├── README.md ├── articles ├── factors-of-flow.md ├── how-to-become-a-hacker.md ├── how-to-build-technical-wealth.md ├── logs-are-streams-not-files.md ├── no-silver-bullet-essence-and-accidents-of-software-engineering.md ├── parallelism-and-concurrency-need-different-tools.md ├── queues-dont-fix-overload.md ├── signs-that-youre-a-good-programmer.md └── the-lava-lair-antipattern.md └── assets └── img ├── lava-layer.png ├── sink1.png ├── sink2.png ├── sink3.png ├── sink4.png ├── sink5.png ├── sink6.png ├── sink7.png └── sink8.png /README.md: -------------------------------------------------------------------------------- 1 | # Programming Related Articles 2 | 3 | A repository for all the great articles I've read over the years to act as a backup in case the originals go down for whatever reason. 4 | 5 | ### Table of contents 6 | 7 | [No Silver Bullet: Essence and Accidents of Software Engineering](https://github.com/montanaflynn/programming-articles/blob/master/articles/no-silver-bullet-essence-and-accidents-of-software-engineering.md) 8 | 9 | [Logs Are Streams, Not Files](https://github.com/montanaflynn/programming-articles/blob/master/articles/logs-are-streams-not-files.md) 10 | 11 | [Parallelism and concurrency need different tools](https://github.com/montanaflynn/programming-articles/blob/master/articles/parallelism-and-concurrency-need-different-tools.md) 12 | 13 | [How To Become A Hacker](https://github.com/montanaflynn/programming-articles/blob/master/articles/how-to-become-a-hacker.md) 14 | 15 | [Queues Don't Fix Overload](https://github.com/montanaflynn/programming-articles/blob/master/articles/queues-dont-fix-overload.md) 16 | 17 | [Factors of flow](https://github.com/montanaflynn/programming-articles/blob/master/articles/factors-of-flow.md) 18 | 19 | [The Lava Layer Anti-Pattern](https://github.com/montanaflynn/programming-articles/blob/master/articles/the-lava-lair-antipattern.md) 20 | 21 | [Signs that you're a good programmer](https://github.com/montanaflynn/programming-articles/blob/master/articles/signs-that-youre-a-good-programmer.md) 22 | 23 | [How To Build Technical Wealth](https://github.com/montanaflynn/programming-articles/blob/master/articles/how-to-build-technical-wealth.md) 24 | -------------------------------------------------------------------------------- /articles/factors-of-flow.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: Swizec Teller 3 | source: https://github.com/Swizec/nightowls 4 | --- 5 | 6 | Ayrton Senna's experience during a qualifying lap at Monaco's 1988 Grand Prix illustrates the importance of flow perfectly. 7 | 8 | > I was already on pole, and I just kept going. Suddenly I was nearly two seconds faster than anybody else, including my team mate with the same car. And suddenly I realised that I was no longer driving the car consciously. I was driving it by a kind of instinct, only it was a different dimension. It was like I was in a tunnel. 9 | 10 | [That lap has been described as the greatest lap of Senna's career](http://youtube.com/watch?v=K9QH8-lKEYY). 11 | 12 | Flow is a state of complete immersion in an activity, a state where the outside and inside world merge into one and the person is simply _present_. Mihaly Csikszentmihaly - the father of flow research in psychology - describes flow as 13 | 14 | > completely focused motivation. A single-minded immersion that represents perhaps the ultimate in harnessing the emotions in service of performing and learning. 15 | 16 | In his essay [Holding a program in one's head](http://paulgraham.com/head.html) Paul Graham explains that to be effective, programmers must be able to hold an entire program in their head while they work. It's the only way the problem and its solution can have the plasticity needed to make sweeping changes and improvements. 17 | 18 | Because programmers tend to work at the limits of their tolerance for complexity holding a program in one's head is only possible in a state of flow. 19 | 20 | But flow is a fragile balance between many opposing forces. The programmer must be willing and able to completely focus on the task at hand, they must have a clear goal to work towards, fresh challenges must be coming in just quickly enough and they must feel a sense of purpose, even joy, in what they're doing. 21 | 22 | ## The autotelic personality 23 | 24 | In his work Csikszentmihalyi describes a type of personality that can experience episodes of flow more easily and more often than most people. 25 | 26 | Researching flow in a lab is too difficult to get proper results, but evidence keeps wagging its eyebrows suggestively, hinting at the validity of something called an "autotelic personality". 27 | 28 | If there is such a thing, I think programmers generally apply: 29 | 30 | * curious 31 | * persistent 32 | * not self-centered 33 | * often performs activities for intrinsic reasons 34 | 35 | It [has been drawn in stick figures](http://xkcd.com/242/) that when a normal person pushes a big button saying "Don't press this" and gets electrocuted, they think to themselves "Heh, I shouldn't have done that." But when an engineer presses that same button they go "Huh, I wonder if that happens every time" 36 | 37 | All programmers are engineers at heart, no matter what their formal qualifications might say. 38 | 39 | Relentlessly hunting down bugs is a source of great pride for most programmers and many would sooner go without sleep than allow a bug to remain in production code. More experienced developers will often go to sleep once they realise they've started making silly mistakes, but they will never forget a bug that's taunting them. 40 | 41 | Even a casual perusal of Github repositories will show that a lot of programmers indulge in so called pet projects - little programs developed for the fun of it, to sharpen the saw if you will. Interestingly enough, flow is much easier to experience on pet projects than large for-money behemoths. 42 | 43 | It could be argued that programmers are very selfless people - the existence of communities like StackOverflow, large numbers of conferences where speakers share their knowledge for free, numerous blogs and free tutorials, not to mention the large opensource community that's given us something as powerful as the Linux operating system and the stack most of the internet runs on. All of those are done holistically ... well, mostly. Everybody gains from having a good standing in the community. 44 | 45 | As a group of people, programmers exhibit all the traits of an autotelic personality. This would suggest flow is much more prevalent amongst programmers than in other groups of people, which might also explain why we programmers are often perceived as night walkers. 46 | 47 | ## Is achieving flow easier at night? 48 | 49 | Yes, flow is indeed much easier to achieve in the evening. At least for those of us who manage to think clearly and who still feel relatively rested. For many, the structure of their life makes this impossible - some then approach the night from its other end, others learn to work during the day. 50 | 51 | ### Decision fatigue 52 | 53 | The biggest enemy to evening and night-time productivity is decision fatigue. 54 | 55 | Most people experiencing decision fatigue don't recognise it for what it is, but if you've ever felt well rested physically and still couldn't think clearly, like your brain was tired, you've experienced decision fatigue. 56 | 57 | [Research](http://www.nytimes.com/2011/08/21/magazine/do-you-suffer-from-decision-fatigue.html) has shown that humans can only accept a limited amount of decisions every day, after that we usually go with the default action, or are so incapable of making a decision we can't perform the action at all. 58 | 59 | Not only that, it's been shown that even the tiniest decision factors into the quota - everything from deciding what to eat for breakfast, to which outfit to wear and what websites to read now or later. This is why the checkout aisle in your supermarket is filled with tiny things you otherwise wouldn't have bought and why people instinctively want to "sleep on it" when making big decisions. 60 | 61 | Depending on how a programmer's day is structured, working in the evening can be a godsend or a spaghetti code mess producing mistake. The important thing is to be well rested regardless of the time when you start working. 62 | 63 | An interesting side-effect of decision fatigue in its early stages is that you can often focus better _because_ you are a bit tired. You know there's no other choice but to power through the work, so you sit down, ignore everything else and just code. 64 | 65 | Eventually decision fatigue catches up and you find yourself thinking long and hard about what variable name to choose. 66 | 67 | ### Clear mind 68 | 69 | Flow is marked by an intense, almost meditative, calm. No other thoughts can invade your brain than what pertains to the task at hand. Everything else ceases to exist. 70 | 71 | But when things are hanging over your head, so to speak, achieving flow is nearly impossible for most people. The mental clarity and emotional calm just isn't there. 72 | 73 | Whether it's an errand you still have to run, or an important email you mustn't forget to send. Perhaps there's just some worrying stuff going on and you aren't Irish enough not to worry about it. Sometimes you simply have an IRC chatroom open in a hidden window. 74 | 75 | If there's something tugging at your brain, you might as well forget about flow. 76 | 77 | Perhaps the biggest thing everyone's got tugging on their brain is the anticipation of imminent distraction. You _know_ something is going to happen in an hour or two. Sometimes it's a big meeting, other times you have to be around to take a call - either way, it can be very distracting even if you do know when the distraction is going to happen. 78 | 79 | Everyone knows those moments where it's 2:35 and you expect something to happen at 3:00. You try to get work done, but you keep looking at the clock every two minutes. Then you might as well check a tweet or two … when 3:00 rolls around you basically haven't done anything useful for almost half an hour. 80 | 81 | Not knowing when, or even if, a distraction is going to happen only makes it worse. You're paying half attention to your code and half attention to your environment at all times. When a colleague sitting across the room says something, you immediately listen in, process what you've heard, decide it wasn't about you and get back to work. 82 | 83 | You'd love to tune out completely, but sometimes it _is_ for you and you have to respond. Random rewards are after all the strongest behaviour creator in existence. 84 | 85 | Something wonderful happens at night, though. There are no more distractions, at least there is no more threat of imminent distraction. Since everyone is asleep you can be fairly certain nobody is going to give you a call, nobody will ask you a random question, nothing interesting is going to happen on the internet. 86 | 87 | For at least ten hours _nothing_ will be expected of you. Nothing. That's a lot of freedom right there. 88 | 89 | ### Habit 90 | 91 | Have you ever had a habitual activity fall through? Say you go to the gym every Wednesday at six in the evening. Or you always watch a certain show on television at exactly 7PM every Thursday. 92 | 93 | When something comes up and you can't do what you're used to it doesn't feel like you've gained an extra hour or two. No, you feel lost, like you suddenly don't know what to do with all this extra time you've been given. So you end up wasting it, clicking one more link on Reddit, or flicking through one more channel on the telly. 94 | 95 | As they say, habit is an iron shirt. 96 | 97 | Getting used to something takes a concerted effort of a few weeks, but once the habit is built it's almost impossible to change. Just remember how odd it feels when you forget to brush your teeth in the morning! 98 | 99 | It's the same thing with flow - it's simply easier at certain times in the day. This depends on a lot of things like how many other obligations you have, what the schedule of the people around you might be and even your internal cycles of activity, which tend to depend a lot on meal times. 100 | 101 | But once you get used to working at a certain point in the day, suddenly your other habits start forming around these patterns instead of the other way around. It's a bit of a self-reinforcing pattern you see. 102 | 103 | Since most programmers start very young, let's take a young programmer for example. Let's say they're still in high school. 104 | 105 | This programmer needs to get up every morning at a predetermined time so they can get to class. They are stuck in class until, say, 3PM. They come home, their mum wants them to do a bunch of chores, there's homework to tend to and before they know it it's already 9PM or 10PM. 106 | 107 | And now, finally, there's nothing hanging over their heads anymore. They can get to programming. 108 | 109 | In a few years this person has come to college, there's much less external time constraints, but after many years of _starting_ coding in the evening they associate the activity with night-time. They simply can't get their mind to coding before 9PM anymore. 110 | 111 | Thus, another night-time hacker is born. With years and piling pressure from the outside, their schedule will shift around, flow to and fro to meet other demands in their lives. 112 | 113 | But when shit hits the fan and something _needs to get done_, I bet they will go back to their safe place and code all night. 114 | 115 | ## Two types of flow and what keeps them going 116 | 117 | Once a programmer gets into flow, it is paramount that they stay there for as long as possible. Not only will they get more work done simply because they're spending more time working, flow is a self-fulfilling prophecy. The more time you spend in flow, the easier getting into flow becomes and the deeper the flow becomes. 118 | 119 | But not all flow is created equal, sometimes it will leave you energised and awash with even better ideas, other times flow might leave you completely exhausted in the end having just squeezed every last ounce of awesome out of you. 120 | 121 | The difference depends mostly on what kept the flow going. 122 | 123 | ### Panic mode 124 | 125 | Every one of us probably knows that feeling of sheer panic that only a hard deadline can bring. The moment you realise The Paper is due next morning, it's already 10pm, you've just come home from an evening of beers and suddenly discovered you _still_ haven't picked up that book from the library. You know, the one with all the info you need to write this thing. 126 | 127 | That's when real productivity begins for most people. That's when they can really tell what they're made of. 128 | 129 | The outside world melts away as you pound the keyboard in a furious panic. Internet stops existing, Skype going off every five minutes could just as well be a supernova going off in a distant galaxy. Nothing exists but you and your work. 130 | 131 | Come morning, you are tired, cranky, effectively two days behind on sleep, but victorious. It might not turn out to be a perfect mark, but it's better than a fail. You know you could have done better if only you had started earlier, but you didn't. 132 | 133 | What just happened was a panic induced flow. 134 | 135 | You spent the entire night essentially in a fight or flight response, adrenaline coursing through your veins made you hyped up, focused. But it was also exhausting, your only reward a deadline not missed. The price, half arsed work and the subsequent day completely wasted because you will be too tired to think. 136 | 137 | ### Deep fulfilment 138 | 139 | The converse of panic flow, is the good kind. A drug-like trance that leaves you energised, ready to take on the world and were it not for annoying physical limitations you would happily spend all your days in such a state. 140 | 141 | For me the typical situation is something like this. 142 | 143 | I have spent the past few days thinking about an interesting problem, it's just been rolling around my mind. Either it's an interesting algorithm I want to try implementing, or a problem I feel could be solved for plenty of people. Sometimes it's as simple as feeling a blogpost must be written to address a topic. 144 | 145 | I let the feeling fester for a while. On purpose. Indulging immediately would ruin the flow because the mind must first have time to process the idea in as much detail as possible. 146 | 147 | One evening I will sit down and begin. 148 | 149 | The outside world melts away, time becomes a blur and before I know it morning comes. Surprising. Startling. When the hell did this happen? Wasn't it just 8pm half an hour ago? 150 | 151 | In reality, I have just spent twelve hours in complete rapture. Engrossed in the bigger picture I solved sub problem after sub problem, each just interesting enough to keep me going, each just difficult enough to keep me going. 152 | 153 | At this point my body has had enough and I do need to get some sleep. 154 | 155 | But the very next day I will jump out of bed - yes literally - and even brushing my teeth might feel like too much waiting to get back into the code. 156 | 157 | That's the good kind of flow. The one you want. Because this is sustainable, were it not for practical limitations one could live for months on end like this. 158 | 159 | Panic flow … that one is only good for one or two consecutive days. Then it takes weeks before the brain is ready again. 160 | 161 | 162 | ## Why is day-time particularly bad for flow? 163 | 164 | Now we can all agree the night is particularly good for flow, it is important to remember this is not the only time when programmers can reach flow. Mornings, especially very early mornings, are a popular choice as well. Even though this is just approaching the night from a different end, the cultural baggage attached is different. 165 | 166 | People who start work early in the morning, especially before dawn, are often seen as hard-working and commendable, while those who choose to work late into the night, often right until dawn, are often seen as lazy slobs who sleep the whole day away. Strange as it may seem. 167 | 168 | Whether a programmer chooses late nights or early mornings, doesn't really change the goal - to avoid working on high intensity flow activities during the day. 169 | 170 | The answer I got when asking programmers why they don't like working during the day sounds very asocial indeed - because other people are awake. 171 | 172 | ### Cost of interruption 173 | 174 | There is a ritual to getting in the flow. 175 | 176 | Usually the first thing people will do when sitting down to code is check Facebook, Twitter and email. Not always in that order, not always all of them, sometimes other social networks completely - Reddit, Hacker News and Stack Overflow are popular options. 177 | 178 | Warm caffeinated beverages go very well with this sort of activity. Making them wastes little time, but introduces an impetus to keep procrastinating until you've finished the cup. 179 | 180 | We call this procrastination, but the term carries connotations that aren't fully warranted. Don't think of it as putting off work, it's more like defragging one's brain. Making sure there are no more lingering thoughts in there, no notifications left unclicked, no important questions left unanswered. 181 | 182 | After anywhere from a few minutes to an hour work can finally begin. 183 | 184 | First, a programmer needs to make sure they are working off the latest version of the project. Then they must check whatever issue tracking system they're using. Having a clear goal is after all paramount to achieving flow and getting anything done. 185 | 186 | Next up, the work environment. 187 | 188 | Swing by a programmer's computer some time and you will likely see a desktop littered with open windows, consoles, IDE's, text editors, documentation. Everything. 189 | 190 | Unlike many (most?) people who use even modern computers as a single application environment and often maximise whatever software they are using to fill the whole screen, programmers are a different beast all together. 191 | 192 | There's usually a text editor and a console or two, perhaps a debugger, a browser with some documentation is almost mandatory as well. Often all these tools will be unified in a single Integrated Development Environment (IDE), which is a mere euphemism for "A bunch of stuff, but in a single window". 193 | 194 | Loading up a work environment like that takes time, not because our computers are slow, but because to know which tools you need, requires understanding what you're about to work on. Keeping things open helps so much that a session crash or a context switch to a different set of tools can be totally devastating to a programmer's productivity. 195 | 196 | Then, the programmer must prime their mind. 197 | 198 | Programs are a conversation between programmer and computer. Computer and user. The programmer's job is breaking down a problem into understandable bits and explaining the solution to the computer in such a way it can be executed perfectly every time. Or at least well enough. 199 | 200 | Loading a program into your brain means learning its language. Functions are verbs, variables are nouns and there's plenty of things in between. Notes can help. Keeping tools open also helps. But nothing will get you over just how much brain power it takes to understand a software project. 201 | 202 | Every time you lose sight of the whole system you have to build it again like constructing a glass palace. Once you understand the language, you have to arrange it in your mind so the space can be freely traversed, you can close your eyes and _see_ how different parts of the system affect each other. 203 | 204 | Only after this stage has been reached is a programmer fully effective. Only then can they have any chance of reaching flow because they don't have to stop every five minutes to figure out how their code fits in the larger picture and whether it's even doing what it's supposed to. 205 | 206 | When you interrupt a programmer, this glass palace can come crashing down in a shower of shattered glass. They have to start the whole process all over again and it's going to take a while before they're back to full steam. 207 | 208 | ### Distractions 209 | 210 | Not all distractions are created equal and not all will cause an interrupt 211 | 212 | You will often see a programmer walking along the street, eyebrow furrowed, peering into the tips of their shoes, lips clenched. Do _not_ try talking to them. This is a programmer who is deep in thought on a problem, struggling hard to prevent the glass palace from tumbling. 213 | 214 | Ask them a question and it can all be gone in the blink of the eye. All that hard work and deep concentration, just gone. Simply because they have to think about something else. 215 | 216 | It almost doesn't matter how simple a question you're asking, anything requiring a proper answer will cause a lot of trouble. Most programmers tend to resort to communicating with grunts and various sounds. Or simply saying "Yeah yeah sure" to everything. 217 | 218 | The danger is not knowing they weren't really listening to you and just said something so you'd go away and stop bothering them. Flailing hands and raised voices can also happen. 219 | 220 | Almost more important than whether the programmer had to do some thinking in response to a distraction is whether they chose to be distracted or it was something outside their control. 221 | 222 | Most people working with computers will often distract themselves. After being focused for a while they will start wasting time on the internet. Nobody told them to do this, often there isn't even a notification saying something new happened, it just feels like "Hey, I haven't done that in a while! I should check if there's anything new!" … there usually isn't. Not much anyway. 223 | 224 | I'm lying, there's always _something_ new. But it usually isn't interesting or something that feels worth the distraction. 225 | 226 | However, random rewards keep us hooked and we can't help ourselves but to check, pressing the button like a lab rat desperately trying to get the next nugget of food from the magical big button. 227 | 228 | If a programmer can prevent themselves from straying off into the wild internet and getting lost for an hour, this sort of distraction won't cause much of a problem. In just a few minutes they will be back to coding at full blast. These sort of distractions are usually just our brain saying we should take a break anyway. 229 | 230 | Conversely even the shortest external distraction can be devastating. 231 | 232 | Most distractions come from coworkers who are just trying to get their work done, but need your involvement - the gordian knot of working in a team of programmers is having two programmers who depend on one another. Both are in a state of flow. One of them has a problem. Do they destroy their productivity by waiting on the other person to naturally come out of flow, or destroy the other person's productivity by interrupting them? 233 | 234 | Hopefully they can work on something else, but this requires a context switch to a different problem as well. 235 | 236 | Programmers also run on a different schedule than most people, which can cause plenty of problems. When the cost of getting started is as high as it is for programmers, it doesn't make sense to work in stretches shorter than a few hours. Everyone else divides their time by hours or even half hours. I've heard lawyers charge in increments of fifteen minutes. 237 | 238 | Even the prototypical calendar app divides the day into hours and half hours ... 239 | 240 | If there's one thing you should know about dealing with programmers is **do not distract them**. Wait for _them_ to come to _you_. Just put your request in the queue. 241 | 242 | Never expect a programmer to do something immediately. 243 | 244 | ## Why programmers work at night 245 | 246 | There's magic in the night-time. The peace and quiet, the internal serenity … There's just you, your work and an infinite abundance of time. You are alone. 247 | 248 | As a society we know that smart, talented people work at night. Often in solitude, they solve problems mere mortals could only dream of. Look no further than your nearest book, movie or TV show about a lone genius ... 249 | 250 | He is a youngish man. A bit of a loner. Doesn't get along with people terribly well. Likes to work at night or before dawn. 251 | 252 | The stereotypical programmer is a friendlier version of that: likes staying up all night, lives off coffee and energy drinks, makes loud noises when disturbed, occasionally writes mean things on the internet. 253 | 254 | ### Culture 255 | 256 | There is a lot to be said about the effect of culture on when people work. Especially when your idea of hard work is sitting behind a computer all day and doing tiny repetitive motions with your fingers. 257 | 258 | One of the greatest benefits of being a programmer is that you can work whenever your mind feels at its best and most productive. You might often have to adjust to other team members either for collaboration or with the explicit purpose of avoiding unwanted interruption, but generally speaking, when you work depends on your personal preference alone. 259 | 260 | A lot of professions can't afford this luxury - journalists have daily deadlines, sales people have to catch people in certain moods, farmers must feed animals at specific times and so on. 261 | 262 | But it wasn't always like that. 263 | 264 | One of the programmers I spoke to while researching this book had the pleasure of working on old mainframe systems a few decades ago. Back then, computing power was a scarce resource and carefully managed. To run a program on a mainframe you needed access. That access was much easier to get at night when other parts of your organisation weren't using the mainframe to do whatever. 265 | 266 | So programmers preferred to stay up late rather than work during the day, IT was simply easier to get the computing resources they needed. 267 | 268 | In the late 1980's and early 1990's computers became commoditised and everyone could have access right from their bedroom. But at the same time the internet came into wider use, amongst programmers especially. 269 | 270 | I was lucky enough never to endure dial-up internet. Right around the time internet access was becoming ubiquitous so was ADSL; I managed to convince my parents "broadband" would be cheaper than internet phone bills. But I still remember a computer enthusiastic family friend explaining his internet strategy when I was about 7 years old. 271 | 272 | > You see, internet use is very limited during the day. Somebody might call your house and your line will be busy. Or your parents will want to use the phone or something. Then in the evening everybody suddenly wants to connect and the internet becomes clogged up, really slow. 273 | 274 | > So your best bet for the internet is after 10pm or 11pm, that's when you can reach full speed. Another option is at 6am or 7am, but that's really hard to get up for. 275 | 276 | I was very impressed by all of this, didn't understand a thing and stuck to windows3.1 games on his computer. Those were fun. The internet did not sound like an interesting place at all. 277 | 278 | A combination of these factors may have greatly contributed to the idea that "computer people" work at night. That they live off coffee and are generally night dwellers who avoid light at all cost. 279 | 280 | Of course programmers aren't doing anything to fight this notion - habitual bragging about how resilient you are to sleep and how you managed to stay up all night to fix a bug is widespread. 281 | 282 | Remember, programmers work at night. Would a young upstart really dare call themselves a real programmer if they didn't enjoy working at night? Of course not. Plus there's all those movies saying that smart people work at night … and you want to be smart, right? 283 | 284 | ### The science of night owls 285 | 286 | A growing body of research gives credence these ideas. People with higher general intelligence are likelier to prefer staying up late and people _are_ more creative in the evening than in the morning. 287 | 288 | [A study published in 2006](http://www.sciencedirect.com/science/article/pii/S0191886999000549) studied USAF trainees in the sixth week of their training. The important part being that as a result of their training, all participants in the study lived the same lifestyle, ate the same food etc. Factors, which have been shown to affect circadian rhythms. 289 | 290 | The study found that people who self-identified as preferring the evening also achieved higher scores in the different cognitive tests measuring things like memory and processing speed. Albeit the correlation was small, it was consistent. 291 | 292 | Interestingly enough, individuals self-identifying as evening types, achieved higher scores in the morning as well. This is despite the USAF population being skewed towards morningness. 293 | 294 | The study also mentions all other morningness-eveningness research would indicate a link between higher general intelligence and a preference for night owl behaviour. Arguing that the sample populations in those studies exhibited a skew towards eveningness and because they were done on university students, who generally have higher intelligence, this would indicate a positive link between the two. 295 | 296 | The research I've found on this topic does not say much about whether being smarter means you will prefer evenings, or preferring evenings means you will be smarter. 297 | 298 | [Another relevant study](http://www.tandfonline.com/doi/abs/10.1080/13546783.2011.625663) from 2011 looked at the difference in solving insight and analytic problems based on morningness-eveningness preference and when you are actually solving the problem. 299 | 300 | Their findings indicate that people perform better at insight-based tasks when they are not at their best time. So a night owl is more creative in the morning, while a lark is more creative in the evening. One of the given explanations supported by a related experiment was that being tired inhibits ones ability to focus, which in turn makes you more capable of thinking widely, beyond the scope of already retrieved knowledge. 301 | 302 | As expected, solving analytical problems shows a positive correlation between time preference and when you're actually solving a problem. Night owls perform better in the evening, larks in the morning. This is very likely due to the fact that peak activity time relates to when we are best able to focus. 303 | 304 | All of this goes somewhat against two of the original hypotheses I proposed before writing this book - that of being more creative in the evening would only appear to be true for morning people, but it seems odd that programmers would habitually work in the evening if they were morning people. Especially since neutral people without a preference do not exhibit creativity gains depending on time of day. 305 | 306 | The other hypothesis that finds itself on shaky ground is the idea that we better manage to focus when tired because a processing power limit is reached and we have no other choice but to focus lest no work get done at all. I will explore this further in the Sleep Science section of the book. 307 | 308 | ### Flow 309 | 310 | > To do high, real good physics work you do need absolutely solid lengths of time, so that when you’re putting ideas together which are vague and hard to remember /../ it needs a lot of concentration 311 | 312 | ~ Richard Feynman, The Pleasure of Finding Things Out 313 | 314 | Just like Feynman's physics example, programming is a process requiring deep thought and a lot of concentration. It's abstractions all the way down. The only way to do good programming is to become completely immersed, so focused you might forget to eat drink and sleep. 315 | 316 | You need flow. 317 | 318 | > Flow is the mental state of operation in which a person performing an activity is fully immersed in a feeling of energised focus, full involvement, and enjoyment in the process of the activity. 319 | 320 | Flow was discovered in 1975 by Mihaly Csikszentmihalyi while he was observing artists who got so lost in their work they would forget to eat and drink and would even stop noticing the passage of time. 321 | 322 | Flow is the natural state of the programmer. 323 | 324 | Any programmer will tell you stories of the wonderful work they've done while in flow, in the zone, wired in or any of a number of euphemisms that all mean the same thing - the programmer had an interesting task, it was challenging, but not too much, and they were left alone to work. 325 | 326 | A lot, if not most, of creative professions experience flow, but it's especially important for programmers. The systems we work on have become so complex they are impossible for an individual to understand fully. 327 | 328 | It's like building a house of cards where every card is slightly bent and torn. You have to balance them carefully, keep all of them in mind when adding a new card and should a single card falter, everything will come tumbling down. 329 | 330 | When a programmer works their brain is playing this delicate game of card house building. Unless fully immersed, the going will be tough, if not impossible. 331 | 332 | More importantly, the work we do provides everything necessary to achieve flow: 333 | 334 | * challenging 335 | * offers immediate feedback 336 | * clear objectives 337 | * never really ends 338 | 339 | It's very easy to get sucked into a programming task. You build the framework in your mind, start working, constantly receive feedback from the computer, make tweaks, beat one small problem after another until suddenly it's morning and birds are chirping outside the window. You're feeling slightly tired, quite hungry, a bit thirsty and confused as to where all the time has vanished off to. 340 | 341 | Right now, if you've ever experienced flow, you are nodding in understanding. If not, you think I'm crazy and am talking complete nonsense. How could you not notice when you get hungry or tired!? Silly person. 342 | 343 | Surely you've played a video game before? 344 | 345 | Video games are specifically designed to get players into a state of flow. To make them fully concentrated on the gaming experience and forget the world around them. The learning curve is always just steep enough to be interesting, the challenges are non-repetitive and you're having fun. 346 | 347 | This is why we hear unfortunate stories of people forgetting to feed their children or starving themselves to death while playing. 348 | 349 | Now imagine your job looked like that. A never-ending string of challenges you can solve with the tiniest bits of focused thinking. 350 | 351 | ### Shifting sleep schedules 352 | 353 | Of course you _could_ code before 3pm, a lot of people do. You likely wouldn't lose a bet saying most employed programmers spend a lot of time at work before 3pm. 354 | 355 | But without an external schedule coding at night just feels more natural. It's when the fun stuff happens. 356 | 357 | You still need to sleep, though. What happens when you go to bed at two in the morning? You wake up at ten in the morning, perhaps nine. If you're really hardcore even at eight. 358 | 359 | Come evening, you don't feel sleepy until midnight. You're just working on something interesting, so you keep working until 1am. Or slightly later. 360 | 361 | Eventually your sleep cycle shifts into a different timezone because you keep going to bed later and later, because you've woken up much later and so on ad infinitum until your alarm clock and society pressure put you in an equilibrium of sorts. 362 | 363 | For a lot of people that equilibrium seems to be waking up at 11am and going to bed at 3am. I don't know why. 364 | 365 | 366 | ### Bright screens 367 | 368 | One of the biggest factors in human sleep-wake cycle is natural light. Scientists have shown that without external stimuli we would all be living on a roughly 26 hour cycle, but there are systems in our bodies that help us sync with the sun. 369 | 370 | This is very important because days aren't equally long and before artificial lighting it wouldn't be very useful to stay awake for the same time in winter days as in summer days. Doesn't make sense and you need to conserve energy when it's cold anyway. 371 | 372 | Modern lighting plays foul with those principles. Computer screens especially. 373 | 374 | This is because the light coming off our screens is very white, aimed right at our eyes and usually insanely bright because monitors are set to stupid brightness levels by default and most people forget to change those. 375 | 376 | All of this just plays back into the not-becoming-sleepy problem and keeps us up longer. Because we aren't sleepy, we might as well work huh? 377 | -------------------------------------------------------------------------------- /articles/how-to-build-technical-wealth.md: -------------------------------------------------------------------------------- 1 | --- 2 | source: http://firstround.com/review/forget-technical-debt-heres-how-to-build-technical-wealth/ 3 | --- 4 | 5 | # Forget Technical Debt — How To Build Technical Wealth 6 | 7 | [**Andrea Goulet**][1] and her business partner sat in her living room, casually reviewing their strategic plan, when an episode of _This Old House_ came on television. It was one of those moments where ideas collide to create something new. They'd been looking for a way to communicate their value proposition — cleaning up legacy code and technical debt for other companies. And here they were, face to face with the perfect analogy. 8 | 9 | "We realized that what we were doing transcended clearing out old code, we were actually **remodeling software** the way you would remodel a house to make it last longer, run better, do more," says Goulet. "It got me thinking about how companies have to invest in mending their code to get more productivity. Just like you have to put a new roof on a house to make it more valuable. It's not sexy, but it's vital, and too many people are doing it wrong." 10 | 11 | Today, she's CEO of [Corgibytes][2] a consulting firm that re-architects and modernizes apps. She's seen all varieties of broken systems, legacy code, and cases of technical debt so extreme it's basically digital hoarding. Here, Goulet argues that **startups need to shift their mindset away from paying down debt toward building technical wealth**, **and away from tearing down old code toward deliberately remodeling it**. She explains this new approach, and how you can do the impossible — actually recruit amazing engineers to tackle this work. 12 | 13 | ## RETHINKING LEGACY CODE 14 | 15 | The most popular definition of legacy code comes from Michael Feathers, author of the aptly titled [_Working Effectively with Legacy Code][3]_: **It's code without test coverage**. That's better than what most people assume — that the term only applies only to really old, archaic systems. But neither definition goes far enough, according to Goulet. "Legacy code has nothing to do with the age of the software. A two year-old app can already be in a legacy state," she says. "**It's all about how difficult that software is to improve.**" 16 | 17 | This means code that isn't written cleanly, that lacks explanation, that contains zero artifacts of your ideas and decision-making processes. A unit test is one type of artifact, but so is any documentation of the rationale and reasoning used to create that code. If there's no way to tell what the developer was thinking when you go to improve it — that's legacy code. 18 | 19 | > Legacy code isn't a technical problem. It's a communication problem. 20 | 21 | ![][4] 22 | 23 | If you linger around in legacy code circles like Goulet does, you'll find that one particular, and rather obscure adage dubbed [Conway's Law][5] will make it's way into nearly every conversation. 24 | 25 | "It's the law that says your codebase will mirror the communication structures across your organization," Goulet says. "If you want to fix your legacy code, you can't do it without also addressing operations, too. That's the missing link that so many people miss." 26 | 27 | Goulet and her team dive into a legacy project much like an archaeologist would. They look for artifacts left behind that give them clues into what past developers were thinking. All of these artifacts together provide context to make new decisions. 28 | 29 | The most important artifact? **Well organized, intention-revealing, clean code.** For example, if you name a variable with generic terms like "foo" or "bar," you might come back six months later and have no idea what that variable is for. 30 | 31 | If the code isn't easy to read, a useful artifact is the source control system, because it provides a history of changes to the code and gives developers an opportunity to write about the changes they're making. 32 | 33 | "A friend of mine says that for commit messages, every summary should be the size of half a tweet, with the description as long as a blog post, if necessary," says Goulet. "You have the chance to tightly couple your rationale with the code that's being changed. It doesn't take a lot of extra time and gives tons of information to people working on the project later, but surprisingly few people do it. It's common to hear developers get so frustrated about working with a piece of code they run 'git blame' in a fit of rage to figure out who wrote that mess, only to find it was themselves." 34 | 35 | Automated tests are also fertile ground for rationale. "There's a reason that so many people like Michael Feathers' definition of legacy code," Goulet explains. "Test suites, especially when used along with [Behavior Driven Development][6] practices like writing out scenarios, are incredibly useful tools for understanding a developer's intention." 36 | 37 | **The lesson here is simple:** If you want to limit your legacy code down the line, pay attention to the details that will make it easier to understand and work with in the future. Write and run unit, acceptance, approval, and integration tests. Explain your commits. Make it easy for future you (and others) to read your mind. 38 | 39 | That said, legacy code will happen no matter what. For reasons both obvious and unexpected. 40 | 41 | Early on at a startup, there's usually a heavy push to get features out the door. Developers are under enormous pressure to deliver, and testing falls by the wayside. The Corgibytes team has encountered many companies that simply couldn't be bothered with testing as they grew — for years. 42 | 43 | Sure, it might not make sense to test compulsively when you're pushing toward a prototype. But once you have a product and users, you need to start investing in maintenance and incremental improvements. "Too many people say, 'Don't worry about the maintenance, the cool things are the features!'" says Goulet. "**If you do this, you're guaranteed to hit a point where you cannot scale. You cannot compete.**" 44 | 45 | As it turns out, the second law of thermodynamics applies to code too: **You'll always be hurtling toward entropy.** You need to constantly battle the chaos of technical debt. And legacy code is simply one type of debt you'll accrue over time. 46 | 47 | "Again the house metaphor applies. You have to keep putting away dishes, vacuuming, taking out the trash," she says. "if you don't, it's going to get harder, until eventually you have to call in the HazMat team." 48 | 49 | Corgibytes gets a lot of calls from CEOs like this one, who said: "Features used to take two weeks to push three years ago. Now they're taking 12 weeks. My developers are super unproductive." 50 | 51 | > Technical debt always reflects an operations problem. 52 | 53 | A lot of CTOs will see the problem coming, but it's hard to convince their colleagues that it's worth spending money to fix what already exists. It seems like backtracking, with no exciting or new outputs. A lot of companies don't move to address technical debt until it starts crippling day-to-day productivity, and by then it can be very expensive to pay down. 54 | 55 | ## FORGET DEBT, BUILD TECHNICAL WEALTH 56 | 57 | You're much more likely to get your CEO, investors and other stakeholders on board if you [reframe your technical debt][7] as an opportunity to accumulate technical wealth — [a term recently coined by agile development coach Declan Whelan][8]. 58 | 59 | "We need to stop thinking about debt as evil. Technical debt can be very useful when you're in the early-stage trenches of designing and building your product," says Goulet. "And when you resolve some debt, you're giving yourself momentum. When you install new windows in your home, yes you're spending a bunch of money, but then you save a hundred dollars a month on your electric bill. The same thing happens with code. Only instead of efficiency, you gain productivity that compounds over time." 60 | 61 | As soon as you see your team not being as productive, you want to identify the technical debt that's holding them back. 62 | 63 | "I talk to so many startups that are killing themselves to acquire talent — they're hiring so many high-paid engineers just to get more work done," she says. "Instead, they should have looked at how to make each of their existing engineers more productive. What debt could you have paid off to get that extra productivity?" 64 | 65 | If you change your perspective and focus on wealth building, you'll end up with a productivity surplus, which can then be reinvested in fixing even more debt and legacy code in a virtuous cycle. Your product will be cruising and getting better all the time. 66 | 67 | > Stop thinking about your software as a project. Start thinking about it as a house you will live in for a long time. 68 | 69 | This is a critical mindset shift, says Goulet. It will take you out of short-term thinking and make you care about maintenance more than you ever have. 70 | 71 | Just like with a house, modernization and upkeep happens in two ways: small, superficial changes ("I bought a new rug!") and big, costly investments that will pay off over time ("I guess we'll replace the plumbing..."). You have to think about both to keep your product current and your team running smoothly. 72 | 73 | This also requires budgeting ahead — if you don't, those bigger purchases are going to hurt. Regular upkeep is the expected cost of home ownership. Shockingly, many companies don't anticipate maintenance as the cost of doing business. 74 | 75 | This is how Goulet coined the term '**software remodeling**.' When something in your house breaks, you don't bulldoze parts of it and rebuild from scratch. Likewise, when you have old, broken code, reaching for a re-write isn't usually the best option. 76 | 77 | **Here are some of the things Corgibytes does when they're called in to 'remodel' a codebase:** 78 | 79 | * Break monolithic apps into micro-services that are lighter weight and more easily maintained. 80 | * Decouple features from each other to make them more extensible. 81 | * Refresh branding and look and feel of the front-end. 82 | * Establish automated testing so that code validates itself. 83 | * Refactor, or edit, codebases to make them easier to work with. 84 | 85 | Remodeling also gets into DevOps territory. For example, Corgibytes often introduces new clients to [Docker][9], making it much easier and faster to set up new developer environments. When you have 30 engineers on your team, cutting the initial setup time from 10 hours to 10 minutes gives you massive leverage to accomplish more tasks. This type of effort can't just be about the software itself, it also has to change how it's built. 86 | 87 | If you know which of these activities will make your code easier to handle and create efficiencies, you should build them into your annual or quarterly roadmap. Don't expect them to happen on their own. But don't put pressure on yourself to implement them all right away either. Goulet sees just as many startups hobbled by their obsession with having 100% test coverage from the very beginning. 88 | 89 | **To get more specific, there are three types of remodeling work every company should plan on:** 90 | 91 | * Automated testing 92 | * Continuous delivery 93 | * Cultural upgrades 94 | 95 | Let's take a closer look at each of these. 96 | 97 | **Automated Testing** 98 | 99 | "One of our clients was going into their Series B and told us they couldn't hire talent fast enough. We helped them introduce an automated testing framework, and it doubled the productivity of their team in under 3 months," says Goulet. "They were able to go to their investors and say, 'We're getting more with a lean team than we would have if we'd doubled the team.'" 100 | 101 | Automated testing is basically a combination of individual tests. You have unit tests which double-check single lines of code. You have integration tests that make sure different parts of the system are playing nice. And you have acceptance tests that ensure features are working as you envisioned. When you write these tests as automated scripts, you can essentially push a button and have your system validate itself rather than having to comb through and manually click through everything. 102 | 103 | Instituting this before hitting product-market fit is probably premature. But as soon as you have a product you're happy with, and that users who depend on it, it's more than worth it to put this framework in place. 104 | 105 | **Continuous Delivery** 106 | 107 | This is the automation of delivery related tasks that used to be manual. The goal is to be able to deploy a small change as soon as it's done and make the feedback loop as short as possible. This can give companies a big competitive advantage over their competition, especially in customer service. 108 | 109 | "Let's say every time you deploy, it's this big gnarly mess. Entropy is out of control," says Goulet. "We've seen deployments take 12 hours or more because it's such a cluster. And when this happens, you're not going to deploy as often. You're going to postpone shipping features because it's too painful. You're going to fall behind and lose to the competition." 110 | 111 | **Other tasks commonly automated during continuous improvement include:** 112 | 113 | * Checking for breaks in the build when commits are made. 114 | * Rolling back in the event of a failure. 115 | * Automated code reviews that check for code quality. 116 | * Scaling computing resources up or down based on demand. 117 | * Making it easy to set up development, testing, and production environments. 118 | 119 | As a simple example, let's say a customer sends in a bug report. The more efficient the developer is in fixing that bug and getting it out, the better. The challenge with bug fixes isn't that making the change is all that difficult, it's that the system isn't set up well and the developer wastes a lot of time doing things other than solving problems, which is what they're best at. 120 | 121 | With continuous improvement, you would become ruthless about determining which tasks are best for the computer and which are best for the human. If a computer is better at it, you automate it. This leaves the developer gleefully solving challenging problems. Customers are happier because their complaints are addressed and fixed quickly. Your backlog of fixes narrows and you're able to spend more time on new creative ways to improve your app even more. **This is the kind of change that generates technical wealth. **Because that developer can ship new code as soon as they fix a bug in one step, they have time and bandwidth to do so much more frequently. 122 | 123 | "You have to constantly ask, 'How can I improve this for my users? How can I make this better? How can I make this more efficient?' But don't stop there," says Goulet. "As soon as you have answers to these questions, you have to ask yourself how you can automate that improvement or efficiency." 124 | 125 | **Cultural Upgrades** 126 | 127 | Every day, Corgibytes sees the same problem: A startup that's built an environment that makes it impossible for its developers to be impactful. The CEO looms over their shoulders wondering why they aren't shipping more often. And the truth is that the culture of the company is working against them. To empower your engineers, you have to look at their environment holistically. 128 | 129 | To make this point, Goulet quotes author Robert Henri: 130 | 131 | > The object isn't to make art, it's to be in that wonderful state which makes art inevitable. 132 | 133 | "That's how you need to start thinking about your software," she says. "Your culture can be that state. Your goal should always be to create an environment where art just happens, and that art is clean code, awesome customer service, happy developers, good product-market fit, profitability, etc. It's all connected." 134 | 135 | This is a culture that prioritizes the resolution of technical debt and legacy code. That's what will truly clear the path for your developers to make impact. And that's what will give you the surplus to build cooler things in the future. You can't remodel your product without making over the environment it's developed in. Changing the overall attitude toward investing in maintenance and modernization is the place to start, ideally at the top with the CEO. 136 | 137 | Here are some of Goulet's suggestions for establishing that flow-state culture: 138 | 139 | * Resist the urge to reward "heroes" who work late nights. Praise effectiveness over effort. 140 | * Get curious about collaboration techniques, such as Woody Zuill's [_Mob Programming][10]_. 141 | * Follow the four [Modern Agile][11] principles: make users awesome, experiment and learn rapidly, make safety a prerequisite, and deliver value continuously. 142 | * Give developers time outside of projects each week for professional development. 143 | * Practice [daily shared journals][12] as a way to enable your team to solve problems proactively. 144 | * Put empathy at the center of everything you do. At Corgibytes, [Brene Brown's CourageWorks][13] training has been invaluable. 145 | 146 | If execs and investors balk at this upgrade, frame it in terms of customer service, Goulet says. Tell them how the end product of this change will be a better experience for the people who matter most to them. It's the most compelling argument you can make. 147 | 148 | ## FINDING THE MOST TALENTED REMODELERS 149 | 150 | It's an industry-wide assumption that badass engineers don't want to work on legacy code. They want to build slick new features. Sticking them in the maintenance department would be a waste, people say. 151 | 152 | **These are misconceptions. You can find incredibly skilled engineers to work on your thorniest debt if you know where and how to look — and how to make them happy when you've got them.** 153 | 154 | "Whenever we speak at conferences, we poll the audience and ask 'Who loves working on legacy code?' It's pretty consistent that less than 10% of any crowd will raise their hands." says Goulet. "But when I talked to these people, I found out they were the engineers who liked the most challenging problems." 155 | 156 | She has clients coming to her with homegrown databases, zero documentation, and no conceivable way to parse out structure. This is the bread and butter of a class of engineers she calls "**menders**." Now she has a team of them working for her at Corgibytes who like nothing more than diving into binary files to see what's really going on. 157 | 158 | ![][14] 159 | 160 | So, how can you find these elite forces? Goulet has tried a lot of things — and a few have worked wonders. 161 | 162 | She launched a community website at [legacycode.rocks][15] that touts the following manifesto: "For too long, those of us who enjoy refactoring legacy code have been treated as second class developers... If you're proud to work on legacy code, welcome!" 163 | 164 | "I started getting all these emails from people saying, 'Oh my god, me too!'" she says. "Just getting out there and spreading this message about how valuable this work is drew in the right people." 165 | 166 | She's also used continuous delivery practices in her recruiting to give these type of developers what they want: Loads of detail and explicit instructions. "It started because I hated repeating myself. If I got more than a few emails asking the same question, I'd put it up on the website, much like I would if I was writing documentation." 167 | 168 | But over time, she noticed that she could refine the application process even further to help her identify good candidates earlier in the process. For example, her application instructions read, "The CEO is going to review your resume, so make sure to address your cover letter to the CEO" without providing a gender. All letters starting with "Dear Sir," or "Mr." are immediately trashed. And this is just the beginning of her recruiting gauntlet. 169 | 170 | "This started because I was annoyed at how many times people assumed that because I'm the CEO of a software company, I must be a man," Goulet said. "So one day, I thought I'd put it on the website as an instruction for applicants to see who was paying attention. To my surprise, it didn't just muffle the less serious candidates. It amplified the folks who had the particular skills for working with legacy code." 171 | 172 | Goulet recalls how one candidate emailed her to say, "I inspected the code on your website (I like the site and hey, it's what I do). There's a weird artifact that seems to be written in PHP but it appears you're running Jekyll which is in Ruby. I was really curious what that's about." 173 | 174 | It turned out that there was a leftover PHP class name in the HTML, CSS, and JavaScript that Goulet got from her designer that she'd been meaning to getting around to but hadn't had a chance. Her response: "Are you looking for a job?" 175 | 176 | Another candidate noticed that she had used the term CTO in an instruction, but that title didn't exist on her team (her business partner is the Chief Code Whisperer). Again, the attention to detail, the curiosity, and the initiative to make it better caught her eye. 177 | 178 | > Menders aren't just detail-oriented, they're compelled by attention to detail. 179 | 180 | Surprisingly, Goulet hasn't been plagued with the recruiting challenges of most tech companies. "Most people apply directly through our website, but when we want to cast a wider net, we use [PowerToFly][16]__ and [WeWorkRemotely][17]__. I really don't have a need for recruiters at the moment. They have a tough time understanding the nuance of what makes menders different." 181 | 182 | If they make it through an initial round, Goulet has a candidate read an article called "[Naming is a Process][18]" by Arlo Belshee. It delves into the very granular specifics of working with indebted code. Her only directions: "Read it and tell me what you think." 183 | 184 | She's looking for understanding of subtleties in their responses, and also the willingness to take a point of view. It's been really helpful in separating deep thinkers with conviction from candidates who just want to get hired. She highly recommends choosing a piece of writing that matters to your operations and will demonstrate how passionate, opinionated, and analytical people are. 185 | 186 | Lastly, she'll have a current team member pair program with the candidate using [Exercism.io][19]. It's an open-source project that allows developers to learn how to code in different languages with a range of test driven development exercises. The first part of the pair programming session allows the candidate to choose a language to build in. For the next exercise, the interviewer gets to pick the language. They get to see how the person deals with surprise, how flexible they are, and whether they're willing to admit they don't know something. 187 | 188 | "When someone has truly transitioned from a practitioner to a master, they freely admit what they don't know," says Goulet. 189 | 190 | Having someone code in a language they aren't that familiar with also gauges their stick-to-it-iveness. "We want someone who will say, 'I'm going to hammer on this problem until it's done.' Maybe they'll even come to us the next day and say, 'I kept at it until I figured it out.' That's the type of behavior that's very indicative of success as a mender." 191 | 192 | > Makers are so lionized in our industry that everyone wants to have them do maintenance too. That's a mistake. The best menders are never the best makers. 193 | 194 | Once she has talented menders in the door, Goulet knows how to set them up for success. Here's what you can do to make this type of developer happy and productive: 195 | 196 | * Give them a generous amount of autonomy. Hand them assignments where you explain the problem, sure, but never dictate how they should solve it. 197 | * If they ask for upgrades to their computers and tooling, do it. They know what they need to maximize efficiency. 198 | * Help them [limit their context-switching][20]. They like to focus until something's done. 199 | 200 | Altogether, this approach has helped Corgibytes build a waiting list of over 20 qualified developers passionate about legacy code. 201 | 202 | ## STABILITY IS NOT A DIRTY WORD 203 | 204 | Most startups don't think past their growth phase. Some may even believe growth should never end. And it doesn't have to, even when you enter the next stage: Stability. **All stability means is that you have the people and processes you need to build technical wealth and spend it on the right priorities.** 205 | 206 | "There's this inflection point between growth and stability where menders must surge, and you start to balance them more equally against the makers focused on new features," says Goulet. "You have your systems. Now you need them to work better." 207 | 208 | This means allocating more of your organization's budget to maintenance and modernization. "You can't afford to think of maintenance as just another line item," she says. "It has to become innate to your culture — something important that will yield greater success in the future." 209 | 210 | Ultimately, the technical wealth you build with these efforts will give rise to a whole new class of developers on your team: scouts that have the time and resources to explore new territory, customer bases and opportunities. When you have the bandwidth to tap into new markets and continuously get better at what you already do — that's when you're truly thriving. 211 | 212 | [1]: https://www.linkedin.com/in/andreamgoulet 213 | [2]: http://corgibytes.com/ 214 | [3]: https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 215 | [4]: http://s3.amazonaws.com/marquee-test-akiaisur2rgicbmpehea/H4y9x4gQj61G9aK4v8Kp_Screen%20Shot%202016-08-11%20at%209.16.38%20AM.png 216 | [5]: https://en.wikipedia.org/wiki/Conway%27s_law 217 | [6]: https://en.wikipedia.org/wiki/Behavior-driven_development 218 | [7]: https://www.agilealliance.org/resources/initiatives/technical-debt/ 219 | [8]: http://legacycoderocks.libsyn.com/technical-wealth-with-declan-wheelan 220 | [9]: https://www.docker.com/ 221 | [10]: http://mobprogramming.org/ 222 | [11]: https://www.industriallogic.com/blog/modern-agile/ 223 | [12]: http://corgibytes.com/blog/2016/08/02/how-we-use-daily-journals/ 224 | [13]: http://www.courageworks.com/ 225 | [14]: http://s3.amazonaws.com/marquee-test-akiaisur2rgicbmpehea/BeX5wWrESmCTaJYsuKhW_Screen%20Shot%202016-08-11%20at%209.17.04%20AM.png 226 | [15]: http://legacycode.rocks/ 227 | [16]: https://www.powertofly.com/ 228 | [17]: https://weworkremotely.com/ 229 | [18]: http://arlobelshee.com/good-naming-is-a-process-not-a-single-step/ 230 | [19]: http://exercism.io/ 231 | [20]: http://corgibytes.com/blog/2016/04/15/inception-layers/ 232 | -------------------------------------------------------------------------------- /articles/logs-are-streams-not-files.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: Adam Wiggins 3 | source: http://adam.herokuapp.com/past/2011/4/1/logs_are_streams_not_files/ 4 | --- 5 | 6 |

Logs Are Streams, Not Files

7 | 8 |

Server daemons (such as PostgreSQL or Nginx) and applications (such as a Rails or Django app) sometimes offer a configuration parameter for a path to the program’s logfile. This can lead us to think of logs as files.

9 | 10 |

But a better conceptual model is to treat logs as time-ordered streams: there is no beginning or end, but rather an ongoing, collated collection of events which we may wish to view in realtime as they happen (e.g. via tail -f or heroku logs --tail) or which we may wish to search in some time window (e.g. via grep or Splunk).

11 | 12 |

Using the power of unix for logs

13 | 14 |

Unix provides some excellent tools for handling streams. There are two default output streams, stdout and stderr, available automatically to all programs. Streams can be turned into files with a redirect operator, but they can also be channeled in more powerful ways, such as splitting the streams to multiple locations or pipelining the stream to another program for further processing.

15 | 16 |

A program that uses stdout for its logging can easily log to any file you wish:

17 | 18 | 
$ mydaemon >> /var/log/mydaemon.log
19 | 20 |

(Typically you would not invoke this command directly, but would run this from an init program such as Upstart or Systemd.)

21 | 22 |

Programs that send their logs directly to a logfile lose all the power and flexibility of unix streams. What’s worse is that they end up reinventing some of these capabilities, badly. How many programs end up re-writing log rotation, for example?

23 | 24 |

Distributed logging with syslog

25 | 26 |

Logging on any reasonably large distributed system will generally end up using the syslog protocol to send logs from many components to a single location. Programs that treat logs as files are now on the wrong path: if they wisht to log to syslog, each program needs to implement syslog internally - and provide yet more logging configuration options to set the various syslog fields.

27 | 28 |

A program using stdout for logging can use syslog without needing to implement any syslog awareness into the program, by piping to the standard logger command available on all modern unixes:

29 | 30 | 
$ mydaemon | logger
31 | 32 |

Perhaps we want to split the stream and log to a local file as well as syslog:

33 | 34 | 
$ mydaemon | tee /var/log/mydaemon.log | logger
35 | 36 |

A program which uses stdout is equipped to log in a variety of ways without adding any weight to its codebase or configuration format.

37 | 38 |

Other distributed logging protocols

39 | 40 |

Syslog is an entrenched standard for distributed logging, but there are other, more modern options as well. Splunk, fast becoming a indispensable tool for anyone running a large software service, can accept syslog; but it also has its own custom protocol which offers additional features like authentication and encryption. Scribe is another example of a modern logging protocol.

41 | 42 |

Programs that log to stdout can be adapted to work with a new protocol without needing to modify the program. Simply pipe the program’s output to a receiving daemon just as you would with the logger program for syslog. Treating your logs as streams is a form of future-proofing for your application.

43 | 44 |

Logging in the Ruby world

45 | 46 |

Most Rack frameworks (Sinatra, Ramaze, etc) and Rack webservers (Mongrel, Thin, etc) do the right thing: they log to stdout. If you run them in the foreground, as is typical of development mode, you see the output right in your terminal. This is exactly what you want. If you run in production mode, you can redirect the output to a file, to syslog, to both, or to any other logging system that can accept an input stream.

47 | 48 |

Unfortunately, Rails stands out as a major exception to this simple principle. It creates its own log directory and writes various files into it; some plugins even take it upon themselves to write their own, separate logfiles. This hurts the local development experience: what you see in your terminal isn’t complete, so you have to open a separate window with tail -f log/*.log to get the information you want. But it hurts the deployment experience even more, because you end up having to tinker around with a bunch of Rails logger configuration options to get your logs from all your web machines to merge into a single stream.

49 | 50 |

Logging on Heroku

51 | 52 |

The need to treat application logs as a stream is especially poignant with Heroku's new logging system. On the backend, we route logs with a syslog router written in Erlang called Logplex.

53 | 54 |

Logplex handles input streams (which we call “sinks”) from many different sources: all the dynos running on the app, system components like our HTTP router, and (currently in alpha) logs from add-on providers. Sinks are merged together into channels (each app has its own channel) which is a unified stream of all logs relevant to the app. This allows developers to see a holistic view of everything happening with their app, or to filter down to logs from a particular type of sink (for example: just logs from the HTTP router, or just logs from worker processes).

55 | 56 |

Further, log streams can also be sent outbound, which we call “drains.” Users can configure syslog drains, and we’re currently working up a technical design for how add-on providers can automatically add drains. This latter item will enable a new class of log search and archival add-on, most notably the emerging syslog-as-a-service products like Loggly and Papertrail.

57 | 58 |

This logging system works quite well, and it gets even better with the new features on the way - but it only works where all programs output their logs as streams. Programs that write logfiles, such as Rails in its default configuration, don’t make sense in this world.

59 | 60 |

As a workaround, Heroku injects the rails_log_stdout plugin into Rails apps at deploy time. We’d prefer not to have to do this (injecting code is a dicey way to solve problems), but it’s the best way to get Rails logs into the app’s logstream without requiring extra configuration from the app developer.

61 | 62 |

Conclusion

63 | 64 |

Logs are a stream, and it behooves everyone to treat them as such. Your programs should log to stdout and/or stderr and omit any attempt to handle log paths, log rotation, or sending logs over the syslog protocol. Directing where the program’s log stream goes can be left up to the runtime container: a local terminal or IDE (in development environments), an Upstart / Systemd launch script (in traditional hosting environments), or a system like Logplex/Heroku (in a platform environment).

65 | 66 | -------------------------------------------------------------------------------- /articles/no-silver-bullet-essence-and-accidents-of-software-engineering.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: Frederick P. Brooks, Jr. 3 | source: http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBullet.html 4 | --- 5 | 6 | No Silver Bullet: Essence and Accidents of Software Engineering 7 | 8 | Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest. 9 | The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet--something to make software costs drop as rapidly as computer hardware costs do. 10 | 11 | But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity. In this article, I shall try to show why, by examining both the nature of the software problem and the properties of the bullets proposed. 12 | 13 | Skepticism is not pessimism, however. Although we see no startling breakthroughs--and indeed, I believe such to be inconsistent with the nature of software--many encouraging innovations are under way. A disciplined, consistent effort to develop, propagate, and exploit these innovations should indeed yield an order-of-magnitude improvement. There is no royal road, but there is a road. 14 | 15 | The first step toward the management of disease was replacement of demon theories and humours theories by the germ theory. That very step, the beginning of hope, in itself dashed all hopes of magical solutions. It told workers that progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today. 16 | 17 | Does It Have to Be Hard?--Essential Difficulties 18 | 19 | Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any--no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware. We cannot expect ever to see twofold gains every two years. 20 | First, one must observe that the anomaly is not that software progress is so slow, but that computer hardware progress is so fast. No other technology since civilization began has seen six orders of magnitude in performance price gain in 30 years. In no other technology can one choose to take the gain in either improved performance or in reduced costs. These gains flow from the transformation of computer manufacture from an assembly industry into a process industry. 21 | 22 | Second, to see what rate of progress one can expect in software technology, let us examine the difficulties of that technology. Following Aristotle, I divide them into essence, the difficulties inherent in the nature of software, and accidents, those difficulties that today attend its production but are not inherent. 23 | 24 | The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract in that such a conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed. 25 | 26 | I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems. 27 | 28 | If this is true, building software will always be hard. There is inherently no silver bullet. 29 | 30 | Let us consider the inherent properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility. 31 | 32 | Complexity. Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound. 33 | 34 | Digital computers are themselves more complex than most things people build: They have very large numbers of states. This makes conceiving, describing, and testing them hard. Software systems have orders-of-magnitude more states than computers do. 35 | 36 | Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger sizes, it is necessarily an increase in the number of different elements. In most cases, the elements interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly. 37 | 38 | The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence. For three centuries, mathematics and the physical sciences made great strides by constructing simplified models of complex phenomena, deriving properties from the models, and verifying those properties by experiment. This paradigm worked because the complexities ignored in the models were not the essential properties of the phenomena. It does not work when the complexities are the essence. 39 | 40 | Many of the classic problems of developing software products derive from this essential complexity and its nonlinear increases with size. From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability. From complexity of function comes the difficulty of invoking function, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. From complexity of structure come the unvisualized states that constitute security trapdoors. 41 | 42 | Not only technical problems, but management problems as well come from the complexity. It makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster. 43 | 44 | Conformity. Software people are not alone in facing complexity. Physics deals with terribly complex objects even at the "fundamental" particle level. The physicist labors on, however, in a firm faith that there are unifying principles to be found, whether in quarks or in unifiedfield theories. Einstein argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. 45 | 46 | No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God. 47 | 48 | In many cases, the software must conform because it is the most recent arrival on the scene. In others, it must conform because it is perceived as the most conformable. But in all cases, much complexity comes from conformation to other interfaces; this complexity cannot be simplified out by any redesign of the software alone. 49 | 50 | Changeability. The software entity is constantly subject to pressures for change. Of course, so are buildings, cars, computers. But manufactured things are infrequently changed after manufacture; they are superseded by later models, or essential changes are incorporated into later-serial-number copies of the same basic design. Call-backs of automobiles are really quite infrequent; field changes of computers somewhat less so. Both are much less frequent than modifications to fielded software. 51 | 52 | In part, this is so because the software of a system embodies its function, and the function is the part that most feels the pressures of change. In part it is because software can be changed more easily--it is pure thought-stuff, infinitely malleable. Buildings do in fact get changed, but the high costs of change, understood by all, serve to dampen the whims of the changers. 53 | 54 | All successful software gets changed. Two processes are at work. First, as a software product is found to be useful, people try it in new cases at the edge of or beyond the original domain. The pressures for extended function come chiefly from users who like the basic function and invent new uses for it. 55 | 56 | Second, successful software survives beyond the normal life of the machine vehicle for which it is first written. If not new computers, then at least new disks, new displays, new printers come along; and the software must be conformed to its new vehicles of opportunity. 57 | 58 | In short, the software product is embedded in a cultural matrix of applications, users, laws, and machine vehicles. These all change continually, and their changes inexorably force change upon the software product. 59 | 60 | Invisibility. Software is invisible and unvisualizable. Geometric abstractions are powerful tools. The floor plan of a building helps both architect and client evaluate spaces, traffic flows, views. Contradictions and omissions become obvious. Scale drawings of mechanical parts and stick-figure models of molecules, although abstractions, serve the same purpose. A geometric reality is captured in a geometric abstraction. 61 | 62 | The reality of software is not inherently embedded in space. Hence, it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. As soon as we attempt to diagram software structure, we find it to constitute not one, but several, general directed graphs superimposed one upon another. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, name-space relationships. These graphs are usually not even planar, much less hierarchical. Indeed, one of the ways of establishing conceptual control over such structure is to enforce link cutting until one or more of the graphs becomes hierarchical. [1] 63 | 64 | In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, and thus do not permit the mind to use some of its most powerful conceptual tools. This lack not only impedes the process of design within one mind, it severely hinders communication among minds. 65 | 66 | Past Breakthroughs Solved Accidental Difficulties 67 | 68 | If we examine the three steps in software technology development that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but that those difficulties have been accidental, not essential, difficulties. We can also see the natural limits to the extrapolation of each such attack. 69 | High-level languages. Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. Most observers credit that development with at least a factor of five in productivity, and with concomitant gains in reliability, simplicity, and comprehensibility. 70 | 71 | What does a high-level language accomplish? It frees a program from much of its accidental complexity. An abstract program consists of conceptual constructs: operations, data types, sequences, and communication. The concrete machine program is concerned with bits, registers, conditions, branches, channels, disks, and such. To the extent that the high-level language embodies the constructs one wants in the abstract program and avoids all lower ones, it eliminates a whole level of complexity that was never inherent in the program at all. 72 | 73 | The most a high-level language can do is to furnish all the constructs that the programmer imagines in the abstract program. To be sure, the level of our thinking about data structures, data types, and operations is steadily rising, but at an ever decreasing rate. And language development approaches closer and closer to the sophistication of users. 74 | 75 | Moreover, at some point the elaboration of a high-level language creates a tool-mastery burden that increases, not reduces, the intellectual task of the user who rarely uses the esoteric constructs. 76 | 77 | Time-sharing. Time-sharing brought a major improvement in the productivity of programmers and in the quality of their product, although not so large as that brought by high-level languages. 78 | 79 | Time-sharing attacks a quite different difficulty. Time-sharing preserves immediacy, and hence enables one to maintain an overview of complexity. The slow turnaround of batch programming means that one inevitably forgets the minutiae, if not the very thrust, of what one was thinking when he stopped programming and called for compilation and execution. This interruption is costly in time, for one must refresh one's memory. The most serious effect may well be the decay of the grasp of all that is going on in a complex system. 80 | 81 | Slow turnaround, like machine-language complexities, is an accidental rather than an essential difficulty of the software process. The limits of the potential contribution of time-sharing derive directly. The principal effect of timesharing is to shorten system response time. As this response time goes to zero, at some point it passes the human threshold of noticeability, about 100 milliseconds. Beyond that threshold, no benefits are to be expected. 82 | 83 | Unified programming environments. Unix and Interlisp, the first integrated programming environments to come into widespread use, seem to have improved productivity by integral factors. Why? 84 | 85 | They attack the accidental difficulties that result from using individual programs together, by providing integrated libraries, unified file formats, and pipes and filters. As a result, conceptual structures that in principle could always call, feed, and use one another can indeed easily do so in practice. 86 | 87 | This breakthrough in turn stimulated the development of whole toolbenches, since each new tool could be applied to any programs that used the standard formats. 88 | 89 | Because of these successes, environments are the subject of much of today's software-engineering research. We look at their promise and limitations in the next section. 90 | 91 | Hopes for the Silver 92 | 93 | Now let us consider the technical developments that are most often advanced as potential silver bullets. What problems do they address--the problems of essence, or the remaining accidental difficulties? Do they offer revolutionary advances, or incremental ones? 94 | Ada and other high-level language advances. One of the most touted recent developments is Ada, a general-purpose high-level language of the 1980's. Ada not only reflects evolutionary improvements in language concepts, but indeed embodies features to encourage modern design and modularization. Perhaps the Ada philosophy is more of an advance than the Ada language, for it is the philosophy of modularization, of abstract data types, of hierarchical structuring. Ada is over-rich, a natural result of the process by which requirements were laid on its design. That is not fatal, for subsetted working vocabularies can solve the learning problem, and hardware advances will give us the cheap MIPS to pay for the compiling costs. Advancing the structuring of software systems is indeed a very good use for the increased MIPS our dollars will buy. Operating systems, loudly decried in the 1960's for their memory and cycle costs, have proved to be an excellent form in which to use some of the MIPS and cheap memory bytes of the past hardware surge. 95 | 96 | Nevertheless, Ada will not prove to be the silver bullet that slays the software productivity monster. It is, after all, just another high-level language, and the biggest payoff from such languages came from the first transition -- the transition up from the accidental complexities of the machine into the more abstract statement of step-by-step solutions. Once those accidents have been removed, the remaining ones will be smaller, and the payoff from their removal will surely be less. 97 | 98 | I predict that a decade from now, when the effectiveness of Ada is assessed, it will be seen to have made a substantial difference, but not because of any particular language feature, nor indeed because of all of them combined. Neither will the new Ada environments prove to be the cause of the improvements. Ada's greatest contribution will be that switching to it occasioned training programmers in modern software-design techniques. 99 | 100 | Object-oriented programming. Many students of the art hold out more hope for object-oriented programming than for any of the other technical fads of the day. [2] I am among them. Mark Sherman of Dartmouth notes on CSnet News that one must be careful to distinguish two separate ideas that go under that name: abstract data types and hierarchical types. The concept of the abstract data type is that an object's type should be defined by a name, a set of proper values, and a set of proper operations rather than by its storage structure, which should be hidden. Examples are Ada packages (with private types) and Modula's modules. 101 | 102 | Hierarchical types, such as Simula-67's classes, allow one to define general interfaces that can be further refined by providing subordinate types. The two concepts are orthogonal_one may have hierarchies without hiding and hiding without hierarchies. Both concepts represent real advances in the art of building software. 103 | 104 | Each removes yet another accidental difficulty from the process, allowing the designer to express the essence of the design without having to express large amounts of syntactic material that add no information content. For both abstract types and hierarchical types, the result is to remove a higher-order kind of accidental difficulty and allow a higher-order expression of design. 105 | 106 | Nevertheless, such advances can do no more than to remove all the accidental difficulties from the expression of the design. The complexity of the design itself is essential, and such attacks make no change whatever in that. An order-of-magnitude gain can be made by object-oriented programming only if the unnecessary type-specification underbrush still in our programming language is itself nine-tenths of the work involved in designing a program product. I doubt it. 107 | 108 | Artificial intelligence. Many people expect advances in artificial intelligence to provide the revolutionary breakthrough that will give order-of-magnitude gains in software productivity and quality. [3] I do not. To see why, we must dissect what is meant by "artificial intelligence." 109 | 110 | D.L. Parnas has clarified the terminological chaos: [4] 111 | 112 | Two quite different definitions of AI are in common use today. AI-1: The use of computers to solve problems that previously could only be solved by applying human intelligence. Al-2: The use of a specific set of programming techniques known as heuristic or rule-based programming. In this approach human experts are studied to determine what heuristics or rules of thumb they use in solving problems.... The program is designed to solve a problem the way that humans seem to solve it. 113 | 114 | The first definition has a sliding meaning.... Something can fit the definition of Al-1 today but, once we see how the program works and understand the problem, we will not think of it as Al any more.... Unfortunately I cannot identify a body of technology that is unique to this field.... Most of the work is problem-specific, and some abstraction or creativity is required to see how to transfer it. 115 | 116 | I agree completely with this critique. The techniques used for speech recognition seem to have little in common with those used for image recognition, and both are different from those used in expert systems. I have a hard time seeing how image recognition, for example, will make any appreciable difference in programming practice. The same problem is true of speech recognition. The hard thing about building software is deciding what one wants to say, not saying it. No facilitation of expression can give more than marginal gains. 117 | 118 | Expert-systems technology, AI-2, deserves a section of its own. 119 | 120 | Expert systems. The most advanced part of the artificial intelligence art, and the most widely applied, is the technology for building expert systems. Many software scientists are hard at work applying this technology to the software-building environment. [3, 5] What is the concept, and what are the prospects? 121 | 122 | An expert system is a program that contains a generalized inference engine and a rule base, takes input data and assumptions, explores the inferences derivable from the rule base, yields conclusions and advice, and offers to explain its results by retracing its reasoning for the user. The inference engines typically can deal with fuzzy or probabilistic data and rules, in addition to purely deterministic logic. 123 | 124 | Such systems offer some clear advantages over programmed algorithms designed for arriving at the same solutions to the same problems: 125 | 126 | Inference-engine technology is developed in an application-independent way, and then applied to many uses. One can justify much effort on the inference engines. Indeed, that technology is well advanced. 127 | The changeable parts of the application-peculiar materials are encoded in the rule base in a uniform fashion, and tools are provided for developing, changing, testing, and documenting the rule base. This regularizes much of the complexity of the application itself. 128 | The power of such systems does not come from ever-fancier inference mechanisms but rather from ever-richer knowledge bases that reflect the real world more accurately. I believe that the most important advance offered by the technology is the separation of the application complexity from the program itself. 129 | 130 | How can this technology be applied to the software-engineering task? In many ways: Such systems can suggest interface rules, advise on testing strategies, remember bug-type frequencies, and offer optimization hints. 131 | 132 | Consider an imaginary testing advisor, for example. In its most rudimentary form, the diagnostic expert system is very like a pilot's checklist, just enumerating suggestions as to possible causes of difficulty. As more and more system structure is embodied in the rule base, and as the rule base takes more sophisticated account of the trouble symptoms reported, the testing advisor becomes more and more particular in the hypotheses it generates and the tests it recommends. Such an expert system may depart most radically from the conventional ones in that its rule base should probably be hierarchically modularized in the same way the corresponding software product is, so that as the product is modularly modified, the diagnostic rule base can be modularly modified as well. 133 | 134 | The work required to generate the diagnostic rules is work that would have to be done anyway in generating the set of test cases for the modules and for the system. If it is done in a suitably general manner, with both a uniform structure for rules and a good inference engine available, it may actually reduce the total labor of generating bring-up test cases, and help as well with lifelong maintenance and modification testing. In the same way, one can postulate other advisors, probably many and probably simple, for the other parts of the software-construction task. 135 | 136 | Many difficulties stand in the way of the early realization of useful expert-system advisors to the program developer. A crucial part of our imaginary scenario is the development of easy ways to get from program-structure specification to the automatic or semiautomatic generation of diagnostic rules. Even more difficult and important is the twofold ,task of knowledge acquisition: finding articulate, self-analytical experts who know why they do things, and developing efficient techniques for extracting what they know and distilling it into rule bases. The essential prerequisite for building an expert system is to have an expert. 137 | 138 | The most powerful contribution by expert systems will surely be to put at the service of the inexperienced programmer the experience and accumulated wisdom of the best programmers. This is no small contribution. The gap between the best software engineering practice and the average practice is very wide_perhaps wider than in any other engineering discipline. A tool that disseminates good practice would be important. 139 | 140 | "Automatic" programming. For almost 40 years, people have been anticipating and writing about "automatic programming," or the generation of a program for solving a problem from a statement of the problem specifications. Some today write as if they expect this technology to provide the next breakthrough. [5] 141 | 142 | Parnas [4] implies that the term is used for glamour, not for semantic content, asserting, 143 | 144 | In short, automatic programming always has been a euphemism for programming with a higher-level language than was presently available to the programmer. 145 | 146 | He argues, in essence, that in most cases it is the solution method, not the problem, whose specification has to be given. 147 | 148 | One can find exceptions. The technique of building generators is very powerful, and it is routinely used to good advantage in programs for sorting. Some systems for integrating differential equations have also permitted direct specification of the problem, and the systems have assessed the parameters, chosen from a library of methods of solution, and generated the programs. 149 | 150 | These applications have very favorable properties: 151 | 152 | The problems are readily characterized by relatively few parameters. 153 | There are many known methods of solution to provide a library of alternatives. 154 | Extensive analysis has led to explicit rules for selecting solution techniques, given problem parameters. 155 | It is hard to see how such techniques generalize to the wider world of the ordinary software system, where cases with such neat properties are the exception. It is hard even to imagine how this breakthrough in generalization could occur. 156 | 157 | Graphical programming. A favorite subject for PhD dissertations in software engineering is graphical, or visual, programming--the application of computer graphics to software design. [6, 7] Sometimes the promise held out by such an approach is postulated by analogy with VLSI chip design, in which computer graphics plays so fruitful a role. Sometimes the theorist justifies the approach by considering flowcharts as the ideal program-design medium and by providing powerful facilities for constructing them. 158 | 159 | Nothing even convincing, much less exciting, has yet emerged from such efforts. I am persuaded that nothing will. 160 | 161 | In the first place, as I have argued elsewhere [8], the flowchart is a very poor abstraction of software structure. Indeed, it is best viewed as Burks, von Neumann, and Goldstine's attempt to provide a desperately needed high-level control language for their proposed computer. In the pitiful, multipage, connection-boxed form to which the flowchart has today been elaborated, it has proved to be useless as a design tool--programmers draw flowcharts after, not before, writing the programs they describe. 162 | 163 | Second, the screens of today are too small, in pixels, to show both the scope and the resolution of any seriously detailed software diagram. The so-called "desktop metaphor" of today's workstation is instead an "airplane-seat" metaphor. Anyone who has shuffled a lap full of papers while seated between two portly passengers will recognize the difference--one can see only a very few things at once. The true desktop provides overview of, and random access to, a score of pages. Moreover, when fits of creativity run strong, more than one programmer or writer has been known to abandon the desktop for the more spacious floor. The hardware technology will have to advance quite substantially before the scope of our scopes is sufficient for the software-design task. 164 | 165 | More fundamentally, as I have argued above, software is very difficult to visualize. Whether one diagrams control flow, variable-scope nesting, variable cross references, dataflow, hierarchical data structures, or whatever, one feels only one dimension of the intricately interlocked software elephant. If one superimposes all the diagrams generated by the many relevant views, it is difficult to extract any global overview. The VLSI analogy is fundamentally misleading--a chip design is a layered two-dimensional description whose geometry reflects its realization in 3-space. A software system is not. 166 | 167 | Program verification. Much of the effort in modern programming goes into testing and the repair of bugs. Is there perhaps a silver bullet to be found by eliminating the errors at the source, in the system-design phase? Can both productivity and product reliability be radically enhanced by following the profoundly different strategy of proving designs correct before the immense effort is poured into implementing and testing them? 168 | 169 | I do not believe we will find productivity magic here. Program verification is a very powerful concept, and it will be very important for such things as secure operating-system kernels. The technology does not promise, however, to save labor. Verifications are so much work that only a few substantial programs have ever been verified. 170 | 171 | Program verification does not mean error-proof programs. There is no magic here, either. Mathematical proofs also can be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it. 172 | 173 | More seriously, even perfect program verification can only establish that a program meets its specification. The hardest part of the software task is arriving at a complete and consistent specification, and much of the essence of building a program is in fact the debugging of the specification. 174 | 175 | Environments and tools. How much more gain can be expected from the exploding researches into better programming environments? One's instinctive reaction is that the big-payoff problems-- hierarchical file systems, uniform file formats to make possible uniform program interfaces, and generalized tools--were the first attacked, and have been solved. Language-specific smart editors are developments not yet widely used in practice, but the most they promise is freedom from syntactic errors and simple semantic errors. 176 | 177 | Perhaps the biggest gain yet to be realized from programming environments is the use of integrated database systems to keep track of the myriad details that must be recalled accurately by the individual programmer and kept current for a group of collaborators on a single system. 178 | 179 | Surely this work is worthwhile, and surely it will bear some fruit in both productivity and reliability. But by its very nature, the return from now on must be marginal. 180 | 181 | Workstations. What gains are to be expected for the software art from the certain and rapid increase in the power and memory capacity of the individual workstation? Well, how many MIPS can one use fruitfully? The composition and editing of programs and documents is fully supported by today's speeds. Compiling could stand a boost, but a factor of 10 in machine speed would surely leave thinktime the dominant activity in the programmer's day. Indeed, it appears to be so now. 182 | 183 | More powerful workstations we surely welcome. Magical enhancements from them we cannot expect. 184 | 185 | Promising Attacks on the Conceptual Essence 186 | 187 | Even though no technological breakthrough promises to give the sort of magical results with which we are so familiar in the hardware area, there is both an abundance of good work going on now, and the promise of steady, if unspectacular progress. 188 | All of the technological attacks on the accidents of the software process are fundamentally limited by the productivity equation: 189 | 190 | 191 | If, as I believe, the conceptual components of the task are now taking most of the time, then no amount of activity on the task components that are merely the expression of the concepts can give large productivity gains. 192 | 193 | Hence we must consider those attacks that address the essence of the software problem, the formulation of these complex conceptual structures. Fortunately, some of these attacks are very promising. 194 | 195 | Buy versus build. The most radical possible solution for constructing software is not to construct it at all. 196 | 197 | Every day this becomes easier, as more and more vendors offer more and better software products for a dizzying variety of applications. While we software engineers have labored on production methodology, the personal-computer revolution has created not one, but many, mass markets for software. Every newsstand carries monthly magazines, which sorted by machine type, advertise and review dozens of products at prices from a few dollars to a few hundred dollars. More specialized sources offer very powerful products for the workstation and other Unix markets. Even software tools and environments can be bought off-the-shelf. I have elsewhere proposed a marketplace for individual modules. [9] 198 | 199 | Any such product is cheaper to buy than to build afresh. Even at a cost of one hundred thousand dollars, a purchased piece of software is costing only about as much as one programmeryear. And delivery is immediate! Immediate at least for products that really exist, products whose developer can refer products to a happy user. Moreover, such products tend to be much better documented and somewhat better maintained than home-grown software. 200 | 201 | The development of the mass market is, I believe, the most profound long-run trend in software engineering. The cost of software has always been development cost, not replication cost. Sharing that cost among even a few users radically cuts the per-user cost. Another way of looking at it is that the use of n copies of a software system effectively multiplies the productivity of its developers by n. That is an enhancement of the productivity of the discipline and of the nation. 202 | 203 | The key issue, of course, is applicability. Can I use an available off-the-shelf package to perform my task? A surprising thing has happened here. During the 1950's and 1960's, study after study showed that users would not use off-the-shelf packages for payroll, inventory control, accounts receivable, and so on. The requirements were too specialized, the case-to-case variation too high. During the 1980's, we find such packages in high demand and widespread use. What has changed? 204 | 205 | Not the packages, really. They may be somewhat more generalized and somewhat more customizable than formerly, but not much. Not the applications, either. If anything, the business and scientific needs of today are more diverse and complicated than those of 20 years ago. 206 | 207 | The big change has been in the hardware/software cost ratio. In 1960, the buyer of a two-million dollar machine felt that he could afford $250,000 more for a customized payroll program, one that slipped easily and nondisruptively into the computer-hostile social environment. Today, the buyer of a $50,000 office machine cannot conceivably afford a customized payroll program, so he adapts the payroll procedure to the packages available. Computers are now so commonplace, if not yet so beloved, that the adaptations are accepted as a matter of course. 208 | 209 | There are dramatic exceptions to my argument that the generalization of software packages has changed little over the years: electronic spreadsheets and simple database systems. These powerful tools, so obvious in retrospect and yet so late in appearing, lend themselves to myriad uses, some quite unorthodox. Articles and even books now abound on how to tackle unexpected tasks with the spreadsheet. Large numbers of applications that would formerly have been written as custom programs in Cobol or Report Program Generator are now routinely done with these tools. 210 | 211 | Many users now operate their own computers day in and day out on various applications without ever writing a program. Indeed, many of these users cannot write new programs for their machines, but they are nevertheless adept at solving new problems with them. 212 | 213 | I believe the single most powerful software-productivity strategy for many organizations today is to equip the computer-naive intellectual workers who are on the firing line with personal computers and good generalized writing, drawing, file, and spreadsheet programs and then to turn them loose. The same strategy, carried out with generalized mathematical and statistical packages and some simple programming capabilities, will also work for hundreds of laboratory scientists. 214 | 215 | Requirements refinement and rapid prototyping. The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later. 216 | 217 | Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements. For the truth is, the client does not know what he wants. The client usually does not know what questions must be answered, and he has almost never thought of the problem in the detail necessary for specification. Even the simple answer_"Make the new software system work like our old manual information-processing system"_is in fact too simple. One never wants exactly that. Complex software systems are, moreover, things that act, that move, that work. The dynamics of that action are hard to imagine. So in planning any software-design activity, it is necessary to allow for an extensive iteration between the client and the designer as part of the system definition. 218 | 219 | I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product. 220 | 221 | Therefore, one of the most promising of the current technological efforts, and one that attacks the essence, not the accidents, of the software problem, is the development of approaches and tools for rapid prototyping of systems as prototyping is part of the iterative specification of requirements. 222 | 223 | A prototype software system is one that simulates the important interfaces and performs the main functions of the intended system, while not necessarily being bound by the same hardware speed, size, or cost constraints. Prototypes typically perform the mainline tasks of the application, but make no attempt to handle the exceptional tasks, respond correctly to invalid inputs, or abort cleanly. The purpose of the prototype is to make real the conceptual structure specified, so that the client can test it for consistency and usability. 224 | 225 | Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy. Hence, they cannot be fixed without fundamental revision--revision that provides for iterative development and specification of prototypes and products. 226 | 227 | Incremental development--grow, don't build, software. I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash he broadened my whole view of the software process. The metaphor shift was powerful, and accurate. Today we understand how like other building processes the construction of software is, and we freely use other elements of the metaphor, such as specifications, assembly of components, and scaffolding. 228 | 229 | The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be specified accurately in advance, and too complex to be built faultlessly, then we must take a radically different approach. 230 | 231 | Let us turn nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and selfrenewing. The secret is that it is grown, not built. 232 | 233 | So it must be with our software-systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development. [10] That is, the system should first be made to run, even if it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit, it should be fleshed out, with the subprograms in turn being developed--into actions or calls to empty stubs in the level below. 234 | 235 | I have seen most dramatic results since I began urging this technique on the project builders in my Software Engineering Laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness. The approach necessitates top-down design, for it is a top-down growing of the software. It allows easy backtracking. It lends itself to early prototypes. Each added function and new provision for more complex data or circumstances grows organically out of what is already there. 236 | 237 | The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build. 238 | 239 | The same benefits can be realized on large projects as on my small ones. [11] 240 | 241 | Great designers. The central question in how to improve the software art centers, as it always has, on people. 242 | 243 | We can get good designs by following good practices instead of poor ones. Good design practices can be taught. Programmers are among the most intelligent part of the population, so they can learn good practice. Hence, a major thrust in the United States is to promulgate good modern practice. New curricula, new literature, new organizations such as the Software Engineering Institute, all have come into being in order to raise the level of our practice from poor to good. This is entirely proper. 244 | 245 | Nevertheless, I do not believe we can make the next step upward in the same way. Whereas the difference between poor conceptual designs and good ones may lie in the soundness of design method, the difference between good designs and great ones surely does not. Great designs come from great designers. Software construction is a creative process. Sound methodology can empower and liberate the creative mind; it cannot inflame or inspire the drudge. 246 | 247 | The differences are not minor--they are rather like the differences between Salieri and Mozart. Study after study shows that the very best designers produce structures that are faster, smaller, simpler, cleaner, and produced with less effort. [12] The differences between the great and the average approach an order of magnitude. 248 | 249 | A little retrospection shows that although many fine, useful software systems have been designed by committees and built as part of multipart projects, those software systems that have excited passionate fans are those that are the products of one or a few designing minds, great designers. Consider Unix, APL, Pascal, Modula, the Smalltalk interface, even Fortran; and contrast them with Cobol, PL/I, Algol, MVS/370, and MS-DOS. (See Table 1.) 250 | 251 | Hence, although I strongly support the technology-transfer and curriculumdevelopment efforts now under way, I think the most important single effort we can mount is to develop ways to grow great designers. 252 | 253 | No software organization can ignore this challenge. Good managers, scarce though they be, are no scarcer than good designers. Great designers and great managers are both very rare. Most organizations spend considerable effort in finding and cultivating the management prospects; I know of none that spends equal effort in finding and developing the great designers upon whom the technical excellence of the products will ultimately depend. 254 | 255 | Table 1. Exciting Vs. Useful But Unexciting Software Products. 256 | Exciting Products 257 | Yes No 258 | Unix Cobol 259 | APL PL/I 260 | Pascal Algol 261 | Modula MVS/370 262 | Smalltalk MS-DOS 263 | Fortran 264 | My first proposal is that each software organization must determine and proclaim that great designers are as important to its success as great managers are, and that they can be expected to be similarly nurtured and rewarded. Not only salary, but the perquisites of recognition--office size, furnishings, personal technical equipment, travel funds, staff support--must be fully equivalent. 265 | 266 | How to grow great designers? Space does not permit a lengthy discussion, but some steps are obvious: 267 | 268 | Systematically identify top designers as early as possible. The best are often not the most experienced. 269 | Assign a career mentor to be responsible for the development of the prospect, and carefully keep a career file. 270 | Devise and maintain a careerdevelopment plan for each prospect, including carefully selected apprenticeships with top designers, episodes of advanced formal education, and short courses, all interspersed with solo-design and technicalleadership assignments. 271 | Provide opportunities for growing designers to interact with and stimulate each other. 272 | Acknowledgments 273 | 274 | I thank Gordon Bell, Bruce Buchanan, Rick Hayes-Roth, Robert Patrick, and, most especially, David Parnas for their insights and stimulating ideas, and Rebekah Bierly for the technical production of this article. 275 | REFERENCES 276 | 277 | [1] D.L. Parnas, "Designing Software for Ease of Extension and Contraction," IEEE Transactions on Software Engineering, Vol. 5, No. 2, March 1979, pp. 128-38. 278 | 279 | [2] G. Booch, "Object-Oriented Design," Software Engineering with Ada, 1983, Menlo Park, Calif.: Benjamin/ Cummings. 280 | 281 | [3] lEEE Transactions on Software Engineering (special issue on artificial intelligence and software engineering), l. Mostow, guest ed., Vol. 11, No. 11, November 1985. 282 | 283 | [4] D.L. Parnas, "Software Aspects of Strategic Defense Systems:" American Scientist, November 1985. 284 | 285 | [5] R. Balzer, "A 15-year Perspective on Automatic Programming," IEEE Transactions on Software Engineering (special issue on artificial intelligence and software engineering), J. Mostow, guest ed., Vol. 11, No. 11 (November 1985), pp. 1257-67. 286 | 287 | [6] Computer (special issue on visual programrning), R.B. Graphton and T. Ichikawa, guest eds., Vol. 18, No. 8, August 1985. 288 | 289 | [7] G. Raeder, "A Survey of Current Graphical Programming Techniques," Computer (special issue on visual programming), R.B. Graphton and T. Ichikawa, guest eds., Vol. 18, No. 8, August 1985, pp. 11-25. 290 | 291 | [8] F.P. Brooks, The Mythical Man Month, Reading, Mass.: Addison-Wesley, 1975, Chapter 14. 292 | 293 | [9] Defense Science Board, Report of the Task Force on Military Software in press. 294 | 295 | [10] H.D. Mills, "Top-Down Programming in Large Systems," in Debugging Techniques in Large Systems, R. Ruskin, ed., Englewood Cliffs, N.J.: Prentice-Hall, 1971. 296 | 297 | [11] B.W. Boehm, "A Spiral Model of Software Development and Enhancement," 1985, TRW Technical Report 21-371-85, TRW, Inc., 1 Space Park, Redondo Beach, Calif. 90278. 298 | 299 | [12] H. Sackman, W.J. Erikson, and E.E. Grant, "Exploratory Experimental Studies Comparing Online and Offline Programming Performance," Communications of the ACM, Vol. 11, No. 1 (January 1968), pp. 3-11. 300 | 301 | Brooks, Frederick P., "No Silver Bullet: Essence and Accidents of Software Engineering," Computer, Vol. 20, No. 4 (April 1987) pp. 10-19. 302 | -------------------------------------------------------------------------------- /articles/parallelism-and-concurrency-need-different-tools.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: Yossi Kreinin 3 | source: http://yosefk.com/blog/parallelism-and-concurrency-need-different-tools.html 4 | --- 5 | 6 |

Parallelism and concurrency need different tools

7 | 8 |

concurrent (noun): Archaic. a rival or competitor.

9 |

– dictionary.com

10 |

Two lines that do not intersect are called parallel lines.

11 |

– Wikipedia

12 |

In this piece, I disagree with Joe Armstrong and Rob Pike, basing my argument on the differences between vending machines and gift boxes (illustrated with drawings carefully prepared in Microsoft Paint).

13 |

Parallelism and concurrency are both very fashionable notions. Lots of languages and tools are advertised as good at these things – often at both things.

14 |

I believe that concurrency and parallelism call for very different tools, and each tool can be really good at either one or the other. To oversimplify:

15 | 19 |

It's possible for both kinds of tools to co-exist in a single language or system. For example, Haskell appears to have good tools for both concurrency and parallelism. But it's still two different sets of tools, and the Haskell wiki explains that you shouldn't use the concurrency tools if you're after parallelism:

20 |

Rule of thumb: use Pure Parallelism if you can, Concurrency otherwise.

21 |

The people behind Haskell realize that one tool for both isn't good enough. It's a similar story with the new language ParaSail – it offers both kinds of tools, and advises to avoid its concurrency features in parallel, non-concurrent programs.

22 |

This is in stark contrast to the opinion of some people primarily interested in concurrency, who think that their tools are a great fit for parallelism. Rob Pike says Go makes parallelism easy because it's a concurrent language:

23 |

Concurrency makes parallelism (and scaling and everything else) easy.

24 |

Likewise, Joe Armstrong claims that Erlang is great for parallelism because it's a concurrent language. He goes as far as to say that seeking to parallelize "legacy" code written in non-concurrent languages is "solving the wrong problem".

25 |

What causes this difference in opinions? Why do Haskell and ParaSail people think you need separate tools for concurrency and parallelism, while Go and Erlang people think that concurrent languages handle parallelism just fine?

26 |

I think people reach these different conclusions because they work on different problems, and develop a different perception of the basic distinction between concurrency and parallelism. Joe Armstrong has drawn a picture to explain how he sees that distinction. I'll draw you a different picture – but first, here's a reproduction of his:

27 |

Concurrency-centric view of concurrency vs parallelism

28 |

Actually many aspects of concurrency are present with just one queue, but I reproduced the 2 queues from Armstrong's original drawing. So what does this picture say?

29 | 36 |

Indeed, that's all there is to it with a vending machine. What about giving gifts to a bunch of kids? Is there a difference between coke cans and presents?

37 |

In fact there is. The vending machine is an event handling system: people come at unpredictable moments, and find an unpredictable amount of cans in the machine. The gift boxes is a computational system: you know the kids, you know what you bought, and you decide who gets what and how.

38 |

In the gift boxes scenario, "concurrent vs parallel" looks very different:

39 |

Parallelism-centric view of concurrency vs parallelism

40 |

How is "concurrent" different from "parallel" in this example?

41 | 47 |

Queueing in front of a gift heap is concurrent in that archaic sense of "rivalry, competition": you want to be there first, so that it's you who gets the Lego set. In Russian, "concurrent" (pronounced kon-koo-rent) is the contemporary word for "competitor".

48 |

With labeled gifts, there's no competition. Logically, labels "connect" each kid with his gift, and these are parallel, non-intersecting, non-conflicting lines. (Why did I draw the arrows so that they do intersect? We'll get back to it later; a good way to think of it is, kids'/processors' paths do cross as they search for their gifts/data – but those intersections are not conflicts over who gets what.)

49 |

Computation vs event handling

50 |

With event handling systems such as vending machines, telephony, web servers and banks, concurrency is inherent to the problem – you must resolve inevitable conflicts between unpredictable requests. Parallelism is a part of the solution - it speeds things up, but the root of the problem is concurrency.

51 |

With computational systems such as gift boxes, graphics, computer vision and scientific computing, concurrency is not a part of the problem – you compute an output from inputs known in advance, without any external events. Parallelism is where the problems start – it speeds things up, but it can introduce bugs.

52 |

Let's proceed to discuss the differences between computational systems and event handling systems. We'll start with obvious things, like determinism, but there are many more subtle consequences.

53 |

Determinism: desirable vs impossible

54 |

In computational systems, determinism is desirable because it makes life easier in many ways. For example, it's nice to test optimizations and refactorings by observing that results didn't change – which you can only do with deterministic programs.

55 |

Determinism is often not strictly required – you may genuinely not care which 100 images you find as long as they all have kittens in them, or exactly what approximation of PI you compute as long as it's somewhere between 3 and 4. Determinism is merely very nice – and possible.

56 |

In event handling systems, however, determinism is impossible, in the sense that it is a requirement for different orders of events to produce different results. If I got there first, I should get the last Coke can, the last dollar in the shared bank account, etc. If you beat me to it by a millisecond, then it should go to you.

57 |

Of course, for basic debugging, you can always run the system completely serially, handling one event a time. But once you make it a bit more realistic and start handling events without first finishing everything that you were doing, you can no longer expect a specific result.

58 |

Even in a debugging environment, if you replay an event log with two users racing to withdraw money from a shared bank account, you'll run the thing twice and it might reach two different final states. The number of possible final states is exponential in the number of conflicts.

59 |

Ouch.

60 |

Sign of parallel safety: determinism vs correctness

61 |

How do you know that you have no bugs due to the different ordering of events?

62 |

In computational systems, if you get the same result every time, you probably have no parallelism bugs – even if the result is busted. Dog pictures instead of cats means you have bugs – but not parallelism bugs if it's the same dogs every time.

63 |

In event handling systems, the only sure sign of not having parallelism bugs is if you always get correct results.

64 |

With two users racing to withdraw money, you can't expect to always reach the same result. What can you expect, assuming the bank isn't buggy? Many things. You (presumably) can never have a negative account balance, you (presumably) can't drain a bank account twice, in effect creating money, etc.

65 |

If none of these wrong things ever happen, then you got yourself a functioning bank, otherwise you have a bug. With a telephony system, it's similar – except that there's a different set of requirements defining "correct results".

66 |

There's no general way to tell if there are timing-related bugs without understanding the aspects of the system having nothing to do with timing.

67 |

Ouch!

68 |

Parallelism bugs: easy to pinpoint vs impossible to define

69 |

With labeled gift boxes, parallelism bugs are trivial to spot – even if the labels are in Chinese, and you can't read Chinese:

70 |

71 |

Even if you can't read the labels, knowing that they exist is enough. If two kids fight over a box, then something is wrong and you call an adult who can read the labels.

72 |

If you're an automated debugging tool, and you don't know which task is supposed to process which data, it's enough to know that a task shouldn't access data modified by another task. If that happens, you tell the programmer, so that he properly assigns data to tasks without conflicts.

73 |

Such tools aren't hypothetical – they're out there. Cilk comes with a tool that does that. Checkedthreads has a Valgrind-based tool that does that.

74 |

Parallel Haskell doesn't have to do it – because you have no side effects to begin with, guaranteeing zero conflicts when things are evaluated in parallel. But even with dynamic checking instead of static guarantees, your parallelism bugs are basically just gone.

75 |

The upshot is that in computational systems, you don't need to know much to flag bugs and to point to the exact problem they cause. "Here's the box they were fighting over".

76 |

In event handling systems, a responsible adult must be much more knowledgeable to maintain discipline.

77 |

To be sure, there still is a weaker, but universal rule which always applies: you can't touch anything if someone is already busy with it. You must wait your turn in a queue. An adult can make sure this rule is followed, which is better than nothing. But it's not enough to prevent many common kinds of mischief:

78 |

79 |

Someone approaching the presents without queuing is clearly wrong – bug detected, order restored.

80 |

But someone unpacking a present, leaving the queue, and coming back to find that the present was taken by someone else could be wrong – or it could be OK. After all, that other kid waited his turn, so the only universal rule of event handling systems was followed – but not necessarily the specific rules of how we give out presents around here.

81 |

Here's how these problems look in code. The following buggy money transfer implementation can be flagged as buggy by an automated debugging tool:

82 | 
src.balance -= amount
 83 | dst.balance += amount
84 |

Here we have no synchronization at all – src.balance can be modified by two processes without one of them waiting for the other to be done with it. So you could lose some of the decrements or what-not. A data race detector like Helgrind will spot this bug by monitoring memory accesses and observing the lack of synchronization – just as easily as Cilk's or checkedthreads' checker.

85 |

However, the version below is probably still buggy, but it would look fine to data race detectors:

86 | 
atomic { src.balance -= amount }
 87 | atomic { dst.balance += amount }
88 |

Here, "atomic" means that everyone has to wait before modifying the balances in some sort of queue – possibly a very fancy sort of queue, but a queue nonetheless; more on that later. Orderly queuing makes data race detectors happy – but is there still a bug?

89 |

A thread could be suspended after decrementing src.balance but before incrementing dst.balance, exposing an intermediate state where money is "temporarily lost". Is this a problem? Perhaps. Do you understand banking? I don't, not really – and Helgrind surely doesn't.

90 |

Here's a more obviously wrong program:

91 | 
if src.balance - amount > 0:
 92 |   atomic { src.balance -= amount }
 93 |   atomic { dst.balance += amount }
94 |

Here, a thread could test that src.balance has enough money to withdraw a given amount, and go to pee. Another thread arrives, makes a similar test and gets busy withdrawing the money. The first thread comes back, is put into some queue, waits his turn and withdraws its own amount – which could have become illegitimate, the first thread having withdrawn too much already.

95 |

This is a lot like coming back and realizing that another kid has walked away with your Apple iPhone®. It's a race condition which isn't a data race – that is, it happens despite everyone's civil and polite queuing.

96 |

What's a race condition? It depends on the application. I'm sure the last snippet is buggy, but I'm less sure about the previous snippet, and it all depends on how the bank works. And if we can't even define "race conditions" without knowing what the program is trying to do, we can't hope to automatically detect them.
97 |

98 |

Of course you can also not have race conditions. All you have to do is implement the entire withdrawal atomically, and of course all approaches to concurrency let you do it.

99 |

The trouble with race conditions though, as opposed to, say, null pointer exceptions, is that you can feed the program bug-triggering inputs again and again, and the thing will only reproduce once in a blue moon. Here's where deterministic, automated debugging could be really handy, flagging the bug every time you feed those inputs. Unfortunately, it's impossible with event handling systems.

100 |

"Ouch" again.

101 |

Queues: implementation detail vs part of the interface

102 |

With labeled gifts, you don't need a queue. Everyone can just go and take their present.

103 |

Or maybe not, not really. Thousands of kids getting thousands of labeled presents will need a queue or several, or else they'd be running into each other. How those queues work doesn't matter in the sense that everyone ends up with the same present anyway. The choice of a queuing method does affect how much time you wait for your present though.

104 |

That's why I drew the "logically parallel" lines connecting everyone to their labeled gifts so that they intersect – even though these aren't "actual conflicts" over who gets what. (I'm very careful with my metaphors and my Microsoft Paint art – very careful.)

105 |

4 processors accessing unrelated data items through a single memory bus is when "logically parallel lines technically cross", and in effect processors will have to wait in some hardware-level queue to make it work. 1000 logically independent tasks distributed to these 4 processors through a load balancing scheduler will again need queues to work.

106 |

There are almost always many queues even in a fully parallel, conflict-free system – but they're an implementation detail and they shouldn't affect results. You get the same thing no matter what your place was in any of the queues:

107 |

108 |

Conversely, in a concurrent system, the queue is right there in the interface:

109 | 116 |

With event systems, you can have simple queues or fancy queues, but they are all explicit queues in the sense that order often affects results, and that's OK – or it could be a race condition, go figure.

117 |

With computational, parallel, conflict-free systems, order shouldn't affect results – and you want tools to make sure the system is indeed conflict-free.

118 |

Rob Pike shows in his slides how easy it is to build a load balancer in Go. Sure it's easy. Go is for concurrency and concurrency is all about queues, and so is load balancing. Not that load balancers are very hard in any language, but yeah it's particularly easy in concurrent languages.

119 |

But that's one part of the parallelism story; the next thing you want is either static guarantees of not having conflicts, or dynamic checking. I'm not saying it can't be done in Go – sure it can be done – just that without having it, you underserve computational systems. And once you do have it, it's a different set of interfaces and tools from channels and goroutines – even if underneath it's implemented with channels and goroutines.

120 |

Importance of preemption

121 |

So conflict prevention/detection is something computational systems need that concurrency tools don't provide. Are there things that concurrency tools do provide that computational systems don't need?

122 |

Sure – explicit queues, for starters. Not only aren't they needed, but explicit queues get in the way of computational systems, because as we've seen, queues lead to race conditions that aren't data races and you can't pinpoint those.

123 |

Another thing computational systems don't really need is very cheap preemptable threads/processes.

124 |

With event handling systems, you sometimes want to react to many different events very quickly, which takes many preemptable processes. You want to have 10000 processes stuck in the middle of something, and process number 10001 can still get activated immediately when an event arrives. This can't work unless preemptable processes are very cheap.

125 |

With computational systems, it's nice to have cheap tasks that you can map to the relatively expensive OS threads – but tasks are not as powerful as processes. You don't activate tasks upon events. What tasks do is they wait in various queues, and get processed when a worker thread becomes idle to run them. Unlike the case with goroutines or the like, you can't have more tasks in flight than you have OS threads – and you don't need to.

126 |

Tasks can be done nicely with fairly traditional runtimes – as opposed to full-blown cheap processes/goroutines/whatever, which seem to me to need more work at the low-level runtime system side of things.

127 |

This shows how platforms aiming at concurrency not only under-serve computational systems, but over-serve them as well.

128 |

(To be fair, it is theoretically conceivable to gain something from preemption in a computational system – namely, it could improve throughput if it lets freshly created tasks which are a part of the critical path preempt in-flight tasks which are not. However, in my long and sad experience, a scheduler actually knowing what the critical paths are is little more than a theoretical possibility. And a dumb greedy scheduler has no use for preemption.)

129 |

Differences among tools from the same family

130 |

Tools aimed at concurrent event systems are not all alike, nor are tools aimed at parallel computational systems. They're from the same family, but there are substantial differences.

131 | 137 |

Of course the other big difference is that you'd be programming in Erlang, Rust, Go and Haskell, respectively. And now to computational systems:

138 | 145 |

I wrote checkedthreads, so this is the advertisement part; checkedthreads is portable, free, safe and available today in the very mainstream languages C and C++, unlike many systems requiring new languages or language extensions.

146 |

With Cilk, there's an effort to standardize it in C++1y or some such, but Cilk wants to add keywords and the C++ people don't like adding keywords. Cilk is available in branches of gcc and LLVM, but it won't run on all platforms at the moment (it extends the ABI) and it hasn't been merged into the mainline for a while. Some newer Cilk features are patented. Not all of it is freely available, etc. etc.

147 |

Cilk's big advantage, however, is that it's backed up by Intel, whereas checkedthreads is backed up by yours truly. And if Cilk suits you and you decide to use it as a result of my checkedthreads-related blogging, I will have achieved my goal of automated debugging of parallel programs getting some much-deserved attention.

148 |

The upshot is that not all concurrency tools are alike, and different parallelism tools are likewise different – I don't even claim to have pointed out the most important differences between my examples; it's hairy stuff. Still, they're two different families, and the first thing to do is to pick the right family.

149 |

Conclusion

150 |

We've discussed differences between parallel, computational systems and concurrent, event handling systems. Areas of differences include:

151 | 158 |

For event handling systems, concurrency is their essence and parallelism is a part of some solutions – typically good ones (two vending machines is better than one). For computational systems, parallelism is their essence and concurrency is a part of some solutions – typically bad ones (a heap of gifts is usually worse than labeled gifts).

159 |

I hope to have gained more clarity than confusion by occasionally conflating "parallelism/concurrency" with "computation/event handling". I also hope I didn't paint too dark a picture of event handling systems – perhaps there are automated verification strategies that I'm not aware of. However it came out, I don't claim that my viewpoint and terminology usage is "the" way of looking at this.

160 |

There's value in the angle preferred by some people interested in event handling systems – "concurrency is dealing with several things at a time, parallelism is doing several things at a time". From this angle, parallelism is an implementation detail, while concurrency is in the structure of the program.

161 |

I believe there's value in my perspective as well – namely, "concurrency is dealing with inevitable timing-related conflicts, parallelism is avoiding unnecessary conflicts" – "vending machines vs labeled gifts". Here's how the two look like – now with parallel arrows disentangled, as they logically are:

162 |

163 |

The most important takeaway is that computational code, as opposed to event handling code, can be made virtually bug-free rather easily, using either automated debugging tools or static guarantees.

164 |

Handling parallelism using its own tools is nothing new. Rob Pike's work on Sawzall, a specialized parallel language where code is always free from parallelism bugs, predates his work on the concurrent language Go.

165 |

However, tools for concurrency appear "louder" than tools for parallelism at the moment – and they can handle parallelism, albeit relatively badly. Loud and bad often crowds out the less visible better. It'll be a pity if better support for parallelism won't be developed as a side effect of "loud concurrency" – or if such support atrophies where already available.

166 |

I'd go as far as replying to Armstrong's "parallelizing serial code is solving the wrong problem" with "using 'bare' concurrency tools for computational code is applying your solution to the wrong problem". The simple fact is that C parallelized with the help of the right tools is faster and safer than Erlang.

167 |

So here's to "using the right tool for the job", and not having anyone walk away with your Apple iPhone®.

168 | -------------------------------------------------------------------------------- /articles/queues-dont-fix-overload.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: Fred Hebert 3 | source: http://ferd.ca/queues-don-t-fix-overload.html 4 | --- 5 | 6 | # Queues Don't Fix Overload 7 | 8 | OK, queues. 9 | 10 | People misuse queues all the time. The most egregious case being to fix issues with slow apps, and consequently, with overload. But to say why, I'll need to take bits of talks and texts I have around the place, and content that I have written in more details about in [Erlang in Anger](http://www.erlang-in-anger.com). 11 | 12 | To oversimplify things, most of the projects I end up working on can be visualized as a very large bathroom sink. User and data input are flowing from the faucet, down 'till the output of the system: 13 | 14 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink1.png) 15 | 16 | So under normal operations, your system can handle all the data that comes in, and carry it out fine: 17 | 18 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink2.png) 19 | 20 | Water goes in, water goes out, everyone's happy. However, from time to time, you'll see temporary overload on your system. If you do messaging, this is going to be around sporting events or events like New Year's Eve. If you're a news site, it's gonna be when a big thing happens (Elections in the US, Royal baby in the UK, someone says they dislike French as a language in Quebec). 21 | 22 | During that time, you may experience that temporary overload: 23 | 24 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink3.png) 25 | 26 | The data that comes out of the system is still limited, and input comes in faster and faster. Web people will use stuff like caches at that point to make it so the input and output required gets to be reduced. Other systems will use a huge buffer (a queue, or in this case, a sink) to hold the temporary data. 27 | 28 | The problem comes when you inevitably encounter prolonged overload. It's when you look at your system load and go "oh crap", and it's not coming down ever. Turns out Obama doesn't want to turn in his birth certificate, the royal baby doesn't look like the father, and someone says Quebec should be better off with Parisian French, and the rumor mill is going for days and weeks at a time: 29 | 30 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink4.png) 31 | 32 | All of a sudden, the buffers, queues, whatever, can't deal with it anymore. You're in a critical state where you can see smoke rising from your servers, or if in the cloud, things are as bad as usual, but more! 33 | 34 | The system inevitably crashes: 35 | 36 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink5.png) 37 | 38 | Woops, everyone is dead, you're in the office at 3am (who knew so many people in the US, disgusted with their "Kenyan" president, now want news on the royal baby, while Quebec people look up 'royale with cheese baby' for some reason) trying to keep things up. 39 | 40 | You look at your stack traces, at your queue, at your DB slow queries, at the APIs you call. You spend weeks at a time optimizing every component, making sure it's always going to be good and solid. Things keeps crashing, but you hit the point where every time, it takes 2-3 days more. 41 | 42 | At the end of it, you see a crapload of problems still happening, but they're a week apart between each failure, which slows down your optimizing in immense ways because it's incredibly hard to measure things when they take weeks to go bad. 43 | 44 | You go "okay I'm all out of ideas, let's buy a bigger server." The system in the end looks like this, and it's still failing: 45 | 46 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink6.png) 47 | 48 | Except now it's an unmaintainable piece of garbage full of dirty hacks to make it work that cost 5 times what it used to, and you've been paid for months optimizing it for no god damn reason because it still dies when overloaded. 49 | 50 | The problem? That red arrow there. You're hitting some hard limit that even through all of your profiling, you didn't consider properly. This can be a database, an API to an external service, disk speed, bandwidth or general I/O limits, paging speed, CPU limits, whatever. 51 | 52 | You've spent months optimizing your super service only to find out at some point in time, you went past its optimal speed without larger changes, and the day your system got to have an operational speed greater than this hard limit, you've doomed yourself to an everlasting series of system failures. 53 | 54 | The disheartening part about it is that you discover that once your system is popular, has people using it and its APIs, and changing it to be better is very expensive and hard. Especially since you'll probably have to revisit assumptions you've made in its core design. Woops. 55 | 56 | So what do you need? You'll need to pick _what has to give_ whenever stuff goes bad. You'll have to pick between blocking on input (back-pressure), or dropping data on the floor (load-shedding). And that happens all the time in the real world, we just don't want to do it as developers, as if it were an admission of failure. 57 | 58 | Bouncers in front of a club, water spillways to go around dams, the pressure mechanism that keeps you from putting more gas in a full tank, and so on. They're all there to impose a system-wide flow control to keep operations safe. 59 | 60 | In [non-critical] software? Who cares! We never shed load because that makes stakeholders angry, and we never think about back-pressure. Usually the back-pressure in the system is implicit: 'tis slow. 61 | 62 | A function/method call to something ends up taking longer? It's slow. Not enough people think of it as back-pressure making its way through your system. In fact, slow distributed systems are often the canary in the overload coal mine. The problem is that everyone just stands around and goes "durr why is everything so slow??" and devs go "I don't know! It just is! It's hard, okay!" 63 | 64 | That's usually because somewhere in the system (possibly the network, or something that is nearly impossible to observe without proper tooling, such as [TCP incast](http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/)), something is clogged and everything else is pushing it back to the edge of your system, to the user. 65 | 66 | And that back-pressure making the system slower? It slows down the rate at which users can input data. It's what is likely keeping your whole stack alive. And you know when people start using queues? Right there. When operations take too long and block stuff up, people introduce a freaking queue in the system. 67 | 68 | And the effects are instant. The application that was sluggish is now fast again. Of course you need to redesign the whole interface and interactions and reporting mechanisms to become asynchronous, but man is it fast! 69 | 70 | Except at some point the queue spills over, and you lose all of the data. There's a serious meeting that then takes place where everyone discusses how this could possibly have happened. Dev #3 suggests more workers are added, Dev #6 recommends the queue gets persistency so that when it crashes, no requests are lost. 71 | 72 | "Cool," says everyone. Off to work. Except at some point, the system dies again. And the queue comes back up, but it's already full and uuugh. Dev #5 goes in and thinks "oh yeah, we could add more queues" (I swear I've seen this unfold back when I didn't know better). People say "oh yeah, that increases capacity" and off they go. 73 | 74 | And then it dies again. And nobody ever thought of that sneaky red arrow there: 75 | 76 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink6.png) 77 | 78 | Maybe they do it without knowing, and decide to go with MongoDB because it's "faster than Postgres" (heh). Who knows. 79 | 80 | The real problem is that everyone involved used queues as an optimization mechanism. With them, new problems are now part of the system, which is a nightmare to maintain. Usually, these problems will come in the form of ruining the [end-to-end principle](https://en.wikipedia.org/wiki/End-to-end_principle) by using a persistent queue as a fire-and-forget mechanisms or assuming tasks can't be replayed or lost. You have more places that can time out, require new ways to detect failures and communicate them back to users, and so on. 81 | 82 | Those can be worked around, don't get me wrong. The issue is that they're being introduced as part of a solution that's not appropriate for the problem it's built to solve. All of this was just premature optimization. Even when everyone involved took measures, reacted to real failures in real pain points, etc. The issue is that nobody considered what the true, central business end of things is, and what its limits are. People considered these limits locally in each sub-component, more or less, and not always. 83 | 84 | But someone should have picked what had to give: do you stop people from inputting stuff in the system, or do you shed load. Those are inescapable choices, where inaction leads to system failure. 85 | 86 | And you know what's cool? If you identify these bottlenecks you have for real in your system, and you put them behind proper back-pressure mechanisms, your system won't even have the right to become slow. 87 | 88 | Step 1. Identify the bottleneck. Step 2: ask the bottleneck for permission to pile more data in: 89 | 90 | ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink7.png) ![](https://raw.githubusercontent.com/montanaflynn/programming-articles/master/assets/img/sink8.png) 91 | 92 | Depending on where you put your probe, you can optimize for different levels of latency and throughput, but what you're going to do is define proper operational limits of your system. 93 | 94 | When people blindly apply a queue as a buffer, all they're doing is creating a bigger buffer to accumulate data that is in-flight, only to lose it sooner or later. You're making failures more rare, but you're making their magnitude worse. 95 | 96 | When you shed load and define proper operational limits to your system, you don't have these. What you may have is customers that are as unhappy (because in either case, they can't do what your system promises right), but with proper back-pressure or load-shedding, you gain: 97 | 98 | - Proper metrics of your quality of service 99 | - An API that will be designed with either in mind (back-pressure lets you know when you're in an overload situation, and when to retry or whatever, and load-shedding lets the user know that some data was lost so they can work around that) 100 | - Fewer night pages 101 | - Fewer critical rushes to get everything fixed because it's dying all the time 102 | - A way to monetize your services through varying account limits and priority lanes 103 | - You act as a more reliable endpoint for everyone who depends on you 104 | 105 | To make stuff usable, a proper idempotent API with end-to-end principles in mind will make it so these instances of back-pressure and load shedding should rarely be a problem for your callers, because they can safely retry requests and know if they worked. 106 | 107 | So when I rant about/against queues, it's because queues will often be (but not always) applied in ways that totally mess up end-to-end principles for no good reason. It's because of bad system engineering, where people are trying to make an 18-wheeler go through a straw and wondering why the hell things go bad. In the end the queue just makes things worse. And when it goes bad, it goes really bad, because everyone tried to close their eyes shut and ignore the fact they built a dam to solve flooding problems upstream of the dam. 108 | 109 | And then of course, there's the use case where you use the queue as a messaging mechanism between front-end threads/processes (think PHP, Ruby, CGI apps in general, and so on) because your language doesn't support inter-process communications. It's marginally better than using a MySQL table (which I've seen done a few times and even took part in), but infinitely worse than picking a tool that supports the messaging mechanisms you need to implement your solution right. 110 | -------------------------------------------------------------------------------- /articles/signs-that-youre-a-good-programmer.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: C. Lawrence Wenham 3 | source: http://www.yacoset.com/Home/signs-that-you-re-a-good-programmer 4 | --- 5 | 6 | ### Signs that you're a good programmer 7 | 8 | The most frequently viewed page on this site is [Signs you're a bad programmer](http://www.yacoset.com/Home/signs-that-you-re-a-bad-programmer), which has also now been published on dead trees by [Hacker Monthly](http://hackermonthly.com/issue-24.html), and I think that behoves me to write its antithesis. "Bad programmer" is also considered inflammatory by some who think I'm speaking down to them. Not so; it was personal catharsis from an author who exhibited many of those problems himself. And what I think made the article popular was the "remedies"--I didn't want someone to get depressed when they recognized themselves, I wanted to be constructive. 9 | 10 | Therefore if you think you're missing any of the qualities below, don't be offended. I didn't pick these up for a while, either, and many of them came from watching other programmers or reading their code 11 | 12 | ### 1\. The instinct to experiment first 13 | 14 | The compiler and runtime can often answer a question faster than a human can. Rather than seek out a senior programmer and ask them "will it work if I do this?", a good programmer will just _try it_ and see if it works before bringing their problem to someone else 15 | 16 | #### Symptoms 17 | 18 | 1. Side projects 19 | 2. Dabbling in other programming languages, especially ones from a different "family" (procedural, stack-based, concurrent, etc.) 20 | 3. Knows what you're talking about when you mention "Arduino" 21 | 4. Old, uncommitted code that duplicates other code's functionality but isn't referenced elsewhere in the project 22 | 5. A tendency to suggest wacky and unrealistic solutions in meetings 23 | 6. A cubicle or desk populated with toys that came from ThinkGeek 24 | 25 | #### How to acquire this trait 26 | 27 | Are you excessively cautious? Are you only comfortable when you have permission? Has anyone ever said that you were passive aggressive? You might consider inviting some friends to visit the local Six Flags or some other roller-coaster park. If you want baptism by fire, then make your first ride the scariest (for me, it was the "[Drop Zone](http://www.themeparkreview.com/photos/kingsdominion/dropzone.htm)" at King's Dominion, with some reinforcement a few years later on the [Kingda Ka](http://www.sixflags.com/greatAdventure/rides/Kingdaka.aspx) at Six Flags). If you consider yourself ready to get off the kiddie rides you might try your hand at hang gliding and windsurfing, which have the benefit of teaching you what you can and cannot control 28 | 29 | Much of what makes people timid to experiment is chemical--your brain has a small number of adrenergic receptors, so a little bit of adrenaline excites your fight-or-flight reflexes too much. But consider why people grow tolerant to coffee: the caffeine's byproducts force their brain to grow more adenosine receptors. So if you force your brain to grow more adrenaline receptors then the same amount of "fear juice" will trigger a lower percentage of them. Find some experience that scares the shit out of you, do it a few times, and you will lose your fear of venture on a physical level 30 | 31 | **Note:** A programmer who "suggests wacky and unrealistic solutions" is not always a bad programmer. It can be a sign of creative thinking from someone who assumes confirmation or correction will come from somewhere else down the line. 32 | 33 | ### 2\. Emotional detachment from code and design 34 | 35 | Code is like kleenex: you use it when it's useful and throw it away when it no longer serves. We all like to think that _code-reuse_ is important, and while it is, it's not meant to be about raising a child. Code doesn't feel. Code doesn't care. Code will turn on you like a Frankenstein monster. Code is just bytes. Code is a liability 36 | 37 | #### Symptoms 38 | 39 | 1. Almost no committed code that is commented out 40 | 2. Willingly throws away weeks or months of work in order to adopt another programmer's superior code 41 | 3. Puts finger to lips, furrows brow and says "hmm" when faults in their work are pointed out, _while looking at the code and not the critic_ 42 | 4. Indifferent to the way the IDE wants to auto-format code, uninterested in "tabs-vs-spaces" arguments 43 | 5. Refers to it as "_the_ code" rather than "_my_ code", unless accepting blame 44 | 6. Has abandoned a design of theirs that was previously used in a successful product 45 | 7. Doesn't become defensive when the boss mentions that they're looking for an off-the-shelf alternative to what they've been writing for the past few years 46 | 47 | #### How to acquire this trait 48 | 49 | Konrad Lorenz, the author of _[On Aggression](http://www.amazon.com/gp/product/0156687410/ref=as_li_ss_tl?ie=UTF8&tag=synesmedia-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0156687410)_, suggested that a scientist should begin each day by throwing out one of his pet theories in order to remain sharp. Consider throwing out one of your pet algorithms or design patterns or exquisite one-line Sodoku solvers every morning to remind yourself that it's you who controls the idea, not the idea that controls you 50 | 51 | Find the code that you're the most proud of and delete it, now re-write it from scratch in a different way. Use a "design pattern" that confuses you, or that you _hate_ (e.g.: the Singleton) and figure out how to make it work. If necessary, delete that after you've got it working and try again with a new pattern or language. Not only will you learn that there's More Than One Way To Do It, but you'll learn that your code is transitory. Code, by its nature, is not just inextricably glued to its language, platform, and the APIs it consumes, but written in the form of ephemeral static charges, orientations of magnetic particles, subject to the whims of the market, Moore's Law, and your employer 52 | 53 | Other techniques to break the abusive relationship 54 | 55 | 1. Maintain somebody else's code 56 | 2. Experience, either by accident or bloody intention, what it's like to lose a week's work to a failed backup or a botched commit and have to re-write it all over again 57 | 3. Work for start-ups where you'll get laid-off when the second or third round of financing doesn't come through 58 | 4. Be stupid enough to post your best code on Reddit 59 | 5. Read the bit about "Destructive pursuit of perfection" further down in this article 60 | 61 | ### 3\. Eager to fix what isn't broken 62 | 63 | Programs are infrastructure: they're built to serve a specific need, but **[needs always change](http://www.yacoset.com/Home/why-nobody-knows-what-they-are-doing)**. Good programmers realize that hard-coded values buried in code are bad, that a [destoryBaghdad() function is immoral](http://www.linfo.org/q_programming.html), and that it's a priority to eliminate "code smells". Not for pride. Not for backslapping attaboys from your peers or the authors of methodology books. But because **you will itch until it is fixed** 64 | 65 | #### Symptoms 66 | 67 | 1. Doesn't take the spec by its word and tries to find out who wrote it and what they were thinking 68 | 2. Hunts down and talks to the people who will use the program each day 69 | 3. Owns a book written by a guy called [Martin Fowler](http://www.amazon.com/gp/product/0201485672/ref=as_li_ss_tl?ie=UTF8&tag=synesmedia-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0201485672) 70 | 4. Tends to express _extreme like_ or _dislike_ for popular technologies such as XML, ORM and REST, and has also **switched positions** on one or more of these in the past 71 | 5. Likes to use abstraction layers, but doesn't like to add more than one layer on top of what's already in the language or platform 72 | 6. Talks about "low cohesion" 73 | 7. At least 10% or more of their commits reduce the line-count of the project without adding new functionality 74 | 8. Before adding a new feature, checks to see if an existing one can be re-designed to perform both tasks or replaced entirely with a better method 75 | 76 | #### How to acquire this trait 77 | 78 | The first attempt to solve a program in code will always bear the artifacts of discovery: discovering the true nature of the problem, discovering the features of the platform, and discovering the best way to solve it. The second attempt will be more stable, but might inherit too much cautionary baggage and become a nightmare to extend. And so many programs today are like the [Firth of Forth Bridge](http://en.wikipedia.org/wiki/Forth_Bridge): disgustingly over-engineered. Sometimes it's the developer's first crack at the problem and looks like a lawn mowed by a dog, sometimes it's their second attempt and looks like the dog installed grass-cutting laser turrets every 2 feet. It can take a third try before the designer understands the problem completely and knows how much, or how _little_ they need to do 79 | 80 | Code lets you learn in stages where you don't need to re-write everything from scratch. You re-write pieces after you understand what they need to do and what they'll never need to do, make them simpler, shorter and beautiful 81 | 82 | Go through your home and repair all the annoying things you've been putting off; fix the crooked picture on the wall, unclog the slow draining sink, repair that gutter drainpipe so your basement doesn't flood, buy a UPS and backup drive for your computer and configure them to shut-down/back-up automatically, replace all the incandescents with efficient bulbs, replace that ethernet cable draped down the hallway with WiFi or some proper wall-jacks and conduit, get a real food-dish for your cat instead of that old cheese-dip container 83 | 84 | Next you should go to your last project and read through the code. Think about what each piece does. There's a loop here, some sorting there, a bit of number crunching, screen updates, HTML generation, database CRUD, that sort of thing 85 | 86 | Now replace the hard-coded HTML with a templating system, get the database CRUD out of your business objects and re-write it to use proper parameterized queries instead of string concatenation, replace all the "writelns" and "MessageBoxes" in your error handlers with a logging framework, refactor code that's trying to borrow methods from other classes, use locale-aware string formatting, stop guessing how big an array should be and use a dynamic collection, delete orphaned code 87 | 88 | Aim for these, in increasing order of importance 89 | 90 | 1. Code that does the same thing, but is shorter or more efficient 91 | 2. Code that does the same thing, but uses an appropriate "wheel" built-into the platform instead of reinventing its own 92 | 3. Code that does the same thing, but is easier to modify for similar needs 93 | 4. Code that does the same thing, but is easier to read and understand 94 | 5. Code that doesn't exist 95 | 96 | Hit #5 and you can call yourself a Zen Apprentice. Do it for a decade until you do it instinctively and you can call yourself a Zen Master 97 | 98 | 99 | ### 4\. Fascinated by the incomprehensible 100 | 101 | I am only just beginning to understand what a Fourier Transform does, but I've been studying them because I have the _damn persistent feeling_ that I could be using them somehow. I don't know what I would use them for yet, but maybe I will someday. What I do know is that _what I don't know will cost me in useless labor_. 102 | 103 | #### Symptoms 104 | 105 | 1. Visits [Lambda The Ultimate](http://lambda-the-ultimate.org/) on a regular basis 106 | 2. Knows what ATP synthase is. Has extracted DNA from a banana in their kitchen 107 | 3. Owns a book with a [dragon on the cover](http://www.amazon.com/gp/product/0321486811/ref=as_li_ss_tl?ie=UTF8&tag=synesmedia-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0321486811), especially if they don't write compilers 108 | 4. Giggles when someone says the phrase "This is recorded on sticky-tape and rust" 109 | 5. Shoves through a crowd at a party to get near someone who just used the word "Bayesian" 110 | 6. Buys drinks for people who work in other industries and seem willing to talk shop when drunk 111 | 7. Has a habit of boring people to tears explaining something tangentially related to the news, such as the cockpit layout of the Airbus 330 112 | 8. Has foreign-language versions of popular songs on their iPod 113 | 9. Envies but doesn't resent people with degrees in something they don't know 114 | 115 | #### How to acquire this trait 116 | 117 | This tends to start in childhood but can be cultivated in adulthood if you can commit to exploring your horizons. Friends are a major gateway: seek social occasions where you'll bump into people you don't know under circumstances where they'll be unhurried and at ease. This may involve alcohol. Don't try to impress them, don't compete with them, but display your ignorance willingly to see if they lean forward to correct and enlighten you. Then shut your fool trap and _listen_ 118 | 119 | When you hear or read something you don't recognize then Google it or hit Wikipedia. For a programmer an equally superior resource is [Ward Cunningham's Wiki](http://c2.com/cgi/wiki), which deserves weeks of your life 120 | 121 | Computer programming has annexed all of the sciences and the feedback loop is so wide it stuns gods. From biology we took Genetic Algorithms. From climatology we took chaos theory. Biologists now use our work to fold proteins. Climatologists now use our simulations to predict armageddon. Everything informs us, and we inform everything. Either probe the unfathomable or retire on a "blub" programmer's salary. 122 | 123 | ### 5\. Compelled to teach 124 | 125 | I once knew someone who thought it was good advice to "never teach everything you know" because they once lost a job after bringing a co-worker up to speed with all their skills. I stared at them with genuine incomprehension. A good manager would never get rid of someone who's not only capable of all their tasks but also demonstrates ability to train new workers. It would be like shooting the goose that lays golden eggs. If you get fired, it's probably for some other reason 126 | 127 | #### Symptoms 128 | 129 | 1. Blogs about their work 130 | 2. Has an active Wikipedia account 131 | 3. Unhesitant to pick up a marker and approach a whiteboard 132 | 4. Commits changes to the repository that consist only of comments 133 | 5. Lets new hires borrow books that cost them $100 to buy 134 | 6. Pauses "[The Andromeda Strain](http://en.wikipedia.org/wiki/The_Andromeda_Strain_(film))" at the part about the sliver of paper getting between the bell and the ringer and grins like a madman 135 | 136 | #### How to acquire this trait 137 | 138 | I can only do this when I'm inspired or "in the mood", and I think that this mood is a product of circumstance, one that's made up of confidence, space, opportunity and provocation. When you're in school your teacher has the space and opportunity already supplied for them and their confidence is hopefully given by their training, but the _inspiration_ is tricky; it's the difference between a good lesson that both the teacher and the student enjoys and a laborious exercise in rote memorization 139 | 140 | Novices in computer programming aren't usually novices in general, because they have lives and friends and family and hobbies and interests that have been going on for even longer. Maybe you do need to bore someone to tears by explaining something that's cool to you, even if it has nothing to do with programming. Maybe you have a younger sibling you can teach the guitar, or your favorite recipe, or how to balance on a pogo stick. Maybe you have a coworker who doesn't know how to ski. It doesn't matter the subject, just that you get a taste of what it's like to program someone else's brain in a positive way 141 | 142 | If you've never taught anything before you will discover, to an embarrassing degree, just how many times you can say "um" and "er" per minute, how badly you're prepared, and how easily you can forget that the student doesn't know details you haven't explained yet 143 | 144 | One of the tricks that worked for me was to volunteer for an opportunity to teach a complex subject (microbiology) to laymen. The first time I tried it I used a Post-It easel and a bunch of markers and tried to draw everything. I was all over the place. It was humiliating. But the audience, fortunately, was friendly 145 | 146 | The next year I tried again, but this time I had an iPad and used Keynote to put together a presentation, which was a lot of fun in itself, but this time the lesson went overwhelmingly more smoothly. I used lots of pictures, very little text, almost no bullet points, a handful of jokes, and just relied on my memory to talk about slides I had designed to provoke my memory more than illustrate anything to the audience 147 | 148 | The experience of doing an awful job the first time _informed_ my next attempt, and now that I've done it three or four more times I find I'm getting slightly better. Not only that, I now know ten times more about the subject because I studied like crazy to help temper my fear of being asked a difficult question. Teaching teaches the teacher 149 | 150 | ## Signs that you're a fantastic programmer 151 | 152 | I only wish I had these traits and I can only write about them because I've observed them in others. Every now and then I have a moment where I think I'm living one of these, but those moments are rare and cherished. They are also debilitating and brush up against the stereotypes of autistic savants, trading one kind of virtue for another: if you want greatness you have to be prepared to pay 153 | 154 | ### 1\. Incorruptible patience 155 | 156 | #### Symptoms 157 | 158 | 1. Fire alarms provoke annoyance more than panic 159 | 2. Cannot name any song that just played on the radio or through their headphones 160 | 3. Is oblivious to how many times their cubicle-mate has gone for coffee, the bathroom, or the hospital 161 | 4. Unbothered by office politics 162 | 5. Can predict a bug before the code is ever run 163 | 164 | #### How to acquire this trait 165 | 166 | Distractions are a product of imagination. The day I wrote this I found myself horribly distracted and annoyed by someone at my gym singing songs in French while I sat in the sauna. The singing moved around outside the sauna and pissed me off. I wished he'd stop because I couldn't concentrate. I pictured a man without concern of others, a douchebag, someone who'd wear a pink shirt and order people around. Then I came out of the sauna and saw it was an old man, chocolate in complexion and as threatening as a worn teddy bear with button eyes. He'd started singing La Vie en rose, which is a song I that I not only loved but that made me wonder, just then, if it was me who'd long since turned into an insufferable asshole 167 | 168 | I don't know how to shut out distractions, but if I had to try I'd guess it'd involve a little bit of deference and so much fascination that it directs your imagination instead of being dictated by it. When I want to be like this I want to take life without taking it personally 169 | 170 | ### 2\. A destructive pursuit of perfection 171 | 172 | The worst optimizations favor profit over beauty, and between the two it's beauty that lasts longer. Perfection isn't the same as obsession, but they're damn close 173 | 174 | #### Symptoms 175 | 176 | 1. Preference for dismissal over compromise 177 | 2. Contempt for delivery dates 178 | 3. Substantial refactoring on the eve of a deadline 179 | 4. Unwilling to accept bonuses, promotion, or stock options for expediency 180 | 5. Fondness for films directed by Stanley Kubrick 181 | 182 | #### How to acquire this trait 183 | 184 | As [Tyler Durden says](http://www.amazon.com/gp/product/B001992NUQ/ref=as_li_ss_tl?ie=UTF8&tag=synesmedia-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=B001992NUQ) you must _know_--not _fear_--_know_ that someday you will die. Your nice condo with Ikea furniture is a side effect, not a reward. If you are not a unique, beautiful snowflake then what you create _has to be_. 185 | 186 | It's also known as pride in one's work. Remember that emotional detachment from _code_ is a virtue, but this doesn't mean emotional detachment from your _work_ is, too. In fact, another way to become emotionally detached from _code_ is to put your interest into the _outcome_ instead. The outcome you should be thinking of is a lady who's going to get fired if she doesn't deliver the output of your program at 4:59pm sharp 187 | 188 | There's a legend about a marketing type who worked for Sam Walton at Wal-Mart and came up with a brilliant campaign to advertise a widget. Sam took a look at the proposal and said something to the effect of "this is great, now take the cost of the campaign and use it to lower the price of the widget instead." According to legend, the widget sold better and made more profit that way than if the campaign had been carried out. 189 | 190 | Let the spirit of the story roll around in your head for a while and think about how it'd map to what you do at work. You boss probably isn't like Sam Walton, but perhaps there's a little bit of Sam in you. Is it better to compromise the way others want, or to make the product just a little bit better 191 | 192 | This could be hazardous to your income, it's risky to your stock options, but when you do a job _right_, when you do things _properly_, when you complete a project the way it _ought to be_, then sometimes time absolves all indulgences. Sometimes the boss calls you back to the carpet to apologize to _you_ 193 | 194 | ### 3\. Encyclopedic grasp of the platform 195 | 196 | Most programmers realize the short lifespan of their tools and don't waste much of their lives memorizing what's doomed to be obsolete. But neither do most programmers appreciate how everything in this industry is a derivative of some earlier thing, sharing syntax and constraints that will live well past our own personal expiration dates. The best programmers have done what Oxford used to insist on: if you learn latin and mathematics then you can fuck all of that other modern nonsense, because you'll have the tools you need to understand _anything_ 197 | 198 | #### Symptoms 199 | 200 | 1. Can recite from memory all of the includables in the C Standard Library 201 | 2. Raises a knowing eyebrow when you mention the "500 mile email" 202 | 3. Has a copy of the OpenDoc Programmer's Guide gathering dust on their shelf 203 | 4. Can complete any sequence of dialogue from Lord of the Rings, Star Wars, Red Dwarf or Monty Python 204 | 5. Rapidly identifies a synchronization bug caused by TCP's Sliding Window algorithm 205 | 6. Recognizes a bug that's caused by a microcode error on the CPU you're testing on 206 | 7. Has a framed personal check for $2.56 from Donald Knuth 207 | 208 | #### How to acquire this trait 209 | 210 | Encyclopedic knowledge takes decades to acquire, but every Guru in the world got there by doing roughly the same three things each day 211 | 212 | 1. Struggling to solve problems they find to be difficult 213 | 2. Writing about how they solved difficult problems 214 | 3. Reflecting on how they solved difficult problems 215 | 216 | Once upon a time a novice programmer was stumped by a bug that he couldn't figure out. The crash report was full of strange numbers he didn't recognize, like -32,760\. Where the hell is that coming from? He hits Ctrl-F and searches all of his code files for "-32760" but it doesn't appear anywhere. Nothing makes any sense. A week goes past during which he goes back to his old college computer-science textbooks, the compiler's manual, everything, and on the last day his glazed eyes rest on a table of numbers. Through the fog of his tired mind he suddenly recognizes one of them: -32,76**8**. He thinks about how remarkable it is that it's so similar to his problem number, and then he notices that this table of numbers is showing the ranges for various integer types and how there can be _signed_ and _unsigned_ versions of both. When the light comes on it's blinding 217 | 218 | Thrilled with his belated insight he writes a blog post about it which disappears into the global ether unread by all but a handful of buddies. That night he lies awake, thinking about that bug and about integer types and the pros and cons of compiler-checked types and so-on 219 | 220 | Ten years later our friend is the lead programmer at the firm, and one day he glances over the shoulder of a junior programmer who's showing evident frustration. Tucked down in the stdout window is a bunch of debugging traces and the number -32,762\. The now-guru programmer taps the newbie on the shoulder and says "are you passing an _unsigned_ int16 to code that's expecting a _signed_ int16? 221 | 222 | If you're not encountering problems that are difficult for you to solve then you need a change of job or hobby or scenery or something. Look for opportunities to work with something new at your job or school, try hacking your Roomba, pick a bug in an open-source project that nobody has touched for months and fix it, try answering tumbleweed questions on [StackOverflow](http://stackoverflow.com/) that force you to look up something you didn't know 223 | 224 | If you could look inside the brain of a guru with a magic magnifying glass you might see clusters of neurons packed around the visual cortex that, like the infamous "[Grandmother cells](http://en.wikipedia.org/wiki/Grandmother_cell)", lie dormant for months but light up when something significant comes into view such as a power of two, or a suspiciously precise delay that points to a DNS timeout, or the signature of the [FDIV bug](http://en.wikipedia.org/wiki/Pentium_FDIV_bug). Those "grandmother cells" can only be made the hard way 225 | 226 | ### 4\. Thinks In Code 227 | 228 | #### Symptoms 229 | 230 | 1. In casual conversation their readiest metaphors come from programming constructs 231 | 2. Spends the majority of their time "goofing off", but commits more bug-free code each day than their colleagues 232 | 3. Glances over your shoulder and points at a bug in your code with their finger 233 | 4. Correctly diagnoses bugs over the phone while drunk or in bed 234 | 5. Comes up with their best code while taking a shower* 235 | 6. When confronted with an obstinate bug, their instinct is to get up and go for a walk 236 | 7. They suddenly pause and stare into space in the middle of a conversation, then abandon you to hurry back to their terminal with no explanation (AKA "A Columbo Moment" or "Gregory House behavior") 237 | 238 | #### How to acquire this trait 239 | 240 | Them darn kids and their cell phones, how does a 12 year-old teenage girl tap text messages on a numeric keypad so fast anyway? It can't be genetic, since all those damn brats can do it, no matter their gender or parentage. It can't be upbringing, cuz kids in every social class can do it. So you rule out this and that and what you're left with is an ancient truth: people think in the language they learned to speak. A teenager's thumbs already know where to go and they _think_ in texting. When writing, typos _feel wrong._ People who learn multiple spoken languages and use them regularly tend to think in multiple languages, too, after they've practiced for so long that they no longer have to do a translation in their heads first. Rather than read a phrase in Russian and translate it to English in their minds before understanding it, they just understand it in Russian. 241 | 242 | You cannot think "_Fire rearward missile_" and then translate it to Russian, you **must think in Russian** 243 | 244 | If you've heard about Sapir-Whorf or read _Nineteen Eighty Four_ and all that jazz then you might already appreciate the implications: words convey ideas, language _is_ thought. Whether that's a syntactic language or a visual or auditory language in your head doesn't matter, it's the way your brain deals with symbols and their rules for manipulation that matter 245 | 246 | The best book you can read (and perform the exercises of) to acquire this trait is [Structure and Interpretation of Computer Programs](http://www.amazon.com/gp/product/0070004846/ref=as_li_ss_tl?ie=UTF8&tag=synesmedia-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0070004846) by Abelson, Sussman and Sussman. It uses Scheme to present its lessons, and even if that's not the language you write programs in it's still one of the best languages to program your brain with. Remember: learn math and latin and you can understand _anything_ 247 | 248 | Whether you have this book or not the key is to practice with coding until you can read and reason with it like your native tongue. You can't acquire this trick in 30 days, it may be more like 30 months. You'll know if you've got it when you begin to _see in code_ as well. 249 | 250 | * - Shaddap Bob, and you too, Sat.