├── hypermedia-rpc.key ├── programming_is_terrible.key ├── programming_is_terrible.pdf ├── hypermedia-rpc.rst └── programming_is_terrible.rst /hypermedia-rpc.key: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tef/emfcamp2012/HEAD/hypermedia-rpc.key -------------------------------------------------------------------------------- /programming_is_terrible.key: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tef/emfcamp2012/HEAD/programming_is_terrible.key -------------------------------------------------------------------------------- /programming_is_terrible.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tef/emfcamp2012/HEAD/programming_is_terrible.pdf -------------------------------------------------------------------------------- /hypermedia-rpc.rst: -------------------------------------------------------------------------------- 1 | .. 2 | draft script for a talk at scottish ruby fringe 2012. 3 | 4 | running time 40 minutes 5 | 6 | a talk on web-like approach to rpc, for ruby and python. 7 | 8 | section titles won't be read out loud, may appear 9 | on slides. 10 | 11 | hypermedia-rpc, or how I learned to love rest 12 | ============================================= 13 | .. 14 | introduction: 15 | me, learning ruby 16 | day job, github, travis 17 | hyperglyph is ducked type web stuff 18 | worksforme 19 | 20 | SLIDE:: 21 | 22 | *blank -or- talk title* 23 | 24 | .. 25 | perhaps put a parody dr-strangelove title, 26 | perhaps section titles in same style. 27 | 28 | hi, my name is tef, and I am a python programmer. 29 | 30 | SLIDE:: 31 | 32 | @tef 33 | 34 | (don't tell anyone, but i've been learning ruby while no-one's been looking) 35 | 36 | python is my day job, and my day job involves getting computers to talk to 37 | each other. there are many names and styles for this, but mostly, they suck. 38 | 39 | I wrote some code to handle the suck for me, and I got paid for doing it too. 40 | 41 | *aside* 42 | Now when I say rpc, I really mean client-server. RPC is a pretty overloaded term, 43 | but I just mean 'I call a function over here, and get a function over there', 44 | at a very primitive level. 45 | 46 | hyperglyph isn't like other rpc systems - it doesn't require code 47 | generation, it's duck typed, and it plays nicely with http. 48 | it does this by being a lot more like a web site than a web service. 49 | 50 | 51 | 52 | 53 | Work also have me their blessing to release it, and I'm porting it to ruby for fun. 54 | 55 | It's called hyperglyph, and it is on github, and travis runs tests in both ruby and python. 56 | 57 | it's made my job suck less, and I hope it can make your job suck less too. 58 | 59 | SLIDE:: 60 | 61 | http://hyperglyph.net 62 | 63 | 64 | Before I begin my story, I'd like to take this moment to thank my employer 65 | for letting me release this code, as well as github and travis. You're all neat. 66 | 67 | -1min 68 | 69 | nobody knows the trouble i've seen 70 | ---------------------------------- 71 | 72 | .. 73 | day job is terrible 74 | rpc i.e client-server 75 | what life wihout rpc looks like 76 | duck typing is nice 77 | 78 | SLIDE:: 79 | 80 | *blank* 81 | 82 | 83 | My day job is writing distributed software, a bit like googlebot, to crawl the web. 84 | 85 | My day job involved writing remote procedure calls. 86 | 87 | My day job sucked. 88 | 89 | 90 | I had some worker processes across a machine pool, and some process to manage 91 | their tasks. I'm simplifying, but a worker needs to fetch a task, crawl it for links, 92 | report them and move onto the next task. 93 | 94 | If *I* was doing it without RPC, all on the same machine, it would probably look a lot like this: 95 | 96 | SLIDE:: 97 | 98 | def process(queue): 99 | task = queue.next_task() 100 | for url in crawl(task.url): 101 | task.found_link(url) 102 | task.complete() 103 | 104 | If *you* were doing it without RPC, it would probably look a lot more like this: 105 | 106 | SLIDE:: 107 | 108 | def process(queue) 109 | task = queue.next_task 110 | crawl(task.url).each do |url| 111 | task.found_link(url) 112 | end 113 | task.complete 114 | end 115 | 116 | *aside* 117 | The ruby examples are works in progress. I hope you'll prefer an idea 118 | of where I'm going in ruby, rather than where I've gone with python. 119 | 120 | As you can see we just call methods, get back objects and call more methods. 121 | It sounded nice in theory, but practice had a lot to answer for. 122 | 123 | SLIDE:: 124 | blank 125 | 126 | HTTP. It's good for you 127 | ----------------------- 128 | 129 | .. 130 | ugly truth of POST+json 131 | other choices and nightmares 132 | http, and http intermediataries 133 | still ugly 134 | 135 | What we started with with looked a lot more like this: 136 | +3min 137 | 138 | SLIDE:: 139 | 140 | def process 141 | 142 | task = http.POST(@next_task_url, {worker:@worker_id) 143 | 144 | crawl(task['url']).each do |url| 145 | http.POST(@found_link_url, {worker:@worker_id, id:task['id'], url:url}) 146 | 147 | end 148 | http.POST(@complete_url, {worker:@worker_id, id:task['id']}) 149 | end 150 | 151 | Lovely, isnt it? 152 | 153 | As you can see, we picked http, out of a number of other approaches, including zeromq, irc, 154 | tcp, soap, and amqp. 155 | 156 | Message queues are nice when things work, but we'd had some terrible 157 | experiences with failure handling. We didn't know zeromq, and we didn't 158 | use irc because we aren't writing a botnet. 159 | 160 | SOAP on the other hand, 161 | 162 | SOAP, well. 163 | 164 | I still get flashbacks. 165 | 166 | I still have nightmares. 167 | 168 | *aside* 169 | I recently heard of a SOAP api where the responses were json 170 | embedded in strings. 171 | 172 | THE S STANDS FOR SIMPLE 173 | 174 | This lead to the obvious conclusion, http. our one true love. 175 | 176 | +4min 177 | 178 | HTTP may not be the best protocol in the world, but it has 179 | things like load balancers, proxies, and caches. one of the 180 | defining features of HTTP is that it is a layered system - 181 | very few protocols offer such a plethora of tools, 182 | let alone enable them 183 | 184 | That said, the resulting code wasn't so elegant, but I didn't blame HTTP. 185 | 186 | 187 | the ugly duckling 188 | ----------------- 189 | 190 | .. 191 | the trouble with objects, urls, stubs 192 | moved on, revisited after growth of stubs 193 | pain of stubs, generating stubs is concrete 194 | had to be a better way that didn't suck: more http 195 | 196 | SLIDE:: 197 | *blank* 198 | 199 | This isn't a unique experience - most rpc libraries don't do anything to ease the 200 | pain. 201 | 202 | URL based apis often have a handful of endpoints with a number of special 203 | ways to construct arguments. Most HTTP based clients suffer from a series 204 | of fixed url templates. 205 | 206 | Other libraries let you avoid the mechanics of making a remote call, and give you an 207 | object. However you only get one and it has all the methods on it. Yay objects! 208 | 209 | Some will offer generate your stubs for you. If you're lucky 210 | the stubs will be for the right version of the server. If you're *really* lucky 211 | they might even compile. 212 | 213 | No matter how I looked at it, writing client code involved hardcoding 214 | assumptions about how requests were made. Ugly, but necessary. 215 | 216 | I moved on. 217 | 218 | I had more important things to work on, I had to grow the product. 219 | Unfortunately as the product grew, so did the network code. 220 | 221 | And then the stubs came. Thousands of them. Dirty dirty stubs. 222 | 223 | SLIDE:: 224 | 225 | class Queue 226 | attr_accessor :worker_id 227 | 228 | def next_task 229 | task_id = http.POST(...) 230 | return Task(@worker_id, task_id) 231 | end 232 | end 233 | 234 | class Task 235 | attr_accessor :worker_id, :task_id, :url 236 | def found_link(url) 237 | http.POST(...) 238 | end 239 | def complete 240 | http.POST(...) 241 | end 242 | ... 243 | end 244 | 245 | I'd just written code just like this server side, too. The stubs were getting everywhere. 246 | At least my client code now looked ok, but stubs came with their own problems. 247 | 248 | Adding new methods became copy and paste. The worst sort of code duplication. The nastiest 249 | issue was state between requests - if we needed a new parameter, we had to change almost every 250 | method. 251 | 252 | Code generation is less painful in C# or Java, but in more dynamic language, 253 | it adds a build step, can't infer types on its own, and rarely handles the dynamic 254 | nature of the code. 255 | 256 | SLIDE:: 257 | 258 | *blank* 259 | 260 | I was fed up. There had to be a better way. 261 | 262 | without duplicating code, by hand or by generation 263 | 264 | without having to make artisan requests for each method, wrangling state between them 265 | 266 | and without abandoning http, and all of its delights. 267 | 268 | 269 | Logically, If the answer is not less http, the answer must be more HTTP. 270 | 271 | 272 | And this is what hyperglyph does. MORE HTTP. 273 | 274 | 275 | HTTP: Still good for you 276 | ------------------------ 277 | 278 | .. 279 | h in http: web page not web service 280 | a sample session/mechanized 281 | client just like objects 282 | queue page, is rest in links, built from object 283 | urls are constructors 284 | similarly for task, & object 285 | hyperglyph is a serialization format 286 | rpc alike but hypertext - ducktyping, flexibility 287 | 288 | Knowing this, how do we embrace http? 289 | 290 | The clue is in the name. Hypertext transfer protocol. 291 | 292 | Hypertext. Links and Forms. Web pages. 293 | 294 | 295 | What if we were to build a *web page* rather than a *web service*? 296 | 297 | .. 298 | possible slide: woah insert. 299 | 300 | 301 | 302 | Perhaps something like this: 303 | 304 | - From the service root 305 | - Go to the queue page, click on next task. 306 | - on the new task, open the link in a new window 307 | - submit any links you find using this form 308 | - when you are finished, click 'complete' 309 | 310 | When we write this down in code, it looks pretty familiar. 311 | 312 | SLIDE:: 313 | 314 | server = Glyph.get('http://local:219') 315 | queue = server.Queue('worker-12') 316 | task = queue.next_task 317 | crawl(task.url).each do |url| 318 | task.found_link(url) 319 | end 320 | 321 | 322 | The initial get of the client fetches the root page. This root 323 | has one attribute, Queue, which is a form to find a queue. 324 | 325 | It submits the form, to get a queue page, and in turn 326 | submits another form to get the next task. 327 | 328 | Although to the client, it looks like objects and methods, it's still 329 | a web page and forms underneath. The client is actually screen scraping web pages. 330 | 331 | But a web page for robots, not humans. 332 | 333 | These pages look something like this: 334 | 335 | SLIDE:: 336 | 337 | Root at / 338 | 339 | , 341 | }> 342 | 343 | Queue at /Queue/?worker_id=bob 344 | 345 | , 347 | }> 348 | 349 | I won't get into the encoding now, but if you think of a json like format, 350 | but with links and forms, you're close. 351 | 352 | Unlike before where we had to pass parameters into each subsequent request, 353 | the url encapsulates the state of the client. 354 | 355 | 356 | We can generate the website from objects too - the Queue page 357 | can be built from an object, and the urls can be built too. 358 | 359 | SLIDE:: 360 | 361 | class Queue < Resource 362 | attr_accessor :worker_id 363 | 364 | def next_task 365 | task_id = db.next_task(....) 366 | return Task(@worker_id, task_id) 367 | end 368 | end 369 | 370 | When a client is returned a Queue, hyperglyph serializes the object 371 | into a resource, with forms mapping to the methods. 372 | 373 | The instance data is smuggled inside the query parameters. 374 | 375 | SLIDE:: 376 | 377 | Root at / 378 | 379 | , 381 | }> 382 | 383 | Queue at /Queue/?worker_id=bob 384 | 385 | , 387 | }> 388 | 389 | 390 | We can see the mapping in the url, from the root it links to a class Queue, 391 | and if we have a queue, we can see the next_task url points to a method, 392 | as well as a particular queue. 393 | 394 | When the client submits next_task, hyperglyph can construct a Queue object, 395 | with the instance arguments in the query parameters, call queue.next_task, 396 | and serialize the response. 397 | 398 | SLIDE:: 399 | 400 | class Queue < Resource 401 | attr_accessor :worker_id 402 | 403 | def next_task 404 | task_id = db.next_task(....) 405 | return Task(@worker_id, task_id) 406 | end 407 | end 408 | 409 | in this case, it returns a Task, which we turn into a webpage. The client 410 | expects three attributes, url, found_link and complete. 411 | 412 | 413 | SLIDE:: 414 | 415 | class Task 416 | attr_accessor :worker_id, :task_id, :url 417 | def found_link(url) 418 | db.found_link(...) 419 | end 420 | def complete 421 | db.complete(...) 422 | end 423 | ... 424 | end 425 | 426 | 427 | SLIDE:: 428 | 429 | Task: /Task/?worker_id=bob&id=uuid 430 | 431 | , 434 | 'complete': , 435 | }> 436 | 437 | This page is generated from the class, like before. The instance data, and sometimes method 438 | are embedded in the url, and used in form attributes. 439 | 440 | ~10m 441 | 442 | Glyph is really a serialization format and supporting library. It handles turning 443 | objects into web pages and back again. 444 | 445 | *aside* 446 | The serialization is an extension of bencoding from bittorrent. 447 | 448 | It's documented, ported to ruby, and it supports all of your favourites 449 | - booleans, fixnums, utf-8 strings, byte arrays, nil, floats, 450 | and iso date times. It also isn't endian specific. 451 | 452 | The serialization format is really all you need client side to start using a 453 | server - and it's start page. Hypertext is what makes hyperglyph different, 454 | and what allows it to map object and methods, dynamically. 455 | 456 | SLIDE:: 457 | 458 | blank 459 | 460 | despite all of the underlying hypertext, at the client 461 | it feels like rpc - you get objects and call methods. 462 | you get objects from the server without having to write them 463 | yourself first. 464 | 465 | on the server it feels like rpc too, but with more flexibility. 466 | the server can add new instance data - the urls change but the 467 | forms don't. the server can add new methods too without breaking 468 | old clients, and without requiring new stubs on the client. 469 | 470 | like with duck typing, the server can return a different 471 | object as long as it has the methods the client is expecting. 472 | 473 | the interface is what is important, the names of things, 474 | rather than the urls. the urls are opaque to the client, 475 | and the server is free to change them, or point to other things. 476 | 477 | hypertext gives freedom to the developer, to grow the api. 478 | 479 | hypermedia is duck typing for apis. 480 | are you on crack? 481 | ----------------- 482 | 483 | It's also cross platform. The same client code in python is 484 | 485 | SLIDE:: 486 | 487 | import hyperglyph 488 | 489 | server = hyperglyph.get('http://local:219') 490 | 491 | queue = server.Queue(worker_id='woz') 492 | task = queue.next_task() 493 | for url in crawl(task.url): 494 | task.found_link(url) 495 | task.complete() 496 | 497 | And if you wanted a server, in python is 498 | 499 | SLIDE:: 500 | 501 | r = hyperglyph.Router() 502 | @r.add() 503 | class Queue(hyperglyph.r): 504 | def __init__(self, worker_id) 505 | self.worker_id = worker_ id 506 | def next_task(self): 507 | return Task(db.next_task(self.worker_id), worker_id) 508 | 509 | @r.add() 510 | class Task(hyperglyph.r): 511 | def __init__(self, uuid, worker_id): 512 | self.uuid, self.worker_id = uuid, worker_id 513 | 514 | def found_link(self, url): 515 | db.found_link(uuid, worker_id, url) 516 | 517 | def complete(self): 518 | db.complete(uuid, worker_id) 519 | 520 | 521 | SLIDE:: 522 | 523 | blank 524 | 525 | hyperglyph: good for you? 526 | -------------------- 527 | 528 | .. 529 | hyperglyph loves http, made things suck less 530 | caching, sharding, embedding, extending 531 | fixed our apis, in internal use 532 | porting it to ruby, my first ruby proggram 533 | demonstrates utility of hypertext 534 | 535 | You don't have to love HTTP to use hyperglyph. Glyph loves HTTP so much 536 | it's going to handle turning your objects into pages and back again for you. 537 | 538 | hyperglyph made my life suck less. I could change the server as I pleased 539 | without breaking clients. Growing the software became a lot less painful. 540 | 541 | SLIDE:: 542 | 543 | caching 544 | sharding 545 | embedding 546 | extending 547 | 548 | we're at the stage where we're adding caching to our services. 549 | the client code won't change. the forms will, the urls might, but the 550 | client won't change. the library, like the browser will handle this for 551 | the client. 552 | 553 | .. 554 | todo: mention embedding/ inlining 555 | 556 | we're looking at sharding next. we'll add a few instance variables 557 | into the server classes, but the methods won't change. as before, 558 | the urls change, but the client code doesn't. 559 | 560 | but when I need to add new behaviour, I don't break the existing 561 | client code, and changing it isn't so hard either. 562 | 563 | once we'd embraced hyperglyph, our apis changed dramatically. much 564 | of the state wrangling disappeared. thinking about web pages 565 | changed how we thought about web services. 566 | 567 | Writing network code still sucks, but now it sucks a lot less. 568 | 569 | SLIDE:: 570 | 571 | blank 572 | 573 | The python code is relatively stable, but I've been changing things 574 | as I've been porting it to ruby. 575 | 576 | 577 | *aside* 578 | It's actually my first ruby program. I wrote it instead of writing 579 | this talk. If you think the talk is terrible, wait until you see my 580 | code. 581 | 582 | I'm also looking at porting it to javascript and erlang, so if this 583 | goes well I might volunteer for an erlang con next. 584 | 585 | I'm consoled by the fact that at least I learned something because 586 | of this talk. 587 | 588 | As much as i've love you to use it - it isn't finished yet and it is still 589 | maturing. If you're curious about what i've done I'd love help and guidance, 590 | especially when it comes to ruby idioms. 591 | 592 | Even if you don't use it, I hope i've shown that you can build loosely coupled 593 | apis over http, when you use what http was built for: hypertext. 594 | 595 | It's there in the name. 596 | 597 | 598 | yes, that's all very good but is it rest? 599 | ----------------------------------------- 600 | 601 | .. 602 | talked about http et all, but not rest 603 | rest is about the web not apis 604 | rpc vs rest is orthogonal - can be complementary 605 | we've been doing this for years with websites for humans 606 | is hyperglyph restful? 607 | 608 | 609 | .. 610 | other questions and answers - authentication, 611 | 612 | Before I end, I should really go back to the beginning. 613 | 614 | I've talked about HTTP, encodings, Hypertext, and URLS 615 | 616 | I've talked about proxies, caches, and load balancers. 617 | 618 | 619 | but I'm yet to mention REST 620 | 621 | 622 | REST isn't about APIS or RPC, it is about the web. 623 | REST is about how all those things fit together, work together. 624 | 625 | 626 | Many people argue that RPC and the web are orthogonal, but I hope 627 | with hyperglyph i've shown you that they are complementary. 628 | 629 | It shouldn't come as a surprise - we've been mapping code to websites 630 | for years, scraping them for years 631 | 632 | 633 | 634 | 635 | As for REST, well if rest is how the web works, and hyperglyph works like the web, Is hyperglyph RESTful? 636 | 637 | No. 638 | 639 | Hyperglyph is an encoding format and library. 640 | 641 | 642 | 643 | On its own, hyperglyph isn't a really a restful api. 644 | 645 | but you can build one *using* hyperglyph, 646 | 647 | and it will make your life suck less. 648 | 649 | thanks 650 | ------ 651 | 652 | SLIDE:: 653 | 654 | @mamund, @steveklabnik, @jon_moore 655 | 656 | I'd like to thank @jon_moore, @mamund, @steveklabnik for 657 | writing some very useful things about hypermedia, 658 | but they're not to blame for this monstrosity. 659 | 660 | SLIDE:: 661 | 662 | http://github.com/hyperglyph 663 | 664 | thank you for your time. 665 | 666 | -------------------------------------------------------------------------------- /programming_is_terrible.rst: -------------------------------------------------------------------------------- 1 | ============================================================== 2 | programming is terrible. lessons learned from a life wasted. 3 | ============================================================== 4 | :Author: tef 5 | 6 | .. 7 | outline 8 | 9 | introduction 10 | good and bad programmers 11 | how culture dictates code 12 | how we destroy learners 13 | learning to be good 14 | 15 | 16 | 17 | Introduction 18 | ============ 19 | 20 | SLIDE:: intro slide, name, talk title 21 | 22 | Hello. I am tef, and I am a bad programmer. 23 | 24 | I started programming because of a desire to take things apart and see how they worked. The more I learn about programming the more I am surprised *that* things work. 25 | 26 | I am not a very good programmer. I forget to write tests, my documentation is sparse, and i'm pretty apologetic at any code review. I also write bugs. Lots of bugs. I have pretty high standards in the naïve belief that it is possible to write software that sucks much, much less than what we put up with. 27 | 28 | 29 | SLIDE:: 30 | good and bad programmers 31 | how culture dictates projects 32 | indoctrinating vs learning 33 | being successful vs being good 34 | 35 | In as much as I'm here to complain, I am also offering survival tips and hope. 36 | 37 | I'm here to talk to you about mistakes I've made, mistakes other people make, and some of the mythology around programming. It's much more about the people than the code. I'll be taking questions at the end. 38 | 39 | A standard disclaimer applies. This is my opinion and doesn't represent my employer, or reality. 40 | 41 | SLIDE:: YMMV. HTH. HAND. 42 | 43 | Some people have found my opinions useful, but for me they normally get me into trouble. 44 | 45 | I'm also wrong a lot of the time. That didn't seem to be a roadblock for the majority of people who write about programming on the internet. 46 | 47 | Let's begin with people being wrong on the internet. It's a good place to start. 48 | 49 | 50 | Good and Bad Programmers 51 | ------------------------ 52 | 53 | .. 54 | me/not me, language choice, political choice, 10x myth, penis having 55 | make mistakes, optimism, pebkac 56 | 57 | SLIDE:: my code is better than your code (sing it) 58 | 59 | Many blogs claim to elcuidate a dichotomy of programmers - good and bad. Upon careful inspection, most of them turn out to actually dictate the following types: 60 | 61 | SLIDE:: 62 | 63 | A. Programmers who are like me. 64 | B. Programmers who are not like me. 65 | 66 | 67 | The assertion is that if you cargo cult their personality, you too can be a successful programmer. Sometimes it is more veiled 68 | 69 | SLIDE:: 70 | 71 | A. Programmers who use my favourite language 72 | B. Programmers who do not use my favourite language 73 | 74 | You may have heard this as 'the blub paradox'. 75 | 76 | ASIDE:: 77 | 78 | the programmer who wrote this, left such a mess of lisp code that the 79 | company he sold its product to, switched to perl. they couldn't find anyone 80 | that could maintain his code. 81 | 82 | yes. perl was more maintainable. 83 | 84 | (he also claimed that if planes worked like lisp, 2001-09-11 wouldn't happen) 85 | 86 | (I am not making this up, he really said this) 87 | 88 | 89 | Sometimes they just *imply* good and bad, without saying it outright. 90 | 91 | SLIDE:: 92 | 93 | A. Programmers who share my political beliefs 94 | B. Programmers who do not share my political beliefs 95 | 96 | This one was spectacularly notable. It mashed everything into a simplified left/right view of politics. 97 | And now programmers who don't understand programming are using political terms they don't understand 98 | too. 99 | 100 | Why do we do this? It’s easy and gets blog hits. Everyone loves a simple answer to a complex problem. Especially 101 | when the two choices are emotionally charged. 102 | 103 | SLIDE:: 104 | 105 | 10x 106 | 107 | Better still, when the good programmers have magical super powers. You'll hear terms 108 | like rockstar, ninja, founder, entrepreneur, all used in the same pre-pubecsent machoism that our industrying 109 | is drowning in. 110 | 111 | Unfortunately, it's total bollocks. The 'some programmers are crazily more productive than others' comes 112 | from a study, on batch processing vs interactive programming, in 1960. On twelve people. In a half hour session. 113 | 114 | We've been repeating this myth endlessly. it's destructive. it's either repeated by idiots who believe 115 | they have nothing to learn from others, or repeated by learners to explain why they shouldn't try to learn. 116 | 117 | *pause* 118 | 119 | Really, when managers talk about 'A programmers' they mean people who will work long hours for little to no pay. 120 | 121 | *pause* 122 | 123 | The worst sort of dichotomy is this 124 | 125 | SLIDE:: 126 | 127 | A. Programmers with a penis 128 | 129 | B. Programmers without a penis. 130 | 131 | If any of you think this. Kill yourself now. This is not a joke. Just fuck off and die. 132 | 133 | 134 | Studies (CITE) have shown, when you tell a group there is a genetic component to ability, the performance 135 | goes down. People use this as a reason to avoid trying to improve, on both sides. 136 | 137 | If you believe in this divide in any way, it is highly likely that you are not only a terrible programmer, you are a terrible 138 | person too. 139 | 140 | I think that once we move past the issues like race, gender, within programming we might have a chance 141 | of handling tabs vs spaces. 142 | 143 | *pause* 144 | 145 | so, are there two types of programmers? Probably not, but if I was to try, i'd say:: 146 | 147 | SLIDE:: 148 | 149 | A. Programmers who know they will make mistakes 150 | B. Programmers who think they will not make mistakes 151 | 152 | I'm a little of column A an d a little of column B. Sometimes I refuse to try, and sometimes I refuse to learn. 153 | 154 | Really, the biggest mistake I make in programming is optimism. 155 | 156 | SLIDE:: 'you would think that' 157 | 158 | Optimism is the classic mistake programmers make, but it is often necessary because the task is so daunting. 159 | 160 | I am yet to meet a programmer who didn't chronically underestimate the time it takes to work. 161 | 162 | 163 | Programmers like to complain, often starting “You would think that...” — Underlying this is the optimism that things can be better (Some like to think that they could have done better). Call me a cynic, but after years of fixing the bugs in software we are still no closer to fixing the behaviours in humans that propagate them. 164 | 165 | SLIDE:: PEBKAC 166 | 167 | The mistakes we make are in part due to the environments we work in. It is just as important to find out why the bug got written as well as how to fix it, if we are to have any hope of learning from our mistakes. 168 | 169 | Software defects aren't endemic, they're systematic. 170 | 171 | 172 | Culture Dictates Code 173 | ===================== 174 | 175 | 176 | SLIDE:: quote Melvin Conway: 177 | 178 | ...organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations 179 | 180 | Essentially the software reflects the social structures of the teams that built it. If you need service orientated architecture, your teams should be structured around providing services to other teams, as opposed to delivering software or code. 181 | 182 | This raises its head in other ways - ‘God’ objects are often caused by ‘God’ programmers. People on the team who hoard responsibility for parts of the code and amass them into a lump. Frequently other programmers make small offerings to the object, and ensure that their code worships at its feet. 183 | 184 | 185 | It's not just individuals, sometimes, we make mistakes as a group, too. 186 | 187 | 188 | SLIDE:: The Bike Shed 189 | 190 | At a design meeting for a nuclear power plant, more time will be spent discussing the colour of the bike shed, than the technical details of the plant. 191 | The bike shed example first appeared in ‘Parkinson's Law’, under the ‘Law of Triviality’: 192 | 193 | To be able to contribute to the techincal discussion, domain expertise is a requirement. To contribute to the bike shed, little or no expertise is required. No matter how well designed the bike shed, someone will always have a change in mind, and arguments will ensue. 194 | 195 | The bike shed example first appeared in ‘Parkinson's Law’, under the ‘Law of Triviality’: 196 | 197 | SLIDE:: “The time spent on any item of the agenda will be in inverse proportion to the sum involved.” 198 | 199 | People love to contribute and feel that they have taken part in a discussion. As the barrier to entry lowers, more and more strive to take part in the discussion. Bikeshedding is the process of arguing over trivia, and how informed discussion is drowned in opinion. When everyone can contribute, nothing gets decided. 200 | 201 | 202 | SLIDE: The classic ‘Group Project‘ 203 | 204 | You have a group of friends. You all want to do something *together*. Everyone pools their ideas and then we'll all work on it. 205 | 206 | Except if any of the ideas were motivating enough, someone would be working on them already. Effectively you're collating all the ideas that people think would be cool if someone else did it for them. 207 | 208 | With no real individual desire to work, the project flounders. Collaboration tends to happen when someone leads by example. 209 | 210 | Leading by example doesn't mean coming up with ideas. Ideas are cheap, plentiful and worthless. Ideas stand as a multiplier of work put in. Only with effort do ideas bring value. Even not so great ideas are successful with enough work. 211 | 212 | Sometimes, ideas is all the group has, and then you get... 213 | 214 | SLIDE: Goon Project 215 | 216 | Enthusiasm didn't get us to the moon, but we've got 18 logos and a wiki. A fatal group project popluated by idea guys and all discussions revolve around the colour of the bike shed. 217 | 218 | When a lot of people want to solve a problem and don't know how, much of the bad ideas above surface and not much else. The most common cause of this is video games. Everyone has played them and not very many people have written them. A lot of enthusiasm goes a long way. Mostly "What should we call it", and "I can make a better logo" 219 | 220 | 221 | SLIDE:: waterfall 222 | 223 | It isn't just the individuls, and structures within teams, the way in which we approach software developmet causes faults too. The Waterfall methodology was introduced as a strawman, and taken seriously and still used today. Mostly because it is easier to bill clients for than actually a good way to bill software. 224 | Project Management is often an attempt to control reality rather than observe it, and react to it. Milestones are handed down upon high with little room for error, because maybe we'll get it right *this time*. 225 | 226 | 227 | SLIDE:: Manhole cover 228 | 229 | It is also evident in how the company acquires workers. We struggle with finding good 230 | programmers Often, companies resort to brainteases. I am asked ' What should I do when I am confronted with a brainteaser question in an interview?' 231 | 232 | SLIDE:: CYA LATER SHITLORDS 233 | 234 | Leave 235 | 236 | ASIDE:: 237 | 238 | There are a couple of experiments that show the context and framing of a problem have a massive effect on how people try and solve it (Wasson Selection Task). Brainteasers are not very effective at determining your ability beyond brainteasers. 239 | 240 | Unless you’ve being hired as a quiz show host, brainteasers in an interview are mostly to make the candidate panic and see how willing you are to put up with bizzare or ludricrous requests. 241 | 242 | I’ve heard people justify them on this basis alone, because the job often involves bizzare or ludricrous requests from management, and they don’t like hearing “no”. 243 | 244 | People tell me it happens at Facebook, they read it in some tech journalism. The same articles were written about Google. Before then it was Microsoft. Brainteasers make for an easy filler article, and so it’s quite a popular urban myth. 245 | 246 | It is a very effective warning sign of a terrible job. 247 | 248 | 249 | SLIDE:: cultists/occultism ? 250 | 251 | Programming is not a science or an art, it’s rituals and cargo-culting at best. Our best practices amount to old wives tales from people who learned to program on punch cards, and we barely test our software, let alone our precious methodologies. At best, It's a craft. 252 | 253 | Programming by and large is learned from maintaing existing software — fixing, testing, and adapting it, not creating it. That it not to deny the value of experimental programming, the adage ‘Top Down the second time’ still rings true. Often a prototype is needed to explore the idea, and understand the consequences of it. It is from maintaining this protoype you learn new approaches. 254 | 255 | We don't just write bad code, we manage it badly and teach it badly. 256 | 257 | 258 | teaching 259 | -------- 260 | 261 | SLIDE:: teaching 262 | 263 | Two largest influences on how programming is taught today are: nostalgia, and the way in which the teacher learns best. It’s a cargo cult approximation to education - do what I do and you will learn what I did. 264 | 265 | 266 | Much of the discussion of education focuses heavily on “what students must know”, rather than more obviously “What do students want to learn, and how do they learn?”. A vital skill of the employed programmer is a willingness to learn on their own, and to explore. We need to encourage this from the outset, instead of dictating their course. 267 | 268 | That said, a little guidance and help goes a long way. 269 | 270 | This is more obvious in adult education - a teacher knows best attitude rarely earns you the respect of the pupils. I’ve learned much of what I know about programming by helping others gain an understanding. The teacher needs to cater to the pupils needs. 271 | 272 | For a start, I’d like to see more appreciation for learning styles - the notion that some people prefer exercises to books, and some prefer talking to pictures. Many believe that the way in which you learn is the best way for everyone to learn. Most teachers will only teach in the way that they prefer, rather than teaching in a way that helps the students. 273 | 274 | 275 | If someone asks you to teach them to program, ask them what they want to create, and then point them in the right direction. 276 | 277 | 278 | SLIDE:: 279 | learning through play 280 | 281 | I encourage people to find a sandbox to play in. Be it a 2d environment with a turtle drawing pictures, or a musical environent, somewhere you can add elements and program them, as well as experiment or change existing programs quickly. 282 | 283 | I try to focus on getting them to explain things to me and asking questions, rather than the drudgery of rote exercises. The computer should be a tool for learning and exploration, driven by the student. 284 | 285 | I must confess that I too am tainted with a nostalgia — one of my earliest experiences of programming was in logo and I had fun. 286 | 287 | 288 | SLIDE:: 289 | Seymour Papert and turtle 290 | 291 | Logo was built by Seymour Papert to create a sort of ‘math world’.His idea was to give people an environment in which to construct their own rules and problems, and try to solve them, rather than a predefined course or structure to work through. Turtle graphics are the canonical example of the ‘math world’. A 2d box to draw in and play. 292 | 293 | I’ve seen a similar idea espoused in math education. Currently it is treated as a death march through formulae to be inscribed into your brain, rather than actually trying to solve problems. Learning is more fun and rewarding when you get to be creative about how you go about it. 294 | 295 | The other influence for me beyond Papert is ‘view-source’. I learned well from copying others and changing things. Fill in the blank exercises are boring to me, as are stepping through a problem in tiny chunks. I enjoyed taking something and tweaking it and manipulaing it to change the behaviour. 296 | 297 | I learned a lot from reading other peoples code and changing it, more than I’ve learned from my own code. Learners need to be able to share and reuse examples easily. Programming is not just explaining things to the computer but working out how things work. 298 | 299 | 300 | SLIDE:: 301 | my first language 302 | 303 | I would start with a relatively useful language from the outset, and by that I mean something:: 304 | - that they can do something useful or fun within an afternoon. 305 | - their friends know and can help them with. 306 | - relatively easy to install and run. 307 | - that doesn’t require navigating an IDE. 308 | - that is general purpose. 309 | 310 | I would advocate any popular scripting language - Python, Ruby, JavaScript, Lua. 311 | 312 | Don’t worry about objects and classes too much. Worry about data structures and algorithms. Get simple functions working to make things happen. 313 | 314 | Learning a language should be a side effect of some larger and more interesting goal. People rarely learn languages for their own merits. 315 | 316 | 317 | SLIDE:: 318 | REAL MEN USE C 319 | 320 | 321 | C is a useful language. Many languages are implemented it it. Much of the libraries and operating system is implemented in it. Unless C is the only option for the project desired, I wouldn’t advocate it as a first language. 322 | 323 | I don’t advocate it because it is hard to do anything immediately useful with it, in a small amount of time. Advocates seem to argue that “C is character building”. Great job! Suffering is such a great learning experience! 324 | 325 | I would advocate *any* scripting language over C first. Even in the grizzly macho world of unix, people learn shell before they learn C. Using C effectively requires much more knowledge of the operating system. 326 | 327 | 328 | SLIDE:: 329 | Industry Standards. 330 | 331 | Using C# and Java are difficult for vastly different reasons to C. For each of those languages, a simpler scripting language is available on the runtime, with access to the same libraries. 332 | 333 | Understanding Object Orientation requires a good understanding of procedural programming first. Focus on the basics before moving on to developing classes and objects. 334 | 335 | Java, C# make better second languages. 336 | 337 | People approach learning with caution, and they generalise on the initial experience. Often they learn with a predisposition for giving up - looking for an excuse to move on to something else. You see this all the time on forums - “Hi I am unconfident about my approach and I don’t want to find out the hard way”. 338 | 339 | 340 | SLIDE:: 341 | 342 | MATHS EH EH EH 343 | 344 | Well, I’d say maths and programming are actually quite related, and the ignorance thereof is where we get things like floating point misconceptions. You need to understand as much mathematics as your program demands, which varies wildly. Not many programs have a high demand of math skills beyond counting. If you can use a spreadsheet, you probably know more than enough to start. 345 | 346 | Part of programming is mathematical, not to say that differential geometry is somehow going to be useful, but reasoning about your program requires the same discipline of thought found in mathematics. I’m not saying that programers need to be mathematicians, but /are/ mathematicians (a class of). proofs are programs, innit. 347 | 348 | Programming is ultimately an interdisciplinary set of skills: Programmers need to be able to write fluently, have critical reasoning skills, engineering dicipline as well as mathematical reasoning. Often overlooked is one of the most vital skills; Domain experience of the problem you are trying to solve. 349 | 350 | 351 | 352 | SLIDE:: 353 | How do I become a successful programmer? 354 | 355 | I'm not qualified to answer this question — I tend to burn out in jobs — but many other programmers I have met have managed to sustain employment and increase pay. I will share with your their winning strategies. 356 | 357 | Although you will be forced to document your software, don't be afraid to write ugly prose, and ensure you leave out failure cases, or data types or arguments. Hopefully you will always be too busy to document and test the code. You have important bugs to fix. 358 | 359 | Write lots of code. Lots of code. Autocomplete Helps. Use your own ad-hoc naming scheme. Write your own wrappers around standard library functions. Reinvent liberally. Learn to use the advanced features of your ide and language and use them everywhere. Don't be afraid to seperate everything out into modules that only make sense when combined. 360 | 361 | Fix problems by creating new ones. Ensure that if you close the bug for now, someone new will re-open it. You can create an equilibrium by constantly shifting the problem around. 362 | 363 | Ensure your tests only pass some of the time. Better if only on your machine with some elaborate setup. Become the central point of failure for the development — those who aren’t will be passed over or lose their job. 364 | 365 | Job security comes from constant creation of work only you can do. If you act like you are the only programmer and this is the only bug you have, you will go far and be rewarded for your solipsist heroics of sabotaging the product. 366 | 367 | 368 | SLIDE:: 369 | How about being a good one? 370 | 371 | Read code every day. Read other peoples code, in order to learn from someone elses mistakes. 372 | 373 | To start with a terrible metaphor - if you met a professional writer you would expect them to be well read — the few I have encountered have a intimidating collection of books. Before you are expected to write a novel, you should have read some novels. Same goes for code bases. 374 | 375 | Yet with programming, much of the education and resources goes into the practice of writing code for the first time, and little towards analysis, debugging or maintenance. 376 | 377 | Programmers often complain that ‘we have to estimate things we’ve never done before’, I cannot help what part of this is due to our institutionalised ignorance of other peoples code and projects. 378 | 379 | 380 | The only other advice I can relay is that you should write code as if it were mistaken, and you will have to change it, again and again. because you will. 381 | 382 | Fail fast and repeatedly. It is easier to get something right by getting wrong a couple of times. It is easier to get it wrong a couple of times if you don't write so much code from the outset. 383 | 384 | Try and think a little more about how the code will be called than how it works. It is far easier to change implementation over interface. 385 | 386 | Don’t be an artist. Don't labour over the ‘right’ way to do things, but don't paint yourself into a corner. Write code that is easy to replace, rather than extend. 387 | 388 | Bear in mind: It is OK to write ugly code. As long as the things using it don't have to write uglier code to use it. 389 | 390 | 391 | As you get further in programming, you will understand the biggest problems are social, not technical. 392 | 393 | 394 | SLIDE: 395 | A final warning 396 | 397 | the software industry is terrible, so is every other industry. retraining won’t help you escape people. 398 | 399 | Thank you. Any questions? 400 | 401 | 402 | --------------------------------------------------------------------------------