├── .gitignore ├── 5nines.slide ├── 5nines ├── config-changed ├── juju-status.txt ├── provisioner.go ├── provisioner_test.go ├── stopping1.go ├── stopping2.go ├── stopping3.go ├── stopping4.go ├── watcher.go └── watcher.txt ├── Makefile ├── absolute-unit.adoc ├── asgppp.jpg ├── clarity.adoc ├── cmp ├── go.mod ├── go.sum └── main.go ├── constant-time.adoc ├── container-camp-2018-interview.adoc ├── context-isnt-for-cancellation.adoc ├── demo.slide ├── demo └── demo.go ├── do-not-fear-first-class-functions ├── actor-ii.go ├── actor-iii.go ├── actor.go ├── calc.go ├── channels.go ├── maps.go ├── mutex.go └── options.go ├── gb.slide ├── gb ├── gb.svg ├── gbee.jpg ├── stars.png └── women-who-go.jpeg ├── gdg-melbourne.adoc ├── go-past-present-future.adoc ├── gof.jpg ├── going-without.adoc ├── golangsyd-october-2015.slide ├── golangsyd-october-2015 ├── cover.png ├── dotgo.png ├── golanguk.svg └── gothamgo.png ├── gopher-puzzlers.slide ├── gopher-puzzlers ├── 1e8ys6.jpg ├── addition-ii.go ├── addition.go ├── bonus.go ├── bonus2.go ├── boom.go ├── cap.go ├── cap2.go ├── compexity.go ├── complexity.go ├── copy.go ├── counting.go ├── declarations.go ├── duration.go ├── empty.go ├── empty2.go ├── falsey.go ├── floaty.go ├── genus.go ├── gofmt.go ├── gofmt2.go ├── gofmt3.go ├── gophercon2016-2.png ├── hampton.jpg ├── hmm-ii.go ├── hmm-iii.go ├── hmm-iv.go ├── hmm.go ├── init.go ├── init2.go ├── init3 ├── init3.go ├── init4.go ├── initb.go ├── iota.go ├── java-puzzlers.jpg ├── keys.go ├── keywords.go ├── keywords2.go ├── mainmain.go ├── maps.go ├── maps2.go ├── missing-panic.go ├── missing-panic2.go ├── missing-panic3.go ├── named-and-unnamed-ii.go ├── named-and-unnamed-iii.go ├── named-and-unnamed-iv.go ├── named-and-unnamed-v.go ├── named-and-unnamed.go ├── neither.go ├── oddity.go ├── one-liner-ii.go ├── one-liner.go ├── one-two-three-ii.go ├── one-two-three.go ├── party.png ├── pointer.go ├── power.go ├── rune.go ├── shift.go ├── size-of-things-ii.go ├── size-of-things-iii.go ├── size-of-things.go ├── sizeclass-ii.go ├── sizeclass.go ├── sizeclassb.go ├── sliced.go ├── slices.go ├── snip.go ├── snip2.go ├── snowman-or-poop.go ├── snowman.go ├── space-packing-ii.go ├── space-packing-iii.go ├── space-packing-iv.go ├── space-packing.go ├── terminator.go ├── terminator2.go ├── tick.go ├── tick2.go ├── tokens-ii.go ├── tokens.go ├── touche.go ├── truthy.go ├── twohundred.go ├── which-is-faster-ii.go ├── which-is-faster.go └── zero.go ├── gophercon-sg-2023.asciidoc ├── gos-hidden-pragmas-examples ├── linkname │ ├── linkname.go │ └── linkname.s ├── noescape │ └── noescape.go ├── norace.go ├── notinheap.go ├── nowritebarrier.go └── redzone.go ├── hde.slide ├── introduction-to-go.slide ├── introduction-to-go ├── concurrency1.go ├── concurrency2.go ├── examples.go ├── go1-vs-tip.png ├── hello.go ├── hellohttp.go ├── histo.go ├── histo0.go ├── histop.go ├── idents.go ├── parallelogram.go ├── pike-2009.png ├── point.go ├── politics.go ├── precision.go ├── recv.go ├── select.go ├── send-recv.go ├── server.go ├── sort.go ├── stringer.go ├── walk.go └── weekday.go ├── percentv ├── go.mod └── main.go ├── performance-without-the-event-loop.slide ├── performance-without-the-event-loop ├── 3357832896_896d98bbaf_z.jpg ├── 640px-Table_of_x86_Registers_svg.svg.png ├── CMOS_Inverter.svg ├── CPU.png ├── Ivy-Bridge_Die_Flat-HR.jpg ├── e450.jpg ├── echo.go ├── grep.go ├── guard-page.png ├── microservices.jpg ├── pager.jpg ├── process.png ├── read.go ├── stack-growth.png ├── sun-ultra-enterprise-450-400mhz-2gb-20-bay-workgroup-server-system-no-hdd-parts_131514071457.jpg ├── threads.png └── turingFull560.jpg ├── practical-go-qcon.adoc ├── practical-go.adoc ├── reproducible-builds-ii.slide ├── reproducible-builds-ii ├── gb.svg └── video.html ├── reproducible-builds.slide ├── reproducible-builds ├── gb.jpg ├── github.png └── goget.jpg ├── seven.slide ├── seven ├── alloc_objects.svg ├── block.svg ├── cpu-mysql-updated.svg ├── flamegraph1.png ├── flamegraph2.png ├── flamegraph3.png ├── good-news.jpg ├── inuse_objects.svg ├── net-http-pprof.go ├── perf.png ├── profile.svg ├── torch.svg └── wharrgarbl.jpg ├── simplicity.slide ├── simplicity ├── 10_ck_chef_hand_10.jpg ├── 16.gif ├── IMG_0095.png ├── IMG_0245.JPG ├── IMG_8124.jpg ├── amor.jpg ├── cover-big.jpg ├── error.png ├── four_string_braid.jpg ├── gofmt.png ├── growing.png ├── history.jpg ├── nasa-mainframe-980x663.jpg ├── pipe.png ├── supermighty.png └── thesis.png ├── split1 ├── go.mod ├── split.go └── split_test.go ├── split10 ├── go.mod ├── go.sum ├── split.go └── split_test.go ├── split11 ├── go.mod ├── go.sum ├── split.go └── split_test.go ├── split2 ├── go.mod ├── split.go └── split_test.go ├── split3 ├── go.mod ├── split.go └── split_test.go ├── split4 ├── go.mod ├── split.go └── split_test.go ├── split5 ├── go.mod ├── split.go └── split_test.go ├── split6 ├── go.mod ├── split.go └── split_test.go ├── split7 ├── go.mod ├── split.go └── split_test.go ├── split8 ├── go.mod ├── split.go └── split_test.go ├── split9 ├── go.mod ├── split.go └── split_test.go ├── testing-gophercon-china.adoc ├── what-have-we-learned-from-the-pdp11.adoc ├── whats-in-a-name.adoc ├── writing-high-performance-go.slide ├── writing-high-performance-go ├── Nehalem_Die_Shot_3.jpg ├── alloc_objects.svg ├── block.svg ├── concat │ └── concat_test.go ├── copy │ └── copy_test.go ├── cpu.svg ├── fib │ └── fib_test.go ├── go17burndown.png ├── grow.go ├── inuse_objects.svg ├── ioloop.go ├── latency.png ├── numbers.png ├── pool.go ├── popcnt │ ├── popcnt2_test.go │ └── popcnt_test.go ├── profile.svg ├── readwrite.go └── semaphore.go └── yow-2018.adoc /.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | -------------------------------------------------------------------------------- /5nines.slide: -------------------------------------------------------------------------------- 1 | Go and Juju at Canonical 2 | GoSF May Meetup 3 | 9 May 2013 4 | 5 | Dave Cheney 6 | Canonical 7 | dave@cheney.net 8 | http://dave.cheney.net/ 9 | @davecheney 10 | 11 | * Who is this guy ? 12 | 13 | - I work for Canonical developing Juju in Go 14 | - Co-organiser of the Sydney Go meetup group with @enneff 15 | - Go Contributor since Feb 2011 16 | - Go Committer since April 2012 17 | - ARM enthusiast 18 | 19 | * Overview 20 | 21 | This is a talk in two parts 22 | 23 | - What is Juju ? 24 | - What are some of the features of Go we use to make Juju reliable ? 25 | 26 | * What is Juju ? 27 | 28 | Before I can talk about how we use Go to build Juju, I need to spend a few minutes explaining Juju. 29 | 30 | * Not a sales presentation 31 | 32 | Come see me afterwards if you _do_ want a sales presentation. 33 | 34 | * Service Orchestration not host configuration 35 | 36 | Juju isn't like Chef or Puppet. 37 | 38 | * Not host configuration ? 39 | 40 | - Hosts are disposable 41 | - Deployments live from minutes to years. 42 | - Host setup is expected to be handled by the provider or baked into the boot image. 43 | - If you need a package to installed, Juju charms can handle that for you. 44 | - If you need a config file to be edited, Juju charms can handle that as well. 45 | 46 | * Services 47 | 48 | - Unlike Chef or Puppet, Juju treats the Service, not the Host, as the central data type. 49 | - A bunch of Services deployed together are called an Environment. 50 | - You describe your Services and how they interrelate and Juju does the rest. 51 | 52 | * Environments 53 | 54 | An Environment is a collection of Services running on an provider like 55 | 56 | - HP Cloud 57 | - EC2 58 | - MaaS 59 | - Openstack 60 | 61 | - more to come 62 | 63 | * Charms (1/2) 64 | 65 | - Services are an instance of a Charm. 66 | 67 | % juju deploy cs:precise/mysql mydatabase 68 | 69 | Output 70 | 71 | % juju status 72 | ... 73 | services: 74 | mydatabase: 75 | charm: cs:precise/mysql-19 76 | exposed: false 77 | units: 78 | mydatabase/0: 79 | agent-state: installed 80 | agent-version: 1.11.0.1 81 | machine: "1" 82 | public-address: ec2-54-241-77-25.us-west-1.compute.amazonaws.com 83 | 84 | * Charms (cont.) 85 | 86 | - just a zip file 87 | 88 | % file ~/.juju/cache/cs_3a_precise_2f_wordpress-11.charm 89 | /home/dfc/.juju/cache/cs_3a_precise_2f_wordpress-11.charm: Zip archive data 90 | 91 | - Hooks named for verbs; install, start, config-changed 92 | - Charms can get details of their configuration via shell commands 93 | 94 | .code 5nines/config-changed /START OMIT/,/END OMIT/ 95 | 96 | - Charms can be written in any language you want. Shell, Python, Ruby, PHP, ... 97 | 98 | * Relations 99 | 100 | - Services are related to one another. 101 | - Relations are a form of bidirectional dependency. 102 | - Relations allow Services to discover configuration. 103 | - Services can query details about other services once a relation has been established. 104 | 105 | * Units 106 | 107 | - A Unit is an instance of a Service. 108 | - Deployed into a machine provided for it in the Environment. 109 | - This usually means a new machine will booted up to handle the Unit. 110 | 111 | * Putting it all together 112 | 113 | .code 5nines/juju-status.txt 114 | 115 | * End of part 1 116 | 117 | So, that was the theory, now on to the implementation. 118 | 119 | * The implementation 120 | 121 | Juju has three parts 122 | 123 | - A database to store the environment (MongoDB) 124 | 125 | - Server side tools 126 | 127 | jujud agents, jujuc tools 128 | 129 | - Client side tools 130 | 131 | juju cli 132 | 133 | * The implementation (cont.) 134 | 135 | At last count, 85k lines of code (182k inc tests), including dependencies developed by Canonical. 136 | 137 | - labix.org/v2/mgo 138 | - launchpad.net/gnuflag 139 | - launchpad.net/goamz 140 | - launchpad.net/gocheck 141 | - launchpad.net/gomaasapi 142 | - launchpad.net/goose 143 | - launchpad.net/goyaml 144 | - launchpad.net/tomb 145 | 146 | * The implementation (cont.) 147 | 148 | - Juju cli alters the data representing the Environment stored in the Mongo database. 149 | - Agents running on various machines inside the Environment observe these changes and react by running a Charm hook. 150 | - Juju Agents are constructed from many goroutines (jobs) selected at runtime for a particular agent role. 151 | - Everything is asynchronous. 152 | 153 | * Two small things 154 | 155 | - Tombs 156 | - Watchers 157 | 158 | * Managing goroutines with Tombs 159 | 160 | For reliable operation we need to 161 | 162 | - Monitor the status of worker goroutines. 163 | - Be able to stop them when needed. 164 | - Know when they _have_ stopped. 165 | 166 | * A contrived example 167 | 168 | .play 5nines/stopping1.go /START OMIT/,/END OMIT/ 169 | 170 | * Improved example 171 | 172 | .play 5nines/stopping2.go /START OMIT/,/END OMIT/ 173 | 174 | * Rewritten using a Tomb 175 | 176 | .play 5nines/stopping3.go /START OMIT/,/END OMIT/ 177 | 178 | * Returning a value 179 | 180 | .play 5nines/stopping4.go /START OMIT/,/END OMIT/ 181 | 182 | * Watching, always watching 183 | 184 | .code 5nines/watcher.txt 185 | 186 | * Watchers, really just Tombs 187 | 188 | .code 5nines/watcher.go /STARTa OMIT/,/ENDa OMIT/ 189 | .code 5nines/watcher.go /STARTb OMIT/,/ENDb OMIT/ 190 | 191 | * Some real code 192 | 193 | .code 5nines/provisioner.go /START OMIT/,/END OMIT/ 194 | 195 | * Thank you 196 | 197 | - Special thanks to @rogpeppe who helped me a _lot_ with my presentation. 198 | 199 | - Big thanks to @enneff for the sweet present tool. 200 | 201 | - Huge thanks to Gary Burd for his talks.godoc.org subsite. This presentation is 202 | .link http://talks.godoc.org/github.com/davecheney/gosf/5nines.slide 203 | 204 | - Check out Juju, it's open source. 205 | .link https://launchpad.net/juju-core 206 | -------------------------------------------------------------------------------- /5nines/config-changed: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ue 4 | 5 | source inc/common 6 | 7 | ## 8 | # This is where things can get a bit hectic. So this long blog is just to 9 | # help preface what's going on here. When this hook fires it will always 10 | # try to re-configure the entire unit. It's like a GOD hook, triggering 11 | # or using data from every other hook with the exception of the install 12 | # and upgrade-charm hooks. 13 | # 14 | # First, get a bunch of configuration values. The idea being this is should 15 | # be the only place that config-get is run as it's "unreliable" anywhere else 16 | # Data should then either be passed to functions or written down - you know 17 | # "YOU SHOULD ALWAYS LEAVE A NOTE!" 18 | # 19 | # From here, the web engine will either be updated or left alone. We always 20 | # assume it's going to be nginx unless you specify apache. So if you put 21 | # "lighttpd" or "i love apache" you're going to get nginx, so don't do that. 22 | # Configuration files are re-written every time. This is to make sure that 23 | # any changes to long running units overtime are included. It may seem 24 | # expensive to keep re-writing the same files but in the end having all 25 | # the units in a sane and identical state is worth it. 26 | # 27 | # Next, we do some small file moving around. Just for debug stuff. 28 | # 29 | # After that tuning levels are decided and executed. This is a bit more 30 | # involved than just running a function, check inc/common for that craziness 31 | # 32 | # After that, it's time to get any user-defined content! do_vcs does that 33 | # and a little more 34 | # 35 | # Caching stuff, basically "DO WE HAVE MEMCACHED, GOOD LETS DO CONFIG" 36 | # 37 | # Then we stop and start everything again and wait for the next round. 38 | ## 39 | 40 | // START OMIT 41 | tuning_level=`config-get tuning` 42 | wp_content_repo=`config-get wp-content` 43 | expose_info=`config-get debug` 44 | engine=`config-get engine` 45 | unit_address=`unit-get private-address` 46 | // END OMIT 47 | 48 | # Make it lower case 49 | tuning_level=${tuning_level,,} 50 | expose_info=${expose_info,,} 51 | engine=${engine,,} 52 | 53 | 54 | if [ "$engine" == "apache" ] || [ "$engine" == "apache2" ]; then 55 | if [ -f .web-engine ]; then 56 | web_engine=`cat .web-engine` 57 | service $web_engine stop 58 | fi 59 | sed -i -e "s/# deb \(.*\) multiverse/deb \1 multiverse/g" /etc/apt/sources.list #for libapache2-mod-fastcgi 60 | apt-get update 61 | apt-get -y purge nginx 62 | apt-get install -y apache2-mpm-worker libapache2-mod-fastcgi 63 | service apache2 stop 64 | 65 | rm -f /var/www/index.html 66 | 67 | rm -f /etc/apache2/sites-enabled/* 68 | a2enmod actions fastcgi alias proxy_balancer proxy_http headers 69 | 70 | install -o root -g root -m 0644 files/charm/apache/etc_apache2_conf-d_php5-fpm.conf /etc/apache2/conf.d/php5-fpm.conf 71 | 72 | juju-log "Installing Apache loadbal config..." 73 | install -o root -g root -m 0644 files/charm/apache/etc_apache2_sites-enabled_loadbalancer /etc/apache2/sites-available/loadbalancer 74 | sed -i -e "s/^ ServerName .*/ ServerName ${unit_address}/" /etc/apache2/sites-available/loadbalancer 75 | a2ensite loadbalancer 76 | 77 | juju-log "Installing Apache wordpress config..." 78 | install -o root -g root -m 0644 files/charm/apache/etc_apache2_sites-enabled_wordpress /etc/apache2/sites-available/wordpress 79 | a2ensite wordpress 80 | 81 | echo "apache2" > .web-engine 82 | else 83 | if [ -f .web-engine ]; then 84 | web_engine=`cat .web-engine` 85 | service $web_engine stop 86 | fi 87 | apt-get -y purge apache2* libapache2* 88 | apt-get install -y nginx 89 | service nginx stop 90 | 91 | juju-log "Cleaning any old or default nginx site configs ..." 92 | rm -f /etc/nginx/sites-enabled/* 93 | rm -f /etc/nginx/conf.d/* 94 | 95 | juju-log "Installing nginx common config ..." 96 | rm -f /etc/nginx/nginx.conf 97 | install -o root -g root -m 0644 files/charm/nginx/etc_nginx_nginx.conf /etc/nginx/nginx.conf 98 | 99 | juju-log "Installing nginx actual site config ..." 100 | #rm -f /etc/nginx/sites-available/ 101 | install -o root -g root -m 0644 files/charm/nginx/etc_nginx_sites-enabled_wordpress /etc/nginx/sites-available/wordpress 102 | ln -sf ../sites-available/wordpress /etc/nginx/sites-enabled/wordpress 103 | 104 | juju-log "Installing nginx loadbal config ..." 105 | rm -f /etc/nginx/sites-available/loadbalancer 106 | install -o root -g root -m 0644 files/charm/nginx/etc_nginx_sites-enabled_loadbalancer /etc/nginx/sites-available/loadbalancer 107 | ln -sf ../sites-available/loadbalancer /etc/nginx/sites-enabled/loadbalancer 108 | 109 | juju-log "Moving nginx var dirs to /mnt storage ..." 110 | rsync -az /var/lib/nginx /mnt/ && rm -rf /var/lib/nginx && ln -s /mnt/nginx /var/lib/ 111 | 112 | echo "nginx" > .web-engine 113 | fi 114 | 115 | # http://i.imgur.com/TUF91.gif 116 | hooks/loadbalancer-rebuild 117 | 118 | juju-log "Restarting Services ..." 119 | source hooks/restart 120 | 121 | if [ ! -f $config_file_path ]; then 122 | juju-log "Nothing to configure, since nothing is installed" 123 | exit 0 124 | fi 125 | 126 | juju-log "Show details? $expose_info" 127 | 128 | if [ "$expose_info" == "yes" ]; then 129 | rsync -az files/_debug $wp_install_path/ 130 | else 131 | rm -rf $wp_install_path/_debug 132 | fi 133 | 134 | juju-log "I will be using this tuning level: $tuning_level" 135 | 136 | if [ "$tuning_level" == "optimized" ]; then 137 | # First and foremost, we need to disable the ability to edit 138 | # themes and upload/update plugins. This breaks a scale-out 139 | # environment. It's sad but true. If you want to update a plugin 140 | # install a theme, etc; take a look at the README. 141 | make_optimized 142 | elif [ "$tuning_level" == "single" ]; then 143 | # We need to prepare an NFS mount, because someone is probably 144 | # going to try to scale out. We also need to vamp up caching. 145 | make_single 146 | elif [ "$tuning_level" == "bare" ]; then 147 | # Okay, you know what you're doing. You're probably going to 148 | # use Gluster to stream-line your files, so you don't need to 149 | # disable anything. We trust you to do what you need to. 150 | make_bare 151 | else 152 | juju-log "Not sure about that tuning level." 153 | exit 1 154 | fi 155 | 156 | do_vcs $wp_content_repo 157 | 158 | if [ -z "$wp_content_repo" ]; then 159 | wp plugin update --path=$wp_install_path --all 160 | fi 161 | 162 | do_cache 163 | 164 | chown -R www-data.www-data $wp_install_path 165 | 166 | . hooks/restart 167 | -------------------------------------------------------------------------------- /5nines/juju-status.txt: -------------------------------------------------------------------------------- 1 | % juju deploy cs:precise/mysql mydatabase 2 | % juju deploy wordpress myblog 3 | % juju add-relation mydatabase myblog 4 | % juju expose myblog 5 | % juju status 6 | machines: 7 | "0": 8 | agent-state: started 9 | agent-version: 1.11.0.1 10 | dns-name: ec2-184-72-15-27.us-west-1.compute.amazonaws.com 11 | instance-id: i-08703b50 12 | series: precise 13 | "1": 14 | agent-state: started 15 | agent-version: 1.11.0.1 16 | dns-name: ec2-54-241-77-25.us-west-1.compute.amazonaws.com 17 | instance-id: i-fe773ca6 18 | series: precise 19 | "2": 20 | agent-state: started 21 | agent-version: 1.11.0.1 22 | dns-name: ec2-50-18-43-73.us-west-1.compute.amazonaws.com 23 | instance-id: i-ba8bc0e2 24 | series: precise 25 | services: 26 | myblog: 27 | charm: cs:precise/wordpress-15 28 | exposed: true 29 | relations: 30 | db: 31 | - mydatabase 32 | loadbalancer: 33 | - myblog 34 | units: 35 | myblog/0: 36 | agent-state: started 37 | agent-version: 1.11.0.1 38 | machine: "2" 39 | public-address: ec2-50-18-43-73.us-west-1.compute.amazonaws.com 40 | mydatabase: 41 | charm: cs:precise/mysql-19 42 | exposed: false 43 | relations: 44 | db: 45 | - myblog 46 | units: 47 | mydatabase/0: 48 | agent-state: started 49 | agent-version: 1.11.0.1 50 | machine: "1" 51 | public-address: ec2-54-241-77-25.us-west-1.compute.amazonaws.com 52 | -------------------------------------------------------------------------------- /5nines/provisioner.go: -------------------------------------------------------------------------------- 1 | package provisioner 2 | 3 | import ( 4 | "errors" 5 | "fmt" 6 | "launchpad.net/juju-core/environs" 7 | "launchpad.net/juju-core/environs/config" 8 | "launchpad.net/juju-core/log" 9 | "launchpad.net/juju-core/state" 10 | "launchpad.net/juju-core/state/api" 11 | "launchpad.net/juju-core/state/api/params" 12 | "launchpad.net/juju-core/state/watcher" 13 | "launchpad.net/juju-core/utils" 14 | "launchpad.net/juju-core/worker" 15 | "launchpad.net/tomb" 16 | "sync" 17 | ) 18 | 19 | // Provisioner represents a running provisioning worker. 20 | type Provisioner struct { 21 | st *state.State 22 | machineId string // Which machine runs the provisioner. 23 | stateInfo *state.Info 24 | apiInfo *api.Info 25 | environ environs.Environ 26 | tomb tomb.Tomb 27 | 28 | // machine.Id => environs.Instance 29 | instances map[string]environs.Instance 30 | // instance.Id => machine id 31 | machines map[state.InstanceId]string 32 | 33 | configObserver 34 | } 35 | 36 | type configObserver struct { 37 | sync.Mutex 38 | observer chan<- *config.Config 39 | } 40 | 41 | // nofity notifies the observer of a configuration change. 42 | func (o *configObserver) notify(cfg *config.Config) { 43 | o.Lock() 44 | if o.observer != nil { 45 | o.observer <- cfg 46 | } 47 | o.Unlock() 48 | } 49 | 50 | // NewProvisioner returns a new Provisioner. When new machines 51 | // are added to the state, it allocates instances from the environment 52 | // and allocates them to the new machines. 53 | func NewProvisioner(st *state.State, machineId string) *Provisioner { 54 | p := &Provisioner{ 55 | st: st, 56 | machineId: machineId, 57 | instances: make(map[string]environs.Instance), 58 | machines: make(map[state.InstanceId]string), 59 | } 60 | go func() { 61 | defer p.tomb.Done() 62 | p.tomb.Kill(p.loop()) 63 | }() 64 | return p 65 | } 66 | 67 | func (p *Provisioner) loop() error { 68 | environWatcher := p.st.WatchEnvironConfig() 69 | defer watcher.Stop(environWatcher, &p.tomb) 70 | 71 | var err error 72 | p.environ, err = worker.WaitForEnviron(environWatcher, p.tomb.Dying()) 73 | if err != nil { 74 | return err 75 | } 76 | 77 | // Get a new StateInfo from the environment: the one used to 78 | // launch the agent may refer to localhost, which will be 79 | // unhelpful when attempting to run an agent on a new machine. 80 | if p.stateInfo, p.apiInfo, err = p.environ.StateInfo(); err != nil { 81 | return err 82 | } 83 | 84 | // Call processMachines to stop any unknown instances before watching machines. 85 | if err := p.processMachines(nil); err != nil { 86 | return err 87 | } 88 | 89 | // Start responding to changes in machines, and to any further updates 90 | // to the environment config. 91 | machinesWatcher := p.st.WatchMachines() 92 | defer watcher.Stop(machinesWatcher, &p.tomb) 93 | // START OMIT 94 | // launchpad.net/juju-core/worker/provisioner/provisioner.go 95 | for { 96 | select { 97 | case <-p.tomb.Dying(): 98 | return tomb.ErrDying 99 | case cfg, ok := <-environWatcher.Changes(): 100 | if !ok { 101 | return watcher.MustErr(environWatcher) 102 | } 103 | if err := p.setConfig(cfg); err != nil { 104 | log.Errorf("worker/provisioner: loaded invalid environment configuration: %v", err) 105 | } 106 | case ids, ok := <-machinesWatcher.Changes(): 107 | if !ok { 108 | return watcher.MustErr(machinesWatcher) 109 | } 110 | if err := p.processMachines(ids); err != nil { 111 | return err 112 | } 113 | } 114 | } 115 | // END OMIT 116 | panic("not reached") 117 | } 118 | 119 | // setConfig updates the environment configuration and notifies 120 | // the config observer. 121 | func (p *Provisioner) setConfig(config *config.Config) error { 122 | if err := p.environ.SetConfig(config); err != nil { 123 | return err 124 | } 125 | p.configObserver.notify(config) 126 | return nil 127 | } 128 | 129 | // Err returns the reason why the Provisioner has stopped or tomb.ErrStillAlive 130 | // when it is still alive. 131 | func (p *Provisioner) Err() (reason error) { 132 | return p.tomb.Err() 133 | } 134 | 135 | // Wait waits for the Provisioner to exit. 136 | func (p *Provisioner) Wait() error { 137 | return p.tomb.Wait() 138 | } 139 | 140 | func (p *Provisioner) String() string { 141 | return "provisioning worker" 142 | } 143 | 144 | // Stop stops the Provisioner and returns any error encountered while 145 | // provisioning. 146 | func (p *Provisioner) Stop() error { 147 | p.tomb.Kill(nil) 148 | return p.tomb.Wait() 149 | } 150 | 151 | func (p *Provisioner) processMachines(ids []string) error { 152 | // Find machines without an instance id or that are dead 153 | pending, dead, err := p.pendingOrDead(ids) 154 | if err != nil { 155 | return err 156 | } 157 | 158 | // Find running instances that have no machines associated 159 | unknown, err := p.findUnknownInstances() 160 | if err != nil { 161 | return err 162 | } 163 | 164 | // Stop all machines that are dead 165 | stopping, err := p.instancesForMachines(dead) 166 | if err != nil { 167 | return err 168 | } 169 | 170 | // It's important that we stop unknown instances before starting 171 | // pending ones, because if we start an instance and then fail to 172 | // set its InstanceId on the machine we don't want to start a new 173 | // instance for the same machine ID. 174 | if err := p.stopInstances(append(stopping, unknown...)); err != nil { 175 | return err 176 | } 177 | 178 | // Start an instance for the pending ones 179 | return p.startMachines(pending) 180 | } 181 | 182 | // findUnknownInstances finds instances which are not associated with a machine. 183 | func (p *Provisioner) findUnknownInstances() ([]environs.Instance, error) { 184 | all, err := p.environ.AllInstances() 185 | if err != nil { 186 | return nil, err 187 | } 188 | instances := make(map[state.InstanceId]environs.Instance) 189 | for _, i := range all { 190 | instances[i.Id()] = i 191 | } 192 | // TODO(dfc) this is very inefficient. 193 | machines, err := p.st.AllMachines() 194 | if err != nil { 195 | return nil, err 196 | } 197 | for _, m := range machines { 198 | if instId, ok := m.InstanceId(); ok { 199 | delete(instances, instId) 200 | } 201 | } 202 | var unknown []environs.Instance 203 | for _, i := range instances { 204 | unknown = append(unknown, i) 205 | } 206 | return unknown, nil 207 | } 208 | 209 | // pendingOrDead looks up machines with ids and retuns those that do not 210 | // have an instance id assigned yet, and also those that are dead. 211 | func (p *Provisioner) pendingOrDead(ids []string) (pending, dead []*state.Machine, err error) { 212 | // TODO(niemeyer): ms, err := st.Machines(alive) 213 | for _, id := range ids { 214 | m, err := p.st.Machine(id) 215 | if state.IsNotFound(err) { 216 | log.Infof("worker/provisioner: machine %q not found in state", m) 217 | continue 218 | } 219 | if err != nil { 220 | return nil, nil, err 221 | } 222 | switch m.Life() { 223 | case state.Dying: 224 | if _, ok := m.InstanceId(); ok { 225 | continue 226 | } 227 | log.Infof("worker/provisioner: killing dying, unprovisioned machine %q", m) 228 | if err := m.EnsureDead(); err != nil { 229 | return nil, nil, err 230 | } 231 | fallthrough 232 | case state.Dead: 233 | dead = append(dead, m) 234 | log.Infof("worker/provisioner: removing dead machine %q", m) 235 | if err := m.Remove(); err != nil { 236 | return nil, nil, err 237 | } 238 | continue 239 | } 240 | if instId, hasInstId := m.InstanceId(); !hasInstId { 241 | status, _, err := m.Status() 242 | if err != nil { 243 | log.Infof("worker/provisioner: cannot get machine %q status: %v", m, err) 244 | continue 245 | } 246 | if status == params.StatusPending { 247 | pending = append(pending, m) 248 | log.Infof("worker/provisioner: found machine %q pending provisioning", m) 249 | continue 250 | } 251 | } else { 252 | log.Infof("worker/provisioner: machine %v already started as instance %q", m, instId) 253 | } 254 | } 255 | return 256 | } 257 | 258 | func (p *Provisioner) startMachines(machines []*state.Machine) error { 259 | for _, m := range machines { 260 | if err := p.startMachine(m); err != nil { 261 | return fmt.Errorf("cannot start machine %v: %v", m, err) 262 | } 263 | } 264 | return nil 265 | } 266 | 267 | func (p *Provisioner) startMachine(m *state.Machine) error { 268 | // TODO(dfc) the state.Info passed to environ.StartInstance remains contentious 269 | // however as the PA only knows one state.Info, and that info is used by MAs and 270 | // UAs to locate the state for this environment, it is logical to use the same 271 | // state.Info as the PA. 272 | stateInfo, apiInfo, err := p.setupAuthentication(m) 273 | if err != nil { 274 | return err 275 | } 276 | cons, err := m.Constraints() 277 | if err != nil { 278 | return err 279 | } 280 | // Generate a unique nonce for the new instance. 281 | uuid, err := utils.NewUUID() 282 | if err != nil { 283 | return err 284 | } 285 | // Generated nonce has the format: "machine-#:UUID". The first 286 | // part is a badge, specifying the tag of the machine the provisioner 287 | // is running on, while the second part is a random UUID. 288 | nonce := fmt.Sprintf("%s:%s", state.MachineTag(p.machineId), uuid.String()) 289 | inst, err := p.environ.StartInstance(m.Id(), nonce, m.Series(), cons, stateInfo, apiInfo) 290 | if err != nil { 291 | // Set the state to error, so the machine will be skipped next 292 | // time until the error is resolved, but don't return an 293 | // error; just keep going with the other machines. 294 | log.Errorf("worker/provisioner: cannot start instance for machine %q: %v", m, err) 295 | if err1 := m.SetStatus(params.StatusError, err.Error()); err1 != nil { 296 | // Something is wrong with this machine, better report it back. 297 | log.Errorf("worker/provisioner: cannot set error status for machine %q: %v", m, err1) 298 | return err1 299 | } 300 | return nil 301 | } 302 | if err := m.SetProvisioned(inst.Id(), nonce); err != nil { 303 | // The machine is started, but we can't record the mapping in 304 | // state. It'll keep running while we fail out and restart, 305 | // but will then be detected by findUnknownInstances and 306 | // killed again. 307 | // 308 | // TODO(dimitern) Stop the instance right away here. 309 | // 310 | // Multiple instantiations of a given machine (with the same 311 | // machine ID) cannot coexist, because findUnknownInstances is 312 | // called before startMachines. However, if the first machine 313 | // had started to do work before being replaced, we may 314 | // encounter surprising problems. 315 | return err 316 | } 317 | // populate the local cache 318 | p.instances[m.Id()] = inst 319 | p.machines[inst.Id()] = m.Id() 320 | log.Noticef("worker/provisioner: started machine %s as instance %s", m, inst.Id()) 321 | return nil 322 | } 323 | 324 | func (p *Provisioner) setupAuthentication(m *state.Machine) (*state.Info, *api.Info, error) { 325 | password, err := utils.RandomPassword() 326 | if err != nil { 327 | return nil, nil, fmt.Errorf("cannot make password for machine %v: %v", m, err) 328 | } 329 | if err := m.SetMongoPassword(password); err != nil { 330 | return nil, nil, fmt.Errorf("cannot set password for machine %v: %v", m, err) 331 | } 332 | stateInfo := *p.stateInfo 333 | stateInfo.Tag = m.Tag() 334 | stateInfo.Password = password 335 | apiInfo := *p.apiInfo 336 | apiInfo.Tag = m.Tag() 337 | apiInfo.Password = password 338 | return &stateInfo, &apiInfo, nil 339 | } 340 | 341 | func (p *Provisioner) stopInstances(instances []environs.Instance) error { 342 | // Although calling StopInstance with an empty slice should produce no change in the 343 | // provider, environs like dummy do not consider this a noop. 344 | if len(instances) == 0 { 345 | return nil 346 | } 347 | if err := p.environ.StopInstances(instances); err != nil { 348 | return err 349 | } 350 | 351 | // cleanup cache 352 | for _, i := range instances { 353 | if id, ok := p.machines[i.Id()]; ok { 354 | delete(p.machines, i.Id()) 355 | delete(p.instances, id) 356 | } 357 | } 358 | return nil 359 | } 360 | 361 | var errNotProvisioned = errors.New("machine has no instance id set") 362 | 363 | // instanceForMachine returns the environs.Instance that represents this machine's instance. 364 | func (p *Provisioner) instanceForMachine(m *state.Machine) (environs.Instance, error) { 365 | inst, ok := p.instances[m.Id()] 366 | if ok { 367 | return inst, nil 368 | } 369 | instId, ok := m.InstanceId() 370 | if !ok { 371 | return nil, errNotProvisioned 372 | } 373 | // TODO(dfc): Ask for all instances at once. 374 | insts, err := p.environ.Instances([]state.InstanceId{instId}) 375 | if err != nil { 376 | return nil, err 377 | } 378 | inst = insts[0] 379 | return inst, nil 380 | } 381 | 382 | // instancesForMachines returns a list of environs.Instance that represent 383 | // the list of machines running in the provider. Missing machines are 384 | // omitted from the list. 385 | func (p *Provisioner) instancesForMachines(ms []*state.Machine) ([]environs.Instance, error) { 386 | var insts []environs.Instance 387 | for _, m := range ms { 388 | switch inst, err := p.instanceForMachine(m); err { 389 | case nil: 390 | insts = append(insts, inst) 391 | case errNotProvisioned, environs.ErrNoInstances: 392 | default: 393 | return nil, err 394 | } 395 | } 396 | return insts, nil 397 | } 398 | -------------------------------------------------------------------------------- /5nines/stopping1.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | import "time" 5 | 6 | // START OMIT 7 | type Worker struct { 8 | stop chan struct{} 9 | } 10 | 11 | func (w *Worker) run() { 12 | defer fmt.Println("All done") 13 | for { 14 | select { 15 | case <-w.stop: 16 | return 17 | case <-time.After(100 * time.Millisecond): 18 | fmt.Println("Waited 100ms") 19 | } 20 | } 21 | } 22 | 23 | func main() { 24 | w := &Worker{stop: make(chan struct{})} 25 | go w.run() 26 | time.Sleep(300 * time.Millisecond) 27 | close(w.stop) 28 | } 29 | 30 | // END OMIT 31 | -------------------------------------------------------------------------------- /5nines/stopping2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | import "time" 5 | import "sync" 6 | 7 | // START OMIT 8 | type Worker struct { 9 | stop chan struct{} 10 | wg sync.WaitGroup 11 | } 12 | 13 | func (w *Worker) run() { 14 | defer w.wg.Done() 15 | defer fmt.Println("All done") 16 | for { 17 | select { 18 | case <-w.stop: 19 | return 20 | case <-time.After(100 * time.Millisecond): 21 | fmt.Println("Waited 100ms") 22 | } 23 | } 24 | } 25 | 26 | func main() { 27 | w := &Worker{stop: make(chan struct{})} 28 | w.wg.Add(1) 29 | go w.run() 30 | time.Sleep(300 * time.Millisecond) 31 | close(w.stop) 32 | w.wg.Wait() 33 | } 34 | 35 | // END OMIT 36 | -------------------------------------------------------------------------------- /5nines/stopping3.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | import "time" 5 | import "launchpad.net/tomb" 6 | 7 | // START OMIT 8 | type Worker struct { 9 | tomb.Tomb 10 | } 11 | 12 | func (w *Worker) run() { 13 | defer w.Tomb.Done() 14 | defer fmt.Println("All done") 15 | for { 16 | select { 17 | case <-w.Tomb.Dying(): 18 | return 19 | case <-time.After(100 * time.Millisecond): 20 | fmt.Println("Waited 100ms") 21 | } 22 | } 23 | } 24 | 25 | func main() { 26 | w := &Worker{} 27 | go w.run() 28 | <-time.After(300 * time.Millisecond) 29 | w.Tomb.Kill(nil) // normal exit 30 | w.Tomb.Wait() 31 | } 32 | 33 | // END OMIT 34 | -------------------------------------------------------------------------------- /5nines/stopping4.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "launchpad.net/tomb" 6 | "math/rand" 7 | ) 8 | 9 | var conn = make(chan int) 10 | 11 | func init() { 12 | go func() { 13 | for { 14 | conn <- rand.Intn(10) 15 | } 16 | }() 17 | } 18 | 19 | // START OMIT 20 | type Worker struct { 21 | tomb.Tomb 22 | } 23 | 24 | func (w *Worker) run() { 25 | defer w.Tomb.Done() 26 | for { 27 | switch v := <-conn; v { 28 | case 0: 29 | w.Tomb.Killf("error! got a zero") 30 | return 31 | case 1: 32 | return 33 | default: 34 | // continue 35 | } 36 | } 37 | } 38 | 39 | func main() { 40 | for i := 0; i < 10; i++ { 41 | var w Worker 42 | go w.run() 43 | fmt.Printf("Worker exited: %v\n", w.Wait()) 44 | } 45 | } 46 | 47 | // END OMIT 48 | -------------------------------------------------------------------------------- /5nines/watcher.txt: -------------------------------------------------------------------------------- 1 | % godoc launchpad.net/juju-core/state EnvironConfigWatcher 2 | PACKAGE DOCUMENTATION 3 | 4 | package state 5 | import "launchpad.net/juju-core/state" 6 | 7 | TYPES 8 | 9 | type EnvironConfigWatcher struct { 10 | // contains filtered or unexported fields 11 | } 12 | EnvironConfigWatcher observes changes to the environment configuration. 13 | 14 | func (w *EnvironConfigWatcher) Changes() <-chan *config.Config 15 | Changes returns a channel that will receive the new environment 16 | configuration when a change is detected. Note that multiple changes may 17 | be observed as a single event in the channel. 18 | 19 | func (w *EnvironConfigWatcher) Err() error 20 | Err returns any error encountered while running or shutting down, or 21 | tomb.ErrStillAlive if the watcher is still running. 22 | 23 | func (w *EnvironConfigWatcher) Stop() error 24 | Stop stops the watcher, and returns any error encountered while running 25 | or shutting down. 26 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: $(patsubst %.adoc,%.html,$(wildcard *.adoc)) 2 | 3 | %.pdf: %.xml 4 | pandoc -f docbook -S $? --latex-engine=xelatex -o $@ 5 | 6 | %.xml: %.adoc 7 | asciidoc -b docbook -d article -o $@ $? 8 | 9 | %.html: %.adoc 10 | asciidoctor -b html5 \ 11 | --failure-level=WARN \ 12 | -d article \ 13 | -o $@ \ 14 | $< 15 | -------------------------------------------------------------------------------- /absolute-unit.adoc: -------------------------------------------------------------------------------- 1 | = Absolute Unit (test) 2 | Dave Cheney 3 | GopherCon Singapore, November 2018 4 | 5 | = Conclusion 6 | 7 | This is more than a talk about testing 8 | -------------------------------------------------------------------------------- /asgppp.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/asgppp.jpg -------------------------------------------------------------------------------- /cmp/go.mod: -------------------------------------------------------------------------------- 1 | module percentv 2 | 3 | go 1.13 4 | 5 | require github.com/google/go-cmp v0.2.0 6 | -------------------------------------------------------------------------------- /cmp/go.sum: -------------------------------------------------------------------------------- 1 | github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ= 2 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 3 | -------------------------------------------------------------------------------- /cmp/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/google/go-cmp/cmp" 7 | ) 8 | 9 | // tag::main[] 10 | func main() { 11 | type T struct { 12 | I int 13 | } 14 | x := []*T{{1}, {2}, {3}} 15 | y := []*T{{1}, {2}, {4}} 16 | 17 | diff := cmp.Diff(x, y) 18 | fmt.Printf(diff) 19 | } 20 | 21 | // end::main[] 22 | -------------------------------------------------------------------------------- /container-camp-2018-interview.adoc: -------------------------------------------------------------------------------- 1 | = Container Camp 2018 Interview 2 | 3 | == Tell us a bit about yourself and your background, how did you get involved in containers and join Heptio? 4 | 5 | My story starts back in the 1990's as a university drop out who ironically got his first break as a lab assistant at the University of Melbourne. 6 | Back in the day each faculty ran their own IT group who looked after their staff and students, and over the next few years I worked my way up to network administrator. 7 | After y2k I decided it was time to move on and worked for Ericsson for a few years, then startups such as Aconex, MailGuard and RedBubble. 8 | In 2008 I accepted a job at Atlassian and moved up to Sydney to work on their nacient SaaS offering. 9 | 10 | I've been involved in orchestration systems. 11 | Starting in 2010 I designed a cost reduced platform for Atlassian's On Demand suite of products using OpenVZ. 12 | In 2012 I joined Canonical and spent several years working on Juju. 13 | 14 | 15 | == What projects most excite you in the container ecosystem at the moment? 16 | 17 | == Is there someone in the container community who you admire for their work? 18 | 19 | As an aside I 20 | 21 | == Without giving too much away, what three things will people learn from your Container Camp AU talk? 22 | 23 | My presention is intended to be a practical talk. 24 | The goal is show the audience 25 | And in doing so, hopefully 26 | -------------------------------------------------------------------------------- /context-isnt-for-cancellation.adoc: -------------------------------------------------------------------------------- 1 | = Context isn't for cancellation 2 | 3 | This is an experience report about the use of, and difficulties with, the context.Context facility. 4 | 5 | Many authors have written about the use, misuse, and how they would change context in a future iteration of Go. 6 | While opinions differs on many subjects, one thing is clear--there is widespead agreement that the values facility on the context tye is orthogonal to its use as a mechanism to control goroutine lifetime. 7 | 8 | Many proposals have emerged that attempt to address this overloading of context with a copy on write bag of values, most aproximate thread local storage and are unlikely to be accepted on ideological grounds. 9 | 10 | == Context is request scoped 11 | 12 | The documentation for context indicaites strongly that context is only for request scoped values--context should not be applied to long lived processes. 13 | [quote, godoc context] 14 | ____ 15 | Do not store Contexts inside a struct type; instead, pass a Context explicitly to each function that needs it. 16 | ____ 17 | Specifically context should only live in function arguments, never stored in a field or global. 18 | This makes context applicable only to the lifetime of resources in a request scope. 19 | While this is a strong use case for context, given Go's lineage on the server, there are other use cases for cancellation where the lifetime of goroutine lives beyond a single request. 20 | 21 | == Freeing resources is not the same as cancellation 22 | 23 | Even removing 24 | 25 | context use overloads 26 | 27 | == context as a hook for cancellation 28 | 29 | The stated goal of the context package is to 30 | [quote, godoc context] 31 | ____ 32 | Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes. 33 | ____ 34 | 35 | Even this description is confused and lacking. 36 | Deadlines are request scoped variables, yet they are treated separately giving context three different responsibilities. 37 | 38 | Yet Contexts most important facility, broadcasting a cancellation signal, is incomplete as there is no way for the canceller to _wait_ for the signal to be acknowledged. 39 | 40 | == Looking to the past 41 | 42 | As part of this experience report, it's germane to highlight some, well, actual experience. 43 | In 2012 Gustavo Niemeier wrote a packaged called tomb for lifecycle management, which was used by Juju for the management of the worker processes (goroutines) within the various agents in the Juju system. 44 | 45 | Tombs are concerned only with lifecycle management. 46 | Importantly this is a generic notion of a lifecycle, not tied exclusively to a request, or a goroutine. 47 | The scope of the resource's lifetime is defined simply by holding a reference to the tomb variable. 48 | 49 | Tombs provided several operations, all without 50 | - the ability to signal the 51 | - the ability to wait on the completion of a goroutine 52 | 53 | Combined with 54 | 55 | == 56 | 57 | == Context should become, well, just context 58 | 59 | The purpose of context is in it's name, context ; add defintion providing supporting information to make decisions. 60 | I propose that context becomes just that; an association list of copy on write values. 61 | 62 | In decoupling the lifetime management nature of context from its apparent use case; storing scoped information, hopefully that will highlight the orthogonal requirement that Go programmers have -- goroutine lifecycle management. 63 | 64 | - goroutine lifecycle management simpler, by 65 | 66 | And best of all, we don't need to wait for Go 2.0 to implement these ideas. 67 | - the the 68 | -------------------------------------------------------------------------------- /demo.slide: -------------------------------------------------------------------------------- 1 | DEMO 2 | 15 Jul 2015 3 | 4 | Dave Cheney 5 | dave@cheney.net 6 | http://dave.cheney.net/ 7 | @davecheney 8 | 9 | * Introduction 10 | 11 | This slide deck introduces you to Go 12 | 13 | * Hello, world 14 | 15 | It all starts with `println`. 16 | 17 | .play -edit demo/demo.go /START4 OMIT/,/END4 OMIT/ 18 | 19 | This is a runnable program, press the *run* button in the bottom corner. 20 | 21 | * Hello, world 22 | 23 | Let's introduce a little more Go 24 | 25 | .play -edit demo/demo.go /START3 OMIT/,/END3 OMIT/ 26 | 27 | This is a runnable program, press the *run* button in the bottom corner. 28 | 29 | * Hello, world 30 | 31 | Let's introduce functions, this is the `main` function. 32 | 33 | .play -edit demo/demo.go /START2 OMIT/,/END2 OMIT/ 34 | 35 | This is a runnable program, press the *run* button in the bottom corner. 36 | 37 | * Hello, world 38 | 39 | The `fmt` package is imported by our program. 40 | 41 | .play -edit demo/demo.go /START1 OMIT/,/END1 OMIT/ 42 | 43 | This is a runnable program, press the *run* button in the bottom corner. 44 | 45 | * Hello, world 46 | 47 | Just like the `fmt` package, our program lives in a package called `main`. 48 | 49 | .play -edit demo/demo.go 50 | 51 | This is a runnable program, press the *run* button in the bottom corner. 52 | 53 | 54 | -------------------------------------------------------------------------------- /demo/demo.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START1 OMIT 4 | import ( 5 | "fmt" 6 | ) 7 | 8 | // START2 OMIT 9 | func main() { 10 | // START3 OMIT 11 | fmt.Println("Hello, world") 12 | // END3 OMIT 13 | 14 | // START4 OMIT 15 | println("hello world") 16 | // END4 OMIT 17 | } 18 | 19 | // END2 OMIT 20 | // END1 OMIT 21 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/actor-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type Mux struct { 4 | ops chan func(*Mux) 5 | } 6 | 7 | func (m *Mux) SendMsg(msg string) { 8 | result := make(chan error, 1) 9 | m.ops <- func(m map[net.Addr]net.Conn) { 10 | for _, conn := range m.conns { 11 | if err := io.WriteString(conn, msg); err != nil { 12 | result <- err 13 | return 14 | } 15 | } 16 | result <- nil 17 | } 18 | return <-result 19 | } 20 | 21 | func (m *Mux) loop() { 22 | conns := make(map[net.Addr]net.Conn) 23 | for _, op := range m.ops { 24 | op(conns) 25 | } 26 | } 27 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/actor-iii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type Mux struct { 4 | ops chan func(*Mux) 5 | } 6 | 7 | func (m *Mux) PrivateMsg(addr net.Addr, msg string) error { 8 | result := make(chan net.Conn, 1) 9 | m.ops <- func(m map[net.Addr]net.Conn) { 10 | result <- m[addr] 11 | } 12 | 13 | conn := <-result 14 | if conn == nil { 15 | return errors.Errorf("client %v not registered", addr) 16 | } 17 | return io.WriteString(conn, msg) 18 | } 19 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/actor.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "io" 5 | "net" 6 | ) 7 | 8 | type Mux struct { 9 | ops chan func(*Mux) 10 | } 11 | 12 | func (m *Mux) Add(conn net.Conn) { 13 | m.ops <- func(m map[net.Addr]net.Conn) { 14 | m[conn.RemoteAddr()] = conn 15 | } 16 | } 17 | 18 | func (m *Mux) Remove(addr net.Addr) { 19 | m.ops <- func(m map[net.Addr]net.Conn) { 20 | delete(m, addr) 21 | } 22 | } 23 | 24 | func (m *Mux) SendMsg(msg string) error { 25 | m.ops <- func(m map[net.Addr]net.Conn) { 26 | for _, conn := range m { 27 | io.WriteString(conn, msg) 28 | } 29 | } 30 | return nil 31 | } 32 | 33 | func (m *Mux) loop() { 34 | conns := make(map[net.Addr]net.Conn) 35 | for _, op := range m.ops { 36 | op(conns) 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/calc.go: -------------------------------------------------------------------------------- 1 | package calc 2 | 3 | type Calculator struct { 4 | acc float64 5 | } 6 | 7 | const ( 8 | OP_ADD = 1 << iota 9 | OP_SUB 10 | OP_MUL 11 | OP_DIV 12 | ) 13 | 14 | func (c *Calculator) Do(op int, v float64) float64 { 15 | switch op { 16 | case OP_ADD: 17 | c.acc += v 18 | case OP_SUB: 19 | c.acc -= v 20 | case OP_MUL: 21 | c.acc *= v 22 | default: 23 | panic("unhandled operation") 24 | } 25 | return v 26 | } 27 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/channels.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "io" 5 | "net" 6 | ) 7 | 8 | type Mux struct { 9 | add chan net.Conn 10 | remove chan net.Addr 11 | sendMsg chan string 12 | } 13 | 14 | func (m *Mux) Add(conn net.Conn) { 15 | m.add <- conn 16 | } 17 | 18 | func (m *Mux) Remove(addr net.Addr) { 19 | m.remove <- add 20 | } 21 | func (m *Mux) SendMsg(msg string) error { 22 | m.sendMsg <- msg 23 | return nil 24 | } 25 | 26 | func (m *Mux) loop() { 27 | conns := make(map[net.Addr]net.Conn) 28 | for { 29 | select { 30 | case conn := <-m.add: 31 | m.conns[conn.RemoteAddr()] = conn 32 | case addr := <-m.remove: 33 | delete(m.conns, addr) 34 | case msg := <-m.sendMsg: 35 | for _, conn := range m.conns { 36 | io.WriteString(conn, msg) 37 | } 38 | } 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/maps.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | type Map func(key string) string 6 | 7 | func (m Map) Add(k, v string) Map { 8 | return func(key string) string { 9 | if k == key { 10 | return v 11 | } 12 | return m(key) 13 | } 14 | } 15 | 16 | func main() { 17 | m := Map(func(string) string { return "" }) 18 | fmt.Println(m("george")) // "" 19 | 20 | m = m.Add("george", "lion") 21 | fmt.Println(m("george")) // "lion" 22 | } 23 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/mutex.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "io" 5 | "net" 6 | "sync" 7 | ) 8 | 9 | type Mux struct { 10 | mu sync.Mutex 11 | conns map[net.Addr]net.Conn 12 | } 13 | 14 | func (m *Mux) Add(conn net.Conn) { 15 | m.mu.Lock() 16 | defer m.mu.Unlock() 17 | m.conns[conn.RemoteAddr()] = conn 18 | } 19 | 20 | func (m *Mux) Remove(addr net.Addr) { 21 | m.mu.Lock() 22 | defer m.mu.Unlock() 23 | delete(m.conns, addr) 24 | } 25 | 26 | func (m *Mux) SendMsg(msg string) error { 27 | m.mu.Lock() 28 | defer m.mu.Unlock() 29 | for _, conn := range m.conns { 30 | if err := io.WriteString(conn, msg); err != nil { 31 | return err 32 | } 33 | } 34 | return nil 35 | } 36 | -------------------------------------------------------------------------------- /do-not-fear-first-class-functions/options.go: -------------------------------------------------------------------------------- 1 | package world 2 | 3 | func WithReticulatedSplines(c *Config) 4 | 5 | type Config struct{} 6 | 7 | type Terrain struct { 8 | config Config 9 | } 10 | 11 | func NewTerrain(options ...func(*Config)) *Terrain { 12 | var t Terrain 13 | for _, option := range options { 14 | option(&t.config) 15 | } 16 | return &t 17 | } 18 | 19 | func main() { 20 | t := NewTerrain(WithReticulatedSplines) 21 | // [ simulation intensifies ] 22 | } 23 | -------------------------------------------------------------------------------- /gb/gb.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 6 | 7 | 11 | 12 | 17 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /gb/gbee.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gb/gbee.jpg -------------------------------------------------------------------------------- /gb/stars.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gb/stars.png -------------------------------------------------------------------------------- /gb/women-who-go.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gb/women-who-go.jpeg -------------------------------------------------------------------------------- /gdg-melbourne.adoc: -------------------------------------------------------------------------------- 1 | = Building a better Ingress for Kubernetes 2 | GDG Devfest Melbourne, 2018 3 | 4 | == Introduction 5 | 6 | Hello, 7 | 8 | My name is David, for the last year, and perhaps the next week, I work for Heptio. 9 | We're a plucky little Seattle based startup working on building tools, automation, and visualisations on top of Kubernetes to make Kubernetes easier to use. 10 | And probably next week the 100 odd of us as joining VMware where we're going to keep doing exactly that. 11 | 12 | Anyway, that's enough of that. 13 | 14 | == What are we talking about today? 15 | 16 | My team, which you're looking at 90% of standing before you today, work within the broad remit of getting client traffic plumbed through to your applications running inside a Kubernetes cluster. 17 | 18 | That means in practical terms; HTTP, TLS and Layer 7. 19 | 20 | Networking people call this ingress traffic, which comes from the latin to "step in". 21 | 22 | There are 23 | 24 | == What should an ingress controller provider? 25 | 26 | == What are the problems with Ingress today 27 | 28 | Where does ingress fail to mean this promise? 29 | 30 | == What are Heptio doing about it? 31 | 32 | == What's next? 33 | 34 | 35 | 36 | 37 | -------------------------------------------------------------------------------- /gof.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gof.jpg -------------------------------------------------------------------------------- /golangsyd-october-2015.slide: -------------------------------------------------------------------------------- 1 | October meetup 2 | Sydney Go Users' Group 3 | 29 Oct 2015 4 | 5 | Dave Cheney 6 | dave@cheney.net 7 | http://dave.cheney.net/ 8 | @davecheney 9 | 10 | * House keeping 11 | 12 | * Agenda 13 | 14 | - Jason Buberel 15 | - Break 16 | - Xuanyi Chew 17 | - Lightning talks: Hemanta, Manish 18 | 19 | * GolangUK 20 | 21 | .image golangsyd-october-2015/golanguk.svg 400 _ 22 | 23 | Writeup: [[https://blog.golang.org/gouk15][blog.golang.org/gouk15]] 24 | 25 | * GothamGo 26 | 27 | .image golangsyd-october-2015/gothamgo.png 400 _ 28 | 29 | Videos (?): [[http://www.gothamgo.com][www.gothamgo.com]] 30 | 31 | * dotGo 32 | 33 | .image golangsyd-october-2015/dotgo.png _ 1024 34 | 35 | November 9th, 2015 36 | 37 | Visit: [[http://www.dotgo.eu][dotgo.eu]] 38 | 39 | * The Go Programming Language 40 | 41 | .image golangsyd-october-2015/cover.png 400 _ 42 | 43 | October 30th, 2015 44 | 45 | Buy: [[http://www.gopl.io][gopl.io]] 46 | 47 | -------------------------------------------------------------------------------- /golangsyd-october-2015/cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/golangsyd-october-2015/cover.png -------------------------------------------------------------------------------- /golangsyd-october-2015/dotgo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/golangsyd-october-2015/dotgo.png -------------------------------------------------------------------------------- /golangsyd-october-2015/gothamgo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/golangsyd-october-2015/gothamgo.png -------------------------------------------------------------------------------- /gopher-puzzlers/1e8ys6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/1e8ys6.jpg -------------------------------------------------------------------------------- /gopher-puzzlers/addition-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | fmt.Printf("%x %x", 'Ѕ', 'S') 7 | } 8 | -------------------------------------------------------------------------------- /gopher-puzzlers/addition.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | Ѕ := 1 7 | a, S := 1, 2 8 | 9 | fmt.Println(a + S + Ѕ) 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/bonus.go: -------------------------------------------------------------------------------- 1 | // Package p contains some sushi that doesn't care. 2 | 3 | package p 4 | 5 | const ಠ_ಠ = `¯\_(ツ)_/¯` 6 | 7 | func すし() string { 8 | return "🍣" 9 | } 10 | -------------------------------------------------------------------------------- /gopher-puzzlers/bonus2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unicode" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | for _, r := range []rune{'ಠ', 'す'} { 11 | fmt.Printf("%c: letter: %v, lowercase: %v, uppercase: %v\n", r, 12 | unicode.IsLetter(r), unicode.IsLower(r), unicode.IsUpper(r)) 13 | } 14 | } 15 | 16 | // END OMIT 17 | -------------------------------------------------------------------------------- /gopher-puzzlers/boom.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | go func() { panic("Boom") }() 7 | for i := 0; i < 10000000; { 8 | fmt.Print(".") 9 | } 10 | fmt.Println("Done") 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/cap.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | s := make([]int, 0, 8) 7 | fmt.Println(len(s)) 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/cap2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | s := make([]int, 0, 8) 7 | r := s[2:6:7] 8 | fmt.Println(len(r), cap(r)) 9 | } 10 | -------------------------------------------------------------------------------- /gopher-puzzlers/compexity.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | var i = 2 6 | 7 | func main() { 8 | j := 2 * 2i 9 | fmt.Println(j) 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/complexity.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "play.ground/complex" 4 | 5 | func main() { 6 | complex.Hello() 7 | } 8 | -- go.mod -- 9 | module play.ground 10 | -- complex/complex.go -- 11 | package complex 12 | 13 | import "fmt" 14 | 15 | func Hello() { 16 | fmt.Println("hello") 17 | } -------------------------------------------------------------------------------- /gopher-puzzlers/copy.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func main() { 8 | fmt.Println(copy([]byte("Tenth"), []byte("Birthday"))) 9 | } 10 | -------------------------------------------------------------------------------- /gopher-puzzlers/counting.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func main() { 8 | i := []int{ 9 | 1, 2, 3, 10 | } 11 | for j := range i { 12 | fmt.Println(j) 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /gopher-puzzlers/declarations.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START OMIT 4 | type ( 5 | T struct{ a, b int } 6 | ) 7 | 8 | import ( 9 | "math/rand" 10 | ) 11 | 12 | const ( 13 | Goku = 9001 14 | ) 15 | 16 | var ( 17 | π = 22 / 7.0 18 | ) 19 | 20 | func ( 21 | seven() int { return 7 } 22 | ) 23 | 24 | // END OMIT 25 | -------------------------------------------------------------------------------- /gopher-puzzlers/duration.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | const Second = uint64(time.Second) 11 | 12 | when := -1 * 5 * Second 13 | 14 | fmt.Printf("this happened %v ago\n", when) 15 | } 16 | 17 | // END OMIT 18 | -------------------------------------------------------------------------------- /gopher-puzzlers/empty.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | p, q := new(int), new(int) 7 | fmt.Println(*p == *q, p == q) 8 | 9 | r, s := new(struct{}), new(struct{}) 10 | fmt.Println(*r == *s, r == s) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/empty2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | var ( 6 | p, q = new(int), new(int) 7 | r, s = new(struct{}), new(struct{}) 8 | ) 9 | 10 | func main() { 11 | fmt.Println(*p == *q, p == q) 12 | fmt.Println(*r == *s, r == s) 13 | } 14 | -------------------------------------------------------------------------------- /gopher-puzzlers/falsey.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func main() { 8 | var err error 9 | _, true := err.(error) 10 | fmt.Println(true) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/floaty.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | var x uint32 = 100.5 + 100.5 + 2147483447.0 7 | fmt.Println(x) 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/genus.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | const ( 6 | Animal = iota 7 | Mineral 8 | Vegetable 9 | ) 10 | 11 | func genus(t uint) { 12 | switch t { 13 | case Animal: 14 | fmt.Println("Animal") 15 | case Mineral: 16 | fmt.Println("Mineral") 17 | case Vegetable: 18 | fmt.Println("Vegetable") 19 | } 20 | } 21 | 22 | func main() { 23 | genus(-Animal) 24 | } 25 | -------------------------------------------------------------------------------- /gopher-puzzlers/gofmt.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import 4 | 5 | // 6 | 7 | "fmt" 8 | 9 | func main() { 10 | fmt.Println("Seattle, WA ☔") 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/gofmt2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | /* 5 | explain why I'm importing fmt 6 | */ 7 | "fmt" 8 | /* 9 | explain why I'm importing pprof 10 | */ 11 | _ "net/http/pprof" 12 | ) 13 | 14 | func main() { 15 | fmt.Println("Seattle, WA ☔") 16 | } 17 | -------------------------------------------------------------------------------- /gopher-puzzlers/gofmt3.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import 4 | 5 | /* 6 | explain why I'm importing fmt 7 | */ 8 | 9 | "fmt" 10 | 11 | import 12 | 13 | /* 14 | explain why I'm importing pprof 15 | */ 16 | 17 | _ "net/http/pprof" 18 | 19 | func main() { 20 | fmt.Println("Seattle, WA ☔") 21 | } 22 | -------------------------------------------------------------------------------- /gopher-puzzlers/gophercon2016-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/gophercon2016-2.png -------------------------------------------------------------------------------- /gopher-puzzlers/hampton.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/hampton.jpg -------------------------------------------------------------------------------- /gopher-puzzlers/hmm-ii.go: -------------------------------------------------------------------------------- 1 | package 100 2 | -------------------------------------------------------------------------------- /gopher-puzzlers/hmm-iii.go: -------------------------------------------------------------------------------- 1 | package ಠ~ಠ 2 | -------------------------------------------------------------------------------- /gopher-puzzlers/hmm-iv.go: -------------------------------------------------------------------------------- 1 | package すし 2 | -------------------------------------------------------------------------------- /gopher-puzzlers/hmm.go: -------------------------------------------------------------------------------- 1 | package ಠ_ಠ 2 | -------------------------------------------------------------------------------- /gopher-puzzlers/init.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func f() {} 4 | func f() {} 5 | 6 | func main() {} 7 | -------------------------------------------------------------------------------- /gopher-puzzlers/init2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func init() { 4 | panic("first init") 5 | } 6 | 7 | func init() { 8 | panic("i am the second init") 9 | } 10 | 11 | func main() {} 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/init3: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/init3 -------------------------------------------------------------------------------- /gopher-puzzlers/init3.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | import "runtime" 5 | 6 | func init() { 7 | var pcs [1]uintptr 8 | runtime.Callers(1, pcs[:]) 9 | fn := runtime.FuncForPC(pcs[0]) 10 | fmt.Println(fn.Name()) 11 | } 12 | 13 | func main() {} 14 | -------------------------------------------------------------------------------- /gopher-puzzlers/init4.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | var x int 6 | 7 | func init() { 8 | x++ 9 | } 10 | 11 | func main() { 12 | init() 13 | fmt.Println(x) 14 | } 15 | -------------------------------------------------------------------------------- /gopher-puzzlers/initb.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func init() {} 4 | func init() {} 5 | 6 | func main() {} 7 | -------------------------------------------------------------------------------- /gopher-puzzlers/iota.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | const ( 8 | c1 int = iota 9 | c2 10 | c3 11 | c4 rune = iota 12 | c5 13 | ) 14 | 15 | func main() { 16 | fmt.Println(c5) 17 | } 18 | -------------------------------------------------------------------------------- /gopher-puzzlers/java-puzzlers.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/java-puzzlers.jpg -------------------------------------------------------------------------------- /gopher-puzzlers/keys.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | fmt.Println(len(map[interface{}]int{ 7 | new(int): 1, 8 | new(int): 2, 9 | new(struct{}): 3, 10 | new(struct{}): 4, 11 | })) 12 | } 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/keywords.go: -------------------------------------------------------------------------------- 1 | // START OMIT 2 | package main 3 | 4 | func A(string string) string { 5 | return string + string 6 | } 7 | 8 | func B(len int) int { 9 | return len+len 10 | } 11 | 12 | func C(val, default string) string { 13 | if val == "" { 14 | return default 15 | } 16 | return val 17 | } 18 | 19 | // END OMIT 20 | 21 | func main() {} 22 | -------------------------------------------------------------------------------- /gopher-puzzlers/keywords2.go: -------------------------------------------------------------------------------- 1 | // START OMIT 2 | package main 3 | 4 | var nil interface{} 5 | 6 | func main() { 7 | var x *int 8 | println(x == nil) 9 | } 10 | -------------------------------------------------------------------------------- /gopher-puzzlers/mainmain.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | var x int 6 | 7 | func init() { 8 | main() 9 | } 10 | 11 | func main() { 12 | x++ 13 | fmt.Println(x) 14 | } 15 | -------------------------------------------------------------------------------- /gopher-puzzlers/maps.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | m := make(map[string]int) 8 | m["foo"]++ 9 | fmt.Println(m["foo"]) 10 | } 11 | 12 | // END OMIT 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/maps2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | m := make(map[string]int) 8 | v := m["foo"] // HL 9 | v++ // HL 10 | m["foo"] = v // HL 11 | fmt.Println(m["foo"]) 12 | } 13 | 14 | // END OMIT 15 | -------------------------------------------------------------------------------- /gopher-puzzlers/missing-panic.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | panic("all is lost") 5 | } 6 | -------------------------------------------------------------------------------- /gopher-puzzlers/missing-panic2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "os" 4 | 5 | func main() { 6 | os.Stderr.Close() 7 | panic("all is lost") 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/missing-panic3.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | ) 7 | 8 | func main() { 9 | os.Stderr.Close() 10 | f, _ := os.Create("/tmp/wut") 11 | fmt.Println(f.Fd()) 12 | defer f.Close() 13 | panic("all is lost") 14 | } 15 | -------------------------------------------------------------------------------- /gopher-puzzlers/named-and-unnamed-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func f() { 4 | // START OMIT 5 | type stack []uintptr 6 | var st stack = make([]uintptr, 20) 7 | // END OMIT 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/named-and-unnamed-iii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START OMIT 4 | type T int 5 | 6 | func F(t T) {} 7 | 8 | func main() { 9 | var q int 10 | F(q) 11 | } 12 | 13 | // END OMIT 14 | -------------------------------------------------------------------------------- /gopher-puzzlers/named-and-unnamed-iv.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START OMIT 4 | type T []int 5 | 6 | func F(t T) {} 7 | 8 | func main() { 9 | var q []int 10 | F(q) 11 | } 12 | 13 | // END OMIT 14 | -------------------------------------------------------------------------------- /gopher-puzzlers/named-and-unnamed-v.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START OMIT 4 | type T []int 5 | 6 | func F(t T) {} 7 | 8 | func main() { 9 | var x []int 10 | var y T 11 | 12 | y = x // HL 13 | F(x) // HL 14 | 15 | _ = y // keep the compiler happy 16 | } 17 | 18 | // END OMIT 19 | -------------------------------------------------------------------------------- /gopher-puzzlers/named-and-unnamed.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type stack []uintptr 4 | 5 | func callers() stack { 6 | return make([]uintptr, 20) 7 | } 8 | 9 | func main() { 10 | callers() 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/neither.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | true := 6 7 | fmt.Println(true > false) 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/oddity.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | var x = 1 6 | 7 | func a() int { 8 | x *= 2 9 | return x 10 | } 11 | 12 | func b() int { 13 | x /= 2 14 | return x 15 | } 16 | 17 | func sub(a, b int) int { 18 | return a - b 19 | } 20 | 21 | func main() { 22 | fmt.Println(sub(a(), b())) 23 | } 24 | -------------------------------------------------------------------------------- /gopher-puzzlers/one-liner-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func f(a int, b uint) { 6 | var min = 0 7 | min = copy(make([]struct{}, a), make([]struct{}, b)) // HL 8 | fmt.Printf("The min of %d and %d is %d\n", a, b, min) 9 | } 10 | 11 | func main() { 12 | f(9000, 314) 13 | } 14 | -------------------------------------------------------------------------------- /gopher-puzzlers/one-liner.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func f(a int, b uint) { 6 | var min = 0 7 | fmt.Printf("The min of %d and %d is %d\n", a, b, min) 8 | } 9 | 10 | func main() { 11 | f(9000, 314) 12 | } 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/one-two-three-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | 7 | func main() { 8 | ch := make(chan int, 1) 9 | go func() { 10 | select { 11 | case ch <- 1: 12 | case ch <- 2: 13 | default: 14 | ch <- 3 15 | } 16 | }() 17 | fmt.Println(<-ch) 18 | } 19 | 20 | // END OMIT 21 | -------------------------------------------------------------------------------- /gopher-puzzlers/one-two-three.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | ) 7 | 8 | // START OMIT 9 | func A() int { 10 | time.Sleep(100 * time.Millisecond) 11 | return 1 12 | } 13 | 14 | func B() int { 15 | time.Sleep(1000 * time.Millisecond) 16 | return 2 17 | } 18 | 19 | func main() { 20 | ch := make(chan int, 1) 21 | go func() { 22 | select { 23 | case ch <- A(): 24 | case ch <- B(): 25 | default: 26 | ch <- 3 27 | } 28 | }() 29 | fmt.Println(<-ch) 30 | } 31 | 32 | // END OMIT 33 | -------------------------------------------------------------------------------- /gopher-puzzlers/party.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gopher-puzzlers/party.png -------------------------------------------------------------------------------- /gopher-puzzlers/pointer.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | type P *int 6 | type Q *int 7 | 8 | func main() { 9 | var p P = new(int) 10 | *p += 8 11 | var x *int = p 12 | var q Q = x 13 | *q++ 14 | fmt.Println(*p, *q) 15 | } 16 | -------------------------------------------------------------------------------- /gopher-puzzlers/power.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | a := 2 ^ 15 5 | b := 4 ^ 15 6 | if a > b { 7 | println("a") 8 | } else { 9 | println("b") 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/rune.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func main() { 8 | var r rune 9 | fmt.Printf("%T\n", r) 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/shift.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func a() { 4 | //START1 OMIT 5 | x := 2 << -1 6 | //END1 OMIT 7 | } 8 | 9 | func b() { 10 | //START2 OMIT 11 | x := 1 12 | y := 2 << x 13 | //END2 OMIT 14 | } 15 | 16 | func c() { 17 | // START3 OMIT 18 | var x = uint(1) 19 | y := 1.0 << x 20 | // END3 OMIT 21 | } 22 | 23 | func d() { 24 | // START4 OMIT 25 | x := uint(0) 26 | y := make([]int, 10) 27 | z := y[1.0<> (^uint(0) >> 63) // HL 11 | type X [n]uint 12 | type Y [n]int 13 | 14 | var x X 15 | var y Y 16 | fmt.Println(unsafe.Sizeof(x), unsafe.Sizeof(y)) 17 | } 18 | 19 | // END OMIT 20 | -------------------------------------------------------------------------------- /gopher-puzzlers/size-of-things-iii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unsafe" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | const n = ^uint(6) % 7 // HL 11 | type X [n]uint 12 | type Y [n]int 13 | 14 | var x X 15 | var y Y 16 | fmt.Println(unsafe.Sizeof(x), unsafe.Sizeof(y)) 17 | } 18 | 19 | // END OMIT 20 | -------------------------------------------------------------------------------- /gopher-puzzlers/size-of-things.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unsafe" 6 | ) 7 | 8 | func main() { 9 | const n = 4 10 | type X [n]uint 11 | type Y [n]int 12 | 13 | var x X 14 | var y Y 15 | fmt.Println(unsafe.Sizeof(x), unsafe.Sizeof(y)) 16 | } 17 | -------------------------------------------------------------------------------- /gopher-puzzlers/sizeclass-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | b := make([]int, 32) 8 | b = append(b, 99) 9 | fmt.Println("len:", len(b), "cap:", cap(b)) 10 | } 11 | 12 | // END OMIT 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/sizeclass.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | b := make([]int, 1023) 8 | b = append(b, 99) 9 | fmt.Println("len:", len(b), "cap:", cap(b)) 10 | } 11 | 12 | // END OMIT 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/sizeclassb.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | b := make([]int, 1024) // HL 8 | b = append(b, 99) 9 | fmt.Println("len:", len(b), "cap:", cap(b)) 10 | } 11 | 12 | // END OMIT 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/sliced.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func main() { 8 | s := make([]int, 3, 8) 9 | s[2:5][0] = 5 10 | fmt.Println(s) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/slices.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | b := []byte("Hello") 7 | r := []rune("Sydney") 8 | i := []int("Gophers") 9 | fmt.Println(b, r, i) 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/snip.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | ) 7 | 8 | func main() { 9 | s := strings.TrimRight("abcdefedcba", "abcdef") 10 | fmt.Printf("%q\n", s) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/snip2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | ) 7 | 8 | func main() { 9 | s := strings.TrimSuffix("abcdefedcba", "abcdef") 10 | fmt.Printf("%q\n", s) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/snowman-or-poop.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "log" 5 | "os" 6 | "time" 7 | ) 8 | 9 | // START OMIT 10 | func main() { 11 | r, w, err := os.Pipe() 12 | if err != nil { 13 | log.Fatal(err) 14 | } 15 | 16 | go func() { 17 | time.Sleep(1 * time.Second) 18 | r.Close() 19 | }() 20 | 21 | var buf [16]byte 22 | _, err = r.Read(buf[:]) 23 | if err != nil { 24 | log.Println("☃") 25 | } else { 26 | log.Println("💩") 27 | } 28 | 29 | w.Close() 30 | } 31 | 32 | // END OMIT 33 | -------------------------------------------------------------------------------- /gopher-puzzlers/snowman.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | r := []rune("☃") 7 | fmt.Println(r) 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/space-packing-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unsafe" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | type T struct { 11 | _ struct{} 12 | a int 13 | } 14 | var t T 15 | fmt.Println(unsafe.Sizeof(t)) 16 | } 17 | 18 | //END OMIT 19 | -------------------------------------------------------------------------------- /gopher-puzzlers/space-packing-iii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | type T struct { 5 | a struct{} 6 | b int 7 | c int 8 | d struct{} 9 | } 10 | 11 | var t T 12 | println(&t.a) 13 | println(&t.b) 14 | println(&t.c) 15 | println(&t.d) 16 | } 17 | -------------------------------------------------------------------------------- /gopher-puzzlers/space-packing-iv.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unsafe" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | type T struct { 11 | a int // 4 or 8 bytes 12 | _ [1]byte // 1 byte, padded to 4 or 8 13 | } 14 | var t T 15 | fmt.Println(unsafe.Sizeof(t)) 16 | } 17 | 18 | //END OMIT 19 | -------------------------------------------------------------------------------- /gopher-puzzlers/space-packing.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "unsafe" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | type T struct { 11 | a int 12 | _ struct{} 13 | } 14 | var t T 15 | fmt.Println(unsafe.Sizeof(t)) 16 | } 17 | 18 | //END OMIT 19 | -------------------------------------------------------------------------------- /gopher-puzzlers/terminator.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | v := []int{1, 2, 3} 7 | for i := range v { 8 | v = append(v, i) 9 | } 10 | fmt.Println(v) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/terminator2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | v := []int{1, 2, 3} 7 | for i, n := 0, len(v); i < n; i++ { 8 | v = append(v, i) 9 | } 10 | fmt.Println(v) 11 | } 12 | -------------------------------------------------------------------------------- /gopher-puzzlers/tick.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "time" 4 | 5 | func main() { 6 | t := time.NewTicker(100) 7 | for range t.C { 8 | t.Stop() 9 | } 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/tick2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | ) 7 | 8 | // START OMIT 9 | func main() { 10 | t := time.NewTicker(100) 11 | t.Stop() 12 | for { 13 | select { 14 | case <-t.C: 15 | fmt.Println("tick") 16 | } 17 | } 18 | } 19 | 20 | // END OMIT 21 | -------------------------------------------------------------------------------- /gopher-puzzlers/tokens-ii.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | // START OMIT 7 | x := 1 * 8 | 2 * 9 | 3 10 | fmt.Println(x) 11 | // END OMIT 12 | } 13 | -------------------------------------------------------------------------------- /gopher-puzzlers/tokens.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type bar0 struct { 4 | n * 5 | bar0 6 | m []bar0 7 | } 8 | 9 | func main() { 10 | } 11 | -------------------------------------------------------------------------------- /gopher-puzzlers/touche.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | s := string("touché") 7 | b := []byte{'t', 'o', 'u', 'c', 'h', 'é'} 8 | fmt.Println(len(s) == len(b)) 9 | } 10 | -------------------------------------------------------------------------------- /gopher-puzzlers/truthy.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | false := true 7 | fmt.Println(true == false == true) 8 | } 9 | -------------------------------------------------------------------------------- /gopher-puzzlers/twohundred.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | // START OMIT 8 | func main() { 9 | x := []int{100, 200, 300, 400, 500, 600, 700} 10 | y := &x[1] 11 | x = append(x, 800) 12 | for i := range x { 13 | x[i]++ 14 | } 15 | z := &x[1] 16 | for i := range x { 17 | x[i]++ 18 | } 19 | fmt.Println(*y, *z) 20 | } 21 | 22 | // END OMIT 23 | -------------------------------------------------------------------------------- /gopher-puzzlers/which-is-faster-ii.go: -------------------------------------------------------------------------------- 1 | package makebench 2 | 3 | import "testing" 4 | 5 | var result []byte 6 | 7 | // START OMIT 8 | func BenchmarkMakeOneByte(b *testing.B) { 9 | var v []byte 10 | for i := 0; i < b.N; i++ { 11 | v = make([]byte, 1) 12 | } 13 | result = v 14 | } 15 | 16 | func BenchmarkOneLiteralByte(b *testing.B) { 17 | var v []byte 18 | for i := 0; i < b.N; i++ { 19 | v = []byte{0} 20 | } 21 | result = v 22 | } 23 | 24 | // END OMIT 25 | -------------------------------------------------------------------------------- /gopher-puzzlers/which-is-faster.go: -------------------------------------------------------------------------------- 1 | package makebench 2 | 3 | import "testing" 4 | 5 | var result []byte 6 | 7 | // START OMIT 8 | func BenchmarkOneLiteralByte(b *testing.B) { 9 | var v []byte 10 | for i := 0; i < b.N; i++ { 11 | v = []byte{0} 12 | } 13 | result = v 14 | } 15 | 16 | func BenchmarkMakeOneByte(b *testing.B) { 17 | var v []byte 18 | for i := 0; i < b.N; i++ { 19 | v = make([]byte, 1) 20 | } 21 | result = v 22 | } 23 | 24 | // END OMIT 25 | -------------------------------------------------------------------------------- /gopher-puzzlers/zero.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type Hash [32]byte 4 | 5 | func MustNotBeZero(h Hash) { 6 | if h == Hash{} { 7 | panic("0") 8 | } 9 | } 10 | 11 | func main() {} 12 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/linkname/linkname.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "sync" 6 | "unsafe" 7 | ) 8 | 9 | func main() { 10 | var wg sync.WaitGroup 11 | wg.Add(10) 12 | for i := 0; i < 10; i++ { 13 | go func() { 14 | defer wg.Done() 15 | fmt.Println("goroutineid:", goroutineid()) 16 | }() 17 | } 18 | wg.Wait() 19 | } 20 | 21 | func goroutineid() int64 { 22 | m := (*m)(unsafe.Pointer(runtime_getm())) 23 | g := m.curg 24 | return g.goid 25 | } 26 | 27 | //go:linkname runtime_getm runtime.getm 28 | func runtime_getm() uintptr 29 | 30 | type m struct { 31 | g0 *g // goroutine with scheduling stack 32 | morebuf gobuf // gobuf arg to morestack 33 | divmod uint32 // div/mod denominator for arm - known to liblink 34 | 35 | // Fields not known to debuggers. 36 | procid uint64 // for debuggers, but offset not hard-coded 37 | gsignal *g // signal-handling g 38 | goSigStack gsignalStack // Go-allocated signal handling stack 39 | sigmask sigset // storage for saved signal mask 40 | tls [6]uintptr // thread-local storage (for x86 extern register) 41 | mstartfn func() 42 | curg *g // current running goroutine 43 | } 44 | 45 | type g struct { 46 | // Stack parameters. 47 | // stack describes the actual stack memory: [stack.lo, stack.hi). 48 | // stackguard0 is the stack pointer compared in the Go stack growth prologue. 49 | // It is stack.lo+StackGuard normally, but can be StackPreempt to trigger a preemption. 50 | // stackguard1 is the stack pointer compared in the C stack growth prologue. 51 | // It is stack.lo+StackGuard on g0 and gsignal stacks. 52 | // It is ~0 on other goroutine stacks, to trigger a call to morestackc (and crash). 53 | stack stack // offset known to runtime/cgo 54 | stackguard0 uintptr // offset known to liblink 55 | stackguard1 uintptr // offset known to liblink 56 | 57 | _panic unsafe.Pointer // innermost panic - offset known to liblink 58 | _defer unsafe.Pointer // innermost defer 59 | m unsafe.Pointer // current m; offset known to arm liblink 60 | sched gobuf 61 | syscallsp uintptr // if status==Gsyscall, syscallsp = sched.sp to use during gc 62 | syscallpc uintptr // if status==Gsyscall, syscallpc = sched.pc to use during gc 63 | stktopsp uintptr // expected sp at top of stack, to check in traceback 64 | param unsafe.Pointer // passed parameter on wakeup 65 | atomicstatus uint32 66 | stackLock uint32 // sigprof/scang lock; TODO: fold in to atomicstatus 67 | goid int64 68 | } 69 | 70 | type sigset uint32 71 | 72 | type guintptr uintptr 73 | 74 | type gobuf struct { 75 | // The offsets of sp, pc, and g are known to (hard-coded in) libmach. 76 | // 77 | // ctxt is unusual with respect to GC: it may be a 78 | // heap-allocated funcval so write require a write barrier, 79 | // but gobuf needs to be cleared from assembly. We take 80 | // advantage of the fact that the only path that uses a 81 | // non-nil ctxt is morestack. As a result, gogo is the only 82 | // place where it may not already be nil, so gogo uses an 83 | // explicit write barrier. Everywhere else that resets the 84 | // gobuf asserts that ctxt is already nil. 85 | sp uintptr 86 | pc uintptr 87 | g guintptr 88 | ctxt unsafe.Pointer // this has to be a pointer so that gc scans it 89 | ret uint64 90 | lr uintptr 91 | bp uintptr // for GOEXPERIMENT=framepointer 92 | } 93 | 94 | // Stack describes a Go execution stack. 95 | // The bounds of the stack are exactly [lo, hi), 96 | // with no implicit data structures on either side. 97 | type stack struct { 98 | lo uintptr 99 | hi uintptr 100 | } 101 | 102 | // gsignalStack saves the fields of the gsignal stack changed by 103 | // setGsignalStack. 104 | type gsignalStack struct { 105 | stack stack 106 | stackguard0 uintptr 107 | stackguard1 uintptr 108 | stktopsp uintptr 109 | } 110 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/linkname/linkname.s: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/gos-hidden-pragmas-examples/linkname/linkname.s -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/noescape/noescape.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | //go:noescape 6 | func length(s string) int 7 | 8 | func main() { 9 | s := "hello world" 10 | l := length(s) 11 | fmt.Println(l) 12 | } 13 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/norace.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | var v int 4 | 5 | //go:norace 6 | func add() { 7 | v++ 8 | } 9 | 10 | func main() { 11 | for i := 0; i < 5; i++ { 12 | go add() 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/notinheap.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | //go:notinheap 6 | type T struct { 7 | a int 8 | } 9 | 10 | //go:noinline 11 | func F() *T { 12 | var t T 13 | return &t 14 | } 15 | 16 | func main() { 17 | fmt.Println(F().a) 18 | } 19 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/nowritebarrier.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | var count int 4 | 5 | //go:nowritebarrier 6 | func increment() { 7 | count++ 8 | } 9 | 10 | func main() { 11 | for i := 0; i < 100; i++ { 12 | increment() 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /gos-hidden-pragmas-examples/redzone.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | type T [256]byte // a large stack allocated type 4 | 5 | //go:nosplit 6 | func A(t T) { 7 | B(t) 8 | } 9 | 10 | //go:nosplit 11 | func B(t T) { 12 | C(t) 13 | } 14 | 15 | //go:nosplit 16 | func C(t T) { 17 | D(t) 18 | } 19 | 20 | //go:nosplit 21 | func D(t T) {} 22 | 23 | func main() { 24 | var t T 25 | A(t) 26 | } 27 | -------------------------------------------------------------------------------- /hde.slide: -------------------------------------------------------------------------------- 1 | HDE, Tokyo, Japan 2 | 8 Dec 2014 3 | 4 | Dave Cheney 5 | dave@cheney.net 6 | http://dave.cheney.net/ 7 | @davecheney 8 | 9 | * Programming philosophy 10 | 11 | The 80's were a golden time for programming language design. 12 | 13 | - ADA, Smalltalk, C++, Self 14 | - First AI winter 15 | - Hardware was large enough to support _huge_ programs, developed by _huge_ numbers of programmers. 16 | 17 | The 90's was a golden time for programming philosophy. 18 | 19 | - SOLID 20 | - TDD, BDD, DDD 21 | - Modularity 22 | 23 | People have thought about designing huge programs since the 80's, don't reinvent their wheel. 24 | 25 | * Overcoming complexity 26 | 27 | "There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." 28 | .caption C. A. R. Hoare, 1980 Turing Award Lecture 29 | 30 | Simplicity is not a synonym for easy, nor is achieving a design which is simple an easy task. Putting this into the context of programming languages, something which is simple may take a little longer, it may be a little more verbose, but it will be more comprehensible, more extensible, more adaptable, and exhibit lower coupling. 31 | 32 | "We know we have to keep it crisp, disentangled, and simple if we refuse to be crushed by the complexities of our own making." 33 | .caption [[https://www.cs.utexas.edu/users/EWD/transcriptions/EWD12xx/EWD1243a.html][Edsger W.Dijkstra]] 34 | 35 | "Simplicity is the unavoidable price which we must pay for reliability." 36 | .caption C. A. R. Hoare 37 | 38 | * Readability 39 | "Programs must be written for people to read, and only incidentally for machines to execute." 40 | .caption Hal Abelson and Gerald Sussman, Structure and Interpretation of Computer Programs 41 | 42 | The source code of a program is an intermediary form, somewhere between the author's concept and the computer's executable notation. In that sense source code is an encoded form of an idea, a process, an algorithm, or a formula. Just like musical notation. 43 | 44 | "We don’t read code, we decode it. We examine it. A piece of code is not literature; it is a specimen." 45 | .caption [[http://www.gigamonkeys.com/code-reading/][Peter Seibel]] 46 | 47 | Go places readability above all other considerations, because source code is written to be read. Many of the choices relating to Go's syntax speak to the needs of the reader. Go's syntax is spartan because of its drive for readability. 48 | 49 | * Change 50 | 51 | "Design is the art of arranging code that needs to work _today_, and to be easy to change _forever_." 52 | .caption Sandi Metz 53 | 54 | Why do Go programmers care _so_much_ about readability? 55 | 56 | Code is written once and read many times. 57 | 58 | Change is what we're paid to do. 59 | 60 | We _must_ make it possible for our software to accommodate business change. Quickly. Safely. Reliably. 61 | 62 | * Design 63 | 64 | "There's a great line in Strunk & White about [where] you'll find places where the rules are broken, but you'll find some compensating merit to make up for it." 65 | .caption Brian Kernighan 66 | 67 | "Some people think that programming should be an art, every construct should be perfectly expressed and abstracted and beautiful, and things like simplicity, portability, and performance are unimportant details. Other people write programs to consume input and produce output, for various values of input and output. The real value of development is the work the program does, not the work the programmer does." 68 | .caption [[https://www.reddit.com/r/golang/comments/30bndg/how_do_you_respond_to_these_antigolang_pieces/cpslzwh][/r/golang]] 69 | 70 | "Design means saying No." 71 | .caption _Me_ 72 | 73 | * Behaviour and type are different things 74 | 75 | Interfaces describe behaviour, structs describe data. 76 | 77 | OO mocks this idea; the _data_class_ is a called a code smell. 78 | 79 | Not everything is an object, some things are just data, and other things are just behaviour. 80 | 81 | These should be separate concepts. 82 | 83 | - If you're talking about behaviour, return functions or interface value. 84 | - If you're talking about data, return a primitive value or struct. 85 | 86 | * Accept interfaces, return structs 87 | 88 | APIs should describe their parameters in terms of their behaviour rather than their type. 89 | 90 | Functions should either consume universe data types or interface values _they_define_. 91 | 92 | ie. `ioutil.ReadAll(io.Reader)` vs `ioutil.ReadAll(*os.File)` 93 | 94 | Where possible, interfaces belong to the consumer. Don't import a package just to agree on an interface defintion. 95 | 96 | So, why return concrete types (or universe types)? 97 | 98 | - Reduce the possibility of typed nils. 99 | - You're already coupled to the name of the function, so the name of a type is no worse. 100 | 101 | * Give things fewer names 102 | 103 | Reduce source level coupling by removing the requirement for everyone to use the same name 104 | 105 | Avoid named return values. 106 | 107 | * Methods on a T vs methods on a *T 108 | Everything in Go is a copy, without exception. 109 | 110 | We also know that method calls are just syntactic sugar for calling a function an passing the receiver as the first parameter. So, what are the rules for how the recover is passed, as a value, or a pointer. 111 | 112 | - Use a pointer receiver when the caller will change the receiver, possibly this could be written as use a pointer receiver when the method is stateful. 113 | - Use a value receiver when the receiver is immutable, one of the few std lib examples of this is the `time.Time` type. 114 | 115 | But even that is incomplete, many methods exist because they need to implement an interfaces, otherwise they'd just be functions, and that seems to cut across this immutable/stateful boundary. 116 | 117 | - In general, use pointer receivers. 118 | - If you use a ptr receiver on one method, the rest should follow suit. 119 | - Return values with methods by reference, and those without by value. 120 | 121 | * Errors are just values 122 | 123 | "Errors are just values." 124 | .caption Rob Pike, Go Proverbs. 125 | 126 | - Errors should be opaque 127 | - Never use nil to indicate failure 128 | - Only handle an error once; handle an error _or_ return it 129 | - Assert errors for behaviour, not type 130 | 131 | * Fewer, larger packages 132 | 133 | Think of your package's name as an elevator pitch to describe what it does, using just one word. 134 | 135 | Go eschewed type hierarchy, don't make the mistake or replacing that with an elaborate package hierarchy. That will lead to too many internal types being made public. 136 | 137 | - Two packages that are always imported together, may be combine 138 | - One package that is never imported, only via a third, maybe combine. 139 | 140 | Each Go package is in effect it’s own small Go program. We see this in the compilation model. 141 | 142 | - In C you ask questions like, "if I change this header file, which source files include it and need rebuilding?" 143 | - In Go the unit of compilation is the package, so we ask, "which packages does this package depend on?" 144 | 145 | * Avoid conflicts with the names of common local variables 146 | Don't steal the name of a common identifier for the name of a package 147 | 148 | The import statement declares a new identifier at the package level. 149 | 150 | Consider the problems naming a package that deals with file descriptors “fd” as fd would be the natural identifier for a file descriptor value returned by some hypothetical fd.Open function. 151 | 152 | * Avoid empty packages. 153 | 154 | An empty package is a sign you developing a package hierarchy. 155 | 156 | An exception to this rule is the use of grouping main packages into a `cmd` directory. 157 | 158 | * The most reasonable channels sizes are usually zero and one. 159 | 160 | Prefer unbuffered channels. 161 | 162 | * How to lay out Go projects 163 | 164 | - single pkg, single lib, Multi paxkage, application, project. 165 | 166 | Use `testdata`, use `vendor/` (if you're an application), use `internal/` 167 | 168 | - vendoring is what it is, use grep and write a Makefile for complex projects. 169 | 170 | Get onboard with `$GOPATH`, use `go`build`, `go`install`, `go`test`-race`. 171 | 172 | Comment everything, use the google shell docs 173 | 174 | * Name your tests with a prefix of they type they test 175 | 176 | Name your tests with a prefix of they type they test so you can run and check the coverage of a type without taking the code out of circuit 177 | 178 | * Only log actionable items 179 | 180 | Go is designed for production, the ones concerned with its output should be told when and only when it goes wrong 181 | 182 | For development add a debug or -v flag 183 | 184 | * Use streaming IO interfaces 185 | 186 | Where-ever possible avoid reading data into a `[]byte` and passing it around. 187 | 188 | Depending on the request you may end up reading megabytes (or more!) of data into memory. This places huge pressure on the GC, which will increase the average latency of your application. 189 | 190 | Instead use `io.Reader` and `io.Writer` to construct processing pipelines to cap the amount of memory in use per request. 191 | 192 | For efficiency, consider implementing `io.ReaderFrom` / `io.WriterTo` if you use a lot of `io.Copy`. These interface are more efficient and avoid copying memory into a temporary buffer. 193 | 194 | * Syntax is irrelevant 195 | 196 | To the extent that readability is paramount, language syntax is mostly irrelevant. 197 | 198 | What matters in a multicore world is the semantics of how multiple processes (goroutines) communicate. 199 | 200 | * Never start a goroutine without knowing how it will finish 201 | 202 | Goroutines are cheap to start and cheap to run, but they do have a finite cost in terms of memory footprint; you cannot create an infinite number of them. 203 | 204 | Every time you use the `go` keyword in your program to launch a goroutine, you must *know* how, and when, that goroutine will exit. 205 | 206 | If you don't know the answer, that's a potential memory leak. 207 | 208 | In your design, some goroutines may run until the program exits. These goroutines are rare enough to not become an exception to the rule. 209 | 210 | *Never*start*a*goroutine*without*knowing*how*it*will*stop*. 211 | 212 | * Don't trade performance for reliability 213 | 214 | "I can make things very fast if they don't have to be correct." 215 | .caption Russ Cox 216 | 217 | Performance and reliability are equally important. 218 | 219 | I see little value in making a very fast server that panics, deadlocks or OOMs on a regular basis. 220 | 221 | Don't trade performance for reliability. 222 | -------------------------------------------------------------------------------- /introduction-to-go/concurrency1.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "time" 8 | ) 9 | 10 | // f START OMIT 11 | func f(msg string, delay time.Duration) { 12 | for { 13 | fmt.Println(msg) 14 | time.Sleep(delay) 15 | } 16 | } 17 | 18 | // f END OMIT 19 | 20 | // main START OMIT 21 | func main() { 22 | go f("A--", 300*time.Millisecond) 23 | go f("-B-", 500*time.Millisecond) 24 | go f("--C", 1100*time.Millisecond) 25 | time.Sleep(20 * time.Second) 26 | } 27 | 28 | // main END OMIT 29 | -------------------------------------------------------------------------------- /introduction-to-go/concurrency2.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "time" 8 | ) 9 | 10 | // f START OMIT 11 | func f(msg string, delay time.Duration, ch chan string) { 12 | for { 13 | ch <- msg 14 | time.Sleep(delay) 15 | } 16 | } 17 | 18 | // f END OMIT 19 | 20 | // main START OMIT 21 | func main() { 22 | ch := make(chan string) 23 | go f("A--", 300*time.Millisecond, ch) 24 | go f("-B-", 500*time.Millisecond, ch) 25 | go f("--C", 1100*time.Millisecond, ch) 26 | 27 | for i := 0; i < 100; i++ { 28 | fmt.Println(i, <-ch) 29 | } 30 | } 31 | 32 | // main END OMIT 33 | -------------------------------------------------------------------------------- /introduction-to-go/examples.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package examples 4 | 5 | // IndexOfAny START OMIT 6 | func IndexOfAny(str string, chars []rune) int { 7 | if len(str) == 0 || len(chars) == 0 { 8 | return -1 9 | } 10 | for i, ch := range str { 11 | for _, match := range chars { 12 | if ch == match { 13 | return i 14 | } 15 | } 16 | } 17 | return -1 18 | } 19 | 20 | // IndexOfAny END OMIT 21 | -------------------------------------------------------------------------------- /introduction-to-go/go1-vs-tip.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/introduction-to-go/go1-vs-tip.png -------------------------------------------------------------------------------- /introduction-to-go/hello.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import "fmt" 6 | 7 | func main() { 8 | fmt.Println("Hello, 世界!") 9 | } 10 | -------------------------------------------------------------------------------- /introduction-to-go/hellohttp.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "net/http" 7 | ) 8 | 9 | func HelloServer(w http.ResponseWriter, req *http.Request) { 10 | log.Println(req.URL) 11 | fmt.Fprintf(w, "Hello, 世界!\nURL = %s\n", req.URL) 12 | } 13 | 14 | func main() { 15 | fmt.Println("please connect to localhost:7777/hello") 16 | http.HandleFunc("/hello", HelloServer) 17 | log.Fatal(http.ListenAndServe(":7777", nil)) 18 | } 19 | -------------------------------------------------------------------------------- /introduction-to-go/histo.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "go/ast" 8 | "go/parser" 9 | "go/token" 10 | "io/ioutil" 11 | "path/filepath" 12 | "runtime" 13 | "sort" 14 | "strings" 15 | "time" 16 | ) 17 | 18 | func walk(dir string, f func(string) bool) bool { 19 | fis, err := ioutil.ReadDir(dir) 20 | if err != nil { 21 | panic(err) 22 | } 23 | // parse all *.go files in directory; 24 | // traverse subdirectories, but don't walk into testdata 25 | for _, fi := range fis { 26 | path := filepath.Join(dir, fi.Name()) 27 | if fi.IsDir() { 28 | if fi.Name() != "testdata" { 29 | if !walk(path, f) { 30 | return false 31 | } 32 | } 33 | } else if strings.HasSuffix(fi.Name(), ".go") && !strings.HasPrefix(fi.Name(), ".") { 34 | if !f(path) { 35 | return false 36 | } 37 | } 38 | } 39 | return true 40 | } 41 | 42 | func walkStdLib(f func(filename string) bool) { 43 | walk(filepath.Join(runtime.GOROOT(), "src"), f) 44 | } 45 | 46 | type histogram map[string]int 47 | 48 | func (h histogram) add(filename string) { 49 | f, err := parser.ParseFile(token.NewFileSet(), filename, nil, 0) 50 | if err != nil { 51 | panic(err) 52 | } 53 | 54 | ast.Inspect(f, func(n ast.Node) bool { 55 | if n, ok := n.(ast.Stmt); ok { 56 | h[fmt.Sprintf("%T", n)]++ 57 | } 58 | return true 59 | }) 60 | } 61 | 62 | // print START OMIT 63 | func (h histogram) print() { 64 | var list []entry 65 | var total int 66 | for key, count := range h { 67 | list = append(list, entry{key, count}) 68 | total += count 69 | } 70 | sort.Sort(byCount(list)) 71 | 72 | percent := 100 / float64(total) 73 | for i, e := range list { 74 | fmt.Printf("%4d. %5.2f%% %5d %s\n", i, float64(e.count)*percent, e.count, e.key) 75 | } 76 | } 77 | 78 | // print END OMIT 79 | 80 | // byCount START OMIT 81 | type entry struct { 82 | key string 83 | count int 84 | } 85 | 86 | type byCount []entry 87 | 88 | func (s byCount) Len() int { return len(s) } 89 | func (s byCount) Swap(i, j int) { s[i], s[j] = s[j], s[i] } 90 | func (s byCount) Less(i, j int) bool { 91 | x, y := s[i], s[j] 92 | if x.count != y.count { 93 | return x.count > y.count // want larger count first 94 | } 95 | return x.key < y.key 96 | } 97 | 98 | // byCount END OMIT 99 | 100 | // main START OMIT 101 | func main() { 102 | start := time.Now() 103 | h := make(histogram) 104 | walkStdLib(func(filename string) bool { 105 | h.add(filename) 106 | return true 107 | }) 108 | 109 | h.print() 110 | fmt.Println(time.Since(start)) 111 | } 112 | 113 | // main END OMIT 114 | -------------------------------------------------------------------------------- /introduction-to-go/histo0.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "go/ast" 8 | "go/parser" 9 | "go/token" 10 | "io/ioutil" 11 | "path/filepath" 12 | "runtime" 13 | "strings" 14 | ) 15 | 16 | func walk(dir string, f func(string) bool) bool { 17 | fis, err := ioutil.ReadDir(dir) 18 | if err != nil { 19 | panic(err) 20 | } 21 | // parse all *.go files in directory; 22 | // traverse subdirectories, but don't walk into testdata 23 | for _, fi := range fis { 24 | path := filepath.Join(dir, fi.Name()) 25 | if fi.IsDir() { 26 | if fi.Name() != "testdata" { 27 | if !walk(path, f) { 28 | return false 29 | } 30 | } 31 | } else if strings.HasSuffix(fi.Name(), ".go") && !strings.HasPrefix(fi.Name(), ".") { 32 | if !f(path) { 33 | return false 34 | } 35 | } 36 | } 37 | return true 38 | } 39 | 40 | func walkStdLib(f func(filename string) bool) { 41 | walk(filepath.Join(runtime.GOROOT(), "src"), f) 42 | } 43 | 44 | // histogram START OMIT 45 | type histogram map[string]int 46 | 47 | // histogram END OMIT 48 | 49 | // add START OMIT 50 | func (h histogram) add(filename string) { 51 | f, err := parser.ParseFile(token.NewFileSet(), filename, nil, 0) 52 | if err != nil { 53 | panic(err) 54 | } 55 | 56 | ast.Inspect(f, func(n ast.Node) bool { 57 | if n, ok := n.(ast.Stmt); ok { // type test: is n an ast.Stmt? 58 | h[fmt.Sprintf("%T", n)]++ 59 | } 60 | return true 61 | }) 62 | } 63 | 64 | // add END OMIT 65 | 66 | // print START OMIT 67 | func (h histogram) print() { 68 | // determine total number of statements 69 | total := 0 70 | for _, count := range h { 71 | total += count 72 | } 73 | 74 | // print map entries 75 | i := 0 76 | percent := 100 / float64(total) 77 | for key, count := range h { 78 | fmt.Printf("%4d. %5.2f%% %5d %s\n", i, float64(count)*percent, count, key) 79 | i++ 80 | } 81 | } 82 | 83 | // print END OMIT 84 | 85 | // main START OMIT 86 | func main() { 87 | // body START OMIT 88 | h := make(histogram) 89 | walkStdLib(func(filename string) bool { 90 | h.add(filename) // does all the hard work 91 | return true 92 | }) 93 | // body END OMIT 94 | h.print() 95 | } 96 | 97 | // main END OMIT 98 | -------------------------------------------------------------------------------- /introduction-to-go/histop.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "go/ast" 8 | "go/parser" 9 | "go/token" 10 | "io/ioutil" 11 | "path/filepath" 12 | "runtime" 13 | "sort" 14 | "strings" 15 | "time" 16 | ) 17 | 18 | func walk(dir string, f func(string) bool) bool { 19 | fis, err := ioutil.ReadDir(dir) 20 | if err != nil { 21 | panic(err) 22 | } 23 | // parse all *.go files in directory; 24 | // traverse subdirectories, but don't walk into testdata 25 | for _, fi := range fis { 26 | path := filepath.Join(dir, fi.Name()) 27 | if fi.IsDir() { 28 | if fi.Name() != "testdata" { 29 | if !walk(path, f) { 30 | return false 31 | } 32 | } 33 | } else if strings.HasSuffix(fi.Name(), ".go") && !strings.HasPrefix(fi.Name(), ".") { 34 | if !f(path) { 35 | return false 36 | } 37 | } 38 | } 39 | return true 40 | } 41 | 42 | func walkStdLib(f func(filename string) bool) { 43 | walk(filepath.Join(runtime.GOROOT(), "src"), f) 44 | } 45 | 46 | type histogram map[string]int 47 | 48 | func (h histogram) add(filename string) { 49 | f, err := parser.ParseFile(token.NewFileSet(), filename, nil, 0) 50 | if err != nil { 51 | panic(err) 52 | } 53 | 54 | ast.Inspect(f, func(n ast.Node) bool { 55 | if n, ok := n.(ast.Stmt); ok { 56 | h[fmt.Sprintf("%T", n)]++ 57 | } 58 | return true 59 | }) 60 | } 61 | 62 | // merge START OMIT 63 | func (h histogram) merge(h1 histogram) { 64 | for key, count := range h1 { 65 | h[key] = h[key] + count 66 | } 67 | } 68 | 69 | // merge END OMIT 70 | 71 | type entry struct { 72 | key string 73 | count int 74 | } 75 | 76 | func (h histogram) print() { 77 | var list []entry 78 | var total int 79 | for key, count := range h { 80 | list = append(list, entry{key, count}) 81 | total += count 82 | } 83 | sort.Sort(byCount(list)) 84 | 85 | percent := 100 / float64(total) 86 | for i, e := range list { 87 | fmt.Printf("%4d. %5.2f%% %5d %s\n", i, float64(e.count)*percent, e.count, e.key) 88 | } 89 | } 90 | 91 | type byCount []entry 92 | 93 | func (s byCount) Len() int { return len(s) } 94 | func (s byCount) Swap(i, j int) { s[i], s[j] = s[j], s[i] } 95 | func (s byCount) Less(i, j int) bool { 96 | x, y := s[i], s[j] 97 | if x.count != y.count { 98 | return x.count > y.count // want larger count first 99 | } 100 | return x.key < y.key 101 | } 102 | 103 | func init() { 104 | n := runtime.NumCPU() 105 | //fmt.Println(n, "cores") 106 | runtime.GOMAXPROCS(n) 107 | } 108 | 109 | // main START OMIT 110 | func main() { 111 | start := time.Now() 112 | ch := make(chan histogram) 113 | count := 0 // goroutine count 114 | walkStdLib(func(filename string) bool { 115 | count++ 116 | // mapper START OMIT 117 | go func() { 118 | h := make(histogram) 119 | h.add(filename) 120 | ch <- h 121 | }() 122 | // mapper END OMIT 123 | return true 124 | }) 125 | 126 | // reducer START OMIT 127 | h := make(histogram) 128 | for count > 0 { 129 | h.merge(<-ch) 130 | count-- 131 | } 132 | // reducer END OMIT 133 | 134 | h.print() 135 | fmt.Println(time.Since(start)) 136 | } 137 | 138 | // main END OMIT 139 | -------------------------------------------------------------------------------- /introduction-to-go/idents.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main // idents.go 4 | 5 | import ( 6 | "fmt" 7 | "os" 8 | "text/scanner" 9 | ) 10 | 11 | func main() { 12 | var s scanner.Scanner 13 | s.Init(os.Stdin) 14 | for { 15 | switch s.Scan() { 16 | case scanner.EOF: 17 | return // all done 18 | case scanner.Ident: 19 | fmt.Println(s.TokenText()) 20 | } 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /introduction-to-go/parallelogram.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START1 OMIT 6 | type Square struct{ length int } 7 | 8 | func (s *Square) Height() int { return s.length } 9 | func (s *Square) Width() int { return s.length } 10 | 11 | type Rectangle struct{ height, width int } 12 | 13 | func (r *Rectangle) Height() int { return r.height } 14 | func (r *Rectangle) Width() int { return r.width } 15 | 16 | // END1 OMIT 17 | 18 | // START2 OMIT 19 | type Parallelogram interface { 20 | Height() int 21 | Width() int 22 | } 23 | 24 | func Area(p Parallelogram) { 25 | fmt.Printf("Height: %d, Width: %d, Area: %d\n", 26 | p.Height(), p.Width(), p.Height()*p.Width()) 27 | } 28 | 29 | func main() { 30 | s := &Square{length: 10} 31 | r := &Rectangle{height: 4, width: 25} 32 | 33 | Area(s) 34 | Area(r) 35 | } 36 | 37 | // END2 OMIT 38 | -------------------------------------------------------------------------------- /introduction-to-go/pike-2009.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/introduction-to-go/pike-2009.png -------------------------------------------------------------------------------- /introduction-to-go/point.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import "fmt" 6 | 7 | // Point START OMIT 8 | type Point struct { 9 | x, y int 10 | } 11 | 12 | // Point END OMIT 13 | 14 | // String START OMIT 15 | func (p Point) String() string { 16 | return fmt.Sprintf("(%d, %d)", p.x, p.y) 17 | } 18 | 19 | // String END OMIT 20 | 21 | // main START OMIT 22 | func main() { 23 | p := Point{2, 3} 24 | fmt.Println(p.String()) 25 | fmt.Println(Point{3, 5}.String()) 26 | } 27 | 28 | // main END OMIT 29 | -------------------------------------------------------------------------------- /introduction-to-go/politics.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | // START OMIT 7 | q := "Tony Abbott" 8 | p := &q 9 | 10 | *p = "Kevin Rudd" 11 | fmt.Println(q) 12 | // END OMIT 13 | } 14 | -------------------------------------------------------------------------------- /introduction-to-go/precision.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // START OMIT 4 | const Ln2 = 0.693147180559945309417232121458176568075500134360255254120680009 5 | const Log2E = 1 / Ln2 // precise reciprocal 6 | // END OMIT 7 | -------------------------------------------------------------------------------- /introduction-to-go/recv.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "time" 4 | import "fmt" 5 | 6 | func main() { 7 | // START OMIT 8 | var work []string 9 | select { 10 | case w := <-incoming: 11 | work = append(work, w) 12 | default: 13 | // will fire if incoming is not ready 14 | } 15 | // END OMIT 16 | } 17 | -------------------------------------------------------------------------------- /introduction-to-go/select.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "time" 4 | import "fmt" 5 | 6 | // START OMIT 7 | func SendWithTimeout(data string, to chan int, timeout time.Duration) error { 8 | select { 9 | case to <- data: 10 | return nil // everything worked 11 | case <-time.After(timeout): 12 | return fmt.Errorf("timeout after %s", timeout) 13 | } 14 | } 15 | 16 | // END OMIT 17 | func main() { 18 | } 19 | -------------------------------------------------------------------------------- /introduction-to-go/send-recv.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func greeting(c chan string) { 7 | words := []string{"Welcome", "to", "Go"} 8 | for _, word := range words { 9 | c <- word 10 | } 11 | close(c) 12 | } 13 | 14 | func main() { 15 | c := make(chan string) 16 | go greeting(c) 17 | 18 | for word := range c { 19 | fmt.Println(word) 20 | } 21 | } 22 | 23 | // END OMIT 24 | -------------------------------------------------------------------------------- /introduction-to-go/server.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "net" 4 | 5 | // START OMIT 6 | func server(listener net.Listener) { 7 | for { 8 | client, _ := listener.Accept() 9 | go handle(client) 10 | } 11 | } 12 | 13 | func handle(client net.Conn) { 14 | // handle connection 15 | } 16 | 17 | func main() { 18 | listener, _ := net.Listen("tcp", ":2000") 19 | go server(listener) 20 | 21 | // do some other stuff 22 | } 23 | 24 | // END OMIT 25 | -------------------------------------------------------------------------------- /introduction-to-go/sort.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "sort" 8 | ) 9 | 10 | type Weekday int 11 | 12 | const ( 13 | Mon Weekday = iota 14 | Tue 15 | Wed 16 | Thu 17 | Fri 18 | Sat 19 | Sun 20 | ) 21 | 22 | var names = [...]string{"Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"} 23 | 24 | func (d Weekday) String() string { // ... 25 | return names[d] 26 | } 27 | 28 | // lexical START OMIT 29 | type lexical []string 30 | 31 | func (a lexical) Len() int { return len(a) } 32 | func (a lexical) Swap(i, j int) { a[i], a[j] = a[j], a[i] } 33 | func (a lexical) Less(i, j int) bool { return a[i] < a[j] } 34 | 35 | // lexical END OMIT 36 | 37 | func main() { 38 | var list []string 39 | for d := Mon; d <= Sun; d++ { 40 | list = append(list, d.String()) 41 | } 42 | 43 | sort.Sort(lexical(list)) 44 | 45 | for i, x := range list { 46 | fmt.Println(i, x) 47 | } 48 | } 49 | -------------------------------------------------------------------------------- /introduction-to-go/stringer.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import "fmt" 6 | 7 | type Point struct { 8 | x, y int 9 | } 10 | 11 | func (p Point) String() string { 12 | return fmt.Sprintf("(%d, %d)", p.x, p.y) 13 | } 14 | 15 | type Weekday int 16 | 17 | const ( 18 | Mon Weekday = iota 19 | Tue 20 | Wed 21 | Thu 22 | Fri 23 | Sat 24 | Sun 25 | ) 26 | 27 | var names = [...]string{"Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"} 28 | 29 | func (d Weekday) String() string { // ... 30 | return names[d] 31 | } 32 | 33 | // Stringer START OMIT 34 | type Stringer interface { 35 | String() string 36 | } 37 | 38 | // Stringer END OMIT 39 | 40 | // main START OMIT 41 | func main() { 42 | var x Stringer 43 | x = Point{2, 3} 44 | fmt.Println("A", x.String()) 45 | 46 | x = Tue 47 | fmt.Println("B", x.String()) 48 | 49 | fmt.Println("C", Point{2, 3}) // fmt.Println knows about Stringer! 50 | fmt.Println("D", Tue) 51 | } 52 | 53 | // main END OMIT 54 | -------------------------------------------------------------------------------- /introduction-to-go/walk.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import ( 6 | "fmt" 7 | "io/ioutil" 8 | "path/filepath" 9 | "runtime" 10 | "strings" 11 | ) 12 | 13 | func walk(dir string, f func(string) bool) bool { 14 | fis, err := ioutil.ReadDir(dir) 15 | if err != nil { 16 | panic(err) 17 | } 18 | // parse all *.go files in directory; 19 | // traverse subdirectories, but don't walk into testdata 20 | for _, fi := range fis { 21 | path := filepath.Join(dir, fi.Name()) 22 | if fi.IsDir() { 23 | if fi.Name() != "testdata" { 24 | if !walk(path, f) { 25 | return false 26 | } 27 | } 28 | } else if strings.HasSuffix(fi.Name(), ".go") && !strings.HasPrefix(fi.Name(), ".") { 29 | if !f(path) { 30 | return false 31 | } 32 | } 33 | } 34 | return true 35 | } 36 | 37 | func walkStdLib(f func(filename string) bool) { 38 | walk(filepath.Join(runtime.GOROOT(), "src"), f) 39 | } 40 | 41 | func _() { 42 | // example START OMIT 43 | n := 0 44 | println := func(s string) bool { 45 | fmt.Println(n, s) 46 | n++ 47 | return n < 10 48 | } 49 | walkStdLib(println) 50 | // example END OMIT 51 | } 52 | 53 | func main() { 54 | // main START OMIT 55 | n := 0 56 | walkStdLib(func(s string) bool { 57 | fmt.Println(n, s) 58 | n++ 59 | return n < 10 60 | }) 61 | // main END OMIT 62 | } 63 | -------------------------------------------------------------------------------- /introduction-to-go/weekday.go: -------------------------------------------------------------------------------- 1 | // +build OMIT 2 | 3 | package main 4 | 5 | import "fmt" 6 | 7 | // type START OMIT 8 | type Weekday int 9 | 10 | // type END OMIT 11 | 12 | const ( 13 | Mon Weekday = iota 14 | Tue 15 | Wed 16 | Thu 17 | Fri 18 | Sat 19 | Sun 20 | ) 21 | 22 | var names = [...]string{"Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"} 23 | 24 | // String START OMIT 25 | func (d Weekday) String() string { // ... 26 | // String END OMIT 27 | return names[d] 28 | } 29 | 30 | // main START OMIT 31 | func main() { 32 | fmt.Println(Mon.String()) 33 | fmt.Println() 34 | 35 | for d := Mon; d <= Sun; d++ { 36 | fmt.Println(d.String()) 37 | } 38 | } 39 | 40 | // main END OMIT 41 | -------------------------------------------------------------------------------- /percentv/go.mod: -------------------------------------------------------------------------------- 1 | module percentv 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /percentv/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | // tag::main[] 8 | func main() { 9 | type T struct { 10 | I int 11 | } 12 | x := []*T{{1}, {2}, {3}} 13 | y := []*T{{1}, {2}, {4}} 14 | 15 | fmt.Printf("%v %v\n", x, y) 16 | fmt.Printf("%#v %#v\n", x, y) 17 | } 18 | 19 | // end::main[] 20 | -------------------------------------------------------------------------------- /performance-without-the-event-loop/3357832896_896d98bbaf_z.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/3357832896_896d98bbaf_z.jpg -------------------------------------------------------------------------------- /performance-without-the-event-loop/640px-Table_of_x86_Registers_svg.svg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/640px-Table_of_x86_Registers_svg.svg.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/CPU.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/CPU.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/Ivy-Bridge_Die_Flat-HR.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/Ivy-Bridge_Die_Flat-HR.jpg -------------------------------------------------------------------------------- /performance-without-the-event-loop/e450.jpg: -------------------------------------------------------------------------------- 1 | 2 | 3 | -------------------------------------------------------------------------------- /performance-without-the-event-loop/echo.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "io" 5 | "net" 6 | "log" 7 | ) 8 | 9 | // START OMIT 10 | func echo(rw io.ReadWriteCloser) { 11 | defer rw.Close() 12 | io.Copy(rw, rw) 13 | } 14 | 15 | func main() { 16 | l, err := net.Listen("tcp", ":8000") 17 | if err != nil { 18 | log.Fatalf("could not listen: %v", err) 19 | } 20 | defer l.Close() 21 | 22 | for { 23 | c, err := l.Accept() 24 | if err != nil { 25 | log.Fatalf("could not accept connection: %v", err) 26 | } 27 | go echo(c) 28 | } 29 | } 30 | // END OMIT 31 | -------------------------------------------------------------------------------- /performance-without-the-event-loop/grep.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bufio" 5 | "fmt" 6 | "io" 7 | "os" 8 | "strings" 9 | ) 10 | 11 | func grep(r io.Reader, needle string) { 12 | br := bufio.NewReader(r) 13 | lines := make(chan string, 20) 14 | 15 | go func() { 16 | defer close(lines) 17 | for { 18 | line, err := br.ReadString('\n') 19 | if err != nil { 20 | return 21 | } 22 | lines <- line 23 | } 24 | }() 25 | 26 | for line := range lines { 27 | if strings.Contains(line, needle) { 28 | fmt.Println(line) 29 | } 30 | } 31 | } 32 | 33 | func main() { 34 | grep(nil, "") 35 | } 36 | -------------------------------------------------------------------------------- /performance-without-the-event-loop/guard-page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/guard-page.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/microservices.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/microservices.jpg -------------------------------------------------------------------------------- /performance-without-the-event-loop/pager.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/pager.jpg -------------------------------------------------------------------------------- /performance-without-the-event-loop/process.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/process.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/read.go: -------------------------------------------------------------------------------- 1 | func (fd *netFD) Read(p []byte) (n int, err error) { 2 | // preamble 3 | for { 4 | n, err = syscall.Read(fd.sysfd, p) // HL 5 | if err != nil { 6 | n = 0 7 | if err == syscall.EAGAIN { 8 | if err = fd.pd.WaitRead(); err == nil { // HL 9 | continue 10 | } 11 | } 12 | } 13 | err = fd.eofError(n, err) 14 | break 15 | } 16 | // epilog 17 | return 18 | } 19 | -------------------------------------------------------------------------------- /performance-without-the-event-loop/stack-growth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/stack-growth.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/sun-ultra-enterprise-450-400mhz-2gb-20-bay-workgroup-server-system-no-hdd-parts_131514071457.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/sun-ultra-enterprise-450-400mhz-2gb-20-bay-workgroup-server-system-no-hdd-parts_131514071457.jpg -------------------------------------------------------------------------------- /performance-without-the-event-loop/threads.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/threads.png -------------------------------------------------------------------------------- /performance-without-the-event-loop/turingFull560.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/performance-without-the-event-loop/turingFull560.jpg -------------------------------------------------------------------------------- /practical-go.adoc: -------------------------------------------------------------------------------- 1 | = Practical Go 2 | Dave Cheney 3 | London Go meetup 4 | 5 | == Introduction 6 | 7 | 8 | We have the Go proverbs, but they are truisms, not guidelines. 9 | 10 | I explored some of these ideas a few years ago at Golang UK 11 | 12 | 13 | 14 | == Conclusion 15 | 16 | 17 | [preface] 18 | == Preface 19 | // The goal of the introduction is to outline the philosophy of the language, and thus this the book. 20 | // Each chapter in turn explains how the topic relates to the philosophy outlined in the introduction 21 | 22 | // Write in the first person, as if one were giving a lecture. 23 | 24 | The sub title of this book is _N suggestions for writing better Go programs_, but that's a little subjective, isn't it? 25 | Who's to say which is better 26 | 27 | This book is not an introduction to the Go programming language. 28 | To steal a quote the author Scott Meyers, his Effective C++ books explain "how to use the language, not what or why the language is", and this too is the goal of _Practical Go_. 29 | This book is a series of a suggestions for writing programs that best capture the philosophy of the Go programming language; readabilty, simplicity, and productivity. 30 | 31 | === What is Practical Go? 32 | 33 | Practical Go is code that is easy to change. 34 | [quote, Sandi Metz] 35 | Design is the art of arranging code to work today, and be changable forever. 36 | 37 | // need attribution for the quote, rubyconf 2013 or 2009? 38 | 39 | If there was a quote that summarises the ethos of writing Pratical Go code it would be this quote from Sandi Metz. 40 | 41 | There is nothing wrong with writing software for fun, or writing a program to solve a problem that once completed renders the program obsolete. 42 | However, for the majority of industrial programming performed today, a program once developed must be deployed, debugged, maintained, extended, and eventually replaced. 43 | These phases define the _software development lifecycle_. 44 | 45 | [quote, Titus Winters, C++Con 2017] 46 | Software engineering is programming intergrated over time. 47 | 48 | This view of writing software differentiates between programming, the exercise at a point in time, and software engineering, the application of engineering principals to the design of software. 49 | Titus' presentation at C\+\+Con in 2017 is one of the purest expressions of the Google software design philosophy from which Go's design philiosphy is a direct descendant. 50 | 51 | === Clarity 52 | 53 | [quote, Titus Winters] 54 | Code is read many more times than it is written. 55 | 56 | The quote above, again from Winters, is by no means exclusive to Google or to C++ programers. 57 | Readability is key to maintainable software, and maintainable software is software that is easy to change. 58 | Therefore if software is to be engineered to be maintainable over a period of time from months to decades, it must be understandable by all who will consult and modify its code. 59 | 60 | // proverb: Clear is better than clever. 61 | [quote, Hal Abelson and Gerald Sussman, Structure and Interpretation of Computer Programs] 62 | Programs must be written for people to read, and only incidentally for machines to execute. 63 | 64 | // explain what the word means in the abstract 65 | The source code of a program is an intermediary form, somewhere between the author's concept and the computer's executable notation. 66 | While the compiler focuses on reducing the source of a program to atoms which have a direct analogue to machine code instructions, readers of your source code must operate at the level of the source as presented. 67 | In that sense source code is an encoded form of an idea, a process, an algorythm, or a formula. 68 | Just like musical notation. 69 | 70 | // explain why it is important to programmers 71 | I doubt anyone would disagree with readability as a core goal of Practical Go. 72 | Go places readability above all other considerations, because source code is written to be read. 73 | However, readability is fundamentaly subjective. 74 | How does one prove that a piece of code is readable? 75 | Many of the choices relating to Go's syntax speak to the needs of the reader. 76 | For example, Go's syntax is sparten because of its drive for readability. 77 | 78 | // http://www.gigamonkeys.com/code-reading/ 79 | [quote, Peter Seibel] 80 | We don’t read code, we decode it. We examine it. A piece of code is not literature; it is a specimen. 81 | 82 | // explain how Go enforces/encourages this property 83 | An excellent example of Go's commitment to readability is +go fmt+, Go's built in source code formatting tool. 84 | But what is it that is so important about +go fmt+, and why is it important to +go fmt+ your source code? 85 | Part of the reason is, to avoid needless debate. 86 | Source code formatting is the most pernicious of these, yet least, important issue, providing that everyone agrees on a single format. 87 | It’s not enough that the code is well formatted according to local custom, but there is precisely one way Go code should be formatted. 88 | The outcome is that nearly all Go code is +go fmt+'ed _by convention_, and adherence to this convention is an indicator of alignment with the values of the language. 89 | 90 | [quote, C. A. R. Hoare, Hints on Programming Language Design 1973] 91 | The readability of programs is immeasurably more important than their writeability. 92 | 93 | //* Familiarity -- Through standardisation, simple convention. 94 | But readability goes deeper than just formatting. 95 | As Go programmers we can pick up a piece of Go code written by anyone in the world and start to read it. 96 | A perchant for concise naming, small interfaces, use of the built in map and slice types, the mantra of composition over inheritance, a simple methodology for declaring what is public or private in your package, all contribute to a convention which provides a sense of familiarity. 97 | All Go code uses these same basic building blocks, so all Go code is accessible to a reader who is versed in the language, not some organization specific dialect. 98 | 99 | .Explicit is better than implicit 100 | Thus, well written Go programs are highly readable. 101 | 102 | https://twitter.com/CodeWisdom/status/851495746402570240?s=09 103 | 104 | “It was an explicit design decision (for readability) that Go does not have ternary expressions. 105 | 106 | Sometimes Go makes you write a few more lines, but we accept that cost for explicitness and legibility.” Bradfitz - issue NNN 107 | 108 | “It definitely reduces typing, but that's not a virtue, as code is read vastly more often than it is written. You gotta optimize for comprehension “ -- me ? 109 | 110 | “I think the point of Go is less about performance and more about providing a simple language that speeds up software development (not necessarily software itself) and allows developers the power of a compiled, GC, and type-safe language without having to deal with the gotchas of languages like C” -- Rob / Ian ? 111 | 112 | [quote, Sameer Ajmani] 113 | Not specifically wrapping, but I have a strong preference to use struct literals with named fields, one per line. 114 | 115 | // https://twitter.com/Sajma/status/875905155887226881?s=09 116 | 117 | http://www.squirrel.nl/pub/PBP_refguide-1.02.00.pdf 118 | 119 | - just because something was hard to write, that does not mean it should be hard to read 120 | https://twitter.com/CodeWisdom/status/887049034069921795?s=09 121 | https://twitter.com/grepory/status/886700568600694784?s=09 122 | 123 | === Simplicity 124 | indexterm:[Simplicity] 125 | 126 | // http://zoo.cs.yale.edu/classes/cs422/2011/bib/hoare81emperor.pdf 127 | 128 | [quote, C. A. R. Hoare, 1980 Turing Award Lecture] 129 | There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. 130 | 131 | // explain what the word means in the abstract 132 | Simplicity is not a synonym for easy, nor is achieving a design which is simple an easy task. 133 | Putting this into the context of programming languages, something which is simple may take a little longer, it may be a little more verbose, but it will be more comprehensible, more estensible, more adaptable, and exibit lower coupling. 134 | 135 | [quote, Edsger W.Dijkstra] 136 | We know we have to keep it crisp, disentangled, and simple if we refuse to be crushed by the complexities of our own making. 137 | // https://www.cs.utexas.edu/users/EWD/transcriptions/EWD12xx/EWD1243a.html 138 | 139 | [quote, Dennis Ritchie] 140 | ____ 141 | A language that doesn’t have everything is actually easier to program in than some that do. 142 | ____ 143 | 144 | [quote, unknown] 145 | That the Go people are perfectly prepared to resort to brute force instead of (excessive) cleverness and thus that when you read the code in, eg, the standard packages you get inspired to do the same. 146 | 147 | // http://www.azquotes.com/quote/596303 148 | 149 | // explain why it is important to programmers 150 | Good programmers write simple programs, not simplistic ones. 151 | They bring their knowledge, their experience, and their failures to new designs, to learn from and avoid mistakes in the future. 152 | You should design your programs with simplicity as a goal, not aim to be pleasantly surprised when your solution happens to be simple. 153 | 154 | Every language introduced in my life time that purports to be simple. 155 | Each new language offers as a justification, and an enticement, their inherent simplicity. 156 | On the other hand, I cannot point to a language introduced in the same time frame with the rallying call of complexity--_more complexity than it's contemporaries_--but many claim instead to be _powerful_. 157 | The idea of proposing a new language which designed to offer inherently higher levels of complexity is clearly laughable, yet this is exactly what so many contemporary languages evolve to become; complicated, baroque, messes. 158 | A parody of the languages they sought to replace. 159 | 160 | So, every language starts out with simplicity as a goal, yet many of them fail to achieve this goal. 161 | Eventually falling back on notions of expressiveness or power of the language as justification for a failure to remain simple. 162 | 163 | One major reason I believe is to be thought successful a language should somehow include the complete set of popular features from its predecessors. 164 | If you would listen to Go's critics, they demand that new programming languages should push forward the boundaries of type theory and computer science. 165 | Thus, clumsy syntax and non-orthogonality is justified by the difficulty of capturing nuanced corner cases of the language, many of them self inflicted by years of careless feature creep. 166 | In reality, this is a veiled request that any new language include all the bits they felt were important in their favourite old language, while still holding true to the promise of whatever it was that drew them to investigate your language in the first place. 167 | 168 | // explain how Go enforces/encourages this property 169 | Simplicity and readablity are closely related, but simplicity goes far deeper than source code on the page. 170 | Go is a language that chooses to be simple, and it does so by deliberately not including many features that other programming languages have accustomed their users to believing are essential. 171 | 172 | Go is a language designed to be simple. 173 | This was the message that spoke to me when I first learned about the language in 2009, and is the message that has stayed with me to this day. 174 | The desire for simplicity is woven through every aspect of the language, a fact which Donovan and Kernighan described as "Go's campaign of radical simplicity"<>>. 175 | Go's focus on simplicity is a feature, not an accident. 176 | Thus, a Practical Go program is inherently simple. 177 | 178 | === Productivity 179 | 180 | // practical go code is rooted in sustainable engineering practices 181 | 182 | The third axiom of Go is productivity. 183 | Go is designed to be used by teams of programmers, even if you may not know each other personally. 184 | Small annoyances such as a lack of warnings, a refusal to allow unused imports, or unused local variables, are all facets of choices designed to help Go work well for large teams. 185 | This does not mean that Go is not suitable for the single developer working alone, or a small program written for a specific need, but speaks to the fact that a number of the choices within the language are aimed at the needs of growing software teams. 186 | 187 | - clean code 188 | - SOLID 189 | - TDD 190 | - anti fragile 191 | 192 | decoupled code, srp 193 | 194 | There is more to the success of Go than just being simple, and this is the realization that for a programming language to be successful, it must coexist inside a larger environment. 195 | Large programs are written by large teams. 196 | I don’t believe this is a controversial statement. 197 | 198 | The inverse is also true. 199 | Large teams of programmers, by their nature, produce large code bases. 200 | Projects with large goals will necessitate large teams, and thus their output will be commensurate. 201 | This is the nature of our work. 202 | 203 | In his 2017 Gophercon keynote footnote:[https://blog.golang.org/toward-go2] Russ Cox used the word _scale_ to denote one of Go's explicit design goals 204 | 205 | [quote, Russ Cox, GopherCon 2017 keynote address] 206 | ____ 207 | The goals we have for Go today are the same as in 2007. We want to make programmers more effective at managing two kinds of scale: production scale, especially concurrent systems interacting with many other servers, exemplified today by cloud software; and development scale, especially large codebases worked on by many engineers coordinating only loosely, exemplified today by modern open-source development. 208 | ____ 209 | 210 | These statements echo earlier ones by Rob Pike 211 | 212 | [quote, Rob Pike, https://talks.golang.org/2012/splash.article] 213 | ____ 214 | The goals of the Go project were to eliminate the slowness and clumsiness of software development at Google, and thereby to make the process more productive and scalable. 215 | The language was designed by and for people who write—and read and debug and maintain—large software systems. 216 | ____ 217 | 218 | Three tenets which are presented in increasing order of subjectivity 219 | 220 | - Simple does not mean easy, we know that, but it also does not mean unsphisticated or crude 221 | -------------------------------------------------------------------------------- /reproducible-builds-ii.slide: -------------------------------------------------------------------------------- 1 | Reproducible Builds (part ii) 2 | Sydney Go Users' Group 3 | 27 May 2015 4 | 5 | Dave Cheney 6 | dave@cheney.net 7 | http://dave.cheney.net/ 8 | @davecheney 9 | 10 | * Warning: this presentation contains nuts 11 | 12 | .html reproducible-builds-ii/video.html 13 | 14 | Dependency management in Go is the single most common question I have been asked consistently for several years now. 15 | 16 | Dependency management is the equivilent of Python's GIL. 17 | 18 | What I see is many people who are actively choosing to stay "standard library only", and I worry that more are silently sitting on the fence until a solution is found. 19 | 20 | * Agenda 21 | 22 | - Problem statement 23 | - Competitive analysis 24 | - Proposal 25 | 26 | * Problem statement 27 | 28 | * Repeatable builds 29 | 30 | I have a requirement that at any time I can fetch the entire graph of source that went into a program, feed that to a compiler and produce a program that is identical to one created in the past. 31 | 32 | This is the requirement I have, and this is the motivation for this talk. If you don't have this requirement, that's fine. 33 | 34 | The plethora of tools that exist in this space shows that Go programmers have multiple, sometimes overlapping requirements. 35 | 36 | This is my solution for my requirements; it is my _hope_ that I can convince you of it's utility to you, but again, if you don't share my requirements, I may not be successful in my arguments. 37 | 38 | * Why I don't have a reproducible build today ? 39 | 40 | OK, so now I've told you what I want; I need to explain to you why I don't feel that I have it today. 41 | 42 | import "github.com/pkg/sftp" # yes, but which revision! 43 | 44 | The most obvious reason is the import statement inside a Go package does not provide enough information for `go`get` to select from a set of revisions available in a remote code repository the specific revision to fetch. 45 | 46 | That information simply isn't there. 47 | 48 | * Interlude 49 | 50 | # env GOPATH=/tmp go get -v github.com/pkg/sftp 51 | # env GOPATH=/tmp go get -v github.com/spf13/hugo 52 | # env GOPATH=/tmp go get -v github.com/juju/juju/... 53 | # env GOPATH=/tmp go get -v launchpad.net/godeps 54 | # cd /tmp/src/github.com/juju/juju 55 | # env GOPATH /tmp/bin/godeps -u dependencies.tsv 56 | 57 | * A brief digression 58 | 59 | * Naming things (part 1) 60 | 61 | There are two rules for successful dependency management in Go. 62 | 63 | Rule 1: Things that are different _must_ have different import paths. 64 | 65 | Who has written a log or logger package, they might all be called "log", but they are not the same package. 66 | 67 | This why we have namespaces. 68 | 69 | github.com/you/log 70 | github.com/me/log 71 | 72 | * Naming things (part 2) 73 | 74 | Rule 2: Things that are the same _must_ have the same import path. 75 | 76 | Are these two packages the same, or are they different ? 77 | 78 | github.com/lib/pq 79 | github.com/davecheney/foo/internal/github.com/lib/pq 80 | 81 | They are the same, this is the same code -- this is obvious to a human, not a computer. 82 | 83 | To a compiler these are different packages; this has serious implications. 84 | 85 | - Type assertions and equality are broken. 86 | - `init()` functions will run multiple times. [[http://godoc.org/database/sql#Register][database/sql.Register]] 87 | 88 | * Versions in the URL 89 | 90 | We cannot embed anything in the import syntax 91 | 92 | import "github.com/project/v7/library" 93 | 94 | - Popular if you want to provide multiple versions of your API at the same time. 95 | - Not accurate enough to checkout a specific revision. 96 | - Not reproducible to ensure everyone has the _same_ revision. 97 | - Every import statement in every file in the package *must* be identical, even using build tags, even using conditional compilation. 98 | - Breaks the rule of naming things, two things which are the same, must have the same import path. 99 | 100 | * The import statement cannot be changed 101 | 102 | This information will not be added to the import syntax for two reasons 103 | 104 | import "github.com/pkg/term" "{hash,tag,version}" 105 | 106 | - Imports are opaque to the language, so some external tool dictating the format of the import declaration to the compiler is not appropriate. 107 | - More importantly, this would be a backward incompatible syntax change. 108 | 109 | * Do Go packages even have versions ? 110 | 111 | Proposition: Go packages do not have versions 112 | 113 | Go packages do not have versions, because there is no widely accepted method of _releasing_ a Go package. 114 | 115 | Go packages are usually tracked in source control, and source control systems give copies of source code in time revision numbers or hashes. 116 | 117 | _Revision_!=_Version_ 118 | 119 | * Competitive Analysis 120 | 121 | * Dude, be a good Gopher, don't break users 122 | 123 | So the first, and longest standing solution to this problem is to always have a stable API. 124 | 125 | - Proposed solution from the Go team for several years. 126 | - Admirable attempt to extend the Go 1 contract to all Go code. 127 | 128 | If it worked, we wouldn't be having this conversation today 129 | 130 | - You _want_ to change the API for your package 131 | - Often the API and the consumer evolve in parallel 132 | - Even putting an identifier in the import path only guarantees I have _a_ version of that package, not _the_ version. 133 | 134 | * I live in the real world 135 | 136 | If my time in system administration taught me anything, it's the unexpected failures that get you. You can plan for the big disasters, but it turns out that the little disasters can be just as disruptive. 137 | 138 | - code.google.com closing down. Will Github still be around in 10 years ? 139 | - codehaus :( 140 | - companies merging, people getting married, someone dies, trademark dispute, etc. 141 | - FoundationDB :( 142 | 143 | These are all little disasters, you can usually find the code again, maybe it's just a quick sed rewrite and you're back again. 144 | 145 | But just like the big disasters, these little disasters are indistinguishable, code which built one day, won't build the next. 146 | 147 | * Don't be this person 148 | 149 | .image reproducible-builds/github.png 150 | 151 | The moral of the story is, if you are responsible for delivering a product written in Go, you need to be responsible for all the source that goes into that product. 152 | 153 | * Tools which manage $GOPATH 154 | 155 | Tools which fixup `$GOPATH` after `go`get` 156 | 157 | - godeps (plural, canonical) 158 | - glock 159 | - gvp 160 | 161 | Problems 162 | 163 | - not reproducible, the upstream can still disappear 164 | - must adjust your `$GOPATH` manually when moving between projects 165 | - near universal dislike for a .lock file in the package 166 | - universal disagreement on the format and layout of a .lock file 167 | 168 | * Tools which vendor packages 169 | 170 | Copying, vendoring, rewriting the source, is the _new_ position from the Go team. 171 | 172 | - godep 173 | 174 | Problems 175 | 176 | - requires source rewriting; many uncomfortable with this 177 | - possibly breaks the naming rules; the same package can exist in the dependency graph under multiple names 178 | - concerns about losing track of the upstream 179 | - ugly long import lines 180 | 181 | * Tools which give you one $GOPATH per project 182 | 183 | Virtual env all the things! 184 | 185 | - gpm https://github.com/pote/gpm 186 | - gvm 187 | - govm 188 | - glide 189 | - /usr/bin/direnv (old skool) 190 | 191 | Problems 192 | 193 | - not isolated from upstream going away 194 | - hard to use, terminal or shell session becomes magic 195 | 196 | * Proposal 197 | 198 | * Stop working around go get 199 | 200 | .image reproducible-builds/goget.jpg _ 400 201 | 202 | Every one of the existing solutions is hamstrung by the fact it is working around the limitations of the `go` tool. 203 | 204 | Stop using `go`get`. Don't use the `go` tool at all. 205 | 206 | * Requirements 207 | 208 | So, we're talking about writing a new build tool for Go, not a wrapper around an existing tool. 209 | 210 | * Project based 211 | 212 | A new build tool should be project based. 213 | 214 | - A project is where main packages live. 215 | - A project is effectively a single $GOPATH. 216 | - Automatic detection, you don't need 'enter' a project. 217 | - The owner of the project decides on the version of a particular import path in use. 218 | - Never more than one copy of a single import per project. 219 | 220 | * No configuration files 221 | 222 | This one I find hard to accept, but Go developers do not want to have any sort of configuration file to build their code. 223 | 224 | I find this hard to rationalize because most repos that I look have had dozens of turds in them, Gruntfiles, Dockerfiles, Wercker configs, etc. 225 | 226 | - Projects are detected from the path, anything that has a `src` in the path is a project. 227 | - Works well in practice, and is backwards compatible with `$GOPATH`. 228 | 229 | * Respect the canonical import path 230 | 231 | package pdf // import "rsc.io/pdf" 232 | 233 | - rsc added this, it's clear what he thinks about the possibility of duplicates in the binary 234 | - Import rewriting has to rewrite the import comment as well, and that sounds like deliberately disabling the safety interlock on a handgun. 235 | 236 | * Leaves source untouched 237 | 238 | - Check the whole source in 239 | - Use submodules or subtrees, svn externals, etc, if you prefer 240 | - Don't touch the source, then you stand a chance of hashing it / diffing it / signing it 241 | 242 | * Introducing gb 243 | 244 | .image reproducible-builds-ii/gb.svg _ 350 245 | 246 | [[http://getgb.io/][http://getgb.io/]] 247 | 248 | - Project based, project is automatically detected 249 | - Dependencies are a property of the project, one copy of any package per project 250 | - Supports multiple working copies concurrently 251 | - Supports vendoring without rewriting via separate vendor/ directory 252 | - Supports plugins (git style) 253 | 254 | * Demo time 255 | 256 | # * This only works for projects, what about packages ? 257 | 258 | # Yes, this solution works for projects, it encourages you to build larger projects. 259 | 260 | # Personally, I think this is what Go needs at the moment. 261 | 262 | # - Projects should think hard about each dependency they bring in -- they aren't free 263 | 264 | # - Packages developed and thrown out there, hoping that someone else will use and popularise them. As Peter Bourgon noted at FOSDEM this year, does Go really need another http mux ? 265 | 266 | # What I want to see is large projects being built with Go, then, when they are proven, lets circle back and peal off parts of those projects that are reusable. But that happens second, you can't have a stable complete package without an anchor tenant, and it makes sense to me that those libraries should be incubated inside their host, not along side them -- it worked for django, it worked for rails, I think it's a message that should be studied. 267 | 268 | * Take aways 269 | 270 | The problem is `go`get`, not the import statement. 271 | 272 | The `go` tool doesn't define the language, we can build a replacement. 273 | 274 | * Try it out 275 | 276 | [[http://getgb.io/][http://getgb.io/]] 277 | 278 | - 100 % compatible with existing Go source. 279 | - Don't even need to change `$GOPATH`. 280 | - Upgrade to a gb project if you like. 281 | - Reusable library for building Go packages, no more shelling out to `go`build` 282 | - Write a plugin, please write a plugin. 283 | 284 | * Stop. Question time. 285 | -------------------------------------------------------------------------------- /reproducible-builds-ii/gb.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 6 | 7 | 11 | 12 | 17 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /reproducible-builds-ii/video.html: -------------------------------------------------------------------------------- 1 |
2 | -------------------------------------------------------------------------------- /reproducible-builds.slide: -------------------------------------------------------------------------------- 1 | Reproducible Builds 2 | GDG Berlin Golang 3 | 20 Apr 2015 4 | 5 | Dave Cheney 6 | dave@cheney.net 7 | http://dave.cheney.net/ 8 | @davecheney 9 | 10 | * Warning: this presentation contains nuts 11 | 12 | I feel a certain degree of trepidation on stage today. Not just because the size of the audience I am addressing, but because of the subject I will be discussing. 13 | 14 | Dependency management in Go is the single most common question I have been asked consistently for several years now. 15 | 16 | Dependency management is the equivilent of Python's GIL, a problem that everyone has, but one that has not been solved. 17 | 18 | What I see is many people who are actively choosing to stay "standard library only", and I worry that more are silently sitting on the fence until a solution is found. 19 | 20 | * Agenda 21 | 22 | - Problem statement 23 | - Competitive analysis 24 | - Proposal 25 | 26 | * Problem statement 27 | 28 | * Repeatable builds 29 | 30 | I have a requirement that at any time I can fetch the entire graph of source that went into a program, feed that to a compiler and produce a program that is identical to one created in the past. 31 | 32 | This is the requirement I have, and this is the motivation for this talk. If you don't have this requirement, that's fine. 33 | 34 | The plethora of tools that exist in this space shows that Go programmers have multiple, sometimes overlapping requirements. Again, that is fine, this is my solution for my requirements; it is my _hope_ that I can convince you of it's utility to you, but again, if you don't share my requirements, I may not be successful in my arguments. 35 | 36 | Out of scope 37 | 38 | - compiler doesn't produce byte for byte comparable binaries 39 | - archiving compiler tool chain versions 40 | 41 | * Why I don't have a reliable builds today 42 | 43 | OK, so now I've told you what I want; I need to explain to you why I don't feel that I have it today. 44 | 45 | import "github.com/pkg/sftp" # yes, but which revision! 46 | 47 | The most obvious reason is the import statement inside a Go package does not provide enough information for `go`get` to select from a set of revisions available in a remote code repository the specific revision to fetch. 48 | 49 | That information simply isn't there. 50 | 51 | * Naming things (part 1) 52 | 53 | There are two rules for successful dependency management in Go. 54 | 55 | Rule 1: Things that are different _must_ have different import paths. 56 | 57 | Who has written a log or logger package, they might all be called "log", but they are not the same package. 58 | 59 | This why we have namespaces, `github.com/you/log`, `github.com/me/log`. 60 | 61 | * Naming things (part 2) 62 | 63 | Rule 2: Things that are the same _must_ have the same import path. 64 | 65 | Are these two packages the same, or are they different ? 66 | 67 | github.com/lib/pq 68 | github.com/davecheney/foo/internal/github.com/lib/pq 69 | 70 | They are the same, this is the same code -- this is obvious to a human, not a computer. 71 | 72 | To a compiler these are different packages. 73 | 74 | - Type assertions and equality are broken. 75 | - `init()` functions will run multiple times. [[http://godoc.org/database/sql#Register][database/sql.Register]] 76 | 77 | * The import statement cannot be changed 78 | 79 | We cannot add anything to the import syntax for two reasons 80 | 81 | import "github.com/pkg/term" "{hash,tag,version}" 82 | 83 | - Imports are opaque to the language, so some external tool dictating the format of the import declaration to the compiler is not appropriate. 84 | - More importantly, this would be a backward incompatible syntax change. 85 | 86 | * Versions in the URL (part 1) 87 | 88 | We cannot embed anything in the import syntax 89 | 90 | import "github.com/project/v7/library" 91 | 92 | - Popular if you want to provide multiple versions of your API at the same time. 93 | - Not accurate enough to checkout a specific revision. 94 | - Not reproducible to ensure everyone has the _same_ revision. 95 | - Every import statement in every file in the package *must* be identical, even using build tags, even using conditional compilation. 96 | - Breaks the rule of naming things, two things which are the same, must have the same import path. 97 | 98 | * Versions in the URL (part 2) 99 | 100 | Leads to nightmarish scenarios where equality and type assertions are broken. 101 | 102 | import "github.com/project/v9/lib" // registers itself as a dialer 103 | import "github.com/project/dialer" 104 | 105 | err := dialer.Dial("someurl") 106 | fmt.Println(err == lib.ErrTimeout) => false 107 | fmt.Printf("%T", err) => "lib.ErrTimeout" 108 | fmt.Println(v7/lib.ErrTimeout == v9/lib.ErrTimeout) => false 109 | 110 | * Competitive Analysis 111 | 112 | * Dude, be a good Gopher, don't break users 113 | 114 | So the first, and longest standing solution to this problem is to always have a stable API. 115 | 116 | - Proposed solution from the Go team for several years. 117 | - Admirable attempt to extend the Go 1 contract to all Go code. 118 | 119 | If it worked, we wouldn't be having this conversation today 120 | 121 | - You _want_ to change the API for your package 122 | - Often the API and the consumer evolve in parallel 123 | - Even putting versions in the import path only guarantees I have _a_ version of that package, not _the_ version. 124 | 125 | * I live in the real world 126 | 127 | If my time in system administration taught me anything, it's the unexpected failures that get you. You can plan for the big disasters, but it turns out that the little disasters can be just as disruptive. 128 | 129 | - code.google.com closing down. Will Github still be around in 10 years ? 130 | - codehaus :( 131 | - companies merging, people getting married, someone dies, trademark dispute, etc. 132 | - FoundationDB :( 133 | 134 | These are all little disasters, you can usually find the code again, maybe it's just a quick sed rewrite and you're back again. 135 | 136 | But just like the big disasters, these little disasters are indistinguishable, code which built one day, won't build the next. 137 | 138 | * Don't be this person 139 | 140 | .image reproducible-builds/github.png 141 | 142 | The moral of the story is, if you are responsible for delivering a product written in Go, you need to be responsible for all the source that goes into that product. 143 | 144 | * Tools which manage $GOPATH 145 | 146 | Tools which fixup `$GOPATH` after `go`get` 147 | 148 | - godeps (plural, canonical) 149 | - glock 150 | - gvp 151 | 152 | Problems 153 | 154 | - not reproducible, the upstream can still disappear 155 | - must adjust your `$GOPATH` manually when moving between projects 156 | - near universal dislike for a .lock file in the package 157 | - universal disagreement on the format and layout of a .lock file 158 | 159 | * Tools which vendor packages 160 | 161 | Copying, vendoring, rewriting the source, is the _new_ position from the Go team. 162 | 163 | - godep 164 | 165 | Problems 166 | 167 | - requires source rewriting; many uncomfortable with this 168 | - possibly breaks the naming rules; the same package can exist in the dependency graph under multiple names 169 | - concerns about losing track of the upstream 170 | - ugly long import lines 171 | 172 | * Tools which give you one $GOPATH per project 173 | 174 | Virtual env all the things! 175 | 176 | - gpm https://github.com/pote/gpm 177 | - gvm 178 | - govm 179 | - glide 180 | - /usr/bin/direnv (old skool) 181 | 182 | Problems 183 | 184 | - not isolated from upstream going away 185 | - hard to use, terminal or shell session becomes magic 186 | 187 | * Proposal 188 | 189 | * Stop working around go get 190 | 191 | .image reproducible-builds/goget.jpg _ 400 192 | 193 | Every one of the existing solutions is hamstrung by the fact it is working around the limitations of the `go` tool. 194 | 195 | Stop using `go`get`. Don't use the `go` tool at all. 196 | 197 | * Requirements 198 | 199 | So, we're talking about writing a new build tool for Go, not a wrapper around an existing tool. 200 | 201 | * Project based 202 | 203 | A new build tool should be project based. 204 | 205 | - A project is where main packages live. 206 | - A project is effectively a single $GOPATH. 207 | - Automatic detection, you don't need 'enter' a project. 208 | - The owner of the project decides on the version of a particular import path in use. 209 | - Never more than one copy of a single import per project. 210 | 211 | * No configuration files 212 | 213 | This one I find hard to accept, but Go developers do not want to have any sort of configuration file to build their code. 214 | 215 | I find this hard to rationalize because most repos that I look have had dozens of turds in them, Gruntfiles, Dockerfiles, Werker configs, etc. 216 | 217 | - Projects are detected from the path, anything that has a `src` in the path is a project. 218 | - Works well in practice, and is backwards compatible with `$GOPATH`. 219 | 220 | * Respect the canonical import path 221 | 222 | package pdf // import "rsc.io/pdf" 223 | 224 | - rsc added this, it's clear what he thinks about the possibility of duplicates in the binary 225 | - Import rewriting has to rewrite the import comment as well, and that sounds like deliberately disabling the safety interlock on a handgun. 226 | 227 | * Leaves source untouched 228 | 229 | - Check the whole source in 230 | - Use submodules or subtrees, svn externals, etc, if you prefer 231 | - Don't touch the source, then you stand a chance of hashing it / diffing it / signing it 232 | 233 | * Annoying things 234 | 235 | If we're going to go the extreme of divorcing ourselves from the `go` tool then maybe we can fix a few other annoyances along the way 236 | 237 | - `-tags`something` now just works 238 | - deleting a file from a package, causes a rebuild 239 | - deleting a package's source, we won't use the stale .a in ~/pkg 240 | - not restricted to `go`get` ideas of correct DVCS usage, ie, can use ssh://github.com/... 241 | 242 | * Introducing gb 243 | 244 | .image reproducible-builds/gb.jpg _ 350 245 | 246 | % /usr/bin/gb 247 | 248 | - Proof of concept 249 | - Project based, project is automatically detected 250 | - Dependencies are a property of the project, one copy of any package per project 251 | - Supports vendoring without rewriting via multiple src/ directories 252 | - Supports plugins (git style) 253 | 254 | * Demo time 255 | 256 | # * This only works for projects, what about packages ? 257 | 258 | # Yes, this solution works for projects, it encourages you to build larger projects. 259 | 260 | # Personally, I think this is what Go needs at the moment. 261 | 262 | # - Projects should think hard about each dependency they bring in -- they aren't free 263 | 264 | # - Packages developed and thrown out there, hoping that someone else will use and popularise them. As Peter Bourgon noted at FOSDEM this year, does Go really need another http mux ? 265 | 266 | # What I want to see is large projects being built with Go, then, when they are proven, lets circle back and peal off parts of those projects that are reusable. But that happens second, you can't have a stable complete package without an anchor tenant, and it makes sense to me that those libraries should be incubated inside their host, not along side them -- it worked for django, it worked for rails, I think it's a message that should be studied. 267 | 268 | * Take aways 269 | 270 | The problem is `go`get`, not the import statement. 271 | 272 | The `go` tool doesn't define the language, we can build a replacement. 273 | 274 | * Try it out 275 | 276 | go get github.com/constabulary/gb/... 277 | 278 | - 100 % compatible with existing Go source. 279 | - Don't even need to change `$GOPATH`. 280 | - Upgrade to a gb project if you like. 281 | - Reusable library for building Go packages, no more shelling out to `go`build` 282 | - Write a plugin, please write a plugin. 283 | 284 | -------------------------------------------------------------------------------- /reproducible-builds/gb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/reproducible-builds/gb.jpg -------------------------------------------------------------------------------- /reproducible-builds/github.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/reproducible-builds/github.png -------------------------------------------------------------------------------- /reproducible-builds/goget.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/reproducible-builds/goget.jpg -------------------------------------------------------------------------------- /seven/flamegraph1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/flamegraph1.png -------------------------------------------------------------------------------- /seven/flamegraph2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/flamegraph2.png -------------------------------------------------------------------------------- /seven/flamegraph3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/flamegraph3.png -------------------------------------------------------------------------------- /seven/good-news.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/good-news.jpg -------------------------------------------------------------------------------- /seven/net-http-pprof.go: -------------------------------------------------------------------------------- 1 | package main // OMIT 2 | // OMIT 3 | import _ "net/http/pprof" 4 | import "log" // OMIT 5 | import "net/http" // OMIT 6 | 7 | func main() { 8 | log.Println(http.ListenAndServe("localhost:3999", nil)) 9 | } 10 | -------------------------------------------------------------------------------- /seven/perf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/perf.png -------------------------------------------------------------------------------- /seven/wharrgarbl.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/seven/wharrgarbl.jpg -------------------------------------------------------------------------------- /simplicity/10_ck_chef_hand_10.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/10_ck_chef_hand_10.jpg -------------------------------------------------------------------------------- /simplicity/16.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/16.gif -------------------------------------------------------------------------------- /simplicity/IMG_0095.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/IMG_0095.png -------------------------------------------------------------------------------- /simplicity/IMG_0245.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/IMG_0245.JPG -------------------------------------------------------------------------------- /simplicity/IMG_8124.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/IMG_8124.jpg -------------------------------------------------------------------------------- /simplicity/amor.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/amor.jpg -------------------------------------------------------------------------------- /simplicity/cover-big.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/cover-big.jpg -------------------------------------------------------------------------------- /simplicity/error.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/error.png -------------------------------------------------------------------------------- /simplicity/four_string_braid.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/four_string_braid.jpg -------------------------------------------------------------------------------- /simplicity/gofmt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/gofmt.png -------------------------------------------------------------------------------- /simplicity/growing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/growing.png -------------------------------------------------------------------------------- /simplicity/history.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/history.jpg -------------------------------------------------------------------------------- /simplicity/nasa-mainframe-980x663.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/nasa-mainframe-980x663.jpg -------------------------------------------------------------------------------- /simplicity/pipe.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/pipe.png -------------------------------------------------------------------------------- /simplicity/supermighty.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/supermighty.png -------------------------------------------------------------------------------- /simplicity/thesis.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/simplicity/thesis.png -------------------------------------------------------------------------------- /split1/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split1/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split1/split_test.go: -------------------------------------------------------------------------------- 1 | // +build none 2 | 3 | // tag::test[] 4 | package split 5 | 6 | import ( 7 | "reflect" 8 | "testing" 9 | ) 10 | 11 | // tag::tests[] 12 | func TestSplit(t *testing.T) { 13 | got := Split("a/b/c", "/") 14 | want := []string{"a", "b", "c"} 15 | if !reflect.DeepEqual(want, got) { 16 | t.Fatalf("expected: %v, got: %v", want, got) 17 | } 18 | } 19 | 20 | // end::test[] 21 | 22 | // tag::test2[] 23 | func TestSplitWrongSep(t *testing.T) { 24 | got := Split("a/b/c", ",") 25 | want := []string{"a/b/c"} // <1> 26 | if !reflect.DeepEqual(want, got) { 27 | t.Fatalf("expected: %v, got: %v", want, got) 28 | } 29 | } 30 | 31 | // end::test2[] 32 | 33 | // tag::empty[] 34 | func testSplitEmptySep(t *testing.T) { 35 | got := Split("a/b/c", "") 36 | want := []string{"a/b/c"} 37 | if !reflect.DeepEqual(want, got) { 38 | t.Fatalf("expected: %v, got: %v", want, got) 39 | } 40 | } 41 | 42 | // end::empty[] 43 | 44 | // tag::test3[] 45 | func testSplitTrailingSep(t *testing.T) { 46 | got := Split("a/b/c/", "/") 47 | want := []string{"a", "b", "c"} 48 | if !reflect.DeepEqual(want, got) { 49 | t.Fatalf("expected: %v, got: %v", want, got) 50 | } 51 | } 52 | 53 | // end::test3[] 54 | // end::tests[] 55 | -------------------------------------------------------------------------------- /split10/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | 5 | require github.com/google/go-cmp v0.2.0 6 | -------------------------------------------------------------------------------- /split10/go.sum: -------------------------------------------------------------------------------- 1 | github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ= 2 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 3 | -------------------------------------------------------------------------------- /split10/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split10/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/google/go-cmp/cmp" 7 | ) 8 | 9 | // tag::test[] 10 | func TestSplit(t *testing.T) { 11 | tests := map[string]struct { 12 | input string 13 | sep string 14 | want []string 15 | }{ 16 | "simple": {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 17 | "wrong sep": {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 18 | "no sep": {input: "abc", sep: "/", want: []string{"abc"}}, 19 | "trailing sep": {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 20 | } 21 | 22 | for name, tc := range tests { 23 | t.Run(name, func(t *testing.T) { 24 | got := Split(tc.input, tc.sep) 25 | diff := cmp.Diff(tc.want, got) 26 | if diff != "" { 27 | t.Fatalf(diff) 28 | } 29 | }) 30 | } 31 | } 32 | 33 | // end::test[] 34 | -------------------------------------------------------------------------------- /split11/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | 5 | require github.com/google/go-cmp v0.2.0 6 | -------------------------------------------------------------------------------- /split11/go.sum: -------------------------------------------------------------------------------- 1 | github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ= 2 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= 3 | -------------------------------------------------------------------------------- /split11/split.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import "strings" 4 | 5 | // tag::split[] 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | if len(s) > 0 { 17 | result = append(result, s) 18 | } 19 | return result 20 | } 21 | 22 | // end::split[] 23 | -------------------------------------------------------------------------------- /split11/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/google/go-cmp/cmp" 7 | ) 8 | 9 | // tag::test[] 10 | func TestSplit(t *testing.T) { 11 | tests := map[string]struct { 12 | input string 13 | sep string 14 | want []string 15 | }{ 16 | "simple": {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 17 | "wrong sep": {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 18 | "no sep": {input: "abc", sep: "/", want: []string{"abc"}}, 19 | "trailing sep": {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 20 | } 21 | 22 | for name, tc := range tests { 23 | t.Run(name, func(t *testing.T) { 24 | got := Split(tc.input, tc.sep) 25 | diff := cmp.Diff(tc.want, got) 26 | if diff != "" { 27 | t.Fatalf(diff) 28 | } 29 | }) 30 | } 31 | } 32 | 33 | // end::test[] 34 | -------------------------------------------------------------------------------- /split2/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split2/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split2/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | type test struct { 11 | input string 12 | sep string 13 | want []string 14 | } 15 | 16 | tests := []test{ 17 | {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 18 | {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 19 | {input: "abc", sep: "/", want: []string{"abc"}}, 20 | } 21 | 22 | for _, tc := range tests { 23 | got := Split(tc.input, tc.sep) 24 | if !reflect.DeepEqual(tc.want, got) { 25 | t.Fatalf("expected: %v, got: %v", tc.want, got) 26 | } 27 | } 28 | } 29 | 30 | // end::test[] 31 | -------------------------------------------------------------------------------- /split3/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split3/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split3/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := []struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | {input: "abc", sep: "/", want: []string{"abc"}}, 18 | } 19 | 20 | for _, tc := range tests { 21 | got := Split(tc.input, tc.sep) 22 | if !reflect.DeepEqual(tc.want, got) { 23 | t.Fatalf("expected: %v, got: %v", tc.want, got) 24 | } 25 | } 26 | } 27 | 28 | // end::test[] 29 | -------------------------------------------------------------------------------- /split4/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split4/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split4/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := []struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | {input: "abc", sep: "/", want: []string{"abc"}}, 18 | {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, // trailing sep 19 | } 20 | 21 | for _, tc := range tests { 22 | got := Split(tc.input, tc.sep) 23 | if !reflect.DeepEqual(tc.want, got) { 24 | t.Fatalf("expected: %v, got: %v", tc.want, got) 25 | } 26 | } 27 | } 28 | 29 | // end::test[] 30 | -------------------------------------------------------------------------------- /split5/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split5/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split5/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := []struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | {input: "abc", sep: "/", want: []string{"abc"}}, 18 | {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, // trailing sep 19 | } 20 | 21 | for i, tc := range tests { 22 | got := Split(tc.input, tc.sep) 23 | if !reflect.DeepEqual(tc.want, got) { 24 | t.Fatalf("test %d: expected: %v, got: %v", i+1, tc.want, got) 25 | } 26 | } 27 | } 28 | 29 | // end::test[] 30 | -------------------------------------------------------------------------------- /split6/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split6/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split6/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := []struct { 11 | name string 12 | input string 13 | sep string 14 | want []string 15 | }{ 16 | {name: "simple", input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 17 | {name: "wrong sep", input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 18 | {name: "no sep", input: "abc", sep: "/", want: []string{"abc"}}, 19 | {name: "trailing sep", input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 20 | } 21 | 22 | for _, tc := range tests { 23 | got := Split(tc.input, tc.sep) 24 | if !reflect.DeepEqual(tc.want, got) { 25 | t.Fatalf("%s: expected: %v, got: %v", tc.name, tc.want, got) 26 | } 27 | } 28 | } 29 | 30 | // end::test[] 31 | -------------------------------------------------------------------------------- /split7/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split7/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split7/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := map[string]struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | "simple": {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | "wrong sep": {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | "no sep": {input: "abc", sep: "/", want: []string{"abc"}}, 18 | "trailing sep": {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 19 | } 20 | 21 | for name, tc := range tests { 22 | got := Split(tc.input, tc.sep) 23 | if !reflect.DeepEqual(tc.want, got) { 24 | t.Fatalf("%s: expected: %v, got: %v", name, tc.want, got) 25 | } 26 | } 27 | } 28 | 29 | // end::test[] 30 | -------------------------------------------------------------------------------- /split8/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split8/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split8/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := map[string]struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | "simple": {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | "wrong sep": {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | "no sep": {input: "abc", sep: "/", want: []string{"abc"}}, 18 | "trailing sep": {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 19 | } 20 | 21 | for name, tc := range tests { 22 | t.Run(name, func(t *testing.T) { 23 | got := Split(tc.input, tc.sep) 24 | if !reflect.DeepEqual(tc.want, got) { 25 | t.Fatalf("expected: %v, got: %v", tc.want, got) 26 | } 27 | }) 28 | } 29 | } 30 | 31 | // end::test[] 32 | -------------------------------------------------------------------------------- /split9/go.mod: -------------------------------------------------------------------------------- 1 | module split 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /split9/split.go: -------------------------------------------------------------------------------- 1 | // tag::split[] 2 | package split 3 | 4 | import "strings" 5 | 6 | // Split slices s into all substrings separated by sep and returns a slice of 7 | // the substrings between those separators. 8 | func Split(s, sep string) []string { 9 | var result []string 10 | i := strings.Index(s, sep) 11 | for i > -1 { 12 | result = append(result, s[:i]) 13 | s = s[i+len(sep):] 14 | i = strings.Index(s, sep) 15 | } 16 | return append(result, s) 17 | } 18 | 19 | // end::split[] 20 | -------------------------------------------------------------------------------- /split9/split_test.go: -------------------------------------------------------------------------------- 1 | package split 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | // tag::test[] 9 | func TestSplit(t *testing.T) { 10 | tests := map[string]struct { 11 | input string 12 | sep string 13 | want []string 14 | }{ 15 | "simple": {input: "a/b/c", sep: "/", want: []string{"a", "b", "c"}}, 16 | "wrong sep": {input: "a/b/c", sep: ",", want: []string{"a/b/c"}}, 17 | "no sep": {input: "abc", sep: "/", want: []string{"abc"}}, 18 | "trailing sep": {input: "a/b/c/", sep: "/", want: []string{"a", "b", "c"}}, 19 | } 20 | 21 | for name, tc := range tests { 22 | t.Run(name, func(t *testing.T) { 23 | got := Split(tc.input, tc.sep) 24 | if !reflect.DeepEqual(tc.want, got) { 25 | t.Fatalf("expected: %#v, got: %#v", tc.want, got) 26 | } 27 | }) 28 | } 29 | } 30 | 31 | // end::test[] 32 | -------------------------------------------------------------------------------- /whats-in-a-name.adoc: -------------------------------------------------------------------------------- 1 | = What's in a name? 2 | dotGo 2019 3 | 4 | == Leaving a trail 5 | 6 | In the last six years I've written Go at four different companies. 7 | I've been priveleged that I'be been able to write Go in a bunch of different verticals and environments, but the downside of this priveliege, is i've left behind a bunch of code; written at a bunch of experience levels, 8 | 9 | Does the code that I have left for others reflect the values of the language, simplicity, readability, maintainabikity. 10 | 11 | In Go we care about readability. A lot. 12 | 13 | "Readability is essential for maintability." 14 | -- Mark Reinhold, JVM language summit 2018 15 | 16 | "There are only two hard things in Computer Science: cache invalidation and naming things." 17 | -— Phil Karlton 18 | 19 | "The most important skill for a programmer is the ability to effectively communicate ideas." 20 | -- https://gaston.life/books/effective-programming/ 21 | 22 | Wait, what? 23 | Aren't we paid to program? 24 | No, of course not, programmers are not paid to program--we're paid to solve problems, with software. 25 | 26 | Hmm, ok, maybe I buy that, but what's all this guff about _effectively communicating ideas_, you just said I'm paid to solve problems with software. 27 | 28 | "This leads to code that communicates intention. The only work that’s reusable is the one that you understand what it communicates." 29 | -- https://gaston.life/books/effective-programming/ 30 | 31 | 32 | 33 | This is a talk about naming things in Go. 34 | 35 | Names are important. 36 | Given the limited syntax of our language, the names we give to things in our programs have an oversized impact on the readability of our programs. 37 | 38 | First, lets set the ground rules. 39 | What are the things in Go we can name. 40 | 41 | To get technical, when I'm talking about naming, I'm talking about naming _identifiers_ in Go programs. 42 | But that's a bit lengthy, so lets just call it naming from now on -- you understand what I mean. 43 | 44 | Anything in Go that is an _identifier_ has a name. 45 | To make this clear this 46 | 47 | * the name of a type, struct, or interface 48 | * the name of a function or a method 49 | * the name of a package 50 | * the name of a constant 51 | * the name of a variable, formal parameter, or return value 52 | 53 | == Identifier length 54 | 55 | Go is not a language that optimises for the shortest line, nor is a language which optimises for the least number of lines in a program. 56 | We're not optimising for the size of the source code on disk, nor how long it takes to type. 57 | 58 | As Rob Pike said, "Go programmers want the _right_ length identifiers" 59 | 60 | Prefer longer names for things which are important. 61 | 62 | If everything’s important, then nobody is 63 | 64 | If its clearer to break a 65 | 66 | ``` 67 | // uint32OrNil returns a *types.UInt32Value containing the v or nil if v is zero. 68 | func uint32OrNil(v int) *types.UInt32Value { 69 | switch v { 70 | case 0: 71 | return nil 72 | default: 73 | return &types.UInt32Value{Value: uint32(v)} 74 | } 75 | } 76 | ``` 77 | 78 | good; v is short, the function is only 7 lines line 79 | bad; the functino talks about returning a uint32, but it takes an int, and the identifeir is v, i would be better. 80 | 81 | == Code is decoded 82 | 83 | Peter Siebal. 84 | 85 | == Verb/noun/etc 86 | 87 | == Kevlins observation that factory suffix conventions convey no information. 88 | 89 | == Use the type system, luke 90 | 91 | == Be consistent 92 | 93 | If you have a set of methods on a type, make sure they all use the same receiver. 94 | If you find that the receiver conflicts with 95 | 96 | == It's contextual 97 | 98 | Sometimes applying rules takes finess. 99 | 100 | an variables name shouldn't be larger than its type's name. 101 | 102 | Adherence to these memes is what I'd conclude is the soul of what we call 'idiomatic' go. 103 | 104 | == Don't be stubborn 105 | 106 | If you're working in a team, or on a piece of code that 107 | 108 | Say they use names which you thing are overly verbose, or perhaps you think their perchant for tiny intendifiers is not your preference -- keep it to yourself. 109 | 110 | The goal is readability, the goal is always readability. 111 | Changing styles in the middle of a file is jarring. 112 | Unifirmity, even if its not your preferred approach, is more valuable for maintenance than it is 113 | 114 | == Names help your code find meaning 115 | 116 | 117 | 118 | == Package naming 119 | 120 | == Machinists have names for everything, so do mathematicians, why don’t programmers, why do we try to fit a thing to a name, not a name to the thing? 121 | 122 | "Poor naming is symptomatic of poor design." 123 | -- Dave Cheney 124 | 125 | == On the subject of nouns 126 | 127 | One of my guilty hobbies is watching metalworking videos on youtube. 128 | I have zero desire to be a machinist, I enjoy thier attention to detail, and find watching them work soothing. 129 | 130 | One thing about machining videos is machinists have a lot of tools, and each tool has a name. 131 | Take making a hole in a piece of metal. 132 | You can use a punch or a drill. You can bore a hole, or ream it, and if you want an irregular shape you can broach it. 133 | The same applies in other parts of machining, what we lay people would call a washer, machinists have many names; bushing. 134 | 135 | To flatting something you can mill it, file it, grind it, lap it, scrape it, flake it, or hone it. 136 | 137 | And so on 138 | 139 | And one fascinating thing about this vocabuilary is one machinist can look at a piece produced by other and not only figure out how to reproduce it, but also use the same terminology. 140 | 141 | And I 142 | 143 | Now my partner, who has a degree in such things, tells me that this is known as an _interpretive repetiour_. 144 | 145 | Now, in programming we have our own repitour, we have words 146 | 147 | 148 | 149 | 150 | == Conclusion 151 | 152 | don't choose a name for your type which is the perfect name for variables of that type. 153 | 154 | this will show up in formal parameters 155 | 156 | but, package names are you friend 157 | 158 | var db sql.DB is good 159 | 160 | list := container.List 161 | 162 | is bad 163 | 164 | * short variable names work well when the distance between their declaration and _last_ use is short. 165 | * long variable names need to justify themselves, the longer they are, the more value they need to bring. Lengthy beurocratic names carry a low amount of signal. 166 | * the name of your varible shouldn't be longer than its type. 167 | * never include the name of your type in the name of your variable. 168 | * constants should describe the value they contain, _not_ where that value is used. 169 | * Single letter variables for loops and branches, single words for parameters and return values, multiple words for functions and package level things, 170 | * Single words for methods, single words for interfaces, and always remember that the name of a package is part of the name the caller uses to to refer to it, so make use of that. 171 | 172 | https://www.reddit.com/r/golang/comments/8wxwgv/why_does_go_encourage_short_variable_names/ 173 | -------------------------------------------------------------------------------- /writing-high-performance-go/Nehalem_Die_Shot_3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/writing-high-performance-go/Nehalem_Die_Shot_3.jpg -------------------------------------------------------------------------------- /writing-high-performance-go/concat/concat_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "net" 7 | "testing" 8 | "time" 9 | ) 10 | 11 | // sink to ensure the compiler does not optimise away dead assignments. 12 | var Result string 13 | 14 | // fake up some values for request and client. 15 | func setup(b *testing.B) (struct{ ID string }, net.Listener) { 16 | request := struct { 17 | ID string 18 | }{"9001"} 19 | client, err := net.Listen("tcp", ":0") 20 | if err != nil { 21 | b.Fatal(err) 22 | } 23 | return request, client 24 | } 25 | 26 | func BenchmarkConcatenate(b *testing.B) { 27 | request, client := setup(b) 28 | defer client.Close() 29 | 30 | b.ResetTimer() 31 | b.ReportAllocs() 32 | var r string 33 | for n := 0; n < b.N; n++ { 34 | s := request.ID 35 | s += " " + client.Addr().String() 36 | s += " " + time.Now().String() 37 | r = s 38 | } 39 | Result = r 40 | } 41 | 42 | func BenchmarkFPrintf(b *testing.B) { 43 | request, client := setup(b) 44 | defer client.Close() 45 | 46 | b.ResetTimer() 47 | b.ReportAllocs() 48 | var r string 49 | for n := 0; n < b.N; n++ { 50 | var b bytes.Buffer 51 | fmt.Fprintf(&b, "%s %v %v", request.ID, client.Addr(), time.Now()) 52 | r = b.String() 53 | } 54 | Result = r 55 | } 56 | 57 | func BenchmarkStrconv(b *testing.B) { 58 | request, client := setup(b) 59 | defer client.Close() 60 | 61 | b.ResetTimer() 62 | b.ReportAllocs() 63 | var r string 64 | for n := 0; n < b.N; n++ { 65 | b := make([]byte, 0, 40) 66 | b = append(b, request.ID...) 67 | b = append(b, ' ') 68 | b = append(b, client.Addr().String()...) 69 | b = append(b, ' ') 70 | b = time.Now().AppendFormat(b, "2006-01-02 15:04:05.999999999 -0700 MST") 71 | r = string(b) 72 | } 73 | Result = r 74 | } 75 | -------------------------------------------------------------------------------- /writing-high-performance-go/copy/copy_test.go: -------------------------------------------------------------------------------- 1 | package copy 2 | 3 | import ( 4 | "testing" 5 | ) 6 | 7 | // START OMIT 8 | var c = []byte("1789678900001234567890") 9 | 10 | func BenchmarkCopy(b *testing.B) { 11 | for i := 0; i < b.N; i++ { 12 | d := make([]byte, len(c)) 13 | copy(d, c) 14 | } 15 | } 16 | 17 | func BenchmarkAppend(b *testing.B) { 18 | for i := 0; i < b.N; i++ { 19 | _ = append([]byte(nil), c...) 20 | } 21 | } 22 | 23 | // END OMIT 24 | -------------------------------------------------------------------------------- /writing-high-performance-go/fib/fib_test.go: -------------------------------------------------------------------------------- 1 | package fib 2 | 3 | // STARTBENCH OMIT 4 | import "testing" 5 | 6 | func BenchmarkFib(b *testing.B) { 7 | for n := 0; n < b.N; n++ { 8 | Fib(20) // run the Fib function b.N times 9 | } 10 | } 11 | 12 | // ENDBENCH OMIT 13 | 14 | // STARTFIB OMIT 15 | // Fib computes the n'th number in the Fibonacci series. 16 | func Fib(n int) int { 17 | if n < 2 { 18 | return n 19 | } 20 | return Fib(n-1) + Fib(n-2) 21 | } 22 | 23 | // ENDFIB OMIT 24 | 25 | func Fib2(n int) int { 26 | a, b := 0, 1 27 | for i := 0; i < n; i++ { 28 | a, b = b, a+b 29 | } 30 | return a 31 | } 32 | 33 | func TestFib(t *testing.T) { 34 | fibs := []int{0, 1, 1, 2, 3, 5, 8, 13, 21} 35 | for n, want := range fibs { 36 | got := Fib(n) 37 | if want != got { 38 | t.Errorf("Fib(%d): want %d, got %d", n, want, got) 39 | } 40 | } 41 | } 42 | 43 | func TestFib2(t *testing.T) { 44 | fibs := []int{0, 1, 1, 2, 3, 5, 8, 13, 21} 45 | for n, want := range fibs { 46 | got := Fib2(n) 47 | if want != got { 48 | t.Errorf("Fib2(%d): want %d, got %d", n, want, got) 49 | } 50 | } 51 | } 52 | 53 | func TestFibFib(t *testing.T) { 54 | for n := 0; n < 30; n++ { 55 | want := Fib(n) 56 | got := Fib2(n) 57 | if want != got { 58 | t.Errorf("Fib2(%d): want %d, got %d", n, want, got) 59 | } 60 | } 61 | } 62 | 63 | func _BenchmarkFib2(b *testing.B) { 64 | for n := 0; n < b.N; n++ { 65 | Fib2(20) 66 | } 67 | } 68 | -------------------------------------------------------------------------------- /writing-high-performance-go/go17burndown.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/writing-high-performance-go/go17burndown.png -------------------------------------------------------------------------------- /writing-high-performance-go/grow.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | func main() { 7 | b := make([]int, 1024) 8 | b = append(b, 99) 9 | fmt.Println("len:", len(b), "cap:", cap(b)) 10 | } 11 | 12 | // END OMIT 13 | -------------------------------------------------------------------------------- /writing-high-performance-go/ioloop.go: -------------------------------------------------------------------------------- 1 | package p 2 | 3 | import ( 4 | "encoding/binary" 5 | "io" 6 | ) 7 | 8 | // START OMIT 9 | type Conn struct { 10 | r io.ReadCloser 11 | ch chan uint32 12 | } 13 | 14 | func (c *Conn) Loop() { 15 | defer r.Close() 16 | var buf [512]byte 17 | for { 18 | b := buf[:] // create slice of buf 19 | n, err := c.r.Read(b) 20 | 21 | for b = b[:n]; len(b) != 0; b = b[4:] { 22 | ch <- binary.BigEndian.Uint32(b) 23 | } 24 | 25 | if err != nil { 26 | return 27 | } 28 | } 29 | } 30 | 31 | // END OMIT 32 | -------------------------------------------------------------------------------- /writing-high-performance-go/latency.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/writing-high-performance-go/latency.png -------------------------------------------------------------------------------- /writing-high-performance-go/numbers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/davecheney/presentations/5a62011c40344043923c2daafb07f3c15e04859c/writing-high-performance-go/numbers.png -------------------------------------------------------------------------------- /writing-high-performance-go/pool.go: -------------------------------------------------------------------------------- 1 | package p 2 | 3 | import "sync" 4 | 5 | // START OMIT 6 | var pool = sync.Pool{New: func() interface{} { return make([]byte, 4096) }} 7 | 8 | func fn() { 9 | buf := pool.Get().([]byte) // takes from pool or calls New 10 | // do work 11 | pool.Put(buf) // returns buf to the pool 12 | } 13 | 14 | // END OMIT 15 | -------------------------------------------------------------------------------- /writing-high-performance-go/popcnt/popcnt2_test.go: -------------------------------------------------------------------------------- 1 | // +build ignore 2 | 3 | package main 4 | 5 | import "testing" 6 | 7 | const m1 = 0x5555555555555555 8 | const m2 = 0x3333333333333333 9 | const m4 = 0x0f0f0f0f0f0f0f0f 10 | const h01 = 0x0101010101010101 11 | 12 | func popcnt(x uint64) int { 13 | x -= (x >> 1) & m1 14 | x = (x & m2) + ((x >> 2) & m2) 15 | x = (x + (x >> 4)) & m4 16 | return int((x * h01) >> 56) 17 | } 18 | 19 | // START OMIT 20 | func BenchmarkPopcnt(b *testing.B) { 21 | for i := 0; i < b.N; i++ { 22 | // optimised away 23 | } 24 | } 25 | 26 | // END OMIT 27 | -------------------------------------------------------------------------------- /writing-high-performance-go/popcnt/popcnt_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "testing" 4 | 5 | // START OMIT 6 | const m1 = 0x5555555555555555 7 | const m2 = 0x3333333333333333 8 | const m4 = 0x0f0f0f0f0f0f0f0f 9 | const h01 = 0x0101010101010101 10 | 11 | func popcnt(x uint64) uint64 { 12 | x -= (x >> 1) & m1 13 | x = (x & m2) + ((x >> 2) & m2) 14 | x = (x + (x >> 4)) & m4 15 | return (x * h01) >> 56 16 | } 17 | 18 | func BenchmarkPopcnt(b *testing.B) { 19 | for i := 0; i < b.N; i++ { 20 | popcnt(uint64(i)) 21 | } 22 | } 23 | 24 | // END OMIT 25 | -------------------------------------------------------------------------------- /writing-high-performance-go/readwrite.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // START OMIT 6 | // sendfile sends the contents of path to the client c. 7 | func sendfile(c net.Conn, path string) error { 8 | r, err := os.Open(path) 9 | if err != nil { 10 | return err 11 | } 12 | defer r.Close() 13 | 14 | // Set the deadline to one minute from now. 15 | c.SetWriteDeadline(time.Now().Add(60 * time.Second)) 16 | 17 | // Copy will send as much of r to the client as it can in 60 seconds. 18 | _, err = io.Copy(c, r) 19 | return err 20 | } 21 | 22 | // END OMIT 23 | -------------------------------------------------------------------------------- /writing-high-performance-go/semaphore.go: -------------------------------------------------------------------------------- 1 | package p 2 | 3 | type Work int 4 | 5 | // START OMIT 6 | var semaphore = make(chan struct{}, 10) 7 | 8 | func processRequest(work *Work) { 9 | semaphore <- struct{}{} // acquire semaphore 10 | // process request 11 | <-semaphore // release semaphore 12 | } 13 | 14 | // END OMIT 15 | -------------------------------------------------------------------------------- /yow-2018.adoc: -------------------------------------------------------------------------------- 1 | = YOW! Contour talk 2 | 3 | 4 | == Introduction 5 | 6 | Who am I? 7 | Who do I work for? 8 | What am I talking about (and why)? 9 | 10 | == Writing Kubernetes components 11 | 12 | Case study, Contour, the ingress controller that my team has built and is currently being rolled as a component in the network 13 | 14 | - like all good infrastructure components, contour is open source, and written in Go. 15 | 16 | == What is k8s? 17 | 18 | I've worked in infrastructure most of my career and now working for a company who employ a number of the core k8s contributors its useful for me to 19 | 20 | == What is k8s 21 | 22 | The marketing blurb for kuberenetes is it is some scalable workload orchestration system. 23 | Honestly, these words don't mean much unless you've already drunk a fifth of kool aid, so here's my descriptino of kubernetes 24 | 25 | if you put your app in a docker container, kuberenetes will take care of making sure that container is always running on a server. 26 | 27 | So that's clearly a flippant description -- albeit fundamenally complete -- but there is a lot that goes into covrering the distance between a python app running in a docker container, and the application being available on the web, on a stable url, with TLS. 28 | 29 | This is where I come in, because part of getting your app deployed on kubernetes is the nextworking parts, specifically the HTTP routers, reverse proxies and load balancers that will get traffic from your users to your app. 30 | 31 | == Basic k8s components 32 | 33 | The secret sauce that holds k8s together is a concept called the apiserver. Which itself is a thin wrapper around etcd which is your typical key/value replicated database. 34 | 35 | the apiserver stores and lets clients query and watch, various objects that represent the desired state in your cluster. 36 | 37 | For the purposes of today, the objects we're interested in are 38 | 39 | - Pod, is a collection of (docker) containers running on the same machine. Critically while their process namespaces are separate, their network namespaces are shared. 40 | - Service, an abstract notion of a layer 4 service. Service objects are identified by name, but also an IP address which is reachable by every pod in the cluster. 41 | - Ingress. An ingress object represents 42 | 43 | 44 | == What is Ingress? 45 | 46 | It's become popular to talk about design philosophies. The principles that guide the design, or the design of the design itself. 47 | 48 | Ingress wasn't part of the original design of kubernetes, it wasn't one of the original set of objects. It was added later. 49 | 50 | So, what was the reason for adding ingress. What are its design principals? 51 | 52 | 53 | 54 | 55 | == What is contour? 56 | 57 | == Why did we choose Envoy? 58 | 59 | == Contour the project 60 | 61 | == Managing Concurrency 62 | 63 | == Dependency management 64 | 65 | == Developing with docker 66 | 67 | == Local deployment 68 | 69 | == Functional testing 70 | 71 | == Care and feeding 72 | 73 | == Conclusion 74 | 75 | --------------------------------------------------------------------------------