├── 2016 ├── interfacer.slide ├── parsers.slide ├── shfmt.slide └── state-machine.png ├── 2017 ├── gogrep.slide ├── goreduce.slide ├── goreduce1a.go ├── goreduce1b.go ├── goreduce2a.go ├── goreduce2b.go ├── helloworld.go └── intro-to-go.slide ├── 2018 ├── benchstat.slide ├── go1.11.slide ├── go1.11_godoc_versions.png ├── go1.12-pre.slide ├── gogrep.slide ├── stringer.slide └── unparam.slide ├── 2020 └── testing-tips │ ├── go.mod │ ├── go.sum │ ├── main.go │ └── main_test.go ├── LICENSE └── README.md /2016/interfacer.slide: -------------------------------------------------------------------------------- 1 | interfacer 2 | Because interfaces rock! 3 | 22 Mar 2016 4 | 5 | Daniel Martí 6 | https://mvdan.cc 7 | mvdan@mvdan.cc 8 | https://github.com/mvdan 9 | 10 | * Interfaces (in Go) are awesome 11 | 12 | - Set of methods 13 | - Implicit implements 14 | - Very useful for mocking 15 | - The Unix™ way - everything is a file! 16 | 17 | * Example 18 | 19 | func ProcessInput(f *os.File) error { 20 | b := make([]byte, 64) 21 | if _, err := f.Read(b); err != nil { 22 | return err 23 | } 24 | // process b 25 | return nil 26 | } 27 | 28 | * Why so specific? 29 | 30 | - Package users can only provide files 31 | - Any mocking needs to mock an entire file 32 | - Plain bad typing, to be honest 33 | 34 | * Example (fixed) 35 | 36 | func ProcessInput(r io.Reader) error { 37 | b := make([]byte, 64) 38 | if _, err := r.Read(b); err != nil { 39 | return err 40 | } 41 | // process b 42 | return nil 43 | } 44 | 45 | * Interface happiness 46 | 47 | - Anything one can read from can be passed as argument 48 | - Files, buffered strings, network sockets, pipes... 49 | - Very easy mocking and unit testing 50 | - Actually correct typing, no grumpy engineers 51 | 52 | * In comes interfacer 53 | 54 | - https://github.com/mvdan/interfacer 55 | 56 | func ProcessInput(f *os.File) error { 57 | b := make([]byte, 64) 58 | if _, err := f.Read(b); err != nil { 59 | return err 60 | } 61 | // process b 62 | return nil 63 | } 64 | 65 | And once run... 66 | 67 | $ interfacer ./... 68 | foo.go:10:19: f can be io.Reader 69 | 70 | * Workflow (1) 71 | 72 | - Grabs interface types from the std and the given packages 73 | 74 | "Error() string": "error", 75 | "Close() error": "io.Closer", 76 | "Close() error; Read([]byte) (int, error)": "io.ReadCloser", 77 | "Read([]byte) (int, error)": "io.Reader", 78 | [...] 79 | 80 | - Also grabs them from the given packages 81 | 82 | * Workflow (2) 83 | 84 | - Discards func signatures one may implement 85 | 86 | "(go/ast.Node) go/ast.Visitor": "go/ast.Visitor.Visit", 87 | "(net/http.ResponseWriter, *net/http.Request)": "net/http.HandlerFunc", 88 | "(string, os.FileInfo, error) error": "path/filepath.WalkFunc", 89 | [...] 90 | 91 | - For example, this function is properly typed: 92 | 93 | func HandleRequest(w http.ResponseWriter, r *http.Request) { 94 | w.Write([]byte{"hello!"}) 95 | } 96 | 97 | - It can't be `io.Writer`! 98 | 99 | * Workflow (3) 100 | 101 | - Walks the source code 102 | - Finds declared functions and notes their parameter usage 103 | 104 | What methods are called on each parameter? 105 | What explicit types is each parameter assigned to? 106 | What explicit types is each parameter passed as? 107 | Can each of the parameters be an interface at all? 108 | 109 | foo.field 110 | foo + 4 111 | foo > 0 112 | foo[2] 113 | 114 | * Workflow (4) 115 | 116 | - We take the parameters that can be interfaces 117 | - Look for a matching interface 118 | - Recommend that interface unless it's already used 119 | 120 | Remember to discard funcs that may be implementing an interface or 121 | function signature! 122 | 123 | * Recent work 124 | 125 | - Treat grouped parameters atomically (all or none) 126 | 127 | // Do not recommend io.Reader! 128 | func Transfer(from, to SomeStruct) { 129 | b := make([]byte, 10) 130 | from.Read(b) 131 | to.SomeOtherMethod(b) 132 | } 133 | 134 | - Notice when funcs specify a type 135 | 136 | // Do not recommend io.Writer! 137 | func WriteToSomeStruct(s SomeStruct) { 138 | b := make([]byte, 10) 139 | s.Write(b) 140 | } 141 | 142 | * Future work 143 | 144 | - More testing! 145 | - Suggesting new interfaces? 146 | -------------------------------------------------------------------------------- /2016/parsers.slide: -------------------------------------------------------------------------------- 1 | Parsers in Go 2 | 26 Apr 2016 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | https://github.com/mvdan 8 | 9 | * Parser generators 10 | 11 | - Grammar as input 12 | - Generated program 13 | - yacc, bison, etc 14 | 15 | if_clause : IF expr THEN stmts FI ; 16 | 17 | * Recursive descent parsers 18 | 19 | - Written by hand 20 | - No formal grammar definition used 21 | - Use recursion (stack) to define/satisfy grammar 22 | 23 | func if() { 24 | want(IF) 25 | expression() 26 | want(THEN) 27 | statements() 28 | want(FI) 29 | } 30 | 31 | * Worth writing by hand? 32 | 33 | Pros: 34 | 35 | - Human-readable and hackable code 36 | - One less tool to learn 37 | - Easier to give better error messages 38 | 39 | Cons: 40 | 41 | - Incredibly easy to screw up 42 | - Large initial investment of time 43 | - Can't rely on formal definition of grammar 44 | 45 | * Generated code 46 | 47 | switch next { 48 | case 32: 49 | case 47: 50 | case 84: 51 | ... 52 | } 53 | 54 | .image state-machine.png 55 | 56 | * Better error messages 57 | 58 | echo -n "for i" | dash 59 | 1: Syntax error: end of file unexpected (expecting "do") 60 | echo -n "foo(bar)" | dash 61 | 1: Syntax error: word unexpected (expecting ")") 62 | 63 | versus... 64 | 65 | echo -n "for i" | shfmt 66 | 1:6: "for foo" must be followed by "in", ; or a newline 67 | echo -n "foo(bar)" | shfmt 68 | 1:5: functions must start like "foo()" 69 | 70 | * In comes Russ Cox 71 | 72 | .link http://research.swtch.com/yyerror 73 | 74 | - Find a bad (mechanical) error message 75 | - Note down current state and input token/byte 76 | - Come up with a better error message 77 | - Write it into the parser 78 | - Repeat 79 | 80 | So sane error messages are possible and sort of easy in generated code 81 | after all. 82 | 83 | * Avoiding pitfalls with Go tools 84 | 85 | A parser is incredibly easy to mess up. Thankfully, we have tools: 86 | 87 | - Code coverage 88 | - TDD 89 | - go-fuzz 90 | - (more?) 91 | 92 | * Why Go moved to a hand-written parser 93 | 94 | - They can now hack on the parser at will 95 | - One less build dependency 96 | - Still easier to give better error messages 97 | 98 | But most importantly... 99 | 100 | - Globals! Parser generators tend to abuse global variables for state. 101 | 102 | This was (is?) keeping the Go compiler from being able to parse multiple 103 | files concurrently. 104 | 105 | * Proof of concept 106 | 107 | .link https://github.com/mvdan/sh 108 | 109 | - 400 commits, 4 months of weekend work 110 | - 1300 lines of code 111 | - fully POSIX compliant (I think?) 112 | 113 | * Let's play a game 114 | 115 | foo; ; 116 | echo if 117 | {foo 118 | foo & bar 119 | if { foo; } then; fi 120 | foo() bar 121 | -------------------------------------------------------------------------------- /2016/shfmt.slide: -------------------------------------------------------------------------------- 1 | Writing a Bash parser and formatter in Go 2 | 19 Oct 2016 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | https://github.com/mvdan 8 | 9 | * Parser generators 10 | 11 | - Grammar as input 12 | - Generated program 13 | - yacc, bison, etc 14 | 15 | if_clause : IF expr THEN stmts FI ; 16 | 17 | * Recursive descent parsers 18 | 19 | - Written by hand 20 | - No formal grammar definition used 21 | - Use recursion (stack) to define/satisfy grammar 22 | 23 | func (p *parser) if() { 24 | p.want(IF) 25 | p.expression() 26 | p.want(THEN) 27 | p.statements() 28 | p.want(FI) 29 | } 30 | 31 | * Worth writing by hand? 32 | 33 | Pros: 34 | 35 | - Human-readable and hackable code 36 | - One less tool to learn 37 | - Easier to give better error messages 38 | 39 | function foo(bar) { ... } 40 | 41 | 1: Syntax error: word unexpected (expecting ")") 42 | 1:10: functions must start like "foo()" 43 | 44 | Cons: 45 | 46 | - Incredibly easy to screw up 47 | - Large initial investment of time 48 | - Can't rely on formal definition of grammar 49 | 50 | * Avoiding pitfalls with Go tools 51 | 52 | A parser is incredibly easy to mess up. Thankfully, we have tools: 53 | 54 | - Tests 55 | - Code coverage 56 | - go-fuzz 57 | - (more?) 58 | 59 | * I want to write my own parser! 60 | 61 | - What language for...? 62 | - I know, Shell! 63 | 64 | Chosen because... 65 | 66 | - No grammar available (?) 67 | - No full parser written as a library or independent program 68 | - Widely used language 69 | 70 | First goal: `gofmt` for the shell, `shfmt` 71 | 72 | * Result 73 | 74 | .link https://github.com/mvdan/sh 75 | 76 | - Supports POSIX Shell and Bash 77 | - parser: 2400 lines of code 78 | - printer: 1100 lines of code 79 | - 9 months of intermittent weekend work 80 | 81 | go get github.com/mvdan/sh/cmd/shfmt 82 | 83 | If you can find a shell program it can't parse or format properly, let 84 | me know! 85 | 86 | * One shell format to rule them all 87 | 88 | Indentation is the only flag 89 | 90 | foo >bar & 91 | 92 | if foo; then 93 | echo $(bar) 94 | fi 95 | 96 | foo() { 97 | bar 98 | } 99 | 100 | foo \ 101 | && bar \ 102 | && more 103 | 104 | * But is Bash really a sane language? 105 | 106 | { foo; } 107 | 108 | {foo; } 109 | 110 | foo() { bar; } 111 | 112 | foo() bar 113 | 114 | foo() { bar; } & 115 | 116 | * It gets worse! 117 | 118 | (foo) $(bar) $((1 + 2)) 119 | 120 | $((echo foo | bar)) 121 | 122 | $((echo foo) | bar) 123 | -------------------------------------------------------------------------------- /2016/state-machine.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mvdan/talks/ae2e20e036e2f1552f8a01fee39f2a1d0a544c8b/2016/state-machine.png -------------------------------------------------------------------------------- /2017/gogrep.slide: -------------------------------------------------------------------------------- 1 | Searching Go code - gogrep 2 | 5 Dec 2017 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | 8 | * Go code at different levels 9 | 10 | - raw bytes 11 | - lines 12 | - lexer tokens 13 | - syntax tree (AST) 14 | - syntax tree with type info 15 | - static single assignment (SSA) form 16 | 17 | * Raw bytes 18 | 19 | { 20 | [...] 21 | if err != nil { 22 | return nil, err 23 | } 24 | } 25 | 26 | -- 27 | 28 | \tif err != nil {\n\t\treturn nil, err\n\t}\n 29 | 30 | * Lines 31 | 32 | " if err != nil {" 33 | " return nil, err" 34 | " }" 35 | 36 | * Lexer tokens 37 | 38 | IF IDENT("err") NEQ IDENT("nil") LBRACE 39 | IDENT("return") IDENT("nil") COMMA IDENT("err") 40 | RBRACE 41 | 42 | * Syntax tree 43 | 44 | IfStmt{ 45 | Cond: BinaryExpr{ 46 | X: Ident{Name: "err"}, 47 | Op: token.NEQ, 48 | Y: Ident{Name: "nil"}, 49 | }, 50 | Body: BlockStmt{List: []Stmt{ 51 | ReturnStmt{Results: []Expr{ 52 | Ident{Name: "nil"}, 53 | Ident{Name: "err"}, 54 | }}, 55 | }}, 56 | } 57 | 58 | * With type info 59 | 60 | TypeOf(Ident{Name: "err"}): builtin error 61 | TypeOf(Ident{Name: "nil"}): untyped nil 62 | 63 | - You can also get where err was declared 64 | 65 | * Go code at different levels 66 | 67 | - raw bytes 68 | - lines 🡐 *grep* 69 | - lexer tokens 70 | - syntax tree (AST) 71 | - syntax tree with type info 72 | - static single assignment (SSA) form 73 | 74 | mostly works thanks to gofmt (spaces count!) 75 | 76 | * Plaintext 77 | 78 | $ git grep -n 'if err != nil {' 79 | load.go:38: if err != nil { 80 | load.go:56: if err != nil { 81 | load.go:116: if err != nil { 82 | main.go:71: if err != nil { 83 | main.go:141: if err != nil { 84 | main.go:146: if err != nil { 85 | main.go:156: if err != nil { 86 | main.go:207: if err != nil { 87 | main.go:324: if err != nil { 88 | 89 | * Complex searches 90 | 91 | if != nil { 92 | return [...], 93 | } 94 | 95 | -- 96 | 97 | if $x != nil { 98 | return $*_, $x 99 | } 100 | 101 | Would match all of: 102 | 103 | if err != nil { return err } 104 | if err != nil { return nil, err } 105 | if err2 != nil { return nil, err2 } 106 | 107 | * Demo time! 108 | 109 | * Go code at different levels 110 | 111 | - raw bytes 112 | - lines 🡐 *grep* 113 | - lexer tokens 114 | - syntax tree (AST) 115 | - syntax tree with type info 🡐 *gogrep* 116 | - static single assignment (SSA) form 117 | 118 | * Multiple statements 119 | 120 | $*_, $x := $_ 121 | if $x != nil { 122 | return $*_, $x 123 | } 124 | 125 | -- 126 | 127 | gogrep '$*_, err := $_; if err != nil { return $*_, $x }' 128 | 129 | * Filtering 130 | 131 | What about finding the ones where we panic? 132 | 133 | $*_, $x := $_ 134 | if $x != nil { 135 | // a panic somewhere 136 | } 137 | 138 | -- 139 | 140 | gogrep -x '$*_, err := $_; if err != nil { $*_ }' -g 'panic($*_)' 141 | 142 | * Negative filtering 143 | 144 | What about finding the ones where we don't return? 145 | 146 | $*_, $x := $_ 147 | if $x != nil { 148 | // a panic somewhere 149 | } 150 | 151 | -- 152 | 153 | gogrep -x '$*_, err := $_; if err != nil { $*_ }' -v 'return $*_' 154 | 155 | * Real use case™ 156 | 157 | - golang/go#18625: compress/flate: Write() causes large and unshrinkable stack growth 158 | 159 | for i, v := range someArray { 160 | [...] 161 | } 162 | 163 | -- 164 | 165 | for i, v := range someArray[:] { 166 | [...] 167 | } 168 | 169 | .link https://github.com/mdempsky/rangerdanger 170 | 171 | * Real use case™ #2 172 | 173 | - Simplify HasPrefix with TrimPrefix 174 | 175 | if strings.HasPrefix(x, s) { 176 | x = x[len(s):] 177 | } 178 | 179 | -- 180 | 181 | x = strings.TrimPrefix(x, s) 182 | 183 | .link https://staticcheck.io/docs/gosimple#S1017 184 | 185 | gogrep 'if strings.HasPrefix($x, $p) { $x = $x[len($p):] }' 186 | 187 | * Aggressive AST matching 188 | 189 | - AST vs SSA 190 | 191 | if (a == b) { ... } | if a == b { ... } 192 | for _ = range x { ... } | for range x { ... } 193 | _, _ = f() | f() 194 | var (a int) | var a int 195 | var a int; a = f() | a := f() 196 | 197 | -- 198 | 199 | gogrep '~ var $_ int' 200 | 201 | Would find *foo* in all of: 202 | 203 | var foo int 204 | var (foo int) 205 | var (foo, bar int) 206 | var (bar uint; foo int) 207 | 208 | * Tradeoffs 209 | 210 | Simple, stupid Complex 211 | -------------------------------------------------------------------------------- 212 | grep literal AST matching gogrep ??? full linter 213 | 214 | - Many commands/operations (range, filter, etc) 215 | - Combine multiple commands 216 | - Use type information 217 | 218 | - Never reaching custom logic nor arbitrary code 219 | 220 | * Code search in PMD 221 | 222 | .link https://pmd.github.io/pmd-5.8.1/pmd-java/rules/java/basic.html#CollapsibleIfStatements 223 | 224 | //IfStatement[@Else='false']/Statement 225 | /IfStatement[@Else='false'] 226 | | 227 | //IfStatement[@Else='false']/Statement 228 | /Block[count(BlockStatement)=1]/BlockStatement 229 | /Statement/IfStatement[@Else='false'] 230 | 231 | -- 232 | 233 | if $_ { 234 | if $_ { 235 | $*_ 236 | } 237 | } 238 | 239 | * Bonus points: gofmt -r 240 | 241 | gofmt -r 'pattern -> replacement' 242 | 243 | -- 244 | 245 | gofmt -l -w -r 'strings.TrimPrefix(x, s) -> strings.TrimSuffix(x, s)' 246 | 247 | - Pattern and replacement can only be expressions 248 | - Only matches ASTs with variables 249 | - Cannot compose commands to do complex queries 250 | 251 | gogrep -x 'if strings.HasPrefix($x, $p) { $x = $x[len($p):] }' 252 | -r '$x = strings.TrimPrefix($x, $p)' 253 | 254 | * Open questions 255 | 256 | - What commands should be added? 257 | - There are no "AND" and "OR" commands 258 | - Allow more custom logic, like: 259 | - filter matches containing at least three return statements 260 | 261 | -- 262 | 263 | Can a more powerful and generic tool exist? 264 | -------------------------------------------------------------------------------- /2017/goreduce.slide: -------------------------------------------------------------------------------- 1 | Reducing Go programs 2 | 17 Oct 2017 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | https://github.com/mvdan 7 | 8 | * Reducing a program 9 | 10 | .play goreduce1a.go /^func main/,/^}/ 11 | 12 | can be reduced to 13 | 14 | .play goreduce1b.go /^func main/,/^}/ 15 | 16 | * But why? 17 | 18 | .link https://github.com/dvyukov/gosmith 19 | 20 | - 50+ compiler bugs, 3 spec bugs 21 | 22 | package p 23 | 24 | var Var3629 = [2]int{} 25 | var Var3555 = [0]int{} 26 | var Var2673 = [][]int{} 27 | var Var2515 = (([...][]func(float64,complex64,error,byte) func([][]float64,[0]interface { Method2486 (complex64,complex64) (interface{},uint,byte,string) 28 | },uint,map[error]complex64) [2][][]*[2]chan int{[]func(float64,complex64,error,byte) func([][]float64,[0]interface { Method2486 (complex64,complex64) (interface{},uint,byte,string) 29 | },uint,map[error]complex64) [2][][]*[2]chan int{},[]func(float64,complex64,error,byte) func([][]float64,[0]interface { Method2486 (complex64,complex64) (interface{},uint,byte,string) 30 | },uint,map[error]complex64) [2][][]*[2]chan int{}})[(((func(error,uintptr,chan [1]bool,float32) struct { Field2544 int 31 | })(nil))(error(nil),uintptr(0),make(chan [1]bool ),float32(1.0))).Field2544])[((Var2673)[(int)(((((*((*((*((([][0]***[0][1][1]int16{})[(Var3555)[((struct { Field3609 int 32 | }{})).Field3609]])[((func(int16) int)(nil))(int16(1))])))))))[(<- make(chan int ,1))])[(((([][1][1]int{})[(<- make(chan int ,1))])[(Var3629)[(make(map[[1]bool]int ,1))[[1]bool{}]]])[0]) + (1)])[(<- make(chan int ))] ,)])[(<- make(chan int ))]](1.0,complex64(1i),error(nil),byte(0)) 33 | 34 | .link https://github.com/golang/go/issues/13277 35 | 36 | - Idea came to exist for C compilers: 37 | 38 | .link https://github.com/csmith-project/csmith 39 | 40 | * How csmith/gosmith generate programs 41 | 42 | - They are language-specific 43 | 44 | - They modify ASTs, not bytes (source code) 45 | 46 | - Thus, they never make syntax errors 47 | 48 | - They run compilers on the programs to search for bugs 49 | 50 | - They can be smart and arrive at a crasher faster 51 | 52 | .link https://github.com/dvyukov/go-fuzz 53 | 54 | * But soon... 55 | 56 | From a C compiler bug report: 57 | 58 | (*l_1209) ^= func_2(func_7(g_13, g_13.f0, g_13.f0, 59 | func_14(func_20(l_25, ((*l_29)--), func_32(func_34((((((*l_963) = 60 | func_37((safe_unary_minus_func_int32_t_s(( 61 | safe_rshift_func_uint16_t_u_s((safe_add_func_uint8_t_u_u(func_47( 62 | l_25), (safe_mul_func_uint16_t_u_u(((safe_add_func_uint64_t_u_u((( 63 | *l_102) = 0xC3DD168C87B4EC18LL), (safe_mul_func_int16_t_s_s(((( 64 | g_117 = (safe_sub_func_int16_t_s_s(((safe_lshift_func_int16_t_s_u(( 65 | g_116 = (g_115 = ((*l_112) ^= (((g_76.f1.f1 , g_110[2]) != &g_111) , 66 | 0xF5F5L)))), l_25)) || g_76.f1.f1), g_76.f1.f2))) < l_118) >= (-8L)), 67 | g_65)))) < l_25), g_13.f0)))), g_76.f1.f3)))), g_119, l_26[1], &g_65)) 68 | == g_964) == 0x24AE7014CC3713C7LL) | l_25), g_966)), g_415), 69 | l_988, g_967, l_118, l_25), l_25), l_118, (*g_676), l_988); 70 | 71 | - Manually reducing these is insane. 72 | 73 | * How does a human reduce manually? 74 | 75 | - Make a change to a program, reducing its size or complexity 76 | 77 | - If it no longer reproduces the bug, undo and retry 78 | 79 | - Otherwise, keep the change and continue 80 | 81 | - Stop when we're happy or bored to death 82 | 83 | * And creduce was born 84 | 85 | .link https://github.com/csmith-project/creduce 86 | 87 | typedef volatile int vint; 88 | vint **depth; 89 | int *b; 90 | vint **get_depth (void) { 91 | return depth; 92 | } 93 | int fn1 (int inc) { 94 | int tmp = 0; 95 | if (get_depth() == &b) 96 | tmp = inc + **depth; 97 | return tmp; 98 | } 99 | 100 | Reduced to: 101 | 102 | volatile int **a; 103 | int *b; 104 | void fn1() { 105 | if (a == &b) **a; 106 | } 107 | 108 | * So here we are 109 | 110 | - Go has a gosmith, but no goreduce 111 | 112 | - I seem to have too much free time on my hands 113 | 114 | - How hard can it be? 115 | 116 | - (surprise: actually hard to get right) 117 | 118 | * Simple rules 119 | 120 | - Remove a statement: `st1; st2 -> `st1` 121 | 122 | - Zero literal values: `"foo" -> ""` 123 | 124 | - Bypass if: `if a { b } -> b` 125 | 126 | And so on. 127 | 128 | * First working example 129 | 130 | .play goreduce2a.go /^func main/,/^}/ 131 | 132 | goreduce -match 'index out of range' . Crasher 133 | 134 | .play goreduce2b.go /^func main/,/^}/ 135 | 136 | * Pain points 137 | 138 | - Had to fork go/ast.Walk (need to modify parent and slices) 139 | 140 | - Undoing changes to an AST isn't as easy as it sounds 141 | 142 | - Go compiler is much grumpier than C ones 143 | 144 | - Calls and imports can have side effects 145 | 146 | * Work ahead 147 | 148 | .link https://github.com/mvdan/goreduce 149 | 150 | - Constant resolution (even appends!) 151 | 152 | - Inline single-use vars 153 | 154 | - Inline single-use funcs (args, returns, name clashes... ugh) 155 | 156 | - Be lazy like a human (remove half of the statements) 157 | 158 | - Avoid changes that make the type system angry 159 | 160 | - Be used in the Go project for compiler bug reports 161 | 162 | Other open questions: 163 | 164 | - Tests are unbearably slow 165 | 166 | - Avoid calling 'go'? 167 | -------------------------------------------------------------------------------- /2017/goreduce1a.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | if 3 == 3 { 5 | panic("crash!") 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /2017/goreduce1b.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | panic("crash!") 5 | } 6 | -------------------------------------------------------------------------------- /2017/goreduce2a.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | a := []int{1, 2, 3} 5 | if true { 6 | a = append(a, 4) 7 | } 8 | a[1] = -2 9 | println(a[10]) 10 | } 11 | -------------------------------------------------------------------------------- /2017/goreduce2b.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | func main() { 4 | a := []int{} 5 | println(a[0]) 6 | } 7 | -------------------------------------------------------------------------------- /2017/helloworld.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | func main() { 6 | fmt.Println("Hello, 世界") 7 | } 8 | -------------------------------------------------------------------------------- /2017/intro-to-go.slide: -------------------------------------------------------------------------------- 1 | Why you should hack in Go 2 | 03 Mar 2017 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | https://github.com/mvdan 7 | 8 | * Why Go exists 9 | 10 | - Simple language 11 | - Fast compilation 12 | - Built-in concurrency 13 | 14 | * Hello World 15 | 16 | .play helloworld.go 17 | 18 | * First impressions 19 | 20 | .play helloworld.go 21 | 22 | - Similar syntax to C/C++/Java 23 | - UTF-8 out of the box 24 | - Package-based 25 | - No semicolons (🙏) 26 | 27 | * Really, really simple 28 | 29 | - `func`, `if`, `for`, `switch` 30 | - `type`, `struct`, `interface` 31 | - `var`, `const`, `map` 32 | 33 | Learn 90% of the language in two days! 34 | 35 | * Concurrency is hard? 36 | 37 | .play helloworld.go 38 | 39 | * Who uses Go? 40 | 41 | - Docker 42 | - Google (Kubernetes, ...) 43 | - Cloudflare 44 | - Netflix 45 | - Uber 46 | - Dropbox 47 | 48 | * Let's have some fun 49 | 50 | Do any of you know what sleep sort is? 51 | -------------------------------------------------------------------------------- /2018/benchstat.slide: -------------------------------------------------------------------------------- 1 | Optimizations in Go are easy! 2 | 04 Apr 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | 8 | * Quiz time! 9 | 10 | func main() { 11 | v := new(int) 12 | err = json.Unmarshal([]byte("null"), &v) 13 | println(v, err != nil) 14 | } 15 | 16 | * Quiz time! 17 | 18 | func main() { 19 | b := true 20 | err := json.Unmarshal([]byte("null"), &b) 21 | println(b, err != nil) 22 | } 23 | 24 | 25 | * Let's start with memory allocations 26 | 27 | var outputs []foo 28 | for _, input := range inputs { 29 | outputs = append(outputs, fn(input)) 30 | ) 31 | 32 | -- 33 | 34 | outputs := make([]foo, 0, len(inputs)) 35 | for _, input := range inputs { 36 | outputs = append(outputs, fn(input)) 37 | ) 38 | 39 | * But sometimes it just isn't that simple 40 | 41 | var s1 = "foo" 42 | var s2 = "bar" 43 | 44 | func f1() { 45 | Sink = s1 + "," + s2 46 | } 47 | 48 | func f2() { 49 | Sink = fmt.Sprintf("%s,%s", s1, s2) 50 | } 51 | 52 | 53 | * Can we measure this? 54 | 55 | func Benchmark1(b *testing.B) { 56 | for i := 0; i < b.N; i++ { 57 | f1() 58 | } 59 | } 60 | 61 | func Benchmark2(b *testing.B) { 62 | for i := 0; i < b.N; i++ { 63 | f2() 64 | } 65 | } 66 | 67 | --- 68 | 69 | $ go test -bench=. -benchmem 70 | Benchmark1-4 30000000 54.5 ns/op 8 B/op 1 allocs/op 71 | Benchmark2-4 10000000 214 ns/op 40 B/op 3 allocs/op 72 | 73 | * Another example 74 | 75 | var n = 200 76 | 77 | func f1() { 78 | list := make([]int, 0, n) 79 | for i := 0; i < n; i++ { 80 | list = append(list, i) 81 | } 82 | } 83 | 84 | func f2() { 85 | list := make([]int, 0, 200) 86 | for i := 0; i < n; i++ { 87 | list = append(list, i) 88 | } 89 | } 90 | 91 | --- 92 | 93 | $ go test -bench=. -benchmem 94 | Benchmark1-4 3000000 455 ns/op 1792 B/op 1 allocs/op 95 | Benchmark2-4 10000000 174 ns/op 0 B/op 0 allocs/op 96 | 97 | * Enter benchcmp 98 | 99 | $ go test -bench=. >old.txt 100 | # make changes 101 | $ go test -bench=. >new.txt 102 | 103 | $ benchcmp old.txt new.txt 104 | benchmark old ns/op new ns/op delta 105 | BenchmarkConcat 523 68.6 -86.88% 106 | 107 | benchmark old allocs new allocs delta 108 | BenchmarkConcat 3 1 -66.67% 109 | 110 | benchmark old bytes new bytes delta 111 | BenchmarkConcat 80 48 -40.00% 112 | 113 | * But CPU time isn't as stable as the number of allocs... 114 | 115 | benchmark old ns/op new ns/op delta 116 | BenchmarkParse-4 530 520 -1.89% 117 | 118 | benchmark old ns/op new ns/op delta 119 | BenchmarkParse-4 530 542 +2.26% 120 | 121 | It's not unusual to see a variance of a few percent in real/user time 122 | measurements. 123 | 124 | * Enter benchstat 125 | 126 | $ go test -bench=. >old.txt 127 | # make changes 128 | $ go test -bench=. >new.txt 129 | 130 | $ benchstat old new 131 | name old time/op new time/op delta 132 | Parse-4 530ns ± 0% 520ns ± 0% ~ (p=1.000 n=1+1) 133 | 134 | * You need much more data to get reliable results 135 | 136 | $ go test -bench=. -count=5 >old.txt 137 | # make changes 138 | $ go test -bench=. -count=5 >new.txt 139 | 140 | BenchmarkParse-4 500000 530 ns/op 141 | BenchmarkParse-4 500000 534 ns/op 142 | BenchmarkParse-4 500000 528 ns/op 143 | BenchmarkParse-4 500000 542 ns/op 144 | BenchmarkParse-4 500000 536 ns/op 145 | 146 | BenchmarkParse-4 500000 528 ns/op 147 | BenchmarkParse-4 500000 524 ns/op 148 | BenchmarkParse-4 500000 515 ns/op 149 | BenchmarkParse-4 500000 532 ns/op 150 | BenchmarkParse-4 500000 526 ns/op 151 | 152 | * You need much more data to get reliable results 153 | 154 | $ benchstat old new 155 | name old time/op new time/op delta 156 | Parse-4 531ns ± 1% 524ns ± 1% ~ (p=0.200 n=3+3) 157 | 158 | $ benchstat old new 159 | name old time/op new time/op delta 160 | Parse-4 534ns ± 1% 525ns ± 2% -1.69% (p=0.040 n=5+5) 161 | 162 | * Let's play with some CPU performance examples 163 | 164 | func f1() { 165 | list := make([]int, 0, 4) 166 | list = append(list, 123) 167 | Sink = list 168 | } 169 | 170 | func f2() { 171 | list := make([]int, 1, 4) 172 | list[0] = 123 173 | Sink = list 174 | } 175 | 176 | BenchmarkF-4 50000000 34.6 ns/op 32 B/op 1 allocs/op 177 | 178 | BenchmarkF-4 50000000 35.1 ns/op 32 B/op 1 allocs/op 179 | 180 | * Results 181 | 182 | No peeking! 183 | 184 | * Results 185 | 186 | $ benchstat old new 187 | name old time/op new time/op delta 188 | F-4 34.8ns ± 1% 35.1ns ± 2% ~ (p=0.143 n=5+5) 189 | 190 | name old alloc/op new alloc/op delta 191 | F-4 32.0B ± 0% 32.0B ± 0% ~ (all equal) 192 | 193 | name old allocs/op new allocs/op delta 194 | F-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) 195 | 196 | * One last example 197 | 198 | var Sink int 199 | var n = 20 200 | 201 | func f1() { 202 | list := make([]int, 0, n) 203 | for i := 0; i < n; i++ { 204 | list = append(list, i) 205 | } 206 | for _, x := range list { 207 | Sink = x 208 | } 209 | } 210 | 211 | func f2() { 212 | list := make([]int, 0, 64) 213 | for i := 0; i < n; i++ { 214 | list = append(list, i) 215 | } 216 | for _, x := range list { 217 | Sink = x 218 | } 219 | } 220 | 221 | * Results 222 | 223 | No peeking! 224 | 225 | * Results 226 | 227 | $ benchstat old new 228 | name old time/op new time/op delta 229 | F-4 78.8ns ± 1% 37.6ns ± 1% -52.35% (p=0.008 n=5+5) 230 | 231 | name old alloc/op new alloc/op delta 232 | F-4 160B ± 0% 0B -100.00% (p=0.008 n=5+5) 233 | 234 | name old allocs/op new allocs/op delta 235 | F-4 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) 236 | 237 | * General advice for better benchstat results 238 | 239 | https://github.com/golang/go/issues/23471 240 | 241 | - Using an idle machine 242 | - High `-count` values (10, 20, 30) 243 | - Using `-benchmem` to get memory info 244 | - Using `-run=-` or `-run=^$` to avoid wasting time on tests 245 | - Any more ideas? Please contribute to the issue above! 246 | -------------------------------------------------------------------------------- /2018/go1.11.slide: -------------------------------------------------------------------------------- 1 | What else is in Go 1.11? 2 | 27 Jun 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | 7 | * Go 1.11 8 | 9 | - WebAssembly and Modules have already been covered. 10 | 11 | - Now onto the changes that won't make headlines! 12 | 13 | Sources: 14 | 15 | .link https://tip.golang.org/doc/go1.11 16 | .link https://dev.golang.org/release#Go1.11 17 | .link https://github.com/golang/go/issues?q=is:open+is:issue+milestone:Go1.11 18 | 19 | * Ports 20 | 21 | Dropped: 22 | 23 | - Windows XP and Vista (300+ lines removed) 24 | 25 | - OSX 10.9 and earlier (120+ lines removed) 26 | 27 | - OpenBSD 6.1 and earlier 28 | 29 | Minor additions, like `-race` on `linux/ppc64le` and `-msan` on `linux/arm64`. 30 | 31 | `riscv` and `riscv64` reserved as `GOARCH` values reserved for the future. 32 | 33 | * Tooling (besides modules) 34 | 35 | - Last release to support *GOCACHE=off* 36 | 37 | - *go*test* now runs *go*vet* by default 38 | 39 | - *go*vet* now requires its input packages to typecheck 40 | 41 | - Last release where *godoc* has a command-line interface 42 | 43 | * Tooling #2 - x/tools/go/packages 44 | 45 | A replacement for `x/tools/go/loader` with several advantages: 46 | 47 | - Support for Modules - critical for third party tools 48 | 49 | - Support for patterns, like `./...` 50 | 51 | - Calls out to *go*list* to find packages 52 | 53 | - Support for build systems like Bazel 54 | 55 | - Support loading dependencies via `GOCACHE` 56 | 57 | * Tooling #3 - gofmt 58 | 59 | var _ = T{ 60 | F1: 1, 61 | F2: 1, 62 | VeryLongNameJustBecause: 1, 63 | F3: 1, 64 | } 65 | 66 | The tweaked heuristic now gives us: 67 | 68 | var _ = T{ 69 | F1: 1, 70 | F2: 1, 71 | VeryLongNameJustBecause: 1, 72 | F3: 1, 73 | } 74 | 75 | * Tooling #4 - godoc versions for std 76 | 77 | .image go1.11_godoc_versions.png _ 900 78 | 79 | * Tooling #5 - debugging 80 | 81 | - Experimental support for calling Go functions in a debugger 82 | 83 | Optimized binaries now include more accurate info, like: 84 | 85 | - Variable locations 86 | 87 | - Line numbers 88 | 89 | - Breakpoint locations 90 | 91 | DWARF sections (debugging info) are now compressed by default 92 | 93 | * Runtime 94 | 95 | - Now uses a sparse heap, so the 512GiB limit is gone 96 | 97 | - Kernel calls on macOS and iOS now go through `libSystem.so` 98 | 99 | - This improves Go's compatibility with future macOS and iOS versions 100 | 101 | * Compiler #1 - indexed export format 102 | 103 | - The old format was sequential - entire packages had to be loaded 104 | 105 | - The new format is indexed, so the compiler only loads what it needs 106 | 107 | - Especially important for large projects and packages 108 | 109 | -- 110 | 111 | Juju/c=4/l=4 46.3s ± 4% 38.0s ± 4% -18.06% (p=0.001 n=7+7) 112 | Kubelet/c=4/l=4 48.1s ± 2% 39.0s ± 5% -18.93% (p=0.002 n=6+6) 113 | 114 | * Compiler #2 - unused type switch variables 115 | 116 | - `gccgo` and `go/types` already errored here 117 | 118 | - The compiler now does too, for consistency 119 | 120 | func f(v interface{}) { 121 | switch x := v.(type) { 122 | } 123 | } 124 | 125 | * Compiler #3 - inlining function calls 126 | 127 | - Funcs that call `panic` can now be inlined 128 | 129 | `-l=4` makes the inlining more agressive, also enabling mid-stack inlining 130 | 131 | - `-l=4` has been tweaked and improved 132 | 133 | - However, `-l=4` still makes some programs larger and slower 134 | 135 | - The heuristic needs more work for mid-stack inlining to be the default 136 | 137 | * Compiler #4 - map clearing idiom 138 | 139 | for k := range m { 140 | delete(m, k) 141 | } 142 | 143 | - Reuses the allocated memory for the map 144 | 145 | - Now skips the expensive range when possible 146 | 147 | GoMapClear/Reflexive/1 92.2ns ± 1% 47.1ns ± 2% -48.89% (p=0.000 n=9+9) 148 | GoMapClear/Reflexive/10 108ns ± 1% 48ns ± 2% -55.68% (p=0.000 n=10+10) 149 | GoMapClear/Reflexive/100 303ns ± 2% 110ns ± 3% -63.56% (p=0.000 n=10+10) 150 | GoMapClear/Reflexive/1000 3.58µs ± 3% 1.23µs ± 2% -65.49% (p=0.000 n=9+10) 151 | GoMapClear/Reflexive/10000 28.2µs ± 3% 10.3µs ± 2% -63.55% (p=0.000 n=9+10) 152 | 153 | * Compiler #5 - slice extension 154 | 155 | append(s, make([]T, n)...) 156 | 157 | - Simpler than manually allocating a new slice and copying 158 | 159 | - Avoids an allocation if there's enough capacity 160 | 161 | - If a new backing array is needed, avoids clearing memory twice 162 | 163 | ExtendSlice/IntSlice 103ns ± 4% 57ns ± 4% -44.55% (p=0.000 n=18+18) 164 | ExtendSlice/PointerSlice 155ns ± 3% 77ns ± 3% -49.93% (p=0.000 n=20+20) 165 | ExtendSlice/NoGrow 50.2ns ± 3% 5.2ns ± 2% -89.67% (p=0.000 n=18+18) 166 | 167 | * Compiler #6 - prove pass 168 | 169 | The prove pass derives facts from code, to be used to delete unnecessary 170 | branches and bounds checks. 171 | 172 | Most importantly, it now recognizes transitive relations: 173 | 174 | - Inside *if*n*<*10*{}*, it can prove *n*<*10* 175 | 176 | - After *s*:=*make([]int,*20)*, it can prove *len(s)*==*20* 177 | 178 | - Globally, *10*<*20*, so *n*<*len(s)* 179 | 180 | - *s*:=*make([]int,*20);*if*n*<*10*{*_*=*s[n]*}* can never panic 181 | 182 | The bounds check is what panics if the index is out of bounds, so in this case 183 | it can be removed. 184 | 185 | * Standard library highlights #1 186 | 187 | Let's begin with some of the most visible changes: 188 | 189 | - Added `os.UserCacheDir`; `$HOME/.cache` on most Unix systems 190 | 191 | - `os/user` adds a `osusergo` build tag use pure Go without `CGO_ENABLED=0` 192 | 193 | - `time` now accepts parsing numeric timezones like `+03` 194 | 195 | - `net/http` adds support for CIDR and ports in `NO_PROXY`, like `NO_PROXY=10.0.0.0/8` 196 | 197 | - `net/http/httputil.ReverseProxy` gained an `ErrorHandler` 198 | 199 | - Some `crypto` funcs now randomly read an extra byte 200 | 201 | * Standard library highlights #2 202 | 203 | - `text/template` can now modify variables via the `=` token: 204 | 205 | {{ $v := "init" }} 206 | {{ if true }} 207 | {{ $v = "changed" }} 208 | {{ end }} 209 | v: {{ $v }} {{/* "changed" */}} 210 | 211 | - `io/ioutil.TempFile` can now be told where to put the random characters: 212 | 213 | ioutil.TempFile("", "foo-") // /tmp/foo-123456 214 | ioutil.TempFile("", "foo-*.txt") // /tmp/foo-123456.txt 215 | 216 | * Standard library highlights #3 217 | 218 | What about performance? 219 | 220 | - Some packages were optimized for `arm64`; especially crypto and byte handling 221 | 222 | - Pieces of `math/big` were rewritten to be much faster 223 | 224 | - Copying data between TCP connections is faster on Linux via the `splice` syscall 225 | 226 | - The mutex profile now includes reader/writer contention for `sync.RWMutex` 227 | -------------------------------------------------------------------------------- /2018/go1.11_godoc_versions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mvdan/talks/ae2e20e036e2f1552f8a01fee39f2a1d0a544c8b/2018/go1.11_godoc_versions.png -------------------------------------------------------------------------------- /2018/go1.12-pre.slide: -------------------------------------------------------------------------------- 1 | What's coming in Go 1.12 2 | 01 Nov 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | 7 | * Go 1.11 8 | 9 | - Modules 10 | - WebAssembly 11 | - XP dropped 12 | - `go`test` runs `go`vet` 13 | 14 | * Go 1.12 15 | 16 | - Freeze on November 5th 17 | - Release early February 2019 18 | 19 | But what sources? 20 | 21 | .link https://github.com/golang/go/issues?q=is:issue+milestone:Go1.12 22 | .link https://github.com/golang/go/commits/master 23 | .link https://go-review.googlesource.com/q/status:merged+RELNOTE%253Dyes 24 | 25 | * Let's start with merged proposals 26 | 27 | They tend to be more interesting. 28 | 29 | * fmt: print maps in sorted order 30 | 31 | .link https://github.com/golang/go/issues/21095 32 | 33 | - Easier debugging 34 | - fmt output is now deterministic 35 | 36 | * testing: add -benchtime=100x 37 | 38 | .link https://github.com/golang/go/issues/24735 39 | 40 | - Run a test N times, instead of for N seconds 41 | - Useful to compare before/after numbers better 42 | 43 | * cmd/godoc: remove CLI support 44 | 45 | .link https://github.com/golang/go/issues/25443 46 | 47 | - You should now use `go`doc` instead of `godoc` 48 | - One more step towards bringing the doc tools in sync 49 | 50 | * strings: add Builder.Cap method 51 | 52 | .link https://github.com/golang/go/issues/26269 53 | 54 | - Mirroring bytes.Buffer.Cap 55 | - Otherwise impossible to tell capacity or free space 56 | 57 | * os: add ProcessState.ExitCode method 58 | 59 | .link https://github.com/golang/go/issues/26539 60 | 61 | - Easier to get a process's exit code 62 | - No need to type assert to a syscall type 63 | 64 | * crypto/rand: warn if blocked 60s+ on a read 65 | 66 | .link https://github.com/golang/go/issues/22614 67 | 68 | - Otherwise programs might hang with no apparent reason 69 | - Makes the issue much easier to debug 70 | 71 | * Onto accepted but still open proposals 72 | 73 | Some of these might be pushed back to 1.13. 74 | 75 | * cmd/go: record language version to support language transitions 76 | 77 | .link https://github.com/golang/go/issues/28221 78 | 79 | - Allow changing the Go language progressively 80 | - Smooth transition for users without 2.0, 3.0, etc 81 | 82 | * cmd/go: add a build -static flag 83 | 84 | .link https://github.com/golang/go/issues/26492 85 | 86 | - Building a fully static binary is *not* trivial 87 | - Make it trivial forever 88 | 89 | * bytes,strings: add ToValidUTF8 90 | 91 | .link https://github.com/golang/go/issues/25805 92 | 93 | - To remove invalid bytes, devs do `string([]rune(s))` 94 | - The compiler may get smart enough to make that a no-op 95 | 96 | * testing: stream t.Log output as it happens 97 | 98 | .link https://github.com/golang/go/issues/24929 99 | 100 | - Log messages have to wait until the test finishes 101 | - Makes slow/hanging tests hard to debug 102 | 103 | * runtime: non-cooperative goroutine preemption 104 | 105 | .link https://github.com/golang/go/issues/24543 106 | 107 | - HUGE deal for large companies 108 | - Possible for certain code patterns to bully the runtime 109 | - Redesign the runtime to remove those edge cases 110 | 111 | * cmd/compile: enable mid-stack inlining 112 | 113 | .link https://github.com/golang/go/issues/19348 114 | 115 | - Has been shelved every release since 1.9 116 | - Likely won't make it into 1.12, very tricky 117 | - Would mean considerable speed-ups for most programs 118 | 119 | * Now onto random stuff 120 | 121 | Randomly picked from what I've seen and found. 122 | 123 | * io: add StringWriter interface 124 | 125 | .link https://github.com/golang/go/issues/27946 126 | 127 | - Lots of copy-paste definitions are no more 128 | 129 | * cmd/go: add GOFLAGS environment variable 130 | 131 | .link https://go-review.googlesource.com/c/go/+/126656 132 | 133 | - Easy way to configure executed Go tools 134 | - Can configure global build flags for a user 135 | 136 | * os: add UserHomeDir 137 | 138 | .link https://github.com/golang/go/issues/26463 139 | 140 | - Using os/user.Current directly is actually bad 141 | - $HOME should be tried first 142 | 143 | * cmd/doc: support printing full documentation 144 | 145 | .link https://github.com/golang/go/issues/25595 146 | 147 | - Another feature ported over from `godoc` 148 | 149 | * math/bits: add extended precision Add, Sub, Mul, Div 150 | 151 | .link https://github.com/golang/go/issues/24813 152 | 153 | - Multi-word arithmetic now much easier (uint128, etc) 154 | - Faster than manual code too, thanks to compiler help 155 | 156 | * text/template: catch panics during func calls 157 | 158 | .link https://github.com/golang/go/issues/28242 159 | 160 | - Panics in user code are now returned as errors 161 | - Simpler to use template, friendlier errors to users 162 | 163 | * Many many more issues 164 | 165 | - Modules support is getting better and more stable 166 | - WebAssembly port is improving every week 167 | - More ports incoming - AIX, HURD 168 | -------------------------------------------------------------------------------- /2018/gogrep.slide: -------------------------------------------------------------------------------- 1 | Searching Go code - gogrep 2 | 30 Aug 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | 7 | * Go code at different levels 8 | 9 | - raw bytes 10 | - lines 11 | - lexer tokens 12 | - syntax tree (AST) 13 | - syntax tree with type info 14 | - static single assignment (SSA) form 15 | 16 | * Raw bytes 17 | 18 | { 19 | [...] 20 | if err != nil { 21 | return nil, err 22 | } 23 | } 24 | 25 | -- 26 | 27 | \tif err != nil {\n\t\treturn nil, err\n\t}\n 28 | 29 | * Lines 30 | 31 | " if err != nil {" 32 | " return nil, err" 33 | " }" 34 | 35 | * Lexer tokens 36 | 37 | IF IDENT("err") NEQ IDENT("nil") LBRACE 38 | IDENT("return") IDENT("nil") COMMA IDENT("err") 39 | RBRACE 40 | 41 | * Syntax tree 42 | 43 | IfStmt{ 44 | Cond: BinaryExpr{ 45 | X: Ident{Name: "err"}, 46 | Op: token.NEQ, 47 | Y: Ident{Name: "nil"}, 48 | }, 49 | Body: BlockStmt{List: []Stmt{ 50 | ReturnStmt{Results: []Expr{ 51 | Ident{Name: "nil"}, 52 | Ident{Name: "err"}, 53 | }}, 54 | }}, 55 | } 56 | 57 | * With type info 58 | 59 | TypeOf(Ident{Name: "err"}): builtin error 60 | TypeOf(Ident{Name: "nil"}): untyped nil 61 | 62 | TypeOf(err != nil): bool 63 | 64 | - You can also get where err was declared 65 | 66 | * Go code at different levels 67 | 68 | - raw bytes 69 | - lines 🡐 *grep* 70 | - lexer tokens 71 | - syntax tree (AST) 72 | - syntax tree with type info 73 | - static single assignment (SSA) form 74 | 75 | mostly works thanks to gofmt (spaces count!) 76 | 77 | * Plaintext 78 | 79 | $ git grep -n 'if err != nil {' 80 | load.go:38: if err != nil { 81 | load.go:56: if err != nil { 82 | load.go:116: if err != nil { 83 | main.go:71: if err != nil { 84 | main.go:141: if err != nil { 85 | main.go:146: if err != nil { 86 | main.go:156: if err != nil { 87 | main.go:207: if err != nil { 88 | main.go:324: if err != nil { 89 | 90 | * Complex searches 91 | 92 | if != nil { 93 | return [...], 94 | } 95 | 96 | -- 97 | 98 | if $x != nil { 99 | return $*_, $x 100 | } 101 | 102 | Would match all of: 103 | 104 | if err != nil { return err } 105 | if err != nil { return nil, err } 106 | if err2 != nil { return nil, err2 } 107 | 108 | * Go code at different levels 109 | 110 | - raw bytes 111 | - lines 🡐 *grep* 112 | - lexer tokens 113 | - syntax tree (AST) 114 | - syntax tree with type info 🡐 *gogrep* 115 | - static single assignment (SSA) form 116 | 117 | * Multiple statements 118 | 119 | $*_, $x := $_ 120 | if $x != nil { 121 | return $*_, $x 122 | } 123 | 124 | -- 125 | 126 | $ gogrep -x '$*_, err := $_; if err != nil { return $*_, $x }' 127 | expr_parse.go:20:2: toks, err := m.tokenize([]byte(expr)); if err != nil { return "", nil, fmt.Errorf("cannot tokenize expr: %v", err); } 128 | expr_parse.go:64:2: exprStr, offs, err := m.transformSource(expr); if err != nil { return nil, err; } 129 | expr_parse.go:296:3: wt, err := m.wildcard(t.pos, next); if err != nil { return nil, err; } 130 | expr_parse.go:343:2: toks, err := m.tokenize([]byte(src)); if err != nil { return nil, err; } 131 | expr_parse.go:373:3: rxStr, err := strconv.Unquote(t.lit); if err != nil { return nil, fmt.Errorf("%v: %v", t.pos, err); } 132 | expr_parse.go:403:3: typeExpr, err := parser.ParseExpr(typeStr); if err != nil { return nil, err; } 133 | expr_parse.go:443:2: n, err := strconv.Atoi(s[len(wildPrefix):]); if err != nil { return -1; } 134 | load.go:37:3: f, err := parser.ParseFile(l.fset, path, nil, parser.ParseComments); if err != nil { return err; } 135 | load.go:55:3: pkg, err := l.ctx.Import(path, l.wd, 0); if err != nil { return err; } 136 | main.go:139:2: cmds, paths, err := m.parseCmds(args); if err != nil { return err; } 137 | main.go:221:4: n, err := strconv.Atoi(cmd.src); if err != nil { return nil, nil, err; } 138 | main.go:227:4: m, err := m.parseAttrs(cmd.src); if err != nil { return nil, nil, fmt.Errorf("cannot parse mods: %v", err); } 139 | main.go:233:4: node, err := m.parseExpr(cmd.src); if err != nil { return nil, nil, err; } 140 | 141 | * Filtering 142 | 143 | What about finding the ones where we panic? 144 | 145 | $*_, $x := $_ 146 | if $x != nil { 147 | // a panic somewhere 148 | } 149 | 150 | -- 151 | 152 | $ gogrep -x '$*_, err := $_; if err != nil { $*_ }' -g 'panic($*_)' 153 | match.go:653:3: pkg, err := m.stdImporter.Import(path); if err != nil { panic(fmt.Sprintf("findScope err: %v", err)); } 154 | write.go:36:3: f, err := os.OpenFile(path, os.O_WRONLY|os.O_TRUNC, 0); if err != nil { panic(err); } 155 | 156 | * Negative filtering 157 | 158 | What about finding the ones where we don't return? 159 | 160 | $*_, $x := $_ 161 | if $x != nil { 162 | // missing return 163 | } 164 | 165 | -- 166 | 167 | $ gogrep -x '$*_, err := $_; if err != nil { $*_ }' -v 'return $*_' 168 | main.go:63:2: err := m.fromArgs(os.Args[1:]); if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1); } 169 | match.go:653:3: pkg, err := m.stdImporter.Import(path); if err != nil { panic(fmt.Sprintf("findScope err: %v", err)); } 170 | write.go:36:3: f, err := os.OpenFile(path, os.O_WRONLY|os.O_TRUNC, 0); if err != nil { panic(err); } 171 | write_test.go:51:2: dir, err := ioutil.TempDir("", "gogrep-write"); if err != nil { t.Fatal(err); } 172 | write_test.go:81:3: gotBs, err := ioutil.ReadFile(path); if err != nil { t.Fatal(err); } 173 | 174 | * Real use case™ 175 | 176 | - Simplify HasPrefix with TrimPrefix 177 | 178 | if strings.HasPrefix(x, s) { 179 | x = x[len(s):] 180 | } 181 | 182 | -- 183 | 184 | x = strings.TrimPrefix(x, s) 185 | 186 | .link https://staticcheck.io/docs/gosimple#S1017 187 | 188 | $ gogrep -x 'if strings.HasPrefix($x, $p) { $x = $x[len($p):] }' 189 | 190 | * Real use case™ #2 191 | 192 | - golang/go#18625: compress/flate: Write() causes large and unshrinkable stack growth 193 | 194 | for i, v := range someArray { 195 | [...] 196 | } 197 | 198 | -- 199 | 200 | for i, v := range someArray[:] { 201 | [...] 202 | } 203 | 204 | .link https://github.com/mdempsky/rangerdanger 205 | 206 | * Real use case™ #2 cont 207 | 208 | $ gogrep -x 'for $_, $_ := range $x { $*_ }' -x '$x' -a 'is(array)' std | grep -vE '_test|vendor/' 209 | archive/tar/reader.go:532:22: blk 210 | compress/flate/inflate.go:235:25: h.chunks 211 | crypto/elliptic/p224.go:154:20: minimal 212 | crypto/elliptic/p256_asm.go:340:20: p.xyz 213 | crypto/elliptic/p256_asm.go:341:3: p.xyz 214 | crypto/md5/md5.go:193:20: d.s 215 | crypto/sha1/sha1.go:229:20: d.h 216 | crypto/sha1/sha1.go:249:20: d.h 217 | crypto/tls/common.go:605:20: c.SessionTicketKey 218 | go/build/gc.go:123:21: [...]string{full, full + ".gox", pkgdir + "lib" + pkg + ".so", ... 219 | go/internal/gccgoimporter/importer.go:45:28: [...]string{pkgfullpath, pkgfullpath + ".gox", ... 220 | go/internal/gcimporter/gcimporter.go:73:22: pkgExts 221 | go/types/stmt.go:800:24: lhs 222 | go/types/stmt.go:848:24: lhs 223 | go/types/universe.go:75:20: aliases 224 | go/types/universe.go:99:20: predeclaredConsts 225 | image/gif/writer.go:28:20: log2Lookup 226 | image/jpeg/huffman.go:158:21: nCodes 227 | image/jpeg/reader.go:727:30: translations 228 | image/jpeg/writer.go:205:20: theHuffmanSpec 229 | 230 | * Aggressive AST matching 231 | 232 | - AST vs SSA 233 | 234 | if (a == b) { ... } | if a == b { ... } 235 | for _ = range x { ... } | for range x { ... } 236 | _, _ = f() | f() 237 | var (a int) | var a int 238 | var a int; a = f() | a := f() 239 | 240 | -- 241 | 242 | gogrep -x '~ var $_ int' 243 | 244 | Would find *foo* in all of: 245 | 246 | var foo int 247 | var (foo int) 248 | var (foo, bar int) 249 | var (bar uint; foo int) 250 | 251 | * Tradeoffs 252 | 253 | Simple, stupid Complex 254 | -------------------------------------------------------------------------------- 255 | grep literal AST matching gogrep ??? full linter 256 | 257 | - Many commands/operations (range, filter, etc) 258 | - Combine multiple commands 259 | - Use type information 260 | 261 | - Never reaching custom logic nor arbitrary code 262 | 263 | * Code search in PMD 264 | 265 | .link https://pmd.github.io/pmd-5.8.1/pmd-java/rules/java/basic.html#CollapsibleIfStatements 266 | 267 | //IfStatement[@Else='false']/Statement 268 | /IfStatement[@Else='false'] 269 | | 270 | //IfStatement[@Else='false']/Statement 271 | /Block[count(BlockStatement)=1]/BlockStatement 272 | /Statement/IfStatement[@Else='false'] 273 | 274 | -- 275 | 276 | if $_ { 277 | if $_ { 278 | $*_ 279 | } 280 | } 281 | 282 | * Bonus points: gofmt -r 283 | 284 | gofmt -r 'pattern -> replacement' 285 | 286 | -- 287 | 288 | gofmt -l -w -r 'strings.TrimPrefix(x, s) -> strings.TrimSuffix(x, s)' 289 | 290 | - Pattern and replacement can only be expressions 291 | - Only matches ASTs with variables 292 | - Cannot compose commands to do complex queries 293 | 294 | gogrep -x 'if strings.HasPrefix($x, $p) { $x = $x[len($p):] }' 295 | -s '$x = strings.TrimPrefix($x, $p)' 296 | 297 | * Open questions 298 | 299 | - What commands should be added? 300 | - There are no "AND" and "OR" commands 301 | - Allow more custom logic, like: 302 | - filter matches containing at least three return statements 303 | 304 | -- 305 | 306 | Can a more powerful and generic tool exist? 307 | -------------------------------------------------------------------------------- /2018/stringer.slide: -------------------------------------------------------------------------------- 1 | 2018's stringer 2 | 17 Jan 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | 8 | * Quiz time! 9 | 10 | package main 11 | 12 | import ( 13 | "os" 14 | "text/template" 15 | ) 16 | 17 | func main() { 18 | tmpl, _ := template.New("").Parse("{{ range $x := . }}{{ $x }}{{ end }}\n") 19 | tmpl.Execute(os.Stdout, map[int]int{1: 2}) 20 | } 21 | 22 | Assuming no errors, what's the output? 23 | 24 | - 0 25 | - 1 26 | - 2 27 | - other 28 | 29 | * String() for enum-like types 30 | 31 | type Pill int 32 | 33 | const ( 34 | Placebo Pill = iota 35 | Aspirin 36 | Ibuprofen 37 | ) 38 | 39 | -- 40 | 41 | Placebo.String() == "Placebo" 42 | Aspirin.String() == "Aspirin" 43 | 44 | * Manually? 45 | 46 | func (p Pill) String() string { 47 | switch p { 48 | case Placebo: 49 | return "Placebo" 50 | case Aspirin: 51 | return "Aspirin" 52 | [...] 53 | } 54 | // or a map, or an array, or a slice... 55 | } 56 | 57 | * golang.org/x/tools/cmd/stringer 58 | 59 | $ cat pill.go 60 | [...] 61 | type Pill int 62 | 63 | const ( 64 | Placebo Pill = iota 65 | Aspirin 66 | Ibuprofen 67 | ) 68 | [...] 69 | $ stringer -type=Pill 70 | $ cat pill_string.go 71 | func (p Pill) String() string { ... } 72 | 73 | -- 74 | 75 | //go:generate stringer -type Pill 76 | type Pill int 77 | 78 | $ go generate 79 | 80 | * Output example 81 | 82 | const _Pill_name = "PlaceboAspirinIbuprofen" 83 | 84 | var _Pill_index = [...]uint8{0, 7, 14, 23} 85 | 86 | func (i Pill) String() string { 87 | if i < 0 || i >= Pill(len(_Pill_index)-1) { 88 | return "Pill(" + strconv.FormatInt(int64(i), 10) + ")" 89 | } 90 | return _Pill_name[_Pill_index[i]:_Pill_index[i+1]] 91 | } 92 | 93 | - No map, no switch 94 | - All the data in two static globals (smaller binary) 95 | - Array, enabling compiler optimizations like removing bounds checks 96 | - Uses smallest index type (uint8, uint16, uint32...) 97 | - Has a built-in fallback 98 | 99 | * Enhancement #1 - prefix trimming 100 | 101 | type Pill int 102 | 103 | const ( 104 | PillPlacebo Pill = iota 105 | PillAspirin 106 | PillIbuprofen 107 | ) 108 | 109 | $ stringer -type Pill -trimprefix Pill 110 | 111 | -- 112 | 113 | PillPlacebo.String() == "Placebo" 114 | PillAspirin.String() == "Aspirin" 115 | 116 | * Enhancement #2 - line comments 117 | 118 | type Token int 119 | 120 | const ( 121 | Invalid Token = iota 122 | Plus // + 123 | Minus // - 124 | Star // * 125 | ) 126 | 127 | $ stringer -type Token -linecomment 128 | 129 | -- 130 | 131 | Plus.String() == "+" 132 | Minus.String() == "-" 133 | Star.String() == "*" 134 | 135 | * Enhancement #3 - this is yet another test 136 | 137 | func (i Pill) String() string { 138 | if i < 0 || i >= Pill(len(_Pill_index)-1) { 139 | return fmt.Sprintf("Pill(%d)", i) 140 | } 141 | return _Pill_name[_Pill_index[i]:_Pill_index[i+1]] 142 | } 143 | 144 | -- 145 | 146 | func (i Pill) String() string { 147 | if i < 0 || i >= Pill(len(_Pill_index)-1) { 148 | return "Pill(" + strconv.FormatInt(int64(i), 10) + ")" 149 | } 150 | return _Pill_name[_Pill_index[i]:_Pill_index[i+1]] 151 | } 152 | 153 | * The levels of the mighty standard library 154 | 155 | // L0 is the lowest level, core, nearly unavoidable packages. 156 | "L0": {"errors", "io", "runtime", "sync", "unsafe", ...}, 157 | 158 | // L1 adds simple functions and strings processing, 159 | // but not Unicode tables. 160 | "L1": {"L0", "math", "sort", "strconv", "unicode/utf8", ...}, 161 | 162 | // L2 adds Unicode and strings processing. 163 | "L2": {"L1", "bufio", "bytes", "path", "strings", "unicode", ...}, 164 | 165 | // L3 adds reflection and some basic utility packages 166 | // and interface definitions, but nothing that makes 167 | // system calls. 168 | "L3": {"L2", "crypto", "encoding/binary", "hash", "image", "reflect", ...}, 169 | 170 | // L4 is defined as L3+fmt+log+time, because in general once 171 | // you're using L3 packages, use of fmt, log, or time is not a big deal. 172 | "L4": {"L3", "fmt", "log", "time"}, 173 | 174 | L4 - L1: path/filepath, os/exec, io/ioutil, ... 175 | 176 | * Enhancements that perished on the way 177 | 178 | Autodetection of prefixes via -autotrimprefix 179 | 180 | - Not worth it, as the go:generate comment is written only once 181 | 182 | Working with both: 183 | 184 | const ( 185 | Placebo Pill = iota 186 | [...] 187 | ) 188 | 189 | const ( 190 | Placebo = Pill(iota) 191 | [...] 192 | ) 193 | 194 | - Former is cleaner and has a simpler AST, meaning less stringer code 195 | - The user only has to rewrite the program once 196 | -------------------------------------------------------------------------------- /2018/unparam.slide: -------------------------------------------------------------------------------- 1 | Unused parameters in Go code 2 | 27 Jun 2018 3 | 4 | Daniel Martí 5 | https://mvdan.cc 6 | mvdan@mvdan.cc 7 | 8 | * Let's define an unused parameter 9 | 10 | Sometimes parameters were never needed, or are no longer needed. 11 | 12 | func NewTemplate(name, body, footer string) Template { 13 | return Template{name: name, body: body} 14 | } 15 | 16 | -- 17 | 18 | Sometimes a parameter should be used but isn't. 19 | 20 | func (t Template) Execute(w io.Writer) error { 21 | _, err := fmt.Fprintf(os.Stdout, "%s %s", t.name, t.body) 22 | return err 23 | } 24 | 25 | * Let's write a linter for this! 26 | 27 | - Using `x/tools/go/loader` to load the packages and Go files 28 | - Using `go/ast` to inspect the funcs 29 | 30 | Enter our first tricky case: 31 | 32 | func NewTemplate(name, body string) Template { 33 | body = sampleBody 34 | return Template{name: name, body: body} 35 | } 36 | 37 | `go/ast` is just the syntax, it has no semantics. 38 | 39 | * Let's add go/types 40 | 41 | - It knows what definition each identifier relates to 42 | 43 | First algorithm in pseudo-Go: 44 | 45 | for _, fn := range allFuncs { 46 | for _, param := range fn.Params { 47 | used := false 48 | for _, node := range fn.NodesRead() { 49 | if types.Is(node, param) { 50 | used = true 51 | break 52 | } 53 | } 54 | if !used { 55 | warnf("unused param: %v", param) 56 | } 57 | } 58 | } 59 | 60 | * Limitations of go/ast and go/types 61 | 62 | - We must manually find if each parametere is used 63 | - `fn.NodesRead` is extremely complex 64 | 65 | Enter our first tricky case, again: 66 | 67 | func NewTemplate(name, body string) Template { 68 | body = sampleBody // not a use of the body variable! 69 | return Template{name: name, body: body} 70 | } 71 | 72 | Too many ways to define a variable: 73 | 74 | var (t Template) 75 | t.body = someValue 76 | 77 | t := Template{} 78 | t.body = someValue 79 | 80 | t := Template{body: someValue} 81 | 82 | * We can make it much simpler with x/tools/go/ssa 83 | 84 | - Much simpler structure thanks to Single Assignment Form 85 | - Already knows the "referrers" to every variable 86 | 87 | New algorithm in pseudo-Go: 88 | 89 | for _, param := range fn.Params { 90 | if len(param.Referrers()) == 0 { 91 | warnf("unused param: %v", param) 92 | } 93 | } 94 | 95 | Because of SSA, our first tricky case becomes something like: 96 | 97 | func NewTemplate(name, body string) Template { 98 | body2 := sampleBody 99 | return Template{name: name, body: body2} 100 | } 101 | 102 | * Common sources of false positives 103 | 104 | Dummy or stub functions: 105 | 106 | func (d devNull) WriteString(s string) error { 107 | return nil 108 | } 109 | func (s storeImpl) CreateFoo(f Foo) error { 110 | return fmt.Errorf("unimplemented") 111 | } 112 | 113 | Public API with compatibility guarantees: 114 | 115 | func NewClient(ctx context.Context, timeout time.Duration) Client { 116 | // no longer using timeout, use context.WithTimeout instead 117 | return Client{ctx: ctx, ...} 118 | } 119 | 120 | * Go has more potential for false positives 121 | 122 | Funcs that satisfy an interface or func signature: 123 | 124 | func pingHandler(w http.ResponseWriter, r *http.Request) { 125 | w.Write([]byte("pong")) 126 | } 127 | -- 128 | os.Expand(s, func(name string) string { 129 | return "SameValue" 130 | }) 131 | 132 | - If we blacklist all known interfaces, it's too conservative 133 | - We need to find how functions may be used and called 134 | - What about type assertions and reflect? 135 | - Once again, complex code to search the entire program 136 | 137 | * Enter our last package: x/tools/go/callgraph 138 | 139 | - Graph where nodes are funcs and edges are calls 140 | - Multiple algorithms that build a callgraph from SSA 141 | 142 | Variance in time cost and accuracy: 143 | 144 | - static: only static, direct calls recorded 145 | - RTA: built dynamically, not very precise 146 | - CHA: conservative, builds an "implements" chain 147 | - PTA: most accurate, and by far most expensive 148 | 149 | A linter must be conservative and reasonably fast, so let's choose CHA. 150 | 151 | * Last version of our program: 152 | 153 | `anyStrongCall` will inspect if any of the calls to fn "locks in" its signature. 154 | 155 | if anyStrongCall(graph.Inbound[fn]) { 156 | continue 157 | } 158 | for _, param := range fn.Params { 159 | if len(param.Referrers()) == 0 { 160 | warnf("unused param: %v", param) 161 | } 162 | } 163 | 164 | Examples of these calls: 165 | 166 | // any func passed as handler cannot change signature 167 | func HandleRequest(handler Handler, req Request) { 168 | handler(req) 169 | } 170 | 171 | // we'd need to change otherFn too, which may not be possible 172 | fn(otherFn()) 173 | 174 | * Summary 175 | 176 | - Unused parameters in Go are tricky 177 | - Use `go/ssa` to inspect each func's code 178 | - Use `go/callgraph` to find how each func is used 179 | - Rinse and repeat with lots of tests until it works OK 180 | 181 | You can try the end result: 182 | 183 | go get -u mvdan.cc/unparam 184 | 185 | Same idea, just with more code to handle endless edge cases. 186 | -------------------------------------------------------------------------------- /2020/testing-tips/go.mod: -------------------------------------------------------------------------------- 1 | module test 2 | 3 | go 1.14 4 | 5 | require github.com/chromedp/chromedp v0.5.3 6 | -------------------------------------------------------------------------------- /2020/testing-tips/go.sum: -------------------------------------------------------------------------------- 1 | github.com/chromedp/cdproto v0.0.0-20200116234248-4da64dd111ac h1:T7V5BXqnYd55Hj/g5uhDYumg9Fp3rMTS6bykYtTIFX4= 2 | github.com/chromedp/cdproto v0.0.0-20200116234248-4da64dd111ac/go.mod h1:PfAWWKJqjlGFYJEidUM6aVIWPr0EpobeyVWEEmplX7g= 3 | github.com/chromedp/chromedp v0.5.3 h1:F9LafxmYpsQhWQBdCs+6Sret1zzeeFyHS5LkRF//Ffg= 4 | github.com/chromedp/chromedp v0.5.3/go.mod h1:YLdPtndaHQ4rCpSpBG+IPpy9JvX0VD+7aaLxYgYj28w= 5 | github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee h1:s+21KNqlpePfkah2I+gwHF8xmJWRjooY+5248k6m4A0= 6 | github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee/go.mod h1:L0fX3K22YWvt/FAX9NnzrNzcI4wNYi9Yku4O0LKYflo= 7 | github.com/gobwas/pool v0.2.0 h1:QEmUOlnSjWtnpRGHF3SauEiOsy82Cup83Vf2LcMlnc8= 8 | github.com/gobwas/pool v0.2.0/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= 9 | github.com/gobwas/ws v1.0.2 h1:CoAavW/wd/kulfZmSIBt6p24n4j7tHgNVCjsfHVNUbo= 10 | github.com/gobwas/ws v1.0.2/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM= 11 | github.com/knq/sysutil v0.0.0-20191005231841-15668db23d08 h1:V0an7KRw92wmJysvFvtqtKMAPmvS5O0jtB0nYo6t+gs= 12 | github.com/knq/sysutil v0.0.0-20191005231841-15668db23d08/go.mod h1:dFWs1zEqDjFtnBXsd1vPOZaLsESovai349994nHx3e0= 13 | github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM= 14 | github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs= 15 | golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 16 | -------------------------------------------------------------------------------- /2020/testing-tips/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "flag" 6 | "fmt" 7 | "log" 8 | 9 | "github.com/chromedp/chromedp" 10 | ) 11 | 12 | func MeetupDate(ctx context.Context, url string) (string, error) { 13 | ctx, cancel := chromedp.NewContext(ctx) 14 | defer cancel() 15 | 16 | var date string 17 | if err := chromedp.Run(ctx, 18 | chromedp.Navigate(url), 19 | chromedp.Text(".eventTimeDisplay", &date, chromedp.ByQuery), 20 | ); err != nil { 21 | return "", err 22 | } 23 | return date, nil 24 | } 25 | 26 | func main() { 27 | flag.Parse() 28 | 29 | actx := context.Background() 30 | ctx, cancel := chromedp.NewContext(actx) 31 | defer cancel() 32 | 33 | for _, arg := range flag.Args() { 34 | fmt.Println(arg) 35 | date, err := MeetupDate(ctx, arg) 36 | if err != nil { 37 | log.Fatal(err) 38 | } 39 | fmt.Println(date) 40 | fmt.Println() 41 | } 42 | } 43 | -------------------------------------------------------------------------------- /2020/testing-tips/main_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "flag" 6 | "testing" 7 | 8 | "github.com/chromedp/chromedp" 9 | ) 10 | 11 | var headful = flag.Bool("headful", false, "") 12 | 13 | func TestMeetupDate(t *testing.T) { 14 | t.Parallel() 15 | 16 | tests := []struct { 17 | name string 18 | url string 19 | want string 20 | }{ 21 | { 22 | "April2020", 23 | "https://www.meetup.com/GoSheffield/events/269190444/", 24 | "Thursday, April 2, 2020\n6:30 PM to 7:15 PM GMT+1", 25 | }, 26 | { 27 | "May2020", 28 | "https://www.meetup.com/GoSheffield/events/270080480/", 29 | "Thursday, May 7, 2020\n6:30 PM to 7:30 PM GMT+1", 30 | }, 31 | { 32 | "June2020", 33 | "https://www.meetup.com/GoSheffield/events/270842085/", 34 | "Thursday, June 4, 2020\n6:30 PM to 7:30 PM GMT+1", 35 | }, 36 | { 37 | "July2020", 38 | "https://www.meetup.com/GoSheffield/events/271453771/", 39 | "Thursday, July 2, 2020\n6:30 PM to 7:30 PM GMT+1", 40 | }, 41 | } 42 | 43 | actx := context.Background() 44 | if *headful { 45 | var cancel func() 46 | actx, cancel = chromedp.NewExecAllocator(actx) 47 | t.Cleanup(cancel) 48 | } 49 | 50 | ctx, cancel := chromedp.NewContext(actx) 51 | t.Cleanup(cancel) 52 | if err := chromedp.Run(ctx); err != nil { 53 | t.Fatal(err) 54 | } 55 | 56 | for _, test := range tests { 57 | t.Run(test.name, func(t *testing.T) { 58 | test := test 59 | t.Parallel() 60 | 61 | got, err := MeetupDate(ctx, test.url) 62 | if err != nil { 63 | t.Fatal(err) 64 | } 65 | if got != test.want { 66 | t.Errorf("want %q, got %q", test.want, got) 67 | } 68 | }) 69 | } 70 | } 71 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Daniel Martí. All rights reserved. 2 | 3 | Redistribution and use in source and binary forms, with or without 4 | modification, are permitted provided that the following conditions are 5 | met: 6 | 7 | * Redistributions of source code must retain the above copyright 8 | notice, this list of conditions and the following disclaimer. 9 | * Redistributions in binary form must reproduce the above 10 | copyright notice, this list of conditions and the following disclaimer 11 | in the documentation and/or other materials provided with the 12 | distribution. 13 | * Neither the name of the copyright holder nor the names of its 14 | contributors may be used to endorse or promote products derived from 15 | this software without specific prior written permission. 16 | 17 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 18 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 19 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 20 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 21 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 22 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 23 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # talks 2 | 3 | Collection of slides from talks I have given. 4 | 5 | Older text-based talks are in files in this repository. 6 | Newer talks are available via Google Slides: 7 | 8 | * 2019 - London Gophers - Reimagining gofmt for Go 2.0 - [slides](https://docs.google.com/presentation/d/14HcyqkNcQ48uyjM_Sas3HxO1LbFvykcfKksFPHaaEcI/edit?usp=sharing) - [video](https://www.youtube.com/watch?v=uiwoTc306TE) 9 | * 2019 - GoSheffield - What's coming in Go 1.13 - [slides](https://docs.google.com/presentation/d/1SHCx5Dw9RXBn2lxfSHeGv5iT3VBBernT1mxc48rAXnE/edit?usp=sharing) 10 | * 2019 - GoSheffield - What's coming in Go 1.14 - [slides](https://docs.google.com/presentation/d/1HfIwlVTmVWQk94OLKfTGvXpQxyp0U4ywG1u5j2tjiuE/edit?usp=sharing) 11 | * 2019 - GopherCon San Diego - Optimizing Go code without a blindfold - [slides](https://docs.google.com/presentation/d/1T974gp66xIA4EVGzOYy7LUA2gVBdMMqdcNkQcopNwrs/edit?usp=sharing) - [video](https://www.youtube.com/watch?v=oE_vm7KeV_E) 12 | * 2019 - GolangLeeds - The hidden cost of initialization - [slides](https://docs.google.com/presentation/d/1p2iNktdFvlhdpUUDSxYj63EY9dAlKJavBC8_8UcrTzo/edit?usp=sharing) 13 | * 2020 - Go Remote Fest - What's coming in Go 1.15 - [slides](https://docs.google.com/presentation/d/1veyF0y6Ynr6AFzd9gXi4foaURlgbMxM-tmB4StDrdAM/edit?usp=sharing) 14 | * 2020 - Golang Barcelona - A Go code obfuscator from scratch - [slides](https://docs.google.com/presentation/d/196hoz6LHrlRxkpQw587gqw-TYRuScjn4IAv-jrjmHkI/edit?usp=sharing) 15 | * 2021 - GoSheffield - Quality of life improvements in Go - [slides](https://docs.google.com/presentation/d/1sBXrDYiSCPVBeguQbCJebxIVptjTFKjGOBsElkbp040/edit?usp=sharing&resourcekey=0-TM-u9xTZZHUyt3-y9-tZow) 16 | * 2021 - Golang North East - Defensive programming in Go - [slides](https://docs.google.com/presentation/d/1b6RsEVcc7o9SFU9NE--dJpw5kDlQvjWlWr3l6uIa12s/edit?usp=sharing) 17 | * 2022 - GopherCon UK - Deep dive into Go's build system - [slides](https://docs.google.com/presentation/d/1tIb04KIka3sTmsF0G8v3RbzKG632oB1UObTmPQ80oq4/edit?usp=sharing) - [video](https://www.youtube.com/watch?v=sTXc_JxmvV0) 18 | * 2022 - GoSheffield - A refreshing take on testing libraries: quicktest - [slides](https://docs.google.com/presentation/d/1vlUhnguW_aZLDhDsQ_opBQdYq3Ld2x9pyFIVb4BERlo/edit?usp=sharing) 19 | * 2022 - GoLab - Defensive programming techniques in Go - [slides](https://docs.google.com/presentation/d/1I6pq08Qk_FTVXVyBLs9SYYp2KbeRCvFHzTOCPfMSrdk/edit?usp=sharing&resourcekey=0-yFJxlTN3sJCnr6UrOEsGkQ) - [video](https://www.youtube.com/watch?v=0bDeoa8287I) 20 | * 2023 - GoLab - From zero to func main: initializing packages - [slides](https://docs.google.com/presentation/d/1Y986Ux0IgBEKKmTudcOcu51K6DrNU8AN8ArI7EFnn4Q/edit?usp=sharing) - [video](https://www.youtube.com/watch?v=cLASOOa_rA0) 21 | * 2023 - London Gophers - What's coming in Go 1.22 - [slides](https://docs.google.com/presentation/d/1f1NQonhMkbpgDDfdsZadYp299lsIpuVjRAjypCqvW8U/edit?usp=sharing&resourcekey=0-q1ukfM5gKtXodK3e4g8koA) - [video](https://www.youtube.com/watch?v=LspUdN3nWPc) 22 | * 2024 - GoLab - Go ten years ago, Go ten years from now - [slides](https://docs.google.com/presentation/d/1pZGklmzsAU3XV_9_S3QiafFFtBQJnh-JwX-PKVW-9ts/edit?usp=sharing) - [video](https://www.youtube.com/watch?v=sqLhUV6yhes) 23 | * 2024 - London Gophers - What's coming in Go 1.24 - [slides](https://docs.google.com/presentation/d/1jvc0RzrshOOjkeEnsyea2jwjs_zVOcyETG558pqPOK0/edit?usp=sharing) 24 | --------------------------------------------------------------------------------