├── .gitignore
├── README.md
├── SUMMARY.md
├── book.json
├── channels.md
├── colophon.md
├── cover.jpg
├── cover_small.jpg
├── databases.md
├── errors.md
├── glossary.md
├── goroutines.md
├── interfaces.md
├── maps.md
├── packages.md
├── scope.md
├── servers.md
├── setup.md
├── slices.md
├── strings.md
├── structs.md
├── styles
└── pdf.css
├── templates.md
├── time.md
└── title.md
/.gitignore:
--------------------------------------------------------------------------------
1 | # Node rules:
2 | ## Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
3 | .grunt
4 |
5 | ## Dependency directory
6 | ## Commenting this out is preferred by some people, see
7 | ## https://docs.npmjs.com/misc/faq#should-i-check-my-node_modules-folder-into-git
8 | node_modules
9 |
10 | # Book build output
11 | _book
12 |
13 | # eBook build output
14 | *.epub
15 | *.mobi
16 | *.pdf
17 |
18 | admin
19 | examples
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Introduction
2 |
3 | ###
4 |
5 | ### About Go
6 |
7 | The Go language spec is deceptively simple to read, with only 25 [keywords](https://golang.org/ref/spec#Keywords) and few data structures.
8 |
9 | The syntax is very like other languages in the C family, and programmers coming from other languages with this heritage will find it easy to get started with the language, however it can be difficult to assimilate the culture of radical minimalism. Some programmers feel keenly the lack of familiar tools which Go simply leaves out - inheritance, assertions, exceptions, the ternary operator, enums, and generics are all missing [by design](https://golang.org/doc/faq#Why_doesnt_Go_have_feature_X). In addition, the strict rules on formatting and the culture of limited dependencies and minimal abstraction can feel restrictive, but these very deliberate omissions also provide some concrete advantages. The corollary to this culture of simplicity, and one of the most rewarding aspects of programming in Go, is the accessibility and simplicity of the standard library code and many of the libraries available - Go code is easy to read as well as write.
10 |
11 | Also invaluable is the clarity and stability of the standard library, encapsulated in the [Go 1 promise](https://golang.org/doc/go1compat) made way back in 2012 \(which still holds today\), or Brad Fitzpatrick's whimsical description of the language as [asymptotically approaching boring](https://golangnews.com/stories/845-video-introducing-go-1.6-asymptotically-approaching-boring-by-brad-fitzpatrick). If you build an app today, it will compile with minimal or no changes until Go 2. While being difficult for the maintainers, this is a liberating promise for Go programmers as it makes this level of the ecosystem very stable.
12 |
13 | If you're looking for a cutting-edge language which explores recent research, or one which favours terse abstractions, Go is not for you – Go is boring by design.
14 |
15 | ### Bugs in Go programs
16 |
17 | Programs in Go, like those in any other language, are subject to bugs.
18 |
19 | Some are caused by unexpected data - a look at the [Trophy Cabinet](https://github.com/dvyukov/go-fuzz#trophies) of bugs found by [Go Fuzz](https://github.com/dvyukov/go-fuzz) should disabuse us of the notion that most bugs are esoteric or difficult to defend against, or even specific to Go as a language. Often the same errors crop up again and again – bounds not checked or nil pointers. The most common bugs encountered \(even by experienced programmers\) are:
20 |
21 | * Index out of range
22 | * Slice bounds out of range
23 | * Nil pointer dereferences
24 |
25 | These are trivial errors, but are easy to make in any sufficiently large program, and are usually due to expectations about data passed in from outside the program which don't hold true \(hence they are easier to find with a fuzzer\).
26 |
27 | There are also a few mistakes which are easy to make when learning Go which a fuzzer might not detect, such as not correctly passing values to a goroutine within a range loop, misuse of an API, or lack of familiarity with the tradeoffs in the design of slices or strings in Go. These are the classes of bugs which this book will hopefully help the reader avoid.
28 |
29 | ### Why read this book?
30 |
31 | Typically Go programmers get up to speed quickly and feel productive within days or weeks, but it can be harder to assimilate the culture and be aware of the subtle problems that can occur. _The Go Bestiary_ provides a quick guide to allow a programmer who has some experience in other languages get started quickly in Go, while being aware of the idioms to use and areas where there might be bugs lurking in their new Go code. The book presents a mix of advice for structuring your Go programs and descriptions of common mistakes to avoid, with liberal code examples, to help ease you into using Go.
32 |
33 | If you're familiar with Go but not an expert, hopefully there will also be a few interesting facts about the language you haven't yet uncovered, and some potential bugs which you might not be aware of. You may want to skip the sections on setup and packages in this case, as they will cover familiar ground.
34 |
35 | If you're new to Go, but experienced in other languages, you'll find succinct descriptions of the most important differences in Go which usually trip up learners \(for example declaring strings, strings vs runes, copying slices, using goroutines within a for loop\), descriptions of bugs which beginners often fall prey to, and plenty of detail about unique aspects of the language like goroutines and slices.
36 |
37 |
--------------------------------------------------------------------------------
/SUMMARY.md:
--------------------------------------------------------------------------------
1 | # Summary
2 |
3 | * [Title](title.md)
4 | * [Introduction](README.md)
5 | * [Setup](setup.md)
6 | * [Strings](strings.md)
7 | * [Structs](structs.md)
8 | * [Slices](slices.md)
9 | * [Maps](maps.md)
10 | * [Goroutines](goroutines.md)
11 | * [Channels](channels.md)
12 | * [Functions](scope.md)
13 | * [Packages](packages.md)
14 | * [Interfaces](interfaces.md)
15 | * [Servers](servers.md)
16 | * [Templates](templates.md)
17 | * [Databases](databases.md)
18 | * [Time](time.md)
19 | * [Errors](errors.md)
20 | * [Glossary](/glossary.md)
21 | * [Colophon](/colophon.md)
22 |
23 |
24 |
25 |
--------------------------------------------------------------------------------
/book.json:
--------------------------------------------------------------------------------
1 | {
2 | "pdf": {
3 | "fontSize": 17,
4 | "paperSize": "legal",
5 | "margin": {
6 | "right": 80,
7 | "left": 80,
8 | "top": 80,
9 | "bottom": 72
10 | }
11 | }
12 | }
--------------------------------------------------------------------------------
/channels.md:
--------------------------------------------------------------------------------
1 | # Channels
2 |
3 | Channels are a queue of values of a specific type which can be used to share information safely between goroutines. You can read more about the uses of channels with goroutines in the [Pipelines](https://blog.golang.org/pipelines) article on the go blog. Before using channels, consider other options, particularly if you just need to control access to shared state. You can use mutexes to control access to state across goroutines, which is much simpler than channels. Use channels to orchestrate more complex behaviour like signalling between goroutines, passing ownership of data, or distributing units of work.
4 |
5 | ## Nil Channels
6 |
7 | Sending to a nil channel blocks forever:
8 |
9 | ```go
10 | func main() {
11 | var c chan string
12 | // send to nil channel blocks forever
13 | c <- "test"
14 | }
15 | ```
16 |
17 | Receive on a nil channel blocks forever:
18 |
19 | ```go
20 | func main() {
21 | var c chan string
22 | // receive on nil blocks forever
23 | fmt.Println(<-c)
24 | }
25 | ```
26 |
27 | ## Closed Channels
28 |
29 | Sending to a closed channel causes a panic
30 |
31 | ```go
32 | package main
33 |
34 | import (
35 | "fmt"
36 | )
37 |
38 | func main() {
39 | output := make(chan int, 1)
40 | write(output, 2) // send
41 | close(output) // close
42 | write(output, 3) // send after close = panic
43 | }
44 | ```
45 |
46 | A receive from a closed channel returns the zero value immediately
47 |
48 | ```go
49 | package main
50 |
51 | import "fmt"
52 |
53 | func main() {
54 | c := make(chan int, 2)
55 | c <- 1
56 | c <- 2
57 | c <- 3
58 | close(c)
59 | for i := 0; i < 5; i++ {
60 | fmt.Printf("%d ", <-c)
61 | }
62 | }
63 | ```
64 |
65 | outputs the channel values then the zero value
66 |
67 | > 1 2 0 0 0
68 |
69 | Instead you can use a range loop to just get the values
70 |
71 | ```go
72 | func main() {
73 | c := make(chan int, 2)
74 | c <- 1
75 | c <- 2
76 | close(c)
77 | for v := range c {
78 | fmt.Printf("%d ", v)
79 | }
80 | }
81 | ```
82 |
83 | ## Stopping a goroutine
84 |
85 | You can use a channel to stop a goroutine by using the channel to signal completion.
86 |
87 | ```go
88 | // create a signal channel
89 | done := make(chan bool)
90 |
91 | // launch the goroutine
92 | go func() {
93 |
94 | // listen for signals
95 | for {
96 | select {
97 | case <- done:
98 | return
99 | default:
100 | // Do something
101 | // ...
102 | }
103 | }
104 | }()
105 |
106 | // Do something
107 | // ...
108 |
109 | // Stop the goroutine
110 | done <- true
111 | ```
112 |
113 | ## Deadlocks
114 |
115 | Go channels created with make\(chan T\) without a size are not buffered. An unbuffered channel is synchronous, you can only send when there is a receiver. If the reads don't match the writes, the anon goroutines deadlock.
116 |
117 | ```go
118 | func main() {
119 | channel := make(chan string)
120 | done_channel := make(chan bool)
121 | go func() {
122 | channel <- "value" // write 1
123 | channel <- "value" // write 2
124 | done_channel <- true
125 | }()
126 | variable := <-channel // read 1
127 | ok := <-done_channel
128 | fmt.Println(variable,ok)
129 | }
130 | ```
131 |
132 | > fatal error: all goroutines are asleep - deadlock!
133 |
134 | ```go
135 | func main() {
136 | channel := make(chan string)
137 | done_channel := make(chan bool)
138 | go func() {
139 | channel <- "write1" // write 1
140 | channel <- "write2" // write 2
141 | done_channel <- true
142 | }()
143 | variable := <-channel // read 1
144 | variable = <-channel // read 2 required to finish
145 | ok := <-done_channel
146 | fmt.Println(variable, ok)
147 | }
148 | ```
149 |
150 | > write2 true
151 |
152 | ## Counting channel elements
153 |
154 | If you want to know how many elements are in a channel, you can just use len\(channel\)
155 |
156 | ```go
157 | func main() {
158 | c := make(chan int, 100)
159 | for i := 0; i < 34; i++ {
160 | c <- 0
161 | }
162 | fmt.Println(len(c))
163 | }
164 | ```
165 |
166 |
167 |
168 |
--------------------------------------------------------------------------------
/colophon.md:
--------------------------------------------------------------------------------
1 | # Colophon
2 |
3 | This book is copyright Kenneth Grant 2017, All Rights Reserved
4 |
5 | Cover image of a Rhinocerous Beetle by James Marchment as part of the Insects Unlocked Project
6 |
7 | You can purchase this book on Amazon in [Kindle](https://www.amazon.com/dp/B07696DMDF) and [Paperback](https://www.amazon.com/dp/1549928376) editions
8 |
9 | The author can be found on twitter [@kennygrant](https://twitter.com/kennygrant)
--------------------------------------------------------------------------------
/cover.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kennygrant/bestiary/be8cd97bf90c1c00abc9f4676a11aa761fbc4f7a/cover.jpg
--------------------------------------------------------------------------------
/cover_small.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kennygrant/bestiary/be8cd97bf90c1c00abc9f4676a11aa761fbc4f7a/cover_small.jpg
--------------------------------------------------------------------------------
/databases.md:
--------------------------------------------------------------------------------
1 | # Databases
2 |
3 | If you're unsure which database to use, Postgresql is a highly reliable, well maintained project and has excellent drivers for Go. the examples shown below use psql, but can be adapted for other databases. Go has drivers for all the mainstream databases, as well as a selection of time series and key/value stores written in Go.
4 |
5 | You can find a list of go database drivers on the Go wiki page [SQL Drivers](https://github.com/golang/go/wiki/SQLDrivers) - prefer a driver which doesn't use cgo and conforms to the [database/sql](https://golang.org/pkg/database/sql/) interface if possible.
6 |
7 | ## Importing a driver
8 |
9 | Importing a database driver is used to register the database driver. In retrospect this is an unfortunate design decision, and it would be better to register drivers explicitly, but it won't change in Go 1. So import the driver as follows in order to register it and use a given database:
10 |
11 | ```go
12 | import (
13 | "database/sql"
14 | // This line registers the database driver
15 | // and exposes the standard database/sql interface
16 | _ "github.com/lib/pq"
17 | )
18 | ```
19 |
20 | If code imports lots of drivers, it will be importing all the code, and registering the drivers on init, so only import those you need to use in a program. If importing drivers with the blank identifier as above you can only use the standard sql interface defined in database/sql. You may find you want to use a higher level package to simplify common operations, or write your own wrapper for the database in complex applications.
21 |
22 | ## Opening a connection
23 |
24 | When you open a connection to the database, you must ping it to ensure it opened correctly
25 |
26 | ```go
27 | // Open the database (no connection is miade)
28 | db, err := sql.Open("postgres","postgres://azurediamond:hunter2@localhost/azurediamond?sslmode=verify-full")
29 | "user:password@tcp(127.0.0.1:3306)/hello")
30 | if err != nil {
31 | return err
32 | }
33 |
34 | // Check the db connection
35 | err = db.Ping()
36 | if err != nil {
37 | return err
38 | }
39 | ```
40 |
41 | ## Closing a connection
42 |
43 | Don't close the connection to the database frequently as it is designed to be recycled, you should ideally create one connection per datastore which lasts for the lifetime of your application. For example you might call defer db.Close in main.
44 |
45 | ```go
46 | defer db.Close()
47 | ```
48 |
49 | ## Querying the database
50 |
51 | Querying the database can be driver specific, especially for more advanced features, so be sure to read the driver documentation for the particular database/sql flavour you're using.
52 |
53 | ### Parameter placeholders
54 |
55 | The different databases use different formats for parameter placeholders, for example they don't all support ? as you might expect. The Mysql driver uses ?, ? etc, the sqlite driver uses ?, ? etc , the Postgresql driver uses $1, $2 etc and the Oracle driver uses :val1, :val2 etc. Sqlx is one of the few drivers to support named query parameters.
56 |
57 | ### Reading values
58 |
59 | To read Values from the database, use db.Query to fetch a result:
60 |
61 | ```go
62 | // Select from the db
63 | sql := "select count from users where status=100"
64 | rows, err := db.Query(sql)
65 | if err != nil {
66 | return err
67 | }
68 | defer rows.Close()
69 | ```
70 |
71 | And rows.Scan to scan in the rows received back:
72 |
73 | ```go
74 | // Read the count (just one row and one col)
75 | var count int
76 | for rows.Next() {
77 | err = rows.Scan(&count)
78 | if err != nil {
79 | return err
80 | }
81 | }
82 | ```
83 |
84 | ### Reading a Row into a struct
85 |
86 | To read a single row into a struct, you can use Query Row. You may want to add a constructor to your models which takes a row and instantiates the model from it, but the code below is a minimal example.
87 |
88 | ```go
89 | // Query a row from the database
90 | row := db.QueryRow("select id,name,created_at from tablename where id=?", id)
91 |
92 | type Resource struct{
93 | ID int
94 | Name string
95 | CreatedAt time.Time
96 | }
97 | var r Resource
98 | err := row.Scan(&r.ID, &r.Name, &r.CreatedAt) // This assumes no values are nil in the database
99 | if err != nil {
100 | return err
101 | }
102 | ```
103 |
104 | ### Scanning into a struct
105 |
106 | Many database libraries use reflect to attempt to introspect struct fields. You should try to avoid using reflect if you can as it is slow, prone to panics, and requires you to use struct tags and a dsl invented by the database library author. Another approach is to generate a list of columns, and pass them to the struct itself to assign, since it knows all about its fields and which values should go into them. This will require the struct to validate the column values and deal with nulls.
107 |
108 | ```go
109 | type User struct {
110 | ID int
111 | Name string
112 | CreatedAt time.Time
113 | }
114 |
115 | func ReadUser(columns map[string]interface{}) *User {
116 | u := &User{}
117 | u.ID = validateInt(cols["id"])
118 | u.Name = validateString(cols["name"])
119 | u.CreatedAt = validateTime(cols["created_at"])
120 | return u
121 | }
122 | ```
123 |
124 | ## Handling Relations
125 |
126 | This can seem a bit more fiddly than other languages, however the best approach is a straightforward one - retrieve the indexes of relations, and then if you require all of their information, retrieve the relations separately from the database.
127 |
128 | You can of course retrieve them with a join at the same time, but in complex apps it helps to separate retrieving relations from retrieving the actual relation records, which is often not necessary \(for example you might need to know a user has 8 images, and which ids they have, but not all the image captions and image data\). Being forced to handle relations manually does have advantages compared to the say ActiveRecord which can pull in far too much data behind the scenes.
129 |
130 | ## Connections are recycled
131 |
132 | Database connections may be called from many goroutines and connections are pooled by the driver. So don't use stateful sql commands like USE, BEGIN or COMMIT, LOCK TABLES etc and instead use the facilities offered by the sql driver. There’s no default limit on the number of connections, so it is possible to exhaust the number of connections allowed by the database. You can use SetMaxOpenConns and SetMaxIdleConns to control this behaviour.
133 |
134 | ```go
135 | db.SetMaxIdleConns(10)
136 | db.SetMaxOpenConns(100)
137 | ```
138 |
139 | ## Null values
140 |
141 | If your database may contain null values, you must guard against them. One way to do this is to scan into a map of empty interface and then assert that the interface{} contains the type you \(as opposed to a special null value\). If it fails, use the zero value of the type instead.
142 |
143 | ## Getting Database columns
144 |
145 | If you wish to know which columns are in a row, you can call:
146 |
147 | ```go
148 | // Get a list of column names for the row
149 | cols, err := rows.Columns()
150 | ```
151 |
152 | This might be useful if you vary selects \(e.g. sometimes select just id and created at, sometimes select the full model\), and yet wish to always use a standard constructor for models which can be passed a map\[string\]interface{} with the values.
153 |
154 | ## Writing to the database
155 |
156 | ```go
157 | // Insert a row
158 | row := db.QueryRow("insert into tablename VALUES($1,$2,$3)", args...)
159 | // Retrieve the inserted id
160 | err = row.Scan(&id)
161 | // Return the id and any error
162 | return id, err
163 | ```
164 |
165 | ## Multiple Statements
166 |
167 | The database/sql interface doesn't specify that drivers should support multiple statements, which means the behaviour is undefined and it's probably best to send single statements, unless your driver explicitly supports it.
168 |
169 |
--------------------------------------------------------------------------------
/errors.md:
--------------------------------------------------------------------------------
1 | # Errors & Logging
2 |
3 | [Errors](https://blog.golang.org/error-handling-and-go) and [logging](https://golang.org/pkg/log/) are one area of Go which does perhaps does deserve the label of simplistic rather than simple. The log package has no levels or interfaces, it simply prints to standard error by default. For many applications though, this is enough, and there are various logging packages available for more sophisticated requirements. The error type is very simple and errors are stored as strings.
4 |
5 | ## The Error Type
6 |
7 | > Errors are values
8 | > – Rob Pike
9 |
10 | The [error](https://blog.golang.org/error-handling-and-go) type in go is a very simple interface, with one method. Errors offer no introspection into what went wrong or storage of other data. Often errors are nested, as one error may be annotated several times as it passes up the stack. Try to prefer handling an error as close to the site of the error as possible.
11 |
12 | ```go
13 | type error interface {
14 | Error() string
15 | }
16 | ```
17 |
18 | You can use your own type for error, as long as it conforms to this interface, in some cases you might want to use a more complex type than the string only errors favoured by the standard library, to record an http status code for example.
19 |
20 | If you're defining interfaces, prefer requiring the error interface rather than a concrete type. You can use type assertions to determine if an error is of the type you're interested in.
21 |
22 | ## to, err := human\(\)
23 | > to, err := human\(\) – Francesc Campoy
24 |
25 | The convention in Go code is always to return an error as the last argument of a function, so if it has multiple arguments. Return an error as the last argument, and try to make it as specific as possible:
26 |
27 | ```go
28 | func DoSomething() error {
29 | to, err := human()
30 | if err != nil [
31 | // Annotate the error and return
32 | return fmt.Errorf("pkgname: failed to human %s",err)
33 | }
34 | // no, error, use value to
35 | ...
36 | return nil // return nil error
37 | }
38 | ```
39 |
40 | ```go
41 | func DoSomethingElse() (value, value, error) {
42 | ...
43 | if err != nil {
44 | // Annotate the error and return nil value + error
45 | return nil, nil, fmt.Errorf("pkgname: failed to do something %s",err)
46 | }
47 | ...
48 | // Return completed values and nil error
49 | return v, vv, nil
50 | }
51 | ```
52 |
53 | The caller should always check for errors before using values. _You should favour returning an error over returning nothing or simply a value_, even if you don't think initially you might encounter many errors, unless your function is just a few lines with no external calls. Functions rarely become simpler over time and it's always better to explicitly report errors rather than silently return zero data.
54 |
55 | ## _Always_ handle Errors
56 |
57 | You should never discard errors using \_ variables in production code. If a function returns an error, make sure you check it before using any values returned by that function. Usually if there is an error returned, values are undefined.
58 |
59 | ## _Either_ log _or_ return an error
60 |
61 | If you're logging an error, you've decided it's not important enough to handle. If you're returning an error \(usually with annotation\), you want the caller to handle it \(which might include logging or reporting it to the user\). Handle the error by logging / reporting, or return it, never both.
62 |
63 | **Don't both log and return errors:**
64 |
65 | ```go
66 | if err != nil {
67 | // Don't do this, either deal with or return an error
68 | log.Printf("pkgname: failed to log stats:%s",err)
69 | return err
70 | }
71 | ```
72 |
73 | If it's not important, log and move on, if it stops processing in this function, return the error and let the caller decide how to handle it \(usually to report to user and/or log\):
74 |
75 | ```go
76 | func DoSomething() error {
77 | // For an error you can handle in this function without telling the user:
78 | if err != nil {
79 | // Not important, just log the error and continue with processing
80 | log.Printf("pkgname: failed to log stats %s",err)
81 | }
82 | ...
83 | // For an important error return it (usually annotated) and let caller decide how to report
84 | if err != nil {
85 | return fmt.Errorf("pkgname: failed to do something %s",err)
86 | }
87 | }
88 | ```
89 |
90 | Dave Cheney has written a talk about handling errors in go – [Don't just check errors, handle them gracefully](https://dave.cheney.net/2016/04/27/dont-just-check-errors-handle-them-gracefully), which covers how to handle errors gracefully and avoid falling into the pitfalls outlined above.
91 |
92 | ## Stack traces on errors
93 |
94 | If you want to get a stack trace at the point of error which is not a panic, you can use the runtime package to determine the caller. You can also use the unofficial [go-errors](https://github.com/go-errors/errors) package, adds stacktrace support to errors, to record the stack trace and understand the state of execution when an error occurred.
95 |
96 | Finally, if you don't mind dumping a stack trace to stderr and terminating the program, you can panic. This is useful for debugging but less so in production programs.
97 |
98 | ## Interpreting a panic stack trace
99 |
100 | If you don't have much experience with stack traces, the output of a panic may seem inscrutable at first. To generate a stack trace, consider this simple program:
101 |
102 | ```go
103 | package main
104 |
105 | type T struct {
106 | s string
107 | }
108 |
109 | func foo(t *T) *T {
110 | defer panic("panic in foo")
111 | return t
112 | }
113 |
114 | func main() {
115 | foo(&T{})
116 | }
117 | ```
118 |
119 | Which produces the following panic:
120 |
121 | ```shell
122 | panic: panic in foo
123 |
124 | goroutine 1 [running]:
125 | main.foo(0x1042bfa4, 0x0, 0x10410008, 0x0)
126 | /tmp/sandbox611247315/main.go:13 +0xdb
127 | main.main()
128 | /tmp/sandbox611247315/main.go:18 +0x40
129 | ```
130 |
131 | The first line gives the message passed to panic, which should be as informative as possible.
132 |
133 | The lines after detail the crashing go routine (by default this is restricted to just the crashing one since Go 1.6).
134 | First we encounter the rather inscrutable message:
135 |
136 | > main.foo(0x1042bfa4, 0x0, 0x10410008, 0x0)
137 |
138 | This details the function with the panic, and may provide some clues if you have a nil pointer exception as to which pointer is nil - it includes the arguments to the function (including the method struct if any, which is always the first argument), and the return values. In this case, because the program uses defer to panic, the return values are filled in before panic, hence the argument and return pointers are the same value of 0x10410008. Without the defer the second two words would both be nil.
139 |
140 | Then the more important line which tells us which file and line the problem occurred at:
141 |
142 | > /tmp/sandbox611247315/main.go:13 +0xdb
143 |
144 | If nothing else, this is the line you need to read, as it clearly states exactly where the problem occurred. Where you have deliberately panicked using panic() this will normally be obvious anyway, but where you have a nil pointer or index out of range it can be useful for tracking down the bug.
145 |
146 | Remember that methods can be called on nil pointers, which can lead to subtle bugs if the callee never expects this and then tries to use the pointer as if it is initialised.
147 |
148 | If you're debugging complex stack traces featuring many goroutines with similar stack traces you may find the [panicparse](https://github.com/maruel/panicparse) library useful.
149 |
150 |
151 | ## Recovering from a panic
152 |
153 | You should recover within a **deferred function** to recover from a [panic](https://blog.golang.org/defer-panic-and-recover). Without a defer to make sure the recover executes last, recover will return nil and have no other effect. For example this will not work:
154 |
155 | ```go
156 | func p() {
157 | // Recover does nothing outwith a defer
158 | if r := recover(); r != nil {
159 | fmt.Println("recover", r)
160 | }
161 | // Something is wrong,
162 | // this panic will end the program
163 | panic("panic")
164 | }
165 | ```
166 |
167 | You need to use defer to recover:
168 |
169 | ```go
170 | func main() {
171 | // Print calling p
172 | fmt.Println("calling p")
173 | // Call p to panic and recover
174 | p()
175 | // Recovered, print panic over
176 | fmt.Println("panic over")
177 | }
178 |
179 | func p() {
180 | // Defer recover to the end of this function
181 | defer func() {
182 | // Recover from panic
183 | if r := recover(); r != nil {
184 | // Print recover
185 | fmt.Println("recover", r)
186 | }
187 | }()
188 |
189 | // Something is wrong, panic
190 | panic("panic")
191 | }
192 | ```
193 |
194 | ## Recover must be in the same goroutine
195 |
196 | You can only recover from panics in the current goroutine. If you have panics two goroutines deep, recovering at the top level won't catch them. For example in a web server handler which spawns a goroutine, you should protect against panics.
197 |
198 | ```go
199 | func a() {
200 | panic("panic a") // this panic will crash the program
201 | }
202 |
203 | func main() {
204 | // This recover does nothing
205 | defer func() {
206 | if x := recover(); x != nil {
207 | fmt.Println("catch panic main")
208 | }
209 | }()
210 | fmt.Println("start")
211 | go a() // use of go means recover above won't work
212 | time.Sleep(1 * time.Second)
213 | }
214 | ```
215 |
216 | In order to catch the panic in a, a recover within that goroutine is required. Note this is not to do with function scope \(removing the go before a\(\) would allow the recover in main to work. For more detail see [Handling Panics](https://golang.org/ref/spec#Handling_panics) in the Go spec.
217 |
218 | ```go
219 | func a() {
220 | // This recover is required to catch the panic in a
221 | defer func() { // recovers panic in a
222 | if x := recover(); x != nil {
223 | fmt.Println("catch panic a")
224 | }
225 | }()
226 | panic("panic a")
227 | }
228 |
229 | func main() {
230 | // this recover does nothing
231 | defer func() { // does nothing
232 | if x := recover(); x != nil {
233 | fmt.Println("catch panic main")
234 | }
235 | }()
236 | fmt.Println("start")
237 | go a() // use of go makes the recover above redundant
238 | time.Sleep(1 * time.Second)
239 | }
240 | ```
241 |
242 | ## Error strings
243 |
244 | Error strings may be used inside other error strings, so they should not be capitalized \(unless beginning with proper nouns or acronyms\) or end with punctuation. Make them composable, so that they can be logged or annotated with other messages:
245 |
246 | ```go
247 | // Annotating an error
248 | return fmt.Errorf("pkgname: failed to read %s: %v", filename, err)
249 |
250 | // Returning an error without comment - prefer annotating if possible
251 | return err
252 | ```
253 |
254 | By convention, errors are annotated with the package name or function involved, to make it easy to find the error. You should try to make your error strings unique so that there is no confusion over where the error may have been emitted.
255 |
256 | ## Don't panic
257 |
258 | [Panic](https://blog.golang.org/defer-panic-and-recover) is intended as a mechanism to report exceptional errors which require the program to exit immediately, or to report programmer error which should be fixed. You don't want to see it in production, nor should you use it to try to reproduce exceptions, which were left out of the language for a reason.
259 |
260 | In web or api servers, you may never need to use the keyword panic, and should prefer not to. Panic is fine for programming errors, or really exceptional situations \(this should never happen\), but try to avoid using it if you can, especially if you're writing a library. Your users will thank you.
261 |
262 | ## Don't use log.Fatalf or log.Panic
263 |
264 | For the same reasons, don't use log.Fatalf or log.Panic except in tests or short programs, because they will halt your program without cleanup and are equivalent to calling panic.
265 |
266 | In almost all cases you should recover gracefully from errors instead of calling a function which terminates the program.
267 |
268 | ```go
269 | // Don't do this, handle the error
270 | log.Fatalf("broken:%s",err)
271 | ```
272 |
273 | ## Asserts & Exceptions
274 |
275 | Go doesn't provide asserts or exceptions by design. There are reasons given for both decisions in the [FAQ](https://golang.org/doc/faq#exceptions) on the Go website.
276 |
277 | ## Go is boring
278 |
279 | Hopefully by now it is clear the Go language is deliberately limited and boring. If you want a stable platform on which to build exciting programs, this is a feature, not a bug. The language keeps getting a little better with every iteration (faster, fewer pauses, bugs fixed) without breaking your programs or introducing surprising new idioms.
280 |
281 | Less, in the case of programming languages, is more.
--------------------------------------------------------------------------------
/glossary.md:
--------------------------------------------------------------------------------
1 | # Glossary
2 |
3 | ## Channels
4 |
5 | Unbuffered channels combine communication — the exchange of a value — with synchronization — guaranteeing that two calculations \(goroutines\) are in a known state.
6 |
7 | ## Goroutines
8 |
9 | A goroutine is a function executing concurrently with other goroutines in the same address space. It is a lightweight alternative to threads invoked by using the go keyword.
10 |
11 | ## Interfaces
12 |
13 | Interfaces in Go provide a way to specify the behaviour of an object by defining a required method set. Commonly used interfaces like io.Writer and io.Reader usually only have very few methods.
14 |
15 | ## Slices
16 |
17 | Slices wrap arrays to give a more general, powerful, and convenient interface to sequences of data. Except for items with explicit dimension such as transformation matrices, most array programming in Go is done with slices rather than simple arrays.
18 |
19 | ## Maps
20 |
21 | Go provides a built-in map type that implements a hash table, which is a data structure offering fast lookups, adds and deletes, but no defined order.
22 |
23 | ## Mutexes
24 |
25 | Mutexes provide a locking mechanism to ensure only one goroutine is running on a critical section of code at any one time. Typically they are used to control access to state like a map between goroutines.
--------------------------------------------------------------------------------
/goroutines.md:
--------------------------------------------------------------------------------
1 | # Goroutines
2 |
3 | Prefix a function or method call with the `go` keyword to run the call in a new goroutine, asynchronously. The `go` keyword is one of the distinguishing features of Go, but it can lead to subtle bugs if not used carefully.
4 |
5 | > Do not communicate by sharing memory; instead, share memory by communicating.
6 | > – Rob Pike
7 |
8 | ## Asynchronous execution
9 |
10 | The simplest usage of a goroutine is simply to execute a function without waiting for a response. For example in a web handler to send a mail, you may not want to delay till the mail is sent before reporting back to the user.
11 |
12 | ```go
13 | // Send a message using the go keyword, without waiting for completion
14 | message := "hello word"
15 | go mail.Send(message, example@example.com)
16 |
17 | // Execution continues on the main goroutine without pausing...
18 | ```
19 |
20 | ## Waiting for go
21 |
22 | A common failure when using goroutines in a simple program is to fail to wait for them to finish. When a program's main ends, all running goroutines that were created by the program will also be stopped. So if your main only spins up some goroutines, those goroutines may not have time to finish, and you're left waiting for godot.
23 |
24 | ```go
25 | func main() {
26 | fmt.Println("waiting...")
27 | go func() {
28 | fmt.Println("godot") // doesn't arrive
29 | }()
30 | }
31 | ```
32 |
33 | > waiting...
34 |
35 | Use a wait group to make sure your goroutines are completed before the app terminates.
36 |
37 | ```go
38 | func main() {
39 | var wg sync.WaitGroup
40 | fmt.Println("hello world")
41 | wg.Add(1)
42 | go func() {
43 | defer wg.Done()
44 | fmt.Println("goodbye, cruel world")
45 | }()
46 | wg.Wait()
47 | }
48 | ```
49 |
50 | > hello world
51 | > goodbye, cruel world
52 |
53 | ## Goroutines & range {#goroutines-range}
54 |
55 | If you range over values and launch a goroutine within your range, the value v sent to the goroutine will not change each time. . This is because the variable v is reused for each iteration of the for loop. Given a func f that prints the arguments:
56 |
57 | ```go
58 | func f(s string) {
59 | fmt.Println(s)
60 | }
61 | ```
62 |
63 | If we run it in a goroutine, the only value printed \(3 times\), is the last one.
64 |
65 | ```go
66 | values := []string{"first", "second", "last"}
67 | for _, v := range values {
68 | go func() { f(v) }()
69 | }
70 | ```
71 |
72 | > last last last
73 |
74 | If instead the value is passed as an argument to the goroutine function, it works as expected:
75 |
76 | ```go
77 | for _, v := range values {
78 | go func(val string) { f(val) }(v)
79 | }
80 | ```
81 |
82 | > first second last
83 |
84 | This is probably the most common error when using goroutines, and is easy to make as it's natural to use the values of range within the loop. Always watch for this if using a goroutine inside a for loop - some linters may warn on this.
85 |
86 | ## Goroutines & Blocking functions
87 |
88 | If you provide a blocking function as the argument to a function in a go routine, the blocking run will be run synchronously first, and the realised argument will be sent to the goroutine function.
89 |
90 | ```go
91 | // if b blocks, b will be called first, then after completion f called with the result
92 | // so b will be called before goroutine is invoked
93 | go f(fb())
94 | ```
95 |
96 | Instead an anonymous function could be used:
97 |
98 | ```go
99 | // b will be called in the goroutine
100 | go func() {
101 | f(fb())
102 | }()
103 | ```
104 |
105 | ## Data Races
106 |
107 | If two goroutines access the same memory without access control, this causes a race condition. Race detection is not yet automatic at compile time, but fortunately it's easy to detect at runtime with the [race detector](https://golang.org/doc/articles/race_detector.html). To find data races, run your program with the -race option and it will warn you if races are detected:
108 |
109 | ```go
110 | go run -race race.go
111 | ```
112 |
113 | An example of a data race:
114 |
115 | ```go
116 | m := make(map[string]string)
117 |
118 | // Access OK
119 | m["1"] = "a"
120 |
121 | go func() {
122 | m["1"] = "a" // First conflicting access.
123 | }()
124 |
125 | m["2"] = "b" // Second conflicting access.
126 | ```
127 |
128 | To avoid races, you can pass copies of values, or wrap access with a mutex from the [sync](https://golang.org/pkg/sync/) package.
129 |
130 | ## `GOMAXPROCS`
131 |
132 | `GOMAXPROCS` sets how many processors the Go runtime uses to run goroutines on. Though it was set to 1 by default in early versions of Go for performance reasons, programs usually don't need to set`GOMAXPROCS,` as it is now set intelligently by the [runtime](https://golang.org/pkg/runtime/#GOMAXPROCS) based on the machine. Parallel programs might benefit from a further increase in `GOMAXPROCS` but be aware that [concurrency is not parallelism](https://blog.golang.org/2013/01/concurrency-is-not-parallelism.html). You may find this set in older source code, but it is very rarely necessary.
133 |
134 | .
135 |
136 |
--------------------------------------------------------------------------------
/interfaces.md:
--------------------------------------------------------------------------------
1 | # Interfaces
2 |
3 | An interface in go is a contract specifying which method signatures a type must have. Crucially, it is specified by the user of the interface, not the code which satisfies it.
4 |
5 | ## Keep interfaces simple
6 |
7 | > The bigger the interface, the weaker the abstraction
8 | > – Rob Pike
9 |
10 | Interfaces are at their most powerful when they express a simple contract that any type can easily conform to. If they start to demand a laundry list of functions \(_more than a few_ is a good rule of thumb\), they have very little advantage over a concrete type as an argument, because the caller is not going to be able to create an alternative type without substantially recreating the original.
11 |
12 | Here are some examples of useful interfaces from the standard library, you'll notice that all of these extremely popular interfaces have one thing in common - they all require very few functions.
13 |
14 | [error](https://golang.org/ref/spec#Errors)
15 |
16 | Error represents an error condition, and only returns a string with a description of the error.
17 |
18 | ```go
19 | type error interface {
20 | Error() string
21 | }
22 | ```
23 |
24 | [fmt.Stringer](https://golang.org/pkg/fmt/#Stringer)
25 |
26 | Used when formatting values for fmt.Printf and friends.
27 |
28 | ```go
29 | type Stringer interface {
30 | String() string
31 | }
32 | ```
33 |
34 | [io.Reader ](https://golang.org/pkg/io/#Reader)
35 |
36 | Read reads len\(p\) bytes from the data stream.
37 |
38 | ```go
39 | type Reader interface {
40 | Read(p []byte) (n int, err error)
41 | }
42 | ```
43 |
44 | [io.Writer](https://golang.org/pkg/io/#Writer)
45 |
46 | Write writes len\(p\) bytes from p to a data .stream.
47 |
48 | ```go
49 | type Writer interface {
50 | Write(p []byte) (n int, err error)
51 | }
52 | ```
53 |
54 | [http.ResponseWriter](https://golang.org/pkg/net/http/#ResponseWriter)
55 |
56 | HTTP handlers are passed a ResponseWriter to write the HTTP response to a request.
57 |
58 | ```go
59 | type ResponseWriter interface {
60 | // Header returns the header map used to set headers.
61 | Header() Header
62 |
63 | // Write writes the data to the connection as part of an HTTP reply.
64 | Write([]byte) (int, error)
65 |
66 | // WriteHeader writes an HTTP response header with just a status code - used for errors.
67 | WriteHeader(int)
68 | }
69 | ```
70 |
71 | ## Avoid the empty Interface
72 |
73 | The empty interface is written like this, but unlike most interfaces it requires no methods \(hence the empty brackets\):
74 |
75 | ```go
76 | interface{}
77 | ```
78 |
79 | It is the equivalent of:
80 |
81 | ```go
82 | type MyInterface interface {
83 | }
84 | ```
85 |
86 | Don't overuse empty interface - _it means nothing_. If you find yourself using empty interface and then switching on type, consider instead defining separate functions which operate on concrete types. Don't try to use empty interface as a poor man's generics - it is possible, but it subverts the type system and makes it very easy to cause panics.
87 |
88 | ## Accept interfaces, return types
89 |
90 | Interfaces are a way of avoiding tight coupling between different packages, so they are most useful when defined at their point of use, and only used there. If you export an interface as a return type, you are forcing others to use this interface only forever, or to attempt to cast your interface back into a concrete type.
91 |
92 | Do not design interfaces for mocking, design them for real world use, and don't add methods to them before you have a concrete use for the methods. The exception to this is of course the extremely common error interface.
93 |
94 | ## Avoid mixing interface and concrete types
95 |
96 | When defining interfaces in a package, don't also provide the implementation in the same package, because the receiver should define interfaces, not the implementer – you may sometimes need to break this guideline.
97 |
98 | * Define interfaces in the package that uses them
99 | * Define concrete types in the package that uses the package that accepts an interface
100 |
101 | ## Don't use pointers to interface
102 |
103 | You probably meant to use a pointer to your real type, or just a plain old Interface. You don't need a pointer to interfaces.
104 |
105 | ## Don't compare interface to nil
106 |
107 | An interface will only be nil when both their type and value fields are nil, so comparing interface to nil can have unexpected results. Don't do that. You can read more about this problem in the, but in summary if any concrete value has ever been stored in the interface, the interface will not be nil.
108 |
109 | ```go
110 | // E is a type conforming to the error interface
111 | type E struct{}
112 | func (e *E) Error() string { return fmt.Sprintf("%p", e) }
113 |
114 | // typederror returns an interface
115 | func typederror() error {
116 | var e *E = nil
117 | return e // e is not nil as expected
118 | }
119 | ```
120 |
121 | You can read more about the internals of interfaces in [Go Data Structures: Interfaces](https://research.swtch.com/interfaces) on Russ Cox's blog.
122 |
123 |
--------------------------------------------------------------------------------
/maps.md:
--------------------------------------------------------------------------------
1 | # Maps
2 |
3 | Maps in Go are hash tables, which provide fast access to values for a given key, one value per key. You can read more about maps in go in the [Go maps in action](https://blog.golang.org/go-maps-in-action) article on the go blog.
4 |
5 | ## Maps and Goroutines
6 |
7 | Go maps are not thread safe. If you need to access a map across goroutines \(for example in a handler\), you should use a [mutex](https://golang.org/pkg/sync/#Mutex) to protect access and make it a private variable protected by that mutex. Any access to the map should lock the mutex first, perform the operation, and then unlock it again. You can also use a RWMutex if you have a lot of reads and few writes.
8 |
9 | ```go
10 | type Data struct {
11 | // mu protects access to values
12 | mu sync.Mutex
13 | // values stores our data
14 | values map[int]int
15 | }
16 |
17 | func (d *Data)Add(k,v int) {
18 | d.mu.Lock
19 | d.values[k] = v
20 | d.mu.Unlock
21 | }
22 | ```
23 |
24 | Be careful when using mutexes or maps not to inadvertently make copies of them \(for example if you passed Data below by value to a function, the mutex would be copied along with the data\). For this reason you will sometimes see mutexes declared as pointers, but the important point is not to copy them \(or their data\).
25 |
26 | It is often clearer to put the mutex right next to the data they protect; convention is to name them mu and place them above the field which they protect. You should very rarely need to explicitly initialise a mutex or use a pointer to a mutex, the zero value can be used directly as above.
27 |
28 | ## Map values
29 |
30 | You cannot take the address of map keys or values. You cannot assign to struct fields for structs stored as values in the map, they are immutable. You should instead store pointers in the map, and you can then mutate the structs those pointers refer to.
31 |
32 | ```go
33 | // Define a type
34 | type T struct {F int}
35 |
36 | // Define a map
37 | m := map[int]T{1:{F:1}}
38 |
39 | // Accessing fields on T is not allowed
40 | m[1].F = 4
41 | ```
42 |
43 | This produces the result:
44 |
45 | > cannot assign to struct field m\[1\].F in map
46 |
47 | ## Key order
48 |
49 | Maps have no defined order, and keys and values returned are randomised to enforce this.
50 |
51 | ```go
52 | // Define a map
53 | m := map[int]int{1: 1, 2: 2, 3: 3, 4: 4, 5: 5}
54 | // Range 5 times
55 | for i := 0; i < 5; i++ {
56 | // The order of keys and values is undefined
57 | for k, v := range m {
58 | fmt.Printf("(%d:%d)", k, v)
59 | }
60 | fmt.Println("")
61 | }
62 | ```
63 |
64 | The output of ranging on a map is not deterministic:
65 |
66 | > \(2:2\)\(3:3\)\(4:4\)\(5:5\)\(1:1\)
67 | >
68 | > \(1:1\)\(2:2\)\(3:3\)\(4:4\)\(5:5\)
69 |
70 | To range map keys in order, get the keys, sort them to some predefined order, then range the keys and get the values.
71 |
72 | ## Keys not in the map
73 |
74 | A map retrieval yields a zero value when the key is not present, so always check a key is present before using it if this distinction is important to you \(for example if you want to treat an empty string stored in the map differently from a missing value\).
75 |
76 | ```go
77 | key := "foo"
78 |
79 | // You can check whether a value exists in the map
80 | v, ok := m[key]
81 | if ok {
82 | // do something with v
83 | }
84 |
85 | // Or you can simply get the value (a zero value if not present)
86 | v := m[key]
87 | ```
88 |
89 | ## Nil Maps
90 |
91 | Maps must always be initialised before use, the zero value is not useful.
92 |
93 | ```go
94 | // Allowed
95 | var s []int
96 | s = append(s,1)
97 |
98 | // You must init a map before use
99 | m := make(map[string]int, 10)
100 |
101 | // This is ok
102 | m["key"] = 1
103 |
104 | // If you assign to a nil map - panic
105 | var mnil map[string]int
106 | mnil["key"] = 1
107 | ```
108 |
109 | If you assign to a nil map panic ensues
110 |
111 | > panic: assignment to entry in nil map
112 |
113 | A nil map behaves like an empty map when reading, so if you are sure you should only read from it, it is safe to use.
114 |
115 | ## Use the right data structure
116 |
117 | If you need fast random access, use a map (for example if storing cache items by key). If you need fast iteration or sorting, use a slice – iterating a map is significantly slower than iterating over a slice.
118 |
119 |
--------------------------------------------------------------------------------
/packages.md:
--------------------------------------------------------------------------------
1 | # Packages
2 |
3 | In Go packages are a way of scoping code, and map exactly to folders. Within a package, code can refer to any identifier within that package, public or private, while those importing the package may only reference public identifiers beginning with an uppercase letter. Each package can contain multiple Go files, and each Go file in a package begins with a declaration at the top of the file for the form:
4 |
5 | ```
6 | // Package x ... this comment explains succinctly what the package does
7 | package x
8 | ```
9 |
10 | where x is the package name. The name should be short, explicit and avoid punctuation - see [Package Names](https://blog.golang.org/package-names) on the Go blog. The line preceding the package declaration can be used to document the package - typically this will be done in the most important file in the package, the one which has the same name as the package itself.
11 |
12 | ## Packages vs Dirs
13 |
14 | Packages are a directory on the file system containing .go files. **You cannot have multiple packages within one directory**. By convention, the package name is the same as the directory name, and the directory should contain a file of the package name with the .go suffix which contains the primary exports. You can if necessary rename packages on import and use a different package name to the folder name.
15 |
16 | ## Workspaces
17 |
18 | A common cause of confusion for beginners with Go is the concept of workspaces, where all your go code lives. These are not checked into version control directly, and when beginning you will just have one workspace which is your gopath. This contains three dirs src, pkg, and bin, as detailed in the getting started section. Projects will typically be hosted under paths like this: src/host/username/project which are used as the import paths, and below that path can contain any directories you want to structure your project.
19 |
20 | ## Project structure
21 |
22 | There are few restrictions on structure in a Go project, but for a beginner there are a few guidelines to keep in mind. The simplest structure is [one main package](https://golang.org/doc/code.html#Command) at the top level of your repository. This can be fetched by go get which is what users will expect. Beyond this you can structure it as you wish, with as many directories as you wish, though each directory with go files must be a separate package, and cannot contain more than one package.
23 |
24 | A few guidelines to live by:
25 |
26 | * Make your project go-gettable by having the main package at the top level
27 | * Err on the side of fewer packages, not more
28 | * Import lower packages from higher levels, and try to avoid interdependencies - low level packages should stand alone
29 | * Store internal libraries not for export in a directoy named internal
30 | * Vendor dependencies in a top level directory named vendor
31 | * Always flatten vendor directories into this one top level folder
32 |
33 | Packages can contain as many files as you want, not just one, and typically should be split by file. Don't split packages by type and have one type per package, a package will normall contain several types.
34 |
35 | ## Blank imports
36 |
37 | You can import packages with the blank identifier solely for their init function, to do this prefix the import with \_
38 |
39 | ```go
40 | import _ "net/http/pprof"
41 | ```
42 |
43 | This makes it clear the package is unused but the init function will be called \(which may introduce side effects in this case registering handlers\).
44 |
45 | ## Don't import main
46 |
47 | You should never try to import the main package. Always try to import from the top level down, so your lowest level packages know nothing about the levels above. This is not always possible but is a good guideline – avoid a fragile web of dependencies between internal packages which makes your program harder to reason about - prefer some copying to interdependent packages.
48 |
49 | ## Cyclic dependencies
50 |
51 | If you run into cyclic imports, you have a package A which imports package B which then imports package A. To avoid this, try to keep your imports in one direction, importing small packages into the top level one. For example for a git command line tool you might have a main command, which imported a package which contains all the git specific structures and could be used as a library by that command, a server or any other type of app. The git package should never know about packages which import it.
52 |
53 | If you structure your apps this way you will avoid cyclic imports and you should consider them a hint that something is wrong with your design.
54 |
55 | ## Using goimports
56 |
57 | If you choose to use [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) instead of go fmt on save, and have several packages with the same name at different places in your `GOPATH`, it may not choose the correct imports, or even those closest to the importing package.
58 |
59 | There is a [fix](https://github.com/golang/go/issues/17557) almost ready for this, but that leaves the problem of imports from third party packages which might be replaced by an import from another package with the same name and similar code by mistake. So if you use the goimports tool use it with caution and always check the imports inserted - it chooses the shortest name by default and may import an unexpected package if you have homonym packages under gopath.
60 |
61 | ## Using go get
62 |
63 | You should be aware that go get fetches the most recent version of any go package. This means if you are sharing code and want reproducible builds, you need to use the vendor directory. In practice because of the Go culture of no breaking changes this is rarely a problem, but sometimes APIs change, and this can lead to subtle breakage in your program if you are not careful. Try to vendor any dependencies you don't control to avoid this problem.
64 |
65 | ## Vendoring packages
66 |
67 | There is no official package manager tool as yet in Go, though work is progressing on one \(go [dep](https://github.com/golang/dep)\). For now you can vendor your packages inside a vendor directory in your project, and those imports will be used instead of the imports currently. You can read more about this behaviour in the go command documentation under [Vendor Directories](https://golang.org/cmd/go/#hdr-Vendor_Directories).
68 |
69 | ## Internal directories
70 |
71 | A subdirectory named 'internal' can be used for packages that can only be imported by nearby code, not by any other package. Packages in an internal directory are only accessible to those in the same file tree, rooted at the parent of the internal directory. So if one project includes 'myproj/example/internal/foo' only packages under myproj can import foo.
72 |
73 | This is typically used for dependencies which should not be shared and are for internal use only \(for example libraries with an unstable api\).
74 |
75 | ## Don't import unsafe
76 |
77 | If you can, avoid using the unsafe or reflect packages in your code – this also applies to libraries you import. Package unsafe bypasses the Go type system. Packages that import unsafe may be non-portable and are not protected by the Go 1 compatibility guidelines.
78 |
79 | ## Don't import reflect
80 |
81 | Package reflect also bypasses the Go type system, so programs using it don't have the benefit of strict type checks, programs that use it will typically be slower and more fragile, and may use interface{} too much. It can be temping when writing a library to use reflect and attempt to handle several types in the same function \(for example an array of types or a single type\), but this makes it confusing for users \(no indication of what type should be passed in\), and easy to miss a type and cause panics.
82 |
83 | ```go
84 | // Think twice!
85 | import "reflect"
86 |
87 | // Using reflect and empty interfaces
88 | // obscures your intent for callers
89 | // and subverts the type system
90 | func MyAPIFunc(myvar interface{}) {
91 | // ... requires typecasts and perhaps reflect
92 | }
93 |
94 | ```
95 |
96 | ## Cgo is not Go
97 |
98 | Cgo is a useful crutch to allow calling into C libraries from Go and vice versa. Build times will be slower, and you won't be able to cross compile code easily. This throws away many of the advantages of writing code in Go. You can use it to do some very useful things like running Go programs on iOS or Android, or interfacing with large libraries of existing code written in C, and it is indispensable for those uses, but if you can avoid using it, do so.
99 |
100 | ## Syscall and cgo are not portable
101 |
102 | If you're using cgo or syscall, your package probably isn't portable, and needs build tags for each platform you're going to target. Unlike other packages in the go ecosystem, the cgo or syscall packages are not portable across platforms. If you import them, you should be aware of this.
103 |
104 | ## Init funcs
105 |
106 | Packages can contain \(one or several\) init\(\) functions to set up state, these are run after all the imports and package variables have been initialised:
107 |
108 | ```go
109 | // init() loads the configuration for this package
110 | func init() {
111 | if user == "" {
112 | log.Fatal("$USER not set")
113 | }
114 | if home == "" {
115 | home = "/home/" + user
116 | }
117 | }
118 | ```
119 |
120 | Avoid using this if you can, and particularly avoid using it to set global state \(like set flags in other packages, or attach handlers to the http.DefaultServeMux\) - doing so can lead to subtle bugs. The init function should be solely concerned with the package it is in.
121 |
122 | ## Flags
123 |
124 | Libraries and packages should not define their own flags or use the flags package. Instead take parameters to define behaviour - the application \(main\) should be in charge of parsing flags and passing the chosen configuration values in to a package. Otherwise it will be difficult to reuse the package or import it anywhere else.
125 |
126 |
--------------------------------------------------------------------------------
/scope.md:
--------------------------------------------------------------------------------
1 | # Functions & Scope
2 |
3 | Go scope is defined by blocks. A block is a possibly empty sequence of declarations and statements within matching brace brackets. The levels of blocks are roughly:
4 |
5 | 1. Universal block - predeclared identifiers, keywords etc
6 | 2. Package block - any top level identifier outside a function
7 | 3. Function block - receivers, arguments, return values and identifiers within a function
8 | 4. Inner Blocks - identifiers declared within an inner block in a function end at the end of that block
9 |
10 | For example package block identifiers include variables and functions within the package:
11 |
12 | ```go
13 | // An exported variable
14 | var Exported = "hello"
15 |
16 | // A private variable
17 | var pkgOnly = 1
18 | ```
19 |
20 | Function block identifiers include receivers, arguments, return values and identifiers within a function:
21 |
22 | ```go
23 | // Functions define a block
24 | func (receiver T)Function(argument int) (returnValue int) {
25 | var v int
26 | }
27 | ```
28 |
29 | A block within a function:
30 |
31 | ```go
32 | func (receiver T) Function(argument int) (returnValue int) {
33 | var v int
34 |
35 | // Inner block begins
36 | if argument > 0 {
37 | var v int // shadows outer v
38 | v = 4 + v
39 | }
40 | // block ends
41 |
42 | fmt.Printf("v=%d\n", f)
43 | return v
44 | }
45 |
46 | func main() {
47 | T{}.Function(1)
48 | }
49 | ```
50 |
51 | The full rules for scope in Go are set out in the [Language Spec](https://golang.org/ref/spec#Declarations_and_scope). Blocks all create a new scope, so an if statement will start a new block, and variables declared within it will go out of scope when it ends. In contrast with Java, blocks protect their variables from the outer scope, you can overwrite a variable in the outer scope by declaring a new one in an inner scope. If required declare the variable before an if block and use it within. Otherwise Go is broadly similar to other languages in the C family, and there are few surprises here apart from shadowing variables inadvertently \(see below\).
52 |
53 | ## Shadowing
54 |
55 | Be careful when using the automatic assignment operator := that you don't accidentally shadow variables from the outer block. In the case below err is created twice, and if you relied on err being set in the outside scope it would not be. In most cases this isn't a problem but it's something to be aware of. Most linters warn about any dangerous instances of shadowing so if you use a linter it is not usually a problem in practice.
56 |
57 | ```go
58 | // err is created
59 | err := f()
60 | if err != nil {
61 | // err is recreated - outer err is not updated
62 | v, err := f()
63 | }
64 | ```
65 |
66 | ## Defer runs after the function
67 |
68 | The defer statement does not run at the end of a block, but at the end of the containing function.
69 |
70 | ## Defer arguments are frozen
71 |
72 | The defer statement freezes its arguments the instant it is evaluated \(the line it occurs in the source code\), not when it executes after leaving the containing function. This means any values changed after it occurs do not get passed to defer.
73 |
74 | ```go
75 | // Define a string s "hello"
76 | s := "hello"
77 |
78 | // At point of defer, s is "hello"
79 | defer fmt.Print(s)
80 |
81 | // Not used, even though defer runs after
82 | s = "hello world"
83 | ```
84 |
85 | ## Switch fall through
86 |
87 | Case statements inside a switch in go **do not fall through by default**, so break is not required. You can use the fallthrough keyword to do so if you require. This should be used sparingly and carefully annotated because of the difference with C - programmers not familiar with Go may be tripped up by it so use this sparingly if at all.
88 |
89 | ```go
90 | switch(i) {
91 | case 0:
92 | // do something here
93 | // ...
94 | case 1:
95 | // do something here
96 | // ...
97 | case 3:
98 | // the fallthrough keyword is required
99 | // to fall through explicitly
100 | fallthrough
101 | default:
102 | // do something
103 | }
104 | ```
105 |
106 | ## Naked Returns
107 |
108 | In go, naked returns \(the use of the keyword return without parameters\) will return the current state of the named return values. This is sometimes used as a shortcut to avoid specifying what is returned, but has fallen somewhat out of favour. Try to avoid using naked returns in your go code, they are unclear, particularly within or at the end of a large function. Instead specify exactly what will be returned, and use nil or zero value for all values when returning an error.
109 |
110 | ## Prefer synchronous functions
111 |
112 | Try to write synchronous functions, which can then be transformed by use of the go keyword into asynchronous ones. Do not attempt to make your API async by default by using go keywords within library functions. It is easy to add concurrency with the go keyword, but impossible to remove it if a function uses it internally.
113 |
114 | ## Prefer functions over methods
115 |
116 | Coming from an object-oriented background, many programmers reach for structs and methods first. Before using a method, you should consider whether you could instead use a function. Functions are independent of the data they work with, and ideally use their inputs and no other state, so that the output is predictable. Use a method where you need to reflect the state of the type the method is attached to.
117 |
118 |
--------------------------------------------------------------------------------
/servers.md:
--------------------------------------------------------------------------------
1 | # Writing Servers in Go
2 |
3 | When writing servers in go, you'll probably make extensive use of the standard library.The net and net/http packages are some of the most widely used in the Go ecosystem, and come with some unique pitfalls. The source to the [net/http package](https://golang.org/pkg/net/http/) is surprisingly straightforward and readable, and if you're going to write servers, you should read through the docs and the source of this package at least once to see what is available.
4 |
5 |
6 | ## Getting Started
7 |
8 | Running a simple server is drop-dead simple in Go, and to get started all you need is a handlerFunc as the http package contains a default router and server you can use. A Handler in go is a function which responds to http requests, usually for a specific route on your server (thought they can be general, like an error handler or a static file handler). It is a very simple function with two arguments: the Request, and an http.ResponseWriter to write a response to. Except for reading the body, handlers should not modify the provided Request.
9 |
10 | ```go
11 | // A handler function in go reads the request
12 | // and writes a response to the ResponseWriter
13 | func HelloServer(w http.ResponseWriter, req *http.Request) {
14 | io.WriteString(w, "hello, world!\n")
15 | }
16 |
17 | func main() {
18 | // Attach our handler function to the default mux
19 | http.HandleFunc("/hello", HelloServer)
20 |
21 | // Ask the http package to start a server
22 | // on port 8080 (note the port is a string)
23 | err := http.ListenAndServe(":8080", nil)
24 | if err != nil {
25 | log.Fatal(err)
26 | }
27 | }
28 | ```
29 |
30 | ## Handlers
31 |
32 | When writing handlers, be aware that the http.Request.Body is an io.ReadCloser, which means it doesn't have to be read into memory all at once, but can be read as a stream. Consider using the io.Reader interface if you need to pass it to a decoder or receive large requests. The request Body will always be non-nil, and the server will close it, so handlers do not need to.
33 |
34 | Similarly, the http.ResponseWriter is an io.Writer, so it can be written to in a stream, and io.Copy can be used to read from one source and write it to the output (for example when reading a file) without reading everything into a buffer first.
35 |
36 | Try to use these interfaces if possible when reading from requests and writing responses.
37 |
38 | ## Servers
39 |
40 | You normally should not use the http.DefaultServeMux, in case other packages have decided to register handlers on it, and you expose those handlers without knowing about it. For example the pprof package by default will install handlers simply by being imported.
41 |
42 | Also set the timeouts on your server explicitly to sensible defaults as the default timeouts are not advisable, but cannot be changed because of the Go 1 promise. You can read more about these timeouts in[ the complete guide to net/http timeouts](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) and [So you want to expose go on the internet](https://blog.cloudflare.com/exposing-go-on-the-internet/) by @filosottile at cloudflare. If you're interested in networking, the cloudflare blog has a lot of good articles on Go.
43 |
44 | ```go
45 | // Setting up your own server,
46 | // specifying some default timeouts
47 | srv := &http.Server{
48 | ReadTimeout: 5 * time.Second,
49 | WriteTimeout: 10 * time.Second,
50 | IdleTimeout: 120 * time.Second,
51 | TLSConfig: tlsConfig,
52 | // Always set the router to avoid using http.DefaultServeMux
53 | // which is accessible to all other packages
54 | Handler: serveMux,
55 | }
56 | log.Println(srv.ListenAndServeTLS("", ""))
57 | ```
58 |
59 | ### Implicit goroutines
60 |
61 | The http server uses goroutines to run your handlers and serve multiple requests in parallel, so **each handler is in a new goroutine**. This means you have to be careful about sharing memory between handlers. If for example you have a global config struct or cache, this must be protected by a mutex. Always run your server programs with a race tester to catch bugs like this as they can creep in if you are not vigilant about finding them.
62 |
63 | ### Listen and Serve Blocks
64 |
65 | If you launch the http server in your main goroutine, don't expect control to return to your main function until the end of the program, because it blocks waiting for input until an error occurs.
66 |
67 | ```go
68 | func main() {
69 | http.HandleFunc("/hello", HelloServer)
70 | err := http.ListenAndServe(":12345", nil)
71 | if err != nil {
72 | log.Fatal(err)
73 | }
74 | log.Println("this line will never execute")
75 | }
76 | ```
77 |
78 | ### Serving Files
79 |
80 | Don't perform **any** file operations on paths before you clean them, and root them at a known good path. If you're serving an entire directory of files, consider using http.FileServer instead of http.ServeFile:
81 |
82 | ```go
83 | fileServer := http.FileServer(http.Dir("./public/static"))
84 | http.Handle("/static/", http.StripPrefix("/static", fileServer))
85 | ```
86 |
87 | If you are using os.Stat, ioutil.ReadAll, or [http.ServeFile](https://www.gitbook.com/book/kennygrant/go-bestiary/edit#) or similar functions with user input \(be that in the request url or params\), be sure to sanitise the file path first, and root it at a known public path. The default mux will usually strip .. from urls before presenting to handlers and ServeFile has some protections against directory traversal, but it is better to be very careful when accessing local files based on anything from user input.
88 |
89 | If you wish to serve static content but present a custom 404 page to users, set up a file handler which checks if files exist and returns 404 or 401 in case of problems accessing the file, but otherwise calls ServeFile.
90 |
91 | ```go
92 | // Clean the path and prefix with our public dir
93 | localPath := "./public" + path.Clean(r.URL.Path)
94 |
95 | // Check the file exists
96 | s, err := os.Stat(localPath)
97 | if err != nil {
98 | // If file not found return 404 page
99 | if os.IsNotExist(err) {
100 | renderNotFound()
101 | return
102 | }
103 |
104 | // For other file errors render unauthorised and return
105 | http.Error(w, "Not Authorized", http.StatusUnauthorized)
106 | return
107 | }
108 |
109 | // If not a file return 404 page
110 | if s.IsDir() {
111 | renderNotFound()
112 | return
113 | }
114 |
115 | // Serve the file content
116 | http.ServeFile(w, r, localPath)
117 | ```
118 |
119 | ### Bad Requests
120 |
121 | The built in Go server will reject bad requests before they hit your handlers, so you will never see them. It will return a code 400 Bad Request to the client. There is at present no way to override this. Normally this isn't a problem but it's something to be aware of. For example this bad request using curl to hit a local go server would never hit your handlers:
122 |
123 | ```
124 | # Perform an invalid request using curl
125 | curl http://localhost:3000/%3
126 | ```
127 |
128 | and would simply return to the client:
129 |
130 | > 400 Bad Request
131 |
132 | If your go server is behind a proxy like nginx or a load balancer those will also intercept such requests.
133 |
134 | ### Panics in goroutines
135 |
136 | If a handler panics, the server assumes that the panic was isolated to the current request, recovers, logs a stack trace to the server log, and closes the connection. So the server will recover from any panics in your handlers, but if your handlers use the go keyword, they must protect against panics **within any separate goroutines** they create, otherwise those goroutines can crash the entire server with a panic. See the errors chapter for more details.
137 |
138 | One approach to this problem is never to make mistakes. However with unpredictable, malformed or downright malicious data coming from outside the application in parameters or files this can be difficult, so it may be worth protecting against panics in any goroutines launched from your handlers.
139 |
140 | ## Cryptography
141 |
142 | If you're trying to work with cryptography in go, definitely view this talk from [George Tankersley](https://www.golangnews.com/stories/1469-video-crypto-for-go-developers-gophercon-crypto) at CoreOS on Crypto for Go developers, which comes with [cryptopasta](https://github.com/gtank/cryptopasta) example code.
143 |
144 | ### Random Numbers
145 |
146 | If generating random numbers for a server, you probably want them to be unpredictable, so use crypto/rand, not math/rand.
147 |
148 | ```go
149 | // Example by George Tankersley
150 | // NewEncryptionKey generates a random 256-bit key
151 | // for Encrypt() and Decrypt().
152 | // It panics if the source of randomness fails.
153 | func NewEncryptionKey() *[32]byte {
154 | key := [32]byte{}
155 | _, err := io.ReadFull(rand.Reader, key[:])
156 | if err != nil {
157 | panic(err)
158 | }
159 | return &key
160 | }
161 | ```
162 |
163 | ### Comparing Passwords
164 |
165 | If you're comparing passwords or other sensitive data, to avoid timing attacks, make use of the [crypto/subtle](https://golang.org/pkg/crypto/subtle/) subtle.ConstantTimeCompare, or better still use the [bcrypt](https://godoc.org/golang.org/x/crypto/bcrypt) package library functions bcrypt.CompareHashAndPassword.
166 |
167 | ### Cookies
168 |
169 | Cookies are just headers passed between client and server, so they're not complex, but go provides some utilities for manipulating them. The server sends cookies with SetCookie on the http.ResponseWriter (not on the request). NB that invalid cookies may be silently dropped when set or received. If you want to use cookies with invalid values including strings like space or @, you will need to use your own cookie implementation.
170 |
171 | ```go
172 | // handler sets a cookie on the request
173 | func handler(w http.ResponseWriter, r *http.Request) {
174 | cookie := http.Cookie{
175 | // Set the name
176 | Name: "session_name",
177 | // Set a value on the cookie
178 | Value: "cookie_value",
179 | // Always set the domain
180 | Domain: "example.com",
181 | // Always set the path
182 | Path: "/",
183 | // Optional expiry time
184 | Expires: time.Now().AddDate(1, 0, 0),
185 | // MaxAge=0 means no Max-Age
186 | MaxAge: 0,
187 | // Allow only the server to access the cookie
188 | HttpOnly: true,
189 | // Set this if using https
190 | Secure: true,
191 | }
192 | // Set the cookie on the response
193 | http.SetCookie(w, &cookie)
194 | // Write the rest of the response
195 | w.WriteHeader(http.StatusOK)
196 | io.WriteString(w, "hello")
197 | }
198 | ```
199 |
200 | If you want to store information in cookies you should try to keep it limited, and encrypt the information stored, as it is stored on the client machine outside your control.
201 |
202 | The [gorilla/sessions](http://www.gorillatoolkit.org/pkg/sessions) package allows you to use encrypted cookies and store session-specific information.
203 |
204 |
205 | ## Making http requests
206 |
207 | Your program may need to fetch http resources, and the net/http package offers http.Get or http.Client in order to help with this, but there are some issues you should be aware of.
208 |
209 | ### Client Timeouts
210 |
211 | The http client has no default timeout, which can be a problem. In this example problem the client waits 10 minutes before exiting.
212 |
213 | ```go
214 | package main
215 | import (
216 | “fmt”
217 | “net/http”
218 | “net/http/httptest”
219 | “time”
220 | )
221 | func main() {
222 | svr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
223 | // handler sleeps for 10 minutes
224 | time.Sleep(10 * time.Minute)
225 | }))
226 | defer svr.Close()
227 |
228 | // Make a get request with default client
229 | fmt.Println(“making request”)
230 | http.Get(svr.URL)
231 | fmt.Println(“finished request”)
232 | }
233 | ```
234 |
235 | Instead, you should create a client explicitly with a timeout:
236 |
237 | ```go
238 | // Create a client with a timeout of 10 seconds
239 | var netClient = &http.Client{
240 | Timeout: time.Second * 10,
241 | }
242 | response, _ := netClient.Get(url)
243 | ```
244 |
245 | ### Closing the response body
246 |
247 | Don't close the response body before you check if there was an error.
248 |
249 | ```go
250 | r, err := http.Get("https://example.com")
251 | // Don't defer close here
252 | if err != nil {
253 | return err
254 | }
255 | // It's safe to defer close
256 | // once you know there was no error
257 | defer r.Body.Close()
258 | ```
259 |
260 | ### Accepting uploads
261 |
262 | If accepting uploads via a multipart form, be aware that temp files created will not be automatically deleted by Go. You should therefore delete them with [Request.Form.RemoveAll](https://golang.org/pkg/mime/multipart/#Form.RemoveAll) after they have been used by the server.
263 |
264 |
265 | ### Check Status Codes
266 |
267 | Always check the status code of the response when making a request with the Go http client. If the status is in the 200 range you can use the response as is. If it is in the 300 range it is a redirection. If it is in the 400 range you have a problem with your request which you should fix \(e.g. invalid headers, invalid URL\). If it is in the 500 range there was a problem on the server end, so you need to inform the user.
268 |
269 | The error returned by http.Get and friends informs you if there was an error reading the response, not if the response was successful.
270 |
271 | ```go
272 | url := "https://example.com"
273 | resp, err := http.Get(url)
274 | if err != nil {
275 | return fmt.Errorf("error getting %s: %v",url,err)
276 | }
277 |
278 | // Unexpected response, inform the user
279 | if resp.StatusCode < 200 || resp.StatusCode > 299 {
280 | return fmt.Errorf("unexpected http status:%d",resp.StatusCode)
281 | }
282 |
283 | // No errors fetching, so the response is ok to use
284 | ```
285 |
286 | ## Utilities
287 |
288 | Under the net/http package are several utility packages which make life easier when debugging or testing http Requests.
289 |
290 | In particular, the [httputil](https://golang.org/pkg/net/http/httputil) package provides:
291 |
292 | [httputil.DumpRequest](https://golang.org/pkg/net/http/httputil/#DumpRequest)
293 |
294 | DumpRequest outputs a nicely formatted version of the request passed to it, making the request content much clearer.
295 |
296 | [httputil.DumpResponse](https://golang.org/pkg/net/http/httputil/#DumpResponse)
297 |
298 | DumpResponse outputs a nicely formatted version of the response passed to it, making the response content much clearer.
299 |
300 | [httputil.ReverseProxy](https://golang.org/pkg/net/http/httputil/#ReverseProxy)
301 |
302 | Provides a simple reverse proxy handler
303 |
304 | And the [httptest](https://golang.org/pkg/net/http/httptest) package provides several useful functions and types:
305 |
306 | [httptest.NewRequest](https://golang.org/pkg/net/http/httptest/#NewRequest)
307 |
308 | Provides a new request suitable for sending to a Handler for testing.
309 |
310 |
311 | [httptest.ResponseRecorder](https://golang.org/pkg/net/http/httptest/#ResponseRecorder)
312 |
313 | ResponseRecorder is a ResponseWriter which records the response data written by a handler so that it can be compared in tests to the expected response.
314 |
315 |
316 | ## Profiling
317 |
318 | The [http/pprof](https://golang.org/pkg/net/http/pprof/) package provides excellent support for profiling your server, and can be used to generate data for use in flame graphs or other graphical representations of your program.
319 |
320 | To use it, simply import the package:
321 |
322 | ```go
323 | import _ "net/http/pprof"
324 | ```
325 |
326 | **Beware** though if you register it there are hidden side effects. The pprof package will attach endpoints to the default serve mux during init, but unfortunately this behaviour cannot change due to the Go 1 promise. For this and performance reasons you should not import it in production, but use it for development only. You can find out more about profiling in [Profiling Go Programs](https://blog.golang.org/profiling-go-programs).
327 |
328 | ## Latency
329 |
330 | When optimising servers it's worth being aware of the comparative times for operations. If your database is in a different data centre for example on another continent, it doesn't matter how optimised your code is if it needs to spend 150ms for every database request.
331 |
332 | 1 CPU cycle takes 0.0000003ms, Cache access 0.0000129ms, Solid state disk 0.05ms, Rotational disk access around 1ms and on internet request around 200ms across continents. So make sure you're optimising the right operations - avoiding network access will always be faster than any other optimisation.
333 |
334 |
--------------------------------------------------------------------------------
/setup.md:
--------------------------------------------------------------------------------
1 | # Setting up Go
2 |
3 | ##### If you have already written and compiled Go code before you can probably skip this section.
4 |
5 | To get up and running with Go, you should follow the [Getting Started](https://golang.org/doc/install) instructions on the go website. If you're on Mac or Windows there is an installer which will make life easier for you and is the recommended route to installation – you can find the latest installers at the [download](https://golang.org/dl/) page.
6 |
7 | You can use homebrew to install go on macs, but you're probably better sticking with the official installer package available on golang.org. Do not use both.
8 |
9 | ## Setting GOPATH
10 |
11 | `GOPATH` is the bugbear of many new users of Go, but it is simply a place to store go code and binaries. This environment variable was introduced in order to make it simple for the Go tool to install dependencies automatically, and to keep all go code in one place. Since `Go 1.7` this has defaulted to:
12 |
13 | ```
14 | $HOME/go
15 | ```
16 |
17 | `GOPATH` contains three directories:
18 |
19 | * bin - for installed binaries
20 | * src - for go source code
21 | * pkg - a cache for intermediate build products
22 |
23 | You may want to add your `$GOPATH/bin` folder to your path so that installed go tools are found automatically.
24 |
25 | So you don't need to set it explicitly, but you do need to put all your go code under `$GOPATH/src` for go tools to work correctly. This is where code will be installed when you run go get, and this is where your dependencies will be stored. There is no concept of versioning in the go get tool, so it will fetch the latest master of any dependencies.
26 |
27 | While it can be frustrating for newcomers, you should accept that all go code will live in your `GOPATH` – there are workarounds to attempt to keep code elsewhere, but the standard tools at present assume you use `GOPATH` for your code. You may find projects sometimes use folders called pkg or src internally inside their main go gettable folder, but this is unrelated to `GOPATH` and is just a way of organising their code. The only special src folder is that at the root of your `GOPATH`.
28 |
29 | The similarly named ENV variable `GOROOT` need only be set** if installing to a custom location**. In most circumstances you can ignore `GOROOT`, as it is not required for a normal Go setup.
30 |
31 | ## Checking your setup
32 |
33 | To test your setup and confirm go is installed correctly, try the go command:
34 | ```bash
35 | go version
36 | ```
37 |
38 | You should see output something like this telling you the go version and the installed platform of the go tools:
39 |
40 | > go version go1.9 darwin/amd64
41 |
42 | Then try a very simple program to check that your go setup is working. First save the following text file in `$GOPATH/src/hello.go` with UTF-8 encoding:
43 |
44 | ```go
45 | // Package main contains the entrypoint of the program
46 | package main
47 |
48 | // The imports list dependencies
49 | import (
50 | "fmt"
51 | )
52 |
53 | // The function main is the entry point of your program
54 | func main() {
55 |
56 | // Print a string to stdout
57 | fmt.Printf("hello, 世界\n")
58 |
59 | }
60 | ```
61 |
62 | and use the go command to run it:
63 |
64 | ```bash
65 | go run $GOPATH/src/hello.go
66 | ```
67 |
68 | You should see the output:
69 |
70 | > hello, 世界
71 |
72 | If you don't, your installation is not working and you should recheck the steps above and check the [Installation](https://golang.org/doc/install) instructions on the Go website.
73 |
74 | ## Cross Compiling
75 |
76 | You can use Go to compile programs not just for the platform you're on, but for another supported platform, like Windows or Linux if you're working on a mac. This makes it very easy to deploy programs as single binaries, without worrying about the dependencies or building on your server. For example if you want to build the hello.go program above for the Linux platform, you could use:
77 |
78 | ```bash
79 | GOOS=linux GOARCH=arm go build -o hello-linux $GOPATH/src/hello.go
80 | ```
81 |
82 | This will give you a binary which runs on linux arm \(or any other platform you choose\), without any extra fuss.
83 |
84 | # Editors
85 |
86 | Some of the most widely used editors for Go programmers, roughly in order of popularity are: [VSCode](https://code.visualstudio.com/), Vim \(with [vim-go](https://github.com/fatih/vim-go)\), [IntelliJ IDEA](https://www.jetbrains.com/idea/), Emacs, and [Atom](https://atom.io/). Since go just requires text files and doesn't require any special support, you can easily switch between editors, but make sure your editor has support for running go fmt on save, and ideally support for looking up definitions from within the editor. Another useful tool for browsing go source code is [sourcegraph](https://sourcegraph.com) \(which happens to be written in Go\).
87 |
88 | You should always run [go fmt](https://blog.golang.org/go-fmt-your-code) on your code, and ideally a linter like gometalinter too. Try to set up your editor initially you run these on save. Almost all public go code is run through go fmt, and if interacting with other go programmers, this is taken as a given. Most editors have a go plugin like [vscode-go](https://github.com/Microsoft/vscode-go/) or [vim-go](https://github.com/fatih/vim-go) which will take care of this for you.
89 |
90 | #### Linter Warnings
91 |
92 | If you pay attention to linter warnings, and make sure you fix them all, you'll find your go code is more idiomatic and you should avoid common errors like executing goroutines with values from a range loop.
93 |
94 | ## Dependencies
95 |
96 | > Duplication is far cheaper than the wrong abstraction
97 | > Sandi Metz
98 |
99 | There is a culture of limiting dependencies in Go, unlike some other ecosystems, the design of Go does not encourage importing libraries for trivial functions. If you're coming from node this will come as a shock, but you should try to adapt to the very different culture, as there are some advantages.
100 |
101 | The biggest problem with dependencies in the long term is entropy over time - the more dependencies you have, the greater the chance of changes which break your build \(for security, or api changes, or new features\), and the more painful it becomes to keep up to date with the ecosystem you've bought into. This is why it is useful to limit your dependencies, and explicitly version those you have, and why vendoring \(taking a copy of dependencies frozen at a given version\) has become an accepted solution in the Go community to importing dependencies.
102 |
103 | Although Go is statically typed which offers some protection, you should not assume that if dependencies change and your code compiles everything is working. A change to a dependency might make subtle changes to defaults or values which while they still compile result in the unexpected behaviour. The only solution to managing dependencies is to freeze them at the import version and inspect changes carefully before upgrading.
104 |
105 | ## Resources
106 |
107 | You should start by working through the[ Go Tour](https://tour.golang.org/welcome/1), you should then skim [Effective Go](https://golang.org/doc/effective_go.html) \(you will want to come back to this later\). You should read the [docs](https://golang.org/pkg/) from the standard library, which contain lots of examples, and finally you might want to refer to [Go By Example](https://gobyexample.com/) - a set of code snippets.
108 |
109 | ### Go Playground
110 |
111 | The Go Playground is a web service that runs on golang.org's servers. The service receives a Go program, compiles, links, and runs the program inside a sandbox, then returns the output. The intention is for go playground links to last forever \(in internet time that's at least a few years\).
112 |
113 | The playground has certain limitations, mostly for security reasons it restricts certain operations like writing files or accessing the network, and the time is the same for each run. There are also limits on execution time and on CPU and memory usage. See [Inside the Go playground](https://blog.golang.org/playground) for more details.
114 |
115 | ### Stack Overflow
116 |
117 | You can find the answer to many questions about Go on [stackoverflow](https://stackoverflow.com/questions/tagged/go), in particular small questions of grammar. A few tips for using this resource:
118 |
119 | * Be sure to search for related questions with the go tag before you post your own.
120 | * Try to post a link to a reproduction of the problem on play.golang.org
121 | * Include the full code and an explanation of the problem
122 | * Accepted answers on stack overflow are sometimes wrong
123 |
124 | ## Community
125 |
126 | There are many partially overlapping online communities for Go which vary greatly in style and content.
127 |
128 | [The Go Forum](https://forum.golangbridge.org/) is a friendly place to ask beginner questions
129 |
130 | The [Go Time](https://changelog.com/gotime) podcast interviews luminaries in the Go community and beyond with a focus on Go
131 |
132 | [Golang News](https://golangnews.com) provides links to articles, videos etc. about Go
133 |
134 | The Go [subreddit](https://www.reddit.com/r/golang/) is another source of links and news
135 |
136 | [Women Who Go](http://www.womenwhogo.org/) organises meetups for women and gender minorities who use Go all over the world
137 |
138 | The [Gopher Slack ](https://gophers.slack.com)has an active community of slackers
139 |
140 | The [Golang Nuts](https://groups.google.com/forum/#!forum/golang-nuts) mailing list has a lot of activity if you prefer mailing lists
141 |
142 | There are many videos from Go conferences at [GopherVids](https://gophervids.appspot.com/)
143 |
144 | [Just for func](https://www.youtube.com/channel/UC_BzFbxG2za3bp5NRRRXJSw) is a great podcast on Go programming by the inimitable [@francesc](https://twitter.com/francesc)
145 |
146 | There are a huge number of meetups and conferences about Go, you can find more on the [Community](https://github.com/golang/go/wiki#the-go-community) page of the wiki.
147 |
148 | ## Writing Idiomatic Go {#style}
149 |
150 | For a guide to idiomatic Go, you can refer to the wiki [Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments), which is a summary of comments made on contributions to the Go project and a lot of good advice on structure and names in Go code.
151 |
152 | Go is opinionated about formatting \(as about so much else\) and provides the tool [go fmt](https://blog.golang.org/go-fmt-your-code) to enforce the suggested formatting. There is one standard style for code, including brace positions, spacing etc, enforced by the tools. This has been embraced by users of go, and it turns out it doesn't really matter where you place your braces or whether you use spaces or tabs, so you can now spend your time worrying about more important things, like whether go should have generics.
153 |
154 |
--------------------------------------------------------------------------------
/slices.md:
--------------------------------------------------------------------------------
1 | # Slices
2 |
3 | Slices are views into arrays of data, offering an efficient way of working with arrays. They can share the same data, and simply store an offset into it. This is useful if you don't need to mutate the data but can lead to some subtle bugs if you assume every slice is independent. To get to grips with how slices work internally in Go, see [Go Slices: usage and internals](https://blog.golang.org/go-slices-usage-and-internals), and for an interesting overview of the reasons slices came to their current form, see [Slices](https://blog.golang.org/slices) by Rob Pike on the Go blog.
4 |
5 | ## Nil slices
6 |
7 | The zero value of a slice is a nil slice. It has no underlying array, and has 0 length. It can be used without initialisation \(unlike maps\).
8 |
9 | ## Slicing shares data
10 |
11 | When you use the slice operator, be aware that the backing data will be shared, using the same underlying array. If you need an independent copy of a slice to manipulate the underlying data, take a copy of it first. After you slice, the new slice will be a view onto the same data, but further operations \(like append\) on that new slice may mean it is copied, so it is usually better not to rely on slices sharing data if you wish to mutate that data. This sharing of a backing store can be very useful if you don't need to change data but just use parts of it as it avoids extra allocations.
12 |
13 | ```go
14 | a := []int{0,1,2,3,4,5}
15 | s := a[:]
16 | s2 := s[1:3]
17 | ```
18 |
19 | This can be confusing though if you don't recognise what is happening. For example if you take two slices of data, then append to one of them, the results for the other view into that data are unpredictable and depend on the length of data:
20 |
21 | ```go
22 | // start with an array
23 | data := []int{0, 1, 2}
24 |
25 | // Get two views on data, showing positions 0 and 1
26 | a := data[0:1] // 0
27 | b := data[1:2] // 1
28 |
29 | a = append(a, 3, 4) // append two numbers to a, which overwrites b
30 | // a = append(a, 3, 4, 5) // append three numbers to a, growing the slice, leaving b alone
31 |
32 | // the new slice a is as expected
33 | println(a[0])
34 | // but the values in b may surprise
35 | println(b[0])
36 | ```
37 |
38 | For this reason if mutating a slice with append, you may need to take a copy of data before using it elsewhere or face unexpected side effects. This bug may hit you if you use [bytes.Split](https://golang.org/pkg/bytes/#Split) or similar functions to return sections of data from a slice of bytes.
39 |
40 |
41 | ## Slicing retains the original
42 |
43 | Slicing a slice doesn't copy the underlying array, so the full array will be kept in memory until it is no longer referenced. If you're just using a small part of a large slice, consider copying that new slice into another shorter one so that the original data can be released.
44 |
45 | ```go
46 | // readHeader reads the file header
47 | func readHeader(file string) {
48 |
49 | // Read data at file path
50 | b, _ := ioutil.ReadFile(file)
51 |
52 | // slice the data to truncate
53 | s := b[:12]
54 |
55 | // Copy the data we want to a new shorter slice
56 | c := make([]byte, len(s))
57 | copy(c, s)
58 |
59 | return c
60 | }
61 | ```
62 |
63 | ## Slicing is limited by cap
64 |
65 | Conversely to indexes \(from 0 to len\), slicing is limited by cap, not by len, as stated in the [spec](https://golang.org/ref/spec#Slice_expressions), which may seem a little counter-intuitive, and you probably shouldn't rely on. This means in some circumstances you can retrieve the backing data for a slice even outside its range, as long as it is between len and cap. If you need to see the original data it is better to keep a copy of the original slice to make the intent clear.
66 |
67 |
68 | ## Index out of range
69 |
70 | One of the most common slice mistakes is to attempt to access an index past the length of an array or slice, due to programmer error. This will result in a panic at runtime:
71 |
72 | > panic: runtime error: index out of range
73 |
74 | While this may seem like a trivial mistake to make, it causes a panic at runtime, and you should take care to avoid it by always bounding index operations at the length of the slice. To catch this kind of error with varied input you should try running your code through [go-fuzz](https://github.com/dvyukov/go-fuzz#trophies). You can read more about how to use Go Fuzz in [Fuzzing a DNS Parser](https://blog.cloudflare.com/dns-parser-meet-go-fuzzer/).
75 |
76 | ## Copying slices
77 |
78 | To copy a slice and duplicate the backing data, use the built-in copy function. The destination comes first in the arguments to copy. Beware when copying - the minimum of len(dst) and len(src) is chosen as the length of the new slice, so the destination will need enough space to fit src. Do not attempt to copy into an empty slice as it will not expand automatically.
79 |
80 | ```go
81 | // Make a new slice to copy
82 | src := []int{1,2,3}
83 | // Make sure dst has enough capacity for src
84 | dst := make([]int, len(src))
85 | // Copy from src to dst, n is the no of elements copied
86 | n := copy(dst, src)
87 | ```
88 |
89 | If you try to copy a src of length 100 into a destination slice of length 1, only 1 entry will be copied into the destination.
90 |
91 | ## Appending elements
92 |
93 | Use the built in append function to add an element to a slice.
94 |
95 | ```go
96 | var s []string
97 | s = append(s,"hello")
98 | fmt.Println(s)
99 | ```
100 |
101 | Appending _may_ create a new backing array, it's best not to rely on the slice argument not changing - the argument may change in length, and the slice returned may not be the slice which goes in. It does not create a copy of the slice and return it for every append, as this would waste a lot of memory. When you definitely need a copy \(for example you need to preserve the original slice as well as mutate it\), take a copy of the slice \(or the part of the slice you need\) first:
102 |
103 | ```go
104 | // Make a new slice
105 | c := make([]byte, len(s))
106 |
107 | // Take a copy
108 | copy(c, s)
109 | ```
110 |
111 | Because of this behaviour it is best to always assign the result of append to the same slice.
112 |
113 | ## Appending slices
114 |
115 | You can use the append function to concatenate slices, in combination with the variadic operator to turn the second slice argument into an array of elements.
116 |
117 | ```go
118 | s = append(s,b...)
119 | ```
120 |
121 | ## Sorting
122 |
123 | To sort a slice you can use [sort.Slice](https://golang.org/pkg/sort/#Slice) from the sort package. This package contains other helpers for operations such as stable sorts, binary search, and reversing slices.
124 |
125 | ```go
126 | people := []struct {
127 | Name string
128 | Age int
129 | }{
130 | {"Gopher", 7},
131 | {"Alice", 55},
132 | {"Vera", 24},
133 | }
134 | sort.Slice(people, func(i, j int) bool {
135 | return people[i].Name < people[j].Name
136 | })
137 | ```
138 |
139 | ## Map on slices
140 |
141 | Slices have been kept intentionally simple, which means you'll be using the for range idiom a lot to perform operations on them. There is no map or forEach function, just use a range on the slice.
142 |
143 | ```go
144 | for _, v := range s {
145 | f(v)
146 | }
147 | ```
148 |
149 | ## Converting slice types
150 |
151 | If you have a slice of \[\]T and wish to convert it to a slice of type \[\]interface{}, or vice versa, you will have to do it by hand with a for loop. They do not have the same representation in memory and there is no convenient way to do it.
152 |
153 | ```go
154 | t := []int{1, 2, 3, 4}
155 | s := make([]interface{}, len(t))
156 | for i, v := range t {
157 | s[i] = v
158 | }
159 | ```
160 |
161 | ## Passing slices to functions
162 |
163 | Be aware when passing a slice to a function that while the slice itself is passed by value, it points to a backing array which does not change, even if the slice is copied, so modifying the elements of the slice passed in will modify elements of the original backing array.
164 |
165 | ## Multi-dimensional slices
166 |
167 | Go doesn't have multi-dimensional slices or maps, you have to create them by hand, initialising each subslice.
168 |
169 |
--------------------------------------------------------------------------------
/strings.md:
--------------------------------------------------------------------------------
1 | # Strings
2 |
3 | Source code in Go is in the UTF-8 encoding. This means string literals you create will be UTF strings, and the source files themselves are UTF-8. The string type offers no such guarantees, it stores an array of bytes, but by convention it is usually UTF-8, and you should convert to Unicode at the boundaries of your app. For a detailed overview of the internal workings of strings in Go see [Strings, bytes, runes and characters in Go](https://blog.golang.org/strings) on the official Go blog.
4 |
5 | ## Declaring strings {#declaring}
6 |
7 | Quotation marks in Go source code behave slightly differently from other languages.
8 |
9 | If you use double quotes you will get a string, and can include escape codes, like \n for newline:
10 |
11 | ```go
12 | // hello is a string containing newline & tab
13 | hello := "hello\n\tworld"
14 | ```
15 |
16 | > hello
17 | > world
18 |
19 | If you use back quotes escape codes will not be interpreted, and the string can contain raw newlines:
20 |
21 | ```go
22 | // hello is a string with no newline
23 | hello := `hello world\n`
24 | ```
25 |
26 | This results in no newline:
27 | > hello world\n
28 |
29 | If you use single quotes, you won't create a string at all, these create a rune:
30 |
31 | ```go
32 | hello := 'h' // a rune
33 | ```
34 |
35 | > h
36 |
37 | This is why if you try to define a string with single quotes, you'll get an error:
38 |
39 | ```
40 | // This is not a valid string
41 | hello := 'hello'
42 | ```
43 |
44 | > 'invalid character literal \(more than one character\)'
45 |
46 | ## Multi-line strings {#multi-line}
47 |
48 | To define multi-line strings, including literal control characters like newline, use back quotes, for example:
49 |
50 | ```go
51 | // A string containing a newline after hello
52 | `hello
53 | world`
54 | ```
55 |
56 | ## Runes {#runes}
57 |
58 | Runes in Go represent a code point in unicode, rather than a character. They may be a single character, or they may be part of a character or a modifier. They are an alias for the type `int32`.
59 |
60 | For example the string:
61 |
62 | Hello, 世界
63 |
64 | Is represented by these bytes:
65 |
66 | > \[72 101 108 108 111 44 32 228 184 150 231 149 140\]
67 |
68 | but is represented by these runes:
69 |
70 | > \[72 101 108 108 111 44 32 19990 30028\]
71 |
72 | as you can see if you range over it (see below).
73 |
74 | ## Range on Strings {#range}
75 |
76 | If you range on a string, you will receive the runes which make up the string, not the bytes. In an ASCII string every byte corresponds exactly to one rune, so ranging "hello" will return:
77 |
78 | > 'h', 'e', 'l', 'l', 'o'
79 |
80 | and ranging over "日本語" will return the runes, not the bytes:
81 |
82 | > '日', '本', '語'
83 |
84 | ## Index on Strings {#indexing-strings}
85 |
86 | Somewhat confusingly, given how ranging works, if you take the index of a string you'll get the byte at that index, not the rune:
87 |
88 | ```go
89 | s := "日本語"
90 | // This returns a byte at index 2
91 | b := s[2]
92 | fmt.Println(b)
93 | ```
94 |
95 | Returns the third byte, not the third rune as you might expect:
96 |
97 | > 165
98 |
99 | ## Trimming strings {#encodings}
100 |
101 | The strings package contains many useful functions for manipulating text, and each one is documented. Some might not do exactly what you expect from the name, for example strings.Trim does not trim a given suffix or prefix:
102 |
103 | ```go
104 | s := "les mots, et les choses"
105 | t := strings.Trim(s,"les ")
106 | fmt.Println(t)
107 | ```
108 |
109 | The result may be unexpected:
110 |
111 | > mots, et les cho
112 |
113 | This function takes a **cutset** string – a list of characters to trim. If you want to trim a full prefix or suffix, use strings.TrimPrefix or strings.TrimSuffix:
114 |
115 | ```go
116 | s := "les mots, et les choses"
117 | t := strings.TrimPrefix(s,"les ")
118 | fmt.Println(t)
119 | ```
120 |
121 | Which will remove the suffix or prefix provided as you would expect.
122 |
123 | ## Formatting Strings {#encodings}
124 |
125 | The usual Printf and Sprintf variants are available in the fmt package, so that you can format strings using a [template string](https://golang.org/pkg/fmt/#hdr-Printing) and variables. The formats accepted are detailed in the strings package documentation. %v prints the value in a default format and is useful for debugging.
126 |
127 | ```go
128 | fmt.Printf("%s at %v", "hello world", time.Now())
129 | // Print to a string
130 | s := fmt.Sprintf("%s at %v", "hello world", time.Now())
131 | ```
132 |
133 | ## Encodings {#encodings}
134 |
135 | Most of the time when working with unicode strings in Go you won't have to do anything special. If you need to convert from another encoding like Windows 1252 you can use the [encoding](https://godoc.org/golang.org/x/text/encoding) package under golang.org/x/text. There are also some other utility packages for dealing with text there. The golang.org/x libraries are slightly less stable than the go standard library and are not part of the official distribution, but have many useful utilities.
136 |
137 | ## Translating encodings {#translating-encodings}
138 |
139 | The go external package [text/encoding](https://godoc.org/golang.org/x/text/encoding) packages provide utilties for translating strings from Go's native UTF-8 to popular encodings and back (such as UTF-16, EUC-JP, GBK and Big5). Encoders and Decoders are available for working with arrays of bytes or strings.
140 |
141 | ```go
142 | // Convert from Go's native UTF-8 to UTF-16
143 | // using golang.org/x/text/encoding/unicode
144 | s := "エヌガミ"
145 |
146 | // Encode native UTF-8 to UTF-16
147 | encoder := unicode.UTF16(unicode.LittleEndian, unicode.UseBOM).NewEncoder()
148 | utf16, err := encoder.String(s)
149 | if err != nil {
150 | fmt.Println(err)
151 | }
152 | fmt.Println("utf16:", utf16)
153 | ```
154 |
155 | ## Strings are immutable {#immutable}
156 |
157 | Strings in Go are immutable, you are not allowed to change the backing bytes, and manipulating bytes directly can be dangerous as the string _may_ contain several bytes for each character/rune.
158 |
159 | ## Don't use pointers to strings {#string-pointers}
160 |
161 | A string value is a fixed size and [should not be passed as a pointer](https://github.com/golang/go/wiki/CodeReviewComments#pass-values).
162 |
163 | ## Strings are never nil {#zero-value}
164 |
165 | You cannot assign nil to a string or compare nil to a string, they have a zero value of "" and can never be nil, only the zero value.
166 |
167 | ## Converting Strings to Ints {#atoi}
168 |
169 | You can use the [strconv](https://golang.org/pkg/strconv/) package to convert Strings to other types like ints and floats and back again.
170 |
171 | ```go
172 | // Itoa converts an int to a string
173 | s := strconv.Itoa(99)
174 | // Atoi converts a string to an int
175 | i,_ := strconv.Atoi(s)
176 | // Print both types
177 | fmt.Printf("string:%s int:%d", s, i)
178 | ```
179 |
180 | Returns
181 |
182 | > string:99 int:99
183 |
184 | ## Parsing floats {#floats}
185 |
186 | Floats are [imprecise ](http://floating-point-gui.de/)and rounding can be unpredictable. When parsing them from strings and working with them, you need to be aware of this.
187 |
188 | ```go
189 | price := "655.18"
190 | f, _ := strconv.ParseFloat(price, 64)
191 | c := int(f * 100)
192 | fmt.Printf("string:%s float:%f cents:%d", price, f, c)
193 | ```
194 |
195 | > string:655.18 float:655.180000 cents:65517
196 |
197 | If parsing a float for a currency value, consider converting the string to a value in cents first, as you will have work to do anyway to strip currency amounts and deal with missing cents. You can then parse as an integer and avoid any problems with storing it as a float.
198 |
199 | ## Reading text files {#reading-files}
200 |
201 | The easiest way to read an entire file is with ioutil, but unless the file is small (e.g. a config file) this can use a large amount of memory:
202 |
203 | ```go
204 | b, err := ioutil.ReadFile("path/to/file")
205 | if err != nil {
206 | fmt.Println("error reading file: %s", err)
207 | }
208 |
209 | fmt.Printf("%s", b)
210 | ```
211 |
212 | So consider reading large files in chunks. You can read a line-based file with [bufio.Scanner](https://golang.org/pkg/bufio/) and the Scan function:
213 |
214 | ```go
215 | file, err := os.Open("path/to/file")
216 | if err != nil {
217 | fmt.Println("error opening file: %s", err)
218 | return
219 | }
220 | defer file.Close()
221 | scanner := bufio.NewScanner(file)
222 | for scanner.Scan() {
223 | fmt.Println("line:%s", scanner.Text())
224 | }
225 | if err := scanner.Err(); err != nil {
226 | fmt.Println("error reading file: %s", err)
227 | }
228 | ```
229 |
230 | Also consider that files conform to io.Reader and io.Writer from the io package, and can be used directly with functions which accept an io.Reader.
--------------------------------------------------------------------------------
/structs.md:
--------------------------------------------------------------------------------
1 | # Structs & Types
2 |
3 | A struct in Go is a sequence of fields, each of which has a name and type public and private rules are the same as names in packages - lowercase names are private, names starting with Uppercase are exported. Structs can also have methods, which allows them to fulfil interfaces. Structs do not inherit from each other, but they can embed another struct to gain its fields and behaviour. The embedder does not know anything about the struct it is embedded in and cannot access it.
4 |
5 | ## Forget inheritance
6 |
7 | _During a Java user group meeting James Gosling \(Java's inventor\) was asked "If you could do Java over again, what would you change?". He replied: "I'd leave out classes"_
8 |
9 | Go eschews inheritance in favour of composition. If you're accustomed to building large hierarchies of types as a way of organising your data, you might find this jarring at first. Go interfaces provide polymorphism, so if your function needs something which Barks, it simply asks for a Barker. Go embedding provides composition, so if your Barker needs to Yap as well, it can embed a struct to do so. Go provides tools which solve the same problems as inheritance, without most of the downsides. You may find you need it less than you think.
10 |
11 | Many developers coming to go from more complex OOP languages start by trying to reintroduce inheritance by using embedding. This will lead to frustration, because embedding does not work in the same way as inheritance, by design.
12 |
13 | ## Composition is not inheritance
14 |
15 | Composition can do some of the same things as inheritance, but it is not the same. For example, don't try to do this:
16 |
17 | ```go
18 | package main
19 |
20 | import (
21 | "fmt"
22 | )
23 |
24 | // Type A knows only about A, it cannot call B methods
25 | type A struct{}
26 |
27 | func (a A) Foo() { a.Bar() }
28 | func (a A) Bar() { fmt.Println("This is a method on A") }
29 |
30 | // Type B knows about A and B
31 | type B struct{
32 | A // Embeds A - note this is not inhertance
33 | }
34 |
35 | // Type B attempts to redefine Bar()
36 | // but type A doesn't know about it
37 | func (b B) Bar() { fmt.Println("This is a method on B") }
38 |
39 | func main() {
40 | B{}.Foo()
41 | }
42 | ```
43 |
44 | A.Foo will call A.Bar, not B.Bar as it might if B inherited from A, because A does not know about B.
45 |
46 | Inheritance was deliberately left out of Go, so don't try to recreate it with composition. If you find yourself frustrated that embedded structs don't know about the embedder, you're not really composing functionality. Always try to use the simplest solution first (separate structs), and only use composition if you definitely need to share the same behaviour between two types.
47 |
48 | ## Don't use this or self
49 |
50 | While it is common in other languages, it is frowned upon in Go to use [self or this](https://github.com/golang/go/wiki/CodeReviewComments#Receiver_Names) in receiver names, instead use the first letter of the receiver. Go has no language support for these words, it does not use them as keywords as other languages do and it is therefore confusing to use them in a Go context.
51 |
52 | ## Type assertions
53 |
54 | Use a type assertion to check the type wrapped in an interface is of the expected type, or to retrieve the value, rather than attempting to use it without checks. If type assertions are used _without_ the comma ok form, a panic ensues if the type is not as expected, so it's always better to use the comma ok form and check the boolean returned.
55 |
56 | ```go
57 | // Convert from interface back to underlying type int
58 | var x interface{} = 7
59 | a := x.(int)
60 | fmt.Println("a is int:", a)
61 |
62 |
63 | // Attempt to convert from interface to int with a string
64 | var y interface{} = "hello"
65 |
66 | // Check before use, no panic
67 | b, ok := y.(int)
68 | if !ok {
69 | fmt.Println("b is not int:", b)
70 | }
71 |
72 | // panic, because y is not of type int
73 | //b = y.(int)
74 | ```
75 |
76 | ## Nil pointer dereference
77 |
78 | This error is usually caused by failing to check errors properly \(i.e. using a value without checking that there were no errors\), or storing a nil pointer and later trying to use it. **Check and handle all errors**. Never use \_ as a shortcut in production code:
79 |
80 | ```go
81 | // Don't do this, value may well be invalid
82 | value, _ := f()
83 | ...
84 | ```
85 |
86 | Instead, always check errors:
87 |
88 | ```go
89 | // Do this
90 | value, err := f()
91 | if err != nil {
92 | // Deal with error
93 | return fmt.Errorf("msg %s",err)
94 | }
95 | // use value
96 | ...
97 | ```
98 |
99 | ## Pointers vs Values for Methods
100 |
101 | You can define methods either on the struct value or struct pointer. If unsure, you can follow these guidelines for deciding which to use:
102 |
103 | * **If in doubt, use a pointer receiver**
104 | * If the method modifies the receiver, it must be on a pointer receiver
105 | * If the receiver is large, e.g. a big struct, it is cheaper to use a pointer receiver
106 | * If the type is stored in a map or interface, it is not addressable and T cannot use \*T methods
107 | * If you need any pointer receivers, make them all pointer receivers
108 |
109 | The outcome of these guidelines is that usually it's best to work with pointers to structs and thus to define methods on the pointer, not on the value. For small structs you may want to use values for efficiency, but for large structs or structs which modify themselves, you will want to use pointer receivers. There is more detail on the rules for pointer receivers at the end of this chapter in the section on Method Sets.
110 |
111 | ## Pointers vs Values in Slices
112 |
113 | As you'll probably want to use pointers to structs elsewhere in your code, it makes sense to use slices of pointers, rather than slices of values. This also lets you update the structs in the slice directly.
114 |
115 | ## Pointers vs Values for constructors
116 |
117 | When writing constructors, default to pointers to structs for the same reasons as above. You can certainly return plain structs or values, particularly for simple values, but typically simple values won't require a constructor.
118 |
119 | ## Why do new and make exist?
120 |
121 | You can probably get away without using `new` at all - it simple allocates a new instance of a type, and you can use &T{} instead.
122 |
123 | `make` is required for used with maps, slices and channels to initialise them with a given size, for example:
124 |
125 | ```go
126 | // Using make with maps and slices
127 | make(map[int]int) // A map with no entries
128 | make(map[int]int,10) // A map with 10 zero entries
129 | make([]int, 10, 100) // A slice with 10 zero entries, and a capacity of 100
130 | c := make(chan int) // An unbuffered channel of integers
131 | ```
132 |
133 | ## Enums
134 |
135 | There are no enums in Go. Use the keyword iota to increment constants from a known base, so the closest to an enum is a set of constants in a file:
136 |
137 | ```go
138 | // Describe the constants here
139 | const (
140 | RoleAnon = iota
141 | RoleReader
142 | RoleAdmin
143 | )
144 | ```
145 |
146 | For more sophisticated control impose limits or provide string values by using a type:
147 |
148 | ```go
149 | type Role struct {
150 | value int
151 | }
152 |
153 | func (e Role)SetValue(v int) {
154 | if v == RoleAnon || v = RoleReader || v = RoleAdmin {
155 | e.value = v
156 | }
157 | }
158 |
159 | func (e Role)Value(v int) {
160 | return e.value
161 | }
162 | ```
163 |
164 | ## Method Sets
165 |
166 | You _normally_ don't have to worry about method sets as the compiler will transform pointers or values to the other in order to use methods defined on the other as a convenience, but this breaks down in some circumstances. The exceptions to this are if a type is stored in a map, or stored in an Interface, or if you want to mutate the value of the receiver within the method. This is a gnarly detail, and effective go is somewhat confusing on this score:
167 |
168 | > The rule about pointers vs. values for receivers is that value methods can be invoked on pointers and values, but pointer methods can only be invoked on pointers.
169 |
170 | It should probably end with can only be invoked on pointers or addressable values. If you attempt to test this, you'll find that you **can** call pointer methods on struct values in most circumstances:
171 |
172 | ```go
173 | type T struct{}
174 |
175 | func (t T) Foo() { fmt.Println("foo") }
176 | func (t *T) Bar() { fmt.Println("bar") }
177 |
178 |
179 | func main() {
180 | // t can call methods Foo and Bar
181 | // as is is addressable
182 | // but beware if Bar() changes
183 | // the receiver changes will be lost
184 | t := T{}
185 | t.Foo()
186 | t.Bar() // compiler converts to (&t).Bar()
187 |
188 | // tp can call methods Foo and Bar
189 | tp := &T{}
190 | t.Foo()
191 | tp.Bar()
192 | }
193 | ```
194 |
195 | The compiler will try to help you by inserting dereferences for pointers or pointers for types where you try to call a method, but this won't work in all instances, it breaks down if your type is stored in a map, or stored in an interface:
196 |
197 | ```go
198 | func main() {
199 |
200 | // You can call the pointer method on a pointer entry in a map
201 | mp := make(map[int]*T)
202 | mp[1] = tp
203 | mp[1].Foo() // Allowed
204 | mp[1].Bar() // Allowed
205 |
206 | // You cannot call the pointer method on a value entry in a map
207 | m := make(map[int]T)
208 | m[1] = t
209 | m[1].Foo() // Allowed
210 | //m[1].Bar() // Not Allowed - cannot take the address of m[1]
211 |
212 | callFoo(tp) // Allowed
213 | callFoo(t) // Allowed
214 |
215 | callBar(tp) // Allowed
216 | //callBar(t) // Not Allowed - Bar method has pointer receiver
217 |
218 | }
219 |
220 | func callBar(i I) { i.Bar() }
221 | func callFoo(i FI) { i.Foo() }
222 | ```
223 |
224 | You can read more about this on this wiki entry on [Method Sets](https://github.com/golang/go/wiki/MethodSets).
225 |
226 |
--------------------------------------------------------------------------------
/styles/pdf.css:
--------------------------------------------------------------------------------
1 | .pdf-header {
2 | margin-top: 60px;
3 | padding-bottom: 20px;
4 | border-bottom: 1px solid #000;
5 | color: #000;
6 | }
7 |
8 | .pdf-footer {
9 | border: none;
10 | color: #000;
11 | }
12 |
13 | a,
14 | a:hover,
15 | a:focus,
16 | .link-inherit,
17 | .link-inherit:hover,
18 | .link-inherit:focus {
19 | color: #000;
20 | }
21 |
22 | h1 {
23 | margin: 2em 0 0.5em 0;
24 | }
25 |
26 | h2 {
27 | margin: 1em 0 0.2em 0;
28 | }
29 |
30 | h3 {
31 | margin: 0.8em 0 0 0;
32 | }
--------------------------------------------------------------------------------
/templates.md:
--------------------------------------------------------------------------------
1 | # Templates
2 |
3 | The Go Standard Library offers templating of both text and html content. The html templates offer a superset of the text template features, including contextual escaping, but the interface can be a little confusing.
4 |
5 | ## The missing docs
6 |
7 | There is extensive documentation of the template packages, but most of it is in the [text/template](https://golang.org/pkg/text/template/) package, not the [html/template](https://golang.org/pkg/html/template/) package, so be sure to read that as well, even if you intend to focus on using html/template. The html template package extends the text/template package and uses much of its functionality, so you need to read the documentation for both.
8 |
9 | ## Delimiters
10 |
11 | If you have problems with the go template delimiters clashing with other libraries, you can use the Delims function in order to set the delimiters to some other string. Note this must be done first, before files are parsed:
12 |
13 | ```go
14 | // Set the delimiters on this template
15 | tmpl, err := template.New("").Delims("[[", "]]").ParseFiles("foo.tmpl", "bar.tmpl")
16 | ```
17 |
18 | ## Dot
19 |
20 | The dot character in go templates represents the data context, and is set on Execution. Within functions like range the dot is set within that scope to the successive elements of the array. To access the parent context within a range use `$.`.
21 |
22 | ## Inline templates
23 |
24 | While normally templates are complex enough to require their own files, you may find it useful to define templates as strings inline in your go code, particularly if sharing them on the playground. When templates are over a few lines this can become cumbersome and you might want to consider storing them in files and reading them in to a template set.
25 |
26 | ```go
27 | // Define a simple template to print data
28 | inline := `Hello, {{.}}`
29 |
30 | // Set data to the string world!
31 | data := `world!`
32 |
33 | // Load the template
34 | t, err := template.New("t").Parse(inline)
35 | if err != nil {
36 | panic(err)
37 | }
38 |
39 | // Print the template 't' to stdout
40 | // using data as the context
41 | err = t.ExecuteTemplate(os.Stdout, "t", data)
42 | ```
43 |
44 | Prints the template "Hello, " + the data "world!"
45 | > Hello, world!
46 |
47 | ## Escaping
48 |
49 | The Go template package assumes that template authors (and by extension the templates) are trusted, but that the data inserted into the template is not. The HTML templates offer contextual escaping, so the package knows about the structure of html and the context in which data is escaped matters - the same data inside a url, an attribute or text content may be escaped differently.
50 |
51 | For example the text "O'Reilly & co" is escaped differently depending on whether it is found inside script tags, an attribute or an href attribute.
52 |
53 | ```go
54 | tmpl := `Hello, {{.}}`
55 | data := `O'Reilly >`
56 |
57 | // Load the template
58 | t, err := template.New("t").Parse(tmpl)
59 | if err != nil {
60 | panic(err)
61 | }
62 |
63 | // Execute the template
64 | err = t.ExecuteTemplate(os.Stdout, "t", data)
65 | ```
66 |
67 | > `Hello, O'Reilly & co`
68 |
69 | If you need to bypass escaping, you can use the appropriate [typed string](https://golang.org/pkg/html/template/#hdr-Typed_Strings) like template.HTML in a function or field, to indicate to the package that this is trusted content. Be very careful not to use user content in such cases, or to escape it properly before use.
70 |
71 | ## Structs, fields and methods
72 |
73 | There is a uniform interface for fields and methods of a struct - the . operator will call either a field or a method. The same is true of map entries. You can also use the built-in function index to access members of Maps and Slices.
74 |
75 | ```go
76 | {{ index mymap 0 }}
77 | ```
78 |
79 | ## Func Maps
80 |
81 | Templates have a set of global functions available, which are documented in the [text/template](https://golang.org/pkg/text/template/#hdr-Functions) package, these offer logical operations, printing and a few other conveniences.
82 |
83 | To extend these Templates offer a FuncMap, which allows you to add functions to templates if you need to format data in a particular way (for example formatting currencies), or provide global data like the title of your website or string translations without threading it through all the different handlers and setting it explicitly as part of the data provided to templates. Don't get too carried away adding functions to your templates though - they are deliberately pared down version of go so that most of the logic resides in your .go files, rather than in template files, and to keep the attack surface to a minimum. Templates are deliberately minimalist in Go and you should strive to keep them so.
84 |
85 | ## Prefix operators
86 |
87 | The boolean operators available in templates are all functions, which means they use prefix notation. Use _and_ x y, not x _and_ y. If you're coming from Ruby this may come as a surprise, as using _and_ with one argument will appear to work, but does not behave as you might expect.
88 |
89 | ```go
90 | {{/* Don't do this, it will always be true if x is non nil */}}
91 | {{ if x and y}}
92 |
93 | {{ end }}
94 |
95 | {{/* Do this instead */}}
96 | {{ if and x y}}
97 |
98 | {{ end }}
99 | ```
100 |
101 | ## Range functions
102 |
103 | Go templates also allow you to range, but unlike the range in Go code, using this function with one argument yields the value, not the index.
104 |
105 | ```go
106 | {{ $v := range .values }}
107 | ```
108 |
109 | There are numerous small differences between Go templates and Go code like this, so don't assume that the template language reflects Go norms.
110 |
111 | ## Writing comma separated lists
112 |
113 | If using range in templates to write a list, which must be comma separated (JSON for example requires commas on all array items, but not the last), the neatest way to do this is to use if $i immediately after the range call:
114 |
115 | ```go
116 | // Print ',' before every element but the first one
117 | {{ $i, $v := range .values }}{{if $i}}, {{end}}
118 | {{.}}
119 | {{ end }}
120 | ```
121 |
122 | ## Printing values
123 |
124 | The printf function is available in templates to print values as strings:
125 |
126 | ```go
127 | {{ printf "%s %d %d" "hello" 1 1 }}
128 | ```
129 |
130 | ## The line eater
131 |
132 | If you need to add newlines in your template for legibility (for example when outputting many columns of csv), you can invoke the line eater with the - symbol beside the braces (either start or end braces). The template:
133 |
134 | ```go
135 | {{- .One }}
136 | {{- .Two -}}
137 | ```
138 |
139 | will output the values with whitespace removed:
140 |
141 | > foobar
142 |
143 | ## Template Sets
144 |
145 | There are several different ways to define templates and name them - they can be named inline in the template, or named by one of the ParseFiles or ParseGlob functions, or named on creation with template.New("t"). If using ParseGlob you'd have to make sure the file names used are always unique, which can be problematic in a large project. Another approach is to use the relative path of template files as the name, to ensure uniqueness, and make it clear when reading templates which file on disk is being included.
146 |
147 | Templates can only include to those in the same set, so you may find it convenient to load all templates into the same set with distinct names (perahps using the location of the templates loaded) so that they can be pulled in to any template by path.
148 |
149 | ## Rendering Nested Templates
150 |
151 | To render a template within another template, assuming they are in the same set, use the template function, which takes the name of the content to place, and the data context to render it with (use the dot operator to obtain the current context).
152 |
153 | ```go
154 |
155 | Hello
156 | {{ template "views/users/content.html.got" . }}
157 |
158 | ```
159 |
160 | This will render the template with name "views/users/content.html.got" inside the main template above.
161 |
162 | ## Template Blocks
163 |
164 | Template blocks, introduced in Go 1.6, can be used to define areas of a template to replace with other content by overlaying it with another template. The master template should be created as normal, then Clone used to copy it with an overlay template. This may seem counter-intuitive at first.
165 |
166 | Another approach to achieve a similar result is to create a layout template with a `.content` key, which can then have content (itself rendered from a separate template) inserted into it using this key.
167 |
168 | ```go
169 |
170 | Hello
171 | {{block "content" .}}
172 | Default content
173 | {{ end }}
174 |
175 | ```
176 |
177 |
178 |
--------------------------------------------------------------------------------
/time.md:
--------------------------------------------------------------------------------
1 | # Time in Go
2 |
3 | Go includes a good set of primitives for dealing with time in the standard library. There are a few oddities detailed below.
4 |
5 | # Time on Go Playground
6 |
7 | The playground starts with the same time for every run to make sure output is deterministic, so examples including time may not function as you expect on the playground. For more details see [Inside the Go Playground.](https://blog.golang.org/playground).
8 |
9 | ## Time Formatting
10 |
11 | Datetime formatting in Go is rather unusual. It uses a format string which also functions as an example time; unfortunately in practice this is rather difficult to remember. The Parse and Format functions take this format layout:
12 |
13 | ```
14 | Mon Jan 2 15:04:05 -0700 MST 2006
15 | ```
16 |
17 | the default time format for dates is and the time package uses the memorable [constant](https://golang.org/src/time/format.go?s=15291:15333#L66) `time.RFC3339` for timestamps in international date format ISO 8601:
18 |
19 | ```
20 | "2006-01-02T15:04:05Z07:00"
21 | ```
22 |
23 | You may want to define some constants for any date formats you normally use, to avoid constantly referring to the time package, for example:
24 |
25 | ```go
26 | const (
27 | Date = "2006-01-02"
28 | DateTime = "2006-01-02 15:04"
29 | )
30 | ```
31 |
32 | ## AddDate
33 |
34 | The AddDate function adjusts dates according to arbitrary overflow rules to ensure that results are reversible. If adding Years or days, the results will generally be as expected, but if adding months, the results may not be as users expect and does not match the output of popular programs like excel.
35 |
36 | For example subtracting one month from October 31 yields October 1st
37 |
38 | ```go
39 | // Oct 31st minus 1 month is Oct 1st,
40 | // not Sept 30th according to Go
41 | input := time.Date(2016, 10, 31, 0, 0, 0, 0, time.UTC)
42 | output := time.Date(2016, 9, 30, 0, 0, 0, 0, time.UTC)
43 | result := input.AddDate(0, -1, 0)
44 | if result != output {
45 | fmt.Printf("got:%v want:%v\n", result, output)
46 | }
47 | ```
48 |
49 | The results will be unpredictable and depend on the day chosen and the number of days in the target month. If the number of days of the start month and the end month do not match, results can be unexpected. This is important if your users expect to add 9 months and 1 day to a given date to conform with accounting rules for example - typically this means 9 calendar months without rollover and then 1 day to be added. There is no way round this except writing your own code to handle adding months in a more predictable way.
50 |
51 | ## Time Zones
52 |
53 | You should **strongly** prefer to store times as **UTC**, and convert them for display. This makes comparisons and calculations straightforward, and allows you to customise display for the user's current location at the time of viewing. When creating times or comparing with the current time, always use the t.UTC\(\) function to be sure you compare the UTC time.
54 |
55 | If you need to convert times, you can convert times from a given zone to a location easily enough:
56 |
57 | ```go
58 | // Load a set location
59 | l, err := time.LoadLocation("America/Mexico_City")
60 |
61 | // Set the time zone
62 | now := t.In(l)
63 | ```
64 |
65 | ## Monotonic time
66 |
67 | Since `Go 1.9`, the [time](https://golang.org/pkg/time/) package now transparently tracks monotonic time in each Time value, making computing durations between two Time values a safe operation in the presence of wall clock adjustments.
68 |
69 | > If Times t and u both contain monotonic clock readings, the operations t.After\(u\), t.Before\(u\), t.Equal\(u\), and t.Sub\(u\) are carried out using the monotonic clock readings alone, ignoring the wall clock readings. If either t or u contains no monotonic clock reading, these operations fall back to using the wall clock readings.
70 |
71 | Times may not contain a monotonic clock reading if they pass through functions like AddDate Round or Truncate, or if they come from parsed external sources, so don't assume they always will have.
72 |
73 | ## Comparing time
74 |
75 | When comparing time you should usually use the [Time.Equal](https://golang.org/pkg/time/#Time.Equal) method rather than ==, as the == operator also compares both Location, which sets the time zone offset but not the absolute time, and the monotonic clock reading, which can be stripped in some circumstances.
76 |
77 | > In general, prefer t.Equal(u) to t == u, since t.Equal uses the most accurate comparison available and correctly handles the case when only one of its arguments has a monotonic clock reading.
78 |
79 | ```go
80 | t := time.Now()
81 | t2 := time.Now().UTC()
82 | if t.Equal(t2) {
83 | // Use t.Equal to compare times
84 | }
85 | ```
86 |
87 | ## Formats vs Values
88 |
89 | Despite their beguiling appearance, time formats are not time values, so if testing parsing, be aware that using a format as a value is often invalid due to formatting directives in the format string which are not valid in a time:
90 |
91 | ```go
92 | // Time formats are not always valid values
93 | _, err := time.Parse(time.RFC3339, time.RFC3339)
94 | fmt.Println("error", err) // RFC3339 is not a valid time
95 | ```
96 |
97 | > error parsing time "2006-01-02T15:04:05Z07:00": extra text: 07:00
98 |
--------------------------------------------------------------------------------
/title.md:
--------------------------------------------------------------------------------
1 | # The Go Bestiary
2 |
3 | ## Bugs, gotchas and guidelines for go programmers
--------------------------------------------------------------------------------