└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # go modules 2 | 3 | This is an overview of using [go modules](https://github.com/golang/go/wiki/Modules), 4 | the differences between it and `$GOPATH`-based development, and some of the tools 5 | and techniques it makes available to go developers. 6 | 7 | This content is also available as [a video recording](https://www.youtube.com/watch?v=OrMA16ASz2U&t=40s). 8 | 9 | - [$GOPATH: the old way](#gopath-the-old-way) 10 | - [Building](#building) 11 | - [What happens](#what-happens) 12 | - [Benefits](#benefits) 13 | - [Potential problems](#potential-problems) 14 | - [Questions/Answers](#questionsanswers) 15 | - [Distribution](#distribution) 16 | - [What happens](#what-happens-1) 17 | - [Benefits](#benefits-1) 18 | - [Potential problems](#potential-problems-1) 19 | - [go modules: the new way](#go-modules-the-new-way) 20 | - [Defining a module](#defining-a-module) 21 | - [What happens](#what-happens-2) 22 | - [Benefits](#benefits-2) 23 | - [Resolving dependencies automatically](#resolving-dependencies-automatically) 24 | - [What happens](#what-happens-3) 25 | - [Benefits](#benefits-3) 26 | - [Distribution](#distribution-1) 27 | - [What happens](#what-happens-4) 28 | - [Benefits](#benefits-4) 29 | - [Replacing resolved module versions or source](#replacing-resolved-module-versions-or-source) 30 | - [Caveats](#caveats) 31 | - [Major versions](#major-versions) 32 | - [Benefits of semantic import versioning](#benefits-of-semantic-import-versioning) 33 | - [Caveats of semantic import versioning](#caveats-of-semantic-import-versioning) 34 | - [Tips](#tips) 35 | - [Getting started](#getting-started) 36 | - [Querying](#querying) 37 | - [Cleaning up](#cleaning-up) 38 | - [Low-level go.mod manipulation](#low-level-gomod-manipulation) 39 | - [Local module cache](#local-module-cache) 40 | - [Module sources](#module-sources) 41 | - [Additional Reading](#additional-reading) 42 | 43 | ## $GOPATH: the old way 44 | 45 | ### Building 46 | 47 | Source was located under `$GOPATH/src`, whether we liked it or not. 48 | 49 | ```sh 50 | go get github.com/liggitt/a 51 | go get github.com/liggitt/b 52 | ``` 53 | 54 | ```sh 55 | cat $GOPATH/src/github.com/liggitt/a/make-cake.go 56 | ``` 57 | 58 | > ```go 59 | > package main 60 | > 61 | > import ( 62 | > "github.com/liggitt/a/helpers" 63 | > "github.com/liggitt/b" 64 | > ) 65 | > 66 | > func main() { 67 | > b.Cake() 68 | > helpers.PrintSuccess() 69 | > } 70 | > ``` 71 | 72 | ```sh 73 | cat $GOPATH/src/github.com/liggitt/a/helpers/helpers.go 74 | ``` 75 | 76 | > ```go 77 | > package helpers 78 | > 79 | > import "fmt" 80 | > 81 | > func PrintSuccess() { 82 | > fmt.Println("cake!") 83 | > } 84 | > ``` 85 | 86 | ```sh 87 | cat $GOPATH/src/github.com/liggitt/b/cake.go 88 | ``` 89 | 90 | > ```go 91 | > package b 92 | > 93 | > func Cake() { 94 | > // TODO: implement. the cake is a lie. 95 | > } 96 | > ``` 97 | 98 | ```sh 99 | cd $GOPATH/src/github.com/liggitt/a 100 | go run . 101 | ``` 102 | 103 | > ```sh 104 | > cake! 105 | > ``` 106 | 107 | #### What happens 108 | 109 | 1. go gathers dependencies of my main package (e.g. `go list -deps .`) 110 | 2. my main package imports `github.com/liggitt/a/helpers` and `github.com/liggitt/b` 111 | 3. go searches for those import paths in the following places, in order: 112 | 1. `github.com/liggitt/a/vendor/` 113 | 2. `$GOPATH/src/` 114 | 3. `$GOROOT/src/` 115 | 4. in this case it finds them in `$GOPATH`, builds, and runs 116 | 117 | #### Benefits 118 | 119 | * Hermetic (if you fully control all the content in `$GOPATH`) 120 | * No network access 121 | * Allows local development 122 | 123 | #### Potential problems 124 | 125 | * What if I need multiple things in my `$GOPATH` at different code levels? 126 | * workaround: have a separate `$GOPATH` per project 127 | * workaround: place a copy of all code inside a vendor dir in each project 128 | * What if I don't want to structure my directories like $GOPATH requires? 129 | * you don't always get what you want 130 | * What if source in `$GOPATH` accidentally drifts from the authoritative source? 131 | * whatever is in `$GOPATH` gets used, for better or worse: 132 | * can be bad if you weren't expecting it 133 | * can be good if you're intentionally developing multiple components at once 134 | 135 | #### Questions/Answers 136 | 137 | **Q: What is the full import path of `$GOPATH/src/github.com/liggitt/b`?** 138 | 139 | A: The relative path under `$GOPATH/src`, so `github.com/liggitt/b` 140 | 141 | **Q: What version of `github.com/liggitt/b` does `github.com/liggitt/a` use?** 142 | 143 | A: Whatever is sitting in `$GOPATH/src/github.com/liggitt/b` 144 | 145 | **Q: What version of `github.com/liggitt/b` does `github.com/liggitt/a` prefer to use?** 146 | 147 | A: What's a version? 148 | 149 | ### Distribution 150 | 151 | ```sh 152 | go get github.com/liggitt/a 153 | ``` 154 | 155 | #### What happens 156 | 157 | * go resolves the version-control location for the specified import path 158 | * go downloads the source to `$GOPATH/src/` 159 | * go also resolves and downloads transitive dependencies to `$GOPATH/src/` 160 | 161 | #### Benefits 162 | 163 | * Simple distribution for simple things 164 | 165 | #### Potential problems 166 | 167 | * No versioning, you always get master 168 | * No versioning of dependencies, you always get master of those as well 169 | * Import path is coupled to location 170 | * Random things get dumped into `$GOPATH` 171 | 172 | ## go modules: the new way 173 | 174 | Note: if you want to work through this demo yourself, all the commands should work as described. 175 | If you want something prepared in advance, the results of walking through this exercise are at 176 | https://github.com/liggitt/a/tree/demo 177 | 178 | ### Defining a module 179 | 180 | Instead of being defined by a path relative to `$GOPATH`, modules are just a tree 181 | of Go source files with a `go.mod` file in the tree's root directory. 182 | The tree can be located anywhere. 183 | 184 | Let's remove our projects from `$GOPATH`: 185 | 186 | ```sh 187 | rm -fr $GOPATH/src/github.com/liggitt/{a,b} 188 | ``` 189 | 190 | And try to build them outside our `$GOPATH`: 191 | 192 | ```sh 193 | mkdir -p $HOME/tmp/modules/can/be/anywhere 194 | cd $HOME/tmp/modules/can/be/anywhere 195 | git clone https://github.com/liggitt/a.git 196 | cd a 197 | go run . 198 | ``` 199 | 200 | We see the problems we expect: 201 | > ``` 202 | > make-cake.go:4:2: cannot find package "github.com/liggitt/a/helpers" in any of: 203 | > /Users/liggitt/.gvm/gos/go1.12.1/src/github.com/liggitt/a/helpers (from $GOROOT) 204 | > /Users/liggitt/go/src/github.com/liggitt/a/helpers (from $GOPATH) 205 | > make-cake.go:5:2: cannot find package "github.com/liggitt/b" in any of: 206 | > /Users/liggitt/.gvm/gos/go1.12.1/src/github.com/liggitt/b (from $GOROOT) 207 | > /Users/liggitt/go/src/github.com/liggitt/b (from $GOPATH) 208 | > ``` 209 | 210 | Without a relative path to `$GOPATH`, go has no way of knowing our `helpers` subpackage is `github.com/liggitt/a/helpers`. 211 | It also doesn't have a way to find `github.com/liggitt/b`. 212 | 213 | Let's turn our package into a go module: 214 | ```sh 215 | go mod init github.com/liggitt/a 216 | ``` 217 | 218 | > ``` 219 | > go: creating new go.mod: module github.com/liggitt/a 220 | > ``` 221 | 222 | ```sh 223 | cat go.mod 224 | ``` 225 | 226 | > ``` 227 | > module github.com/liggitt/a 228 | > 229 | > go 1.12 230 | > ``` 231 | 232 | Commit the initial version of our go.mod file: 233 | ```sh 234 | git add . && git commit -m "initial go.mod file" 235 | ``` 236 | 237 | #### What happens 238 | 239 | From `go help go.mod`: 240 | * The `module` verb defines the module path 241 | * The `go` verb sets the expected language version 242 | 243 | Now when we run, go can figure out our `helpers` package is `github.com/liggitt/a/helpers` 244 | by finding the closest parent dir containing a `go.mod` file, looking at the name of the module it defines, 245 | then appending the relative path to the `helpers` directory to get the full import path. 246 | 247 | #### Benefits 248 | 249 | * Allows developing outside of `$GOPATH` 250 | 251 | ### Resolving dependencies automatically 252 | 253 | Since we can no longer assume all go source is located under `$GOPATH`, how does go find source for dependencies? 254 | `go run` (and `go build`, `go list`, etc) will now fetch dependencies automatically from their canonical locations 255 | (the same place `go get` would fetch them) at run time, if needed: 256 | 257 | ```sh 258 | go run . 259 | ``` 260 | 261 | > ```sh 262 | > go: finding github.com/liggitt/b v1.0.0 263 | > go: downloading github.com/liggitt/b v1.0.0 264 | > go: extracting github.com/liggitt/b v1.0.0 265 | > cake! 266 | > ``` 267 | 268 | #### What happens 269 | 270 | That did a few things: 271 | 272 | 1. It noticed a dependency that wasn't represented in our `go.mod` file, so it resolved and added a `require` directive for it (`v1.0.0` happened to be the latest version): 273 | 274 | ```sh 275 | git diff go.mod 276 | ``` 277 | 278 | > ```diff 279 | > diff --git a/go.mod b/go.mod 280 | > index 34c6e02..9869e5e 100644 281 | > --- a/go.mod 282 | > +++ b/go.mod 283 | > @@ -1,3 +1,5 @@ 284 | > module github.com/liggitt/a 285 | > 286 | > go 1.12 287 | > + 288 | > +require github.com/liggitt/b v1.0.0 289 | > ``` 290 | 291 | From `go help go.mod`: 292 | * The `require` verb requires a particular module at a given version or later (editors note: the "or later" will be important) 293 | 294 | The syntax of a `require` directive is `require `. 295 | 296 | You can also group multiple `require` directives into a block, just like imports: 297 | ``` 298 | require ( 299 | example.com/thing1 v2.3.4 300 | example.com/thing2 v1.2.3 301 | ) 302 | ``` 303 | 304 | You can specify any resolveable tag, branch name, or SHA as a `require` version, 305 | and it will be canonicalized the next time the module graph is computed, 306 | and the `go.mod` file automatically updated. 307 | 308 | You can also run `go get @`, and go will resolve and add a 309 | `require` directive for the specified module and version to your `go.mod` file: 310 | 311 | ```sh 312 | go get github.com/liggitt/b@v1.0.0 313 | ``` 314 | 315 | Commit the updated go.mod file: 316 | ```sh 317 | git add . && git commit -m "require github.com/liggitt/b@v1.0.0" 318 | ``` 319 | 320 | 2. `go run` also downloaded the new dependency to a local module cache: 321 | 322 | ```sh 323 | find $GOPATH/pkg/mod/cache/download 324 | ``` 325 | 326 | > ```sh 327 | > /Users/liggitt/go/pkg/mod/cache/download 328 | > /Users/liggitt/go/pkg/mod/cache/download/github.com 329 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt 330 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b 331 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v 332 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/v1.0.0.mod 333 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/list.lock 334 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/v1.0.0.zip 335 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/v1.0.0.lock 336 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/v1.0.0.ziphash 337 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/v1.0.0.info 338 | > /Users/liggitt/go/pkg/mod/cache/download/github.com/liggitt/b/@v/list 339 | > ``` 340 | 341 | This cache is maintained automatically, but has a few associated commands: 342 | 343 | To clean the module cache: 344 | ```sh 345 | go clean -modcache 346 | ``` 347 | 348 | To force downloading dependencies: 349 | ```sh 350 | go mod download 351 | ``` 352 | 353 | You can prevent downloading modules from the network by setting the environment variable `GOPROXY` to `off`. 354 | If dependencies are not present in the module cache, and network requests are disabled, operations will fail: 355 | ```sh 356 | go run . 357 | ``` 358 | 359 | > ```sh 360 | > cake! 361 | > ``` 362 | 363 | ```sh 364 | GOPROXY=off go run . 365 | ``` 366 | 367 | > ```sh 368 | > cake! 369 | > ``` 370 | 371 | ```sh 372 | go clean -modcache 373 | GOPROXY=off go run . 374 | ``` 375 | 376 | > ```sh 377 | > go: github.com/liggitt/b@v1.0.0: module lookup disabled by GOPROXY=off 378 | > go: error loading module requirements 379 | > ``` 380 | 381 | You can also choose to copy all required dependency packages into a local vendor directory: 382 | ```sh 383 | go mod vendor 384 | find vendor 385 | ``` 386 | 387 | > ```sh 388 | > vendor 389 | > vendor/github.com 390 | > vendor/github.com/liggitt 391 | > vendor/github.com/liggitt/b 392 | > vendor/github.com/liggitt/b/cake.go 393 | > vendor/modules.txt 394 | > ``` 395 | 396 | Then run with `-mod=vendor` to tell go to use the local vendor directory when resolving dependency source: 397 | 398 | ```sh 399 | go clean -modcache 400 | GOPROXY=off GOFLAGS=-mod=vendor go run . 401 | ``` 402 | 403 | > ```sh 404 | > cake! 405 | > ``` 406 | 407 | Clean up the vendor directory before continuing: 408 | ```sh 409 | rm -fr vendor 410 | ``` 411 | 412 | #### Benefits 413 | 414 | * Dependencies don't drift from canonical source (the module cache computes checksums, which it compares with checksums in the optional `go.sum` file of the main module, and complains about differences) 415 | * There is (some) control over what versions of dependencies are used 416 | * There are built-in tools for caching and vendoring dependencies, and ensuring a hermetic build with no network access 417 | 418 | ### Distribution 419 | 420 | Someone who wanted to fetch a particular version of your module could do this: 421 | ```sh 422 | go get github.com/liggitt/a@v1.0.0 423 | ``` 424 | 425 | #### What happens 426 | 427 | * go resolves the version-control location for the specified import path, and resolves the specified version 428 | * go downloads the dependency to the module cache 429 | * go adds a `require` directive to the current module's `go.mod` file recording the specified version 430 | 431 | #### Benefits 432 | 433 | * Simple distribution for simple things 434 | * `go get` is version-aware (both for the requested module, and for its transitive dependencies) 435 | * No stomping of versions in a single `$GOPATH` location when developing multiple modules 436 | 437 | ### Replacing resolved module versions or source 438 | 439 | Let's see what happens when multiple modules in a build require different versions of the same module. 440 | 441 | First, let's add a new dependency: 442 | 443 | ```sh 444 | go get github.com/liggitt/c@v1.0.0 445 | ``` 446 | 447 | > ```diff 448 | > git diff go.mod 449 | > diff --git a/go.mod b/go.mod 450 | > index 9869e5e..16d2163 100644 451 | > --- a/go.mod 452 | > +++ b/go.mod 453 | > @@ -2,4 +2,7 @@ module github.com/liggitt/a 454 | > 455 | > go 1.12 456 | > 457 | > -require github.com/liggitt/b v1.0.0 458 | > +require ( 459 | > + github.com/liggitt/b v1.0.0 460 | > + github.com/liggitt/c v1.0.0 // indirect 461 | > +) 462 | > ``` 463 | 464 | The `// indirect` decoration is added because we are not actually using any packages from the module yet. 465 | 466 | Now change our `github.com/liggitt/a/make-cake.go` file to use the new module: 467 | 468 | ```diff 469 | package main 470 | 471 | import ( 472 | + "fmt" 473 | "github.com/liggitt/a/helpers" 474 | "github.com/liggitt/b" 475 | + "github.com/liggitt/c" 476 | ) 477 | 478 | func main() { 479 | b.Cake() 480 | + fmt.Println(c.CakeOrDeath()) 481 | helpers.PrintSuccess() 482 | } 483 | ``` 484 | 485 | ```sh 486 | go run . 487 | ``` 488 | 489 | > ```sh 490 | > cake, please 491 | > cake! 492 | > ``` 493 | 494 | Since we are now using a package from the module, the `// indirect` 495 | decoration was removed from `go.mod` when `go run` computed the module graph. 496 | 497 | Commit the changes: 498 | ```sh 499 | git add . && git commit -m "require github.com/liggitt/c@v1.0.0" 500 | ``` 501 | 502 | Now add and use another dependency in `github.com/liggitt/a/make-cake.go`: 503 | 504 | ```diff 505 | @@ -5,10 +5,12 @@ import ( 506 | "github.com/liggitt/a/helpers" 507 | "github.com/liggitt/b" 508 | "github.com/liggitt/c" 509 | + "github.com/liggitt/d" 510 | ) 511 | 512 | func main() { 513 | b.Cake() 514 | fmt.Println(c.CakeOrDeath()) 515 | + fmt.Println(d.Ingredients()) 516 | helpers.PrintSuccess() 517 | } 518 | ``` 519 | 520 | ```sh 521 | go get github.com/liggitt/d@v1.0.0 522 | go run . 523 | ``` 524 | 525 | > ``` 526 | > sorry, all out of cake... your choice is 'or death' 527 | > [eggs flour sugar butter] 528 | > cake! 529 | > ``` 530 | 531 | Wait... why did the output from `github.com/liggitt/c#CakeOrDeath()` change? 532 | 533 | ```sh 534 | git diff go.mod 535 | ``` 536 | 537 | > ```diff 538 | > diff --git a/go.mod b/go.mod 539 | > index c6e8ae8..7e155cf 100644 540 | > --- a/go.mod 541 | > +++ b/go.mod 542 | > @@ -4,5 +4,6 @@ go 1.12 543 | > 544 | > require ( 545 | > github.com/liggitt/b v1.0.0 546 | > - github.com/liggitt/c v1.0.0 547 | > + github.com/liggitt/c v1.1.0 548 | > + github.com/liggitt/d v1.0.0 549 | > ) 550 | > ``` 551 | 552 | Our version of `github.com/liggitt/c` changed from `v1.0.0` to `v1.1.0`. 553 | To see why, we can inspect our module dependency graph by running `go mod graph`: 554 | 555 | ```sh 556 | go mod graph 557 | ``` 558 | 559 | > ``` 560 | > github.com/liggitt/a github.com/liggitt/b@v1.0.0 561 | > github.com/liggitt/a github.com/liggitt/c@v1.1.0 562 | > github.com/liggitt/a github.com/liggitt/d@v1.0.0 563 | > github.com/liggitt/d@v1.0.0 github.com/liggitt/c@v1.1.0 564 | > ``` 565 | 566 | Here we see that `github.com/liggitt/d@v1.0.0` requires `github.com/liggitt/c@v1.1.0`. 567 | 568 | If we go look at [that module definition](https://github.com/liggitt/d/blob/v1.0.0/go.mod), that version is indeed required: 569 | 570 | ``` 571 | module github.com/liggitt/d 572 | 573 | go 1.12 574 | 575 | require github.com/liggitt/c v1.1.0 576 | ``` 577 | 578 | If multiple modules are involved in a build, and they require different versions 579 | of the same module, the maximum required version of the module is selected, 580 | and the `go.mod` file of the main module is updated to reflect that version. 581 | 582 | From `go help go.mod`: 583 | > The go command automatically updates go.mod each time it uses the 584 | > module graph, to make sure go.mod always accurately reflects reality 585 | > and is properly formatted. 586 | > 587 | > The update removes redundant or misleading requirements. 588 | > For example, if A v1.0.0 itself requires B v1.2.0 and C v1.0.0, 589 | > then go.mod's requirement of B v1.0.0 is misleading (superseded by 590 | > A's need for v1.2.0), and its requirement of C v1.0.0 is redundant 591 | > (implied by A's need for the same version), so both will be removed. 592 | > If module M contains packages that directly import packages from B or 593 | > C, then the requirements will be kept but updated to the actual 594 | > versions being used. 595 | > 596 | > Because the module graph defines the meaning of import statements, any 597 | > commands that load packages also use and therefore update go.mod, 598 | > including go build, go get, go install, go list, go test, go mod graph, 599 | > go mod tidy, and go mod why. 600 | 601 | This auto-updating of the go.mod file to reflect reality is alternately helpful and maddening. 602 | See https://github.com/golang/go/issues/29452 for discussion about this. 603 | 604 | Whatever the reason, we really want to use `github.com/liggitt/c@v1.0.0` when running our build. 605 | Fortunately, go modules allow the main module (the one where the go commands are run) to override 606 | the selected version of a module. This is done with a `replace` directive in the `go.mod` file. 607 | 608 | ```sh 609 | go mod edit -replace github.com/liggitt/c=github.com/liggitt/c@v1.0.0 610 | go run . 611 | ``` 612 | 613 | > ``` 614 | > cake, please 615 | > [eggs flour sugar butter] 616 | > cake! 617 | > ``` 618 | 619 | We're back to the behavior we wanted, let's see what that added to our `go.mod` file: 620 | 621 | ```sh 622 | git diff go.mod 623 | ``` 624 | 625 | > ```diff 626 | > diff --git a/go.mod b/go.mod 627 | > index c6e8ae8..8aae820 100644 628 | > --- a/go.mod 629 | > +++ b/go.mod 630 | > @@ -4,5 +4,8 @@ go 1.12 631 | > 632 | > require ( 633 | > github.com/liggitt/b v1.0.0 634 | > - github.com/liggitt/c v1.0.0 635 | > + github.com/liggitt/c v1.1.0 636 | > + github.com/liggitt/d v1.0.0 637 | > ) 638 | > + 639 | > +replace github.com/liggitt/c => github.com/liggitt/c v1.0.0 640 | > ``` 641 | 642 | The maximum version still appears as a `require` directive, 643 | but we are now forcing the version we want to be selected with a `replace` directive. 644 | 645 | Commit the changes: 646 | ```sh 647 | git add . && git commit -m "require github.com/liggitt/d v1.0.0, pin github.com/liggitt/c v1.0.0" 648 | ``` 649 | 650 | `replace` directives have many different uses, not just pinning versions. From `go help go.mod`: 651 | * The `replace` verb replaces a module version with a different module version 652 | 653 | Possible applications: 654 | 655 | * pin a dependency to a specific version: 656 | 657 | ```sh 658 | go mod edit -replace example.com/some/thing=example.com/some/thing@v1.0.0 659 | ``` 660 | produces this in `go.mod`: 661 | ``` 662 | replace example.com/some/thing => example.com/some/thing v1.0.0 663 | ``` 664 | 665 | * change the remote source for a dependency (useful for building/testing with forks): 666 | 667 | ```sh 668 | go mod edit -replace example.com/some/thing@v1.0.0=example.com/my/fork@v1.0.0 669 | ``` 670 | produces this in `go.mod`: 671 | ``` 672 | replace example.com/some/thing v1.0.0 => example.com/my/fork v1.0.0 673 | ``` 674 | 675 | * use a local source for a dependency (useful for developing multiple modules locally): 676 | 677 | ```sh 678 | go mod edit -replace example.com/some/thing=../local/path/to/source 679 | ``` 680 | produces this in `go.mod`: 681 | ``` 682 | replace some/thing => ../local/path/to/source 683 | ``` 684 | 685 | #### Caveats 686 | 687 | * `replace` directives apply only in the main module's `go.mod` and are ignored in dependencies, 688 | so they are not effective in modules intended to be used as libraries by other modules. 689 | * Modules you depend on via local path `replace` directives must also be published at their canonical 690 | locations in order for components that use your module to be able to resolve them 691 | * Changing the remote source for a dependency (for example, to point to a fork), 692 | must point to a drop-in-compatible location. No import rewriting is performed. 693 | 694 | ### Major versions 695 | 696 | What happens if a breaking change is made to a struct type, interface, or method signature, 697 | and two modules in the same build depend on incompatible versions? 698 | 699 | go modules allow module publishers to provide different major versions of a module, 700 | by adding a major version suffix to the module import path. This results in a completely 701 | new import tree, which can be used alongside the old tree. 702 | This is called [semantic import versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning). 703 | 704 | Let's make a [breaking change](https://github.com/liggitt/c/commit/6bfb5f89f40e7e6b10ce18771af841bf682044eb#diff-a6413ab6421e2f42b61888f9ac5b2db1) to `github.com/liggitt/c`: 705 | 706 | ```diff 707 | -func CakeOrDeath() string { 708 | +func CakeOrDeath(preference string) string { 709 | + if preference == "death" { 710 | + return "death it is" 711 | + } 712 | return "sorry, all out of cake... your choice is 'or death'" 713 | } 714 | ``` 715 | 716 | We also change the module name in `go.mod` file to append a major version suffix: 717 | 718 | ```diff 719 | -module github.com/liggitt/c 720 | +module github.com/liggitt/c/v2 721 | ``` 722 | 723 | Now we can tag that as [`v2.0.0`](https://github.com/liggitt/c/releases/tag/v2.0.0). 724 | 725 | Versions prior to v2.0.0 are special-cased by go modules and don't need a major version suffix, 726 | so callers that want to use v0.x or v1.x don't need to do anything special. 727 | 728 | Callers that want to use v2.0.0+ of a module that uses semantic import versioning 729 | have to [rewrite their go imports and change the required version of the module](https://github.com/liggitt/d/commit/c3bdd9d5b06e8387886616e30753140afdd39184) to add the versioned suffix: 730 | 731 | ```diff 732 | diff --git a/go.mod b/go.mod 733 | index 7d67056..fa289cd 100644 734 | --- a/go.mod 735 | +++ b/go.mod 736 | @@ -2,4 +2,4 @@ module github.com/liggitt/d 737 | 738 | go 1.12 739 | 740 | -require github.com/liggitt/c v1.1.0 741 | +require github.com/liggitt/c/v2 v2.0.0 742 | diff --git a/ingredients.go b/ingredients.go 743 | index 1053743..c2bcfd5 100644 744 | --- a/ingredients.go 745 | +++ b/ingredients.go 746 | @@ -3,7 +3,7 @@ package d 747 | import ( 748 | "strings" 749 | 750 | - "github.com/liggitt/c" 751 | + c "github.com/liggitt/c/v2" 752 | ) 753 | 754 | func Ingredients() []string { 755 | @@ -11,5 +11,5 @@ func Ingredients() []string { 756 | } 757 | 758 | func CakeAvailable() bool { 759 | - return !strings.Contains(c.CakeOrDeath(), "death") 760 | + return !strings.Contains(c.CakeOrDeath("cake"), "death") 761 | } 762 | ``` 763 | 764 | Because `github.com/liggitt/c` made that incompatible change in what is effectively a different package 765 | (`github.com/liggitt/c/v2`), we can upgrade to a version of `github.com/liggitt/d` that uses that new version, 766 | and our existing use of `github.com/liggitt/c` is unaffected: 767 | 768 | ```sh 769 | go get github.com/liggitt/d@v1.1.0 770 | go run . 771 | ``` 772 | 773 | > ``` 774 | > cake, please 775 | > [eggs flour sugar butter] 776 | > cake! 777 | > ``` 778 | 779 | We can see both versions are in use: 780 | 781 | ```sh 782 | go mod graph 783 | ``` 784 | 785 | > ``` 786 | > github.com/liggitt/a github.com/liggitt/b@v1.0.0 787 | > github.com/liggitt/a github.com/liggitt/c@v1.1.0 788 | > github.com/liggitt/a github.com/liggitt/d@v1.1.0 789 | > github.com/liggitt/d@v1.1.0 github.com/liggitt/c/v2@v2.0.0 790 | > ``` 791 | 792 | Commit the updated go.mod file: 793 | ```sh 794 | git add . && git commit -m "require github.com/liggitt/d v1.1.0" 795 | ``` 796 | 797 | As we transition through multiple versions of dependencies, the checksum file (`go.sum`) 798 | can accumulate checksums for dependencies we no longer use. To prune unused `require` directives 799 | and unused checksums, running `go mod tidy` is recommended before publishing a module: 800 | 801 | ```sh 802 | go mod tidy 803 | git diff 804 | ``` 805 | 806 | ```diff 807 | diff --git a/go.sum b/go.sum 808 | index fa373b9..8262cc2 100644 809 | --- a/go.sum 810 | +++ b/go.sum 811 | @@ -2,11 +2,7 @@ github.com/liggitt/b v1.0.0 h1:b2PD0maWheor82stUPtMmVt15kSIal//bKvvXGW6m/c= 812 | github.com/liggitt/b v1.0.0/go.mod h1:ELIy9WS4GN+KTnXJ4EeReLowuaddQ7tnfsXV6BxuPiw= 813 | github.com/liggitt/c v1.0.0 h1:jLi0imaDl5DUIPX63+zI6vB5w2UF6wDa8F7bLMBDYko= 814 | github.com/liggitt/c v1.0.0/go.mod h1:F/PN0EPwqEcaXbrAc1E8sqvkXqQjvWH2whyWJoSkgA0= 815 | -github.com/liggitt/c v1.1.0 h1:Uc7ExZIjGtvzmUjpGvpaSEWG2xHEQ/mJbK7sScoCYEQ= 816 | -github.com/liggitt/c v1.1.0/go.mod h1:F/PN0EPwqEcaXbrAc1E8sqvkXqQjvWH2whyWJoSkgA0= 817 | github.com/liggitt/c/v2 v2.0.0 h1:7ITR3NLf81hJyIaFoUPUUl1QrFm7lnOjoH2v+fqsZf0= 818 | github.com/liggitt/c/v2 v2.0.0/go.mod h1:KRLLWBp7oLaMdqk8sLfRgalmCgDMNvQ4TQH5UrZevtI= 819 | -github.com/liggitt/d v1.0.0 h1:7CL2m7s17MyF3C2R2Y7jCMt8Czp/Ats2DClddZoN+88= 820 | -github.com/liggitt/d v1.0.0/go.mod h1:4GgUKOe83Va2bM/CVugCwU5aIM0V9Y5NpL1bg1rKZGE= 821 | github.com/liggitt/d v1.1.0 h1:MlVbFLjh4c5OZtN94h6Gj8x6BSpqXKi4epUIgSUdnFE= 822 | github.com/liggitt/d v1.1.0/go.mod h1:kLQGcLlWWIApPBG+JqB8mWJamzof4jHZ2nIhzfaullk= 823 | ``` 824 | 825 | The checksums for the unused versions of `github.com/liggitt/c` and `github.com/liggitt/d` are removed. 826 | 827 | Commit the updated go.mod file: 828 | ```sh 829 | git add . && git commit -m "go mod tidy" 830 | ``` 831 | 832 | #### Benefits of semantic import versioning 833 | 834 | * Allows side-by-side use of different major versions in the same build 835 | * Allows a module to make use of other major versions of itself if it wants to (`github.com/liggitt/c/v2` could use `github.com/liggitt/c`) 836 | 837 | #### Caveats of semantic import versioning 838 | 839 | * It's not magic... it behaves like an unrelated import path, so all callers have to change their imports to pick up the new version 840 | * Even internal imports in the versioned module have to use the version-qualified import paths 841 | * Deeply nested modules with lots of internal imports are painful to increment 842 | 843 | ### Tips 844 | 845 | #### Getting started 846 | 847 | Opt in to using go modules: 848 | ```sh 849 | export GO111MODULE=on 850 | ``` 851 | 852 | Create a new module: 853 | ``` 854 | go mod init 855 | ``` 856 | 857 | #### Querying 858 | 859 | New commands (or module-aware versions of existing commands) for extracting module and dependency info: 860 | ``` 861 | go mod graph 862 | go mod why 863 | go list -m all 864 | go list -m -json all 865 | go mod edit -json 866 | ``` 867 | 868 | Be aware that all of these (except `go mod edit -json`) have the potential to 869 | modify the `go.mod` file in the process of computing the module graph. 870 | 871 | #### Cleaning up 872 | 873 | Tidy a go.mod file (resolve references, remove unused replace directives, sort, etc), 874 | and populate the `go.sum` file with checksums for the selected module's versions: 875 | ```sh 876 | go mod tidy 877 | ``` 878 | 879 | #### Low-level go.mod manipulation 880 | 881 | These commands all perform direct manipulation of the go.mod file. 882 | They do not resolve the module graph, so they do not consult the 883 | network or module cache, or modify required versions. 884 | All the invocations which modify the `go.mod` file also format it (`-fmt` is implied). 885 | 886 | Dump to JSON: 887 | ```sh 888 | go mod edit -json 889 | ``` 890 | 891 | Add a require directive: 892 | ```sh 893 | go mod edit -require github.com/liggitt/a@v1.0.0 894 | ``` 895 | 896 | Remove a require directive: 897 | ```sh 898 | go mod edit -droprequire github.com/liggitt/a 899 | ``` 900 | 901 | Add a replace directive to pin versions: 902 | ```sh 903 | go mod edit -replace github.com/liggitt/c=github.com/liggitt/c@v1.0.0 904 | ``` 905 | 906 | Add a replace directive to use a forked version: 907 | ```sh 908 | go mod edit -replace github.com/liggitt/c=github.com/example/c-fork@v1.0.0 909 | ``` 910 | 911 | Add a replace directive to use a local path: 912 | ```sh 913 | go mod edit -replace github.com/liggitt/c=../path/to/source 914 | ``` 915 | 916 | Remove a replace directive: 917 | ```sh 918 | go mod edit -dropreplace github.com/liggitt/c 919 | ``` 920 | 921 | Format the `go.mod` file: 922 | ```sh 923 | go mod edit -fmt 924 | ``` 925 | 926 | See `go help mod edit` for details 927 | 928 | #### Local module cache 929 | 930 | Ensure the local module cache has all required dependencies: 931 | ```sh 932 | go mod download 933 | ``` 934 | 935 | Clear the local module cache: 936 | ```sh 937 | go clean -modcache 938 | ``` 939 | 940 | #### Module sources 941 | 942 | By default, missing modules are searched/fetched using the network. 943 | This can happen in response to go commands that were previously local-only, 944 | like `go list`. 945 | 946 | To prevent go operations from hitting the network (and fail if they need to): 947 | ```sh 948 | export GOPROXY=off 949 | ``` 950 | 951 | To use a directory instead of the network, do `GOPROXY=file:///...`. 952 | The module cache maintained by go is in the expected structure to be used this way: 953 | ```sh 954 | export GOPROXY=file://$GOPATH/pkg/mod/cache/download 955 | ``` 956 | 957 | You can also point GOPROXY at a network location of a module proxy. 958 | 959 | See `go help goproxy` for details 960 | 961 | ### Additional Reading 962 | 963 | * `go help` topics: 964 | 965 | ```sh 966 | go help modules 967 | go help go.mod 968 | go help mod 969 | go help module-get 970 | go help goproxy 971 | ``` 972 | 973 | * https://github.com/golang/go/wiki/Modules, especially: 974 | * https://github.com/golang/go/wiki/Modules#semantic-import-versioning 975 | * https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release 976 | * https://golang.org/cmd/go/#hdr-The_go_mod_file 977 | * https://golang.org/cmd/go/#hdr-Maintaining_module_requirements 978 | * https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning 979 | * https://blog.golang.org/using-go-modules 980 | * https://blog.golang.org/modules2019 981 | * [Open golang issues related to modules](https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+label%3Amodules) 982 | --------------------------------------------------------------------------------