├── README.md ├── boring01.go ├── boring02.go ├── boring03.go ├── boring04.go ├── boring05.go ├── boring06.go ├── boring07.go ├── boring08.go ├── boring09.go ├── boring10.go ├── boring11.go ├── boring12.go ├── boring13.go ├── boring14.go ├── daisyChain.go ├── googleSearch01.go ├── googleSearch02.go ├── googleSearch03.go ├── googleSearch04.go ├── googleSearch05.go └── imgs ├── chinese_whispers.png └── fan_in.png /README.md: -------------------------------------------------------------------------------- 1 | Go concurrency patterns 2 | ======================= 3 | 4 | >该文档源自 Rob Pike 在google IO中的演讲，版权归原作者所有。如果你在天朝，可以点击[这里](http://v.youku.com/v_show/id_XNDI1NjgxMTAw.html)观看演讲视频。所有的代码皆为本人根据演讲稿内容编写，并调试通过。如发现bug欢迎提交更新。 5 | 6 | ## Concurrency features in Go ## 7 | 8 | People seemed fascinated by the concurrency features of Go when the language was firest announced. 9 | 10 | Questions: 11 | 12 | - Why is concurrency supported? 13 | - What is concurrency, anyway? 14 | - Where does the idea come from? 15 | - What is it good for? 16 | - How do i use it? 17 | 18 | ## Why? ## 19 | 20 | Look around you, What do you see? 21 | 22 | Do you see a single-stepping world doing one thing at a time? 23 | 24 | Or do you see a complex world of interacting, independently behaving pieces? 25 | 26 | That's why. Sequential processing on its own does not model the world's behavior. 27 | 28 | ## What is concurrency? ## 29 | 30 | Concurrency is the composition of independently executing computations. 31 | 32 | Concurrency is a way to structure software, particaularly as a way to write clean code that interacts well with the real world. 33 | 34 | It is not parallelism. 35 | 36 | ## Concurrency is not paralleism ## 37 | 38 | Concurrency is not paralleism, although it enables parallelism. 39 | 40 | If you have only one processor, your program can still be concurrent but it cannot be parallel. 41 | 42 | On the other hand, a well-written concurrent program might run efficiently in parallel on a multiprocessor. That property could be important... 43 | 44 | See [tinyurl.com/goconcnotpar](http://tinyurl.com/goconcnotpar) for more on that distinction. Too much to discuss here. 45 | 46 | ## A model for software construction ## 47 | 48 | Easy to understand. 49 | 50 | Easy to use. 51 | 52 | Easy to reason about. 53 | 54 | You don't need to be an expert! 55 | 56 | (Much nicer than dealing with the minutiae of parallelism (threads, semaphores, locks, barries, etc.)) 57 | 58 | ## History ## 59 | 60 | To many, the concurrency features of Go seemed new. 61 | 62 | But they are rooted in a long history, reaching back to Hoare's CSP in 1978 and even Dijkstra's guarded commands(1975). 63 | 64 | Languages with similar features: 65 | 66 | - Occam (May, 1983) 67 | - Erlang (Armstrong, 1986) 68 | - Newsqueak (Pike, 1988) 69 | - Concurrent ML (Reppy, 1993) 70 | - Alef (Winterbottom, 1995) 71 | - Limbo (Dorward, Pike, Winterbottom, 1996) 72 | 73 | ## Distinction ## 74 | 75 | Go is the latest on the Newsqueak-Alef-Limbo branch, distinguished by first-class channels. 76 | 77 | Erlang is closer to the original CSP, where you communicate to a process by name rather than over a channel. 78 | 79 | The models are equivalent but express things differently 80 | 81 | Rough analogy: writing to a file by name(process, Erlang) vs. writing to a file descriptor (channel, Go). 82 | 83 | ## Basic Examples ## 84 | 85 | ## A boring function ## 86 | 87 | We need an example to show the interesting properties of the concurrency primitives. 88 | 89 | To avoid distraction, we make it a boring example. 90 | 91 | func boring(msg string) { 92 | for i := 0; ; i++ { 93 | fmt.Println(msg, i) 94 | time.Sleep(time.Second) 95 | } 96 | } 97 | 98 | ## Slightly less boring ## 99 | 100 | Make the intervals between messages unpredictable (still under a second). 101 | 102 | func boring(msg string) { 103 | for i := 0; ; i++ { 104 | fmt.Println(msg, i) 105 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 106 | } 107 | } 108 | 109 | [code]() 110 | 111 | ## Running it ## 112 | 113 | The boring function runs on forever, like a boring party guest. 114 | 
func main() {
115 | 	boring("boring!")
116 | }
117 | 
118 | func boring(msg string) {
119 | 	for i := 0; ; i++ {
120 | 		fmt.Println(msg, i)
121 | 		time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
122 | 	}
123 | }
124 |
125 | [code]() 126 | 127 | ## Ignoring it ## 128 | 129 | The go statement runs the function as usual, but doesn't make the caller wait. 130 | 131 | It launches a goroutine. 132 | 133 | The functionality is analogous to the & on the end of a shell command. 134 | 
package main
135 | 
136 | import (
137 | 	"fmt"
138 | 	"time"
139 | 	"math/rand"
140 | 	)
141 | 
142 | func main() {
143 | 	go boring("boring!")
144 | }
145 |
146 | [code]() 147 | 148 | ## Ignoring it a little less ## 149 | 150 | When main returns, the program exits and takes the boring function down with it. 151 | 152 | We can hang around a little, and on the way show that both main and the launched goroutine are running. 153 | 154 | func main() { 155 | go boring("boring!") 156 | fmt.Println("I'm listening") 157 | time.Sleep(2 * time.Second) 158 | fmt.Println("You're boring; I'm leaving.") 159 | } 160 | 161 | [code]() 162 | 163 | ## Goroutines ## 164 | 165 | What is a goroutine? It's an independently executing function, launched by a go statement. 166 | 167 | It has its own call stack, which grows and shrinks as required. 168 | 169 | It's very cheap. It's practical to have thousands, even hundreds of thousands of goroutines. 170 | 171 | It's not a thread. 172 | 173 | There might be only one thread in a program with thousands of goroutines. 174 | 175 | Instead, goroutines are multiplexed dynamically onto threads are needed to keep all the goroutines running. 176 | 177 | But if you think of it as a very cheap thread, you won't be far off. 178 | 179 | ## Communication ## 180 | 181 | Our boring examples cheated: the main function couldn't see the output from the other goroutine. 182 | 183 | It was just printed to the screen, where we pretended we saw a conversation. 184 | 185 | Real conversations require communication. 186 | 187 | ## Channels ## 188 | 189 | A channel in Go provides a connection betwwen two goroutines, allowing them to communicate. 190 | 
// Declaring and initializing.
191 | var c chan int
192 | c = make(chan int)
193 | // or
194 | c := make(chan int)
195 |
196 | 197 | 
// Sending on a channel.
198 | c <- 1
199 |
200 | 201 | 
// Receiving from a channel.
202 | // The "arrow" indicates the direction of data flow.
203 | value = <- c
204 |
205 | 206 | ## Using channels ## 207 | 208 | A channel connects the main and boring goroutines so they can communicate. 209 | 210 | 
func main() {
211 | 	c := make(chan string)
212 | 	go boring("boring!", c)
213 | 	for i := 0; i < 5; i++ {
214 | 		fmt.Printf("You say: %q\n", <-c) // Receive expression is just a value.
215 | 	}
216 | 	fmt.Println("You're boring; I'm leaving.")
217 | }
218 | 
219 | func boring(msg string, c chan string) {
220 | 	for i := 0; ; i++ {
221 | 		c <- fmt.Sprintf("%s %d", msg, i) // Expression to be sent can be any suitable value.
222 | 		time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
223 | 	}
224 | }
225 |
226 | 227 | [code]() 228 | 229 | ## Synchroniztion ## 230 | 231 | When the main function executes <-c, it will wait for a value to be sent. 232 | 233 | Similarly, when the boring function executes c <- value, it waits for a receiver to be ready. 234 | 235 | A sender and receiver must both be ready to play their part in the communication. Otherwise we wait until they are. 236 | 237 | Thus channels both communicate and synchronize. 238 | 239 | ## An aside about buffered channels ## 240 | 241 | Note for experts: Go channels can also be created with a buffer. 242 | 243 | Buffering removes synchronization. 244 | 245 | Buffering makes them more like Erlang's mailboxes. 246 | 247 | Buffered channels can be important for some problems but they are more subtle to reason about. 248 | 249 | We won't need them today. 250 | 251 | ## The Go approach ## 252 | 253 | > Don't communicate by sharing memory, share memory by communicating. 254 | 255 | ## Patterns ## 256 | 257 | ## Generator: function that returns a channel ## 258 | 259 | Channels are first-class values, just like strings or integers. 260 | 
func main() {
261 | 	c := boring("boring!") // Function returning a channel.
262 | 	for i := 0; i < 5; i++ {
263 | 		fmt.Printf("You say: %q\n", <-c) 
264 | 	}
265 | 	fmt.Println("You're boring; I'm leaving.")
266 | }
267 | 
268 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings.
269 | 	c := make(chan string)
270 | 	go func() {
271 | 		for i := 0; ; i++ {
272 | 			c <- fmt.Sprintf("%s %d", msg, i) 
273 | 			time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
274 | 		}
275 | 	}()
276 | 	return c // Return the channel to the caller.
277 | }
278 |
279 | 280 | [code]() 281 | 282 | ## Channels as a handle on a service ## 283 | 284 | Our boring function returns a channel that lets us communicate with the boring service it provides. 285 | 286 | We can have more instances of the service. 287 | 288 | 
func main(){
289 | 	joe := boring("Joe")
290 | 	ann := boring("Ann")
291 | 	for i := 0; i < 5; i++ {
292 | 		fmt.Println(<-joe)
293 | 		fmt.Println(<-ann)
294 | 	}
295 | 	fmt.Println("You're both boring; I'm leaving.")
296 | }
297 |
298 | [code]() 299 | 300 | ## Multiplexing ## 301 | 302 | These programs make Joe and Ann count in lockstep. 303 | We can instead use a fan-in function to let whosoever is ready talk. 304 | 305 | 
func fanIn(input1, input2 <-chan string) <-chan string {
306 | 	c := make(chan string)
307 | 	go func() { for { c <- <-input1 } }()
308 | 	go func() { for { c <- <-input2 } }()
309 | 	return c
310 | }
311 |
312 | 313 | 
func main(){
314 | 	c := fanIn(boring("Joe"), boring("Ann"))
315 | 	for i := 0; i < 10; i++ {
316 | 		fmt.Println(<-c)
317 | 	}
318 | 	fmt.Println("You're both boring; I'm leaving.")
319 | }
320 |
321 | [code]() 322 | 323 | ## Fan-in ## 324 | ![Fan-in](imgs/fan_in.png?raw=true) 325 | 326 | ## Restoring sequencing ## 327 | 328 | Send a channel on a channel, making goroutine wait its turn. 329 | 330 | Receive all messages, then enable them again by sending on a private channel. 331 | 332 | First we define a message type that contains a channel for the reply. 333 | 334 | 
type Message struct {
335 | 	str string
336 | 	wait chan bool
337 | }
338 |
339 | 340 | ## Restoring sequencing ## 341 | 342 | Each speaker must wait for a go-ahead. 343 | 344 | for i := 0; i < 5; i++ { 345 | msg1 := <-c; fmt.Println(msg1.str) 346 | msg2 := <-c; fmt.Println(msg2.str) 347 | msg1.wait <- true 348 | msg2.wait <- true 349 | } 350 | 351 | 352 | 353 | waitForIt := make(chan bool) // Shared between all messages. 354 | 355 | 356 | 357 | c <- Message( fmt.Sprintf("%s: %d", msg, i), waitForIt ) 358 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 359 | <- waitForIt 360 | 361 | [code]() 362 | 363 | ## Select ## 364 | 365 | A control structure unique to concurrency. 366 | 367 | The reason channels and goroutines are built into the language. 368 | 369 | ## Select ## 370 | 371 | The select statement provides another way to handle multiple channels. 372 | It's like a switch, but each case is a communication: 373 | 374 | - All channels are evaluated. 375 | - Selection blocks until one communication can proceed, which then does. 376 | - If multiple can proceed, select chooses pseudo-randomly. 377 | - A default clause, if present, executes immediately if no channel is ready. 378 | 379 | 380 | 381 | select { 382 | case v1 := <-c1: 383 | fmt.Printf("received %v from c1\n", v1) 384 | case v2 := <-c2: 385 | fmt.Printf("received %v from c2\n", v2) 386 | case c3 <- 23: 387 | fmt.Printf("sent %v to c3\n", 23) 388 | default: 389 | fmt.Printf("no one was ready to communiction\n") 390 | } 391 | 392 | ## Fan-in again ## 393 | 394 | Rewrite our original fanin function. Only one goroutine is needed. Old: 395 | 396 | 
func fanIn(input1, input2 <-chan string) <-chan string {
397 | 	c := make(chan string)
398 | 	go func() { for { c <- <-input1 } }()
399 | 	go func() { for { c <- <-input2 } }()
400 | 	return c
401 | }
402 |
403 | 404 | ## Fan-in using select ## 405 | 406 | Rewrite our original fanin function. Only one goroutine is needed. New: 407 | 408 | 
func fanIn(input1, input2 <-chan string) <-chan string {
409 | 	c := make(chan string)
410 | 	go func() {
411 | 		for {
412 | 			select {
413 | 				case s := <-input1: c <- s
414 | 				case s := <-input2: c <- s
415 | 			}
416 | 		}
417 | 	}()
418 | 	return c
419 | }
420 |
421 | [code]() 422 | 423 | ## Timeout using select ## 424 | 425 | The time.After function returns a channel that blocks for the specified duration. 426 | After the interval, the channel delivers the current time, once. 427 | 428 | 
func main(){
429 | 	c := boring("Joe")
430 | 	for {
431 | 		select {
432 | 		case s:= <-c:
433 | 			fmt.Println(s)
434 | 		case <-time.After(1 * time.Second):
435 | 			fmt.Println("You're too slow.")
436 | 			return
437 | 		}
438 | 		
439 | 	}
440 | 	fmt.Println("You're both boring; I'm leaving.")
441 | }
442 |
443 | [code]() 444 | 445 | ## Timeout for whole conversation using select ## 446 | 447 | Create the timer once, outside the loop, to time out the entire conversation. 448 | (In the previous program, we had a timeout for each message.) 449 | 450 | 
func main(){
451 | 	c := boring("Joe")
452 | 	timeout := time.After(5 * time.Second)
453 | 	for {
454 | 		select {
455 | 		case s:= <-c:
456 | 			fmt.Println(s)
457 | 		case <-timeout:
458 | 			fmt.Println("You're talk too much.")
459 | 			return
460 | 		}
461 | 		
462 | 	}
463 | 	fmt.Println("You're both boring; I'm leaving.")
464 | }
465 |
466 | [code]() 467 | 468 | ## Quit channel ## 469 | 470 | We can turn this around and tell Joe to stop when we're tired of listening to him. 471 | 472 | 
    quit := make(chan bool)
473 | 	c := boring("Joe", quit)
474 | 	for i := rand.Intn(20); i >= 0; i-- { fmt.Println(<-c) }
475 | 	quit <- true
476 |
477 | 478 | 
    select {
479 | 	case c <- fmt.Sprintf("%s %d", msg, i):
480 | 		// do nothing
481 | 	case <-quit:
482 | 		return
483 | 	}
484 |
485 | [code]() 486 | 487 | ## Receive on quit channel ## 488 | 489 | How do we know it's finished? Wait for it to tell us it's done: receive on the quit channel 490 | 491 | 
    quit := make(chan string)
492 | 	c := boring("Joe", quit)
493 | 	for i := rand.Intn(20); i >= 0; i-- { fmt.Println(<-c) }
494 | 	quit <- "Bye!"
495 | 	fmt.Printf("Joe says: %q\n", <-quit)
496 |
497 | 498 | 
			   
499 | 			select {
500 | 			case c <- fmt.Sprintf("%s %d", msg, i):
501 | 				// do nothing
502 | 			case <-quit:
503 | 				cleanup()
504 | 				quit <- "See you!"
505 | 				return
506 | 			}
507 |
508 | [code]() 509 | 510 | ## Daisy-chain ## 511 | 512 | func f(left, right chan int) { 513 | left <- 1 + <-right 514 | } 515 | 516 | func main() { 517 | const n = 10000 518 | leftmost := make(chan int) 519 | right := leftmost 520 | left := leftmost 521 | for i := 0; i < n; i++ { 522 | right = make(chan int) 523 | go f(left, right) 524 | left = right 525 | } 526 | go func(c chan int) { c <- 1}(right) 527 | fmt.Println(<-leftmost) 528 | } 529 | 530 | [code]() 531 | 532 | ## Chinese whispers, gopher style ## 533 | ![](imgs/chinese_whispers.png?raw=true) 534 | 535 | ## Systems software ## 536 | 537 | Go was designed for writing systems software. 538 | Let's see how the concurrency features come into play. 539 | 540 | ## Example: Google Search ## 541 | 542 | Q: What does Google search do? 543 | 544 | A: Given a query, return a page of search results (and some ads). 545 | 546 | Q: How do we get the search results? 547 | 548 | A: Send the query to Web search, Image search, YouTube, Maps, News, etc., then mix the results. 549 | 550 | How do we implement this? 551 | 552 | ## Google Search: A fake framework ## 553 | 554 | We can simulate the search function, much as we simulated conversation before. 555 | 556 | ## Google Search 1.0 ## 557 | 558 | The Google functio takes a query and returns a slice of Results (which are just strings). 559 | 560 | Google invokes Web, Image, and Video searches serially, appending them to the results slice. 561 | 562 | func Google(query string) []Result{ 563 | results := make([]Result, 3, 10) 564 | results = append(results, Web(query)) 565 | results = append(results, Image(query)) 566 | results = append(results, Video(query)) 567 | return results 568 | } 569 | 570 | [code]() 571 | 572 | ## Google Search 2.0 ## 573 | 574 | Run the Web, Image, and Video searchs concurently, and wait for all results. 575 | 576 | No locks, No condition variables. No callbacks. 577 | 578 | func Google(query string) (results []Result) { 579 | c := make(chan Result) 580 | go func() { c <- Web(query) } () 581 | go func() { c <- Image(query) } () 582 | go func() { c <- Video(query) } () 583 | 584 | for i :=0; i < 3; i++ { 585 | result := <-c 586 | results = append(results, result) 587 | } 588 | return 589 | } 590 | 591 | [code]() 592 | 593 | ## Google Search 2.1 ## 594 | 595 | Don't wait for slow servers. No locks. No condition variables. No callbacks. 596 | 597 | func Google(query string) (results []Result) { 598 | c := make(chan Result) 599 | go func() { c <- Web(query) } () 600 | go func() { c <- Image(query) } () 601 | go func() { c <- Video(query) } () 602 | 603 | timeout := time.After(80 * time.Millisecond) 604 | for i :=0; i < 3; i++ { 605 | select { 606 | case result := <-c: 607 | results = append(results, result) 608 | case <-timeout: 609 | fmt.Println("timed out") 610 | return 611 | } 612 | 613 | } 614 | return 615 | } 616 | 617 | [code]() 618 | 619 | ## Avoid timeout ## 620 | 621 | Q: How do we avoid discarding srsults from slow servers? 622 | 623 | A: Replicate the servers. Send request to multiple replicas, and use the first response. 624 | 625 | func First(query string, replicas ...Search) Result { 626 | c := make(chan Result) 627 | searchReplica := func(i int) { c <- replicas[i](query) } 628 | for i := range replicas { 629 | go searchReplica(i) 630 | } 631 | return <-c 632 | } 633 | 634 | ## Using the First function ## 635 | 636 | func main() { 637 | rand.Seed(time.Now().UnixNano()) 638 | start := time.Now() 639 | result := First("golang", fakeSearch("replica 1"), 640 | fakeSearch("replica 2")) 641 | elapsed := time.Since(start) 642 | fmt.Println(results) 643 | fmt.Println(elapsed) 644 | } 645 | 646 | [code]() 647 | 648 | ## Google Search 3.0 ## 649 | 650 | Reduce tail latency using replicated search servers. 651 | 652 | func Google(query string) (results []Result) { 653 | c := make(chan Result) 654 | go func() { c <- First(query, Web1, Web2) } () 655 | go func() { c <- First(query, Image1, Image2) } () 656 | go func() { c <- First(query, Video1, Video2) } () 657 | 658 | timeout := time.After(80 * time.Millisecond) 659 | for i :=0; i < 3; i++ { 660 | select { 661 | case result := <-c: 662 | results = append(results, result) 663 | case <-timeout: 664 | fmt.Println("timed out") 665 | return 666 | } 667 | 668 | } 669 | return 670 | } 671 | 672 | [code]() 673 | 674 | ## And still... ## 675 | 676 | > No locks. No condition variables. No callbacks. 677 | 678 | ## Summary ## 679 | 680 | In just a few simple transformations we used Go's concurrency primitives to convert a 681 | 682 | - slow 683 | - sequential 684 | - failure-sensitive 685 | 686 | program into one that is 687 | 688 | - fast 689 | - concurrent 690 | - replicated 691 | - robust. 692 | 693 | ## More party tricks ## 694 | 695 | There are endless ways to use these tools, many presented elsewhere. 696 | 697 | Chatroulette toy: 698 | 699 | [tinyurl.com/gochatroulette](http://tinyurl.com/gochatroulette) 700 | 701 | Load balancer: 702 | 703 | [tinyurl.com/goloadbalancer](http://tinyurl.com/goloadbalancer) 704 | 705 | Concurrent prime sieve. 706 | 707 | [tinyurl.com/gosieve](http://tinyurl.com/gosieve) 708 | 709 | Concurrent power series (by Mcllroy): 710 | 711 | [tinyurl.com/gopowerseries](http://tinyurl.com/gopowerseries) 712 | 713 | ## Don't overdo it ## 714 | 715 | The're fun to play with, but don't overuse these ideas 716 | 717 | Goroutines and channels are big ideas. They're tools for program construnction. 718 | 719 | But sometimes all you need is a reference counter. 720 | 721 | Go has "sync" and "sync/atomic" packages that provide mutexes, condition variables, etc. They provide tools for smaller problems. 722 | 723 | Ofter, these things will work together to solve a bigger problem. 724 | 725 | Always use the right tool for the job. 726 | 727 | ## Conclusions ## 728 | 729 | Goroutines and channels make it easy to express complex operations dealing with 730 | 731 | - multiple inputs 732 | - multiple outputs 733 | - timeouts 734 | - failure 735 | 736 | And they're fun to use. 737 | 738 | ## Links ## 739 | Go Home Page: 740 | 741 | [golang.org](http://golang.org) 742 | 743 | Go Tour (learn Go in your browser) 744 | 745 | [tour.golang.org](http://tour.golang.org) 746 | 747 | Package documentation: 748 | 749 | [golang.org/pkg](http://golang.org/pkg) 750 | 751 | Articles galore: 752 | 753 | [golang.org/doc](http://golang.org/doc) 754 | 755 | Concurrency is not parallelism: 756 | 757 | [tinyurl.com/goconcnotpar](http://tinyurl.com/goconcnotpar) 758 | 759 | -------------------------------------------------------------------------------- /boring01.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | ) 7 | 8 | func boring(msg string) { 9 | for i := 0; ; i++ { 10 | fmt.Println(msg, i) 11 | time.Sleep(time.Second) 12 | } 13 | } 14 | 15 | func main() { 16 | boring("boring!") 17 | } 18 | -------------------------------------------------------------------------------- /boring02.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func boring(msg string) { 10 | for i := 0; ; i++ { 11 | fmt.Println(msg, i) 12 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 13 | } 14 | } 15 | 16 | func main() { 17 | boring("boring!") 18 | } 19 | -------------------------------------------------------------------------------- /boring03.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main() { 10 | go boring("boring!") 11 | } 12 | 13 | func boring(msg string) { 14 | for i := 0; ; i++ { 15 | fmt.Println(msg, i) 16 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 17 | } 18 | } -------------------------------------------------------------------------------- /boring04.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main() { 10 | go boring("boring!") 11 | fmt.Println("I'm listening") 12 | time.Sleep(2 * time.Second) 13 | fmt.Println("You're boring; I'm leaving.") 14 | } 15 | 16 | func boring(msg string) { 17 | for i := 0; ; i++ { 18 | fmt.Println(msg, i) 19 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 20 | } 21 | } -------------------------------------------------------------------------------- /boring05.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main() { 10 | c := make(chan string) 11 | go boring("boring!", c) 12 | for i := 0; i < 5; i++ { 13 | fmt.Printf("You say: %q\n", <-c) // Receive expression is just a value. 14 | } 15 | fmt.Println("You're boring; I'm leaving.") 16 | } 17 | 18 | func boring(msg string, c chan string) { 19 | for i := 0; ; i++ { 20 | c <- fmt.Sprintf("%s %d", msg, i) // Expression to be sent can be any suitable value. 21 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 22 | } 23 | } -------------------------------------------------------------------------------- /boring06.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main() { 10 | c := boring("boring!") // Function returning a channel. 11 | for i := 0; i < 5; i++ { 12 | fmt.Printf("You say: %q\n", <-c) 13 | } 14 | fmt.Println("You're boring; I'm leaving.") 15 | } 16 | 17 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 18 | c := make(chan string) 19 | go func() { // We launch the goroutine from inside the function. 20 | for i := 0; ; i++ { 21 | c <- fmt.Sprintf("%s %d", msg, i) 22 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 23 | } 24 | }() 25 | return c // Return the channel to the caller. 26 | } -------------------------------------------------------------------------------- /boring07.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | joe := boring("Joe") 11 | ann := boring("Ann") 12 | for i := 0; i < 5; i++ { 13 | fmt.Println(<-joe) 14 | fmt.Println(<-ann) 15 | } 16 | fmt.Println("You're both boring; I'm leaving.") 17 | } 18 | 19 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 20 | c := make(chan string) 21 | go func() { // We launch the goroutine from inside the function. 22 | for i := 0; ; i++ { 23 | c <- fmt.Sprintf("%s %d", msg, i) 24 | time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond) 25 | } 26 | }() 27 | return c // Return the channel to the caller. 28 | } -------------------------------------------------------------------------------- /boring08.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | c := fanIn(boring("Joe"), boring("Ann")) 11 | for i := 0; i < 10; i++ { 12 | fmt.Println(<-c) 13 | } 14 | fmt.Println("You're both boring; I'm leaving.") 15 | } 16 | 17 | func fanIn(input1, input2 <-chan string) <-chan string { 18 | c := make(chan string) 19 | go func() { for { c <- <-input1 } }() 20 | go func() { for { c <- <-input2 } }() 21 | return c 22 | } 23 | 24 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 25 | c := make(chan string) 26 | go func() { // We launch the goroutine from inside the function. 27 | for i := 0; ; i++ { 28 | c <- fmt.Sprintf("%s %d", msg, i) 29 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 30 | } 31 | }() 32 | return c // Return the channel to the caller. 33 | } -------------------------------------------------------------------------------- /boring09.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | type Message struct { 10 | str string 11 | wait chan bool 12 | } 13 | 14 | func main(){ 15 | c := fanIn(boring("Joe"), boring("Ann")) 16 | for i := 0; i < 5; i++ { 17 | msg1 := <-c; fmt.Println(msg1.str) 18 | msg2 := <-c; fmt.Println(msg2.str) 19 | msg1.wait <- true 20 | msg2.wait <- true 21 | } 22 | fmt.Println("You're both boring; I'm leaving.") 23 | } 24 | 25 | func fanIn(input1, input2 <-chan Message) <-chan Message { 26 | c := make(chan Message) 27 | go func() { for { c <- <-input1 } }() 28 | go func() { for { c <- <-input2 } }() 29 | return c 30 | } 31 | 32 | func boring(msg string) <-chan Message{ // Returns receive-only channel of strings. 33 | waitForIt := make(chan bool) 34 | c := make(chan Message) 35 | go func() { // We launch the goroutine from inside the function. 36 | for i := 0; ; i++ { 37 | c <- Message{ fmt.Sprintf("%s %d", msg, i), waitForIt } 38 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 39 | <-waitForIt 40 | } 41 | }() 42 | return c // Return the channel to the caller. 43 | } -------------------------------------------------------------------------------- /boring10.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | c := fanIn(boring("Joe"), boring("Ann")) 11 | for i := 0; i < 10; i++ { 12 | fmt.Println(<-c) 13 | } 14 | fmt.Println("You're both boring; I'm leaving.") 15 | } 16 | 17 | func fanIn(input1, input2 <-chan string) <-chan string { 18 | c := make(chan string) 19 | go func() { 20 | for { 21 | select { 22 | case s := <-input1: c <- s 23 | case s := <-input2: c <- s 24 | } 25 | } 26 | }() 27 | return c 28 | } 29 | 30 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 31 | c := make(chan string) 32 | go func() { // We launch the goroutine from inside the function. 33 | for i := 0; ; i++ { 34 | c <- fmt.Sprintf("%s %d", msg, i) 35 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 36 | } 37 | }() 38 | return c // Return the channel to the caller. 39 | } -------------------------------------------------------------------------------- /boring11.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | c := boring("Joe") 11 | for { 12 | select { 13 | case s:= <-c: 14 | fmt.Println(s) 15 | case <-time.After(1 * time.Second): 16 | fmt.Println("You're too slow.") 17 | return 18 | } 19 | 20 | } 21 | fmt.Println("You're both boring; I'm leaving.") 22 | } 23 | 24 | func fanIn(input1, input2 <-chan string) <-chan string { 25 | c := make(chan string) 26 | go func() { 27 | for { 28 | select { 29 | case s := <-input1: c <- s 30 | case s := <-input2: c <- s 31 | } 32 | } 33 | }() 34 | return c 35 | } 36 | 37 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 38 | c := make(chan string) 39 | go func() { // We launch the goroutine from inside the function. 40 | for i := 0; ; i++ { 41 | c <- fmt.Sprintf("%s %d", msg, i) 42 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 43 | } 44 | }() 45 | return c // Return the channel to the caller. 46 | } -------------------------------------------------------------------------------- /boring12.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | c := boring("Joe") 11 | timeout := time.After(5 * time.Second) 12 | for { 13 | select { 14 | case s:= <-c: 15 | fmt.Println(s) 16 | case <-timeout: 17 | fmt.Println("You're talk too much.") 18 | return 19 | } 20 | 21 | } 22 | fmt.Println("You're both boring; I'm leaving.") 23 | } 24 | 25 | func fanIn(input1, input2 <-chan string) <-chan string { 26 | c := make(chan string) 27 | go func() { 28 | for { 29 | select { 30 | case s := <-input1: c <- s 31 | case s := <-input2: c <- s 32 | } 33 | } 34 | }() 35 | return c 36 | } 37 | 38 | func boring(msg string) <-chan string{ // Returns receive-only channel of strings. 39 | c := make(chan string) 40 | go func() { // We launch the goroutine from inside the function. 41 | for i := 0; ; i++ { 42 | c <- fmt.Sprintf("%s %d", msg, i) 43 | time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 44 | } 45 | }() 46 | return c // Return the channel to the caller. 47 | } -------------------------------------------------------------------------------- /boring13.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | // "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | quit := make(chan bool) 11 | c := boring("Joe", quit) 12 | for i := rand.Intn(20); i >= 0; i-- { fmt.Println(<-c) } 13 | quit <- true 14 | fmt.Println("You're both boring; I'm leaving.") 15 | } 16 | 17 | func fanIn(input1, input2 <-chan string) <-chan string { 18 | c := make(chan string) 19 | go func() { 20 | for { 21 | select { 22 | case s := <-input1: c <- s 23 | case s := <-input2: c <- s 24 | } 25 | } 26 | }() 27 | return c 28 | } 29 | 30 | func boring(msg string, quit chan bool) <-chan string{ // Returns receive-only channel of strings. 31 | c := make(chan string) 32 | go func() { // We launch the goroutine from inside the function. 33 | for i := 0; ; i++ { 34 | select { 35 | case c <- fmt.Sprintf("%s %d", msg, i): 36 | // do nothing 37 | case <-quit: 38 | return 39 | } 40 | 41 | //time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 42 | } 43 | }() 44 | return c // Return the channel to the caller. 45 | } -------------------------------------------------------------------------------- /boring14.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | // "time" 6 | "math/rand" 7 | ) 8 | 9 | func main(){ 10 | quit := make(chan string) 11 | c := boring("Joe", quit) 12 | for i := rand.Intn(20); i >= 0; i-- { fmt.Println(<-c) } 13 | quit <- "Bye!" 14 | fmt.Printf("Joe says: %q\n", <-quit) 15 | } 16 | 17 | func fanIn(input1, input2 <-chan string) <-chan string { 18 | c := make(chan string) 19 | go func() { 20 | for { 21 | select { 22 | case s := <-input1: c <- s 23 | case s := <-input2: c <- s 24 | } 25 | } 26 | }() 27 | return c 28 | } 29 | 30 | func boring(msg string, quit chan string) <-chan string{ // Returns receive-only channel of strings. 31 | c := make(chan string) 32 | go func() { // We launch the goroutine from inside the function. 33 | for i := 0; ; i++ { 34 | select { 35 | case c <- fmt.Sprintf("%s %d", msg, i): 36 | // do nothing 37 | case <-quit: 38 | cleanup() 39 | quit <- "See you!" 40 | return 41 | } 42 | 43 | //time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond) 44 | } 45 | }() 46 | return c // Return the channel to the caller. 47 | } 48 | 49 | func cleanup() {} -------------------------------------------------------------------------------- /daisyChain.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func f(left, right chan int) { 8 | left <- 1 + <-right 9 | } 10 | 11 | func main() { 12 | const n = 10000 13 | leftmost := make(chan int) 14 | right := leftmost 15 | left := leftmost 16 | for i := 0; i < n; i++ { 17 | right = make(chan int) 18 | go f(left, right) 19 | left = right 20 | } 21 | go func(c chan int) { c <- 1}(right) 22 | fmt.Println(<-leftmost) 23 | } 24 | -------------------------------------------------------------------------------- /googleSearch01.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | var ( 10 | Web = fakeSearch("web") 11 | Image = fakeSearch("image") 12 | Video = fakeSearch("video") 13 | ) 14 | 15 | type Result string 16 | 17 | type Search func(query string) Result 18 | 19 | func fakeSearch(kind string) Search { 20 | return func (query string) Result { 21 | time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) 22 | return Result(fmt.Sprintf("%s result for %q\n", kind, query)) 23 | } 24 | } 25 | 26 | func Google(query string) []Result{ 27 | results := make([]Result, 3, 10) 28 | results = append(results, Web(query)) 29 | results = append(results, Image(query)) 30 | results = append(results, Video(query)) 31 | return results 32 | } 33 | 34 | func main() { 35 | rand.Seed(time.Now().UnixNano()) 36 | start := time.Now() 37 | results := Google("golang") 38 | elapsed := time.Since(start) 39 | fmt.Println(results) 40 | fmt.Println(elapsed) 41 | } 42 | -------------------------------------------------------------------------------- /googleSearch02.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | var ( 10 | Web = fakeSearch("web") 11 | Image = fakeSearch("image") 12 | Video = fakeSearch("video") 13 | ) 14 | 15 | type Result string 16 | 17 | type Search func(query string) Result 18 | 19 | func fakeSearch(kind string) Search { 20 | return func (query string) Result { 21 | time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) 22 | return Result(fmt.Sprintf("%s result for %q\n", kind, query)) 23 | } 24 | } 25 | 26 | func Google(query string) (results []Result) { 27 | c := make(chan Result) 28 | go func() { c <- Web(query) } () 29 | go func() { c <- Image(query) } () 30 | go func() { c <- Video(query) } () 31 | 32 | for i :=0; i < 3; i++ { 33 | result := <-c 34 | results = append(results, result) 35 | } 36 | return 37 | } 38 | 39 | func main() { 40 | rand.Seed(time.Now().UnixNano()) 41 | start := time.Now() 42 | results := Google("golang") 43 | elapsed := time.Since(start) 44 | fmt.Println(results) 45 | fmt.Println(elapsed) 46 | } 47 | -------------------------------------------------------------------------------- /googleSearch03.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | var ( 10 | Web = fakeSearch("web") 11 | Image = fakeSearch("image") 12 | Video = fakeSearch("video") 13 | ) 14 | 15 | type Result string 16 | 17 | type Search func(query string) Result 18 | 19 | func fakeSearch(kind string) Search { 20 | return func (query string) Result { 21 | time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) 22 | return Result(fmt.Sprintf("%s result for %q\n", kind, query)) 23 | } 24 | } 25 | 26 | func Google(query string) (results []Result) { 27 | c := make(chan Result) 28 | go func() { c <- Web(query) } () 29 | go func() { c <- Image(query) } () 30 | go func() { c <- Video(query) } () 31 | 32 | timeout := time.After(80 * time.Millisecond) 33 | for i :=0; i < 3; i++ { 34 | select { 35 | case result := <-c: 36 | results = append(results, result) 37 | case <-timeout: 38 | fmt.Println("timed out") 39 | return 40 | } 41 | 42 | } 43 | return 44 | } 45 | 46 | func main() { 47 | rand.Seed(time.Now().UnixNano()) 48 | start := time.Now() 49 | results := Google("golang") 50 | elapsed := time.Since(start) 51 | fmt.Println(results) 52 | fmt.Println(elapsed) 53 | } 54 | -------------------------------------------------------------------------------- /googleSearch04.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | var ( 10 | Web = fakeSearch("web") 11 | Image = fakeSearch("image") 12 | Video = fakeSearch("video") 13 | ) 14 | 15 | type Result string 16 | 17 | type Search func(query string) Result 18 | 19 | func fakeSearch(kind string) Search { 20 | return func (query string) Result { 21 | time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) 22 | return Result(fmt.Sprintf("%s result for %q\n", kind, query)) 23 | } 24 | } 25 | 26 | func Google(query string) (results []Result) { 27 | c := make(chan Result) 28 | go func() { c <- Web(query) } () 29 | go func() { c <- Image(query) } () 30 | go func() { c <- Video(query) } () 31 | 32 | timeout := time.After(80 * time.Millisecond) 33 | for i :=0; i < 3; i++ { 34 | select { 35 | case result := <-c: 36 | results = append(results, result) 37 | case <-timeout: 38 | fmt.Println("timed out") 39 | return 40 | } 41 | 42 | } 43 | return 44 | } 45 | 46 | func First(query string, replicas ...Search) Result { 47 | c := make(chan Result) 48 | searchReplica := func(i int) { c <- replicas[i](query) } 49 | for i := range replicas { 50 | go searchReplica(i) 51 | } 52 | return <-c 53 | } 54 | 55 | func main() { 56 | rand.Seed(time.Now().UnixNano()) 57 | start := time.Now() 58 | result := First("golang", fakeSearch("replica 1"), 59 | fakeSearch("replica 2")) 60 | elapsed := time.Since(start) 61 | fmt.Println(result) 62 | fmt.Println(elapsed) 63 | } 64 | -------------------------------------------------------------------------------- /googleSearch05.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | "math/rand" 7 | ) 8 | 9 | var ( 10 | Web1 = fakeSearch("web1") 11 | Web2 = fakeSearch("web2") 12 | Image1 = fakeSearch("image1") 13 | Image2 = fakeSearch("image2") 14 | Video1 = fakeSearch("video1") 15 | Video2 = fakeSearch("video2") 16 | ) 17 | 18 | type Result string 19 | 20 | type Search func(query string) Result 21 | 22 | func fakeSearch(kind string) Search { 23 | return func (query string) Result { 24 | time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) 25 | return Result(fmt.Sprintf("%s result for %q\n", kind, query)) 26 | } 27 | } 28 | 29 | func Google(query string) (results []Result) { 30 | c := make(chan Result) 31 | go func() { c <- First(query, Web1, Web2) } () 32 | go func() { c <- First(query, Image1, Image2) } () 33 | go func() { c <- First(query, Video1, Video2) } () 34 | 35 | timeout := time.After(80 * time.Millisecond) 36 | for i :=0; i < 3; i++ { 37 | select { 38 | case result := <-c: 39 | results = append(results, result) 40 | case <-timeout: 41 | fmt.Println("timed out") 42 | return 43 | } 44 | 45 | } 46 | return 47 | } 48 | 49 | func First(query string, replicas ...Search) Result { 50 | c := make(chan Result) 51 | searchReplica := func(i int) { c <- replicas[i](query) } 52 | for i := range replicas { 53 | go searchReplica(i) 54 | } 55 | return <-c 56 | } 57 | 58 | func main() { 59 | rand.Seed(time.Now().UnixNano()) 60 | start := time.Now() 61 | results := Google("golang") 62 | elapsed := time.Since(start) 63 | fmt.Println(results) 64 | fmt.Println(elapsed) 65 | } 66 | -------------------------------------------------------------------------------- /imgs/chinese_whispers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lintide/GoConcurrencyPatterns/37781be9f4a087dd3e44077dbbfef0e6656b49e8/imgs/chinese_whispers.png -------------------------------------------------------------------------------- /imgs/fan_in.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lintide/GoConcurrencyPatterns/37781be9f4a087dd3e44077dbbfef0e6656b49e8/imgs/fan_in.png --------------------------------------------------------------------------------