├── .github ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md └── FUNDING.yml ├── .gitignore ├── .hound.yml ├── .travis.yml ├── LICENSE.md ├── README.md ├── benchmark_test.go ├── doc.go ├── example_test.go ├── go.mod ├── pool.go └── pool_test.go /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. 6 | 7 | ## Our Standards 8 | 9 | Examples of behavior that contributes to creating a positive environment include: 10 | 11 | * Using welcoming and inclusive language 12 | * Being respectful of differing viewpoints and experiences 13 | * Gracefully accepting constructive criticism 14 | * Focusing on what is best for the community 15 | * Showing empathy towards other community members 16 | 17 | Examples of unacceptable behavior by participants include: 18 | 19 | * The use of sexualized language or imagery and unwelcome sexual attention or advances 20 | * Trolling, insulting/derogatory comments, and personal or political attacks 21 | * Public or private harassment 22 | * Publishing others' private information, such as a physical or electronic address, without explicit permission 23 | * Other conduct which could reasonably be considered inappropriate in a professional setting 24 | 25 | ## Our Responsibilities 26 | 27 | Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. 28 | 29 | Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. 30 | 31 | ## Scope 32 | 33 | This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. 34 | 35 | ## Enforcement 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at vardius@gmail.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. 38 | 39 | Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. 40 | 41 | ## Attribution 42 | 43 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] 44 | 45 | [homepage]: http://contributor-covenant.org 46 | [version]: http://contributor-covenant.org/version/1/4/ 47 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | We are open to, and grateful for, any contributions made by the community. By contributing to **worker-pool**, you agree to abide by the [code of conduct](https://github.com/vardius/worker-pool/blob/master/CODE_OF_CONDUCT.md). 4 | 5 | ## Reporting Issues and Asking Questions 6 | 7 | Before opening an issue, please search the [issue tracker](https://github.com/vardius/worker-pool/issues) to make sure your issue hasn't already been reported. 8 | 9 | ### Bugs and Improvements 10 | 11 | We use the issue tracker to keep track of bugs and improvements to **worker-pool** itself, its examples, and the documentation. We encourage you to open issues to discuss improvements, architecture, theory, internal implementation, etc. If a topic has been discussed before, we will ask you to join the previous discussion. 12 | 13 | Any pull requests that involve breaking changes should be made against the `next` branch. 14 | 15 | ### Getting Help 16 | 17 | For support or usage questions like “how do I do X with **worker-pool**” and “my code doesn't work”, we encourage you to post an issue. 18 | 19 | Please be considerate when doing this as this is not the primary purpose of the issue tracker. 20 | 21 | ## Development 22 | 23 | Visit the [issue tracker](https://github.com/vardius/worker-pool/issues) to find a list of open issues that need attention. 24 | 25 | Fork, then clone the repo: 26 | 27 | ``` 28 | git clone https://github.com/your-username/worker-pool.git 29 | ``` 30 | 31 | ### Docs 32 | 33 | Improvements to the documentation are always welcome. In the docs we abide by typographic rules, so instead of ' you should use '. Same goes for “ ” and dashes (—) where appropriate. These rules only apply to the text, not to code blocks. 34 | 35 | ### Sending a Pull Request 36 | 37 | For non-trivial changes, please open an issue with a proposal for a new feature or refactoring before starting on the work. We don't want you to waste your efforts on a pull request that we won't want to accept. 38 | 39 | On the other hand, sometimes the best way to start a conversation *is* to send a pull request. Use your best judgement! 40 | 41 | In general, the contribution workflow looks like this: 42 | 43 | * Open a new issue in the [Issue tracker](https://github.com/vardius/worker-pool/issues). 44 | * Fork the repo. 45 | * Create a new feature branch based off the `master` branch. 46 | * Make sure all tests pass and there are no linting errors. 47 | * Submit a pull request, referencing any issues it addresses. 48 | 49 | Please try to keep your pull request focused in scope and avoid including unrelated commits. 50 | 51 | After you have submitted your pull request, we'll try to get back to you as soon as possible. We may suggest some changes or improvements. 52 | 53 | Thank you for contributing! 54 | -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: [vardius] # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] 4 | patreon: # Replace with a single Patreon username 5 | open_collective: # Replace with a single Open Collective username 6 | ko_fi: # Replace with a single Ko-fi username 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry 9 | liberapay: # Replace with a single Liberapay username 10 | issuehunt: # Replace with a single IssueHunt username 11 | otechie: # Replace with a single Otechie username 12 | custom: # Replace with a single custom sponsorship URL 13 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Binaries for programs and plugins 2 | *.exe 3 | *.dll 4 | *.so 5 | *.dylib 6 | 7 | # Test binary, build with `go test -c` 8 | *.test 9 | 10 | # Output of the go coverage tool, specifically when used with LiteIDE 11 | *.out 12 | 13 | # Project-local glide cache, RE: https://github.com/Masterminds/glide/issues/736 14 | .glide/ 15 | -------------------------------------------------------------------------------- /.hound.yml: -------------------------------------------------------------------------------- 1 | go: 2 | enabled: true 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: go 2 | go: 3 | - "1.12" 4 | - tip 5 | script: 6 | - go build 7 | - go test -v -race -cover -coverprofile=coverage.txt -covermode=atomic 8 | - go test -bench=. -cpu=4 -benchmem 9 | after_script: 10 | - bash <(curl -s https://codecov.io/bash) 11 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2017-present Rafał Lorenz 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 👨‍🔧 worker-pool 2 | ================ 3 | [![Build Status](https://travis-ci.org/vardius/worker-pool.svg?branch=master)](https://travis-ci.org/vardius/worker-pool) 4 | [![Go Report Card](https://goreportcard.com/badge/github.com/vardius/worker-pool)](https://goreportcard.com/report/github.com/vardius/worker-pool) 5 | [![codecov](https://codecov.io/gh/vardius/worker-pool/branch/master/graph/badge.svg)](https://codecov.io/gh/vardius/worker-pool) 6 | [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fvardius%2Fworker-pool.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fvardius%2Fworker-pool?ref=badge_shield) 7 | [![](https://godoc.org/github.com/vardius/worker-pool?status.svg)](https://pkg.go.dev/github.com/vardius/worker-pool) 8 | [![license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://github.com/vardius/worker-pool/blob/master/LICENSE.md) 9 | 10 | logo 11 | 12 | Go simple async worker pool. 13 | 14 | 📖 ABOUT 15 | ================================================== 16 | 17 | Worker pool is a software design pattern for achieving concurrency of task execution. Maintains multiple workers waiting for tasks to be allocated for concurrent execution. By maintaining a pool of workers, the model increases performance and avoids latency in execution. The number of available workers might be tuned to the computing resources available. 18 | 19 | 20 | 21 | You can read more about worker pools in Go [here](https://brandur.org/go-worker-pool). 22 | 23 | Contributors: 24 | 25 | * [Rafał Lorenz](http://rafallorenz.com) 26 | 27 | Want to contribute ? Feel free to send pull requests! 28 | 29 | Have problems, bugs, feature ideas? 30 | We are using the github [issue tracker](https://github.com/vardius/worker-pool/issues) to manage them. 31 | 32 | ## 📚 Documentation 33 | 34 | For __examples__ **visit [godoc#pkg-examples](http://godoc.org/github.com/vardius/worker-pool#pkg-examples)** 35 | 36 | For **GoDoc** reference, **visit [pkg.go.dev](https://pkg.go.dev/github.com/vardius/worker-pool)** 37 | 38 | 🚏 HOW TO USE 39 | ================================================== 40 | 41 | ## 🚅 Benchmark 42 | **CPU: 3,3 GHz Intel Core i7** 43 | 44 | **RAM: 16 GB 2133 MHz LPDDR3** 45 | 46 | ```bash 47 | ➜ worker-pool git:(master) ✗ go test -bench=. -cpu=4 -benchmem 48 | goos: darwin 49 | goarch: amd64 50 | pkg: github.com/vardius/worker-pool/v2 51 | BenchmarkWorker1-4 3944299 284 ns/op 56 B/op 3 allocs/op 52 | BenchmarkWorker1Parallel-4 7394715 138 ns/op 48 B/op 2 allocs/op 53 | BenchmarkWorker100-4 1657569 693 ns/op 56 B/op 3 allocs/op 54 | BenchmarkWorker100Parallel-4 3673483 368 ns/op 48 B/op 2 allocs/op 55 | BenchmarkWorkerNumCPU-4 2590293 445 ns/op 56 B/op 3 allocs/op 56 | BenchmarkWorkerNumCPUParallel-4 3591553 298 ns/op 48 B/op 2 allocs/op 57 | PASS 58 | ok github.com/vardius/worker-pool/v2 9.511s 59 | ``` 60 | 61 | ## 🏫 Basic example 62 | ```go 63 | package main 64 | 65 | import ( 66 | "fmt" 67 | "sync" 68 | 69 | "github.com/vardius/worker-pool/v2" 70 | ) 71 | 72 | func main() { 73 | var wg sync.WaitGroup 74 | 75 | poolSize := 1 76 | jobsAmount := 3 77 | workersAmount := 2 78 | 79 | // create new pool 80 | pool := workerpool.New(poolSize) 81 | out := make(chan int, jobsAmount) 82 | worker := func(i int) { 83 | defer wg.Done() 84 | out <- i 85 | } 86 | 87 | for i := 1; i <= workersAmount; i++ { 88 | if err := pool.AddWorker(worker); err != nil { 89 | panic(err) 90 | } 91 | } 92 | 93 | wg.Add(jobsAmount) 94 | 95 | for i := 0; i < jobsAmount; i++ { 96 | if err := pool.Delegate(i); err != nil { 97 | panic(err) 98 | } 99 | } 100 | 101 | go func() { 102 | // stop all workers after jobs are done 103 | wg.Wait() 104 | close(out) 105 | pool.Stop() // stop removes all workers from pool, to resume work add them again 106 | }() 107 | 108 | sum := 0 109 | for n := range out { 110 | sum += n 111 | } 112 | 113 | fmt.Println(sum) 114 | // Output: 115 | // 3 116 | } 117 | ``` 118 | 119 | 📜 [License](LICENSE.md) 120 | ------- 121 | 122 | This package is released under the MIT license. See the complete license in the package 123 | 124 | [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fvardius%2Fworker-pool.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fvardius%2Fworker-pool?ref=badge_large) 125 | -------------------------------------------------------------------------------- /benchmark_test.go: -------------------------------------------------------------------------------- 1 | package workerpool 2 | 3 | import ( 4 | "runtime" 5 | "testing" 6 | ) 7 | 8 | func poolDelegate(b *testing.B, pool Pool) { 9 | for n := 0; n < b.N; n++ { 10 | if err := pool.Delegate(n); err != nil { 11 | b.Fatal(err) 12 | } 13 | } 14 | } 15 | 16 | func poolDelegateParallel(b *testing.B, pool Pool) { 17 | b.RunParallel(func(pb *testing.PB) { 18 | for pb.Next() { 19 | if err := pool.Delegate(1); err != nil { 20 | b.Fatal(err) 21 | } 22 | } 23 | }) 24 | } 25 | 26 | func runBenchmark(b *testing.B, workersAmount int, runInParallel bool) { 27 | pool := New(b.N) 28 | defer pool.Stop() 29 | 30 | worker := func(i int) {} 31 | 32 | for i := 1; i <= workersAmount; i++ { 33 | if err := pool.AddWorker(worker); err != nil { 34 | b.Fatal(err) 35 | } 36 | } 37 | 38 | b.ResetTimer() 39 | 40 | if runInParallel { 41 | poolDelegateParallel(b, pool) 42 | } else { 43 | poolDelegate(b, pool) 44 | } 45 | } 46 | 47 | func BenchmarkWorker1(b *testing.B) { 48 | runBenchmark(b, 1, false) 49 | } 50 | 51 | func BenchmarkWorker1Parallel(b *testing.B) { 52 | runBenchmark(b, 1, true) 53 | } 54 | 55 | func BenchmarkWorker100(b *testing.B) { 56 | runBenchmark(b, 100, false) 57 | } 58 | 59 | func BenchmarkWorker100Parallel(b *testing.B) { 60 | runBenchmark(b, 100, true) 61 | } 62 | 63 | func BenchmarkWorkerNumCPU(b *testing.B) { 64 | runBenchmark(b, runtime.NumCPU()+1, false) 65 | } 66 | 67 | func BenchmarkWorkerNumCPUParallel(b *testing.B) { 68 | runBenchmark(b, runtime.NumCPU()+1, true) 69 | } 70 | -------------------------------------------------------------------------------- /doc.go: -------------------------------------------------------------------------------- 1 | /* 2 | Package workerpool provides simple async workers 3 | */ 4 | package workerpool 5 | -------------------------------------------------------------------------------- /example_test.go: -------------------------------------------------------------------------------- 1 | package workerpool_test 2 | 3 | import ( 4 | "fmt" 5 | "strconv" 6 | "sync" 7 | 8 | workerpool "github.com/vardius/worker-pool/v2" 9 | ) 10 | 11 | func Example() { 12 | var wg sync.WaitGroup 13 | 14 | poolSize := 1 15 | jobsAmount := 3 16 | workersAmount := 2 17 | 18 | // create new pool 19 | pool := workerpool.New(poolSize) 20 | out := make(chan int, jobsAmount) 21 | worker := func(i int) { 22 | defer wg.Done() 23 | out <- i 24 | } 25 | 26 | for i := 1; i <= workersAmount; i++ { 27 | if err := pool.AddWorker(worker); err != nil { 28 | panic(err) 29 | } 30 | } 31 | 32 | wg.Add(jobsAmount) 33 | 34 | for i := 0; i < jobsAmount; i++ { 35 | if err := pool.Delegate(i); err != nil { 36 | panic(err) 37 | } 38 | } 39 | 40 | go func() { 41 | // stop all workers after jobs are done 42 | wg.Wait() 43 | close(out) 44 | pool.Stop() 45 | }() 46 | 47 | sum := 0 48 | for n := range out { 49 | sum += n 50 | } 51 | 52 | fmt.Println(sum) 53 | // Output: 54 | // 3 55 | } 56 | 57 | func Example_second() { 58 | poolSize := 2 59 | jobsAmount := 8 60 | workersAmount := 3 61 | 62 | ch := make(chan int, jobsAmount) 63 | defer close(ch) 64 | 65 | // create new pool 66 | pool := workerpool.New(poolSize) 67 | defer pool.Stop() 68 | 69 | worker := func(i int, out chan<- int) { out <- i } 70 | 71 | for i := 1; i <= workersAmount; i++ { 72 | if err := pool.AddWorker(worker); err != nil { 73 | panic(err) 74 | } 75 | } 76 | 77 | go func() { 78 | for n := 0; n < jobsAmount; n++ { 79 | if err := pool.Delegate(n, ch); err != nil { 80 | panic(err) 81 | } 82 | } 83 | }() 84 | 85 | var sum = 0 86 | for sum < jobsAmount { 87 | select { 88 | case <-ch: 89 | sum++ 90 | } 91 | } 92 | 93 | fmt.Println(sum) 94 | // Output: 95 | // 8 96 | } 97 | 98 | func Example_third() { 99 | poolSize := 2 100 | jobsAmount := 8 101 | workersAmount := 3 102 | 103 | var wg sync.WaitGroup 104 | wg.Add(jobsAmount) 105 | 106 | // allocate queue 107 | pool := workerpool.New(poolSize) 108 | worker := func(s string) { 109 | defer wg.Done() 110 | defer fmt.Println("job " + s + " is done !") 111 | fmt.Println("job " + s + " is running ..") 112 | } 113 | 114 | // mock arg 115 | argx := make([]string, jobsAmount) 116 | for j := 0; j < jobsAmount; j++ { 117 | argx[j] = "_" + strconv.Itoa(j) + "_" 118 | } 119 | 120 | // start workers 121 | for i := 1; i <= workersAmount; i++ { 122 | if err := pool.AddWorker(worker); err != nil { 123 | panic(err) 124 | } 125 | } 126 | 127 | // assign jobs 128 | for i := 0; i < jobsAmount; i++ { 129 | go func(i int) { 130 | if err := pool.Delegate(argx[i]); err != nil { 131 | panic(err) 132 | } 133 | }(i) 134 | } 135 | 136 | // clean up 137 | wg.Wait() 138 | pool.Stop() 139 | 140 | // Unordered output: 141 | // job _0_ is running .. 142 | // job _0_ is done ! 143 | // job _1_ is running .. 144 | // job _1_ is done ! 145 | // job _2_ is running .. 146 | // job _2_ is done ! 147 | // job _3_ is running .. 148 | // job _3_ is done ! 149 | // job _4_ is running .. 150 | // job _4_ is done ! 151 | // job _5_ is running .. 152 | // job _5_ is done ! 153 | // job _6_ is running .. 154 | // job _6_ is done ! 155 | // job _7_ is running .. 156 | // job _7_ is done ! 157 | } 158 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/vardius/worker-pool/v2 2 | 3 | go 1.13 4 | -------------------------------------------------------------------------------- /pool.go: -------------------------------------------------------------------------------- 1 | package workerpool 2 | 3 | import ( 4 | "fmt" 5 | "reflect" 6 | "sync" 7 | ) 8 | 9 | // Pool implements worker pool 10 | type Pool interface { 11 | // Delegate job to a workers 12 | // will block if channel is full, you might want to wrap it with goroutine to avoid it 13 | // will panic if called after Stop() 14 | Delegate(args ...interface{}) error 15 | 16 | // AddWorker adds worker to the pool 17 | AddWorker(fn interface{}) error 18 | // RemoveWorker removes worker from the pool 19 | RemoveWorker(fn interface{}) error 20 | 21 | // WorkersNum returns number of workers in the pool 22 | WorkersNum() int 23 | 24 | // Stop removes all workers workers 25 | // to resume work add them again 26 | Stop() 27 | } 28 | 29 | type quitCh chan struct{} 30 | type workers map[reflect.Value][]quitCh 31 | 32 | type pool struct { 33 | queue chan []reflect.Value 34 | workers workers 35 | mtx sync.RWMutex 36 | } 37 | 38 | func (p *pool) Delegate(args ...interface{}) error { 39 | if len(p.workers) == 0 { 40 | return fmt.Errorf("there is no workers in pool") 41 | } 42 | 43 | p.queue <- buildQueueValue(args) 44 | 45 | return nil 46 | } 47 | 48 | func (p *pool) AddWorker(fn interface{}) error { 49 | if err := isValidHandler(fn); err != nil { 50 | return err 51 | } 52 | 53 | worker := reflect.ValueOf(fn) 54 | 55 | p.mtx.Lock() 56 | defer p.mtx.Unlock() 57 | 58 | q := make(quitCh) 59 | 60 | if _, ok := p.workers[worker]; !ok { 61 | p.workers[worker] = []quitCh{q} 62 | } else { 63 | p.workers[worker] = append(p.workers[worker], q) 64 | } 65 | 66 | go func() { 67 | for { 68 | select { 69 | case args := <-p.queue: 70 | worker.Call(args) 71 | case <-q: 72 | return 73 | } 74 | } 75 | }() 76 | 77 | return nil 78 | } 79 | 80 | func (p *pool) RemoveWorker(fn interface{}) error { 81 | if err := isValidHandler(fn); err != nil { 82 | return err 83 | } 84 | 85 | worker := reflect.ValueOf(fn) 86 | 87 | p.mtx.Lock() 88 | defer p.mtx.Unlock() 89 | 90 | if len(p.workers[worker]) > 0 { 91 | close(p.workers[worker][len(p.workers[worker])-1]) 92 | 93 | p.workers[worker] = p.workers[worker][:len(p.workers[worker])-1] 94 | } else { 95 | delete(p.workers, worker) 96 | } 97 | 98 | return nil 99 | } 100 | 101 | func (p *pool) WorkersNum() int { 102 | sum := 0 103 | for _, qChs := range p.workers { 104 | sum += len(qChs) 105 | } 106 | 107 | return sum 108 | } 109 | 110 | func (p *pool) Stop() { 111 | for _, qChs := range p.workers { 112 | for _, ch := range qChs { 113 | close(ch) 114 | } 115 | } 116 | } 117 | 118 | func isValidHandler(fn interface{}) error { 119 | if reflect.TypeOf(fn).Kind() != reflect.Func { 120 | return fmt.Errorf("%s is not a reflect.Func", reflect.TypeOf(fn)) 121 | } 122 | 123 | return nil 124 | } 125 | 126 | func buildQueueValue(args []interface{}) []reflect.Value { 127 | reflectedArgs := make([]reflect.Value, 0) 128 | 129 | for _, arg := range args { 130 | reflectedArgs = append(reflectedArgs, reflect.ValueOf(arg)) 131 | } 132 | 133 | return reflectedArgs 134 | } 135 | 136 | // New creates new worker pool with a given job queue length 137 | func New(queueLength int) Pool { 138 | return &pool{ 139 | queue: make(chan []reflect.Value, queueLength), 140 | workers: make(workers), 141 | } 142 | } 143 | -------------------------------------------------------------------------------- /pool_test.go: -------------------------------------------------------------------------------- 1 | package workerpool 2 | 3 | import ( 4 | "fmt" 5 | "sync" 6 | "testing" 7 | ) 8 | 9 | func TestNew(t *testing.T) { 10 | pool := New(2) 11 | defer pool.Stop() 12 | 13 | if pool == nil { 14 | t.Fail() 15 | } 16 | } 17 | 18 | func TestRemoveWorker(t *testing.T) { 19 | pool := New(2) 20 | defer pool.Stop() 21 | 22 | worker := func(i int) {} 23 | 24 | for i := 1; i <= 2; i++ { 25 | if err := pool.AddWorker(worker); err != nil { 26 | t.Fatal(err) 27 | } 28 | } 29 | 30 | if err := pool.RemoveWorker(worker); err != nil { 31 | t.Fatal(err) 32 | } 33 | 34 | if pool.WorkersNum() != 1 { 35 | t.Fatal("should have one worker left") 36 | } 37 | 38 | if err := pool.Delegate(1); err != nil { 39 | t.Fatal(err) 40 | } 41 | } 42 | 43 | func TestInvalidWorker(t *testing.T) { 44 | pool := New(2) 45 | defer pool.Stop() 46 | 47 | if pool.AddWorker("worker") == nil { 48 | t.Fail() 49 | } 50 | } 51 | 52 | func TestWorkers(t *testing.T) { 53 | delegateWorkToWorkers(t, 2, 3, 3) // same workers as jobs 54 | delegateWorkToWorkers(t, 2, 3, 2) // less workers then jobs 55 | delegateWorkToWorkers(t, 2, 3, 5) // more workers than jobs 56 | } 57 | 58 | func delegateWorkToWorkers(t *testing.T, poolSize int, jobsAmount int, workersAmount int) { 59 | var wg sync.WaitGroup 60 | 61 | pool := New(poolSize) 62 | out := make(chan int, jobsAmount) 63 | worker := func(i int) { 64 | defer wg.Done() 65 | out <- i 66 | } 67 | 68 | for i := 1; i <= workersAmount; i++ { 69 | if err := pool.AddWorker(worker); err != nil { 70 | t.Fatal(err) 71 | } 72 | } 73 | 74 | wg.Add(jobsAmount) 75 | 76 | for i := 1; i <= jobsAmount; i++ { 77 | if err := pool.Delegate(i); err != nil { 78 | t.Fatal(err) 79 | } 80 | } 81 | 82 | go func() { 83 | wg.Wait() 84 | close(out) 85 | pool.Stop() 86 | }() 87 | 88 | sum := 0 89 | for n := range out { 90 | sum += n 91 | } 92 | 93 | if sum == 0 { 94 | fmt.Printf("Delegating job %d to %d workers failed", jobsAmount, workersAmount) 95 | t.Fail() 96 | } 97 | } 98 | --------------------------------------------------------------------------------