├── .gitignore ├── .ruby-version ├── Dockerfile ├── Gemfile ├── Gemfile.lock ├── README.md └── lib ├── auto_client.rb ├── configuration.rb ├── gateway.rb ├── keyboard_client.rb ├── logger.rb ├── receiver.rb └── worker.rb /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled source # 2 | ################### 3 | *.com 4 | *.class 5 | *.dll 6 | *.exe 7 | *.o 8 | *.so 9 | 10 | # Packages # 11 | ############ 12 | # it's better to unpack these files and commit the raw source 13 | # git has its own built in compression methods 14 | *.7z 15 | *.dmg 16 | *.gz 17 | *.iso 18 | *.jar 19 | *.rar 20 | *.tar 21 | *.zip 22 | 23 | # OS generated files # 24 | ###################### 25 | .DS_Store 26 | .DS_Store? 27 | ._* 28 | .Spotlight-V100 29 | .Trashes 30 | Icon? 31 | ehthumbs.db 32 | Thumbs.db 33 | 34 | #Rails specific stuff 35 | config/database.yml 36 | config/env.yml 37 | config/app_config.yml 38 | config/mongoid.yml 39 | 40 | *.log 41 | tmp/ 42 | *.sql 43 | *.sqlite 44 | *.sqlite3 45 | .rvmrc 46 | public/system 47 | public/spree 48 | public/uploads 49 | /public/system* 50 | 51 | .idea/ 52 | .sass-cache/ 53 | .bundle/ 54 | .jhw-cache/ 55 | 56 | ## UNIX TEMP FILES 57 | *~ 58 | 59 | 60 | -------------------------------------------------------------------------------- /.ruby-version: -------------------------------------------------------------------------------- 1 | 2.1.1 2 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM jpamaya/ruby-2.1.1 2 | 3 | # Add project files 4 | ADD . / 5 | 6 | # Bundle 7 | RUN bundle install 8 | 9 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gem 'eventmachine' 4 | gem 'bunny' 5 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | amq-protocol (1.9.2) 5 | bunny (1.6.3) 6 | amq-protocol (>= 1.9.2) 7 | eventmachine (1.0.3) 8 | 9 | PLATFORMS 10 | ruby 11 | 12 | DEPENDENCIES 13 | bunny 14 | eventmachine 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Microservices tests 2 | 3 | This is a repo for microservice tests using rabbitmq. 4 | 5 | ## Instructions 6 | 7 | 1. [Install rabbitmq](https://www.rabbitmq.com/download.html) 8 | 2. Clone this repo 9 | 3. go into the cloned project directory 10 | 11 | ```bash 12 | cd microservices-tests 13 | ``` 14 | 4. do bundle install 15 | 16 | ```bash 17 | bundle install 18 | ``` 19 | 5. cd into lib, where all the exectuables are, from now on, you will execute any commands in different terminals, keeping the processes (microservices) alive. 20 | 21 | ```bash 22 | cd lib 23 | ``` 24 | 6. edit configuration.rb to set the connection url to the rabbitmq server if necessary (just in case you are using non-default values for the ports, user, password, etc.) 25 | 7. run the keyboard client, so you can type messages, hit enter to send. Messages will be 26 | queued in rabbitmq until a consumer appears 27 | 28 | ```bash 29 | ruby keyboard_client.rb 30 | ``` 31 | 8. run the gateway service which takes messages and replays them to the workers and logger through a fanout exchange. 32 | 33 | ```bash 34 | ruby gateway.rb 35 | ``` 36 | 9. run ANY number of workers. In order to control how much time workers perform a task, just type dots "." in the message you send using the keyboard client, example: "Hello!...." will take 4 seconds to process, because it has 4 dots ".". 37 | 38 | ```bash 39 | ruby worker.rb 40 | ``` 41 | 9. Run the receiver, which is the last stop for processed messages. Processed messages are the ones that have any digits in the message body, complying to the pattern /\d/. Processed messages are upcased. 42 | 43 | ```bash 44 | ruby receiver.rb 45 | ``` 46 | 10. Run the logger, which will log the important interactions in the system. The other microservices send messages to the "logger" queue for this. 47 | 48 | ```bash 49 | ruby logger.rb 50 | ``` 51 | 11. Finally, in order to not input messages manually, run: 52 | 53 | ```bash 54 | ruby auto_client.rb 55 | ``` 56 | This will basically send a message to "queue_a" every second so you can see how the overall system works by looking a the various console outputs for each of the microservices. 57 | 58 | 59 | ## Interesting thing you can do now 60 | 61 | 1. Leave the system running and see how it behaves (basically is a message passing game) 62 | 2. Using the keyboard client, put various messages with multiple dots "." to simulate load for the workers. 63 | 3. Turn off services randomly and rerun them, you will see how messages are queued and processed when the microservices become available again. 64 | 4. Enjoy! 65 | 66 | 67 | ## Running each micro-service in docker container 68 | 69 | This steps assume you have docker installed and working in your system. 70 | 71 | ### Run RabbitMQ Server 72 | 73 | 1. Create and image for rabbitmq server: 74 | 75 | ```bash 76 | docker build -t="dockerfile/rabbitmq" github.com/dockerfile/rabbitmq 77 | ``` 78 | 2. Run a container from that image with: 79 | 80 | ```bash 81 | run --name rabbitmq -d -p 5672:5672 -p 15672:15672 dockerfile/rabbitmq 82 | ``` 83 | 84 | ### Run microservices in docker 85 | 86 | 1. Create and image for micro-services (with ruby 2.1.1 and microservices bundle) doing: 87 | 88 | ```bash 89 | docker build -t="microservices/client" . 90 | ``` 91 | 2. Run each microservice in a terminal as in steps 7 to 11 of the **Instructions** part, replacing *microservice.rb* with the corresponding ruby file. 92 | 93 | ```bash 94 | docker run --link rabbitmq:amq -t -i microservices/client ruby lib/microservice.rb 95 | ``` 96 | 97 | -------------------------------------------------------------------------------- /lib/auto_client.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | ###################### 5 | ## Automatic Client ## 6 | ###################### 7 | 8 | require "bunny" 9 | 10 | require 'rubygems' 11 | 12 | # Require configuration 13 | require_relative 'configuration' 14 | 15 | conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 16 | conn.start 17 | 18 | ch = conn.create_channel 19 | q = ch.queue("gateway_queue") 20 | 21 | stamp = 0 22 | 23 | loop do 24 | stamp = stamp + 1 25 | random_string = ('a'..'z').to_a.shuffle[0,3].join 26 | data = "[auto] - %03d - #{random_string}" % stamp 27 | ch.default_exchange.publish(data, :routing_key => q.name) 28 | puts " [Auto] Sent '#{data}'" 29 | sleep 1 30 | end 31 | 32 | conn.close 33 | -------------------------------------------------------------------------------- /lib/configuration.rb: -------------------------------------------------------------------------------- 1 | #################### 2 | ## Configuration ## 3 | #################### 4 | 5 | # Configuration, to be more DRY 6 | # add connection urls here 7 | class Configuration 8 | def self.rabbitmq_url 9 | boxen_url || docker_url || default_url 10 | end 11 | 12 | def self.default_url 13 | "amqp://guest:guest@localhost:5672" 14 | end 15 | 16 | def self.docker_url 17 | amqp_port_address = ENV['AMQ_PORT_5672_TCP_ADDR'] 18 | amqp_port_address ? "amqp://guest:guest@#{amqp_port_address}:5672" : nil 19 | end 20 | 21 | def self.boxen_url 22 | ENV['BOXEN_RABBITMQ_URL'] 23 | end 24 | end 25 | -------------------------------------------------------------------------------- /lib/gateway.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | ############################### 5 | ## Intermediate microservice ## 6 | ############################### 7 | 8 | require "bunny" 9 | 10 | # Require configuration 11 | require_relative 'configuration' 12 | 13 | def send_message(data) 14 | send_conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 15 | send_conn.start 16 | send_ch = send_conn.create_channel 17 | send_ex = send_ch.fanout("gateway_exchange") 18 | send_ex.publish(data) 19 | puts " [x] Mediating '#{data}'" 20 | send_conn.close 21 | end 22 | 23 | conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 24 | conn.start 25 | 26 | ch = conn.create_channel 27 | q = ch.queue("gateway_queue") 28 | 29 | begin 30 | puts " [*] Intermediate step ------" 31 | puts " [*] Waiting for messages..." 32 | puts " [*] To exit press CTRL+C" 33 | q.subscribe(:block => true) do |delivery_info, properties, body| 34 | send_message("Mediated: #{body}") 35 | end 36 | rescue Interrupt => _ 37 | conn.close 38 | 39 | exit(0) 40 | end 41 | -------------------------------------------------------------------------------- /lib/keyboard_client.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | ##################### 5 | ## Keyboard Client ## 6 | ##################### 7 | 8 | require "bunny" 9 | 10 | require 'rubygems' 11 | require 'eventmachine' 12 | 13 | # Require configuration 14 | require_relative 'configuration' 15 | 16 | module MyKeyboardHandler 17 | include EM::Protocols::LineText2 18 | def receive_line data 19 | conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 20 | conn.start 21 | 22 | ch = conn.create_channel 23 | q = ch.queue("gateway_queue") 24 | 25 | ch.default_exchange.publish(data, :routing_key => q.name, :persistent => true) 26 | puts " [x] Keyboard Sent '#{data}'" 27 | 28 | conn.close 29 | end 30 | end 31 | 32 | EM.run { 33 | puts " [*] Keyboard client ------" 34 | puts " [*] Type text and press Enter to send a message" 35 | puts " [*] To exit press CTRL+C" 36 | puts 37 | EM.open_keyboard(MyKeyboardHandler) 38 | } 39 | -------------------------------------------------------------------------------- /lib/logger.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | ######################### 5 | ## Logger microservice ## 6 | ######################### 7 | 8 | require "bunny" 9 | 10 | # Require configuration 11 | require_relative 'configuration' 12 | 13 | conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 14 | conn.start 15 | 16 | ch = conn.create_channel 17 | x = ch.fanout("gateway_exchange") 18 | q = ch.queue("logger_queue") 19 | q.bind(x) 20 | 21 | begin 22 | puts " [*] Logger ------" 23 | puts " [*] Waiting for messages. To exit press CTRL+C" 24 | q.subscribe(:block => true) do |delivery_info, properties, body| 25 | puts " [x] #{body}" 26 | end 27 | rescue Interrupt => _ 28 | conn.close 29 | 30 | exit(0) 31 | end 32 | -------------------------------------------------------------------------------- /lib/receiver.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | #################### 5 | ## Final receiver ## 6 | #################### 7 | 8 | require "bunny" 9 | 10 | # Require configuration 11 | require_relative 'configuration' 12 | 13 | conn = Bunny.new(Configuration.rabbitmq_url, automatically_recover: false) 14 | conn.start 15 | 16 | ch = conn.create_channel 17 | q = ch.queue("receiver_queue") 18 | 19 | begin 20 | puts " [*] Final receiver ------" 21 | puts " [*] Receiving processed messages..." 22 | puts " [*] To exit press CTRL+C" 23 | q.subscribe(:block => true) do |delivery_info, properties, body| 24 | puts " [x] Received '#{body}'" 25 | end 26 | rescue Interrupt => _ 27 | conn.close 28 | 29 | exit(0) 30 | end 31 | -------------------------------------------------------------------------------- /lib/worker.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # encoding: utf-8 3 | 4 | ############################## 5 | ## Message Processor/Worker ## 6 | ############################## 7 | 8 | require "bunny" 9 | 10 | # Require configuration 11 | require_relative 'configuration' 12 | 13 | # A simple patter to use in a simple logic 14 | pattern = /\d/ 15 | 16 | def send_message(data, queue) 17 | send_conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 18 | send_conn.start 19 | send_ch = send_conn.create_channel 20 | send_q = send_ch.queue(queue) 21 | send_ch.default_exchange.publish(data, :routing_key => send_q.name, :persistent => true) 22 | puts " [x] Processed '#{data}'" 23 | send_conn.close 24 | end 25 | 26 | conn = Bunny.new(Configuration.rabbitmq_url, :automatically_recover => false) 27 | conn.start 28 | 29 | ch = conn.create_channel 30 | x = ch.fanout("gateway_exchange") 31 | q = ch.queue("work_queue", :exclusive => true) 32 | q.bind(x) 33 | 34 | begin 35 | puts " [*] Worker ------" 36 | puts " [*] Waiting for messages to process" 37 | puts " [*] To exit press CTRL+C" 38 | q.subscribe(:block => true) do |delivery_info, properties, body| 39 | 40 | # Sleep a determinate amount of time, simulating a long running task 41 | work_time = body.count(".").to_i 42 | 43 | puts " - Intensive task running #{work_time} seconds" 44 | sleep work_time 45 | puts " [OK] done!" 46 | 47 | # Only if contains a number, process it, otherwise discard it 48 | if body.match(pattern) 49 | send_message(body.upcase, "receiver_queue") 50 | send_message("Processed: #{body}", "logger_queue") 51 | else 52 | send_message("Discarded: #{body}", "logger_queue") 53 | end 54 | end 55 | rescue Interrupt => _ 56 | conn.close 57 | 58 | exit(0) 59 | end 60 | --------------------------------------------------------------------------------