├── .github └── workflows │ └── ruby.yml ├── .gitignore ├── Gemfile ├── README.md ├── Rakefile ├── _config.yml ├── bin └── tuktuk ├── lib ├── tuktuk.rb └── tuktuk │ ├── bounce.rb │ ├── cache.rb │ ├── dns.rb │ ├── package.rb │ ├── rails.rb │ ├── tuktuk.rb │ └── version.rb ├── spec └── deliver_spec.rb └── tuktuk.gemspec /.github/workflows/ruby.yml: -------------------------------------------------------------------------------- 1 | name: Ruby 2 | on: [push] 3 | 4 | jobs: 5 | build: 6 | runs-on: ubuntu-latest 7 | 8 | steps: 9 | - uses: actions/checkout@v1 10 | - name: Set up Ruby 2.6 11 | uses: actions/setup-ruby@v1 12 | with: 13 | ruby-version: 2.6.x 14 | - name: Build and test with Rake 15 | run: | 16 | gem install bundler 17 | gem install rspec 18 | bundle install --jobs 4 --retry 3 19 | rspec 20 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | *.gem 3 | test 4 | log 5 | pkg 6 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source :rubygems 2 | 3 | # specified in tuktuk.gemspec 4 | gemspec 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Tuktuk - SMTP client for Ruby 2 | ============================= 3 | 4 | Unlike famous ol' Pony gem (which is friggin' awesome by the way), Tuktuk does not rely on 5 | `sendmail` or a separate SMTP server in order to deliver email. Tuktuk looks up the 6 | MX servers of the destination address and connects directly using Net::SMTP. 7 | This way you don't need to install Exim or Postfix and you can actually handle 8 | response status codes -- like bounces, 5xx -- within your application. 9 | 10 | Plus, it supports DKIM out of the box. 11 | 12 | Delivering mail 13 | --------------- 14 | 15 | ``` ruby 16 | require 'tuktuk' 17 | 18 | message = { 19 | :from => 'you@username.com', 20 | :to => 'user@yoursite.com', 21 | :body => 'Hello there', 22 | :subject => 'Hiya' 23 | } 24 | 25 | response, email = Tuktuk.deliver(message) 26 | ``` 27 | 28 | HTML (multipart) emails are supported, of course. 29 | 30 | ``` ruby 31 | 32 | message = { 33 | :from => 'you@username.com', 34 | :to => 'user@yoursite.com', 35 | :body => 'Hello there', 36 | :html_body => '

Hello there

', 37 | :subject => 'Hiya in colours' 38 | } 39 | 40 | response, email = Tuktuk.deliver(message) 41 | ``` 42 | 43 | The `response` is either a [Net::SMTP::Response](http://ruby-doc.org/stdlib-2.0.0/libdoc/net/smtp/rdoc/Net/SMTP/Response.html) object, or a Bounce exception (HardBounce or SoftBounce, depending on the cause). `email` is a [mail](https://github.com/mikel/mail) object. So, to handle bounces you'd do: 44 | 45 | ``` ruby 46 | [...] 47 | 48 | response, email = Tuktuk.deliver(message) 49 | 50 | if response.is_a?(Tuktuk::Bounce) 51 | puts 'Email bounced. Type: ' + response.class.name # => HardBounce or SoftBounce 52 | else 53 | puts 'Email delivered! Server responded: ' + response.message 54 | end 55 | ``` 56 | 57 | You can also call `Tuktuk.deliver!` (with a trailing `!`), in which case it will automatically raise an exception if the response was either a `HardBounce` or a `SoftBounce`. This is useful when running in the background via Resque or Sidekiq, because it makes you aware of which emails are not getting through, and you can requeue those jobs to have them redelivered. 58 | 59 | Email options 60 | ------------- 61 | 62 | Attachments are supported, as you'd expect. 63 | 64 | ``` rb 65 | message = { 66 | :from => 'john@lennon.com', 67 | :to => 'paul@maccartney.com', 68 | :subject => 'Question for you', 69 | :body => 'How do you sleep?', 70 | :reply_to => '', 71 | :return_path => 'bounces@server.com', 72 | :attachments => [ '/home/john/walrus.png' ] 73 | } 74 | ``` 75 | 76 | Attachments can be either a path to a file or a hash containing the file's name and content, like this: 77 | 78 | ``` rb 79 | message = { 80 | ... 81 | :attachments => [ 82 | { :filename => 'walrus.png', :content => File.read('/home/john/walrus.png') } 83 | ] 84 | } 85 | ``` 86 | 87 | These are the email headers Tuktuk is able to set for you. Just pass them as part of the hash and they'll be automatically set. 88 | 89 | ``` 90 | :return_path => '', # will actually set three headers, Return-Path, Bounces-To and Errors-To 91 | :reply_to => '', 92 | :in_reply_to => '', 93 | :list_unsubscribe => ', ', 94 | :list_archive => '', 95 | :list_id => '' 96 | ``` 97 | 98 | Delivering multiple 99 | ------------------- 100 | 101 | With Tuktuk, you can also deliver multiple messages at once. Depending on the `max_workers` config parameter, Tuktuk will either connect sequentially to the target domain's MX servers, or do it in parallel by spawning threads. 102 | 103 | Tuktuk will try to send all emails targeted for a specific domain on the same SMTP session. If a MX server is not responding -- or times out in the middle --, Tuktuk will try to deliver the remaining messages to next MX server, and so on. 104 | 105 | To #deliver_many, you need to pass an array of messages, and you'll receive an array of [response, email] elements, just as above. 106 | 107 | ``` ruby 108 | messages = [ { ... }, { ... }, { ... }, { ... } ] # array of messages 109 | 110 | result = Tuktuk.deliver_many(messages) 111 | 112 | result.each do |response, email| 113 | 114 | if response.is_a?(Tuktuk::Bounce) 115 | puts 'Email bounced. Type: ' + response.class.name 116 | else 117 | puts 'Email delivered!' 118 | end 119 | 120 | end 121 | ``` 122 | 123 | Options & DKIM 124 | -------------- 125 | 126 | Now, if you want to enable DKIM (and you _should_): 127 | 128 | ``` ruby 129 | require 'tuktuk' 130 | 131 | Tuktuk.options = { 132 | :dkim => { 133 | :domain => 'yoursite.com', 134 | :selector => 'mailer', 135 | :private_key => IO.read('ssl/yoursite.com.key') 136 | } 137 | } 138 | 139 | message = { ... } 140 | 141 | response, email = Tuktuk.deliver(message) 142 | ``` 143 | 144 | For DKIM to work, you need to set up some TXT records in your domain's DNS. You can use [this tool](http://www.socketlabs.com/domainkey-dkim-generation-wizard/) to generate the key. You should also create [SPF records](http://www.spfwizard.net/) if you haven't. Then use [this tool](https://www.mail-tester.com/spf-dkim-check) to verify that they're both correctly in place. 145 | 146 | All available options, with their defaults: 147 | 148 | ``` ruby 149 | Tuktuk.options = { 150 | :log_to => nil, # e.g. log/mailer.log or STDOUT 151 | :helo_domain => nil, # your server's domain goes here 152 | :max_workers => 0, # controls number of threads for delivering_many emails (read below) 153 | :open_timeout => 20, # max seconds to wait for opening a connection 154 | :read_timeout => 20, # 20 seconds to wait for a response, once connected 155 | :verify_ssl => true, # whether to skip SSL keys verification or not 156 | :debug => false, # connects and delivers email to localhost, instead of real target server. CAUTION! 157 | :dkim => { ... } 158 | } 159 | ``` 160 | 161 | You can set the `max_workers` option to `auto`, which will spawn the necessary threads to connect in paralell to all target MX servers when delivering multiple messages. When set to `0`, these batches will be delivered sequentially. 162 | 163 | In other words, if you have three emails targeted to Gmail users and two for Hotmail users, using `auto` Tuktuk will spawn two threads and connect to both servers at once. Using `0` will have your emails delivered to one host, and then the other. 164 | 165 | Using with Rails 166 | ---------------- 167 | 168 | Tuktuk comes with ActionMailer support out of the box. In your environment.rb or environments/{env}.rb: 169 | 170 | ``` ruby 171 | require 'tuktuk/rails' 172 | 173 | [...] 174 | 175 | config.action_mailer.delivery_method = :tuktuk 176 | ``` 177 | 178 | Since Tuktuk delivers email directly to the user's MX servers, it's probably a good idea to set `config.action_mailer.raise_delivery_errors` to true. That way you can actually know if an email couldn't make it to its destination. 179 | 180 | When used with ActionMailer, you can pass options using ActionMailer's interface, like this: 181 | 182 | ``` ruby 183 | 184 | config.action_mailer.delivery_method = :tuktuk 185 | 186 | config.action_mailer.tuktuk_settings = { 187 | :log_to => 'log/mailer.log', # when not set, Tuktuk will use Rails.logger 188 | :dkim => { 189 | :domain => 'yoursite.com', 190 | :selector => 'mailer', 191 | :private_key => IO.read('ssl/yoursite.com.key') 192 | } 193 | } 194 | ``` 195 | 196 | # Example SPK/DKIM/DMARC settings 197 | 198 | If you're sending email from yoursite.com, the SPF record should be set for the APEX/root host, and look like this: 199 | 200 | v=spf1 ip4:[ipv4_address] ip6:[ipv6_address] mx a include:[other_host] ~all 201 | 202 | For example: 203 | 204 | v=spf1 ip4:12.34.56.78 ip6:2600:3c05::f07c:92ff:fe48:b2fd mx a include:mailgun.org ~all 205 | 206 | This tells the receiving server to accept email sent from a) the addresses explicitly mentioned (`ip4` and `ip6`), 207 | b) from the hosts mentioned in the `include` statements, as well as c) the hosts listed as `MX` and `A` records for that domain. 208 | 209 | As for DKIM, you should add two TXT records. The first is a simple, short one that goes under the `_domainkey` host, 210 | and should contain the following: 211 | 212 | t=y;o=~; 213 | 214 | Then, a second DKIM record should be placed under `[selector]._domainkey` (e.g. `mailer._domainkey`), and should look like this: 215 | 216 | k=rsa; p=MIIBIBANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA[...]DAQAB (public key) 217 | 218 | And finally, your DMARC record goes under the `_dmarc` host, and goes like this: 219 | 220 | v=DMARC1; p=none; rua=mailto:postmaster@yoursite.com; ruf=mailto:postmaster@yoursite.com 221 | 222 | So, in summary: 223 | 224 | (SPF) @.yoursite.com --> v=spf1 ip4:[ipv4_address] ip6:[ipv6_address] mx a include:[other_host] ~all 225 | (DKIM1) _domainkey.yoursite.com --> t=y;o=~; 226 | (DKIM2) [selector]._domainkey.yoursite.com --> k=rsa; p=MIIBIBANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA[...]DAQAB 227 | (DMARC) _dmarc.yoursite.com --> v=DMARC1; p=none; rua=mailto:postmaster@yoursite.com; ruf=mailto:postmaster@yoursite.com 228 | 229 | Now, to check wether your records are OK, you can use the `dig` command like follows: 230 | 231 | dig yoursite.com TXT +short # should output the SPF record, under the root domain 232 | dig mailer._domainkey.yoursite.com TXT +short # should output the DKIM record containing the key 233 | dig _domainkey.yoursite.com TXT +short # should output the other (short) DKIM 234 | dig _dmarc.yoursite.com TXT +short # should output the DMARC record 235 | 236 | Remember you can query your DNS server directly with the `dig` command by adding `@name.server.com` 237 | after the `dig` command (e.g. `dig @ns1.linode.com yoursite.com TXT`). 238 | 239 | # Contributions 240 | 241 | You're more than welcome. Send a pull request, including tests, and make sure you don't break anything. That's it. 242 | 243 | # Copyright 244 | 245 | (c) Fork Limited. MIT license. 246 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'bundler' 2 | Bundler::GemHelper.install_tasks 3 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-minimal -------------------------------------------------------------------------------- /bin/tuktuk: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require File.expand_path(File.dirname(__FILE__)) + '/../lib/tuktuk' 4 | require 'optparse' 5 | 6 | def missing_keys?(opts) 7 | [:body, :from, :subject, :to] - opts.keys 8 | end 9 | 10 | options = {} 11 | 12 | OptionParser.new do |opts| 13 | opts.banner = "Usage: tuktuk [options]" 14 | 15 | opts.on("-f", "--from your@email.com", String, "From email") do |val| 16 | options[:from] = val 17 | end 18 | 19 | opts.on("-t", "--to email1,email2", Array, "List of destination emails") do |list| 20 | options[:to] = list 21 | end 22 | 23 | opts.on("-s","--subject 'This is a test.'", String, "Email subject") do |subject| 24 | options[:subject] = subject 25 | end 26 | 27 | opts.on("-b", "--body 'Hello there.'", String, "Email body") do |body| 28 | options[:body] = body 29 | end 30 | 31 | opts.on_tail("-h", "--help", "Show this message") do 32 | puts opts 33 | exit 34 | end 35 | 36 | opts.on_tail("-v", "--version", "Show version") do 37 | puts Tuktuk::VERSION 38 | exit 39 | end 40 | 41 | end.parse! 42 | 43 | if list = missing_keys?(options) and list.any? 44 | puts "Missing option(s): #{list.join(', ')}" 45 | puts "Run with -h or --help for all options." 46 | exit 47 | end 48 | 49 | Tuktuk.options = { 50 | :max_workers => 0, 51 | :log_to => STDOUT 52 | } 53 | 54 | response, email = Tuktuk.deliver(options) 55 | 56 | if response.is_a?(Tuktuk::Bounce) 57 | puts 'Email bounced. Type: ' + response.class.name # => HardBounce or SoftBounce 58 | puts response.message 59 | else 60 | puts 'Email delivered!' 61 | end 62 | -------------------------------------------------------------------------------- /lib/tuktuk.rb: -------------------------------------------------------------------------------- 1 | require File.expand_path(File.dirname(__FILE__)) + '/tuktuk/tuktuk' 2 | -------------------------------------------------------------------------------- /lib/tuktuk/bounce.rb: -------------------------------------------------------------------------------- 1 | module Tuktuk 2 | 3 | class Bounce < RuntimeError 4 | 5 | HARD_BOUNCE_CODES = [ 6 | 501, # Bad address syntax (eg. "i.user.@hotmail.com") 7 | 504, # mailbox is disabled 8 | 511, # sorry, no mailbox here by that name (#5.1.1 - chkuser) 9 | 540, # recipient's email account has been suspended. 10 | 550, # Requested action not taken: mailbox unavailable 11 | 552, # Spam Message Rejected -- Requested mail action aborted: exceeded storage allocation 12 | 554, # Recipient address rejected: Policy Rejection- Abuse. Go away -- This user doesn't have a yahoo.com account 13 | 563, # ERR_MSG_REJECT_BLACKLIST, message has blacklisted content and thus I reject it 14 | 571 # Delivery not authorized, message refused 15 | ] 16 | 17 | def self.type(e) 18 | if e.is_a?(Net::SMTPFatalError) and code = e.to_s[0..2] and HARD_BOUNCE_CODES.include?(code.to_i) 19 | HardBounce.new(e) 20 | else 21 | SoftBounce.new(e) # either soft mailbox bounce, timeout or server bounce 22 | end 23 | end 24 | 25 | def code 26 | if str = to_s[0..2] and str.gsub(/[^0-9]/, '') != '' 27 | str.to_i 28 | end 29 | end 30 | 31 | def status 32 | code 33 | end 34 | 35 | end 36 | 37 | class HardBounce < Bounce; end 38 | class SoftBounce < Bounce; end 39 | 40 | end 41 | -------------------------------------------------------------------------------- /lib/tuktuk/cache.rb: -------------------------------------------------------------------------------- 1 | module Tuktuk 2 | 3 | class Cache 4 | 5 | attr_reader :store, :max_keys 6 | 7 | def initialize(max_keys = 1000) 8 | @store = {} 9 | @max_keys = max_keys 10 | end 11 | 12 | def get(key) 13 | store[key] 14 | end 15 | 16 | def set(key, value) 17 | return if store[key] == value 18 | pop if store.length > max_keys 19 | store[key] = value 20 | end 21 | 22 | def pop 23 | store.delete(store.keys.last) 24 | end 25 | 26 | def show 27 | store.each { |k,v| puts "#{k} -> #{v}" }; nil 28 | end 29 | 30 | end 31 | 32 | end -------------------------------------------------------------------------------- /lib/tuktuk/dns.rb: -------------------------------------------------------------------------------- 1 | begin 2 | require 'resolv' 3 | rescue 4 | require 'net/dns/resolve' 5 | end 6 | 7 | module Tuktuk 8 | 9 | module DNS 10 | 11 | class << self 12 | 13 | def get_mx(host) 14 | if defined?(Resolv::DNS) 15 | get_using_resolve(host) 16 | else 17 | get_using_net_dns(host) 18 | end 19 | end 20 | 21 | def get_using_resolve(host) 22 | Resolv::DNS.open do |dns| 23 | if res = dns.getresources(host, Resolv::DNS::Resource::IN::MX) 24 | sort_mx(res) 25 | end 26 | end 27 | end 28 | 29 | def get_using_net_dns(host) 30 | if res = Net::DNS::Resolver.new.mx(host) 31 | sort_mx(res) 32 | end 33 | end 34 | 35 | def sort_mx(res) 36 | res.sort {|x,y| x.preference <=> y.preference}.map { |rr| rr.exchange.to_s } 37 | end 38 | 39 | end 40 | 41 | end 42 | 43 | end 44 | -------------------------------------------------------------------------------- /lib/tuktuk/package.rb: -------------------------------------------------------------------------------- 1 | require 'mail' 2 | 3 | class Mail::Message 4 | attr_accessor :array_index 5 | end 6 | 7 | module Tuktuk 8 | 9 | module Package 10 | 11 | class << self 12 | 13 | def build(message, index = nil) 14 | mail = message.is_a?(Hash) ? new(message) : message.is_a?(Mail) ? message : Mail.read_from_string(message.to_s) 15 | mail.array_index = index if index 16 | mail 17 | end 18 | 19 | def new(message) 20 | mail = message[:html_body] ? mixed(message) : plain(message) 21 | mail.charset = 'UTF-8' 22 | 23 | mail['In-Reply-To'] = message[:in_reply_to] if message[:in_reply_to] 24 | mail['List-Unsubscribe'] = message[:list_unsubscribe] if message[:list_unsubscribe] 25 | mail['List-Archive'] = message[:list_archive] if message[:list_archive] 26 | mail['List-Id'] = message[:list_id] if message[:list_id] 27 | mail['X-Mailer'] = "Tuktuk SMTP v#{Tuktuk::VERSION}" 28 | 29 | if message[:return_path] 30 | mail['Return-Path'] = message[:return_path] 31 | mail['Bounces-To'] = message[:return_path] 32 | mail['Errors-To'] = message[:return_path] 33 | end 34 | 35 | if (message[:attachments] || []).any? 36 | message[:attachments].each do |file| 37 | mail.add_file(file) 38 | end 39 | end 40 | 41 | mail 42 | end 43 | 44 | def plain(message) 45 | mail = Mail.new do 46 | from message[:from] 47 | to message[:to] 48 | reply_to message[:reply_to] if message[:reply_to] 49 | # sender message[:sender] if message[:sender] 50 | subject message[:subject] 51 | message_id message[:message_id] if message[:message_id] 52 | body message[:body] 53 | end 54 | end 55 | 56 | def mixed(message) 57 | mail = Mail.new do 58 | from message[:from] 59 | to message[:to] 60 | reply_to message[:reply_to] if message[:reply_to] 61 | # sender message[:sender] if message[:sender] 62 | subject message[:subject] 63 | message_id message[:message_id] if message[:message_id] 64 | text_part do 65 | body message[:body] 66 | end 67 | html_part do 68 | content_type 'text/html; charset=UTF-8' 69 | body message[:html_body] 70 | end 71 | end 72 | end 73 | 74 | end 75 | 76 | end 77 | 78 | end 79 | -------------------------------------------------------------------------------- /lib/tuktuk/rails.rb: -------------------------------------------------------------------------------- 1 | if ActionMailer::Base.respond_to?(:add_delivery_method) 2 | 3 | ActionMailer::Base.add_delivery_method :tuktuk, Tuktuk 4 | 5 | module Tuktuk 6 | 7 | def self.new(options) 8 | self.options = options 9 | self 10 | end 11 | 12 | end 13 | 14 | else 15 | 16 | require 'tuktuk' 17 | 18 | class ActionMailer::Base 19 | 20 | def self.tuktuk_settings=(opts) 21 | Tuktuk.options = opts 22 | end 23 | 24 | def perform_delivery_tuktuk(mail) 25 | Tuktuk.deliver!(mail) 26 | end 27 | 28 | end 29 | 30 | end -------------------------------------------------------------------------------- /lib/tuktuk/tuktuk.rb: -------------------------------------------------------------------------------- 1 | require 'net/smtp' 2 | require 'dkim' 3 | require 'logger' 4 | require 'work_queue' 5 | 6 | module Tuktuk; end 7 | 8 | this_path = File.expand_path(File.dirname(__FILE__)) 9 | 10 | %w(package cache dns bounce).each { |lib| require this_path + "/#{lib}" } 11 | require_relative this_path + '/version' unless defined?(Tuktuk::VERSION) 12 | 13 | DEFAULTS = { 14 | :helo_domain => nil, 15 | :max_workers => 0, 16 | :read_timeout => 20, 17 | :open_timeout => 20, 18 | :verify_ssl => true, 19 | :debug => false, 20 | :log_to => nil # $stdout, 21 | } 22 | 23 | # overwrite Net::SMTP#quit since the connection might have been closed 24 | # before we got a chance to say goodbye. swallow the error in that case. 25 | class Net::SMTP 26 | def quit 27 | getok('QUIT') 28 | rescue EOFError => e 29 | # nil 30 | end 31 | end 32 | 33 | module Tuktuk 34 | 35 | class << self 36 | 37 | def cache 38 | @cache ||= Cache.new(100) 39 | end 40 | 41 | def deliver(message, opts = {}) 42 | # raise 'Please pass a valid message object.' unless message[:to] 43 | bcc = opts.delete(:bcc) || [] 44 | bcc = [bcc] if bcc.is_a?(String) 45 | 46 | self.options = opts if opts.any? 47 | mail = Package.build(message) 48 | response = lookup_and_deliver(mail, bcc) 49 | return response, mail 50 | end 51 | 52 | # same as deliver but raises error. used by ActionMailer 53 | def deliver!(mail, opts = {}) 54 | @logger = Rails.logger if defined?(Rails) and !config[:log_to] 55 | resp, email = deliver(mail, opts) 56 | if resp.is_a?(Exception) 57 | raise resp 58 | else 59 | return resp, email 60 | end 61 | end 62 | 63 | def deliver_many(messages, opts = {}) 64 | raise ArgumentError, "Not an array of messages: #{messages.inspect}" unless messages.any? 65 | self.options = opts if opts.any? 66 | messages_by_domain = reorder_by_domain(messages) 67 | lookup_and_deliver_many(messages_by_domain) 68 | end 69 | 70 | def options=(hash) 71 | if dkim_opts = hash.delete(:dkim) 72 | self.dkim = dkim_opts 73 | end 74 | config.merge!(hash) 75 | end 76 | 77 | def dkim=(dkim_opts) 78 | Dkim::domain = dkim_opts[:domain] 79 | Dkim::selector = dkim_opts[:selector] 80 | Dkim::private_key = dkim_opts[:private_key] 81 | end 82 | 83 | private 84 | 85 | def config 86 | @config ||= DEFAULTS 87 | end 88 | 89 | def use_dkim? 90 | !Dkim::domain.nil? 91 | end 92 | 93 | def logger 94 | @logger ||= Logger.new(config[:log_to]) 95 | end 96 | 97 | def get_domain(email_address) 98 | email_address && email_address.to_s[/@([a-z0-9\._-]+)/i, 1] 99 | end 100 | 101 | def reorder_by_domain(array) 102 | hash = {} 103 | array.each_with_index do |message, i| 104 | mail = Package.build(message, i) 105 | if mail.destinations.count != 1 106 | raise ArgumentError, "Invalid destination count: #{mail.destinations.count}" 107 | end 108 | 109 | if to = mail.destinations.first and domain = get_domain(to) 110 | domain = domain.downcase 111 | hash[domain] = [] if hash[domain].nil? 112 | hash[domain].push(mail) 113 | end 114 | end 115 | hash 116 | end 117 | 118 | def smtp_servers_for_domain(domain) 119 | unless servers = cache.get(domain) 120 | if servers = DNS.get_mx(domain) and servers.any? 121 | cache.set(domain, servers) 122 | end 123 | end 124 | servers.any? && servers 125 | end 126 | 127 | def lookup_and_deliver(mail, bcc = []) 128 | if mail.destinations.empty? 129 | raise "No destinations found! You need to pass a :to field." 130 | end 131 | 132 | response = nil 133 | mail.destinations.each do |to| 134 | 135 | domain = get_domain(to) 136 | raise "Empty domain: #{domain}" if domain.to_s.strip == '' 137 | 138 | unless servers = smtp_servers_for_domain(domain) 139 | return HardBounce.new("588 No MX records for domain #{domain}") 140 | end 141 | 142 | last_error = nil 143 | servers.each do |server| 144 | begin 145 | response = send_now(mail, server, to, bcc) 146 | break 147 | rescue Exception => e # explicitly rescue Exception so we catch Timeout:Error's too 148 | logger.error "Error: #{e}" 149 | last_error = e 150 | end 151 | end 152 | return Bounce.type(last_error) if last_error 153 | end 154 | response 155 | end 156 | 157 | def lookup_and_deliver_many(by_domain) 158 | if config[:max_workers] && config[:max_workers] != 0 159 | lookup_and_deliver_many_threaded(by_domain) 160 | else 161 | lookup_and_deliver_many_sync(by_domain) 162 | end 163 | end 164 | 165 | def lookup_and_deliver_many_threaded(by_domain) 166 | count = config[:max_workers].is_a?(Integer) ? config[:max_workers] : nil 167 | queue = WorkQueue.new(count) 168 | responses = [] 169 | 170 | logger.info("Delivering emails to #{by_domain.keys.count} domains...") 171 | by_domain.each do |domain, mails| 172 | queue.enqueue_b(domain, mails) do |domain, mails| 173 | # send emails and then assign responses to array according to mail index 174 | rr = lookup_and_deliver_by_domain(domain, mails) 175 | rr.each do |resp, mail| 176 | responses[mail.array_index] = [resp, mail] 177 | end 178 | end # worker 179 | end 180 | 181 | queue.join # wait for threads to finish 182 | queue.kill # terminate queue 183 | responses 184 | end 185 | 186 | def lookup_and_deliver_many_sync(by_domain) 187 | responses = [] 188 | 189 | logger.info("Delivering emails to #{by_domain.keys.count} domains...") 190 | by_domain.each do |domain, mails| 191 | # send emails and then assign responses to array according to mail index 192 | rr = lookup_and_deliver_by_domain(domain, mails) 193 | rr.each do |resp, mail| 194 | responses[mail.array_index] = [resp, mail] 195 | end 196 | end 197 | responses 198 | end 199 | 200 | def lookup_and_deliver_by_domain(domain, mails) 201 | responses = [] 202 | total = mails.count 203 | 204 | unless servers = smtp_servers_for_domain(domain) 205 | err = HardBounce.new("588 No MX Records for domain #{domain}") 206 | mails.each { |mail| responses.push [err, mail] } 207 | return responses 208 | end 209 | 210 | servers.each do |server| 211 | send_many_now(server, mails).each do |mail, resp| 212 | responses.push [resp, mail] 213 | mails.delete(mail) # remove it from list, to avoid duplicate delivery 214 | end 215 | logger.info "#{responses.count}/#{total} mails processed on #{domain}'s MX: #{server}." 216 | break if responses.count == total 217 | end 218 | 219 | # if we still have emails in queue, mark them with the last error which prevented delivery 220 | if mails.any? and @last_error 221 | bounce = Bounce.type(@last_error) 222 | logger.info "#{mails.count} mails still pending. Marking as #{bounce.class}..." 223 | mails.each { |m| responses.push [bounce, m] } 224 | end 225 | 226 | responses 227 | end 228 | 229 | def send_now(mail, server, to, bcc = []) 230 | logger.info "#{to} - Delivering email at #{server}..." 231 | logger.info "Including these destinations: #{bcc.inspect}" if bcc && bcc.any? 232 | from = get_from(mail) 233 | 234 | response = nil 235 | socket = init_connection(server) 236 | socket.start(get_helo_domain(from), nil, nil, nil) do |smtp| 237 | response = smtp.send_message(get_raw_mail(mail), from, to, *bcc) 238 | logger.info "#{to} - [SENT] #{response.message.strip}" 239 | end 240 | 241 | response 242 | end 243 | 244 | def send_many_now(server, mails) 245 | logger.info "Delivering #{mails.count} mails at #{server}..." 246 | responses = {} 247 | 248 | socket = init_connection(server) 249 | socket.start(get_helo_domain, nil, nil, nil) do |smtp| 250 | mails.each do |mail| 251 | begin 252 | resp = smtp.send_message(get_raw_mail(mail), get_from(mail), mail.to) 253 | smtp.send(:getok, 'RSET') if server['hotmail'] # fix for '503 Sender already specified' 254 | rescue Net::SMTPFatalError, Net::SMTPServerBusy => e # error code 5xx, except for 500, like: 550 Mailbox not found 255 | resp = Bounce.type(e) 256 | end 257 | responses[mail] = resp 258 | logger.info "#{mail.to} [#{responses[mail].class}] #{responses[mail].message.strip}" # both error and response have this method 259 | end 260 | end 261 | 262 | responses 263 | rescue Exception => e # SMTPServerBusy, SMTPSyntaxError, SMTPUnsupportedCommand, SMTPUnknownError (unexpected reply code) 264 | logger.error "[SERVER ERROR: #{server}] #{e.class} -> #{e.message}" 265 | @last_error = e 266 | responses 267 | end 268 | 269 | def get_raw_mail(mail) 270 | use_dkim? ? Dkim.sign(mail.to_s).to_s : mail.to_s 271 | end 272 | 273 | def get_from(mail) 274 | mail.return_path || mail.sender || mail.from_addrs.first 275 | end 276 | 277 | def get_helo_domain(from = nil) 278 | Dkim::domain || config[:helo_domain] || (from && get_domain(from)) 279 | end 280 | 281 | def init_connection(server) 282 | context = OpenSSL::SSL::SSLContext.new 283 | context.verify_mode = config[:verify_ssl] ? 284 | OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE 285 | context.set_params # Configures default certificate store 286 | 287 | port = nil 288 | if config[:debug] 289 | if config[:debug].is_a?(String) 290 | server = config[:debug].split(':').first 291 | port = config[:debug].split(':').last if config[:debug][':'] 292 | else 293 | server = 'localhost' 294 | end 295 | logger.warn "Debug option enabled. Connecting to #{server}!" 296 | end 297 | 298 | smtp = Net::SMTP.new(server, port) 299 | smtp.enable_starttls_auto(context) 300 | smtp.read_timeout = config[:read_timeout] if config[:read_timeout] 301 | smtp.open_timeout = config[:open_timeout] if config[:open_timeout] 302 | smtp 303 | end 304 | 305 | end 306 | 307 | end 308 | -------------------------------------------------------------------------------- /lib/tuktuk/version.rb: -------------------------------------------------------------------------------- 1 | module Tuktuk 2 | MAJOR = 0 3 | MINOR = 9 4 | PATCH = 0 5 | 6 | VERSION = [MAJOR, MINOR, PATCH].join('.') 7 | end 8 | -------------------------------------------------------------------------------- /spec/deliver_spec.rb: -------------------------------------------------------------------------------- 1 | require './lib/tuktuk/tuktuk' 2 | require 'rspec/mocks' 3 | 4 | def email(attrs = {}) 5 | { :to => "user#{rand(1000)}@domain.com", 6 | :from => 'me@company.com', 7 | :subject => 'Test email', 8 | :body => 'Hello world.' 9 | }.merge(attrs) 10 | end 11 | 12 | describe 'deliver' do 13 | 14 | before(:each) do 15 | @mock_smtp = double('Net::SMTP', enable_starttls_auto: true, :read_timeout= => true, :open_timeout= => true) 16 | @mock_conn = double('SMTP Connection') 17 | @mock_smtp.stub(:start).and_yield(@mock_conn) 18 | @mock_resp = double('SMTP::Response', message: '250 OK') 19 | 20 | Net::SMTP.stub(:new).and_return(@mock_smtp) 21 | end 22 | 23 | describe 'single recipient' do 24 | 25 | describe 'when destination is valid (has MX servers)' do 26 | 27 | before do 28 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com'] 29 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers) 30 | end 31 | 32 | it 'sends message' do 33 | msg = email 34 | expect(@mock_conn).to receive(:send_message).with(String, msg[:from], msg[:to]).and_return(@mock_resp) 35 | Tuktuk.deliver(msg) 36 | end 37 | 38 | describe 'and bcc is given' do 39 | 40 | let(:bcc_email) { 'bcc@test.com' } 41 | 42 | it 'includes it in destination list' do 43 | msg = email 44 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to], bcc_email).and_return(@mock_resp) 45 | Tuktuk.deliver(msg, bcc: [bcc_email]) 46 | end 47 | 48 | it 'also works if not passed as array' do 49 | msg = email 50 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to], bcc_email).and_return(@mock_resp) 51 | Tuktuk.deliver(msg, bcc: bcc_email) 52 | end 53 | 54 | end 55 | 56 | end 57 | 58 | end 59 | 60 | describe 'multiple recipients (string list)' do 61 | 62 | describe 'when destination is valid (has MX servers)' do 63 | 64 | before do 65 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com'] 66 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers) 67 | end 68 | 69 | it 'sends message' do 70 | msg = email(to: 'some@one.com, another@one.com') 71 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first).and_return(@mock_resp) 72 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last).and_return(@mock_resp) 73 | Tuktuk.deliver(msg) 74 | end 75 | 76 | describe 'and bcc is given' do 77 | 78 | let(:bcc_email) { 'bcc@test.com' } 79 | 80 | it 'includes it in destination list' do 81 | msg = email(to: 'some@one.com, another@one.com') 82 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first, bcc_email).and_return(@mock_resp) 83 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last, bcc_email).and_return(@mock_resp) 84 | Tuktuk.deliver(msg, bcc: [bcc_email]) 85 | end 86 | 87 | it 'also works if not passed as array' do 88 | msg = email(to: 'some@one.com, another@one.com') 89 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first, bcc_email).and_return(@mock_resp) 90 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last, bcc_email).and_return(@mock_resp) 91 | Tuktuk.deliver(msg, bcc: bcc_email) 92 | end 93 | 94 | end 95 | 96 | end 97 | 98 | end 99 | 100 | describe 'multiple recipients (array)' do 101 | 102 | describe 'when destination is valid (has MX servers)' do 103 | 104 | before do 105 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com'] 106 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers) 107 | end 108 | 109 | it 'sends message' do 110 | msg = email(to: ['some@one.com', 'another@one.com']) 111 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first).and_return(@mock_resp) 112 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last).and_return(@mock_resp) 113 | Tuktuk.deliver(msg) 114 | end 115 | 116 | describe 'and bcc is given' do 117 | 118 | let(:bcc_email) { 'bcc@test.com' } 119 | 120 | it 'includes it in destination list' do 121 | msg = email(to: ['some@one.com', 'another@one.com']) 122 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first, bcc_email).and_return(@mock_resp) 123 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last, bcc_email).and_return(@mock_resp) 124 | Tuktuk.deliver(msg, bcc: [bcc_email]) 125 | end 126 | 127 | it 'also works if not passed as array' do 128 | msg = email(to: ['some@one.com', 'another@one.com']) 129 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first, bcc_email).and_return(@mock_resp) 130 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last, bcc_email).and_return(@mock_resp) 131 | Tuktuk.deliver(msg, bcc: bcc_email) 132 | end 133 | 134 | end 135 | 136 | end 137 | 138 | end 139 | 140 | end 141 | 142 | describe 'deliver many' do 143 | 144 | before(:each) do 145 | @mock_smtp = double('Net::SMTP') 146 | Net::SMTP.stub(:new).and_return(@mock_smtp) 147 | end 148 | 149 | describe 'when no emails are passed' do 150 | 151 | it 'raises' do 152 | lambda do 153 | Tuktuk.deliver_many [] 154 | end.should raise_error(ArgumentError) 155 | end 156 | 157 | end 158 | 159 | describe 'when one email contains multiple addresses' do 160 | 161 | it 'raises' do 162 | lambda do 163 | Tuktuk.deliver_many [ email, email(:to => 'one@user.com, two@user.com') ] 164 | end.should raise_error(ArgumentError) 165 | end 166 | 167 | end 168 | 169 | describe 'when emails are valid' do 170 | 171 | it 'groups them by domain' do 172 | 173 | end 174 | 175 | describe 'and max_workers is 0' do 176 | 177 | it 'does not start any threads' do 178 | 179 | end 180 | 181 | end 182 | 183 | describe 'and max_workers is >0' do 184 | 185 | it 'does not spawn any more threads than the max allowed' do 186 | 187 | end 188 | 189 | end 190 | 191 | describe 'and max workers is auto' do 192 | 193 | it 'spawns a new thread for each domain' do 194 | 195 | end 196 | 197 | end 198 | 199 | describe 'when delivering to domain' do 200 | 201 | before do 202 | @mock_smtp.stub(:start).and_yield('foo') 203 | @emails = [email, email, email] 204 | 205 | @success = double('Net::SMTP::Response') 206 | @soft_email_bounce = Tuktuk::SoftBounce.new('503 Sender already specified') 207 | @hard_email_bounce = Tuktuk::HardBounce.new('505 Mailbox not found') 208 | @soft_server_bounce = Tuktuk::SoftBounce.new('Be back in a sec') 209 | @hard_server_bounce = Tuktuk::HardBounce.new('No MX records found.') 210 | end 211 | 212 | describe 'when domain exists' do 213 | 214 | before do 215 | @domain = 'domain.com' 216 | end 217 | 218 | describe 'and has valid MX servers' do 219 | 220 | before do 221 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com'] 222 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers) 223 | end 224 | 225 | it 'starts by delivering to first one' do 226 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', [1]).and_return([[1, 'ok']]) 227 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', [1]) 228 | end 229 | 230 | describe 'and first server processes all our mail' do 231 | 232 | describe 'and all mail goes through' do 233 | 234 | before do 235 | @responses = [] 236 | @emails.each { |e| @responses.push [e, @success] } 237 | end 238 | 239 | it 'does not try to connect to second server' do 240 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return(@responses) 241 | Tuktuk.should_not_receive(:send_many_now).with('mx2.domain.com') 242 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 243 | end 244 | 245 | end 246 | 247 | describe 'and all emails were hard failures (bounces)' do 248 | 249 | before do 250 | @responses = [] 251 | @emails.each { |e| @responses.push [e, @hard_email_bounce] } 252 | end 253 | 254 | it 'does not try to connect to second server' do 255 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return(@responses) 256 | Tuktuk.should_not_receive(:send_many_now).with('mx2.domain.com') 257 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 258 | end 259 | 260 | end 261 | 262 | end 263 | 264 | describe 'and first server is down' do 265 | 266 | before do 267 | @responses = [] 268 | @emails.each { |e| @responses.push [e, @success] } 269 | end 270 | 271 | it 'does not raise error' do 272 | Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_raise('Unable to connect.') 273 | Tuktuk.should_receive(:init_connection).once.with('mx2.domain.com').and_raise('Unable to connect.') 274 | Tuktuk.should_receive(:init_connection).once.with('mx3.domain.com').and_raise('Unable to connect.') 275 | # Tuktuk.should_receive(:init_connection).once.and_raise('Unable to connect.') 276 | expect do 277 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 278 | end.not_to raise_error # (RuntimeError) 279 | end 280 | 281 | it 'returns empty responses' do 282 | Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_raise('Unable to connect.') 283 | responses = Tuktuk.send(:send_many_now, 'mx1.domain.com', @emails) 284 | responses.should be_empty 285 | end 286 | 287 | it 'tries to connect to second server' do 288 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return([]) 289 | Tuktuk.should_receive(:send_many_now).once.with('mx2.domain.com', @emails).and_return(@responses) 290 | Tuktuk.should_not_receive(:send_many_now).with('mx3.domain.com') 291 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 292 | end 293 | 294 | end 295 | 296 | describe 'and first server receives only one email' do 297 | 298 | before do 299 | @first = [@emails[0], @success] 300 | @last_two = [[@emails[1], @success], [@emails[2], @soft_email_bounce]] 301 | end 302 | 303 | it 'does not try to send that same email to second server' do 304 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return([@first]) 305 | last_two_emails = @emails.last(2) 306 | last_two_emails.include?(@emails.first).should be false 307 | Tuktuk.should_receive(:send_many_now).once.with('mx2.domain.com', last_two_emails).and_return(@last_two) 308 | Tuktuk.should_not_receive(:send_many_now).with('mx3.domain.com') 309 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 310 | end 311 | 312 | describe 'and other servers are down' do 313 | 314 | before do 315 | # TODO: for some reason the :init_connection on line 138 is affecting this 316 | # this test should pass when running on its own 317 | # Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_return(@mock_smtp) 318 | # Tuktuk.should_receive(:init_connection).once.with('mx2.domain.com').and_raise('Unable to connect.') 319 | # Tuktuk.should_receive(:init_connection).once.with('mx3.domain.com').and_raise('Unable to connect.') 320 | end 321 | 322 | it 'should not mark first email as bounced' do 323 | Tuktuk.should_receive(:send_many_now).and_return([@first], [], []) 324 | responses = Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails) 325 | responses[1][0].should be_a(Tuktuk::Bounce) if responses[1] 326 | responses[0][0].should_not be_a(Tuktuk::Bounce) 327 | end 328 | 329 | end 330 | 331 | end 332 | 333 | end 334 | 335 | end 336 | 337 | end 338 | 339 | end 340 | 341 | end 342 | -------------------------------------------------------------------------------- /tuktuk.gemspec: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | require File.expand_path("../lib/tuktuk/version", __FILE__) 3 | 4 | Gem::Specification.new do |s| 5 | s.name = "tuktuk" 6 | s.version = Tuktuk::VERSION 7 | s.platform = Gem::Platform::RUBY 8 | s.authors = ['Tomás Pollak'] 9 | s.email = ['tomas@forkhq.com'] 10 | s.homepage = "https://github.com/tomas/tuktuk" 11 | s.summary = "SMTP client for Ruby with DKIM support." 12 | s.description = "Easy way of sending DKIM-signed emails from Ruby, no dependencies needed." 13 | 14 | s.required_rubygems_version = ">= 1.3.6" 15 | s.rubyforge_project = "tuktuk" 16 | 17 | s.add_development_dependency "bundler", ">= 1.0.0" 18 | s.add_runtime_dependency "net-dns", "= 0.6.1" 19 | s.add_runtime_dependency "mail", "~> 2.3" 20 | s.add_runtime_dependency "dkim", "~> 0.0.2" 21 | s.add_runtime_dependency 'work_queue', '~> 2.5.0' 22 | 23 | s.files = `git ls-files`.split("\n") 24 | s.executables = `git ls-files`.split("\n").map{|f| f =~ /^bin\/(.*)/ ? $1 : nil}.compact 25 | s.require_path = 'lib' 26 | end 27 | --------------------------------------------------------------------------------