├── .github
└── workflows
│ └── ruby.yml
├── .gitignore
├── Gemfile
├── README.md
├── Rakefile
├── _config.yml
├── bin
└── tuktuk
├── lib
├── tuktuk.rb
└── tuktuk
│ ├── bounce.rb
│ ├── cache.rb
│ ├── dns.rb
│ ├── package.rb
│ ├── rails.rb
│ ├── tuktuk.rb
│ └── version.rb
├── spec
└── deliver_spec.rb
└── tuktuk.gemspec
/.github/workflows/ruby.yml:
--------------------------------------------------------------------------------
1 | name: Ruby
2 | on: [push]
3 |
4 | jobs:
5 | build:
6 | runs-on: ubuntu-latest
7 |
8 | steps:
9 | - uses: actions/checkout@v1
10 | - name: Set up Ruby 2.6
11 | uses: actions/setup-ruby@v1
12 | with:
13 | ruby-version: 2.6.x
14 | - name: Build and test with Rake
15 | run: |
16 | gem install bundler
17 | gem install rspec
18 | bundle install --jobs 4 --retry 3
19 | rspec
20 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | *~
2 | *.gem
3 | test
4 | log
5 | pkg
6 |
--------------------------------------------------------------------------------
/Gemfile:
--------------------------------------------------------------------------------
1 | source :rubygems
2 |
3 | # specified in tuktuk.gemspec
4 | gemspec
5 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | Tuktuk - SMTP client for Ruby
2 | =============================
3 |
4 | Unlike famous ol' Pony gem (which is friggin' awesome by the way), Tuktuk does not rely on
5 | `sendmail` or a separate SMTP server in order to deliver email. Tuktuk looks up the
6 | MX servers of the destination address and connects directly using Net::SMTP.
7 | This way you don't need to install Exim or Postfix and you can actually handle
8 | response status codes -- like bounces, 5xx -- within your application.
9 |
10 | Plus, it supports DKIM out of the box.
11 |
12 | Delivering mail
13 | ---------------
14 |
15 | ``` ruby
16 | require 'tuktuk'
17 |
18 | message = {
19 | :from => 'you@username.com',
20 | :to => 'user@yoursite.com',
21 | :body => 'Hello there',
22 | :subject => 'Hiya'
23 | }
24 |
25 | response, email = Tuktuk.deliver(message)
26 | ```
27 |
28 | HTML (multipart) emails are supported, of course.
29 |
30 | ``` ruby
31 |
32 | message = {
33 | :from => 'you@username.com',
34 | :to => 'user@yoursite.com',
35 | :body => 'Hello there',
36 | :html_body => '
Hello there
',
37 | :subject => 'Hiya in colours'
38 | }
39 |
40 | response, email = Tuktuk.deliver(message)
41 | ```
42 |
43 | The `response` is either a [Net::SMTP::Response](http://ruby-doc.org/stdlib-2.0.0/libdoc/net/smtp/rdoc/Net/SMTP/Response.html) object, or a Bounce exception (HardBounce or SoftBounce, depending on the cause). `email` is a [mail](https://github.com/mikel/mail) object. So, to handle bounces you'd do:
44 |
45 | ``` ruby
46 | [...]
47 |
48 | response, email = Tuktuk.deliver(message)
49 |
50 | if response.is_a?(Tuktuk::Bounce)
51 | puts 'Email bounced. Type: ' + response.class.name # => HardBounce or SoftBounce
52 | else
53 | puts 'Email delivered! Server responded: ' + response.message
54 | end
55 | ```
56 |
57 | You can also call `Tuktuk.deliver!` (with a trailing `!`), in which case it will automatically raise an exception if the response was either a `HardBounce` or a `SoftBounce`. This is useful when running in the background via Resque or Sidekiq, because it makes you aware of which emails are not getting through, and you can requeue those jobs to have them redelivered.
58 |
59 | Email options
60 | -------------
61 |
62 | Attachments are supported, as you'd expect.
63 |
64 | ``` rb
65 | message = {
66 | :from => 'john@lennon.com',
67 | :to => 'paul@maccartney.com',
68 | :subject => 'Question for you',
69 | :body => 'How do you sleep?',
70 | :reply_to => '',
71 | :return_path => 'bounces@server.com',
72 | :attachments => [ '/home/john/walrus.png' ]
73 | }
74 | ```
75 |
76 | Attachments can be either a path to a file or a hash containing the file's name and content, like this:
77 |
78 | ``` rb
79 | message = {
80 | ...
81 | :attachments => [
82 | { :filename => 'walrus.png', :content => File.read('/home/john/walrus.png') }
83 | ]
84 | }
85 | ```
86 |
87 | These are the email headers Tuktuk is able to set for you. Just pass them as part of the hash and they'll be automatically set.
88 |
89 | ```
90 | :return_path => '', # will actually set three headers, Return-Path, Bounces-To and Errors-To
91 | :reply_to => '',
92 | :in_reply_to => '',
93 | :list_unsubscribe => ', ',
94 | :list_archive => '',
95 | :list_id => ''
96 | ```
97 |
98 | Delivering multiple
99 | -------------------
100 |
101 | With Tuktuk, you can also deliver multiple messages at once. Depending on the `max_workers` config parameter, Tuktuk will either connect sequentially to the target domain's MX servers, or do it in parallel by spawning threads.
102 |
103 | Tuktuk will try to send all emails targeted for a specific domain on the same SMTP session. If a MX server is not responding -- or times out in the middle --, Tuktuk will try to deliver the remaining messages to next MX server, and so on.
104 |
105 | To #deliver_many, you need to pass an array of messages, and you'll receive an array of [response, email] elements, just as above.
106 |
107 | ``` ruby
108 | messages = [ { ... }, { ... }, { ... }, { ... } ] # array of messages
109 |
110 | result = Tuktuk.deliver_many(messages)
111 |
112 | result.each do |response, email|
113 |
114 | if response.is_a?(Tuktuk::Bounce)
115 | puts 'Email bounced. Type: ' + response.class.name
116 | else
117 | puts 'Email delivered!'
118 | end
119 |
120 | end
121 | ```
122 |
123 | Options & DKIM
124 | --------------
125 |
126 | Now, if you want to enable DKIM (and you _should_):
127 |
128 | ``` ruby
129 | require 'tuktuk'
130 |
131 | Tuktuk.options = {
132 | :dkim => {
133 | :domain => 'yoursite.com',
134 | :selector => 'mailer',
135 | :private_key => IO.read('ssl/yoursite.com.key')
136 | }
137 | }
138 |
139 | message = { ... }
140 |
141 | response, email = Tuktuk.deliver(message)
142 | ```
143 |
144 | For DKIM to work, you need to set up some TXT records in your domain's DNS. You can use [this tool](http://www.socketlabs.com/domainkey-dkim-generation-wizard/) to generate the key. You should also create [SPF records](http://www.spfwizard.net/) if you haven't. Then use [this tool](https://www.mail-tester.com/spf-dkim-check) to verify that they're both correctly in place.
145 |
146 | All available options, with their defaults:
147 |
148 | ``` ruby
149 | Tuktuk.options = {
150 | :log_to => nil, # e.g. log/mailer.log or STDOUT
151 | :helo_domain => nil, # your server's domain goes here
152 | :max_workers => 0, # controls number of threads for delivering_many emails (read below)
153 | :open_timeout => 20, # max seconds to wait for opening a connection
154 | :read_timeout => 20, # 20 seconds to wait for a response, once connected
155 | :verify_ssl => true, # whether to skip SSL keys verification or not
156 | :debug => false, # connects and delivers email to localhost, instead of real target server. CAUTION!
157 | :dkim => { ... }
158 | }
159 | ```
160 |
161 | You can set the `max_workers` option to `auto`, which will spawn the necessary threads to connect in paralell to all target MX servers when delivering multiple messages. When set to `0`, these batches will be delivered sequentially.
162 |
163 | In other words, if you have three emails targeted to Gmail users and two for Hotmail users, using `auto` Tuktuk will spawn two threads and connect to both servers at once. Using `0` will have your emails delivered to one host, and then the other.
164 |
165 | Using with Rails
166 | ----------------
167 |
168 | Tuktuk comes with ActionMailer support out of the box. In your environment.rb or environments/{env}.rb:
169 |
170 | ``` ruby
171 | require 'tuktuk/rails'
172 |
173 | [...]
174 |
175 | config.action_mailer.delivery_method = :tuktuk
176 | ```
177 |
178 | Since Tuktuk delivers email directly to the user's MX servers, it's probably a good idea to set `config.action_mailer.raise_delivery_errors` to true. That way you can actually know if an email couldn't make it to its destination.
179 |
180 | When used with ActionMailer, you can pass options using ActionMailer's interface, like this:
181 |
182 | ``` ruby
183 |
184 | config.action_mailer.delivery_method = :tuktuk
185 |
186 | config.action_mailer.tuktuk_settings = {
187 | :log_to => 'log/mailer.log', # when not set, Tuktuk will use Rails.logger
188 | :dkim => {
189 | :domain => 'yoursite.com',
190 | :selector => 'mailer',
191 | :private_key => IO.read('ssl/yoursite.com.key')
192 | }
193 | }
194 | ```
195 |
196 | # Example SPK/DKIM/DMARC settings
197 |
198 | If you're sending email from yoursite.com, the SPF record should be set for the APEX/root host, and look like this:
199 |
200 | v=spf1 ip4:[ipv4_address] ip6:[ipv6_address] mx a include:[other_host] ~all
201 |
202 | For example:
203 |
204 | v=spf1 ip4:12.34.56.78 ip6:2600:3c05::f07c:92ff:fe48:b2fd mx a include:mailgun.org ~all
205 |
206 | This tells the receiving server to accept email sent from a) the addresses explicitly mentioned (`ip4` and `ip6`),
207 | b) from the hosts mentioned in the `include` statements, as well as c) the hosts listed as `MX` and `A` records for that domain.
208 |
209 | As for DKIM, you should add two TXT records. The first is a simple, short one that goes under the `_domainkey` host,
210 | and should contain the following:
211 |
212 | t=y;o=~;
213 |
214 | Then, a second DKIM record should be placed under `[selector]._domainkey` (e.g. `mailer._domainkey`), and should look like this:
215 |
216 | k=rsa; p=MIIBIBANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA[...]DAQAB (public key)
217 |
218 | And finally, your DMARC record goes under the `_dmarc` host, and goes like this:
219 |
220 | v=DMARC1; p=none; rua=mailto:postmaster@yoursite.com; ruf=mailto:postmaster@yoursite.com
221 |
222 | So, in summary:
223 |
224 | (SPF) @.yoursite.com --> v=spf1 ip4:[ipv4_address] ip6:[ipv6_address] mx a include:[other_host] ~all
225 | (DKIM1) _domainkey.yoursite.com --> t=y;o=~;
226 | (DKIM2) [selector]._domainkey.yoursite.com --> k=rsa; p=MIIBIBANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA[...]DAQAB
227 | (DMARC) _dmarc.yoursite.com --> v=DMARC1; p=none; rua=mailto:postmaster@yoursite.com; ruf=mailto:postmaster@yoursite.com
228 |
229 | Now, to check wether your records are OK, you can use the `dig` command like follows:
230 |
231 | dig yoursite.com TXT +short # should output the SPF record, under the root domain
232 | dig mailer._domainkey.yoursite.com TXT +short # should output the DKIM record containing the key
233 | dig _domainkey.yoursite.com TXT +short # should output the other (short) DKIM
234 | dig _dmarc.yoursite.com TXT +short # should output the DMARC record
235 |
236 | Remember you can query your DNS server directly with the `dig` command by adding `@name.server.com`
237 | after the `dig` command (e.g. `dig @ns1.linode.com yoursite.com TXT`).
238 |
239 | # Contributions
240 |
241 | You're more than welcome. Send a pull request, including tests, and make sure you don't break anything. That's it.
242 |
243 | # Copyright
244 |
245 | (c) Fork Limited. MIT license.
246 |
--------------------------------------------------------------------------------
/Rakefile:
--------------------------------------------------------------------------------
1 | require 'bundler'
2 | Bundler::GemHelper.install_tasks
3 |
--------------------------------------------------------------------------------
/_config.yml:
--------------------------------------------------------------------------------
1 | theme: jekyll-theme-minimal
--------------------------------------------------------------------------------
/bin/tuktuk:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env ruby
2 |
3 | require File.expand_path(File.dirname(__FILE__)) + '/../lib/tuktuk'
4 | require 'optparse'
5 |
6 | def missing_keys?(opts)
7 | [:body, :from, :subject, :to] - opts.keys
8 | end
9 |
10 | options = {}
11 |
12 | OptionParser.new do |opts|
13 | opts.banner = "Usage: tuktuk [options]"
14 |
15 | opts.on("-f", "--from your@email.com", String, "From email") do |val|
16 | options[:from] = val
17 | end
18 |
19 | opts.on("-t", "--to email1,email2", Array, "List of destination emails") do |list|
20 | options[:to] = list
21 | end
22 |
23 | opts.on("-s","--subject 'This is a test.'", String, "Email subject") do |subject|
24 | options[:subject] = subject
25 | end
26 |
27 | opts.on("-b", "--body 'Hello there.'", String, "Email body") do |body|
28 | options[:body] = body
29 | end
30 |
31 | opts.on_tail("-h", "--help", "Show this message") do
32 | puts opts
33 | exit
34 | end
35 |
36 | opts.on_tail("-v", "--version", "Show version") do
37 | puts Tuktuk::VERSION
38 | exit
39 | end
40 |
41 | end.parse!
42 |
43 | if list = missing_keys?(options) and list.any?
44 | puts "Missing option(s): #{list.join(', ')}"
45 | puts "Run with -h or --help for all options."
46 | exit
47 | end
48 |
49 | Tuktuk.options = {
50 | :max_workers => 0,
51 | :log_to => STDOUT
52 | }
53 |
54 | response, email = Tuktuk.deliver(options)
55 |
56 | if response.is_a?(Tuktuk::Bounce)
57 | puts 'Email bounced. Type: ' + response.class.name # => HardBounce or SoftBounce
58 | puts response.message
59 | else
60 | puts 'Email delivered!'
61 | end
62 |
--------------------------------------------------------------------------------
/lib/tuktuk.rb:
--------------------------------------------------------------------------------
1 | require File.expand_path(File.dirname(__FILE__)) + '/tuktuk/tuktuk'
2 |
--------------------------------------------------------------------------------
/lib/tuktuk/bounce.rb:
--------------------------------------------------------------------------------
1 | module Tuktuk
2 |
3 | class Bounce < RuntimeError
4 |
5 | HARD_BOUNCE_CODES = [
6 | 501, # Bad address syntax (eg. "i.user.@hotmail.com")
7 | 504, # mailbox is disabled
8 | 511, # sorry, no mailbox here by that name (#5.1.1 - chkuser)
9 | 540, # recipient's email account has been suspended.
10 | 550, # Requested action not taken: mailbox unavailable
11 | 552, # Spam Message Rejected -- Requested mail action aborted: exceeded storage allocation
12 | 554, # Recipient address rejected: Policy Rejection- Abuse. Go away -- This user doesn't have a yahoo.com account
13 | 563, # ERR_MSG_REJECT_BLACKLIST, message has blacklisted content and thus I reject it
14 | 571 # Delivery not authorized, message refused
15 | ]
16 |
17 | def self.type(e)
18 | if e.is_a?(Net::SMTPFatalError) and code = e.to_s[0..2] and HARD_BOUNCE_CODES.include?(code.to_i)
19 | HardBounce.new(e)
20 | else
21 | SoftBounce.new(e) # either soft mailbox bounce, timeout or server bounce
22 | end
23 | end
24 |
25 | def code
26 | if str = to_s[0..2] and str.gsub(/[^0-9]/, '') != ''
27 | str.to_i
28 | end
29 | end
30 |
31 | def status
32 | code
33 | end
34 |
35 | end
36 |
37 | class HardBounce < Bounce; end
38 | class SoftBounce < Bounce; end
39 |
40 | end
41 |
--------------------------------------------------------------------------------
/lib/tuktuk/cache.rb:
--------------------------------------------------------------------------------
1 | module Tuktuk
2 |
3 | class Cache
4 |
5 | attr_reader :store, :max_keys
6 |
7 | def initialize(max_keys = 1000)
8 | @store = {}
9 | @max_keys = max_keys
10 | end
11 |
12 | def get(key)
13 | store[key]
14 | end
15 |
16 | def set(key, value)
17 | return if store[key] == value
18 | pop if store.length > max_keys
19 | store[key] = value
20 | end
21 |
22 | def pop
23 | store.delete(store.keys.last)
24 | end
25 |
26 | def show
27 | store.each { |k,v| puts "#{k} -> #{v}" }; nil
28 | end
29 |
30 | end
31 |
32 | end
--------------------------------------------------------------------------------
/lib/tuktuk/dns.rb:
--------------------------------------------------------------------------------
1 | begin
2 | require 'resolv'
3 | rescue
4 | require 'net/dns/resolve'
5 | end
6 |
7 | module Tuktuk
8 |
9 | module DNS
10 |
11 | class << self
12 |
13 | def get_mx(host)
14 | if defined?(Resolv::DNS)
15 | get_using_resolve(host)
16 | else
17 | get_using_net_dns(host)
18 | end
19 | end
20 |
21 | def get_using_resolve(host)
22 | Resolv::DNS.open do |dns|
23 | if res = dns.getresources(host, Resolv::DNS::Resource::IN::MX)
24 | sort_mx(res)
25 | end
26 | end
27 | end
28 |
29 | def get_using_net_dns(host)
30 | if res = Net::DNS::Resolver.new.mx(host)
31 | sort_mx(res)
32 | end
33 | end
34 |
35 | def sort_mx(res)
36 | res.sort {|x,y| x.preference <=> y.preference}.map { |rr| rr.exchange.to_s }
37 | end
38 |
39 | end
40 |
41 | end
42 |
43 | end
44 |
--------------------------------------------------------------------------------
/lib/tuktuk/package.rb:
--------------------------------------------------------------------------------
1 | require 'mail'
2 |
3 | class Mail::Message
4 | attr_accessor :array_index
5 | end
6 |
7 | module Tuktuk
8 |
9 | module Package
10 |
11 | class << self
12 |
13 | def build(message, index = nil)
14 | mail = message.is_a?(Hash) ? new(message) : message.is_a?(Mail) ? message : Mail.read_from_string(message.to_s)
15 | mail.array_index = index if index
16 | mail
17 | end
18 |
19 | def new(message)
20 | mail = message[:html_body] ? mixed(message) : plain(message)
21 | mail.charset = 'UTF-8'
22 |
23 | mail['In-Reply-To'] = message[:in_reply_to] if message[:in_reply_to]
24 | mail['List-Unsubscribe'] = message[:list_unsubscribe] if message[:list_unsubscribe]
25 | mail['List-Archive'] = message[:list_archive] if message[:list_archive]
26 | mail['List-Id'] = message[:list_id] if message[:list_id]
27 | mail['X-Mailer'] = "Tuktuk SMTP v#{Tuktuk::VERSION}"
28 |
29 | if message[:return_path]
30 | mail['Return-Path'] = message[:return_path]
31 | mail['Bounces-To'] = message[:return_path]
32 | mail['Errors-To'] = message[:return_path]
33 | end
34 |
35 | if (message[:attachments] || []).any?
36 | message[:attachments].each do |file|
37 | mail.add_file(file)
38 | end
39 | end
40 |
41 | mail
42 | end
43 |
44 | def plain(message)
45 | mail = Mail.new do
46 | from message[:from]
47 | to message[:to]
48 | reply_to message[:reply_to] if message[:reply_to]
49 | # sender message[:sender] if message[:sender]
50 | subject message[:subject]
51 | message_id message[:message_id] if message[:message_id]
52 | body message[:body]
53 | end
54 | end
55 |
56 | def mixed(message)
57 | mail = Mail.new do
58 | from message[:from]
59 | to message[:to]
60 | reply_to message[:reply_to] if message[:reply_to]
61 | # sender message[:sender] if message[:sender]
62 | subject message[:subject]
63 | message_id message[:message_id] if message[:message_id]
64 | text_part do
65 | body message[:body]
66 | end
67 | html_part do
68 | content_type 'text/html; charset=UTF-8'
69 | body message[:html_body]
70 | end
71 | end
72 | end
73 |
74 | end
75 |
76 | end
77 |
78 | end
79 |
--------------------------------------------------------------------------------
/lib/tuktuk/rails.rb:
--------------------------------------------------------------------------------
1 | if ActionMailer::Base.respond_to?(:add_delivery_method)
2 |
3 | ActionMailer::Base.add_delivery_method :tuktuk, Tuktuk
4 |
5 | module Tuktuk
6 |
7 | def self.new(options)
8 | self.options = options
9 | self
10 | end
11 |
12 | end
13 |
14 | else
15 |
16 | require 'tuktuk'
17 |
18 | class ActionMailer::Base
19 |
20 | def self.tuktuk_settings=(opts)
21 | Tuktuk.options = opts
22 | end
23 |
24 | def perform_delivery_tuktuk(mail)
25 | Tuktuk.deliver!(mail)
26 | end
27 |
28 | end
29 |
30 | end
--------------------------------------------------------------------------------
/lib/tuktuk/tuktuk.rb:
--------------------------------------------------------------------------------
1 | require 'net/smtp'
2 | require 'dkim'
3 | require 'logger'
4 | require 'work_queue'
5 |
6 | module Tuktuk; end
7 |
8 | this_path = File.expand_path(File.dirname(__FILE__))
9 |
10 | %w(package cache dns bounce).each { |lib| require this_path + "/#{lib}" }
11 | require_relative this_path + '/version' unless defined?(Tuktuk::VERSION)
12 |
13 | DEFAULTS = {
14 | :helo_domain => nil,
15 | :max_workers => 0,
16 | :read_timeout => 20,
17 | :open_timeout => 20,
18 | :verify_ssl => true,
19 | :debug => false,
20 | :log_to => nil # $stdout,
21 | }
22 |
23 | # overwrite Net::SMTP#quit since the connection might have been closed
24 | # before we got a chance to say goodbye. swallow the error in that case.
25 | class Net::SMTP
26 | def quit
27 | getok('QUIT')
28 | rescue EOFError => e
29 | # nil
30 | end
31 | end
32 |
33 | module Tuktuk
34 |
35 | class << self
36 |
37 | def cache
38 | @cache ||= Cache.new(100)
39 | end
40 |
41 | def deliver(message, opts = {})
42 | # raise 'Please pass a valid message object.' unless message[:to]
43 | bcc = opts.delete(:bcc) || []
44 | bcc = [bcc] if bcc.is_a?(String)
45 |
46 | self.options = opts if opts.any?
47 | mail = Package.build(message)
48 | response = lookup_and_deliver(mail, bcc)
49 | return response, mail
50 | end
51 |
52 | # same as deliver but raises error. used by ActionMailer
53 | def deliver!(mail, opts = {})
54 | @logger = Rails.logger if defined?(Rails) and !config[:log_to]
55 | resp, email = deliver(mail, opts)
56 | if resp.is_a?(Exception)
57 | raise resp
58 | else
59 | return resp, email
60 | end
61 | end
62 |
63 | def deliver_many(messages, opts = {})
64 | raise ArgumentError, "Not an array of messages: #{messages.inspect}" unless messages.any?
65 | self.options = opts if opts.any?
66 | messages_by_domain = reorder_by_domain(messages)
67 | lookup_and_deliver_many(messages_by_domain)
68 | end
69 |
70 | def options=(hash)
71 | if dkim_opts = hash.delete(:dkim)
72 | self.dkim = dkim_opts
73 | end
74 | config.merge!(hash)
75 | end
76 |
77 | def dkim=(dkim_opts)
78 | Dkim::domain = dkim_opts[:domain]
79 | Dkim::selector = dkim_opts[:selector]
80 | Dkim::private_key = dkim_opts[:private_key]
81 | end
82 |
83 | private
84 |
85 | def config
86 | @config ||= DEFAULTS
87 | end
88 |
89 | def use_dkim?
90 | !Dkim::domain.nil?
91 | end
92 |
93 | def logger
94 | @logger ||= Logger.new(config[:log_to])
95 | end
96 |
97 | def get_domain(email_address)
98 | email_address && email_address.to_s[/@([a-z0-9\._-]+)/i, 1]
99 | end
100 |
101 | def reorder_by_domain(array)
102 | hash = {}
103 | array.each_with_index do |message, i|
104 | mail = Package.build(message, i)
105 | if mail.destinations.count != 1
106 | raise ArgumentError, "Invalid destination count: #{mail.destinations.count}"
107 | end
108 |
109 | if to = mail.destinations.first and domain = get_domain(to)
110 | domain = domain.downcase
111 | hash[domain] = [] if hash[domain].nil?
112 | hash[domain].push(mail)
113 | end
114 | end
115 | hash
116 | end
117 |
118 | def smtp_servers_for_domain(domain)
119 | unless servers = cache.get(domain)
120 | if servers = DNS.get_mx(domain) and servers.any?
121 | cache.set(domain, servers)
122 | end
123 | end
124 | servers.any? && servers
125 | end
126 |
127 | def lookup_and_deliver(mail, bcc = [])
128 | if mail.destinations.empty?
129 | raise "No destinations found! You need to pass a :to field."
130 | end
131 |
132 | response = nil
133 | mail.destinations.each do |to|
134 |
135 | domain = get_domain(to)
136 | raise "Empty domain: #{domain}" if domain.to_s.strip == ''
137 |
138 | unless servers = smtp_servers_for_domain(domain)
139 | return HardBounce.new("588 No MX records for domain #{domain}")
140 | end
141 |
142 | last_error = nil
143 | servers.each do |server|
144 | begin
145 | response = send_now(mail, server, to, bcc)
146 | break
147 | rescue Exception => e # explicitly rescue Exception so we catch Timeout:Error's too
148 | logger.error "Error: #{e}"
149 | last_error = e
150 | end
151 | end
152 | return Bounce.type(last_error) if last_error
153 | end
154 | response
155 | end
156 |
157 | def lookup_and_deliver_many(by_domain)
158 | if config[:max_workers] && config[:max_workers] != 0
159 | lookup_and_deliver_many_threaded(by_domain)
160 | else
161 | lookup_and_deliver_many_sync(by_domain)
162 | end
163 | end
164 |
165 | def lookup_and_deliver_many_threaded(by_domain)
166 | count = config[:max_workers].is_a?(Integer) ? config[:max_workers] : nil
167 | queue = WorkQueue.new(count)
168 | responses = []
169 |
170 | logger.info("Delivering emails to #{by_domain.keys.count} domains...")
171 | by_domain.each do |domain, mails|
172 | queue.enqueue_b(domain, mails) do |domain, mails|
173 | # send emails and then assign responses to array according to mail index
174 | rr = lookup_and_deliver_by_domain(domain, mails)
175 | rr.each do |resp, mail|
176 | responses[mail.array_index] = [resp, mail]
177 | end
178 | end # worker
179 | end
180 |
181 | queue.join # wait for threads to finish
182 | queue.kill # terminate queue
183 | responses
184 | end
185 |
186 | def lookup_and_deliver_many_sync(by_domain)
187 | responses = []
188 |
189 | logger.info("Delivering emails to #{by_domain.keys.count} domains...")
190 | by_domain.each do |domain, mails|
191 | # send emails and then assign responses to array according to mail index
192 | rr = lookup_and_deliver_by_domain(domain, mails)
193 | rr.each do |resp, mail|
194 | responses[mail.array_index] = [resp, mail]
195 | end
196 | end
197 | responses
198 | end
199 |
200 | def lookup_and_deliver_by_domain(domain, mails)
201 | responses = []
202 | total = mails.count
203 |
204 | unless servers = smtp_servers_for_domain(domain)
205 | err = HardBounce.new("588 No MX Records for domain #{domain}")
206 | mails.each { |mail| responses.push [err, mail] }
207 | return responses
208 | end
209 |
210 | servers.each do |server|
211 | send_many_now(server, mails).each do |mail, resp|
212 | responses.push [resp, mail]
213 | mails.delete(mail) # remove it from list, to avoid duplicate delivery
214 | end
215 | logger.info "#{responses.count}/#{total} mails processed on #{domain}'s MX: #{server}."
216 | break if responses.count == total
217 | end
218 |
219 | # if we still have emails in queue, mark them with the last error which prevented delivery
220 | if mails.any? and @last_error
221 | bounce = Bounce.type(@last_error)
222 | logger.info "#{mails.count} mails still pending. Marking as #{bounce.class}..."
223 | mails.each { |m| responses.push [bounce, m] }
224 | end
225 |
226 | responses
227 | end
228 |
229 | def send_now(mail, server, to, bcc = [])
230 | logger.info "#{to} - Delivering email at #{server}..."
231 | logger.info "Including these destinations: #{bcc.inspect}" if bcc && bcc.any?
232 | from = get_from(mail)
233 |
234 | response = nil
235 | socket = init_connection(server)
236 | socket.start(get_helo_domain(from), nil, nil, nil) do |smtp|
237 | response = smtp.send_message(get_raw_mail(mail), from, to, *bcc)
238 | logger.info "#{to} - [SENT] #{response.message.strip}"
239 | end
240 |
241 | response
242 | end
243 |
244 | def send_many_now(server, mails)
245 | logger.info "Delivering #{mails.count} mails at #{server}..."
246 | responses = {}
247 |
248 | socket = init_connection(server)
249 | socket.start(get_helo_domain, nil, nil, nil) do |smtp|
250 | mails.each do |mail|
251 | begin
252 | resp = smtp.send_message(get_raw_mail(mail), get_from(mail), mail.to)
253 | smtp.send(:getok, 'RSET') if server['hotmail'] # fix for '503 Sender already specified'
254 | rescue Net::SMTPFatalError, Net::SMTPServerBusy => e # error code 5xx, except for 500, like: 550 Mailbox not found
255 | resp = Bounce.type(e)
256 | end
257 | responses[mail] = resp
258 | logger.info "#{mail.to} [#{responses[mail].class}] #{responses[mail].message.strip}" # both error and response have this method
259 | end
260 | end
261 |
262 | responses
263 | rescue Exception => e # SMTPServerBusy, SMTPSyntaxError, SMTPUnsupportedCommand, SMTPUnknownError (unexpected reply code)
264 | logger.error "[SERVER ERROR: #{server}] #{e.class} -> #{e.message}"
265 | @last_error = e
266 | responses
267 | end
268 |
269 | def get_raw_mail(mail)
270 | use_dkim? ? Dkim.sign(mail.to_s).to_s : mail.to_s
271 | end
272 |
273 | def get_from(mail)
274 | mail.return_path || mail.sender || mail.from_addrs.first
275 | end
276 |
277 | def get_helo_domain(from = nil)
278 | Dkim::domain || config[:helo_domain] || (from && get_domain(from))
279 | end
280 |
281 | def init_connection(server)
282 | context = OpenSSL::SSL::SSLContext.new
283 | context.verify_mode = config[:verify_ssl] ?
284 | OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE
285 | context.set_params # Configures default certificate store
286 |
287 | port = nil
288 | if config[:debug]
289 | if config[:debug].is_a?(String)
290 | server = config[:debug].split(':').first
291 | port = config[:debug].split(':').last if config[:debug][':']
292 | else
293 | server = 'localhost'
294 | end
295 | logger.warn "Debug option enabled. Connecting to #{server}!"
296 | end
297 |
298 | smtp = Net::SMTP.new(server, port)
299 | smtp.enable_starttls_auto(context)
300 | smtp.read_timeout = config[:read_timeout] if config[:read_timeout]
301 | smtp.open_timeout = config[:open_timeout] if config[:open_timeout]
302 | smtp
303 | end
304 |
305 | end
306 |
307 | end
308 |
--------------------------------------------------------------------------------
/lib/tuktuk/version.rb:
--------------------------------------------------------------------------------
1 | module Tuktuk
2 | MAJOR = 0
3 | MINOR = 9
4 | PATCH = 0
5 |
6 | VERSION = [MAJOR, MINOR, PATCH].join('.')
7 | end
8 |
--------------------------------------------------------------------------------
/spec/deliver_spec.rb:
--------------------------------------------------------------------------------
1 | require './lib/tuktuk/tuktuk'
2 | require 'rspec/mocks'
3 |
4 | def email(attrs = {})
5 | { :to => "user#{rand(1000)}@domain.com",
6 | :from => 'me@company.com',
7 | :subject => 'Test email',
8 | :body => 'Hello world.'
9 | }.merge(attrs)
10 | end
11 |
12 | describe 'deliver' do
13 |
14 | before(:each) do
15 | @mock_smtp = double('Net::SMTP', enable_starttls_auto: true, :read_timeout= => true, :open_timeout= => true)
16 | @mock_conn = double('SMTP Connection')
17 | @mock_smtp.stub(:start).and_yield(@mock_conn)
18 | @mock_resp = double('SMTP::Response', message: '250 OK')
19 |
20 | Net::SMTP.stub(:new).and_return(@mock_smtp)
21 | end
22 |
23 | describe 'single recipient' do
24 |
25 | describe 'when destination is valid (has MX servers)' do
26 |
27 | before do
28 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com']
29 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers)
30 | end
31 |
32 | it 'sends message' do
33 | msg = email
34 | expect(@mock_conn).to receive(:send_message).with(String, msg[:from], msg[:to]).and_return(@mock_resp)
35 | Tuktuk.deliver(msg)
36 | end
37 |
38 | describe 'and bcc is given' do
39 |
40 | let(:bcc_email) { 'bcc@test.com' }
41 |
42 | it 'includes it in destination list' do
43 | msg = email
44 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to], bcc_email).and_return(@mock_resp)
45 | Tuktuk.deliver(msg, bcc: [bcc_email])
46 | end
47 |
48 | it 'also works if not passed as array' do
49 | msg = email
50 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to], bcc_email).and_return(@mock_resp)
51 | Tuktuk.deliver(msg, bcc: bcc_email)
52 | end
53 |
54 | end
55 |
56 | end
57 |
58 | end
59 |
60 | describe 'multiple recipients (string list)' do
61 |
62 | describe 'when destination is valid (has MX servers)' do
63 |
64 | before do
65 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com']
66 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers)
67 | end
68 |
69 | it 'sends message' do
70 | msg = email(to: 'some@one.com, another@one.com')
71 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first).and_return(@mock_resp)
72 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last).and_return(@mock_resp)
73 | Tuktuk.deliver(msg)
74 | end
75 |
76 | describe 'and bcc is given' do
77 |
78 | let(:bcc_email) { 'bcc@test.com' }
79 |
80 | it 'includes it in destination list' do
81 | msg = email(to: 'some@one.com, another@one.com')
82 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first, bcc_email).and_return(@mock_resp)
83 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last, bcc_email).and_return(@mock_resp)
84 | Tuktuk.deliver(msg, bcc: [bcc_email])
85 | end
86 |
87 | it 'also works if not passed as array' do
88 | msg = email(to: 'some@one.com, another@one.com')
89 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').first, bcc_email).and_return(@mock_resp)
90 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].split(', ').last, bcc_email).and_return(@mock_resp)
91 | Tuktuk.deliver(msg, bcc: bcc_email)
92 | end
93 |
94 | end
95 |
96 | end
97 |
98 | end
99 |
100 | describe 'multiple recipients (array)' do
101 |
102 | describe 'when destination is valid (has MX servers)' do
103 |
104 | before do
105 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com']
106 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers)
107 | end
108 |
109 | it 'sends message' do
110 | msg = email(to: ['some@one.com', 'another@one.com'])
111 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first).and_return(@mock_resp)
112 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last).and_return(@mock_resp)
113 | Tuktuk.deliver(msg)
114 | end
115 |
116 | describe 'and bcc is given' do
117 |
118 | let(:bcc_email) { 'bcc@test.com' }
119 |
120 | it 'includes it in destination list' do
121 | msg = email(to: ['some@one.com', 'another@one.com'])
122 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first, bcc_email).and_return(@mock_resp)
123 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last, bcc_email).and_return(@mock_resp)
124 | Tuktuk.deliver(msg, bcc: [bcc_email])
125 | end
126 |
127 | it 'also works if not passed as array' do
128 | msg = email(to: ['some@one.com', 'another@one.com'])
129 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].first, bcc_email).and_return(@mock_resp)
130 | expect(@mock_conn).to receive(:send_message).with(instance_of(String), msg[:from], msg[:to].last, bcc_email).and_return(@mock_resp)
131 | Tuktuk.deliver(msg, bcc: bcc_email)
132 | end
133 |
134 | end
135 |
136 | end
137 |
138 | end
139 |
140 | end
141 |
142 | describe 'deliver many' do
143 |
144 | before(:each) do
145 | @mock_smtp = double('Net::SMTP')
146 | Net::SMTP.stub(:new).and_return(@mock_smtp)
147 | end
148 |
149 | describe 'when no emails are passed' do
150 |
151 | it 'raises' do
152 | lambda do
153 | Tuktuk.deliver_many []
154 | end.should raise_error(ArgumentError)
155 | end
156 |
157 | end
158 |
159 | describe 'when one email contains multiple addresses' do
160 |
161 | it 'raises' do
162 | lambda do
163 | Tuktuk.deliver_many [ email, email(:to => 'one@user.com, two@user.com') ]
164 | end.should raise_error(ArgumentError)
165 | end
166 |
167 | end
168 |
169 | describe 'when emails are valid' do
170 |
171 | it 'groups them by domain' do
172 |
173 | end
174 |
175 | describe 'and max_workers is 0' do
176 |
177 | it 'does not start any threads' do
178 |
179 | end
180 |
181 | end
182 |
183 | describe 'and max_workers is >0' do
184 |
185 | it 'does not spawn any more threads than the max allowed' do
186 |
187 | end
188 |
189 | end
190 |
191 | describe 'and max workers is auto' do
192 |
193 | it 'spawns a new thread for each domain' do
194 |
195 | end
196 |
197 | end
198 |
199 | describe 'when delivering to domain' do
200 |
201 | before do
202 | @mock_smtp.stub(:start).and_yield('foo')
203 | @emails = [email, email, email]
204 |
205 | @success = double('Net::SMTP::Response')
206 | @soft_email_bounce = Tuktuk::SoftBounce.new('503 Sender already specified')
207 | @hard_email_bounce = Tuktuk::HardBounce.new('505 Mailbox not found')
208 | @soft_server_bounce = Tuktuk::SoftBounce.new('Be back in a sec')
209 | @hard_server_bounce = Tuktuk::HardBounce.new('No MX records found.')
210 | end
211 |
212 | describe 'when domain exists' do
213 |
214 | before do
215 | @domain = 'domain.com'
216 | end
217 |
218 | describe 'and has valid MX servers' do
219 |
220 | before do
221 | @servers = ['mx1.domain.com', 'mx2.domain.com', 'mx3.domain.com']
222 | Tuktuk.stub(:smtp_servers_for_domain).and_return(@servers)
223 | end
224 |
225 | it 'starts by delivering to first one' do
226 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', [1]).and_return([[1, 'ok']])
227 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', [1])
228 | end
229 |
230 | describe 'and first server processes all our mail' do
231 |
232 | describe 'and all mail goes through' do
233 |
234 | before do
235 | @responses = []
236 | @emails.each { |e| @responses.push [e, @success] }
237 | end
238 |
239 | it 'does not try to connect to second server' do
240 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return(@responses)
241 | Tuktuk.should_not_receive(:send_many_now).with('mx2.domain.com')
242 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
243 | end
244 |
245 | end
246 |
247 | describe 'and all emails were hard failures (bounces)' do
248 |
249 | before do
250 | @responses = []
251 | @emails.each { |e| @responses.push [e, @hard_email_bounce] }
252 | end
253 |
254 | it 'does not try to connect to second server' do
255 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return(@responses)
256 | Tuktuk.should_not_receive(:send_many_now).with('mx2.domain.com')
257 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
258 | end
259 |
260 | end
261 |
262 | end
263 |
264 | describe 'and first server is down' do
265 |
266 | before do
267 | @responses = []
268 | @emails.each { |e| @responses.push [e, @success] }
269 | end
270 |
271 | it 'does not raise error' do
272 | Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_raise('Unable to connect.')
273 | Tuktuk.should_receive(:init_connection).once.with('mx2.domain.com').and_raise('Unable to connect.')
274 | Tuktuk.should_receive(:init_connection).once.with('mx3.domain.com').and_raise('Unable to connect.')
275 | # Tuktuk.should_receive(:init_connection).once.and_raise('Unable to connect.')
276 | expect do
277 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
278 | end.not_to raise_error # (RuntimeError)
279 | end
280 |
281 | it 'returns empty responses' do
282 | Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_raise('Unable to connect.')
283 | responses = Tuktuk.send(:send_many_now, 'mx1.domain.com', @emails)
284 | responses.should be_empty
285 | end
286 |
287 | it 'tries to connect to second server' do
288 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return([])
289 | Tuktuk.should_receive(:send_many_now).once.with('mx2.domain.com', @emails).and_return(@responses)
290 | Tuktuk.should_not_receive(:send_many_now).with('mx3.domain.com')
291 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
292 | end
293 |
294 | end
295 |
296 | describe 'and first server receives only one email' do
297 |
298 | before do
299 | @first = [@emails[0], @success]
300 | @last_two = [[@emails[1], @success], [@emails[2], @soft_email_bounce]]
301 | end
302 |
303 | it 'does not try to send that same email to second server' do
304 | Tuktuk.should_receive(:send_many_now).once.with('mx1.domain.com', @emails).and_return([@first])
305 | last_two_emails = @emails.last(2)
306 | last_two_emails.include?(@emails.first).should be false
307 | Tuktuk.should_receive(:send_many_now).once.with('mx2.domain.com', last_two_emails).and_return(@last_two)
308 | Tuktuk.should_not_receive(:send_many_now).with('mx3.domain.com')
309 | Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
310 | end
311 |
312 | describe 'and other servers are down' do
313 |
314 | before do
315 | # TODO: for some reason the :init_connection on line 138 is affecting this
316 | # this test should pass when running on its own
317 | # Tuktuk.should_receive(:init_connection).once.with('mx1.domain.com').and_return(@mock_smtp)
318 | # Tuktuk.should_receive(:init_connection).once.with('mx2.domain.com').and_raise('Unable to connect.')
319 | # Tuktuk.should_receive(:init_connection).once.with('mx3.domain.com').and_raise('Unable to connect.')
320 | end
321 |
322 | it 'should not mark first email as bounced' do
323 | Tuktuk.should_receive(:send_many_now).and_return([@first], [], [])
324 | responses = Tuktuk.send(:lookup_and_deliver_by_domain, 'domain.com', @emails)
325 | responses[1][0].should be_a(Tuktuk::Bounce) if responses[1]
326 | responses[0][0].should_not be_a(Tuktuk::Bounce)
327 | end
328 |
329 | end
330 |
331 | end
332 |
333 | end
334 |
335 | end
336 |
337 | end
338 |
339 | end
340 |
341 | end
342 |
--------------------------------------------------------------------------------
/tuktuk.gemspec:
--------------------------------------------------------------------------------
1 | # -*- encoding: utf-8 -*-
2 | require File.expand_path("../lib/tuktuk/version", __FILE__)
3 |
4 | Gem::Specification.new do |s|
5 | s.name = "tuktuk"
6 | s.version = Tuktuk::VERSION
7 | s.platform = Gem::Platform::RUBY
8 | s.authors = ['Tomás Pollak']
9 | s.email = ['tomas@forkhq.com']
10 | s.homepage = "https://github.com/tomas/tuktuk"
11 | s.summary = "SMTP client for Ruby with DKIM support."
12 | s.description = "Easy way of sending DKIM-signed emails from Ruby, no dependencies needed."
13 |
14 | s.required_rubygems_version = ">= 1.3.6"
15 | s.rubyforge_project = "tuktuk"
16 |
17 | s.add_development_dependency "bundler", ">= 1.0.0"
18 | s.add_runtime_dependency "net-dns", "= 0.6.1"
19 | s.add_runtime_dependency "mail", "~> 2.3"
20 | s.add_runtime_dependency "dkim", "~> 0.0.2"
21 | s.add_runtime_dependency 'work_queue', '~> 2.5.0'
22 |
23 | s.files = `git ls-files`.split("\n")
24 | s.executables = `git ls-files`.split("\n").map{|f| f =~ /^bin\/(.*)/ ? $1 : nil}.compact
25 | s.require_path = 'lib'
26 | end
27 |
--------------------------------------------------------------------------------