├── .github └── CODEOWNERS ├── .gitignore ├── Gemfile ├── LICENSE ├── README.md ├── Rakefile ├── clearbit.gemspec ├── examples ├── company.rb ├── discovery.rb ├── enrichment.rb ├── logo.rb ├── name_domain.rb ├── person.rb ├── prospector.rb ├── reveal.rb ├── risk.rb ├── risk_flag.rb ├── version.rb └── watchlist.rb ├── lib ├── clearbit.rb └── clearbit │ ├── analytics.rb │ ├── analytics │ ├── LICENSE │ ├── README.md │ ├── backoff_policy.rb │ ├── client.rb │ ├── defaults.rb │ ├── field_parser.rb │ ├── logging.rb │ ├── message_batch.rb │ ├── request.rb │ ├── response.rb │ ├── utils.rb │ └── worker.rb │ ├── audiences.rb │ ├── autocomplete.rb │ ├── base.rb │ ├── discovery.rb │ ├── enrichment.rb │ ├── enrichment │ ├── company.rb │ ├── news.rb │ ├── person.rb │ └── person_company.rb │ ├── errors │ └── invalid_webhook_signature.rb │ ├── logo.rb │ ├── mash.rb │ ├── name_domain.rb │ ├── pending.rb │ ├── prospector.rb │ ├── resource.rb │ ├── reveal.rb │ ├── risk.rb │ ├── version.rb │ ├── watchlist.rb │ └── webhook.rb └── spec ├── lib └── clearbit │ ├── analytics_spec.rb │ ├── discovery_spec.rb │ ├── enrichment_spec.rb │ ├── logo_spec.rb │ ├── prospector_spec.rb │ └── webhook_spec.rb ├── spec_helper.rb └── support └── helpers.rb /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | * @bentona 2 | * @jcutrell 3 | 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | *.rbc 3 | .bundle 4 | .config 5 | .yardoc 6 | Gemfile.lock 7 | InstalledFiles 8 | _yardoc 9 | coverage 10 | doc/ 11 | lib/bundler/man 12 | pkg 13 | rdoc 14 | spec/reports 15 | test/tmp 16 | test/version_tmp 17 | tmp 18 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | # Specify your gem's dependencies in clearbit.gemspec 4 | gemspec 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Clearbit. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## ⚠️ DEPRECATION WARNING 2 | 3 | This package is no longer being maintained. If you're looking to integrate with Clearbit's API we recommend looking at the HTTP requests available in our documentation at [clearbit.com/docs](https://clearbit.com/docs) 4 | 5 | # Clearbit 6 | 7 | A Ruby API client to [https://clearbit.com](https://clearbit.com). 8 | 9 | ## Installation 10 | 11 | Add this line to your application's Gemfile: 12 | 13 | ``` ruby 14 | gem 'clearbit' 15 | ``` 16 | 17 | And then execute: 18 | 19 | $ bundle 20 | 21 | Or install it yourself as: 22 | 23 | $ gem install clearbit 24 | 25 | ## Usage 26 | 27 | First authorize requests by setting the API key found on your [account's settings page](https://clearbit.com/keys). 28 | 29 | ``` ruby 30 | Clearbit.key = ENV['CLEARBIT_KEY'] 31 | ``` 32 | 33 | Then you can lookup people by email address: 34 | 35 | ``` ruby 36 | result = Clearbit::Enrichment.find(email: 'alex@alexmaccaw.com', stream: true) 37 | 38 | person = result.person 39 | company = result.company 40 | ``` 41 | 42 | Passing the `stream` option makes the operation blocking - it could hang for 4-5 seconds if we haven't seen the email before. Alternatively you can use our [webhook](https://clearbit.com/docs#webhooks) API. 43 | 44 | Without the `stream` option, the operation is non-blocking, and we will immediately return either the enriched data or `Clearbit::Pending` object. 45 | 46 | ```ruby 47 | result = Clearbit::Enrichment.find(email: 'alex@alexmaccaw.com') 48 | 49 | if result.pending? 50 | # Lookup queued - try again later 51 | end 52 | 53 | # Later 54 | unless result.pending? 55 | person = result.person 56 | company = result.company 57 | end 58 | 59 | ``` 60 | 61 | In either case, if a person or company can't be found, the result will be `nil`. 62 | 63 | See the [documentation](https://clearbit.com/docs#person-api) for more information. 64 | ## Name to Domain 65 | 66 | To find the domain based on the name of a resource, you can use the `NameDomain` API. 67 | 68 | ```ruby 69 | name = Clearbit::NameDomain.find(name: 'Uber') 70 | ``` 71 | For more information look at the [documentation](https://dashboard.clearbit.com/docs?ruby#name-to-domain-api). 72 | 73 | ## Company lookup 74 | 75 | You can lookup company data by domain name: 76 | 77 | ``` ruby 78 | company = Clearbit::Enrichment::Company.find(domain: 'uber.com', stream: true) 79 | ``` 80 | 81 | If the company can't be found, then `nil` will be returned. 82 | 83 | See the [documentation](https://clearbit.com/docs#company-api) for more information. 84 | 85 | ## Analytics 86 | 87 | *NOTE:* We **strongly** recommend using `clearbit.js` for Analytics and integrating with Clearbit X. It handles a lot of complexity, like generating `anonymous_id`s and associating them with `user_id`s when a user is identified. It also automatically tracks `page` views with the full data set. 88 | 89 | ### Identifying Users 90 | 91 | Identify users by sending their `user_id`, and adding details like their `email` and `company_domain` to create People and Companies inside of Clearbit X. 92 | 93 | ```ruby 94 | Clearbit::Analytics.identify( 95 | user_id: '1234', # Required if no anonymous_id is sent. The user's ID in your database. 96 | anonymous_id: session[:anonymous_id], # Required if no user_id is sent. A UUID to track anonymous users. 97 | traits: { 98 | email: 'david@clearbitexample.com', # Optional, but strongly recommended 99 | company_domain: 'clearbit.com', # Optional, but strongly recommended 100 | first_name: 'David', # Optional 101 | last_name: 'Lumley', # Optional 102 | # … other analytical traits can also be sent, like the plan a user is on etc 103 | }, 104 | context: { 105 | ip: '89.102.33.1' # Optional, but strongly recommended when identifying users 106 | } # as they sign up, or log in 107 | ) 108 | ``` 109 | 110 | ### Page Views 111 | 112 | Use the `page` method, and send the users `anonymous_id` along with the `url` they're viewing, and the `ip` the request comes from in order to create Companies inside of Clearbit X and track their page views. 113 | 114 | ```ruby 115 | Clearbit::Analytics.page( 116 | user_id: '1234', # Required if no anonymous_id is sent. The user's ID in your database. 117 | anonymous_id: session[:anonymous_id], # Required if no user_id is sent. A UUID to track anonymous users. 118 | name: 'Clearbit Ruby Library', # Optional, but strongly recommended 119 | properties: { 120 | url: 'https://github.com/clearbit/clearbit-ruby?utm_source=google', # Required. Likely to be request.referer 121 | path: '/clearbit/clearbit-ruby', # Optional, but strongly recommended 122 | search: '?utm_source=google', # Optional, but strongly recommended 123 | referrer: nil, # Optional. Unlikely to be request.referrer. 124 | }, 125 | context: { 126 | ip: '89.102.33.1', # Optional, but strongly recommended. 127 | }, 128 | ) 129 | ``` 130 | 131 | ## Other APIs 132 | 133 | For more info on our other APIs (such as the Watchlist or Discover APIs), please see our [main documentation](https://clearbit.com/docs). 134 | 135 | ## Webhooks 136 | 137 | For rack apps use the `Clearbit::Webhook` module to wrap deserialization and verify the webhook is from trusted party: 138 | 139 | ``` ruby 140 | post '/v1/webhooks/clearbit' do 141 | begin 142 | webhook = Clearbit::Webhook.new(request.env) 143 | webhook.type #=> 'person' 144 | webhook.body.name.given_name #=> 'Alex' 145 | 146 | # ... 147 | rescue Clearbit::Errors::InvalidWebhookSignature => e 148 | # ... 149 | end 150 | end 151 | ``` 152 | 153 | The global Clearbit.key can be overriden for multi-tenant apps using multiple Clearbit keys like so: 154 | 155 | ```ruby 156 | webhook = Clearbit::Webhook.new(request.env, 'CLEARBIT_KEY') 157 | ``` 158 | 159 | ## Proxy Support 160 | 161 | Passing the proxy option allows you to specify a proxy server to pass the request through. 162 | 163 | ``` ruby 164 | company = Clearbit::Enrichment::Company.find( 165 | domain: 'uber.com', 166 | proxy: 'https://user:password@proxyserver.tld:8080' 167 | ) 168 | ``` 169 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require "bundler/gem_tasks" 2 | 3 | begin 4 | require 'rspec/core/rake_task' 5 | 6 | RSpec::Core::RakeTask.new(:spec) 7 | 8 | task :default => :spec 9 | rescue LoadError 10 | # no rspec available 11 | end 12 | -------------------------------------------------------------------------------- /clearbit.gemspec: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | lib = File.expand_path('../lib', __FILE__) 3 | $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) 4 | require 'clearbit/version' 5 | 6 | Gem::Specification.new do |spec| 7 | spec.name = "clearbit" 8 | spec.version = Clearbit::VERSION 9 | spec.authors = ["Alex MacCaw"] 10 | spec.email = ["alex@clearbit.com"] 11 | spec.description = %q{API client for clearbit.com} 12 | spec.summary = %q{API client for clearbit.com} 13 | spec.homepage = "https://github.com/maccman/clearbit-ruby" 14 | spec.license = "MIT" 15 | 16 | spec.files = `git ls-files`.split($/) 17 | spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) } 18 | spec.test_files = spec.files.grep(%r{^(test|spec|features)/}) 19 | spec.require_paths = ["lib"] 20 | 21 | spec.add_dependency 'json', ['~> 1.7'] if RUBY_VERSION < '1.9' 22 | 23 | spec.add_development_dependency 'bundler' 24 | spec.add_development_dependency 'net-http-spy' 25 | spec.add_development_dependency 'pry' 26 | spec.add_development_dependency 'rake' 27 | spec.add_development_dependency 'rspec' 28 | spec.add_development_dependency 'rack' 29 | spec.add_development_dependency 'webmock' 30 | spec.add_dependency 'nestful', '~> 1.1.0' 31 | end 32 | -------------------------------------------------------------------------------- /examples/company.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Enrichment::Company.find(domain: 'stripe.com', stream: true) 5 | -------------------------------------------------------------------------------- /examples/discovery.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Discovery.search(query: {tech: 'marketo'}) 5 | -------------------------------------------------------------------------------- /examples/enrichment.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Enrichment.find(email: 'alex@alexmaccaw.com') 5 | -------------------------------------------------------------------------------- /examples/logo.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Logo.url(domain: 'clearbit.com') 5 | -------------------------------------------------------------------------------- /examples/name_domain.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::NameDomain.find(name: 'Stripe') 5 | -------------------------------------------------------------------------------- /examples/person.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Enrichment::Person.find(email: 'alex@alexmaccaw.com') 5 | -------------------------------------------------------------------------------- /examples/prospector.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | 3 | response = Clearbit::Prospector.search(domain: 'clearbit.com', page: 1) 4 | 5 | puts "Displaying #{response[:results].size} of #{response[:total]} results:" 6 | 7 | response[:results].each_with_index do |person, index| 8 | puts " #{index + 1}. #{person.name.full_name} (#{person.email})" 9 | end 10 | -------------------------------------------------------------------------------- /examples/reveal.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'csv' 3 | 4 | puts 'domain,fuzzy' 5 | 6 | STDIN.read.each_line do |line| 7 | result = Clearbit::Reveal.find(ip: line.strip) 8 | next puts unless result 9 | puts CSV.generate_line([result.domain, result.fuzzy]) 10 | end 11 | 12 | -------------------------------------------------------------------------------- /examples/risk.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | 3 | # Step 1) 4 | # Add the JS library + load it 5 | # https://clearbit.com/docs#risk-api-javascript-library 6 | 7 | # Step 2) 8 | result = Clearbit::Risk.calculate( 9 | email: 'test@example.com', 10 | ip: '0.0.0.0' 11 | ) 12 | 13 | p result 14 | -------------------------------------------------------------------------------- /examples/risk_flag.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | require 'optparse' 4 | 5 | options = {} 6 | OptionParser.new do |opts| 7 | opts.banner = "Usage: risk_flag.rb [options]" 8 | 9 | opts.on("-iIP", "--ip=IP", "IP address") do |ip| 10 | options[:ip] = ip 11 | end 12 | 13 | opts.on("-eEMAIL", "--email=EMAIL", "Email address") do |email| 14 | options[:email] = email 15 | end 16 | 17 | opts.on("-tTYPE", "--type=TYPE", "Type") do |type| 18 | options[:type] = type 19 | end 20 | 21 | opts.on("-h", "--help", "Prints this help") do 22 | puts opts 23 | exit 24 | end 25 | end.parse! 26 | 27 | options[:type] ||= 'spam' 28 | 29 | begin 30 | Clearbit::Risk.flag(options) 31 | rescue Nestful::Error => err 32 | pp err.decoded 33 | else 34 | puts 'Successfully flagged!' 35 | end 36 | -------------------------------------------------------------------------------- /examples/version.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | Clearbit::Enrichment::PersonCompany.version = '2015-05-30' 5 | 6 | p Clearbit::Enrichment::PersonCompany.find(email: 'alex@clearbit.com', webhook_url: 'http://requestb.in/18owk611') 7 | -------------------------------------------------------------------------------- /examples/watchlist.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit' 2 | require 'pp' 3 | 4 | pp Clearbit::Watchlist.search(name: 'Smith') -------------------------------------------------------------------------------- /lib/clearbit.rb: -------------------------------------------------------------------------------- 1 | require 'nestful' 2 | require 'clearbit/version' 3 | require 'clearbit/analytics' 4 | 5 | module Clearbit 6 | def self.api_key=(value) 7 | Base.key = value 8 | end 9 | 10 | def self.key=(value) 11 | Base.key = value 12 | end 13 | 14 | def self.key 15 | Base.key 16 | end 17 | 18 | def self.key! 19 | key || raise('Clearbit.key not set') 20 | end 21 | 22 | autoload :Audiences, 'clearbit/audiences' 23 | autoload :Autocomplete, 'clearbit/autocomplete' 24 | autoload :Base, 'clearbit/base' 25 | autoload :Discovery, 'clearbit/discovery' 26 | autoload :Enrichment, 'clearbit/enrichment' 27 | autoload :Logo, 'clearbit/logo' 28 | autoload :Mash, 'clearbit/mash' 29 | autoload :NameDomain, 'clearbit/name_domain' 30 | autoload :Pending, 'clearbit/pending' 31 | autoload :Prospector, 'clearbit/prospector' 32 | autoload :Resource, 'clearbit/resource' 33 | autoload :Reveal, 'clearbit/reveal' 34 | autoload :Risk, 'clearbit/risk' 35 | autoload :Watchlist, 'clearbit/watchlist' 36 | autoload :Webhook, 'clearbit/webhook' 37 | 38 | module Errors 39 | autoload :InvalidWebhookSignature, 'clearbit/errors/invalid_webhook_signature' 40 | end 41 | 42 | if clearbit_key = ENV['CLEARBIT_KEY'] 43 | Clearbit.key = clearbit_key 44 | end 45 | 46 | # Backwards compatibility 47 | Person = Enrichment::Person 48 | Company = Enrichment::Company 49 | end 50 | -------------------------------------------------------------------------------- /lib/clearbit/analytics.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit/analytics/defaults' 2 | require 'clearbit/analytics/utils' 3 | require 'clearbit/analytics/field_parser' 4 | require 'clearbit/analytics/client' 5 | require 'clearbit/analytics/worker' 6 | require 'clearbit/analytics/request' 7 | require 'clearbit/analytics/response' 8 | require 'clearbit/analytics/logging' 9 | 10 | module Clearbit 11 | class Analytics 12 | # Proxy identify through to a client instance, in order to keep the client 13 | # consistent with how the other Clearbit APIs are accessed 14 | def self.identify(args) 15 | analytics = new(write_key: Clearbit.key) 16 | analytics.identify(args) 17 | analytics.flush 18 | end 19 | 20 | # Proxy page through to a client instance, in order to keep the client 21 | # consistent with how the other Clearbit APIs are accessed 22 | def self.page(args) 23 | analytics = new(write_key: Clearbit.key) 24 | analytics.page(args) 25 | analytics.flush 26 | end 27 | 28 | # Proxy group through to a client instance, in order to keep the client 29 | # consistent with how the other Clearbit APIs are accessed 30 | def self.group(args) 31 | analytics = new(write_key: Clearbit.key) 32 | analytics.group(args) 33 | analytics.flush 34 | end 35 | 36 | # Initializes a new instance of {Clearbit::Analytics::Client}, to which all 37 | # method calls are proxied. 38 | # 39 | # @param options includes options that are passed down to 40 | # {Clearbit::Analytics::Client#initialize} 41 | # @option options [Boolean] :stub (false) If true, requests don't hit the 42 | # server and are stubbed to be successful. 43 | def initialize(options = {}) 44 | Request.stub = options[:stub] if options.has_key?(:stub) 45 | @client = Clearbit::Analytics::Client.new options 46 | end 47 | 48 | def method_missing(message, *args, &block) 49 | if @client.respond_to? message 50 | @client.send message, *args, &block 51 | else 52 | super 53 | end 54 | end 55 | 56 | def respond_to_missing?(method_name, include_private = false) 57 | @client.respond_to?(method_name) || super 58 | end 59 | 60 | include Logging 61 | end 62 | end 63 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2013 Segment Inc. 4 | Copyright (c) 2019 Clearbit. 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 7 | 8 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 9 | 10 | THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 11 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/README.md: -------------------------------------------------------------------------------- 1 | # Getting Started with Clearbit::Analytics 2 | 3 | ## Initialization 4 | 5 | Make sure you set your `Clearbit.key` as part of your apps initialization, or before you make use of `Clearbit::Analytics`. You can find your secret key at [https://dashboard.clearbit.com/api](https://dashboard.clearbit.com/api) 6 | 7 | `Clearbit.key = 'sk_…'`. 8 | 9 | We also recommend ensuring that every person has a randomly generated `anonymous_id` to aid tracking page events and assist in matching up anonymous traffic to signed in users. 10 | 11 | ```ruby 12 | class ApplicationController 13 | before_action :set_anonymous_id 14 | … 15 | 16 | def set_anonymous_id 17 | session[:anonymous_id] ||= SecureRandom.uuid 18 | end 19 | 20 | … 21 | end 22 | ``` 23 | 24 | ## Importing Users into X 25 | 26 | You'll want to loop through your users, and call `.identify` for each one, passing the users `id`, and their `email` along with any other traits you value. 27 | 28 | ```ruby 29 | User.find_in_batches do |users| 30 | users.each do |user| 31 | Clearbit::Analytics.identify( 32 | user_id: user.id, # Required. 33 | traits: { 34 | email: user.email, # Required. 35 | company_domain: user.domain || user.email.split('@').last, # Optional, strongly recommended. 36 | first_name: user.first_name, # Optional. 37 | last_name: user.last_name, # Optional. 38 | # … other analytical traits can also be sent, like the plan a user is on etc. 39 | }, 40 | ) 41 | end 42 | end 43 | ``` 44 | 45 | ## Identifying Users on sign up / log in 46 | 47 | Identifying users on sign up, or on log in will help Clearbit X associate the user with anonymous page views. 48 | 49 | ```ruby 50 | class SessionsController 51 | … 52 | 53 | def create 54 | … 55 | 56 | identify(current_user) 57 | 58 | … 59 | end 60 | 61 | private 62 | 63 | def identify(user) 64 | Clearbit::Analytics.identify( 65 | user_id: user.id, # Required 66 | anonymous_id: session[:anonymous_id], # Optional, strongly recommended. Helps Clearbit X associate with anonymous visits. 67 | traits: { 68 | email: user.email, # Required. 69 | company_domain: user.domain || user.email.split('@').last, # Optional, strongly recommended. 70 | first_name: user.first_name, # Optional. 71 | last_name: user.last_name, # Optional. 72 | # … other analytical traits can also be sent, like the plan a user is on etc. 73 | }, 74 | context: { 75 | ip: request.ip, # Optional, but strongly recommended. Helps Clearbit X associate with anonymous visits. 76 | }, 77 | ) 78 | end 79 | 80 | … 81 | end 82 | ``` 83 | 84 | ## Sending Page views 85 | 86 | Tracking page views is best done using a `anonymous_id` you generate for each user. You'll need to sent the `url` as part of the `properties` hash, and the `ip` as part of the `context` hash. Any other data you can provide will greatly help with segmentation inside of Clearbit X. 87 | 88 | 89 | ```ruby 90 | uri = URI(request.referer) # You'll likely want to do some error handling here. 91 | 92 | Clearbit::Analytics.page( 93 | user_id: current_user&.id, # Optional 94 | anonymous_id: session[:anonymous_id], # Required 95 | properties: { 96 | url: request.referrer, # Required. Because this is a backend integration, the referrer is the URL that was visited. 97 | path: format_path(uri.path), # Optional, but strongly recommended. 98 | search: format_search(uri.params), # Optional, but strongly recommended. 99 | referrer: nil, # Optional. Not the `request.referer`, but the referrer of the original page view. 100 | }, 101 | context: { 102 | ip: request.ip, # Required. Helps Clearbit X associate anonymous visits with Companies. 103 | }, 104 | ) 105 | 106 | def format_path(path) 107 | path.presence || '/' 108 | end 109 | 110 | def format_search(search) 111 | return unless search 112 | '?' + search 113 | end 114 | ``` 115 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/backoff_policy.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit/analytics/defaults' 2 | 3 | module Clearbit 4 | class Analytics 5 | class BackoffPolicy 6 | include Clearbit::Analytics::Defaults::BackoffPolicy 7 | 8 | # @param [Hash] opts 9 | # @option opts [Numeric] :min_timeout_ms The minimum backoff timeout 10 | # @option opts [Numeric] :max_timeout_ms The maximum backoff timeout 11 | # @option opts [Numeric] :multiplier The value to multiply the current 12 | # interval with for each retry attempt 13 | # @option opts [Numeric] :randomization_factor The randomization factor 14 | # to use to create a range around the retry interval 15 | def initialize(opts = {}) 16 | @min_timeout_ms = opts[:min_timeout_ms] || MIN_TIMEOUT_MS 17 | @max_timeout_ms = opts[:max_timeout_ms] || MAX_TIMEOUT_MS 18 | @multiplier = opts[:multiplier] || MULTIPLIER 19 | @randomization_factor = opts[:randomization_factor] || RANDOMIZATION_FACTOR 20 | 21 | @attempts = 0 22 | end 23 | 24 | # @return [Numeric] the next backoff interval, in milliseconds. 25 | def next_interval 26 | interval = @min_timeout_ms * (@multiplier**@attempts) 27 | interval = add_jitter(interval, @randomization_factor) 28 | 29 | @attempts += 1 30 | 31 | [interval, @max_timeout_ms].min 32 | end 33 | 34 | private 35 | 36 | def add_jitter(base, randomization_factor) 37 | random_number = rand 38 | max_deviation = base * randomization_factor 39 | deviation = random_number * max_deviation 40 | 41 | if random_number < 0.5 42 | base - deviation 43 | else 44 | base + deviation 45 | end 46 | end 47 | end 48 | end 49 | end 50 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/client.rb: -------------------------------------------------------------------------------- 1 | require 'thread' 2 | require 'time' 3 | 4 | require 'clearbit/analytics/defaults' 5 | require 'clearbit/analytics/logging' 6 | require 'clearbit/analytics/utils' 7 | require 'clearbit/analytics/worker' 8 | 9 | module Clearbit 10 | class Analytics 11 | class Client 12 | include Clearbit::Analytics::Utils 13 | include Clearbit::Analytics::Logging 14 | 15 | # @param [Hash] opts 16 | # @option opts [String] :write_key Your project's write_key 17 | # @option opts [FixNum] :max_queue_size Maximum number of calls to be 18 | # remain queued. 19 | # @option opts [Proc] :on_error Handles error calls from the API. 20 | def initialize(opts = {}) 21 | symbolize_keys!(opts) 22 | 23 | @queue = Queue.new 24 | @write_key = opts[:write_key] 25 | @max_queue_size = opts[:max_queue_size] || Defaults::Queue::MAX_SIZE 26 | @worker_mutex = Mutex.new 27 | @worker = Worker.new(@queue, @write_key, opts) 28 | 29 | check_write_key! 30 | 31 | at_exit { @worker_thread && @worker_thread[:should_exit] = true } 32 | end 33 | 34 | # Synchronously waits until the worker has flushed the queue. 35 | # 36 | # Use only for scripts which are not long-running, and will specifically 37 | # exit 38 | def flush 39 | while !@queue.empty? || @worker.is_requesting? 40 | ensure_worker_running 41 | sleep(0.1) 42 | end 43 | end 44 | 45 | # @!macro common_attrs 46 | # @option attrs [String] :anonymous_id ID for a user when you don't know 47 | # who they are yet. (optional but you must provide either an 48 | # `anonymous_id` or `user_id`) 49 | # @option attrs [Hash] :context ({}) 50 | # @option attrs [Hash] :integrations What integrations this event 51 | # goes to (optional) 52 | # @option attrs [String] :message_id ID that uniquely 53 | # identifies a message across the API. (optional) 54 | # @option attrs [Time] :timestamp When the event occurred (optional) 55 | # @option attrs [String] :user_id The ID for this user in your database 56 | # (optional but you must provide either an `anonymous_id` or `user_id`) 57 | # @option attrs [Hash] :options Options such as user traits (optional) 58 | 59 | # Tracks an event 60 | # 61 | # @see https://segment.com/docs/sources/server/ruby/#track 62 | # 63 | # @param [Hash] attrs 64 | # 65 | # @option attrs [String] :event Event name 66 | # @option attrs [Hash] :properties Event properties (optional) 67 | # @macro common_attrs 68 | def track(attrs) 69 | symbolize_keys! attrs 70 | enqueue(FieldParser.parse_for_track(attrs)) 71 | end 72 | 73 | # Identifies a user 74 | # 75 | # @see https://segment.com/docs/sources/server/ruby/#identify 76 | # 77 | # @param [Hash] attrs 78 | # 79 | # @option attrs [Hash] :traits User traits (optional) 80 | # @macro common_attrs 81 | def identify(attrs) 82 | symbolize_keys! attrs 83 | enqueue(FieldParser.parse_for_identify(attrs)) 84 | end 85 | 86 | # Aliases a user from one id to another 87 | # 88 | # @see https://segment.com/docs/sources/server/ruby/#alias 89 | # 90 | # @param [Hash] attrs 91 | # 92 | # @option attrs [String] :previous_id The ID to alias from 93 | # @macro common_attrs 94 | def alias(attrs) 95 | symbolize_keys! attrs 96 | enqueue(FieldParser.parse_for_alias(attrs)) 97 | end 98 | 99 | # Associates a user identity with a group. 100 | # 101 | # @see https://segment.com/docs/sources/server/ruby/#group 102 | # 103 | # @param [Hash] attrs 104 | # 105 | # @option attrs [String] :group_id The ID of the group 106 | # @option attrs [Hash] :traits User traits (optional) 107 | # @macro common_attrs 108 | def group(attrs) 109 | symbolize_keys! attrs 110 | enqueue(FieldParser.parse_for_group(attrs)) 111 | end 112 | 113 | # Records a page view 114 | # 115 | # @see https://segment.com/docs/sources/server/ruby/#page 116 | # 117 | # @param [Hash] attrs 118 | # 119 | # @option attrs [String] :name Name of the page 120 | # @option attrs [Hash] :properties Page properties (optional) 121 | # @macro common_attrs 122 | def page(attrs) 123 | symbolize_keys! attrs 124 | enqueue(FieldParser.parse_for_page(attrs)) 125 | end 126 | 127 | # Records a screen view (for a mobile app) 128 | # 129 | # @param [Hash] attrs 130 | # 131 | # @option attrs [String] :name Name of the screen 132 | # @option attrs [Hash] :properties Screen properties (optional) 133 | # @option attrs [String] :category The screen category (optional) 134 | # @macro common_attrs 135 | def screen(attrs) 136 | symbolize_keys! attrs 137 | enqueue(FieldParser.parse_for_screen(attrs)) 138 | end 139 | 140 | # @return [Fixnum] number of messages in the queue 141 | def queued_messages 142 | @queue.length 143 | end 144 | 145 | private 146 | 147 | # private: Enqueues the action. 148 | # 149 | # returns Boolean of whether the item was added to the queue. 150 | def enqueue(action) 151 | # add our request id for tracing purposes 152 | action[:messageId] ||= uid 153 | 154 | if @queue.length < @max_queue_size 155 | @queue << action 156 | ensure_worker_running 157 | 158 | true 159 | else 160 | logger.warn( 161 | 'Queue is full, dropping events. The :max_queue_size ' \ 162 | 'configuration parameter can be increased to prevent this from ' \ 163 | 'happening.' 164 | ) 165 | false 166 | end 167 | end 168 | 169 | # private: Checks that the write_key is properly initialized 170 | def check_write_key! 171 | raise ArgumentError, 'Write key must be initialized' if @write_key.nil? 172 | end 173 | 174 | def ensure_worker_running 175 | return if worker_running? 176 | @worker_mutex.synchronize do 177 | return if worker_running? 178 | @worker_thread = Thread.new do 179 | @worker.run 180 | end 181 | end 182 | end 183 | 184 | def worker_running? 185 | @worker_thread && @worker_thread.alive? 186 | end 187 | end 188 | end 189 | end 190 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/defaults.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Analytics 3 | module Defaults 4 | module Request 5 | HOST = 'x.clearbit.com' 6 | PORT = 443 7 | PATH = '/v1/import' 8 | SSL = true 9 | HEADERS = { 'Accept' => 'application/json', 10 | 'Content-Type' => 'application/json', 11 | 'User-Agent' => "clearbit-ruby/#{Clearbit::VERSION}" } 12 | RETRIES = 10 13 | end 14 | 15 | module Queue 16 | MAX_SIZE = 10000 17 | end 18 | 19 | module Message 20 | MAX_BYTES = 32768 # 32Kb 21 | end 22 | 23 | module MessageBatch 24 | MAX_BYTES = 512_000 # 500Kb 25 | MAX_SIZE = 100 26 | end 27 | 28 | module BackoffPolicy 29 | MIN_TIMEOUT_MS = 100 30 | MAX_TIMEOUT_MS = 10000 31 | MULTIPLIER = 1.5 32 | RANDOMIZATION_FACTOR = 0.5 33 | end 34 | end 35 | end 36 | end 37 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/field_parser.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Analytics 3 | # Handles parsing fields according to the Segment Spec 4 | # 5 | # @see https://segment.com/docs/spec/ 6 | class FieldParser 7 | class << self 8 | include Clearbit::Analytics::Utils 9 | 10 | # In addition to the common fields, track accepts: 11 | # 12 | # - "event" 13 | # - "properties" 14 | def parse_for_track(fields) 15 | common = parse_common_fields(fields) 16 | 17 | event = fields[:event] 18 | properties = fields[:properties] || {} 19 | 20 | check_presence!(event, 'event') 21 | check_is_hash!(properties, 'properties') 22 | 23 | isoify_dates! properties 24 | 25 | common.merge({ 26 | :type => 'track', 27 | :event => event.to_s, 28 | :properties => properties 29 | }) 30 | end 31 | 32 | # In addition to the common fields, identify accepts: 33 | # 34 | # - "traits" 35 | def parse_for_identify(fields) 36 | common = parse_common_fields(fields) 37 | 38 | traits = fields[:traits] || {} 39 | check_is_hash!(traits, 'traits') 40 | isoify_dates! traits 41 | 42 | common.merge({ 43 | :type => 'identify', 44 | :traits => traits 45 | }) 46 | end 47 | 48 | # In addition to the common fields, alias accepts: 49 | # 50 | # - "previous_id" 51 | def parse_for_alias(fields) 52 | common = parse_common_fields(fields) 53 | 54 | previous_id = fields[:previous_id] 55 | check_presence!(previous_id, 'previous_id') 56 | 57 | common.merge({ 58 | :type => 'alias', 59 | :previousId => previous_id 60 | }) 61 | end 62 | 63 | # In addition to the common fields, group accepts: 64 | # 65 | # - "group_id" 66 | # - "traits" 67 | def parse_for_group(fields) 68 | common = parse_common_fields(fields) 69 | 70 | group_id = fields[:group_id] 71 | traits = fields[:traits] || {} 72 | 73 | check_presence!(group_id, 'group_id') 74 | check_is_hash!(traits, 'traits') 75 | 76 | isoify_dates! traits 77 | 78 | common.merge({ 79 | :type => 'group', 80 | :groupId => group_id, 81 | :traits => traits 82 | }) 83 | end 84 | 85 | # In addition to the common fields, page accepts: 86 | # 87 | # - "name" 88 | # - "properties" 89 | def parse_for_page(fields) 90 | common = parse_common_fields(fields) 91 | 92 | name = fields[:name] || '' 93 | properties = fields[:properties] || {} 94 | 95 | check_is_hash!(properties, 'properties') 96 | 97 | isoify_dates! properties 98 | 99 | common.merge({ 100 | :type => 'page', 101 | :name => name.to_s, 102 | :properties => properties 103 | }) 104 | end 105 | 106 | # In addition to the common fields, screen accepts: 107 | # 108 | # - "name" 109 | # - "properties" 110 | # - "category" (Not in spec, retained for backward compatibility" 111 | def parse_for_screen(fields) 112 | common = parse_common_fields(fields) 113 | 114 | name = fields[:name] 115 | properties = fields[:properties] || {} 116 | category = fields[:category] 117 | 118 | check_presence!(name, 'name') 119 | check_is_hash!(properties, 'properties') 120 | 121 | isoify_dates! properties 122 | 123 | parsed = common.merge({ 124 | :type => 'screen', 125 | :name => name, 126 | :properties => properties 127 | }) 128 | 129 | parsed[:category] = category if category 130 | 131 | parsed 132 | end 133 | 134 | private 135 | 136 | def parse_common_fields(fields) 137 | timestamp = fields[:timestamp] || Time.new 138 | message_id = fields[:message_id].to_s if fields[:message_id] 139 | context = fields[:context] || {} 140 | 141 | check_user_id! fields 142 | check_timestamp! timestamp 143 | 144 | add_context! context 145 | 146 | parsed = { 147 | :context => context, 148 | :messageId => message_id, 149 | :timestamp => datetime_in_iso8601(timestamp) 150 | } 151 | 152 | parsed[:userId] = fields[:user_id] if fields[:user_id] 153 | parsed[:anonymousId] = fields[:anonymous_id] if fields[:anonymous_id] 154 | parsed[:integrations] = fields[:integrations] if fields[:integrations] 155 | 156 | # Not in spec, retained for backward compatibility 157 | parsed[:options] = fields[:options] if fields[:options] 158 | 159 | parsed 160 | end 161 | 162 | def check_user_id!(fields) 163 | unless fields[:user_id] || fields[:anonymous_id] 164 | raise ArgumentError, 'Must supply either user_id or anonymous_id' 165 | end 166 | end 167 | 168 | def check_timestamp!(timestamp) 169 | raise ArgumentError, 'Timestamp must be a Time' unless timestamp.is_a? Time 170 | end 171 | 172 | def add_context!(context) 173 | context[:library] = { :name => 'clearbit-ruby', :version => Clearbit::VERSION.to_s } 174 | end 175 | 176 | # private: Ensures that a string is non-empty 177 | # 178 | # obj - String|Number that must be non-blank 179 | # name - Name of the validated value 180 | def check_presence!(obj, name) 181 | if obj.nil? || (obj.is_a?(String) && obj.empty?) 182 | raise ArgumentError, "#{name} must be given" 183 | end 184 | end 185 | 186 | def check_is_hash!(obj, name) 187 | raise ArgumentError, "#{name} must be a Hash" unless obj.is_a? Hash 188 | end 189 | end 190 | end 191 | end 192 | end 193 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/logging.rb: -------------------------------------------------------------------------------- 1 | require 'logger' 2 | 3 | module Clearbit 4 | class Analytics 5 | # Wraps an existing logger and adds a prefix to all messages 6 | class PrefixedLogger 7 | def initialize(logger, prefix) 8 | @logger = logger 9 | @prefix = prefix 10 | end 11 | 12 | def debug(msg) 13 | @logger.debug("#{@prefix} #{msg}") 14 | end 15 | 16 | def info(msg) 17 | @logger.info("#{@prefix} #{msg}") 18 | end 19 | 20 | def warn(msg) 21 | @logger.warn("#{@prefix} #{msg}") 22 | end 23 | 24 | def error(msg) 25 | @logger.error("#{@prefix} #{msg}") 26 | end 27 | end 28 | 29 | module Logging 30 | class << self 31 | def logger 32 | return @logger if @logger 33 | 34 | base_logger = if defined?(Rails) 35 | Rails.logger 36 | else 37 | logger = Logger.new STDOUT 38 | logger.progname = 'Clearbit::Analytics' 39 | logger 40 | end 41 | @logger = PrefixedLogger.new(base_logger, '[clearbit-ruby]') 42 | end 43 | 44 | attr_writer :logger 45 | end 46 | 47 | def self.included(base) 48 | class << base 49 | def logger 50 | Logging.logger 51 | end 52 | end 53 | end 54 | 55 | def logger 56 | Logging.logger 57 | end 58 | end 59 | end 60 | end 61 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/message_batch.rb: -------------------------------------------------------------------------------- 1 | require 'forwardable' 2 | require 'clearbit/analytics/logging' 3 | 4 | module Clearbit 5 | class Analytics 6 | # A batch of `Message`s to be sent to the API 7 | class MessageBatch 8 | class JSONGenerationError < StandardError; end 9 | 10 | extend Forwardable 11 | include Clearbit::Analytics::Logging 12 | include Clearbit::Analytics::Defaults::MessageBatch 13 | 14 | def initialize(max_message_count) 15 | @messages = [] 16 | @max_message_count = max_message_count 17 | @json_size = 0 18 | end 19 | 20 | def <<(message) 21 | begin 22 | message_json = message.to_json 23 | rescue StandardError => e 24 | raise JSONGenerationError, "Serialization error: #{e}" 25 | end 26 | 27 | message_json_size = message_json.bytesize 28 | if message_too_big?(message_json_size) 29 | logger.error('a message exceeded the maximum allowed size') 30 | else 31 | @messages << message 32 | @json_size += message_json_size + 1 # One byte for the comma 33 | end 34 | end 35 | 36 | def full? 37 | item_count_exhausted? || size_exhausted? 38 | end 39 | 40 | def clear 41 | @messages.clear 42 | @json_size = 0 43 | end 44 | 45 | def_delegators :@messages, :to_json 46 | def_delegators :@messages, :empty? 47 | def_delegators :@messages, :length 48 | 49 | private 50 | 51 | def item_count_exhausted? 52 | @messages.length >= @max_message_count 53 | end 54 | 55 | def message_too_big?(message_json_size) 56 | message_json_size > Defaults::Message::MAX_BYTES 57 | end 58 | 59 | # We consider the max size here as just enough to leave room for one more 60 | # message of the largest size possible. This is a shortcut that allows us 61 | # to use a native Ruby `Queue` that doesn't allow peeking. The tradeoff 62 | # here is that we might fit in less messages than possible into a batch. 63 | # 64 | # The alternative is to use our own `Queue` implementation that allows 65 | # peeking, and to consider the next message size when calculating whether 66 | # the message can be accomodated in this batch. 67 | def size_exhausted? 68 | @json_size >= (MAX_BYTES - Defaults::Message::MAX_BYTES) 69 | end 70 | end 71 | end 72 | end 73 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/request.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit/analytics/defaults' 2 | require 'clearbit/analytics/utils' 3 | require 'clearbit/analytics/response' 4 | require 'clearbit/analytics/logging' 5 | require 'clearbit/analytics/backoff_policy' 6 | require 'net/http' 7 | require 'net/https' 8 | require 'json' 9 | 10 | module Clearbit 11 | class Analytics 12 | class Request 13 | include Clearbit::Analytics::Defaults::Request 14 | include Clearbit::Analytics::Utils 15 | include Clearbit::Analytics::Logging 16 | 17 | # public: Creates a new request object to send analytics batch 18 | # 19 | def initialize(options = {}) 20 | options[:host] ||= HOST 21 | options[:port] ||= PORT 22 | options[:ssl] ||= SSL 23 | @headers = options[:headers] || HEADERS 24 | @path = options[:path] || PATH 25 | @retries = options[:retries] || RETRIES 26 | @backoff_policy = 27 | options[:backoff_policy] || Clearbit::Analytics::BackoffPolicy.new 28 | 29 | http = Net::HTTP.new(options[:host], options[:port]) 30 | http.use_ssl = options[:ssl] 31 | http.read_timeout = 8 32 | http.open_timeout = 4 33 | 34 | @http = http 35 | end 36 | 37 | # public: Posts the write key and batch of messages to the API. 38 | # 39 | # returns - Response of the status and error if it exists 40 | def post(write_key, batch) 41 | logger.debug("Sending request for #{batch.length} items") 42 | 43 | last_response, exception = retry_with_backoff(@retries) do 44 | status_code, body = send_request(write_key, batch) 45 | error = JSON.parse(body)['error'] 46 | should_retry = should_retry_request?(status_code, body) 47 | logger.debug("Response status code: #{status_code}") 48 | logger.debug("Response error: #{error}") if error 49 | 50 | [Response.new(status_code, error), should_retry] 51 | end 52 | 53 | if exception 54 | logger.error(exception.message) 55 | exception.backtrace.each { |line| logger.error(line) } 56 | Response.new(-1, exception.to_s) 57 | else 58 | last_response 59 | end 60 | end 61 | 62 | private 63 | 64 | def should_retry_request?(status_code, body) 65 | if status_code >= 500 66 | true # Server error 67 | elsif status_code == 429 68 | true # Rate limited 69 | elsif status_code >= 400 70 | logger.error(body) 71 | false # Client error. Do not retry, but log 72 | else 73 | false 74 | end 75 | end 76 | 77 | # Takes a block that returns [result, should_retry]. 78 | # 79 | # Retries upto `retries_remaining` times, if `should_retry` is false or 80 | # an exception is raised. `@backoff_policy` is used to determine the 81 | # duration to sleep between attempts 82 | # 83 | # Returns [last_result, raised_exception] 84 | def retry_with_backoff(retries_remaining, &block) 85 | result, caught_exception = nil 86 | should_retry = false 87 | 88 | begin 89 | result, should_retry = yield 90 | return [result, nil] unless should_retry 91 | rescue StandardError => e 92 | should_retry = true 93 | caught_exception = e 94 | end 95 | 96 | if should_retry && (retries_remaining > 1) 97 | logger.debug("Retrying request, #{retries_remaining} retries left") 98 | sleep(@backoff_policy.next_interval.to_f / 1000) 99 | retry_with_backoff(retries_remaining - 1, &block) 100 | else 101 | [result, caught_exception] 102 | end 103 | end 104 | 105 | # Sends a request for the batch, returns [status_code, body] 106 | def send_request(write_key, batch) 107 | payload = JSON.generate( 108 | :sentAt => datetime_in_iso8601(Time.now), 109 | :batch => batch 110 | ) 111 | request = Net::HTTP::Post.new(@path, @headers) 112 | request.basic_auth(write_key, nil) 113 | 114 | if self.class.stub 115 | logger.debug "stubbed request to #{@path}: " \ 116 | "write key = #{write_key}, batch = JSON.generate(#{batch})" 117 | 118 | [200, '{}'] 119 | else 120 | response = @http.request(request, payload) 121 | [response.code.to_i, response.body] 122 | end 123 | end 124 | 125 | class << self 126 | attr_writer :stub 127 | 128 | def stub 129 | @stub || ENV['STUB'] 130 | end 131 | end 132 | end 133 | end 134 | end 135 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/response.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Analytics 3 | class Response 4 | attr_reader :status, :error 5 | 6 | # public: Simple class to wrap responses from the API 7 | # 8 | # 9 | def initialize(status = 200, error = nil) 10 | @status = status 11 | @error = error 12 | end 13 | end 14 | end 15 | end 16 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/utils.rb: -------------------------------------------------------------------------------- 1 | require 'securerandom' 2 | 3 | module Clearbit 4 | class Analytics 5 | module Utils 6 | extend self 7 | 8 | # public: Return a new hash with keys converted from strings to symbols 9 | # 10 | def symbolize_keys(hash) 11 | hash.each_with_object({}) do |(k, v), memo| 12 | memo[k.to_sym] = v 13 | end 14 | end 15 | 16 | # public: Convert hash keys from strings to symbols in place 17 | # 18 | def symbolize_keys!(hash) 19 | hash.replace symbolize_keys hash 20 | end 21 | 22 | # public: Return a new hash with keys as strings 23 | # 24 | def stringify_keys(hash) 25 | hash.each_with_object({}) do |(k, v), memo| 26 | memo[k.to_s] = v 27 | end 28 | end 29 | 30 | # public: Returns a new hash with all the date values in the into iso8601 31 | # strings 32 | # 33 | def isoify_dates(hash) 34 | hash.each_with_object({}) do |(k, v), memo| 35 | memo[k] = datetime_in_iso8601(v) 36 | end 37 | end 38 | 39 | # public: Converts all the date values in the into iso8601 strings in place 40 | # 41 | def isoify_dates!(hash) 42 | hash.replace isoify_dates hash 43 | end 44 | 45 | # public: Returns a uid string 46 | # 47 | def uid 48 | arr = SecureRandom.random_bytes(16).unpack('NnnnnN') 49 | arr[2] = (arr[2] & 0x0fff) | 0x4000 50 | arr[3] = (arr[3] & 0x3fff) | 0x8000 51 | '%08x-%04x-%04x-%04x-%04x%08x' % arr 52 | end 53 | 54 | def datetime_in_iso8601(datetime) 55 | case datetime 56 | when Time 57 | time_in_iso8601 datetime 58 | when DateTime 59 | time_in_iso8601 datetime.to_time 60 | when Date 61 | date_in_iso8601 datetime 62 | else 63 | datetime 64 | end 65 | end 66 | 67 | def time_in_iso8601(time, fraction_digits = 3) 68 | fraction = if fraction_digits > 0 69 | ('.%06i' % time.usec)[0, fraction_digits + 1] 70 | end 71 | 72 | "#{time.strftime('%Y-%m-%dT%H:%M:%S')}#{fraction}#{formatted_offset(time, true, 'Z')}" 73 | end 74 | 75 | def date_in_iso8601(date) 76 | date.strftime('%F') 77 | end 78 | 79 | def formatted_offset(time, colon = true, alternate_utc_string = nil) 80 | time.utc? && alternate_utc_string || seconds_to_utc_offset(time.utc_offset, colon) 81 | end 82 | 83 | def seconds_to_utc_offset(seconds, colon = true) 84 | (colon ? UTC_OFFSET_WITH_COLON : UTC_OFFSET_WITHOUT_COLON) % [(seconds < 0 ? '-' : '+'), (seconds.abs / 3600), ((seconds.abs % 3600) / 60)] 85 | end 86 | 87 | UTC_OFFSET_WITH_COLON = '%s%02d:%02d' 88 | UTC_OFFSET_WITHOUT_COLON = UTC_OFFSET_WITH_COLON.sub(':', '') 89 | end 90 | end 91 | end 92 | -------------------------------------------------------------------------------- /lib/clearbit/analytics/worker.rb: -------------------------------------------------------------------------------- 1 | require 'clearbit/analytics/defaults' 2 | require 'clearbit/analytics/message_batch' 3 | require 'clearbit/analytics/request' 4 | require 'clearbit/analytics/utils' 5 | 6 | module Clearbit 7 | class Analytics 8 | class Worker 9 | include Clearbit::Analytics::Utils 10 | include Clearbit::Analytics::Defaults 11 | include Clearbit::Analytics::Logging 12 | 13 | # public: Creates a new worker 14 | # 15 | # The worker continuously takes messages off the queue 16 | # and makes requests to the segment.io api 17 | # 18 | # queue - Queue synchronized between client and worker 19 | # write_key - String of the project's Write key 20 | # options - Hash of worker options 21 | # batch_size - Fixnum of how many items to send in a batch 22 | # on_error - Proc of what to do on an error 23 | # 24 | def initialize(queue, write_key, options = {}) 25 | symbolize_keys! options 26 | @queue = queue 27 | @write_key = write_key 28 | @on_error = options[:on_error] || proc { |status, error| } 29 | batch_size = options[:batch_size] || Defaults::MessageBatch::MAX_SIZE 30 | @batch = MessageBatch.new(batch_size) 31 | @lock = Mutex.new 32 | end 33 | 34 | # public: Continuously runs the loop to check for new events 35 | # 36 | def run 37 | until Thread.current[:should_exit] 38 | return if @queue.empty? 39 | 40 | @lock.synchronize do 41 | consume_message_from_queue! until @batch.full? || @queue.empty? 42 | end 43 | 44 | res = Request.new.post @write_key, @batch 45 | @on_error.call(res.status, res.error) unless res.status == 200 46 | 47 | @lock.synchronize { @batch.clear } 48 | end 49 | end 50 | 51 | # public: Check whether we have outstanding requests. 52 | # 53 | def is_requesting? 54 | @lock.synchronize { !@batch.empty? } 55 | end 56 | 57 | private 58 | 59 | def consume_message_from_queue! 60 | @batch << @queue.pop 61 | rescue MessageBatch::JSONGenerationError => e 62 | @on_error.call(-1, e.to_s) 63 | end 64 | end 65 | end 66 | end 67 | -------------------------------------------------------------------------------- /lib/clearbit/audiences.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Audiences < Base 3 | endpoint 'https://audiences.clearbit.com' 4 | path '/v1/audiences' 5 | 6 | def self.add_email(values = {}) 7 | post('email', values) 8 | end 9 | 10 | def self.add_domain(values = {}) 11 | post('domain', values) 12 | end 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /lib/clearbit/autocomplete.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Autocomplete 3 | class Company < Base 4 | endpoint 'https://autocomplete.clearbit.com' 5 | path '/v1' 6 | 7 | def self.suggest(query) 8 | response = get 'companies/suggest', query: query 9 | 10 | self.new(response) 11 | end 12 | end 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /lib/clearbit/base.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Base < Resource 3 | endpoint 'https://api.clearbit.com' 4 | options :format => :json 5 | 6 | def self.version=(value) 7 | add_options headers: {'API-Version' => value} 8 | end 9 | 10 | def self.key=(value) 11 | add_options auth_type: :bearer, 12 | password: value 13 | 14 | @key = value 15 | end 16 | 17 | def self.key 18 | @key 19 | end 20 | 21 | def pending? 22 | false 23 | end 24 | end 25 | end 26 | -------------------------------------------------------------------------------- /lib/clearbit/discovery.rb: -------------------------------------------------------------------------------- 1 | require 'delegate' 2 | 3 | module Clearbit 4 | class Discovery < Base 5 | endpoint 'https://discovery.clearbit.com' 6 | path '/v1/companies/search' 7 | 8 | class PagedResult < Delegator 9 | def initialize(params, response) 10 | @params = params 11 | super Mash.new(response) 12 | end 13 | 14 | def __getobj__ 15 | @response 16 | end 17 | 18 | def __setobj__(obj) 19 | @response = obj 20 | end 21 | 22 | def each(&block) 23 | return enum_for(:each) unless block_given? 24 | 25 | results.each do |result| 26 | yield result 27 | end 28 | 29 | if results.any? 30 | search = Discovery.search( 31 | @params.merge(page: page + 1) 32 | ) 33 | search.each(&block) 34 | end 35 | end 36 | 37 | def map(&block) 38 | each.map(&block) 39 | end 40 | end 41 | 42 | def self.search(values = {}) 43 | response = post('', values) 44 | 45 | PagedResult.new(values, response) 46 | end 47 | end 48 | end 49 | -------------------------------------------------------------------------------- /lib/clearbit/enrichment.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Enrichment extend self 3 | autoload :Company, 'clearbit/enrichment/company' 4 | autoload :News, 'clearbit/enrichment/news' 5 | autoload :Person, 'clearbit/enrichment/person' 6 | autoload :PersonCompany, 'clearbit/enrichment/person_company' 7 | 8 | def find(values) 9 | if values.key?(:domain) 10 | result = Company.find(values) 11 | 12 | if result && result.pending? 13 | Pending.new 14 | else 15 | PersonCompany.new(company: result) 16 | end 17 | else 18 | PersonCompany.find(values) 19 | end 20 | end 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /lib/clearbit/enrichment/company.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Enrichment 3 | class Company < Base 4 | endpoint 'https://company.clearbit.com' 5 | path '/v2/companies' 6 | 7 | def self.find(values) 8 | unless values.is_a?(Hash) 9 | values = { id: values } 10 | end 11 | 12 | if values.key?(:domain) 13 | response = get(uri(:find), values) 14 | elsif id = values.delete(:id) 15 | response = get(id, values) 16 | else 17 | raise ArgumentError, 'Invalid values' 18 | end 19 | 20 | if response.status == 202 21 | Pending.new 22 | else 23 | self.new(response) 24 | end 25 | rescue Nestful::ResourceNotFound 26 | end 27 | 28 | class << self 29 | alias_method :[], :find 30 | end 31 | 32 | def flag!(attrs = {}) 33 | self.class.post(uri(:flag), attrs) 34 | end 35 | end 36 | end 37 | end 38 | -------------------------------------------------------------------------------- /lib/clearbit/enrichment/news.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Enrichment 3 | class News < Base 4 | endpoint 'https://company.clearbit.com' 5 | path '/v1/news' 6 | 7 | def self.articles(values) 8 | if values.key?(:domain) 9 | response = get(uri(:articles), values) 10 | else 11 | raise ArgumentError, 'Invalid values' 12 | end 13 | 14 | new(response) 15 | end 16 | end 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /lib/clearbit/enrichment/person.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Enrichment 3 | class Person < Base 4 | endpoint 'https://person.clearbit.com' 5 | path '/v2/people' 6 | 7 | def self.find(values) 8 | unless values.is_a?(Hash) 9 | values = { id: values } 10 | end 11 | 12 | if values.key?(:email) 13 | response = get(uri(:find), values) 14 | elsif id = values.delete(:id) 15 | response = get(id, values) 16 | else 17 | raise ArgumentError, 'Invalid values' 18 | end 19 | 20 | if response.status == 202 21 | Pending.new 22 | else 23 | self.new(response) 24 | end 25 | 26 | rescue Nestful::ResourceNotFound 27 | end 28 | 29 | class << self 30 | alias_method :[], :find 31 | end 32 | 33 | def flag!(attrs = {}) 34 | self.class.post(uri(:flag), attrs) 35 | end 36 | end 37 | end 38 | end 39 | -------------------------------------------------------------------------------- /lib/clearbit/enrichment/person_company.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Enrichment 3 | class PersonCompany < Base 4 | endpoint 'https://person.clearbit.com' 5 | path '/v2/combined' 6 | 7 | def self.find(values) 8 | unless values.is_a?(Hash) 9 | values = { email: values } 10 | end 11 | 12 | if values.key?(:email) 13 | response = get(uri(:find), values) 14 | else 15 | raise ArgumentError, 'Invalid values' 16 | end 17 | 18 | if response.status == 202 19 | Pending.new 20 | else 21 | self.new(response) 22 | end 23 | rescue Nestful::ResourceNotFound 24 | end 25 | 26 | class << self 27 | alias_method :[], :find 28 | end 29 | end 30 | end 31 | end 32 | -------------------------------------------------------------------------------- /lib/clearbit/errors/invalid_webhook_signature.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | module Errors 3 | # Raised when the Webhook Request Signature doesn't validate. 4 | class InvalidWebhookSignature < StandardError 5 | def to_s 6 | 'Clearbit Webhook Request Signature was invalid.' 7 | end 8 | end 9 | end 10 | end 11 | -------------------------------------------------------------------------------- /lib/clearbit/logo.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Logo 3 | ENDPOINT = 'https://logo.clearbit.com' 4 | 5 | def self.url(values) 6 | params = values.delete(params) || {} 7 | 8 | if size = values.delete(:size) 9 | params.merge!(size: size) 10 | end 11 | 12 | if format = values.delete(:format) 13 | params.merge!(format: format) 14 | end 15 | 16 | if greyscale = values.delete(:greyscale) 17 | params.merge!(greyscale: greyscale) 18 | end 19 | 20 | encoded_params = URI.encode_www_form(params) 21 | 22 | if domain = values.delete(:domain) 23 | raise ArgumentError, 'Invalid domain' unless domain =~ /^[a-z0-9-]+(\.[a-z0-9-]+)*\.[a-z]{2,}$/ 24 | if encoded_params.empty? 25 | "#{ENDPOINT}/#{domain}" 26 | else 27 | "#{ENDPOINT}/#{domain}?#{encoded_params}" 28 | end 29 | else 30 | raise ArgumentError, 'Invalid values' 31 | end 32 | end 33 | 34 | class << self 35 | alias_method :[], :url 36 | end 37 | end 38 | end 39 | -------------------------------------------------------------------------------- /lib/clearbit/mash.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Mash < Hash 3 | def self.new(value = nil, *args) 4 | if value.respond_to?(:each) && 5 | !value.respond_to?(:each_pair) 6 | value.map {|v| super(v) } 7 | else 8 | super 9 | end 10 | end 11 | 12 | alias_method :to_s, :inspect 13 | 14 | def initialize(source_hash = nil, default = nil, &blk) 15 | deep_update(source_hash.to_hash) if source_hash 16 | default ? super(default) : super(&blk) 17 | end 18 | 19 | class << self; alias [] new; end 20 | 21 | def id #:nodoc: 22 | self['id'] 23 | end 24 | 25 | def type #:nodoc: 26 | self['type'] 27 | end 28 | 29 | alias_method :regular_reader, :[] 30 | alias_method :regular_writer, :[]= 31 | 32 | # Retrieves an attribute set in the Mash. Will convert 33 | # any key passed in to a string before retrieving. 34 | def custom_reader(key) 35 | value = regular_reader(convert_key(key)) 36 | yield value if block_given? 37 | value 38 | end 39 | 40 | # Sets an attribute in the Mash. Key will be converted to 41 | # a string before it is set, and Hashes will be converted 42 | # into Mashes for nesting purposes. 43 | def custom_writer(key,value) #:nodoc: 44 | regular_writer(convert_key(key), convert_value(value)) 45 | end 46 | 47 | alias_method :[], :custom_reader 48 | alias_method :[]=, :custom_writer 49 | 50 | # This is the bang method reader, it will return a new Mash 51 | # if there isn't a value already assigned to the key requested. 52 | def initializing_reader(key) 53 | ck = convert_key(key) 54 | regular_writer(ck, self.class.new) unless key?(ck) 55 | regular_reader(ck) 56 | end 57 | 58 | # This is the under bang method reader, it will return a temporary new Mash 59 | # if there isn't a value already assigned to the key requested. 60 | def underbang_reader(key) 61 | ck = convert_key(key) 62 | if key?(ck) 63 | regular_reader(ck) 64 | else 65 | self.class.new 66 | end 67 | end 68 | 69 | def fetch(key, *args) 70 | super(convert_key(key), *args) 71 | end 72 | 73 | def delete(key) 74 | super(convert_key(key)) 75 | end 76 | 77 | alias_method :regular_dup, :dup 78 | # Duplicates the current mash as a new mash. 79 | def dup 80 | self.class.new(self, self.default) 81 | end 82 | 83 | def key?(key) 84 | super(convert_key(key)) 85 | end 86 | alias_method :has_key?, :key? 87 | alias_method :include?, :key? 88 | alias_method :member?, :key? 89 | 90 | # Performs a deep_update on a duplicate of the 91 | # current mash. 92 | def deep_merge(other_hash, &blk) 93 | dup.deep_update(other_hash, &blk) 94 | end 95 | alias_method :merge, :deep_merge 96 | 97 | # Recursively merges this mash with the passed 98 | # in hash, merging each hash in the hierarchy. 99 | def deep_update(other_hash, &blk) 100 | other_hash.each_pair do |k,v| 101 | key = convert_key(k) 102 | if regular_reader(key).is_a?(Mash) and v.is_a?(::Hash) 103 | custom_reader(key).deep_update(v, &blk) 104 | else 105 | value = convert_value(v, true) 106 | value = blk.call(key, self[k], value) if blk 107 | custom_writer(key, value) 108 | end 109 | end 110 | self 111 | end 112 | alias_method :deep_merge!, :deep_update 113 | alias_method :update, :deep_update 114 | alias_method :merge!, :update 115 | 116 | # Performs a shallow_update on a duplicate of the current mash 117 | def shallow_merge(other_hash) 118 | dup.shallow_update(other_hash) 119 | end 120 | 121 | # Merges (non-recursively) the hash from the argument, 122 | # changing the receiving hash 123 | def shallow_update(other_hash) 124 | other_hash.each_pair do |k,v| 125 | regular_writer(convert_key(k), convert_value(v, true)) 126 | end 127 | self 128 | end 129 | 130 | def replace(other_hash) 131 | (keys - other_hash.keys).each { |key| delete(key) } 132 | other_hash.each { |key, value| self[key] = value } 133 | self 134 | end 135 | 136 | # Will return true if the Mash has had a key 137 | # set in addition to normal respond_to? functionality. 138 | def respond_to?(method_name, include_private=false) 139 | camelized_name = camelize(method_name.to_s) 140 | 141 | if key?(method_name) || 142 | key?(camelized_name) || 143 | method_name.to_s.slice(/[=?!_]\Z/) 144 | return true 145 | end 146 | 147 | super 148 | end 149 | 150 | def method_missing(method_name, *args, &blk) 151 | return self.[](method_name, &blk) if key?(method_name) 152 | 153 | camelized_name = camelize(method_name.to_s) 154 | 155 | if key?(camelized_name) 156 | return self.[](camelized_name, &blk) 157 | end 158 | 159 | match = method_name.to_s.match(/(.*?)([?=!_]?)$/) 160 | 161 | case match[2] 162 | when "=" 163 | self[match[1]] = args.first 164 | when "?" 165 | !!self[match[1]] 166 | when "!" 167 | initializing_reader(match[1]) 168 | when "_" 169 | underbang_reader(match[1]) 170 | else 171 | default(method_name, *args, &blk) 172 | end 173 | end 174 | 175 | protected 176 | 177 | def camelize(string) 178 | string = string.to_s 179 | string = string.sub(/^(?:(?=\b|[A-Z_])|\w)/) { $&.downcase } 180 | string.gsub(/(?:_|(\/))([a-z\d]*)/i) { "#{$1}#{$2.capitalize}" }.gsub('/', '::') 181 | end 182 | 183 | def convert_key(key) #:nodoc: 184 | key.to_s 185 | end 186 | 187 | def convert_value(val, duping=false) #:nodoc: 188 | case val 189 | when self.class 190 | val.dup 191 | when ::Hash 192 | val = val.dup if duping 193 | Mash.new(val) 194 | when ::Array 195 | val.map {|e| convert_value(e) } 196 | else 197 | val 198 | end 199 | end 200 | end 201 | end 202 | -------------------------------------------------------------------------------- /lib/clearbit/name_domain.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class NameDomain < Base 3 | endpoint 'https://company.clearbit.com' 4 | path '/v1/domains' 5 | 6 | def self.find(values) 7 | response = get(uri(:find), values) 8 | Mash.new(response.decoded) 9 | rescue Nestful::ResourceNotFound 10 | end 11 | 12 | class << self 13 | alias_method :[], :find 14 | end 15 | end 16 | end 17 | 18 | -------------------------------------------------------------------------------- /lib/clearbit/pending.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Pending 3 | def pending? 4 | true 5 | end 6 | 7 | def queued? 8 | true 9 | end 10 | 11 | def inspect 12 | 'Your request is pending - please try again in few seconds, or pass the :stream option as true.' 13 | end 14 | end 15 | end 16 | -------------------------------------------------------------------------------- /lib/clearbit/prospector.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Prospector < Base 3 | endpoint 'https://prospector.clearbit.com' 4 | path '/v1/people' 5 | 6 | def self.search(values = {}) 7 | self.new get('search', values) 8 | end 9 | 10 | def self.find(values) 11 | unless values.is_a?(Hash) 12 | values = {:id => values} 13 | end 14 | 15 | if id = values.delete(:id) 16 | response = get(id, values) 17 | 18 | else 19 | raise ArgumentError, 'Invalid values' 20 | end 21 | 22 | self.new(response) 23 | rescue Nestful::ResourceNotFound 24 | end 25 | 26 | class << self 27 | alias_method :[], :find 28 | end 29 | 30 | def email 31 | self[:email] || email_response.email 32 | end 33 | 34 | def verified 35 | self[:verified] || email_response.verified 36 | end 37 | 38 | alias_method :verified?, :verified 39 | 40 | protected 41 | 42 | def email_response 43 | @email_response ||= begin 44 | response = self.class.get(uri(:email)) 45 | Mash.new(response.decoded) 46 | end 47 | end 48 | end 49 | end 50 | -------------------------------------------------------------------------------- /lib/clearbit/resource.rb: -------------------------------------------------------------------------------- 1 | require 'uri' 2 | 3 | module Clearbit 4 | class Resource < Mash 5 | def self.endpoint(value = nil) 6 | @endpoint = value if value 7 | return @endpoint if @endpoint 8 | superclass.respond_to?(:endpoint) ? superclass.endpoint : nil 9 | end 10 | 11 | def self.path(value = nil) 12 | @path = value if value 13 | return @path if @path 14 | superclass.respond_to?(:path) ? superclass.path : nil 15 | end 16 | 17 | def self.options(value = nil) 18 | @options ||= {} 19 | @options.merge!(value) if value 20 | 21 | if superclass <= Resource && superclass.respond_to?(:options) 22 | Nestful::Helpers.deep_merge(superclass.options, @options) 23 | else 24 | @options 25 | end 26 | end 27 | 28 | class << self 29 | alias_method :endpoint=, :endpoint 30 | alias_method :path=, :path 31 | alias_method :options=, :options 32 | alias_method :add_options, :options 33 | end 34 | 35 | def self.url(options = {}) 36 | URI.join(endpoint.to_s, path.to_s).to_s 37 | end 38 | 39 | def self.uri(*parts) 40 | # If an absolute URI already 41 | if (uri = parts.first) && uri.is_a?(URI) 42 | return uri if uri.host 43 | end 44 | 45 | value = Nestful::Helpers.to_path(url, *parts) 46 | 47 | URI.parse(value) 48 | end 49 | 50 | OPTION_KEYS = %i{ 51 | params key headers stream 52 | proxy user password auth_type 53 | timeout ssl_options request 54 | } 55 | 56 | def self.parse_values(values) 57 | params = values.reject {|k,_| OPTION_KEYS.include?(k) } 58 | options = values.select {|k,_| OPTION_KEYS.include?(k) } 59 | 60 | if request_options = options.delete(:request) 61 | options.merge!(request_options) 62 | end 63 | 64 | if key = options.delete(:key) 65 | options.merge!( 66 | auth_type: :bearer, 67 | password: key 68 | ) 69 | end 70 | 71 | [params, options] 72 | end 73 | 74 | def self.get(action = '', values = {}) 75 | params, options = parse_values(values) 76 | 77 | request( 78 | uri(action), 79 | options.merge(method: :get, params: params)) 80 | end 81 | 82 | def self.put(action = '', values = {}) 83 | params, options = parse_values(values) 84 | 85 | request( 86 | uri(action), 87 | options.merge(method: :put, params: params, format: :json)) 88 | end 89 | 90 | def self.post(action = '', values = {}) 91 | params, options = parse_values(values) 92 | 93 | request( 94 | uri(action), 95 | options.merge(method: :post, params: params, format: :json)) 96 | end 97 | 98 | def self.delete(action = '', values = {}) 99 | params, options = parse_values(values) 100 | 101 | request( 102 | uri(action), 103 | options.merge(method: :delete, params: params)) 104 | end 105 | 106 | def self.request(uri, options = {}) 107 | options = Nestful::Helpers.deep_merge(self.options, options) 108 | 109 | if options[:stream] 110 | uri.host = uri.host.gsub('.clearbit.com', '-stream.clearbit.com') 111 | end 112 | 113 | response = Nestful::Request.new( 114 | uri, options 115 | ).execute 116 | 117 | if notice = response.headers['X-API-Warn'] 118 | Kernel.warn notice 119 | end 120 | 121 | response 122 | end 123 | 124 | def uri(*parts) 125 | self.class.uri(*[id, *parts].compact) 126 | end 127 | end 128 | end 129 | -------------------------------------------------------------------------------- /lib/clearbit/reveal.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Reveal < Base 3 | endpoint 'https://reveal.clearbit.com' 4 | path '/v1/companies' 5 | 6 | def self.find(values) 7 | self.new(get(:find, values)) 8 | rescue Nestful::ResourceNotFound 9 | end 10 | 11 | class << self 12 | alias_method :[], :find 13 | end 14 | end 15 | end 16 | -------------------------------------------------------------------------------- /lib/clearbit/risk.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Risk < Base 3 | endpoint 'https://risk.clearbit.com' 4 | path '/v1' 5 | 6 | def self.calculate(values = {}) 7 | self.new post('calculate', values) 8 | end 9 | 10 | def self.confirmed(values = {}) 11 | post('confirmed', values) 12 | end 13 | 14 | def self.flag(values = {}) 15 | post('flag', values) 16 | end 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /lib/clearbit/version.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | VERSION = '0.3.3' 3 | end 4 | -------------------------------------------------------------------------------- /lib/clearbit/watchlist.rb: -------------------------------------------------------------------------------- 1 | module Clearbit 2 | class Watchlist < Base 3 | endpoint 'https://watchlist.clearbit.com' 4 | path '/v1/search/all' 5 | 6 | def self.search(values) 7 | response = post('', values) 8 | self.new(response) 9 | end 10 | 11 | class Individual < Watchlist 12 | path '/v1/search/individuals' 13 | end 14 | 15 | class Entity < Watchlist 16 | path '/v1/search/entities' 17 | end 18 | 19 | class Candidate < Watchlist 20 | path '/v1/candidates' 21 | 22 | def self.find(id, values) 23 | response = get(id, values) 24 | self.new(response) 25 | end 26 | 27 | def self.all(values) 28 | response = get('', values) 29 | self.new(response) 30 | end 31 | 32 | def self.create(values) 33 | response = post('', values) 34 | self.new(response) 35 | end 36 | 37 | def destroy 38 | self.class.delete(id) 39 | end 40 | end 41 | end 42 | end 43 | -------------------------------------------------------------------------------- /lib/clearbit/webhook.rb: -------------------------------------------------------------------------------- 1 | require 'openssl' 2 | require 'rack' 3 | require 'json' 4 | 5 | module Clearbit 6 | class Webhook < Mash 7 | def self.clearbit_key 8 | Clearbit.key! 9 | end 10 | 11 | def self.valid?(request_signature, body, key = nil) 12 | return false unless request_signature && body 13 | 14 | # The global Clearbit.key can be overriden for multi-tenant apps using multiple Clearbit keys 15 | key = (key || clearbit_key).gsub(/\A(pk|sk)_/, '') 16 | 17 | signature = generate_signature(key, body) 18 | Rack::Utils.secure_compare(request_signature, signature) 19 | end 20 | 21 | def self.valid!(signature, body, key = nil) 22 | valid?(signature, body, key) ? true : raise(Errors::InvalidWebhookSignature.new) 23 | end 24 | 25 | def self.generate_signature(key, body) 26 | signed_body = body 27 | signed_body = JSON.dump(signed_body) unless signed_body.is_a?(String) 28 | 'sha1=' + OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha1'), key, signed_body) 29 | end 30 | 31 | def initialize(env, key = nil) 32 | request = Rack::Request.new(env) 33 | 34 | request.body.rewind 35 | 36 | signature = request.env['HTTP_X_REQUEST_SIGNATURE'] 37 | body = request.body.read 38 | 39 | self.class.valid!(signature, body, key) 40 | 41 | merge!(JSON.parse(body)) 42 | end 43 | end 44 | end 45 | -------------------------------------------------------------------------------- /spec/lib/clearbit/analytics_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | require 'base64' 3 | 4 | describe Clearbit::Analytics do 5 | before do |example| 6 | Clearbit.key = 'clearbit_key' 7 | end 8 | 9 | describe '#identify' do 10 | it 'sends an identify request to Clearbit X' do 11 | basic_auth = Base64.encode64('clearbit_key:').strip 12 | x_stub = stub_request(:post, 'https://x.clearbit.com/v1/import'). 13 | with( 14 | headers: { 'Authorization' => "Basic #{basic_auth}" } 15 | ).to_return(status: 200, body: {success: true}.to_json) 16 | 17 | Clearbit::Analytics.identify( 18 | user_id: '123', 19 | traits: { 20 | email: 'david@clearbit.com', 21 | }, 22 | ) 23 | 24 | expect(x_stub).to have_been_requested 25 | end 26 | end 27 | 28 | describe '#page' do 29 | it 'sends an identify request to Clearbit X' do 30 | basic_auth = Base64.encode64('clearbit_key:').strip 31 | x_stub = stub_request(:post, 'https://x.clearbit.com/v1/import'). 32 | with( 33 | headers: { 'Authorization' => "Basic #{basic_auth}" } 34 | ).to_return(status: 200, body: {success: true}.to_json) 35 | 36 | Clearbit::Analytics.page( 37 | user_id: '123', 38 | traits: { 39 | email: 'david@clearbit.com', 40 | }, 41 | ) 42 | 43 | expect(x_stub).to have_been_requested 44 | end 45 | end 46 | 47 | describe '#group' do 48 | it 'sends an identify request to Clearbit X' do 49 | basic_auth = Base64.encode64('clearbit_key:').strip 50 | x_stub = stub_request(:post, 'https://x.clearbit.com/v1/import'). 51 | with( 52 | headers: { 'Authorization' => "Basic #{basic_auth}" } 53 | ).to_return(status: 200, body: {success: true}.to_json) 54 | 55 | Clearbit::Analytics.group( 56 | user_id: '123', 57 | group_id: '123', 58 | traits: { 59 | domain: 'clearbit.com', 60 | }, 61 | ) 62 | 63 | expect(x_stub).to have_been_requested 64 | end 65 | end 66 | end 67 | -------------------------------------------------------------------------------- /spec/lib/clearbit/discovery_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe Clearbit::Discovery do 4 | before do |example| 5 | Clearbit.key = 'clearbit_key' 6 | end 7 | 8 | it 'returns results from the Discovery API' do 9 | body = [] 10 | query = {query: {name: 'stripe'}} 11 | 12 | stub_request(:post, "https://discovery.clearbit.com/v1/companies/search"). 13 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}, body: query.to_json). 14 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 15 | 16 | Clearbit::Discovery.search(query: {name: 'stripe'}) 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /spec/lib/clearbit/enrichment_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe Clearbit::Enrichment do 4 | before do |example| 5 | Clearbit.key = 'clearbit_key' 6 | end 7 | 8 | context 'combined API' do 9 | it 'should call out to the combined API' do 10 | body = { 11 | person: nil, 12 | company: nil 13 | } 14 | 15 | stub_request(:get, 'https://person.clearbit.com/v2/combined/find?email=test@example.com'). 16 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 17 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 18 | 19 | Clearbit::Enrichment.find(email: 'test@example.com') 20 | end 21 | 22 | it 'uses streaming option' do 23 | body = { 24 | person: nil, 25 | company: nil 26 | } 27 | 28 | stub_request(:get, 'https://person-stream.clearbit.com/v2/combined/find?email=test@example.com'). 29 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 30 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 31 | 32 | Clearbit::Enrichment.find(email: 'test@example.com', stream: true) 33 | end 34 | 35 | it 'accepts request option' do 36 | body = { 37 | person: nil, 38 | company: nil 39 | } 40 | 41 | stub_request(:get, 'https://person.clearbit.com/v2/combined/find?email=test@example.com'). 42 | with(:headers => {'Authorization'=>'Bearer clearbit_key', 'X-Rated' => 'true'}). 43 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 44 | 45 | Clearbit::Enrichment.find(email: 'test@example.com', request: {headers: {'X-Rated' => 'true'}}) 46 | end 47 | 48 | it 'returns pending? if 202 response' do 49 | body = { 50 | person: nil, 51 | company: nil 52 | } 53 | 54 | stub_request(:get, 'https://person.clearbit.com/v2/combined/find?email=test@example.com'). 55 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 56 | to_return(:status => 202, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 57 | 58 | result = Clearbit::Enrichment.find(email: 'test@example.com') 59 | 60 | expect(result.pending?).to be true 61 | end 62 | 63 | it 'should use the Company API if domain is provided' do 64 | body = {} 65 | 66 | stub_request(:get, 'https://company.clearbit.com/v2/companies/find?domain=example.com'). 67 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 68 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 69 | 70 | Clearbit::Enrichment.find(domain: 'example.com') 71 | end 72 | end 73 | 74 | context 'person API' do 75 | it 'should call out to the person API' do 76 | body = {} 77 | 78 | stub_request(:get, 'https://person.clearbit.com/v2/people/find?email=test@example.com'). 79 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 80 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 81 | 82 | Clearbit::Enrichment::Person.find(email: 'test@example.com') 83 | end 84 | end 85 | 86 | context 'company API' do 87 | it 'should call out to the company API' do 88 | body = {} 89 | 90 | stub_request(:get, 'https://company.clearbit.com/v2/companies/find?domain=example.com'). 91 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 92 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 93 | 94 | Clearbit::Enrichment::Company.find(domain: 'example.com') 95 | end 96 | end 97 | end 98 | -------------------------------------------------------------------------------- /spec/lib/clearbit/logo_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe Clearbit::Logo do 4 | context 'domain validation' do 5 | 6 | def check_invalid_domain(domain) 7 | end 8 | 9 | it 'passes for simple domains' do 10 | expect { 11 | Clearbit::Logo.url(domain: 'clearbit.com') 12 | }.to_not raise_error 13 | end 14 | 15 | it 'passes for dashed domains' do 16 | expect { 17 | Clearbit::Logo.url(domain: 'clear-bit.com') 18 | }.to_not raise_error 19 | end 20 | 21 | it 'passes for multi-dot TLDs' do 22 | expect { 23 | Clearbit::Logo.url(domain: 'bbc.co.uk') 24 | }.to_not raise_error 25 | 26 | expect { 27 | Clearbit::Logo.url(domain: 'clear-bit.co.uk') 28 | }.to_not raise_error 29 | end 30 | 31 | it 'passes for new-style tlds' do 32 | expect { 33 | Clearbit::Logo.url(domain: 'clearbit.museum') 34 | }.to_not raise_error 35 | end 36 | 37 | it 'fails for invalid urls' do 38 | expect { 39 | Clearbit::Logo.url(domain: 'clearbit') 40 | }.to raise_error(ArgumentError) 41 | end 42 | end 43 | end 44 | -------------------------------------------------------------------------------- /spec/lib/clearbit/prospector_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe Clearbit::Prospector do 4 | before do |example| 5 | Clearbit.key = 'clearbit_key' 6 | end 7 | 8 | context 'Prospector API' do 9 | it 'calls the Prospector API' do 10 | body = { page: 1, page_size: 5, total: 723, results: [] } 11 | 12 | stub_request(:get, "https://prospector.clearbit.com/v1/people/search?domain=stripe.com"). 13 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 14 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 15 | 16 | Clearbit::Prospector.search(domain: 'stripe.com') 17 | end 18 | 19 | it 'can page through records' do 20 | body = { page: 2, page_size: 10, total: 12, results: [] } 21 | 22 | stub_request(:get, "https://prospector.clearbit.com/v1/people/search?domain=stripe.com&page=2&page_size=10"). 23 | with(:headers => {'Authorization'=>'Bearer clearbit_key'}). 24 | to_return(:status => 200, :body => body.to_json, headers: {'Content-Type' => 'application/json'}) 25 | 26 | Clearbit::Prospector.search(domain: 'stripe.com', page: 2, page_size: 10) 27 | end 28 | end 29 | end 30 | -------------------------------------------------------------------------------- /spec/lib/clearbit/webhook_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe Clearbit::Webhook, 'valid!' do 4 | let(:clearbit_key) { 'clearbit_key' } 5 | 6 | context 'clearbit key set globally' do 7 | before do 8 | Clearbit.key = clearbit_key 9 | end 10 | 11 | context 'valid signature' do 12 | it 'returns true' do 13 | signature = generate_signature(clearbit_key, 'A-OK') 14 | 15 | result = Clearbit::Webhook.valid!(signature, 'A-OK') 16 | 17 | expect(result).to eq true 18 | end 19 | end 20 | 21 | context 'invalid signature' do 22 | it 'returns false' do 23 | signature = generate_signature(clearbit_key, 'A-OK') 24 | 25 | expect { 26 | Clearbit::Webhook.valid!(signature, 'TAMPERED_WITH_BODY_BEWARE!') 27 | }.to raise_error(Clearbit::Errors::InvalidWebhookSignature) 28 | end 29 | end 30 | end 31 | 32 | context 'clearbit key set locally' do 33 | context 'valid signature' do 34 | it 'returns true' do 35 | clearbit_key = 'clearbit_key' 36 | signature = generate_signature(clearbit_key, 'A-OK') 37 | 38 | result = Clearbit::Webhook.valid!(signature, 'A-OK', clearbit_key) 39 | 40 | expect(result).to eq true 41 | end 42 | end 43 | 44 | context 'invalid signature' do 45 | it 'returns false' do 46 | clearbit_key = 'clearbit_key' 47 | signature = generate_signature(clearbit_key, 'A-OK') 48 | 49 | expect { 50 | Clearbit::Webhook.valid!(signature, 'TAMPERED_WITH_BODY_BEWARE!', clearbit_key) 51 | }.to raise_error(Clearbit::Errors::InvalidWebhookSignature) 52 | end 53 | end 54 | end 55 | end 56 | 57 | describe Clearbit::Webhook, 'initialize' do 58 | let(:clearbit_key) { 'clearbit_key' } 59 | 60 | let(:env) do 61 | request_body = JSON.dump(id:'123', type: 'person', body: nil, status: 404) 62 | 63 | Rack::MockRequest.env_for('/webhook', 64 | method: 'POST', 65 | input: request_body, 66 | 'HTTP_X_REQUEST_SIGNATURE' => generate_signature(clearbit_key, request_body) 67 | ) 68 | end 69 | 70 | context 'clearbit key set globally' do 71 | it 'returns a mash' do 72 | Clearbit.key = 'clearbit_key' 73 | 74 | webhook = Clearbit::Webhook.new(env) 75 | 76 | expect(webhook.status).to eq 404 77 | end 78 | end 79 | 80 | context 'clearbit key set locally' do 81 | it 'returns a mash' do 82 | webhook = Clearbit::Webhook.new(env, 'clearbit_key') 83 | 84 | expect(webhook.status).to eq 404 85 | end 86 | end 87 | end 88 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | $LOAD_PATH << File.join(File.dirname(__FILE__), '..', 'lib') 2 | $LOAD_PATH << File.join(File.dirname(__FILE__)) 3 | 4 | # External 5 | require 'rubygems' 6 | require 'rspec' 7 | require 'pry' 8 | require 'webmock/rspec' 9 | 10 | # Library 11 | require 'clearbit' 12 | 13 | Dir[File.expand_path('spec/support/**/*.rb')].each { |file| require file } 14 | 15 | RSpec.configure do |config| 16 | config.include Spec::Support::Helpers 17 | config.order = 'random' 18 | 19 | config.expect_with :rspec do |c| 20 | c.syntax = :expect 21 | end 22 | 23 | config.before :each do 24 | Clearbit.key = nil 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /spec/support/helpers.rb: -------------------------------------------------------------------------------- 1 | module Spec 2 | module Support 3 | module Helpers 4 | def generate_signature(clearbit_key, webhook_body) 5 | signed_body = webhook_body 6 | signed_body = JSON.dump(signed_body) unless signed_body.is_a?(String) 7 | 'sha1=' + OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha1'), clearbit_key, signed_body) 8 | end 9 | end 10 | end 11 | end 12 | --------------------------------------------------------------------------------