├── .github └── FUNDING.yml ├── .gitignore ├── Dockerfile ├── README.md ├── UNLICENSE ├── haproxy.cfg.erb ├── newnym.sh ├── start.rb └── uncachable /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | github: mattes 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:14.04 2 | MAINTAINER Matthias Kadenbach 3 | 4 | RUN echo 'deb http://deb.torproject.org/torproject.org trusty main' | tee /etc/apt/sources.list.d/torproject.list 5 | RUN gpg --keyserver keys.gnupg.net --recv 886DDD89 6 | RUN gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | apt-key add - 7 | 8 | RUN echo 'deb http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu trusty main' | tee /etc/apt/sources.list.d/ruby.list 9 | RUN gpg --keyserver keyserver.ubuntu.com --recv C3173AA6 10 | RUN gpg --export 80f70e11f0f0d5f10cb20e62f5da5f09c3173aa6 | apt-key add - 11 | 12 | RUN apt-get update && \ 13 | apt-get install -y tor polipo haproxy ruby2.1 libssl-dev wget curl build-essential zlib1g-dev libyaml-dev libssl-dev && \ 14 | ln -s /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/libssl.so.1.0.0 15 | 16 | RUN update-rc.d -f tor remove 17 | RUN update-rc.d -f polipo remove 18 | 19 | RUN gem install excon -v 0.44.4 20 | 21 | ADD start.rb /usr/local/bin/start.rb 22 | RUN chmod +x /usr/local/bin/start.rb 23 | 24 | ADD newnym.sh /usr/local/bin/newnym.sh 25 | RUN chmod +x /usr/local/bin/newnym.sh 26 | 27 | ADD haproxy.cfg.erb /usr/local/etc/haproxy.cfg.erb 28 | ADD uncachable /etc/polipo/uncachable 29 | 30 | EXPOSE 5566 4444 31 | 32 | CMD /usr/local/bin/start.rb 33 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | docker-rotating-proxy 2 | ===================== 3 | 4 | [![Docker Pulls](https://img.shields.io/docker/pulls/mattes/rotating-proxy.svg)](https://hub.docker.com/r/mattes/rotating-proxy/) 5 | 6 | ``` 7 | Docker Container 8 | ------------------------------------- 9 | <-> Polipo 1 <-> Tor Proxy 1 10 | Client <----> HAproxy <-> Polipo 2 <-> Tor Proxy 2 11 | <-> Polipo n <-> Tor Proxy n 12 | ``` 13 | 14 | __Why:__ Lots of IP addresses. One single endpoint for your client. 15 | Load-balancing by HAproxy. 16 | 17 | Usage 18 | ----- 19 | 20 | ```bash 21 | # build docker container 22 | docker build -t mattes/rotating-proxy:latest . 23 | 24 | # ... or pull docker container 25 | docker pull mattes/rotating-proxy:latest 26 | 27 | # start docker container 28 | docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy 29 | 30 | # test with ... 31 | curl --proxy 127.0.0.1:5566 https://api.my-ip.io/ip 32 | 33 | # monitor 34 | http://127.0.0.1:4444/haproxy?stats 35 | ``` 36 | 37 | 38 | Further Readings 39 | ---------------- 40 | 41 | * [Tor Manual](https://www.torproject.org/docs/tor-manual.html.en) 42 | * [Tor Control](https://www.thesprawl.org/research/tor-control-protocol/) 43 | * [HAProxy Manual](http://cbonte.github.io/haproxy-dconv/configuration-1.5.html) 44 | * [Polipo](http://www.pps.univ-paris-diderot.fr/~jch/software/polipo/) 45 | 46 | -------------- 47 | 48 | Please note: Tor offers a SOCKS Proxy only. In order to allow communication 49 | from HAproxy to Tor, Polipo is used to translate from HTTP proxy to SOCKS proxy. 50 | HAproxy is able to talk to HTTP proxies only. 51 | 52 | -------------------------------------------------------------------------------- /UNLICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to -------------------------------------------------------------------------------- /haproxy.cfg.erb: -------------------------------------------------------------------------------- 1 | global 2 | maxconn 1024 3 | daemon 4 | pidfile <%= pid_file %> 5 | 6 | defaults 7 | mode http 8 | maxconn 1024 9 | option httplog 10 | option dontlognull 11 | retries 3 12 | timeout connect 5s 13 | timeout client 60s 14 | timeout server 60s 15 | 16 | 17 | listen stats *:4444 18 | mode http 19 | log global 20 | maxconn 10 21 | clitimeout 100s 22 | srvtimeout 100s 23 | contimeout 100s 24 | timeout queue 100s 25 | stats enable 26 | stats hide-version 27 | stats refresh 30s 28 | stats show-node 29 | stats uri /haproxy?stats 30 | 31 | 32 | frontend rotating_proxies 33 | bind *:<%= port %> 34 | default_backend tor 35 | option http_proxy 36 | 37 | backend tor 38 | option http_proxy 39 | balance leastconn # http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#balance 40 | 41 | <% backends.each do |b| %> 42 | server <%= b[:name] %><%= b[:port] %> <%= b[:addr] %>:<%= b[:port] %> 43 | <% end %> 44 | -------------------------------------------------------------------------------- /newnym.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | CONTROLPORT=$1 4 | 5 | cat <<'EOF' | nc localhost $CONTROLPORT 6 | authenticate "" 7 | signal newnym 8 | quit 9 | EOF 10 | -------------------------------------------------------------------------------- /start.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | require 'erb' 3 | require 'excon' 4 | require 'logger' 5 | 6 | $logger = Logger.new(STDOUT, ENV['DEBUG'] ? Logger::DEBUG : Logger::INFO) 7 | 8 | module Service 9 | class Base 10 | attr_reader :port 11 | 12 | def initialize(port) 13 | @port = port 14 | end 15 | 16 | def service_name 17 | self.class.name.downcase.split('::').last 18 | end 19 | 20 | def start 21 | ensure_directories 22 | $logger.info "starting #{service_name} on port #{port}" 23 | end 24 | 25 | def ensure_directories 26 | %w{lib run log}.each do |dir| 27 | path = "/var/#{dir}/#{service_name}" 28 | Dir.mkdir(path) unless Dir.exists?(path) 29 | end 30 | end 31 | 32 | def data_directory 33 | "/var/lib/#{service_name}" 34 | end 35 | 36 | def pid_file 37 | "/var/run/#{service_name}/#{port}.pid" 38 | end 39 | 40 | def executable 41 | self.class.which(service_name) 42 | end 43 | 44 | def stop 45 | $logger.info "stopping #{service_name} on port #{port}" 46 | if File.exists?(pid_file) 47 | pid = File.read(pid_file).strip 48 | begin 49 | self.class.kill(pid.to_i) 50 | rescue => e 51 | $logger.warn "couldn't kill #{service_name} on port #{port}: #{e.message}" 52 | end 53 | else 54 | $logger.info "#{service_name} on port #{port} was not running" 55 | end 56 | end 57 | 58 | def self.kill(pid, signal='SIGINT') 59 | Process.kill(signal, pid) 60 | end 61 | 62 | def self.fire_and_forget(*args) 63 | $logger.debug "running: #{args.join(' ')}" 64 | pid = Process.fork 65 | if pid.nil? then 66 | # In child 67 | exec args.join(" ") 68 | else 69 | # In parent 70 | Process.detach(pid) 71 | end 72 | end 73 | 74 | def self.which(executable) 75 | path = `which #{executable}`.strip 76 | if path == "" 77 | return nil 78 | else 79 | return path 80 | end 81 | end 82 | end 83 | 84 | 85 | class Tor < Base 86 | attr_reader :port, :control_port 87 | 88 | def initialize(port, control_port) 89 | @port = port 90 | @control_port = control_port 91 | end 92 | 93 | def data_directory 94 | "#{super}/#{port}" 95 | end 96 | 97 | def start 98 | super 99 | self.class.fire_and_forget(executable, 100 | "--SocksPort #{port}", 101 | "--ControlPort #{control_port}", 102 | "--NewCircuitPeriod 15", 103 | "--MaxCircuitDirtiness 15", 104 | "--UseEntryGuards 0", 105 | "--UseEntryGuardsAsDirGuards 0", 106 | "--CircuitBuildTimeout 5", 107 | "--ExitRelay 0", 108 | "--RefuseUnknownExits 0", 109 | "--ClientOnly 1", 110 | "--AllowSingleHopCircuits 1", 111 | "--DataDirectory #{data_directory}", 112 | "--PidFile #{pid_file}", 113 | "--Log \"warn syslog\"", 114 | '--RunAsDaemon 1', 115 | "| logger -t 'tor' 2>&1") 116 | end 117 | 118 | def newnym 119 | self.class.fire_and_forget('/usr/local/bin/newnym.sh', 120 | "#{control_port}", 121 | "| logger -t 'newnym'") 122 | end 123 | end 124 | 125 | class Polipo < Base 126 | def initialize(port, tor:) 127 | super(port) 128 | @tor = tor 129 | end 130 | 131 | def start 132 | super 133 | # https://gitweb.torproject.org/torbrowser.git/blob_plain/1ffcd9dafb9dd76c3a29dd686e05a71a95599fb5:/build-scripts/config/polipo.conf 134 | if File.exists?(pid_file) 135 | File.delete(pid_file) 136 | end 137 | self.class.fire_and_forget(executable, 138 | "proxyPort=#{port}", 139 | "socksParentProxy=127.0.0.1:#{tor_port}", 140 | "socksProxyType=socks5", 141 | "diskCacheRoot=''", 142 | "disableLocalInterface=true", 143 | "allowedClients=127.0.0.1", 144 | "localDocumentRoot=''", 145 | "disableConfiguration=true", 146 | "dnsUseGethostbyname='yes'", 147 | "logSyslog=true", 148 | "daemonise=true", 149 | "pidFile=#{pid_file}", 150 | "disableVia=true", 151 | "allowedPorts='1-65535'", 152 | "tunnelAllowedPorts='1-65535'", 153 | "| logger -t 'polipo' 2>&1") 154 | end 155 | 156 | def tor_port 157 | @tor.port 158 | end 159 | end 160 | 161 | class Proxy 162 | attr_reader :id 163 | attr_reader :tor, :polipo 164 | 165 | def initialize(id) 166 | @id = id 167 | @tor = Tor.new(tor_port, tor_control_port) 168 | @polipo = Polipo.new(polipo_port, tor: tor) 169 | end 170 | 171 | def start 172 | $logger.info "starting proxy id #{id}" 173 | @tor.start 174 | @polipo.start 175 | end 176 | 177 | def stop 178 | $logger.info "stopping proxy id #{id}" 179 | @tor.stop 180 | @polipo.stop 181 | end 182 | 183 | def restart 184 | stop 185 | sleep 5 186 | start 187 | end 188 | 189 | def tor_port 190 | 10000 + id 191 | end 192 | 193 | def tor_control_port 194 | 30000 + id 195 | end 196 | 197 | def polipo_port 198 | tor_port + 10000 199 | end 200 | alias_method :port, :polipo_port 201 | 202 | def test_url 203 | ENV['test_url'] || 'http://icanhazip.com' 204 | end 205 | 206 | def working? 207 | Excon.get(test_url, proxy: "http://127.0.0.1:#{port}", :read_timeout => 10).status == 200 208 | rescue 209 | false 210 | end 211 | end 212 | 213 | class Haproxy < Base 214 | attr_reader :backends 215 | 216 | def initialize(port = 5566) 217 | @config_erb_path = "/usr/local/etc/haproxy.cfg.erb" 218 | @config_path = "/usr/local/etc/haproxy.cfg" 219 | @backends = [] 220 | super(port) 221 | end 222 | 223 | def start 224 | super 225 | compile_config 226 | self.class.fire_and_forget(executable, 227 | "-f #{@config_path}", 228 | "| logger 2>&1") 229 | end 230 | 231 | def soft_reload 232 | self.class.fire_and_forget(executable, 233 | "-f #{@config_path}", 234 | "-p #{pid_file}", 235 | "-sf #{File.read(pid_file)}", 236 | "| logger 2>&1") 237 | end 238 | 239 | def add_backend(backend) 240 | @backends << {:name => 'tor', :addr => '127.0.0.1', :port => backend.port} 241 | end 242 | 243 | private 244 | def compile_config 245 | File.write(@config_path, ERB.new(File.read(@config_erb_path)).result(binding)) 246 | end 247 | end 248 | end 249 | 250 | haproxy = Service::Haproxy.new 251 | proxies = [] 252 | 253 | tor_instances = ENV['tors'] || 10 254 | tor_instances.to_i.times.each do |id| 255 | proxy = Service::Proxy.new(id) 256 | haproxy.add_backend(proxy) 257 | proxy.start 258 | proxies << proxy 259 | end 260 | 261 | haproxy.start 262 | 263 | sleep 60 264 | 265 | loop do 266 | $logger.info "resetting circuits" 267 | proxies.each do |proxy| 268 | $logger.info "reset nym for #{proxy.id} (port #{proxy.port})" 269 | proxy.tor.newnym 270 | end 271 | 272 | $logger.info "testing proxies" 273 | proxies.each do |proxy| 274 | $logger.info "testing proxy #{proxy.id} (port #{proxy.port})" 275 | proxy.restart unless proxy.working? 276 | end 277 | 278 | $logger.info "sleeping for 60 seconds" 279 | sleep 60 280 | end 281 | -------------------------------------------------------------------------------- /uncachable: -------------------------------------------------------------------------------- 1 | ipinfo.io 2 | icanhazip.com 3 | echoip.com 4 | --------------------------------------------------------------------------------