├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── docker-tcp-switchboard.py ├── example ├── Dockerfile └── docker-tcp-switchboard.conf ├── requirements.txt └── travis-ci-test ├── README ├── client.py ├── config.ini ├── config.ini.d ├── echoserv.ini └── upperserv.ini ├── runtest_basic.sh ├── runtest_kill.sh ├── runtest_rebuild.sh ├── setupenv.sh └── testimages ├── Dockerfile.echoserv ├── Dockerfile.echoserv-mangled ├── Dockerfile.upperserv └── echoserv.py /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "3.5" 4 | sudo: required 5 | before_install: 6 | - pushd travis-ci-test && ./setupenv.sh && popd 7 | 8 | script: 9 | - pushd travis-ci-test && ./runtest_basic.sh && popd 10 | - pushd travis-ci-test && ./runtest_kill.sh && popd 11 | - pushd travis-ci-test && ./runtest_rebuild.sh && popd 12 | 13 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Steven Van Acker 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Docker TCP Switchboard 2 | 3 | [![Build Status](https://api.travis-ci.org/OverTheWireOrg/docker-tcp-switchboard.svg?branch=master)](https://travis-ci.org/OverTheWireOrg/docker-tcp-switchboard) 4 | 5 | This project is part of [OverTheWire]'s infrastructure and used to provide 6 | players of OverTheWire wargames with a fresh Docker container each time they 7 | log into SSH. 8 | 9 | At this point in time, docker-tcp-switchboard only really supports SSH instead 10 | of arbitrary TCP connections, since it makes a connection to the backend and 11 | expects to receive a banner in order to determine that the Docker containers 12 | has started up successfully. 13 | 14 | Some features, current and future: 15 | 16 | * Allocate a new Docker instance per connection 17 | * Ability to reuse Docker instances for multiple connections. 18 | * Ability to limit the amount of running containers to avoid resource exhaustion. 19 | * [future] Ability to set quota (time-limit, network traffic limit) per container. 20 | * [future] Ability to delay network communication for incoming connections, to 21 | prevent that a flood of incoming connections spawns of a flood of containers 22 | that overwhelm the Docker host. 23 | 24 | ## Quickstart 25 | Attention: This is just a quick-start and not suitable for production. 26 | 27 | Prerequisites: 28 | - A docker image of your choice is needed 29 | - The image requires a running ssh-server and a known user/password (See `\example\Dockerfile` for a simple example) 30 | - root or root-privileges are needed for setup 31 | 32 | ````bash 33 | # start in your home directory 34 | cd ~ 35 | # clone this repository 36 | git clone https://github.com/OverTheWireOrg/docker-tcp-switchboard.git 37 | # install and start docker. You'll be able to control docker without root 38 | sudo apt-get -y install docker-ce 39 | sudo service docker start 40 | sudo usermod -a -G docker **yourusername** 41 | # install requirements 42 | cd /docker-tcp-switchboard 43 | sudo apt install python3-pip 44 | pip3 install -r requirements.txt 45 | # setup logfile 46 | touch /var/log/docker-tcp-switchboard.log 47 | chmod a+w /var/log/docker-tcp-switchboard.log 48 | # create the configuration file 49 | vi /etc/docker-tcp-switchboard.conf #paste your configuration file here (see below) 50 | # start docker-tcp-switchboard. It'll run in the foreground. 51 | python3 docker-tcp-switchboard.py 52 | ```` 53 | Done! Now connect to your `outerport` to start a fresh container. 54 | 55 | 56 | ## Example configuration file 57 | ````ini 58 | [global] 59 | logfile = /var/log/docker-tcp-switchboard.log 60 | loglevel = DEBUG 61 | 62 | [profile:firstcontainer] 63 | innerport = 22 64 | outerport = 32768 65 | container = imagename 66 | limit = 10 67 | reuse = false 68 | 69 | [profile:differentcontainer] 70 | innerport = 22 71 | outerport = 32769 72 | container = differentimagename 73 | limit = 5 74 | reuse = false 75 | 76 | [dockeroptions:differentcontainer] 77 | ports={"8808/tcp":null} 78 | volumes={"/home/ubuntu/mountthisfolder/": {"bind": "/mnd/folderincointainer/", "mode": "rw"}} 79 | 80 | ```` 81 | 82 | ### misc 83 | - See logfile for debugging (`tail -f /var/log/docker-tcp-switchboard.log`) 84 | - To auto-disconnect when idle, use SSHD config options "ClientAliveInterval" and "ServerAliveCountMax" 85 | - Remember to unblock "outerport" in your firewall 86 | - See [Docker SDK for Python](https://docker-py.readthedocs.io/en/stable/containers.html) for troubleshooting and available dockeroptions 87 | 88 | 89 | 90 | 91 | [OverTheWire]: http://overthewire.org 92 | -------------------------------------------------------------------------------- /docker-tcp-switchboard.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from twisted.protocols.portforward import * 4 | from twisted.internet import reactor 5 | 6 | import time, socket 7 | import configparser, glob 8 | import random, string 9 | import pprint 10 | import json 11 | import docker 12 | import copy 13 | 14 | import logging 15 | import logging.handlers 16 | logger = logging.getLogger("docker-tcp-switchboard") 17 | 18 | # this is a global object that keeps track of the free ports 19 | # when requested, it allocated a new docker instance and returns it 20 | 21 | class DockerPorts(): 22 | CONFIG_PROFILEPREFIX = "profile:" 23 | CONFIG_DOCKEROPTIONSPREFIX = "dockeroptions:" 24 | 25 | def __init__(self): 26 | self.instancesByName = dict() 27 | self.imageParams = dict() 28 | 29 | def _getProfilesList(self, config): 30 | out = [] 31 | for n in config.sections(): 32 | if n.startswith(self.CONFIG_PROFILEPREFIX): 33 | out += [n[len(self.CONFIG_PROFILEPREFIX):]] 34 | return out 35 | 36 | def _readProfileConfig(self, config, profilename): 37 | fullprofilename = "{}{}".format(self.CONFIG_PROFILEPREFIX, profilename) 38 | innerport = self._parseInt(config[fullprofilename]["innerport"]) 39 | checkupport = self._parseInt(config[fullprofilename]["checkupport"]) if "checkupport" in config[fullprofilename] else innerport 40 | return { 41 | "outerport": int(config[fullprofilename]["outerport"]), 42 | "innerport": innerport, 43 | "containername": config[fullprofilename]["container"], 44 | "checkupport": checkupport, 45 | "limit": self._parseInt(config[fullprofilename]["limit"]) if "limit" in config[fullprofilename] else 0, 46 | "reuse": self._parseTruthy(config[fullprofilename]["reuse"]) if "reuse" in config[fullprofilename] else False, 47 | "dockeroptions": self._getDockerOptions(config, profilename, innerport, checkupport) 48 | } 49 | 50 | def _addDockerOptionsFromConfigSection(self, config, sectionname, base={}): 51 | import collections 52 | def update(d, u): 53 | for k, v in u.items(): 54 | if isinstance(v, collections.Mapping): 55 | r = update(d.get(k, {}), v) 56 | d[k] = r 57 | else: 58 | d[k] = u[k] 59 | return d 60 | 61 | # we may need to read json values 62 | def guessvalue(v): 63 | if v in ["True", "False"] or all(c in string.digits for c in v) or v.startswith("[") or v.startswith("{"): 64 | return json.loads(v) 65 | return v 66 | 67 | # if sectionname doesn't exist, return base 68 | # otherwise, read keywords and values, add them to base 69 | if sectionname in config.sections(): 70 | newvals = dict(config[sectionname]) 71 | fixedvals = {} 72 | for (k,v) in newvals.items(): 73 | fixedvals[k] = guessvalue(v) 74 | base = update(base, fixedvals) 75 | 76 | return base # FIXME 77 | 78 | def _getDockerOptions(self, config, profilename, innerport, checkupport): 79 | out = {} 80 | out = self._addDockerOptionsFromConfigSection(config, "dockeroptions", {}) 81 | out = self._addDockerOptionsFromConfigSection(config, "{}{}".format(self.CONFIG_DOCKEROPTIONSPREFIX, profilename), out) 82 | 83 | out["detach"] = True 84 | if "ports" not in out: 85 | out["ports"] = {} 86 | out["ports"][innerport] = None 87 | out["ports"][checkupport] = None 88 | # cannot use detach and remove together 89 | # See https://github.com/docker/docker-py/issues/1477 90 | #out["remove"] = True 91 | #out["auto_remove"] = True 92 | return out 93 | 94 | def readConfig(self, fn): 95 | # read the configfile. 96 | config = configparser.ConfigParser() 97 | logger.debug("Reading configfile from {}".format(fn)) 98 | config.read(fn) 99 | 100 | # set log file 101 | if "global" in config.sections() and "logfile" in config["global"]: 102 | if "global" in config.sections() and "rotatelogfileat" in config["global"]: 103 | handler = logging.handlers.TimedRotatingFileHandler(config["global"]["logfile"], when=config["global"]["rotatelogfileat"]) 104 | else: 105 | handler = logging.FileHandler(config["global"]["logfile"]) 106 | formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') 107 | handler.setFormatter(formatter) 108 | logger.addHandler(handler) 109 | 110 | # set log level 111 | if "global" in config.sections() and "loglevel" in config["global"]: 112 | #global logger 113 | logger.setLevel(logging.getLevelName(config["global"]["loglevel"])) 114 | 115 | # if there is a configdir directory, reread everything 116 | if "global" in config.sections() and "splitconfigfiles" in config["global"]: 117 | fnlist = [fn] + [f for f in glob.glob(config["global"]["splitconfigfiles"])] 118 | logger.debug("Detected configdir directive. Reading configfiles from {}".format(fnlist)) 119 | config = configparser.ConfigParser() 120 | config.read(fnlist) 121 | 122 | if len(self._getProfilesList(config)) == 0: 123 | logger.error("invalid configfile. No docker images") 124 | sys.exit(1) 125 | 126 | for profilename in self._getProfilesList(config): 127 | conf = self._readProfileConfig(config, profilename) 128 | logger.debug("Read config for profile {} as:\n {}".format(profilename, pprint.pformat(conf))) 129 | self.registerProxy(profilename, conf) 130 | 131 | return dict([(name, self.imageParams[name]["outerport"]) for name in self.imageParams.keys()]) 132 | 133 | def _parseInt(self, x): 134 | return int(x) 135 | 136 | def _parseTruthy(self, x): 137 | if x.lower() in ["0", "false", "no"]: 138 | return False 139 | if x.lower() in ["1", "true", "yes"]: 140 | return True 141 | 142 | raise "Unknown truthy value {}".format(x) 143 | 144 | def registerProxy(self, profilename, conf): 145 | self.imageParams[profilename] = copy.deepcopy(conf) 146 | 147 | def create(self, profilename): 148 | containername = self.imageParams[profilename]["containername"] 149 | dockeroptions = self.imageParams[profilename]["dockeroptions"] 150 | imagelimit = self.imageParams[profilename]["limit"] 151 | reuse = self.imageParams[profilename]["reuse"] 152 | innerport = self.imageParams[profilename]["innerport"] 153 | checkupport = self.imageParams[profilename]["checkupport"] 154 | 155 | icount = 0 156 | if profilename in self.instancesByName: 157 | icount = len(self.instancesByName[profilename]) 158 | 159 | if imagelimit > 0 and icount >= imagelimit: 160 | logger.warn("Reached max count of {} (currently {}) for image {}".format(imagelimit, icount, profilename)) 161 | return None 162 | 163 | instance = None 164 | 165 | if reuse and icount > 0: 166 | logger.debug("Reusing existing instance for image {}".format(profilename)) 167 | instance = self.instancesByName[profilename][0] 168 | else: 169 | instance = DockerInstance(profilename, containername, innerport, checkupport, dockeroptions) 170 | instance.start() 171 | 172 | if profilename not in self.instancesByName: 173 | self.instancesByName[profilename] = [] 174 | 175 | # in case of reuse, the list will have duplicates 176 | self.instancesByName[profilename] += [instance] 177 | 178 | return instance 179 | 180 | def destroy(self, instance): 181 | profilename = instance.getProfileName() 182 | reuse = self.imageParams[profilename]["reuse"] 183 | 184 | # in case of reuse, the list will have duplicates, but remove() does not care 185 | self.instancesByName[profilename].remove(instance) 186 | 187 | # stop the instance if there is no reuse, or if this is the last instance for a reused image 188 | if not reuse or len(self.instancesByName[profilename]) == 0: 189 | instance.stop() 190 | 191 | 192 | # this class represents a single docker instance listening on a certain middleport. 193 | # The middleport is managed by the DockerPorts global object 194 | # After the docker container is started, we wait until the middleport becomes reachable 195 | # before returning 196 | class DockerInstance(): 197 | def __init__(self, profilename, containername, innerport, checkupport, dockeroptions): 198 | self._profilename = profilename 199 | self._containername = containername 200 | self._dockeroptions = dockeroptions 201 | self._innerport = innerport 202 | self._checkupport = checkupport 203 | self._instance = None 204 | 205 | def getDockerOptions(self): 206 | return self._dockeroptions 207 | 208 | def getContainerName(self): 209 | return self._containername 210 | 211 | def getMappedPort(self, inp): 212 | try: 213 | return int(self._instance.attrs["NetworkSettings"]["Ports"]["{}/tcp".format(inp)][0]["HostPort"]) 214 | except Exception as e: 215 | logger.warn("Failed to get port information for port {} from {}: {}".format(inp, self.getInstanceID(), e)) 216 | return None 217 | 218 | def getMiddlePort(self): 219 | return self.getMappedPort(self._innerport) 220 | 221 | def getMiddleCheckupPort(self): 222 | return self.getMappedPort(self._checkupport) 223 | 224 | def getProfileName(self): 225 | return self._profilename 226 | 227 | def getInstanceID(self): 228 | try: 229 | return self._instance.id 230 | except Exception as e: 231 | logger.warn("Failed to get instanceid: {}".format(e)) 232 | return "None" 233 | 234 | def start(self): 235 | # get docker client 236 | client = docker.from_env() 237 | 238 | # start instance 239 | try: 240 | logger.debug("Starting instance {} of container {} with dockeroptions {}".format(self.getProfileName(), self.getContainerName(), pprint.pformat(self.getDockerOptions()))) 241 | clientres = client.containers.run(self.getContainerName(), **self.getDockerOptions()) 242 | self._instance = client.containers.get(clientres.id) 243 | logger.debug("Done starting instance {} of container {}".format(self.getProfileName(), self.getContainerName())) 244 | except Exception as e: 245 | logger.debug("Failed to start instance {} of container {}: {}".format(self.getProfileName(), self.getContainerName(), e)) 246 | self.stop() 247 | return False 248 | 249 | # wait until container's checkupport is available 250 | logger.debug("Started instance on middleport {} with ID {}".format(self.getMiddlePort(), self.getInstanceID())) 251 | if self.__waitForOpenPort(self.getMiddleCheckupPort()): 252 | logger.debug("Started instance on middleport {} with ID {} has open port {}".format(self.getMiddlePort(), self.getInstanceID(), self.getMiddleCheckupPort())) 253 | return True 254 | else: 255 | logger.debug("Started instance on middleport {} with ID {} has closed port {}".format(self.getMiddlePort(), self.getInstanceID(), self.getMiddleCheckupPort())) 256 | self.stop() 257 | return False 258 | 259 | def stop(self): 260 | mp = self.getMiddlePort() 261 | cid = self.getInstanceID() 262 | logger.debug("Killing and removing {} (middleport {})".format(cid, mp)) 263 | try: 264 | self._instance.remove(force=True) 265 | except Exception as e: 266 | logger.warn("Failed to remove instance for middleport {}, id {}".format(mp, cid)) 267 | return False 268 | return True 269 | 270 | def __isPortOpen(self, port, readtimeout=0.1): 271 | s = socket.socket() 272 | ret = False 273 | logger.debug("Checking whether port {} is open...".format(port)) 274 | if port == None: 275 | time.sleep(readtimeout) 276 | else: 277 | try: 278 | s.connect(("0.0.0.0", port)) 279 | # just connecting is not enough, we should try to read and get at least 1 byte back 280 | # since the daemon in the container might not have started accepting connections yet, while docker-proxy does 281 | s.settimeout(readtimeout) 282 | data = s.recv(1) 283 | ret = len(data) > 0 284 | except socket.error: 285 | ret = False 286 | 287 | logger.debug("result = ".format(ret)) 288 | s.close() 289 | return ret 290 | 291 | def __waitForOpenPort(self, port, timeout=5, step=0.1): 292 | started = time.time() 293 | 294 | while started + timeout >= time.time(): 295 | if self.__isPortOpen(port): 296 | return True 297 | time.sleep(step) 298 | return False 299 | 300 | class LoggingProxyClient(ProxyClient): 301 | def dataReceived(self, data): 302 | payloadlen = len(data) 303 | self.factory.server.upBytes += payloadlen 304 | self.peer.transport.write(data) 305 | 306 | class LoggingProxyClientFactory(ProxyClientFactory): 307 | protocol = LoggingProxyClient 308 | 309 | class DockerProxyServer(ProxyServer): 310 | clientProtocolFactory = LoggingProxyClientFactory 311 | reactor = None 312 | 313 | def __init__(self): 314 | super().__init__() 315 | self.downBytes = 0 316 | self.upBytes = 0 317 | self.sessionID = "".join([random.choice(string.ascii_letters) for _ in range(16)]) 318 | self.sessionStart = time.time() 319 | 320 | # This is a reimplementation, except that we want to specify host and port... 321 | def connectionMade(self): 322 | # Don't read anything from the connecting client until we have 323 | # somewhere to send it to. 324 | self.transport.pauseProducing() 325 | 326 | client = self.clientProtocolFactory() 327 | client.setServer(self) 328 | 329 | if self.reactor is None: 330 | from twisted.internet import reactor 331 | self.reactor = reactor 332 | global globalDockerPorts 333 | self.dockerinstance = globalDockerPorts.create(self.factory.profilename) 334 | if self.dockerinstance == None: 335 | self.transport.write(bytearray("Maximum connection-count reached. Try again later.\r\n", "utf-8")) 336 | self.transport.loseConnection() 337 | else: 338 | logger.info("[Session {}] Incoming connection for image {} from {} at {}".format(self.sessionID, self.dockerinstance.getProfileName(), 339 | self.transport.getPeer(), self.sessionStart)) 340 | self.reactor.connectTCP("0.0.0.0", self.dockerinstance.getMiddlePort(), client) 341 | 342 | def connectionLost(self, reason): 343 | profilename = "" 344 | if self.dockerinstance != None: 345 | global globalDockerPorts 346 | globalDockerPorts.destroy(self.dockerinstance) 347 | profilename = self.dockerinstance.getProfileName() 348 | self.dockerinstance = None 349 | super().connectionLost(reason) 350 | timenow = time.time() 351 | logger.info("[Session {}] server disconnected session for image {} from {} (start={}, end={}, duration={}, upBytes={}, downBytes={}, totalBytes={})".format( 352 | self.sessionID, profilename, self.transport.getPeer(), 353 | self.sessionStart, timenow, timenow-self.sessionStart, 354 | self.upBytes, self.downBytes, self.upBytes + self.downBytes)) 355 | 356 | def dataReceived(self, data): 357 | payloadlen = len(data) 358 | self.downBytes += payloadlen 359 | self.peer.transport.write(data) 360 | 361 | 362 | class DockerProxyFactory(ProxyFactory): 363 | protocol = DockerProxyServer 364 | 365 | def __init__(self, profilename): 366 | self.profilename = profilename 367 | 368 | 369 | if __name__ == "__main__": 370 | import sys 371 | 372 | globalDockerPorts = DockerPorts() 373 | portsAndNames = globalDockerPorts.readConfig(sys.argv[1] if len(sys.argv) > 1 else '/etc/docker-tcp-switchboard.conf') 374 | 375 | for (name, outerport) in portsAndNames.items(): 376 | logger.debug("Listening on port {}".format(outerport)) 377 | reactor.listenTCP(outerport, DockerProxyFactory(name), interface=sys.argv[2] if len(sys.argv) > 2 else '') 378 | reactor.run() 379 | 380 | 381 | -------------------------------------------------------------------------------- /example/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | 3 | #--------- Install usefull tools ----------- 4 | RUN apt-get update && apt-get install -y \ 5 | openssh-server \ 6 | git \ 7 | curl \ 8 | vim \ 9 | apt-utils \ 10 | iputils-ping \ 11 | sudo 12 | 13 | #--------- SETUP System ----------- 14 | 15 | # add user and sudo 16 | RUN useradd -ms /bin/bash -g sudo sshuser 17 | # username= sshuser, password= password 18 | RUN echo 'sshuser:password' | chpasswd 19 | RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers 20 | 21 | # Config SSH 22 | # Set SSH timeout 23 | RUN mkdir /var/run/sshd 24 | #set timeout to auto-disconnect when idle (see man sshd) 25 | #RUN echo 'ClientAliveInterval 60' >> /etc/ssh/sshd_config 26 | #RUN echo 'ClientAliveCountMax 10' >> /etc/ssh/sshd_config 27 | #RUN echo 'TCPKeepAlive no' >> /etc/ssh/sshd_config 28 | 29 | #--------- SETUP USER ----------- 30 | 31 | USER sshuser 32 | WORKDIR /home/sshuser/ 33 | 34 | #------------- ROOT ------------- 35 | 36 | USER root 37 | 38 | # Setup SSH 39 | EXPOSE 22 40 | # Start SSH Deamon in "not detach" mode. Once SSH connction breaks the container stops 41 | CMD ["/usr/sbin/sshd", "-D"] 42 | -------------------------------------------------------------------------------- /example/docker-tcp-switchboard.conf: -------------------------------------------------------------------------------- 1 | [global] 2 | splitconfigfiles = /etc/docker-tcp-switchboard.d/*.conf 3 | logfile = /var/log/docker-tcp-switchboard.log 4 | loglevel = DEBUG 5 | 6 | [testssh] 7 | innerport = 22 8 | outerport = 32768 9 | container = testcontainer 10 | limit = 5 11 | reuse = false -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Twisted 2 | docker 3 | -------------------------------------------------------------------------------- /travis-ci-test/README: -------------------------------------------------------------------------------- 1 | setupenv.sh starts docker, builds the images and installs some packages to run the switchboard. 2 | runtest.sh starts the switchboard and then runs client.py, which runs the tests against echoserv and upperserv. 3 | -------------------------------------------------------------------------------- /travis-ci-test/client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from twisted.internet.protocol import Protocol, ClientFactory 4 | from twisted.internet import reactor 5 | import twisted.internet.error 6 | import sys, pprint, time, threading 7 | 8 | # for client: a translate function to be passed to the protocol class, to indicate what the expect in return 9 | 10 | errorcode = 0 11 | lock = threading.Lock() 12 | connectCount = 0 13 | 14 | class Echo(Protocol): 15 | def __init__(self, factory, repeats = 10, data = "xxx", translationFunction = lambda x: x, delay = 0): 16 | self.lines = [] 17 | self.factory = factory 18 | self.repeats = repeats 19 | self.sendData = data 20 | self.translationFunction = translationFunction 21 | self.delay = delay 22 | self.counter = 0 23 | 24 | def dataReceived(self, data): 25 | self.lines += ["S: {}".format(data.decode("utf-8"))] 26 | reply = "" 27 | if self.counter >= self.repeats: 28 | reply = "quit\n" 29 | else: 30 | reply = self.sendData + "\n" 31 | self.lines += ["C: {}".format(reply)] 32 | self.transport.write(reply.encode("utf-8")) 33 | self.counter += 1 34 | time.sleep(self.delay) 35 | 36 | def connectionLost(self, reason): 37 | #self.lines += ["?: {}".format(reason)] 38 | self.lines += ["?: Done"] 39 | res = self.verifyOutcome() 40 | self.factory.logResult(res) 41 | if "weird" == res: 42 | global errorcode 43 | errorcode = 1 44 | 45 | def verifyOutcome(self): 46 | successexpected = ['S: Hello, this is an echo service!\n'] 47 | successexpected += ['C: '+self.sendData+'\n', 'S: '+self.translationFunction(self.sendData+'\n')] * self.repeats 48 | successexpected += ['C: quit\n', 'S: Goodbye.\n'+self.translationFunction('quit\n'), 'C: quit\n', '?: Done'] 49 | 50 | fullexpected = ['S: Maximum connection-count reached. Try again later.\r\n', 'C: xxx\n', '?: Done'] 51 | 52 | if successexpected == self.lines: 53 | return "success" 54 | if fullexpected == self.lines: 55 | return "full" 56 | else: 57 | print("Got weird lines ::::") 58 | pprint.pprint(self.lines) 59 | print("--> Expected ::::") 60 | pprint.pprint(successexpected) 61 | return "weird" 62 | 63 | class UpperEcho(Echo): 64 | def __init__(self, factory, repeats = 10, data = "xxx", delay = 0): 65 | super().__init__(factory, repeats, data, lambda x: x.upper(), delay) 66 | 67 | 68 | class EchoClientFactory(ClientFactory): 69 | def __init__(self, protocol = None, goodconn = None, maxconn = None): 70 | self.protocol = protocol 71 | self.goodconn = goodconn 72 | self.maxconn = maxconn 73 | self.results = [] 74 | 75 | def logResult(self, x): 76 | self.results += [x] 77 | 78 | fullcount = len([n for n in self.results if n == "full"]) 79 | successcount = len([n for n in self.results if n == "success"]) 80 | weirdcount = len([n for n in self.results if n == "weird"]) 81 | total = len(self.results) 82 | 83 | if weirdcount > 0: 84 | raise Exception("Detected weird connections. Abort") 85 | 86 | if total == self.maxconn and successcount != self.goodconn: 87 | raise Exception("All {} connections finished, but success count ({}) is not the expected {}".format(total, successcount, self.goodconn)) 88 | 89 | if total > self.maxconn: 90 | raise Exception("Counted more connections ({}) than the {} connections anticipated".format(total, self.maxconn)) 91 | 92 | 93 | 94 | def startedConnecting(self, connector): 95 | global connectCount, lock 96 | with lock: 97 | connectCount += 1 98 | print('Started to connect.%d' % connectCount) 99 | 100 | def buildProtocol(self, addr): 101 | global connectCount 102 | print('Connected.%d' % connectCount) 103 | return self.protocol(self) 104 | 105 | def clientConnectionLost(self, connector, reason): 106 | global connectCount, lock 107 | with lock: 108 | connectCount -= 1 109 | print('Lost connection.%d' % connectCount) 110 | #print('Lost connection.%d Reason:' % connectCount, reason) 111 | if connectCount == 0: 112 | reactor.stop() 113 | 114 | def clientConnectionFailed(self, connector, reason): 115 | global connectCount, lock 116 | with lock: 117 | connectCount -= 1 118 | print('Failed connection.%d' % connectCount) 119 | #print('Failed connection.%d Reason:' % connectCount, reason) 120 | global errorcode 121 | errorcode = 1 122 | if connectCount == 0: 123 | reactor.stop() 124 | 125 | echoserv_successcount = int(sys.argv[1]) 126 | echoserv_totalcount = int(sys.argv[2]) 127 | upperserv_successcount = int(sys.argv[3]) 128 | upperserv_totalcount = int(sys.argv[4]) 129 | 130 | ecf = EchoClientFactory(Echo, echoserv_successcount, echoserv_totalcount) 131 | uecf = EchoClientFactory(UpperEcho, upperserv_successcount, upperserv_totalcount) 132 | 133 | for x in range(echoserv_totalcount): 134 | reactor.connectTCP("localhost", 2222, ecf) 135 | for x in range(upperserv_totalcount): 136 | reactor.connectTCP("localhost", 2223, uecf) 137 | 138 | reactor.run() 139 | 140 | sys.exit(errorcode) 141 | -------------------------------------------------------------------------------- /travis-ci-test/config.ini: -------------------------------------------------------------------------------- 1 | [global] 2 | loglevel = DEBUG 3 | logfile = /tmp/logfile 4 | splitconfigfiles = ./config.ini.d/*.ini 5 | 6 | [dockeroptions] 7 | dns = [ "8.8.8.8", "1.2.3.4" ] 8 | 9 | -------------------------------------------------------------------------------- /travis-ci-test/config.ini.d/echoserv.ini: -------------------------------------------------------------------------------- 1 | [profile:echoserv] 2 | container = echoserv 3 | outerport = 2222 4 | innerport = 8000 5 | limit = 4 6 | -------------------------------------------------------------------------------- /travis-ci-test/config.ini.d/upperserv.ini: -------------------------------------------------------------------------------- 1 | [profile:upperserv] 2 | container = upperserv 3 | outerport = 2223 4 | innerport = 8000 5 | limit = 7 6 | -------------------------------------------------------------------------------- /travis-ci-test/runtest_basic.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # start the switchboard 4 | ../docker-tcp-switchboard.py config.ini & 5 | DAEMONPID=$! 6 | function cleanup { 7 | echo "Cleaning up..." 8 | kill -9 $DAEMONPID || true 9 | cat /tmp/logfile 10 | rm -f /tmp/logfile 11 | } 12 | trap cleanup EXIT 13 | 14 | sleep 2 # give time to startup 15 | 16 | timeout --signal=KILL 90 ./client.py 4 10 7 10 17 | sleep 3 18 | 19 | if [ $(docker ps -aq|wc -l) -eq 0 ]; 20 | then 21 | echo "Success: All containers are gone"; 22 | else 23 | echo "Fail: Some containers remain"; 24 | docker ps -a; 25 | false; 26 | fi 27 | -------------------------------------------------------------------------------- /travis-ci-test/runtest_kill.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # start the switchboard 4 | ../docker-tcp-switchboard.py config.ini & 5 | DAEMONPID=$! 6 | function cleanup { 7 | echo "Cleaning up..." 8 | kill -9 $DAEMONPID || true # daemon could already be dead 9 | kill -9 $NCPID || true # netcat is hopefully disconnected already 10 | rm -f /tmp/logfile 11 | } 12 | trap cleanup EXIT 13 | 14 | sleep 2 # give time to startup 15 | 16 | # open a connection 17 | ((while true; do echo hi; sleep 1; done; sleep 1000) | nc 0 2222 ) & 18 | NCPID=$! 19 | sleep 10 20 | 21 | # show running containers, for debugging the test 22 | docker ps -a 23 | 24 | # now kill the server, which should clean up everything 25 | kill -s SIGTERM $DAEMONPID 26 | sleep 2 27 | 28 | # Show logfile 29 | cat /tmp/logfile 30 | 31 | if [ $(docker ps -aq|wc -l) -eq 0 ]; 32 | then 33 | echo "Success: All containers are gone"; 34 | else 35 | echo "Fail: Some containers remain"; 36 | docker ps -a; 37 | false; 38 | fi 39 | -------------------------------------------------------------------------------- /travis-ci-test/runtest_rebuild.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # this test does a rebuild of one of the images while 4 | # a connection is active 5 | 6 | # start the switchboard 7 | ../docker-tcp-switchboard.py config.ini & 8 | DAEMONPID=$! 9 | function cleanup { 10 | echo "Cleaning up..." 11 | kill -9 $DAEMONPID || true # daemon could already be dead 12 | kill -9 $NCPID || true # netcat is hopefully disconnected already 13 | cat /tmp/logfile 14 | rm -f /tmp/logfile 15 | } 16 | trap cleanup EXIT 17 | 18 | sleep 2 # give time to startup 19 | 20 | # open a connection 21 | ((while true; do echo hi; sleep 1; done; sleep 1000) | nc 0 2222 ) & 22 | NCPID=$! 23 | sleep 10 24 | 25 | # show running containers, for debugging the test 26 | docker ps -a 27 | 28 | # now rebuild the image 29 | docker build -t echoserv -f testimages/Dockerfile.echoserv-mangled testimages 30 | sleep 2 31 | 32 | # kill the client 33 | kill -s SIGTERM $NCPID 34 | sleep 2 35 | 36 | # show running containers, for debugging the test 37 | docker ps -a 38 | sleep 2 39 | 40 | # and now stop the server 41 | kill -s SIGTERM $DAEMONPID 42 | sleep 2 43 | 44 | if [ $(docker ps -aq|wc -l) -eq 0 ]; 45 | then 46 | echo "Success: All containers are gone"; 47 | else 48 | echo "Fail: Some containers remain"; 49 | docker ps -a; 50 | false; 51 | fi 52 | 53 | # restore the old image 54 | docker build -t echoserv -f testimages/Dockerfile.echoserv testimages 55 | -------------------------------------------------------------------------------- /travis-ci-test/setupenv.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # installing some packages before docker install maybe messes things up 4 | #sudo apt-get update 5 | #apt-cache search python3-pip || true 6 | #apt-cache search pip3 || true 7 | #sudo apt-get install -y python3-pip 8 | sudo pip install -r ../requirements.txt 9 | 10 | # install latest docker if it doesn't already exist 11 | if [ ! -e /etc/init.d/docker ]; then 12 | curl -sSL https://get.docker.com/ | sudo sh 13 | fi 14 | # start docker, and wait until it's likely up 15 | (sudo /etc/init.d/docker start && sleep 3 ) || true 16 | 17 | # give 'everyone' access... 18 | sudo chmod o+rw /var/run/docker.sock 19 | 20 | # build echoserv and upperserv 21 | docker build -t echoserv -f testimages/Dockerfile.echoserv testimages 22 | docker build -t upperserv -f testimages/Dockerfile.upperserv testimages 23 | sleep 2 24 | 25 | 26 | -------------------------------------------------------------------------------- /travis-ci-test/testimages/Dockerfile.echoserv: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get -y upgrade && apt-get -y install python-twisted 3 | EXPOSE 8000 4 | ADD echoserv.py /server.py 5 | CMD /server.py 6 | -------------------------------------------------------------------------------- /travis-ci-test/testimages/Dockerfile.echoserv-mangled: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get -y upgrade && apt-get -y install python-twisted 3 | EXPOSE 8000 4 | ADD echoserv.py /server.py 5 | RUN echo hello > /world 6 | CMD /server.py 7 | -------------------------------------------------------------------------------- /travis-ci-test/testimages/Dockerfile.upperserv: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get -y upgrade && apt-get -y install python-twisted 3 | EXPOSE 8000 4 | ADD echoserv.py /server.py 5 | CMD /server.py upper 6 | -------------------------------------------------------------------------------- /travis-ci-test/testimages/echoserv.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Copyright (c) Twisted Matrix Laboratories. 4 | # See LICENSE for details. 5 | 6 | from twisted.internet.protocol import Protocol, Factory 7 | from twisted.internet import reactor 8 | import sys 9 | 10 | ### Protocol Implementation 11 | 12 | makeUpper = False 13 | 14 | # This is just about the simplest possible protocol 15 | class Echo(Protocol): 16 | def connectionMade(self): 17 | self.transport.write("Hello, this is an echo service!\n") 18 | 19 | def dataReceived(self, data): 20 | if data.lower().startswith("quit"): 21 | self.transport.write("Goodbye.\n".encode("utf-8")) 22 | self.transport.loseConnection() 23 | global makeUpper 24 | if makeUpper: 25 | data = data.upper() 26 | self.transport.write(data) 27 | 28 | 29 | def main(): 30 | f = Factory() 31 | f.protocol = Echo 32 | reactor.listenTCP(8000, f) 33 | reactor.run() 34 | 35 | if __name__ == '__main__': 36 | if len(sys.argv) > 1 and sys.argv[1] == "upper": 37 | makeUpper = True 38 | main() 39 | --------------------------------------------------------------------------------