├── .gitignore ├── LICENSE ├── README-old.md ├── README.md ├── bin ├── mesh ├── mesos-docker └── mesos-docker-setup ├── tutorial.md └── tutorial ├── mesos-docker.1.png └── mesos-docker.2.png /.gitignore: -------------------------------------------------------------------------------- 1 | /.reviewboardrc 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, and 10 | distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by the copyright 13 | owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all other entities 16 | that control, are controlled by, or are under common control with that entity. 17 | For the purposes of this definition, "control" means (i) the power, direct or 18 | indirect, to cause the direction or management of such entity, whether by 19 | contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the 20 | outstanding shares, or (iii) beneficial ownership of such entity. 21 | 22 | "You" (or "Your") shall mean an individual or Legal Entity exercising 23 | permissions granted by this License. 24 | 25 | "Source" form shall mean the preferred form for making modifications, including 26 | but not limited to software source code, documentation source, and configuration 27 | files. 28 | 29 | "Object" form shall mean any form resulting from mechanical transformation or 30 | translation of a Source form, including but not limited to compiled object code, 31 | generated documentation, and conversions to other media types. 32 | 33 | "Work" shall mean the work of authorship, whether in Source or Object form, made 34 | available under the License, as indicated by a copyright notice that is included 35 | in or attached to the work (an example is provided in the Appendix below). 36 | 37 | "Derivative Works" shall mean any work, whether in Source or Object form, that 38 | is based on (or derived from) the Work and for which the editorial revisions, 39 | annotations, elaborations, or other modifications represent, as a whole, an 40 | original work of authorship. For the purposes of this License, Derivative Works 41 | shall not include works that remain separable from, or merely link (or bind by 42 | name) to the interfaces of, the Work and Derivative Works thereof. 43 | 44 | "Contribution" shall mean any work of authorship, including the original version 45 | of the Work and any modifications or additions to that Work or Derivative Works 46 | thereof, that is intentionally submitted to Licensor for inclusion in the Work 47 | by the copyright owner or by an individual or Legal Entity authorized to submit 48 | on behalf of the copyright owner. For the purposes of this definition, 49 | "submitted" means any form of electronic, verbal, or written communication sent 50 | to the Licensor or its representatives, including but not limited to 51 | communication on electronic mailing lists, source code control systems, and 52 | issue tracking systems that are managed by, or on behalf of, the Licensor for 53 | the purpose of discussing and improving the Work, but excluding communication 54 | that is conspicuously marked or otherwise designated in writing by the copyright 55 | owner as "Not a Contribution." 56 | 57 | "Contributor" shall mean Licensor and any individual or Legal Entity on behalf 58 | of whom a Contribution has been received by Licensor and subsequently 59 | incorporated within the Work. 60 | 61 | 2. Grant of Copyright License. 62 | 63 | Subject to the terms and conditions of this License, each Contributor hereby 64 | grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, 65 | irrevocable copyright license to reproduce, prepare Derivative Works of, 66 | publicly display, publicly perform, sublicense, and distribute the Work and such 67 | Derivative Works in Source or Object form. 68 | 69 | 3. Grant of Patent License. 70 | 71 | Subject to the terms and conditions of this License, each Contributor hereby 72 | grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, 73 | irrevocable (except as stated in this section) patent license to make, have 74 | made, use, offer to sell, sell, import, and otherwise transfer the Work, where 75 | such license applies only to those patent claims licensable by such Contributor 76 | that are necessarily infringed by their Contribution(s) alone or by combination 77 | of their Contribution(s) with the Work to which such Contribution(s) was 78 | submitted. If You institute patent litigation against any entity (including a 79 | cross-claim or counterclaim in a lawsuit) alleging that the Work or a 80 | Contribution incorporated within the Work constitutes direct or contributory 81 | patent infringement, then any patent licenses granted to You under this License 82 | for that Work shall terminate as of the date such litigation is filed. 83 | 84 | 4. Redistribution. 85 | 86 | You may reproduce and distribute copies of the Work or Derivative Works thereof 87 | in any medium, with or without modifications, and in Source or Object form, 88 | provided that You meet the following conditions: 89 | 90 | You must give any other recipients of the Work or Derivative Works a copy of 91 | this License; and 92 | You must cause any modified files to carry prominent notices stating that You 93 | changed the files; and 94 | You must retain, in the Source form of any Derivative Works that You distribute, 95 | all copyright, patent, trademark, and attribution notices from the Source form 96 | of the Work, excluding those notices that do not pertain to any part of the 97 | Derivative Works; and 98 | If the Work includes a "NOTICE" text file as part of its distribution, then any 99 | Derivative Works that You distribute must include a readable copy of the 100 | attribution notices contained within such NOTICE file, excluding those notices 101 | that do not pertain to any part of the Derivative Works, in at least one of the 102 | following places: within a NOTICE text file distributed as part of the 103 | Derivative Works; within the Source form or documentation, if provided along 104 | with the Derivative Works; or, within a display generated by the Derivative 105 | Works, if and wherever such third-party notices normally appear. The contents of 106 | the NOTICE file are for informational purposes only and do not modify the 107 | License. You may add Your own attribution notices within Derivative Works that 108 | You distribute, alongside or as an addendum to the NOTICE text from the Work, 109 | provided that such additional attribution notices cannot be construed as 110 | modifying the License. 111 | You may add Your own copyright statement to Your modifications and may provide 112 | additional or different license terms and conditions for use, reproduction, or 113 | distribution of Your modifications, or for any such Derivative Works as a whole, 114 | provided Your use, reproduction, and distribution of the Work otherwise complies 115 | with the conditions stated in this License. 116 | 117 | 5. Submission of Contributions. 118 | 119 | Unless You explicitly state otherwise, any Contribution intentionally submitted 120 | for inclusion in the Work by You to the Licensor shall be under the terms and 121 | conditions of this License, without any additional terms or conditions. 122 | Notwithstanding the above, nothing herein shall supersede or modify the terms of 123 | any separate license agreement you may have executed with Licensor regarding 124 | such Contributions. 125 | 126 | 6. Trademarks. 127 | 128 | This License does not grant permission to use the trade names, trademarks, 129 | service marks, or product names of the Licensor, except as required for 130 | reasonable and customary use in describing the origin of the Work and 131 | reproducing the content of the NOTICE file. 132 | 133 | 7. Disclaimer of Warranty. 134 | 135 | Unless required by applicable law or agreed to in writing, Licensor provides the 136 | Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, 137 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, 138 | including, without limitation, any warranties or conditions of TITLE, 139 | NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are 140 | solely responsible for determining the appropriateness of using or 141 | redistributing the Work and assume any risks associated with Your exercise of 142 | permissions under this License. 143 | 144 | 8. Limitation of Liability. 145 | 146 | In no event and under no legal theory, whether in tort (including negligence), 147 | contract, or otherwise, unless required by applicable law (such as deliberate 148 | and grossly negligent acts) or agreed to in writing, shall any Contributor be 149 | liable to You for damages, including any direct, indirect, special, incidental, 150 | or consequential damages of any character arising as a result of this License or 151 | out of the use or inability to use the Work (including but not limited to 152 | damages for loss of goodwill, work stoppage, computer failure or malfunction, or 153 | any and all other commercial damages or losses), even if such Contributor has 154 | been advised of the possibility of such damages. 155 | 156 | 9. Accepting Warranty or Additional Liability. 157 | 158 | While redistributing the Work or Derivative Works thereof, You may choose to 159 | offer, and charge a fee for, acceptance of support, warranty, indemnity, or 160 | other liability obligations and/or rights consistent with this License. However, 161 | in accepting such obligations, You may act only on Your own behalf and on Your 162 | sole responsibility, not on behalf of any other Contributor, and only if You 163 | agree to indemnify, defend, and hold each Contributor harmless for any liability 164 | incurred by, or claims asserted against, such Contributor by reason of your 165 | accepting any such warranty or additional liability. 166 | 167 | END OF TERMS AND CONDITIONS 168 | 169 | APPENDIX: How to apply the Apache License to your work 170 | 171 | To apply the Apache License to your work, attach the following boilerplate 172 | notice, with the fields enclosed by brackets "[]" replaced with your own 173 | identifying information. (Don't include the brackets!) The text should be 174 | enclosed in the appropriate comment syntax for the file format. We also 175 | recommend that a file or class name and description of purpose be included on 176 | the same "printed page" as the copyright notice for easier identification within 177 | third-party archives. 178 | 179 | Copyright 2013 Mesosphere, Inc. 180 | 181 | Licensed under the Apache License, Version 2.0 (the "License"); 182 | you may not use this file except in compliance with the License. 183 | You may obtain a copy of the License at 184 | 185 | http://www.apache.org/licenses/LICENSE-2.0 186 | 187 | Unless required by applicable law or agreed to in writing, software 188 | distributed under the License is distributed on an "AS IS" BASIS, 189 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 190 | See the License for the specific language governing permissions and 191 | limitations under the License. 192 | -------------------------------------------------------------------------------- /README-old.md: -------------------------------------------------------------------------------- 1 | # Mesos / Docker Integration 2 | 3 | NOTE: This is a proof of concept for running Dockers on Mesos via Marathon. It 4 | has a number of limitations and is not recommended for production use. 5 | First class Docker support is coming up in Mesos 0.19 via [Deimos](https://github.com/mesosphere/deimos). 6 | 7 | ## Summary 8 | 9 | Docker containers provide a consistent, compact and flexible means of packaging application builds. Delivering applications with Docker on Mesos promises a truly elastic, efficient and consistent platform for delivering a range of applications on premises or in the cloud. 10 | This repository combines Docker with the [Mesos](http://mesos.apache.org) cluster scheduler and [Marathon](https://github.com/mesosphere/marathon) but could also be used with other Mesos frameworks such as [Chronos](https://github.com/airbnb/chronos). 11 | 12 | ## Getting Started 13 | 14 | This [tutorial](https://github.com/mesosphere/mesos-docker/blob/master/tutorial.md) shows you how you can get started with Mesos / Docker. 15 | 16 | ## Help 17 | 18 | If you have questions, please post on the [Marathon Framework Group](https://groups.google.com/forum/?hl=en#!forum/marathon-framework) email list. 19 | You can find Mesos support in the `#mesos` channel on [freenode][freenode] (IRC). 20 | The team at [Mesosphere](http://mesosphere.io) is also happy to answer any questions. 21 | 22 | ## Authors 23 | 24 | * [Jason Dusek](https://github.com/solidsnack) 25 | 26 | ## Requirements 27 | 28 | * [Mesos][Mesos] 0.14+ 29 | 30 | [Mesos]: http://incubator.apache.org/mesos/ "Apache Mesos" 31 | [freenode]: http://freenode.net/ "IRC channels" 32 | 33 | ## How Mesos-Docker Integration Works 34 | 35 | A Mesos cluster comprises a few masters and many slaves. As work is farmed out 36 | to them, Mesos slaves delegate to _executors_ for the setup and teardown of 37 | individual tasks. It is the executor that manages communication between the 38 | slave and the Docker daemon. The Docker daemon manages caching and launch of 39 | Docker images, which can be pulled from the global Docker index or a local 40 | registry. 41 | 42 | ![Marathon can launch and monitor service containers from one or more Docker registries using the Docker executor for Mesos.](tutorial/mesos-docker.1.png) 43 | 44 | When a Docker container is started as a Mesos task, it runs beneath the Docker 45 | daemon on a slave server. Although the Docker container does not run as a true 46 | child process of the executor, as it would have under the old Docker 47 | standalone mode, the executor is able to manage translation of resource 48 | limits, signals and Mesos process control messages to appropriate calls of the 49 | `docker` tool. Fine-grained resource monitoring, forthcoming in mainline Mesos 50 | and part of some service management tools, is hard to do right if the 51 | container does not run directly under the executor -- so this architecture may 52 | be revised in the future. 53 | 54 | ![The executor, monitored by the Mesos slave, delegates to the local Docker daemon for image discovery and management. The executor communicates with Marathon via the Mesos master and ensures that Docker enfores the specified resource limitations.](tutorial/mesos-docker.2.png) 55 | 56 | When a user requests a Docker container with Marathon, using the web UI or the 57 | HTTP API as in the examples above, the request is ultimately delegated to the 58 | LXC tools, by way of Mesos and Docker. 59 | 60 | * Marathon makes a resource request from the Mesos master and then waits to 61 | accept an appropriate offer. 62 | 63 | * Once an offer is accepted, the Mesos master sends the task's specification 64 | to the slave. 65 | 66 | * On the slave server, the Mesos slave daemon calls the Mesos-Docker executor 67 | which in turn calls the `docker` command line tool. 68 | 69 | * The `docker` command line tool talks to the local Docker daemon, which 70 | manages interactions with the cache of images and the LXC tools. 71 | 72 | * If the image is cached, it will be run from the cache. Otherwise, the Docker 73 | daemon contacts a Docker registry to retrieve it. 74 | 75 | * The Docker daemon runs the container under the LXC tools. 76 | 77 | 78 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Deprecated! 2 | 3 | This project was a proof of concept for Docker support in Mesos via an executor and is no longer maintained. 4 | 5 | As of version 0.20.0, Docker containers are natively supported by Mesos! To learn how to configure Docker and Mesos, see the [Mesosphere tutorial](https://mesosphere.com/learn/launch-docker-container-on-mesosphere/). 6 | 7 | Marathon offers [easy scaling and deployment of Docker containers on Mesos](https://mesosphere.github.io/marathon/docs/native-docker.html). -------------------------------------------------------------------------------- /bin/mesh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import json 3 | import logging 4 | import logging.handlers 5 | import os 6 | import signal 7 | import subprocess 8 | import sys 9 | import threading 10 | import time 11 | import traceback 12 | 13 | import mesos 14 | import mesos_pb2 15 | 16 | proc = None # Global process object 17 | # This script is meant to execute one and only one subprocess at a time. This 18 | # simplifies the interaction with Python threads and the driver thread. We 19 | # don't do any cleanup of the driver or other threads, instead we just shut 20 | # everthing down with os._exit(). So you probably shouldn't use this program 21 | # as a module in your project... 22 | 23 | log = logging.getLogger(os.path.basename(sys.argv[0])) 24 | log.setLevel(logging.DEBUG) 25 | # TODO: Use '/var/run/syslog' on OS X (check for one, then the other) 26 | syslog = logging.handlers.SysLogHandler(address = '/dev/log') 27 | syslog_format = logging.Formatter("%(name)s[%(process)d]: %(message)s") 28 | syslog.setFormatter(syslog_format) 29 | log.addHandler(syslog) 30 | 31 | # Handles signals, passed as negative numbers, and ensures worker process is 32 | # cleaned up if it exists. 33 | # 34 | # This function is called many times throughout the code, but because it's 35 | # final statement is a call to os._exit(), it will only ever actually be 36 | # called one time. 37 | def exit(returncode): 38 | try: 39 | cleanup_proc() 40 | except Exception, e: 41 | log_exc() 42 | finally: 43 | os._exit( ((-returncode) + 128) if returncode < 0 else returncode ) 44 | 45 | def log_exc(): 46 | for line in traceback.format_exc().splitlines(): 47 | log.error(line) 48 | 49 | def json_pp(thing): 50 | s = json.dumps(thing, indent=2, separators=(',', ': '), sort_keys=True) 51 | data_lines = s.splitlines()[1:-1] 52 | return "{ " + '\n'.join([data_lines[0][2:]] + data_lines[1:]) + " }" 53 | 54 | def terminate_process_and_group(pid, sleep=2): 55 | if proc.poll() is None: 56 | os.kill(pid, signal.SIGTERM) 57 | time.sleep(2) 58 | try: # I hope the PID was not recycled yet... 59 | os.killpg(pid, signal.SIGKILL) 60 | except OSError, e: 61 | pass # There were no processes in the group; hurray. 62 | 63 | cleaned_up_already = False 64 | def cleanup_proc(sleep=2): 65 | global proc 66 | if proc is not None and not cleaned_up_already: 67 | log.info('Terminating process PID %d and subprocesses' % proc.pid) 68 | terminate_process_and_group(proc.pid, sleep) 69 | else: 70 | log.info('Nothing to shutdown') 71 | 72 | class ShExecutor(mesos.Executor): 73 | def __init__(self, args): 74 | self.args = args 75 | self.task = None 76 | self.driver = None 77 | self.runner_thread = None 78 | self.shutdown_thread = None 79 | self.sleep = 2 80 | 81 | def run(self): 82 | global proc 83 | exitcode = 2 84 | finalstate = mesos_pb2.TASK_FAILED 85 | try: 86 | log.info('ARGV ' + ' '.join(str(arg) for arg in self.args)) 87 | proc = subprocess.Popen(self.args, preexec_fn=os.setsid) 88 | self.send_state(mesos_pb2.TASK_RUNNING) 89 | log.info('Waiting for process...') 90 | proc.wait() 91 | log.info('Process exited with code: %d' % proc.returncode) 92 | exitcode = proc.returncode 93 | try: 94 | # If the process failed to kills its subprocesses, try to shut 95 | # them down gracefully. 96 | os.killpg(-proc.pid, signal.SIGTERM) 97 | except OSError, e: 98 | pass # There were no processes in the group; hurray. 99 | if proc.returncode == 0: 100 | finalstate = mesos_pb2.TASK_FINISHED 101 | else: 102 | if self.shutdown_thread: 103 | finalstate = mesos_pb2.TASK_KILLED 104 | else: 105 | finalstate = mesos_pb2.TASK_FAILED 106 | except Exception, e: 107 | log_exc() 108 | finally: 109 | self.send_state(finalstate) 110 | exit(exitcode) 111 | 112 | def spawn_shutdown_thread(self): 113 | # Only kill the main process. We allow it to cleanup its subprocesses 114 | # and then make an effort to cleanup elsewhere. 115 | self.shutdown_thread = threading.Thread(target=cleanup_proc) 116 | self.shutdown_thread.daemon = True 117 | self.shutdown_thread.start() 118 | 119 | def send_state(self, state): 120 | try: 121 | update = mesos_pb2.TaskStatus() 122 | update.task_id.value = self.task.task_id.value 123 | update.state = state 124 | self.driver.sendStatusUpdate(update) 125 | except Exception, e: 126 | log_exc() 127 | 128 | #### Mesos Executor API methods #### 129 | 130 | def registered(self, driver, executorInfo, frameworkInfo, slaveInfo): 131 | log.info('Registered with Mesos slave') 132 | 133 | def reregistered(driver, slaveInfo): 134 | log.info('Reregistered with Mesos slave') 135 | 136 | def disconnected(driver): 137 | log.warning('Disconnected from Mesos slave') 138 | 139 | def launchTask(self, driver, task): 140 | self.task = task 141 | self.driver = driver 142 | log.info('Task is: %s' % task.task_id.value) 143 | data = {} 144 | try: 145 | data = json.loads(task.data) if task.data else {} 146 | for line in json_pp(data).splitlines(): 147 | log.info(line) 148 | except Exception, e: 149 | log.error('JSON from framework is rubbish') 150 | log_exc() 151 | try: 152 | self.run_thread = threading.Thread(target=self.run) 153 | self.run_thread.daemon = True 154 | self.run_thread.start() 155 | except Exception, e: 156 | log_exc() 157 | self.send_state(mesos_pb2.TASK_FAILED) 158 | driver.stop() 159 | exit(2) 160 | 161 | def killTask(self, driver, task_id): 162 | if self.task.task_id.value == task_id.value: 163 | log.info('Asked to shutdown managed task %s' % task_id.value) 164 | self.spawn_shutdown_thread() 165 | else: 166 | log.info('Asked to shutdown unknown task %s' % task_id.value) 167 | 168 | def shutdown(self, driver): 169 | self.spawn_shutdown_thread() 170 | 171 | if __name__ == '__main__': 172 | def handler(signum, _): 173 | log.info('Exiting due to signal: '+str(signum)) 174 | exit(-signum) 175 | signal.signal(signal.SIGINT, handler) 176 | signal.signal(signal.SIGTERM, handler) 177 | signal.signal(signal.SIGABRT, handler) 178 | signal.signal(signal.SIGPIPE, handler) 179 | signal.signal(signal.SIGSEGV, handler) 180 | driver = mesos.MesosExecutorDriver(ShExecutor(sys.argv[1:])) 181 | log.info('Ready to serve!') 182 | exit(0 if driver.run() == mesos_pb2.DRIVER_STOPPED else 1) 183 | 184 | -------------------------------------------------------------------------------- /bin/mesos-docker: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import itertools 3 | import json 4 | import logging 5 | import logging.handlers 6 | import os 7 | import re 8 | import signal 9 | import subprocess 10 | import sys 11 | import threading 12 | import time 13 | import traceback 14 | 15 | import mesos 16 | import mesos_pb2 17 | 18 | def main(): 19 | def handler(signum, _): 20 | log.info('Exiting due to signal: '+str(signum)) 21 | exit(-signum) 22 | signal.signal(signal.SIGINT, handler) 23 | signal.signal(signal.SIGTERM, handler) 24 | signal.signal(signal.SIGABRT, handler) 25 | signal.signal(signal.SIGPIPE, handler) 26 | signal.signal(signal.SIGSEGV, handler) 27 | driver = mesos.MesosExecutorDriver(DockerExecutor(sys.argv[1:])) 28 | log.info('Ready to serve!') 29 | exit(0 if driver.run() == mesos_pb2.DRIVER_STOPPED else 1) 30 | 31 | # This script wraps one and only one container. 32 | cid = None 33 | cidfile = None 34 | def read_cid(): 35 | global cid 36 | if cidfile is not None: 37 | try: 38 | with open(cidfile) as f: 39 | cid = f.read().strip() 40 | except IOError: 41 | pass 42 | 43 | log = logging.getLogger('mesos-docker') 44 | log.setLevel(logging.DEBUG) 45 | syslog = logging.handlers.SysLogHandler(address = '/dev/log') 46 | syslog_format = logging.Formatter("%(name)s[%(process)d]: %(message)s") 47 | syslog.setFormatter(syslog_format) 48 | log.addHandler(syslog) 49 | 50 | class DockerExecutor(mesos.Executor): 51 | def __init__(self, args): 52 | self.args = args 53 | self.task = None 54 | self.driver = None 55 | self.image = None 56 | self.docker_options = [] 57 | self.runner_thread = None 58 | self.shutdown_thread = None 59 | self.data = {} 60 | self.env = {} 61 | 62 | def run(self): 63 | global proc 64 | exitcode = 2 65 | finalstate = mesos_pb2.TASK_FAILED 66 | resources = self.task.resources 67 | try: 68 | if self.image is None: 69 | self.image = self.args[0] 70 | args = self.args[1:] 71 | else: 72 | args = self.args[0:] 73 | cpus = [ r.scalar.value for r in resources if r.name == 'cpus' ] 74 | megs = [ r.scalar.value for r in resources if r.name == 'mem' ] 75 | relative_cpu = [ int(1024 * r) for r in cpus ] + [ None ] 76 | memory_bytes = [ int((2**20) * r) for r in megs ] + [ None ] 77 | ports = self.allocated_ports() 78 | proc = run_with_settings(self.image, self.docker_options, 79 | args, self.env, ports, 80 | relative_cpu[0], memory_bytes[0]) 81 | self.send_state(mesos_pb2.TASK_RUNNING) 82 | proc.wait() 83 | log.info('Container exited with code: %d' % proc.returncode) 84 | exitcode = proc.returncode 85 | if proc.returncode == 0: 86 | finalstate = mesos_pb2.TASK_FINISHED 87 | else: 88 | if self.shutdown_thread: 89 | finalstate = mesos_pb2.TASK_KILLED 90 | else: 91 | finalstate = mesos_pb2.TASK_FAILED 92 | except Exception, e: 93 | log_exc() 94 | finally: 95 | self.send_state(finalstate) 96 | exit(exitcode) 97 | 98 | def send_state(self, state): 99 | try: 100 | update = mesos_pb2.TaskStatus() 101 | update.task_id.value = self.task.task_id.value 102 | update.state = state 103 | self.driver.sendStatusUpdate(update) 104 | except Exception, e: 105 | log_exc() 106 | 107 | #### Mesos Executor API methods #### 108 | 109 | def registered(self, driver, executorInfo, frameworkInfo, slaveInfo): 110 | log.info('Registered with Mesos slave') 111 | 112 | def reregistered(driver, slaveInfo): 113 | log.info('Reregistered with Mesos slave') 114 | 115 | def disconnected(driver): 116 | log.warning('Disconnected from Mesos slave') 117 | 118 | def launchTask(self, driver, task): 119 | if self.task is not None: 120 | log.error('Executor was reused but this executor is not reuseable') 121 | exit(2) 122 | self.task = task 123 | self.driver = driver 124 | log.info('Task is: %s' % task.task_id.value) 125 | try: 126 | self.data = json.loads(task.data) if task.data else {} 127 | for line in json_pp(self.data).splitlines(): 128 | log.info(line) 129 | for i, port in enumerate(self.allocated_ports()): 130 | if port != 0: 131 | self.env['PORT'+str(i)] = str(port) 132 | if 'PORT0' in self.env: 133 | self.env['PORT'] = self.env['PORT0'] 134 | if 'env' in self.data: # NB: user-specified PORT* vars override ours 135 | self.env.update(self.data['env']) 136 | if 'container' in self.data: 137 | container = self.data['container'] 138 | m = re.match(r'^docker:///(.+)$', container['image']) 139 | if m is None: 140 | raise ValueError('container.image must be a docker:/// URL') 141 | self.image = m.group(1) 142 | self.docker_options = container['options'] 143 | for k, v in self.env.items(): 144 | os.environ[k] = str(v) 145 | except Exception, e: 146 | log.error('JSON from framework is rubbish') 147 | log_exc() 148 | try: 149 | self.run_thread = threading.Thread(target=self.run) 150 | self.run_thread.daemon = True 151 | self.run_thread.start() 152 | except Exception, e: 153 | log_exc() 154 | self.send_state(mesos_pb2.TASK_FAILED) 155 | exit(2) 156 | 157 | def killTask(self, driver, task_id): 158 | if self.task.task_id.value == task_id.value: 159 | log.info('Asked to shutdown managed task %s' % task_id.value) 160 | self.cleanup() 161 | else: 162 | log.info('Asked to shutdown unknown task %s' % task_id.value) 163 | 164 | def cleanup(self): 165 | if self.shutdown_thread is None: 166 | self.shutdown_thread = threading.Thread(target=cleanup_container) 167 | self.shutdown_thread.daemon = True 168 | self.shutdown_thread.start() 169 | 170 | def allocated_ports(self): 171 | range_resources = [ _.ranges.range for _ in self.task.resources 172 | if _.name == 'ports' ] 173 | ranges = itertools.chain(*range_resources) 174 | # NB: Casting long() to int() so there's no trailing 'L' in later 175 | # stringifications. Ports should only ever be shorts, anyways. 176 | ports = [ range(int(_.begin), int(_.end)+1) for _ in ranges ] 177 | return list(itertools.chain(*ports)) 178 | 179 | def shutdown(self, driver): 180 | self.cleanup() 181 | 182 | # Handles signals, passed as negative numbers, and ensures worker process is 183 | # cleaned up if it exists. 184 | # 185 | # This function shows up in many places but because it's final statement is a 186 | # call to os._exit() we can be sure it is only ever called one time. 187 | def exit(returncode): 188 | try: 189 | cleanup_container() 190 | except Exception, e: 191 | log_exc() 192 | finally: 193 | os._exit( ((-returncode) + 128) if returncode < 0 else returncode ) 194 | 195 | def log_exc(): 196 | for line in traceback.format_exc().splitlines(): 197 | log.error(line) 198 | 199 | def json_pp(thing): 200 | s = json.dumps(thing, indent=2, separators=(',', ': '), sort_keys=True) 201 | data_lines = s.splitlines()[1:-1] 202 | return "{ " + '\n'.join([data_lines[0][2:]] + data_lines[1:]) + " }" 203 | 204 | def ensure_image(f): 205 | def f_(image, *args, **kwargs): 206 | pull_once(image) 207 | return f(image, *args, **kwargs) 208 | return f_ 209 | 210 | def try_cid(f): 211 | def f_(*args, **kwargs): 212 | if cid is None: 213 | read_cid() 214 | return f(*args, **kwargs) 215 | return f_ 216 | 217 | cleaning_up_already = False # Hackish lock. 218 | @try_cid 219 | def cleanup_container(): 220 | global cid 221 | global cidfile 222 | global cleaning_up_already 223 | if cid is not None and not cleaning_up_already: 224 | cleaning_up_already = True 225 | log.info('Cleaning up container %s' % cid) 226 | subprocess.check_call(['docker', 'stop', '-t=2', cid]) 227 | subprocess.check_call(['docker', 'rm', cid]) 228 | subprocess.check_call(['rm', '-f', cidfile]) 229 | cid = None 230 | cidfile = None 231 | 232 | @ensure_image 233 | def run_with_settings(image, docker_opts=[], args=[], env={}, ports=[], 234 | relative_cpu=None, memory_bytes=None): 235 | global cidfile 236 | cidfile = '/tmp/docker_cid.' + os.urandom(8).encode('hex') 237 | cmd = [ 'run', '--cidfile', cidfile ] 238 | if relative_cpu is not None: 239 | cmd += [ '-c', str(relative_cpu) ] 240 | if memory_bytes is not None: 241 | cmd += [ '-m', str(memory_bytes) ] 242 | for k, v in env.items(): 243 | cmd += [ '-e', '%s=%s' % (k,v) ] 244 | log.info('Mesos ports: ' + str(ports)) 245 | docker_ports = inner_ports(image) 246 | log.info('Docker ports: ' + str(docker_ports)) 247 | port_pairings = itertools.izip_longest(ports, docker_ports) 248 | for allocated, target in port_pairings: 249 | if allocated is None: 250 | log.warning('Too few ports were allocated to this image.') 251 | break 252 | if target is None: 253 | log.warning('Too many ports were allocated to this image.') 254 | break 255 | cmd += [ '-p', '%d:%d' % (allocated, target) ] 256 | argv = ['docker'] + cmd + docker_opts + [ image ] + [ arg for arg in args ] 257 | log.info('ARGV ' + ' '.join(str(arg) for arg in argv)) 258 | return subprocess.Popen(argv) 259 | 260 | try: 261 | subprocess.check_output 262 | except: 263 | # For python 2.6 or earlier. 264 | def check_output(*args): 265 | p = subprocess.Popen(stdout=subprocess.PIPE, *args) 266 | stdout = p.communicate()[0] 267 | exitcode = p.wait() 268 | if exitcode: 269 | raise subprocess.CalledProcessError(exitcode, args[0]) 270 | return stdout 271 | subprocess.check_output = check_output 272 | 273 | @ensure_image 274 | def inner_ports(image): 275 | text = subprocess.check_output(['docker', 'inspect', image]) 276 | parsed = json.loads(text)[0] 277 | config = None 278 | if 'Config' in parsed: 279 | config = parsed['Config'] 280 | if 'config' in parsed and config is None: 281 | config = parsed['config'] 282 | if config: 283 | exposed = config.get('ExposedPorts', {}) 284 | if exposed and isinstance(exposed, dict): 285 | return sorted( int(k.split('/')[0]) for k in exposed.keys() ) 286 | specs = config.get('PortSpecs', []) 287 | if specs and isinstance(specs, list): 288 | return sorted( int(v.split(':')[-1]) for v in specs ) 289 | return [] # If all else fails... 290 | 291 | def pull(image): 292 | subprocess.check_call(['docker', 'pull', image]) 293 | refresh_docker_image_info(image) 294 | 295 | def pull_once(image): 296 | if not image_info(image): 297 | pull(image) 298 | 299 | def image_info(image): 300 | if image in images: 301 | return images[image] 302 | else: 303 | return refresh_docker_image_info(image) 304 | 305 | images = {} 306 | def refresh_docker_image_info(image): 307 | delim = re.compile(' +') 308 | text = subprocess.check_output(['docker', 'images', image]) 309 | records = [ delim.split(line) for line in text.splitlines() ] 310 | for record in records: 311 | if record[0] == image: 312 | images[image] = image 313 | return record 314 | 315 | if __name__ == '__main__': 316 | main() 317 | -------------------------------------------------------------------------------- /bin/mesos-docker-setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -o errexit -o nounset -o pipefail 3 | function -h { 4 | cat < /etc/apt/sources.list.d/docker.list 51 | } 52 | 53 | function mesos_egg { 54 | curl_ "$egg" --output mesos.egg 55 | easy_install ./mesos.egg 56 | } 57 | 58 | function install_mesos { 59 | curl_ "$deb" --output mesos.deb 60 | dpkg -i mesos.deb 61 | } 62 | 63 | function install_marathon { 64 | curl_ "$marathon" --output /tmp/marathon.deb 65 | dpkg -i /tmp/marathon.deb 66 | } 67 | 68 | function install_executor { 69 | mkdir -p /var/lib/mesos/executors 70 | curl_ "$executor" --output /var/lib/mesos/executors/docker 71 | } 72 | 73 | function install_httpie { 74 | easy_install httpie 75 | } 76 | 77 | # Allow user to import Python packages installed with easy_install, for 78 | # running HTTPie. This is a problem of some longevity on Ubuntu: 79 | # http://superuser.com/questions/573699/how-do-i-install-python-with-proper-permissions 80 | # Putting the user in the staff group seems like the least invasive and most 81 | # easily reversable fix. 82 | function mangle_groups { 83 | local user="${SUDO_USER:-$(id -un)}" 84 | [[ $user = root ]] || gpasswd staff -a "$user" 85 | } 86 | 87 | function curl_ { 88 | local stat cmd=( curl -sSfL "$@" ) 89 | if "${cmd[@]}" 90 | then 91 | return 0 92 | else 93 | local stat=$? 94 | { printf ': exit %3s ;' "$stat" 95 | printf ' %q' "${cmd[@]}" 96 | printf '\n' ;} >&2 97 | return $stat 98 | fi 99 | } 100 | 101 | function msg { out "$*" >&2 ;} 102 | function err { local x=$? ; msg "$*" ; return $(( $x == 0 ? 1 : $x )) ;} 103 | function out { printf '%s\n' "$*" ;} 104 | 105 | if [[ ${1:-} ]] && declare -F | cut -d' ' -f3 | fgrep -qx -- "${1:-}" 106 | then "$@" 107 | else main "$@" 108 | fi 109 | -------------------------------------------------------------------------------- /tutorial.md: -------------------------------------------------------------------------------- 1 | 2 | Docker containers provide a consistent, compact and flexible means of 3 | packaging application builds. Delivering applications with Docker on Mesos 4 | promises a truly elastic, efficient and consistent platform for delivering a 5 | range of applications on premises or in the cloud. 6 | 7 | ## Service Launch and Discovery 8 | 9 | Let's launch a service in a Docker container and use Marathon's discovery 10 | features to find and connect to it from another Docker container. In this 11 | example, we'll launch Redis within a Docker container and then connect to it 12 | using the Redis CLI. 13 | 14 | Launching Docker containers with Marathon is a matter of assigning the 15 | "executor" parameter to point to a Docker executor and making the container 16 | name the first word of the command. 17 | 18 | :; http POST http://localhost:8080/v1/apps/start \ 19 | id=multidis instances=2 mem=512 cpus=1 \ 20 | executor=/var/lib/mesos/executors/docker \ 21 | cmd='johncosta/redis' 22 | 23 | >> 24 | 25 | HTTP/1.1 204 No Content 26 | Content-Type: application/json 27 | Server: Jetty(8.y.z-SNAPSHOT) 28 | 29 | Once we have two Redis instances running, we can query Marathon to find them 30 | and connect to them. 31 | 32 | :; http GET http://localhost:8080/v1/endpoints 33 | 34 | >> 35 | 36 | HTTP/1.1 200 OK 37 | Content-Type: text/plain 38 | Server: Jetty(8.y.z-SNAPSHOT) 39 | Transfer-Encoding: chunked 40 | 41 | multidis 10445 ip-10-184-7-73.ec2.internal:31001 ip-10-184-7-73.ec2.internal:31000 42 | 43 | Using one of the `ip:port` pairs above, we can connect to one of our Redis 44 | instances, using the same Docker image, to get access to a working Redis CLI: 45 | 46 | :; sudo docker run -i -t johncosta/redis \ 47 | redis-cli -h ip-10-184-7-73.ec2.internal -p 31001 48 | 49 | redis ip-10-184-7-73.ec2.internal:31001> SET x 7 50 | OK 51 | redis ip-10-184-7-73.ec2.internal:31001> GET x 52 | "7" 53 | 54 | There's no need to install Redis or its supporting libraries on your Mesos 55 | hosts. 56 | 57 | ## Reproducing This Example 58 | 59 | Installation of the many components used here is straightforward. Mesos and 60 | the Python bindings for Mesos are available as packages: 61 | 62 | * [Ubuntu 13.04 Package](http://downloads.mesosphere.io/master/ubuntu/13.04/mesos_0.14.0_amd64.deb) 63 | * [Python Mesos Bindings](http://downloads.mesosphere.io/master/ubuntu/13.04/mesos-0.14.0-py2.7-linux-x86_64.egg) 64 | 65 | Marathon, a Scala program, can be run from a standalone Jar. We've made an 66 | Upstart script for it, too. 67 | 68 | * [Marathon Executable Jar](http://downloads.mesosphere.io/marathon/marathon-0.0.6-SNAPSHOT-jar-with-dependencies.jar) 69 | * [Marathon Upstart Script](http://downloads.mesosphere.io/marathon/marathon.conf) 70 | 71 | With Mesos and Marathon, you have everything you need to start running 72 | reliable services across your cluster. 73 | 74 | The Docker project provides [Ubuntu 13.04 installation instructions](http://docs.docker.io/en/latest/installation/ubuntulinux/#ubuntu-raring). 75 | 76 | To follow along with the examples above, install the Docker executor as 77 | `/var/lib/mesos/executors/docker`. 78 | 79 | [Docker Executor for Mesos](https://github.com/mesosphere/mesos-docker/blob/master/bin/mesos-docker) 80 | 81 | For a standalone installation, we've put together a script that installs all 82 | the needed components and configures them on Ubuntu 13.04. Just pipe it to a 83 | shell running under `sudo` and you're all set: 84 | 85 | curl -fL https://raw.github.com/mesosphere/mesos-docker/master/bin/mesos-docker-setup | sudo bash 86 | 87 | -------------------------------------------------------------------------------- /tutorial/mesos-docker.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/d2iq-archive/mesos-docker/f2e7326a5647ccf87f65d9134af110c36f073208/tutorial/mesos-docker.1.png -------------------------------------------------------------------------------- /tutorial/mesos-docker.2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/d2iq-archive/mesos-docker/f2e7326a5647ccf87f65d9134af110c36f073208/tutorial/mesos-docker.2.png --------------------------------------------------------------------------------