├── requirements.txt ├── docker-compose.yml ├── src ├── notifier.py └── event_checker.py ├── Dockerfile ├── LICENSE └── README.md /requirements.txt: -------------------------------------------------------------------------------- 1 | requests>=2.20.0 2 | docker==2.5.1 3 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2' 2 | services: 3 | eventer: 4 | build: . 5 | volumes: 6 | - /var/run/docker.sock:/var/run/docker.sock 7 | 8 | -------------------------------------------------------------------------------- /src/notifier.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import requests 3 | 4 | 5 | def notify(message, subject): 6 | mailer(subject, message) 7 | 8 | 9 | def mailer(subject, message): 10 | return requests.post("https://api.mailgun.net/*******", 11 | auth=("api", "*******"), 12 | data={"from": "noreply@mexample.com", 13 | "to": "******", 14 | "subject": subject, 15 | "text": message}) 16 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | 3 | ENV PYTHONUNBUFFERED 1 4 | ENV DEBIAN_FRONTEND noninteractive 5 | 6 | RUN apt-get update 7 | RUN apt-get install apt-transport-https ca-certificates curl software-properties-common -y 8 | RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - 9 | RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" 10 | RUN apt-get update 11 | RUN apt-get install docker-ce -y 12 | RUN apt-get install python3 -y 13 | RUN apt-get install python3-pip -y 14 | RUN pip3 install --upgrade pip 15 | ADD requirements.txt / 16 | RUN pip3 install -r requirements.txt 17 | 18 | WORKDIR /app 19 | 20 | ADD src /app 21 | 22 | CMD ["python3", "event_checker.py"] -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Gerard Keating 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/event_checker.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import time 3 | from collections import Counter 4 | from pprint import pformat 5 | import datetime 6 | 7 | import docker 8 | 9 | import notifier 10 | 11 | MAX_NOTIFY_TIME = 60 12 | 13 | 14 | def notify(events): 15 | message = "" 16 | message += "\n%d events\n" % len(events) 17 | message += pformat(Counter(row['Action'] for row in events)) 18 | message += "\n" 19 | message += pformat(events) 20 | subject = "[DOCKER_EVNETS] %d docker events\n" % len(events) 21 | print("Sending e-mail %s", subject) 22 | notifier.notify(message, subject) 23 | 24 | 25 | def run(): 26 | client = docker.from_env() 27 | while True: 28 | for event in client.events(decode=True): 29 | notify([event]) 30 | break 31 | since = datetime.datetime.now() 32 | # Do not want to bombard the e-mailer with messages so sleeping 33 | print("Sleeping for %d seconds" % MAX_NOTIFY_TIME) 34 | time.sleep(MAX_NOTIFY_TIME) 35 | print("Woke") 36 | until = datetime.datetime.now() 37 | notify([event for event in client.events(since=since, until=until, decode=True)]) 38 | 39 | 40 | def main(): 41 | run() 42 | 43 | 44 | if __name__=="__main__": 45 | main() 46 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Docker Event 2 | 3 | ## The Problem 4 | 5 | I had a problem recently. I had a couple Docker containers on one machine 6 | for a personal project which was a django web app called  7 | [issueinfinity.com](https://www.issueinfinity.com). 8 | One of the containers, that was doing some image manipulation, kept 9 | crashing and I did not know until I logged into the machine and looked at it 10 | using `docker ps`. So I wanted an e-mail notification if any of my 11 | containers stopped or if anything happened to them. 12 | 13 | ## The Command 14 | I discovered a command `docker events`. This is a very handy command to see what has 15 | happened to your docker containers. 16 | 17 | To see what has happened to your containers in the last hour you can run: 18 | 19 | docker events --format {{json .}} --since "1h" --until "0s" 20 | 21 | The `--since 1h` simply means get everything since an hour ago. 22 | 23 | The `--until 0s` means until 0 seconds ago, without the `--until` the command 24 | will just continually stream docker events. 25 | 26 | The `--format "{{json .}}"` argument outputs the events as 27 | [JSON lines](http://jsonlines.org) which basically means a JSON document for 28 | every event. 29 | 30 | As well as the above you can add filters to the events like 31 | 32 | docker events --filter 'container=test' --filter 'event=stop' 33 | 34 | This will output any stop event on the test container. For more about the 35 | filters and more details on the other the arguments see the 36 | [official docker docs](https://docs.docker.com/engine/reference/commandline/events) 37 | 38 | ## The Python Script 39 | So now we have enough to create a Python script: 40 | ```python 41 | import sys 42 | import subprocess 43 | import time 44 | import json 45 | from collections import Counter 46 | from pprint import pprint, pformat 47 | 48 | from notifier import notify 49 | 50 | 51 | def run(): 52 | proc = subprocess.Popen('docker events --format "{{json .}}" --since "1h" --until "0s"', 53 | stdout=subprocess.PIPE, 54 | stderr=subprocess.STDOUT, shell=True) 55 | 56 | retcode = proc.poll() 57 | wait_time = 0.1 58 | while retcode is None: 59 | time.sleep(wait_time) 60 | retcode = proc.poll() 61 | res = proc.stdout.read() 62 | res = res.decode("utf-8") 63 | event_data = [] 64 | for line in res.split("\n"): 65 | line = line.strip() 66 | if line: 67 | event_data.append(json.loads(line)) 68 | pprint(event_data[-1]) 69 | print("-"*5) 70 | if event_data: 71 | message = "" 72 | message += "\n%d events\n" % len(event_data) 73 | message += pformat(Counter(row['Action'] for row in event_data)) 74 | message += "\n" 75 | message += pformat(event_data) 76 | subject = "[DOCKER_EVENTS] %d docker events\n" % len(event_data) 77 | notify(message, subject) 78 | print("e-mail sent") 79 | 80 | 81 | def main(_): 82 | while True: 83 | run() 84 | print("Now sleeping for an hour") 85 | time.sleep(60*60) 86 | 87 | 88 | 89 | if __name__=="__main__": 90 | main(sys.argv[1:]) 91 | ``` 92 | 93 | Here I am simply running the docker event in a subprocess. 94 | I am the putting the stderr of the process into stdout and putting 95 | stdout into a temporary file. The temporary file avoids issues with 96 | buffer overflows. Then the script sends an e-mail(not shown) if there is 97 | any events 98 | and parses the json to create a summary of the work. 99 | 100 | Although this works the subprocess adds overhead so to simplify the 101 | solution we can just use the docker python api. 102 | 103 | ## Using the Docker Python API 104 | Simply run `pip install docker` 105 | to install the python docker api. 106 | 107 | Now the script becomes simpler: 108 | 109 | ```python 110 | import sys 111 | import time 112 | from collections import Counter 113 | from pprint import pprint, pformat 114 | import datetime 115 | 116 | from notifier import notify 117 | import docker 118 | 119 | 120 | def run(): 121 | client = docker.from_env() 122 | until = datetime.datetime.utcnow() 123 | since = until - datetime.timedelta(hours=1) 124 | event_data = [] 125 | for event in client.events(since=since, until=until, decode=True): 126 | event_data.append(event) 127 | pprint(event_data[-1]) 128 | print("-" * 5) 129 | if event_data: 130 | message = "" 131 | message += "\n%d events\n" % len(event_data) 132 | message += pformat(Counter(row['Action'] for row in event_data)) 133 | message += "\n" 134 | message += pformat(event_data) 135 | subject = "%d docker events\n" % len(event_data) 136 | notify(message, subject) 137 | print("e-mail sent") 138 | 139 | 140 | def main(_): 141 | while True: 142 | run() 143 | print("Now sleeping for an hour") 144 | time.sleep(60*60) 145 | 146 | 147 | if __name__=="__main__": 148 | main(sys.argv[1:]) 149 | ``` 150 | 151 | The docker client is created from the environment with `docker.from_env`, 152 | which means it works like the command line would. 153 | The client.events function works very similar to the command line version. 154 | The `decode=True` means the output are python dictionaries. 155 | ## I Put Docker in your Docker 156 | Now I could put this python script into supervisord and run on the docker 157 | machine but if we are using docker why not use docker. 158 | To get this working in a docker container we need to install the docker 159 | client inside the container. The simplest way to do this is to run 160 | `apt-get install docker.io` in the Dockerfile. 161 | 162 | Then to make it work we need to mount the docker socket into the container 163 | with `/var/run/docker.sock:/var/run/docker.sock`. 164 | 165 | You can see this in the docker-compose.yml file. 166 | 167 | And voila a docker event notifier. 168 | 169 | ## Update, now streaming 170 | A comment on [reddit](https://www.reddit.com/r/Python/comments/7bx7qe/an_explanation_and_code_of_a_python_script_to/dpm39e5/) said: 171 | > Though preferably it would be rather when an event happens than just polling for it. 172 | 173 | And I agree but I still did not want to be bombarded with e-mails so here's what I came up with 174 | ```python 175 | #!/usr/bin/env python3 176 | import time 177 | from collections import Counter 178 | from pprint import pformat 179 | import datetime 180 | 181 | import docker 182 | 183 | import notifier 184 | 185 | MAX_NOTIFY_TIME = 60 186 | 187 | 188 | def notify(events): 189 | message = "" 190 | message += "\n%d events\n" % len(events) 191 | message += pformat(Counter(row['Action'] for row in events)) 192 | message += "\n" 193 | message += pformat(events) 194 | subject = "[DOCKER_EVNETS] %d docker events\n" % len(events) 195 | print("Sending e-mail %s", subject) 196 | notifier.notify(message, subject) 197 | 198 | 199 | def run(): 200 | client = docker.from_env() 201 | while True: 202 | for event in client.events(decode=True): 203 | notify([event]) 204 | break 205 | since = datetime.datetime.now() 206 | # Do not want to bombard the e-mailer with messages so sleeping 207 | print("Sleeping for %d seconds" % MAX_NOTIFY_TIME) 208 | time.sleep(MAX_NOTIFY_TIME) 209 | print("Woke") 210 | until = datetime.datetime.now() 211 | notify([event for event in client.events(since=since, until=until, decode=True)]) 212 | 213 | 214 | def main(): 215 | run() 216 | 217 | 218 | if __name__=="__main__": 219 | main() 220 | 221 | ``` 222 | It is a simple solution, it gets the first event, e-mails about it and then sleeps. After the program stops sleeping it gets all the events that happened while it was sleeping and sends them and repeats. 223 | 224 | ## Still to do: 225 | - [ ] make it more configurable 226 | - [x] make it send notification on the first event and not just once an hour 227 | 228 | 229 | --------------------------------------------------------------------------------