├── machine └── Dockerfile ├── .dockerignore ├── .gitignore ├── hn ├── .gitignore ├── Dockerfile.run ├── Dockerfile.build ├── Makefile └── main.go ├── riemann-dash ├── config.rb └── Dockerfile ├── curl └── Dockerfile ├── pinger └── docker-compose.yml ├── tailf ├── entrypoint.sh └── Dockerfile ├── ansible ├── conf │ └── ansible.cfg ├── docker-compose.yml ├── Dockerfile ├── playbooks │ ├── trick-out.yml │ ├── secure.yml │ ├── swapfile.yml │ └── bootstrap.yml ├── entrypoint.sh └── machine.py ├── riemann-health ├── entrypoint.sh └── Dockerfile ├── ab └── Dockerfile ├── folding-at-home ├── docker-compose.yml └── Dockerfile ├── rethinkdb └── docker-compose.yml ├── Makefile ├── riemann-server ├── Dockerfile ├── docker-compose.yml └── riemann.config ├── keras └── Dockerfile └── caffe └── Dockerfile /machine/Dockerfile: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | .git 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | -------------------------------------------------------------------------------- /hn/.gitignore: -------------------------------------------------------------------------------- 1 | hn 2 | -------------------------------------------------------------------------------- /riemann-dash/config.rb: -------------------------------------------------------------------------------- 1 | set :bind, "0.0.0.0" 2 | -------------------------------------------------------------------------------- /curl/Dockerfile: -------------------------------------------------------------------------------- 1 | from alpine 2 | 3 | run apk add --update curl 4 | -------------------------------------------------------------------------------- /pinger/docker-compose.yml: -------------------------------------------------------------------------------- 1 | pinger: 2 | image: busybox 3 | command: ping google.com 4 | -------------------------------------------------------------------------------- /tailf/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | while ! journalctl -f -u "$1"; do sleep 1; done 4 | -------------------------------------------------------------------------------- /ansible/conf/ansible.cfg: -------------------------------------------------------------------------------- 1 | [defaults] 2 | host_key_checking = False 3 | transport = paramiko 4 | -------------------------------------------------------------------------------- /tailf/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | add /entrypoint.sh /entrypoint.sh 4 | entrypoint ["/entrypoint.sh"] 5 | -------------------------------------------------------------------------------- /riemann-health/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | riemann-health --host ${RIEMANN_HEALTH_SERVER_HOSTNAME} --event-host $(cat /etc/hostname) 4 | -------------------------------------------------------------------------------- /ab/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | run apt-get update && \ 4 | apt-get install -y --no-install-recommends apache2-utils && \ 5 | rm -rf /var/lib/apt/lists/* 6 | 7 | entrypoint ["ab"] 8 | -------------------------------------------------------------------------------- /folding-at-home/docker-compose.yml: -------------------------------------------------------------------------------- 1 | fahclient: 2 | image: nathanleclaire/folding-at-home 3 | cpu_shares: 50 4 | environment: 5 | - "FAH_PASSKEY" 6 | - "affinity:com.docker.compose.service!=fahclient" 7 | -------------------------------------------------------------------------------- /hn/Dockerfile.run: -------------------------------------------------------------------------------- 1 | FROM alpine 2 | MAINTAINER Nathan LeClaire 3 | 4 | COPY /hn /usr/local/bin/hn 5 | RUN chmod +x /usr/local/bin/hn 6 | RUN adduser -D hn 7 | USER hn 8 | 9 | ENTRYPOINT ["hn"] 10 | -------------------------------------------------------------------------------- /riemann-dash/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian 2 | 3 | run apt-get update && apt-get install -y ruby && \ 4 | gem install --no-ri --no-rdoc riemann-dash 5 | env RIEMANN_DASH_CONFIG /config.rb 6 | expose 4567/tcp 7 | copy ./config.rb /config.rb 8 | cmd ["riemann-dash"] 9 | -------------------------------------------------------------------------------- /hn/Dockerfile.build: -------------------------------------------------------------------------------- 1 | FROM golang:1.7-alpine 2 | MAINTAINER Nathan LeClaire 3 | 4 | RUN mkdir -p /go/src/github.com/nathanleclaire/hn 5 | COPY . /go/src/github.com/nathanleclaire/hn 6 | RUN go install github.com/nathanleclaire/hn 7 | 8 | CMD ["cat", "/go/bin/hn"] 9 | -------------------------------------------------------------------------------- /rethinkdb/docker-compose.yml: -------------------------------------------------------------------------------- 1 | leader: 2 | container_name: rethinkleader 3 | image: rethinkdb 4 | mem_limit: "450m" 5 | ports: 6 | - "8080:8080" 7 | restart: always 8 | follower: 9 | image: rethinkdb 10 | mem_limit: "450m" 11 | command: rethinkdb --join rethinkleader 12 | restart: always 13 | -------------------------------------------------------------------------------- /riemann-health/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian 2 | 3 | run apt-get update && apt-get install -y ruby ruby-dev build-essential zlib1g-dev && \ 4 | gem install --no-ri --no-rdoc riemann-tools 5 | 6 | env RIEMANN_HEALTH_SERVER_HOSTNAME riemann-server 7 | copy ./entrypoint.sh /entrypoint.sh 8 | entrypoint ["/entrypoint.sh"] 9 | -------------------------------------------------------------------------------- /ansible/docker-compose.yml: -------------------------------------------------------------------------------- 1 | provision: 2 | image: nathanleclaire/ansibleprovision 3 | net: host 4 | volumes: 5 | - /root/.ssh:/hostssh 6 | environment: 7 | - affinity:container!=*provision* 8 | consul: 9 | image: progrium/consul 10 | hostname: consul 11 | command: -server -bootstrap 12 | ports: 13 | - "8500:8500" 14 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | USER := nathanleclaire 2 | TAG := latest 3 | IMAGE := pinger 4 | 5 | FULLNAME := $(USER)/$(IMAGE):$(TAG) 6 | 7 | default: build 8 | 9 | clean: 10 | docker rmi $(FULLNAME) 11 | 12 | build: 13 | docker build -t $(FULLNAME) $(IMAGE) 14 | 15 | tag: 16 | docker tag -f $(USER)/$(IMAGE) $(FULLNAME) 17 | 18 | push: 19 | docker push $(FULLNAME) 20 | -------------------------------------------------------------------------------- /ansible/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | run apt-get update && \ 4 | apt-get install -y ansible ssh && \ 5 | rm -rf /var/lib/apt/lists/* 6 | add ./machine.py /machine.py 7 | add ./playbooks /playbooks 8 | add ./conf/ansible.cfg /etc/ansible/ansible.cfg 9 | add ./entrypoint.sh /entrypoint.sh 10 | entrypoint ["/entrypoint.sh"] 11 | cmd ["/playbooks/bootstrap.yml"] 12 | -------------------------------------------------------------------------------- /folding-at-home/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian 2 | 3 | add https://fah.stanford.edu/file-releases/public/release/fahclient/debian-testing-64bit/v7.4/fahclient_7.4.4_amd64.deb /fah.deb 4 | run dpkg -i --force-depends /fah.deb 5 | 6 | env FAH_USER Nathan_LeClaire 7 | env FAH_TEAM DockerSwarm 8 | env FAH_CPU_USAGE 30 9 | cmd FAHClient --user="$FAH_USER" --team="$FAH_TEAM" --passkey="$FAH_PASSKEY" --cpu-usage="$FAH_CPU_USAGE" 10 | -------------------------------------------------------------------------------- /hn/Makefile: -------------------------------------------------------------------------------- 1 | USER := nathanleclaire 2 | COMPILE_IMAGE := "$(USER)/hnbuild" 3 | TAG := "latest" 4 | RUN_IMAGE := "$(USER)/hn:$(TAG)" 5 | 6 | default: compile 7 | 8 | push: 9 | docker push $(RUN_IMAGE) 10 | 11 | compile: compilerbuild 12 | docker build -t $(RUN_IMAGE) -f Dockerfile.run . 13 | 14 | compilerbuild: 15 | docker build -t $(COMPILE_IMAGE) -f Dockerfile.build . 16 | docker run $(COMPILE_IMAGE) >hn 17 | -------------------------------------------------------------------------------- /riemann-server/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | env RIEMANN_VERSION 0.2.10 4 | 5 | run apt-get update && apt-get install -y default-jre 6 | add https://aphyr.com/riemann/riemann_${RIEMANN_VERSION}_all.deb /riemann_${RIEMANN_VERSION}_all.deb 7 | run dpkg -i riemann_${RIEMANN_VERSION}_all.deb 8 | expose 5556/tcp 5555/udp 9 | copy ./riemann.config /etc/riemann/riemann.config 10 | cmd ["riemann", "/etc/riemann/riemann.config"] 11 | -------------------------------------------------------------------------------- /keras/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | run apt-get update && \ 4 | apt-get install -y python python-pip python-numpy python-scipy python-dev python-h5py python-pip python-nose g++ libopenblas-dev git gfortran libhdf5-dev && \ 5 | rm -rf /var/lib/apt/lists/* 6 | run pip install Theano 7 | run pip install Cython 8 | run git clone https://github.com/fchollet/keras keras 9 | workdir /keras 10 | run python setup.py install 11 | -------------------------------------------------------------------------------- /ansible/playbooks/trick-out.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Trick out Ubuntu server 3 | hosts: all 4 | gather_facts: False 5 | 6 | tasks: 7 | - name: Install htop 8 | apt: > 9 | package=htop 10 | state=present 11 | update_cache=yes 12 | - name: Install tree 13 | apt: > 14 | package=tree 15 | state=present 16 | - name: Install jq 17 | apt: > 18 | package=jq 19 | state=present 20 | -------------------------------------------------------------------------------- /hn/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "log" 5 | "net/http" 6 | "os" 7 | ) 8 | 9 | func main() { 10 | http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { 11 | h, err := os.Hostname() 12 | if err != nil { 13 | http.Error(w, err.Error(), http.StatusInternalServerError) 14 | return 15 | } 16 | 17 | if _, err := w.Write([]byte(`⚡ The container serving this request is ` + h + ` ⚡`)); err != nil { 18 | http.Error(w, err.Error(), http.StatusInternalServerError) 19 | return 20 | } 21 | }) 22 | log.Println("Listening on :8080") 23 | log.Fatal(http.ListenAndServe(":8080", nil)) 24 | } 25 | -------------------------------------------------------------------------------- /riemann-server/docker-compose.yml: -------------------------------------------------------------------------------- 1 | riemannserver: 2 | container_name: riemann-server 3 | image: "nathanleclaire/riemann-server:article" 4 | mem_limit: "1500m" 5 | ports: 6 | - "127.0.0.1:5556:5556" 7 | restart: always 8 | riemanndash: 9 | image: "nathanleclaire/riemann-dash:article" 10 | mem_limit: "500m" 11 | ports: 12 | - "127.0.0.1:4567:4567" 13 | restart: always 14 | riemannhealth: 15 | image: "nathanleclaire/riemann-health:article" 16 | pid: host 17 | environment: 18 | - "affinity:container!=*riemannhealth*" 19 | volumes: 20 | - "/etc/hostname:/etc/hostname:ro" 21 | restart: always 22 | -------------------------------------------------------------------------------- /ansible/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [[ ! -d /hostssh ]]; then 4 | echo "Must mount the host SSH directory at /hostssh, e.g. 'docker run --net host -v /root/.ssh:/hostssh nathanleclaire/ansible" 5 | exit 1 6 | fi 7 | 8 | # Generate temporary SSH key to allow access to the host machine. 9 | mkdir -p /root/.ssh 10 | ssh-keygen -f /root/.ssh/id_rsa -P "" 11 | 12 | cp /hostssh/authorized_keys /hostssh/authorized_keys.bak 13 | cat /root/.ssh/id_rsa.pub >>/hostssh/authorized_keys 14 | 15 | ansible all -i "localhost," -m raw -a "apt-get install -y python-minimal" 16 | ansible-playbook -i "localhost," "$@" 17 | 18 | mv /hostssh/authorized_keys.bak /hostssh/authorized_keys 19 | -------------------------------------------------------------------------------- /riemann-server/riemann.config: -------------------------------------------------------------------------------- 1 | ; -*- mode: clojure; -*- 2 | ; vim: filetype=clojure 3 | 4 | (logging/init {:file "/var/log/riemann/riemann.log"}) 5 | 6 | ; Listen on the local interface over TCP (5555), UDP (5555), and websockets 7 | ; (5556) 8 | (let [host "0.0.0.0"] 9 | (tcp-server {:host host}) 10 | (udp-server {:host host}) 11 | (ws-server {:host host})) 12 | 13 | ; Expire old events from the index every 5 seconds. 14 | (periodically-expire 5) 15 | 16 | (let [index (index)] 17 | ; Inbound events will be passed to these streams: 18 | (streams 19 | (default :ttl 60 20 | ; Index all events immediately. 21 | index 22 | 23 | ; Log expired events. 24 | (expired 25 | (fn [event] (info "expired" event)))))) 26 | -------------------------------------------------------------------------------- /ansible/playbooks/secure.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Secure Ubuntu server 3 | hosts: all 4 | gather_facts: False 5 | 6 | tasks: 7 | - name: Reset UFW firewall 8 | ufw: 9 | state=reset 10 | - name: Allow SSH access on instance 11 | ufw: > 12 | rule=allow 13 | name=OpenSSH 14 | - name: Open Docker daemon, HTTP(S), and Swarm ports 15 | ufw: > 16 | rule=allow 17 | port={{ item }} 18 | proto=tcp 19 | with_items: 20 | - 80 # Default HTTP port 21 | - 443 # Default HTTPS port 22 | - 2376 # Docker daemon API port 23 | - 3376 # Swarm API port 24 | - name: Set to deny incoming requests by default 25 | ufw: > 26 | default=deny 27 | - name: Turn on UFW 28 | ufw: > 29 | state=enabled 30 | - name: Install fail2ban 31 | apt: name=fail2ban state=present update_cache=yes 32 | -------------------------------------------------------------------------------- /caffe/Dockerfile: -------------------------------------------------------------------------------- 1 | from debian:jessie 2 | 3 | run apt-get update && \ 4 | apt-get install -y libprotobuf-dev \ 5 | libleveldb-dev \ 6 | libsnappy-dev \ 7 | libopencv-dev \ 8 | libhdf5-serial-dev \ 9 | protobuf-compiler \ 10 | libatlas-base-dev \ 11 | libgflags-dev \ 12 | libgoogle-glog-dev \ 13 | liblmdb-dev \ 14 | build-essential \ 15 | python \ 16 | git \ 17 | python-pip && \ 18 | apt-get install -y --no-install-recommends libboost-all-dev && \ 19 | rm -rf /var/lib/apt/lists/* 20 | run git clone https://github.com/BVLC/caffe 21 | workdir /caffe 22 | add Makefile.config /caffe/Makefile.config 23 | 24 | # Debian lib files have a slightly different name than the Ubuntu version expected. 25 | run ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.8.0.2 /usr/lib/x86_64-linux-gnu/libhdf5.so && \ 26 | ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.8.0.2 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so 27 | run apt-get install -y gcc-4.6 g++-4.6 gcc-4.6-multilib g++-4.6-multilib 28 | 29 | run make all -j$(cat /proc/cpuinfo | awk '/^processor/{print $3}' | tail -1) 30 | -------------------------------------------------------------------------------- /ansible/playbooks/swapfile.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Set up swap file for droplet 3 | hosts: all 4 | vars: 5 | - swap_file_path: /root/swapfile 6 | - swap_file_size: 1GB 7 | gather_facts: False 8 | 9 | tasks: 10 | - name: Create swap file 11 | command: dd if=/dev/zero of={{ swap_file_path }} bs=1024 count={{ swap_file_size }} 12 | creates="{{ swap_file_path }}" 13 | tags: 14 | - swap.file.create 15 | - name: Change swap file permissions 16 | file: path="{{ swap_file_path }}" 17 | owner=root 18 | group=root 19 | mode=0600 20 | tags: 21 | - swap.file.permissions 22 | - name: Check swap file type 23 | command: file {{ swap_file_path }} 24 | register: swapfile 25 | tags: 26 | - swap.file.mkswap 27 | - name: Make swap file 28 | command: "sudo mkswap {{ swap_file_path }}" 29 | when: swapfile.stdout.find('swap file') == -1 30 | tags: 31 | - swap.file.mkswap 32 | - name: Write swap entry in fstab 33 | mount: name=none 34 | src={{ swap_file_path }} 35 | fstype=swap 36 | opts=sw 37 | passno=0 38 | dump=0 39 | state=present 40 | tags: 41 | - swap.fstab 42 | - name: Mount swap 43 | command: "swapon {{ swap_file_path }}" 44 | tags: 45 | - swap.file.swampon 46 | -------------------------------------------------------------------------------- /ansible/machine.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | """ 4 | Example Usage: 5 | $ ansible -i machine.py machinename -m ping 6 | """ 7 | 8 | 9 | import argparse 10 | import subprocess 11 | 12 | try: 13 | import json 14 | except ImportError: 15 | import simplejson as json 16 | 17 | def dm(*args): 18 | return subprocess.check_output(["docker-machine"] + list(args)).strip() 19 | 20 | def dminspect(fmt, mcn): 21 | return dm("inspect", "-f", fmt, mcn) 22 | 23 | def get_host_and_vars(m): 24 | hosts = [dminspect("{{.Driver.IPAddress}}", m)] 25 | ssh_vars = { 26 | "ansible_ssh_user": dminspect("{{.Driver.SSHUser}}", m), 27 | "ansible_ssh_port": dminspect("{{.Driver.SSHPort}}", m), 28 | "ansible_ssh_private_key_file": dminspect("{{.Driver.SSHKeyPath}}", m) 29 | } 30 | data = {"hosts": hosts, "vars": ssh_vars} 31 | return data 32 | 33 | class DockerMachineInventory(object): 34 | def __init__(self): 35 | self.inventory = {} # Ansible Inventory 36 | 37 | parser = argparse.ArgumentParser(description='Produce an Ansible Inventory file based on Docker Machine status') 38 | parser.add_argument('--list', action='store_true', help='List all active Droplets as Ansible inventory (default: True)') 39 | self.args = parser.parse_args() 40 | 41 | machines = dm("ls", "-q").splitlines() 42 | json_data = {m: get_host_and_vars(m) for m in machines} 43 | 44 | print json.dumps(json_data) 45 | 46 | DockerMachineInventory() 47 | -------------------------------------------------------------------------------- /ansible/playbooks/bootstrap.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Trick out Debian server 3 | hosts: all 4 | gather_facts: False 5 | 6 | tasks: 7 | - name: Install desired packages 8 | become: true 9 | apt: > 10 | package={{ item }} 11 | state=present 12 | update_cache=yes 13 | with_items: 14 | - htop 15 | - tree 16 | - jq 17 | - fail2ban 18 | - vim 19 | - mosh 20 | - ufw 21 | 22 | - name: Get simple .vimrc 23 | get_url: url=https://raw.githubusercontent.com/amix/vimrc/master/vimrcs/basic.vim dest=/home/admin/.vimrc 24 | 25 | - name: Reset UFW firewall 26 | become: true 27 | ufw: 28 | state=reset 29 | 30 | - name: Allow SSH access on instance 31 | become: true 32 | ufw: > 33 | rule=allow 34 | name=OpenSSH 35 | 36 | - name: Open Docker daemon, HTTP(S), and Swarm ports 37 | become: true 38 | ufw: > 39 | rule=allow 40 | port={{ item }} 41 | proto=tcp 42 | with_items: 43 | - 80 # Default HTTP port 44 | - 443 # Default HTTPS port 45 | - 2375 # Docker daemon API port (non-TLS) 46 | - 2376 # Docker daemon API port (TLS) 47 | - 3376 # Swarm API port 48 | - 7946 # Serf port (libnetwork) 49 | 50 | - name: Open VXLAN and Serf UDP ports 51 | become: true 52 | ufw: > 53 | rule=allow 54 | port={{ item }} 55 | proto=udp 56 | with_items: 57 | - 7946 # Serf 58 | - 4789 # VXLAN 59 | 60 | - name: Set to deny incoming requests by default 61 | become: true 62 | ufw: > 63 | default=deny 64 | 65 | - name: Turn on UFW 66 | become: true 67 | ufw: > 68 | state=enabled 69 | 70 | - name: Set memory limit in GRUB 71 | become: true 72 | lineinfile: > 73 | dest=/etc/default/grub 74 | regexp=^GRUB_CMDLINE_LINUX_DEFAULT 75 | line='GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"' 76 | 77 | - name: Load new GRUB config 78 | become: true 79 | command: update-grub 80 | 81 | - name: Set hostname 82 | become: true 83 | hostname: name={{ inventory_hostname }} 84 | 85 | - name: Set properly in /etc/hosts 86 | become: true 87 | lineinfile: dest=/etc/hosts regexp='^127\.0\.0\.1' line='127.0.0.1 {{ inventory_hostname }}' owner=root group=root mode=0644 88 | 89 | --------------------------------------------------------------------------------