├── doc └── img │ ├── consul.png │ ├── jaeger.png │ ├── logspout-cli.png │ ├── traefik-dash.png │ ├── grafana-docker.png │ ├── traefik-routes.png │ └── grafana-telegraf.png ├── Makefile.COPYME ├── targets.mk ├── LICENSE ├── conf └── telegraf │ └── telegraf.conf ├── docker-compose.yml └── README.md /doc/img/consul.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/consul.png -------------------------------------------------------------------------------- /doc/img/jaeger.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/jaeger.png -------------------------------------------------------------------------------- /doc/img/logspout-cli.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/logspout-cli.png -------------------------------------------------------------------------------- /doc/img/traefik-dash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/traefik-dash.png -------------------------------------------------------------------------------- /doc/img/grafana-docker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/grafana-docker.png -------------------------------------------------------------------------------- /doc/img/traefik-routes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/traefik-routes.png -------------------------------------------------------------------------------- /doc/img/grafana-telegraf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCricketeer/gryllidae/HEAD/doc/img/grafana-telegraf.png -------------------------------------------------------------------------------- /Makefile.COPYME: -------------------------------------------------------------------------------- 1 | install: 2 | @echo "installing!" 3 | 4 | clean: 5 | @echo "cleaning!" 6 | 7 | include .instrument/targets.mk 8 | -------------------------------------------------------------------------------- /targets.mk: -------------------------------------------------------------------------------- 1 | INSTRU_DOCKER_FLAGS=--rm -ti --volume '$(HOME)/.ssh':/.ssh \ 2 | --volume /var/run/docker.sock:/var/run/docker.sock \ 3 | --volume $(HOST_OS_BASEDIR):/workdir 4 | 5 | GITREMOTE=$(shell git config --get remote.origin.url ) 6 | GITREPO=$(shell echo $(GITREMOTE) | awk -F '/' '{print $$NF}' | cut -d. -f1 ) 7 | GITORG=$(shell echo $(GITREMOTE) | awk -F ':' '{print $$NF}' | cut -d/ -f1 ) 8 | 9 | DOCKER_REGISTRY ?= '' 10 | 11 | .PHONY: ult-instrument 12 | ult-instrument: clean install ult-instrument-infra 13 | REGISTRY=$(DOCKER_REGISTRY) docker-compose -f "$(CURDIR)/docker-compose.yml" up 14 | 15 | .PHONY: ult-instrument-infra 16 | ult-instrument-infra: 17 | @REGISTRY=$(DOCKER_REGISTRY) docker-compose -f "$(CURDIR)/.instrument/docker-compose.yml" up -d 18 | 19 | .PHONY: ult-instrument-infra-clean 20 | ult-instrument-infra-clean: 21 | @REGISTRY=$(DOCKER_REGISTRY) docker-compose -f "$(CURDIR)/.instrument/docker-compose.yml" rm -sfv 22 | 23 | .PHONY: ult-instrument-clean 24 | ult-instrument-clean: ult-instrument-infra-clean 25 | @REGISTRY=$(DOCKER_REGISTRY) docker-compose -f "$(CURDIR)/docker-compose.yml" rm -sfv 26 | 27 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The Clear BSD License 2 | 3 | Copyright (c) 2020 Jordan M. Moore 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted (subject to the limitations in the disclaimer 8 | below) provided that the following conditions are met: 9 | 10 | * Redistributions of source code must retain the above copyright notice, 11 | this list of conditions and the following disclaimer. 12 | 13 | * Redistributions in binary form must reproduce the above copyright 14 | notice, this list of conditions and the following disclaimer in the 15 | documentation and/or other materials provided with the distribution. 16 | 17 | * Neither the name of the copyright holder nor the names of its 18 | contributors may be used to endorse or promote products derived from this 19 | software without specific prior written permission. 20 | 21 | NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY 22 | THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND 23 | CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 25 | PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR 26 | CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 27 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 28 | PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR 29 | BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 30 | IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 31 | ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 32 | POSSIBILITY OF SUCH DAMAGE. 33 | -------------------------------------------------------------------------------- /conf/telegraf/telegraf.conf: -------------------------------------------------------------------------------- 1 | # Configuration for telegraf agent 2 | [agent] 3 | interval = "10s" 4 | round_interval = true 5 | metric_batch_size = 1000 6 | metric_buffer_limit = 10000 7 | 8 | collection_jitter = "3s" 9 | flush_interval = "10s" 10 | flush_jitter = "5s" 11 | 12 | debug = false 13 | quiet = false 14 | logfile = "/var/log/telegraf/telegraf.log" 15 | logfile_rotation_interval = "0d" 16 | logfile_rotation_max_size = "1MB" 17 | logfile_rotation_max_archives = 5 18 | 19 | hostname = "" 20 | 21 | ############################################################################### 22 | # OUTPUT PLUGINS # 23 | ############################################################################### 24 | 25 | # Configuration for sending metrics to InfluxDB 26 | [[outputs.influxdb]] 27 | urls = ["http://influxdb:8086"] # required 28 | database = "telegraf" # required 29 | username = "telegraf" 30 | password = "instrument" 31 | ## If true, the database tag will not be added to the metric. 32 | exclude_database_tag = false 33 | retention_policy = "" 34 | write_consistency = "any" 35 | timeout = "5s" 36 | ## If true, no CREATE DATABASE queries will be sent. Set to true when using 37 | ## Telegraf with a user without permissions to create databases or when the 38 | ## database already exists. 39 | skip_database_creation = false 40 | 41 | ############################################################################### 42 | # INPUT PLUGINS # 43 | ############################################################################### 44 | 45 | # Read metrics about cpu usage 46 | [[inputs.cpu]] 47 | ## Whether to report per-cpu stats or not 48 | percpu = true 49 | ## Whether to report total system cpu stats or not 50 | totalcpu = true 51 | ## If true, collect raw CPU time metrics. 52 | collect_cpu_time = false 53 | report_active = false 54 | 55 | # Read metrics about disk usage by mount point 56 | [[inputs.disk]] 57 | ## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually 58 | ## present on /run, /var/run, /dev/shm or /dev). 59 | ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"] 60 | 61 | # Read metrics about disk IO by device 62 | [[inputs.diskio]] 63 | ## Setting devices will restrict the stats to the specified devices. 64 | # devices = ["sda", "sdb"] 65 | 66 | # Get kernel statistics from /proc/stat 67 | [[inputs.kernel]] 68 | # no configuration 69 | 70 | # Read metrics about memory usage 71 | [[inputs.mem]] 72 | # no configuration 73 | 74 | # Get the number of processes and group them by status 75 | [[inputs.processes]] 76 | # no configuration 77 | 78 | # Read metrics about swap memory usage 79 | [[inputs.swap]] 80 | # no configuration 81 | 82 | # Read metrics about system load & uptime 83 | [[inputs.system]] 84 | ## Uncomment to remove deprecated metrics. 85 | # fielddrop = ["uptime_format"] 86 | 87 | [[inputs.internal]] 88 | collect_memstats = true 89 | 90 | # # Read metrics about network interface usage 91 | [[inputs.net]] 92 | interfaces = ["eth*"] 93 | 94 | # # Read metrics about docker containers 95 | [[inputs.docker]] 96 | endpoint = "unix:///var/run/docker.sock" 97 | timeout = "5s" 98 | 99 | # Statsd Server 100 | [[inputs.statsd]] 101 | ## Address and port to host UDP listener on 102 | service_address = ":8125" 103 | 104 | ## Percentiles to calculate for timing & histogram stats. 105 | percentiles = [50.0, 75.0, 99.0, 99.9] 106 | 107 | ## Delete gauges every interval (default=false) 108 | delete_gauges = true 109 | ## Delete counters every interval (default=false) 110 | delete_counters = true 111 | 112 | ## separator to use between elements of a statsd metric 113 | metric_separator = "_" 114 | 115 | ## convert measurement names, “.” to “_” and “-” to “__” 116 | convert_names = false 117 | 118 | ## used to parse StatD variable name correctly for InfluxDB 119 | ## it enables to have counters/gauges grouped by measurement 120 | templates = [ 121 | "* measurement.field" 122 | ] 123 | 124 | ## Parses extensions to statsd in the datadog statsd format 125 | ## currently supports metrics and datadog tags. 126 | ## http://docs.datadoghq.com/guides/dogstatsd/ 127 | datadog_extensions = true 128 | 129 | ## Number of UDP messages allowed to queue up, once filled, 130 | ## the statsd server will start dropping packets 131 | allowed_pending_messages = 10000 132 | 133 | ## Number of timing/histogram values to track per-measurement in the 134 | ## calculation of percentiles. Raising this limit increases the accuracy 135 | ## of percentiles but also increases the memory usage and cpu time. 136 | percentile_limit = 1000 137 | 138 | 139 | # [[inputs.docker_log]] 140 | # ## To use TCP, set endpoint = "tcp://[ip]:[port]" 141 | # ## To use environment variables (ie, docker-machine), set endpoint = "ENV" 142 | # endpoint = "unix:///var/run/docker.sock" 143 | 144 | # ## When true, container logs are read from the beginning; otherwise 145 | # ## reading begins at the end of the log. 146 | # # from_beginning = false 147 | 148 | # ## docker labels to include. Globs accepted. 149 | # ## Note that an empty array for both will include all labels as tags 150 | # docker_label_include = ["logcapture"] 151 | 152 | # ## Set the source tag for the metrics to the container ID hostname, eg first 12 chars 153 | # source_tag = true 154 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | ## Ultimate Instrumentation 2 | ## Provides 3 | ## - log collection and forwarding (logspout) (localhost:8000/logs) 4 | ## - http/tcp/udp proxying (traefik) (localhost:8080/dashboard/) 5 | ## - service discovery + dns + health-checks (consul) (localhost:8500/ui) (depends on registrator) 6 | ## - time-series DB storage (influxdb) (localhost:8086) 7 | ## - metric & event visualization (grafana) (localhost:3000) 8 | ## - distributed tracing (jaegar) (localhost:16686) 9 | ## http routing provided by traefik on `localhost/` (see SERVICE_TAGS vars for routes) 10 | 11 | version: '3' 12 | 13 | services: 14 | # logging 15 | logspout: 16 | image: "${REGISTRY}gliderlabs/logspout:master" 17 | volumes: ['/var/run/docker.sock:/tmp/docker.sock:ro'] 18 | networks: ['infra','instrument_web'] 19 | ports: ['8000:80'] # logs avail at http://localhost:8000/logs 20 | environment: 21 | EXCLUDE_LABEL: logspout.exclude 22 | # registrator 23 | SERVICE_TAGS: 'traefik.enable=true,traefik.port=8000,traefik.docker.network=instrument_web,traefik.http.routers.router0.rule=PathPrefix(`/logs`)' 24 | 25 | # service discovery (consul + registrator) 26 | consul: 27 | image: "${REGISTRY}consul:1.7" 28 | restart: always 29 | networks: ['infra','instrument_web'] 30 | ports: 31 | - '8500:8500' # web ui 32 | # - '8300:8300' # server rpc 33 | # - '8301:8301' # lan serf tcp 34 | # - '8301:8301/udp' # lan serf udp 35 | # - '8600:8600' # dns tcp 36 | # - '8600:8600/udp' # dns udp 37 | environment: 38 | # registrator 39 | SERVICE_TAGS: "traefik.enable=true,traefik.docker.network=instrument_web" 40 | labels: 41 | - logspout.exclude 42 | registrator: 43 | image: "${REGISTRY}gliderlabs/registrator:master" 44 | depends_on: ['consul'] 45 | networks: ['infra'] 46 | volumes: ['/var/run/docker.sock:/tmp/docker.sock:ro'] 47 | command: ["-internal=true", "consul://consul:8500"] 48 | 49 | # load balancing 50 | traefik: 51 | image: "${REGISTRY}traefik:2.2" 52 | depends_on: ['registrator', 'consul'] 53 | networks: ['infra','instrument_web'] 54 | ports: 55 | - '8080:8080' # web ui 56 | - '80:80' 57 | command: 58 | # - "--log.level=DEBUG" 59 | - "--api.insecure=true" 60 | - "--api.dashboard=true" 61 | - "--providers.consulcatalog=true" 62 | - "--providers.consulcatalog.endpoint.address=http://consul:8500" 63 | - "--providers.consulcatalog.endpoint.datacenter=dc1" 64 | - "--providers.consulcatalog.cache=true" 65 | - "--providers.consulcatalog.exposedByDefault=false" 66 | labels: 67 | - logspout.exclude 68 | 69 | #monitoring 70 | ## TODO - get loki working (in grafana) 71 | # loki: 72 | # image: ${REGISTRY}/grafana/loki:latest 73 | # depends_on: ['registrator', 'consul'] 74 | # ports: ["3100:3100"] 75 | # command: ["-config.file=/etc/loki/local-config.yaml"] 76 | # networks: ['infra'] 77 | # environment: 78 | # # LOGSPOUT: ignore 79 | # SERVICE_TAGS: "traefik.enable=true,traefik.http.routers.router0.rule=PathPrefix(`/loki`)" 80 | 81 | telegraf: 82 | image: "${REGISTRY}telegraf:1.14-alpine" 83 | restart: unless-stopped 84 | networks: ['infra', 'monitor'] 85 | depends_on: ['influxdb'] 86 | labels: 87 | - logspout.exclude 88 | volumes: 89 | - ./conf/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro 90 | # For docker stats 91 | - /var/run/docker.sock:/var/run/docker.sock:ro 92 | 93 | influxdb: 94 | image: "${REGISTRY}influxdb:1.8-alpine" 95 | restart: always 96 | ports: ['8086:8086'] 97 | networks: ['monitor'] 98 | labels: 99 | - logspout.exclude 100 | environment: 101 | INFLUXDB_DB: telegraf 102 | INFLUXDB_USER: telegraf 103 | INFLUXDB_USER_PASSWORD: 'instrument' 104 | INFLUXDB_REPORTING_DISABLED: 'true' 105 | # volumes: 106 | # - influxdb-volume:/var/lib/influxdb 107 | 108 | grafana: 109 | image: "${REGISTRY}grafana/grafana:6.7.3" 110 | depends_on: ['influxdb'] 111 | ports: ["3000:3000"] 112 | networks: ['monitor', 'instrument_web'] 113 | user: "0" 114 | labels: 115 | - logspout.exclude 116 | environment: 117 | GF_SECURITY_ADMIN_PASSWORD: 'instrument' 118 | GF_USERS_ALLOW_SIGN_UP: 'false' 119 | # registrator 120 | SERVICE_TAGS: 'traefik.enable=true,traefik.docker.network=instrument_web,traefik.http.routers.router0.rule=PathPrefix(`/grafana`)' 121 | 122 | # tracing 123 | jaeger: 124 | image: "${REGISTRY}jaegertracing/all-in-one:1.17" 125 | networks: ['monitor','tracing', 'instrument_web'] 126 | environment: 127 | COLLECTOR_ZIPKIN_HTTP_PORT: 9411 128 | labels: 129 | - logspout.exclude 130 | environment: 131 | # registrator 132 | SERVICE_TAGS: 'traefik.enable=true,traefik.docker.network=instrument_web,traefik.http.routers.router0.rule=PathPrefix(`/jaeger`)' 133 | ports: 134 | - '9411:9411' # zipkin 135 | - '16686:16686' # web ui 136 | # - '5778:5778' # http serve configs 137 | - '6831:6831/udp' # jaeger.thrift (compact) 138 | # - '6832:6832/udp' # jaeger.thrift (binary) 139 | # - '14268:14268' # http accept jaeger.thrift directly 140 | # - '14250:14250' # http accept model.proto 141 | 142 | # Create a network for intrastructure components 143 | networks: 144 | infra: 145 | monitor: 146 | tracing: 147 | instrument_web: 148 | external: true 149 | 150 | # Create local persistent volumes 151 | # volumes: 152 | # grafana-volume: 153 | # influxdb-volume: 154 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Gryllidae 🦗 2 | 3 | An (opininated) collection of open-source [CNCF](https://cncf.io)-based Docker services that assist in making [12factor](https://12factor.net) applications. 4 | 5 | **Future work** - Contribute to [awesome-compose](https://github.com/docker/awesome-compose). 6 | 7 | ## Table of Contents 8 | 9 | - [What you get](#what-you-get) 10 | - [What you end up with](#what-you-end-up-with) 11 | - [Getting Started](#getting-started) 12 | - [**tl;dr** - How do we run this thing?](#are-we-there-yet) 13 | - [Extending](#extending) 14 | - [Troubleshooting](#troubleshooting) 15 | - [What is my Docker Compose "`image`"?](#what-is-my-image) 16 | - [I only have a `Dockerfile`... How do I use this?](#i-only-use-dockerfile-not-maven-what-now) 17 | - [The sirens are sounding, and there is no shelter nearby!!!](#what-is-going-wrong-everything-is-falling-apart) 18 | - [canihazk8s-plz](#extras)? 19 | 20 | ## What you get 21 | 22 | - Service Discovery 23 | - [Consul](https://www.consul.io/) 24 | - [Registrator](https://github.com/gliderlabs/registrator) (registers Docker containers with Consul as services) 25 | - [Traefik](https://docs.traefik.io/) for load balancing & proxy against Consul \[1\] services 26 | - [Logspout](https://github.com/gliderlabs/logspout) for [Log collection](https://12factor.net/logs) 27 | - Metrics - The TIG Stack 28 | - [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/) (metrics aggregator and pipeline) w/ [DataDog StatsD parser](https://docs.datadoghq.com/developers/dogstatsd/). Refer to [telegraf.conf](conf/telegraf/telegraf.conf) for customization options. 29 | - [InfluxDB 1.x](https://www.influxdata.com/) (metrics storage) 30 | - [Grafana](https://grafana.com/) (visualization) 31 | - **TODO**: Prometheus (PR's welcome) (Telegraf+Influx work fine, though. Telegraf can be configured to scrape Prometheus Metrics) 32 | - Instrumentation 33 | - [Jaeger](https://www.jaegertracing.io/) (distributed tracing) 34 | 35 | Any passwords for services are `instrument`. For example, Grafana creds are `admin:instrument`. InfluxDB can be queried using `instrument` as the password. 36 | 37 | ## What you end up with 38 | 39 | ### Consul - 40 | 41 | ![consul](doc/img/consul.png) 42 | 43 | ### Logspout - 44 | 45 | ![logspout](doc/img/logspout-cli.png) 46 | 47 | ### Traefik - 48 | 49 | ![traefik dash](doc/img/traefik-dash.png) 50 | 51 | ![traefik routes](doc/img/traefik-routes.png) 52 | 53 | ### Grafana - 54 | 55 | [Docker dashboard](https://grafana.com/grafana/dashboards/893) - id:`893` 56 | 57 | ![grafana-docker](doc/img/grafana-docker.png) 58 | 59 | [Telegraf dashboard](https://grafana.com/grafana/dashboards/928) - id:`928`, id:`5955` 60 | 61 | ![grafana-telegraf](doc/img/grafana-telegraf.png) 62 | 63 | ### Jaeger - 64 | 65 | ![jaeger](doc/img/jaeger.png) 66 | 67 | ## Getting Started 68 | 69 | First, download this repo as a ZIP (use the clone button) and extract it as a folder into your project as `.instrument`. 70 | 71 | Next, create a Docker network for the components 72 | 73 | ```sh 74 | docker network create instrument_web 75 | ``` 76 | 77 | Then, if you are using your own `Makefile`, then add the `.instrument` make targets to it 78 | 79 | ```sh 80 | echo -e '\ninclude .instrument/targets.mk' >> Makefile 81 | ``` 82 | 83 | Otherwise, since we've provided `make` targets for you, go ahead and create your own `Makefile`. (Trust me, using one is nicer than memorizing a bunch of Docker commands) 84 | 85 | Here's a starting template. Note: `clean` and `install` should be updated to actually do things that are dependent on your own code. Also important: tabs matter when updating a `Makefile`. 86 | 87 | ```Makefile 88 | install: 89 | @echo "installing!" 90 | 91 | clean: 92 | @echo "cleaning!" 93 | 94 | include .instrument/targets.mk 95 | ``` 96 | 97 | > _Alright, alright, alright!_ 98 | 99 | Having the infrastructure in place is great, **but** it doesn't help, you, _the application developer_, ensure your app will run on these services. 100 | In order to test your _own app_ in this environment, [make your own `docker-compose.yml` file](https://docs.docker.com/compose/compose-file/) 101 | 102 | Here's a starting template 103 | 104 | ```yaml 105 | version: '3' 106 | networks: 107 | instrument_web: 108 | external: true 109 | 110 | ## Update here 111 | services: 112 | app: 113 | image: containous/whoami # Replace with your own image 114 | ports: # Update with your own ports 115 | - "8081:80" 116 | networks: ['instrument_web'] # This attaches to the underlying infrastructure network 117 | environment: # Update with your environment 118 | FOO: bar 119 | ``` 120 | 121 | Next, add additional services that specify: 122 | 123 | 1. Any dependent services (such as [databases](https://github.com/bitnami/bitnami-docker-mariadb/blob/5.5.48/docker-compose.yml), [Kafka](https://github.com/confluentinc/cp-all-in-one/blob/5.5.0-post/cp-all-in-one-community/docker-compose.yml), etc.). Make sure to only copy the internal sections of any `services` block. 124 | 2. (Optional) Any [links to existing, external services](https://docs.docker.com/compose/networking/). 125 | 126 | If you use a remote service over the network, it is up to you to ensure you have the appropriate network connectivity and firewall options from your machine to those. 127 | 128 | Best practices say to configure such connections via the `environment` block of Compose or [in-app config wiring](https://12factor.net/config). 129 | 130 | 3. Externalized secrets 131 | 132 | > ***Note***: For simplicity, Hashicorp Vault is excluded from this stack. 133 | > 134 | >[Docker Compose can reference a `.env` file](https://docs.docker.com/compose/environment-variables/#the-env-file), should you need local credentials, otherwise use dummy credentials for test databases and such. 135 | > 136 | >**No one** responsible leaking access credentials in Git repos but yourself._ 137 | 138 | ```sh 139 | # add to your gitignore 140 | echo -e '\n.env' >> .gitignore 141 | 142 | vim .env 143 | ``` 144 | 145 | ### Are we there, yet? 146 | 147 | YES!!! 148 | 149 | With all that in place, write in your services (refer to Compose docs above as needed), then get ready to run your application(s)! 150 | 151 | ```sh 152 | make ult-instrument 153 | ``` 154 | 155 | This will run until stopped via Ctrl + C. 156 | 157 | ## Extending 158 | 159 | Hopefully the sevices listed above in [what you get](#what-you-get) are enough. Of course, feel free to mix-and-match with what you think is necessary. 160 | 161 | ## Troubleshooting 162 | 163 | ### It doesn't seem to work 164 | 165 | Make sure you have the following file structure. Any extra files should include documentation and your local application code + build processes. As mentioned below, this has mostly been tested with Apache Maven, but NPM, or similar tooling could be build around this process. 166 | 167 | ```txt 168 | .instrument/ 169 | conf/ 170 | grafana/ 171 | telegraf/ 172 | telegraf.conf 173 | targets.mk 174 | docker-compose.yml 175 | Makefile 176 | docker-compose.yml 177 | ``` 178 | 179 | ### What is my `image`? 180 | 181 | Are you stuck here? 182 | 183 | ```yaml 184 | version: '3' 185 | services: 186 | myapp: 187 | image: ??? 188 | ``` 189 | 190 | You can either pull an image directly off [Docker Hub](https://hub.docker.com/), or more commonly, you are in development mode, and you are testing services locally. When an image is local, you can find it with `docker images`. 191 | 192 | When using any Docker image, the full image name would look like 193 | 194 | ```sh 195 | [docker-registry]/[git-org]/[image-name]:[image-version] 196 | ``` 197 | 198 | Where each part of a full Docker image reference are: 199 | 200 | 1. (optional) Docker Registry 201 | 2. (optional) Docker Org/User 202 | 3. (**required**) Docker Image 203 | 4. (**preferred**) Image version 204 | 205 | Without a registry specified, the default is [Docker Hub](https://hub.docker.com/). Use `docker images` to see what images are already downloaded on your local machine. Creating a Docker accont is free, and will let you create your own Docker Org/User where you can push images for others to use. 206 | 207 | If you exclude the image version, then it defaults to `latest`. Best practices of Docker say to always use a defined version. Preferabbly [SemVer](https://semver.org/), by which the `maven-release-plugin` can generate alongside the [Fabric8 `docker-maven-plugin`](https://dmp.fabric8.io/). 208 | 209 | > Wait... [***Apache Maven***](https://apache.maven.org)? 210 | 211 | Yes, you heard me right... Read on. 212 | 213 | ### I only use Dockerfile, not Maven, what now? 214 | 215 | Maven is not only for Java apps! You _will_ need Java installed, but the featureset of Maven outweighs that burden. 216 | 217 | As mentioned, the Fabric8 plugin works fine and has been tested with this project, so refer its documentation for configuration options. In general, it works similarly to the `maven-assembly-plugin` in that it bundles up the final build artifacts into a Docker image. 218 | 219 | Other options for building Docker images from Maven include 220 | 221 | - (My favorite) : [`jib-maven-plugin` by Google](https://github.com/GoogleContainerTools/jib/tree/master/jib-maven-plugin). Note: This is being merged into the Fabric8 Maven plugins - see [Eclipse JKube](https://github.com/eclipse/jkube). 222 | - [Spotify Docker Maven Plugin (INACTIVE)](https://github.com/spotify/docker-maven-plugin) - It is stable and functional, but Jib builds more optimized images. 223 | 224 | If you find Gradle, SBT, or another build tool works better for you, feel free to let us know. 225 | 226 | ### What is going wrong? Everything is falling apart! 227 | 228 | *Relax. Breeeaathee.* 229 | 230 | If everything started okay, logs from `stdout` / `stderr` of all the services will be `tail`'d. 231 | They can also be followed in another tab (termainal or browser) via `curl http://localhost:8000/logs`. Refer [Logspout documentation](https://github.com/gliderlabs/logspout#including-specific-containers) on performing filters. Of course, `grep` works great here too. 232 | 233 | Should a container die, you'll need to debug it. 234 | 235 | Useful [commands](https://docs.docker.com/compose/reference/): 236 | 237 | - `docker-compose ps` - See what's running (must be ran in same folder as the compose file) 238 | - `docker-compose logs ` - dump the logs of that image. Include `logs -f ` to follow the logs. 239 | - `docker-compose exec bash` - can be used to shell into a container to inspect files and processes like any other terminal session. 240 | 241 | ## Extras 242 | 243 | ### I really like Minikube/Minishift and Helm 244 | 245 | They are nice, sure, but Kube YAML is needlessly verbose for a local environment. 246 | --------------------------------------------------------------------------------