├── doc ├── grafana-graph.png ├── grafana-dash-err.png ├── grafana-dash-ok.png ├── grafana-graph-setup.png ├── grafana-single-stat.png └── grafana-single-stat-setup.png ├── data ├── docker-entrypoint.sh ├── etc │ └── supervisord.conf ├── httpd.py ├── update-metrics.sh └── src │ └── aws-ec2-sg-exporter ├── LICENSE ├── .travis.yml ├── Makefile ├── Dockerfile └── README.md /doc/grafana-graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-graph.png -------------------------------------------------------------------------------- /doc/grafana-dash-err.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-dash-err.png -------------------------------------------------------------------------------- /doc/grafana-dash-ok.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-dash-ok.png -------------------------------------------------------------------------------- /doc/grafana-graph-setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-graph-setup.png -------------------------------------------------------------------------------- /doc/grafana-single-stat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-single-stat.png -------------------------------------------------------------------------------- /doc/grafana-single-stat-setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-single-stat-setup.png -------------------------------------------------------------------------------- /data/docker-entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Be strict 4 | set -e 5 | set -u 6 | set -o pipefail 7 | 8 | 9 | ### 10 | ### Create metrics before starting up anything 11 | ### 12 | START=$(date +%s) 13 | if /usr/bin/aws-ec2-sg-exporter > /var/www/index.html.new; then 14 | END=$(date +%s) 15 | mv -f /var/www/index.html.new /var/www/index.html 16 | printf "[OK] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "docker-entrypoint.sh" "Metrics updated in $(( ${END} - ${START} )) sec" 17 | else 18 | END=$(date +%s) 19 | >&2 printf "[ERR] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "docker-entrypoint.sh" "Failed to update metrics after $(( ${END} - ${START} )) sec" 20 | exit 1 21 | fi 22 | 23 | 24 | ### 25 | ### Start up 26 | ### 27 | exec /usr/bin/supervisord -c /etc/supervisord.conf 28 | -------------------------------------------------------------------------------- /data/etc/supervisord.conf: -------------------------------------------------------------------------------- 1 | [supervisord] 2 | user = root 3 | nodaemon = true 4 | logfile = /var/log/supervisord.log 5 | pidfile = /var/run/supervisord.pid 6 | childlogdir = /var/log/ 7 | 8 | [program:httpd] 9 | command = /usr/bin/httpd.py 10 | autostart = true 11 | autorestart = true 12 | stdout_logfile = /dev/stdout 13 | stdout_logfile_maxbytes = 0 14 | stdout_events_enabled = true 15 | stderr_logfile = /dev/stderr 16 | stderr_logfile_maxbytes = 0 17 | stderr_events_enabled = true 18 | 19 | [program:update-metrics] 20 | command = /usr/bin/update-metrics.sh 21 | autostart = true 22 | autorestart = true 23 | stdout_logfile = /dev/stdout 24 | stdout_logfile_maxbytes = 0 25 | stdout_events_enabled = true 26 | stderr_logfile = /dev/stderr 27 | stderr_logfile_maxbytes = 0 28 | stderr_events_enabled = true 29 | -------------------------------------------------------------------------------- /data/httpd.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Credits: https://pythonbasics.org/webserver/ 4 | 5 | from http.server import BaseHTTPRequestHandler, HTTPServer 6 | 7 | hostName = "0.0.0.0" 8 | serverPort = 8080 9 | staticFile = "/var/www/index.html" 10 | 11 | 12 | class SimpleHTTPRequestHandler(BaseHTTPRequestHandler): 13 | 14 | def do_GET(self): 15 | self.send_response(200) 16 | self.send_header("Content-type", "text/plain") 17 | self.end_headers() 18 | 19 | with open(staticFile, 'r') as content_file: 20 | content = content_file.read() 21 | self.wfile.write(bytes(content, "utf-8")) 22 | 23 | if __name__ == "__main__": 24 | httpd = HTTPServer((hostName, serverPort), SimpleHTTPRequestHandler) 25 | print("Server started http://%s:%s" % (hostName, serverPort)) 26 | 27 | try: 28 | httpd.serve_forever() 29 | except KeyboardInterrupt: 30 | pass 31 | 32 | httpd.server_close() 33 | print("Server stopped.") 34 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 cytopia 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | ### 4 | ### Enable sudo (required for docker service) 5 | ### 6 | sudo: required 7 | 8 | 9 | ### 10 | ### Language 11 | ### 12 | language: minimal 13 | 14 | 15 | ### 16 | ### Add services 17 | ### 18 | services: 19 | - docker 20 | 21 | 22 | ### 23 | ### Build Matrix 24 | ### 25 | env: 26 | matrix: 27 | - VERSION=latest 28 | 29 | 30 | ### 31 | ### Install requirements 32 | ### 33 | install: 34 | - retry() { 35 | for ((n=0; n<10; n++)); do 36 | echo "[${n}] ${*}"; 37 | if eval "${*}"; then 38 | return 0; 39 | fi; 40 | done; 41 | return 1; 42 | } 43 | 44 | 45 | ### 46 | ### Check generation changes, build and test 47 | ### 48 | before_script: 49 | - retry make lint 50 | - retry make build 51 | 52 | 53 | ### 54 | ### Push to Dockerhub 55 | ### 56 | script: 57 | # Push to docker hub on success 58 | - if [ "${TRAVIS_PULL_REQUEST}" == "false" ]; then 59 | while ! make login USER="${DOCKER_USERNAME}" PASS="${DOCKER_PASSWORD}"; do sleep 1; done; 60 | if [ -n "${TRAVIS_TAG}" ]; then 61 | while ! make push TAG="${TRAVIS_TAG}"; do sleep 1; done; 62 | elif [ "${TRAVIS_BRANCH}" == "master" ]; then 63 | while ! make push; do sleep 1; done; 64 | elif [[ ${TRAVIS_BRANCH} =~ ^(release-[.0-9]+)$ ]]; then 65 | while ! make push TAG="${TRAVIS_BRANCH}"; do sleep 1; done; 66 | else 67 | echo "Skipping branch ${TRAVIS_BRANCH}"; 68 | fi 69 | else 70 | echo "Skipping push on PR"; 71 | fi 72 | -------------------------------------------------------------------------------- /data/update-metrics.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | 8 | # ------------------------------------------------------------------------------------------------- 9 | # F U N C T I O N S 10 | # ------------------------------------------------------------------------------------------------- 11 | 12 | ### 13 | ### Function called by trap 14 | ### 15 | do_this_on_ctrl_c(){ 16 | exit 0 17 | } 18 | 19 | 20 | # ------------------------------------------------------------------------------------------------- 21 | # E N T R Y P O I N T 22 | # ------------------------------------------------------------------------------------------------- 23 | 24 | ### 25 | ### Add Trap for Ctrl+c 26 | ### 27 | trap 'do_this_on_ctrl_c' SIGINT 28 | 29 | 30 | ### 31 | ### How often to update the metrics 32 | ### 33 | if env | grep -q '^UPDATE_TIME='; then 34 | UPDATE_TIME="$( env | grep '^UPDATE_TIME=' | sed 's/^UPDATE_TIME=//g' )" 35 | else 36 | UPDATE_TIME=60 37 | fi 38 | 39 | 40 | ### 41 | ### Update metrics endlessly 42 | ### 43 | while true; do 44 | sleep "${UPDATE_TIME}" 45 | START=$(date +%s) 46 | if /usr/bin/aws-ec2-sg-exporter > /var/www/index.html.new; then 47 | END=$(date +%s) 48 | mv -f /var/www/index.html.new /var/www/index.html 49 | printf "[OK] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "update-metrics.sh" "Metrics updated in $(( ${END} - ${START} )) sec" 50 | else 51 | END=$(date +%s) 52 | >&2 printf "[ERR] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "update-metrics.sh" "Failed to update metrics after $(( ${END} - ${START} )) sec" 53 | false 54 | fi 55 | done 56 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | ifneq (,) 2 | .error This Makefile requires GNU Make. 3 | endif 4 | 5 | .PHONY: build rebuild lint test _test-version _test-run tag pull login push enter 6 | 7 | CURRENT_DIR = $(dir $(abspath $(lastword $(MAKEFILE_LIST)))) 8 | 9 | DIR = . 10 | FILE = Dockerfile 11 | IMAGE = cytopia/aws-ec2-sg-exporter 12 | TAG = latest 13 | 14 | build: 15 | docker build -t $(IMAGE) -f $(DIR)/$(FILE) $(DIR) 16 | 17 | rebuild: pull 18 | docker build --no-cache -t $(IMAGE) -f $(DIR)/$(FILE) $(DIR) 19 | 20 | lint: 21 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-cr --text --ignore '.git/,.github/,tests/' --path . 22 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-crlf --text --ignore '.git/,.github/,tests/' --path . 23 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-trailing-single-newline --text --ignore '.git/,.github/,tests/' --path . 24 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-trailing-space --text --ignore '.git/,.github/,tests/' --path . 25 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-utf8 --text --ignore '.git/,.github/,tests/' --path . 26 | @docker run --rm -v $(CURRENT_DIR):/data cytopia/file-lint file-utf8-bom --text --ignore '.git/,.github/,tests/' --path . 27 | 28 | test: 29 | true 30 | 31 | 32 | tag: 33 | docker tag $(IMAGE) $(IMAGE):$(TAG) 34 | 35 | pull: 36 | @grep -E '^\s*FROM' Dockerfile \ 37 | | sed -e 's/^FROM//g' -e 's/[[:space:]]*as[[:space:]]*.*$$//g' \ 38 | | xargs -n1 docker pull; 39 | 40 | login: 41 | yes | docker login --username $(USER) --password $(PASS) 42 | 43 | push: 44 | @$(MAKE) tag TAG=$(TAG) 45 | docker push $(IMAGE):$(TAG) 46 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM alpine:3.9 as builder 2 | 3 | RUN set -x \ 4 | && apk add --no-cache \ 5 | gcc \ 6 | libffi-dev \ 7 | make \ 8 | musl-dev \ 9 | openssl-dev \ 10 | python3 \ 11 | python3-dev 12 | 13 | RUN set -x \ 14 | && pip3 install --no-cache-dir --no-compile awscli \ 15 | && rm -r /usr/lib/python3.6/site-packages/awscli/examples \ 16 | && find /usr/lib/ -name '__pycache__' -print0 | xargs -0 -n1 rm -rf \ 17 | && find /usr/lib/ -name '*.pyc' -print0 | xargs -0 -n1 rm -rf \ 18 | && aws --version 2>&1 | grep -E '^aws-cli/[.0-9]+' 19 | 20 | RUN set -x \ 21 | && pip3 install --no-cache-dir --no-compile supervisor \ 22 | && find /usr/lib/ -name '__pycache__' -print0 | xargs -0 -n1 rm -rf \ 23 | && find /usr/lib/ -name '*.pyc' -print0 | xargs -0 -n1 rm -rf 24 | 25 | 26 | FROM alpine:3.9 as production 27 | LABEL \ 28 | maintainer="cytopia " \ 29 | repo="https://github.com/cytopia/aws-ec2-sg-exporter" 30 | 31 | RUN set -eux \ 32 | && mkdir -p /var/www \ 33 | && apk add --no-cache \ 34 | bash \ 35 | bind-tools \ 36 | curl \ 37 | jq \ 38 | python3 \ 39 | && ln -sf /usr/bin/python3 /usr/bin/python \ 40 | && find /usr/lib/ -name '__pycache__' -print0 | xargs -0 -n1 rm -rf \ 41 | && find /usr/lib/ -name '*.pyc' -print0 | xargs -0 -n1 rm -rf 42 | 43 | COPY --from=builder /usr/lib/python3.6/site-packages/ /usr/lib/python3.6/site-packages/ 44 | COPY --from=builder /usr/bin/aws /usr/bin/aws 45 | COPY --from=builder /usr/bin/supervisord /usr/bin/supervisord 46 | 47 | COPY data/docker-entrypoint.sh /docker-entrypoint.sh 48 | COPY data/httpd.py /usr/bin/httpd.py 49 | COPY data/update-metrics.sh /usr/bin/update-metrics.sh 50 | 51 | COPY data/etc/supervisord.conf /etc/supervisord.conf 52 | COPY data/src/aws-ec2-sg-exporter /usr/bin/aws-ec2-sg-exporter 53 | 54 | ENTRYPOINT ["/docker-entrypoint.sh"] 55 | -------------------------------------------------------------------------------- /data/src/aws-ec2-sg-exporter: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | set -u 5 | set -o pipefail 6 | 7 | 8 | # ------------------------------------------------------------------------------------------------- 9 | # V A R I A B L E S 10 | # ------------------------------------------------------------------------------------------------- 11 | 12 | ### 13 | ### The Prometheus metric name 14 | ### 15 | METRIC_NAME="aws_ec2_sg_compare" 16 | 17 | 18 | 19 | # ------------------------------------------------------------------------------------------------- 20 | # F U N C T I O N S 21 | # ------------------------------------------------------------------------------------------------- 22 | 23 | ### 24 | ### Check if exactly one security group exists by filter 25 | ### Output's all found security ids separated by newline. 26 | ### 27 | ### Returns 0 if one security group id was found 28 | ### Returns 1 if no security group id was found 29 | ### Returns 2 if more than 1 security group id was found 30 | ### 31 | get_sg_ids() { 32 | local name="${1}" 33 | local region="${2}" 34 | local count=0 35 | 36 | for sg_id in $( \ 37 | aws ec2 describe-security-groups \ 38 | --region "${region}" \ 39 | --filters "Name=tag:Name,Values=${name}" \ 40 | --query 'SecurityGroups[*].GroupId' \ 41 | --output text \ 42 | ); do 43 | count=$(( count + 1)) 44 | echo "${sg_id}" 45 | done 46 | 47 | # No security group found 48 | if [ "${count}" -eq "0" ]; then 49 | return 1 50 | # More than 1 security group found 51 | elif [ "${count}" -gt "1" ]; then 52 | return 2 53 | fi 54 | 55 | # Exactly one security group found 56 | return 0 57 | } 58 | 59 | 60 | ### 61 | ### Get newline-separated AWS Security Group IPv4 CIDR by name, protocol and from_port 62 | ### 63 | get_applied_sg_ip4_addr() { 64 | local name="${1}" 65 | local region="${2}" 66 | local from_port="${3}" 67 | local proto="${4}" 68 | 69 | aws ec2 describe-security-groups \ 70 | --region "${region}" \ 71 | --filters "Name=tag:Name,Values=${name}" \ 72 | --query 'SecurityGroups[*].IpPermissions' \ 73 | | dd obs=1M 2>/dev/null \ 74 | | jq -r ".[][] 75 | | select(.IpProtocol | contains(\"${proto}\") ) 76 | | select(.FromPort | contains(${from_port}) ) 77 | | .IpRanges[].CidrIp" 78 | } 79 | 80 | 81 | ### 82 | ### Get newline-separated AWS Security Group IPv6 CIDR by name, protocol and from_port 83 | ### 84 | get_applied_sg_ip6_addr() { 85 | local name="${1}" 86 | local region="${2}" 87 | local from_port="${3}" 88 | local proto="${4}" 89 | 90 | aws ec2 describe-security-groups \ 91 | --region "${region}" \ 92 | --filters "Name=tag:Name,Values=${name}" \ 93 | --query 'SecurityGroups[*].IpPermissions' \ 94 | | dd obs=1M 2>/dev/null \ 95 | | jq -r ".[][] 96 | | select(.IpProtocol | contains(\"${proto}\") ) 97 | | select(.FromPort | contains(${from_port}) ) 98 | | .Ipv6Ranges[].CidrIpv6" 99 | } 100 | 101 | 102 | ### 103 | ### Evalute command to get newline-separated wanted IPv4 CIDR 104 | ### 105 | get_wanted_sg_ip4_addr() { 106 | eval "${1}" | dd obs=1M 2>/dev/null | sort 107 | } 108 | 109 | 110 | ### 111 | ### Evalute command to get newline-separated wanted IPv6 CIDR 112 | ### 113 | get_wanted_sg_ip6_addr() { 114 | eval "${1}" | dd obs=1M 2>/dev/null | sort 115 | } 116 | 117 | 118 | ### 119 | ### Print one metric line in Prometheus format 120 | ### 121 | print_prom_metric() { 122 | local name="${1}" 123 | local region="${2}" 124 | local proto="${3}" 125 | local from_port="${4}" 126 | local version="${5}" 127 | local cidr="${6}" 128 | local sg_id="${7}" 129 | local errno="${8}" 130 | local error="${9}" 131 | local counter="${10}" 132 | 133 | printf "${METRIC_NAME}{name=\"%s\",region=\"%s\",proto=\"%s\",from_port=\"%s\",ip=\"%s\",cidr=\"%s\",id=\"%s\",errno=\"%s\",error=\"%s\"} %s\n" \ 134 | "${name}" \ 135 | "${region}" \ 136 | "${proto}" \ 137 | "${from_port}" \ 138 | "${version}" \ 139 | "${cidr}" \ 140 | "${sg_id}" \ 141 | "${errno}" \ 142 | "${error}" \ 143 | "${counter}" 144 | } 145 | 146 | 147 | ### 148 | ### Compare applied vs wanted IPv4/6 CIDR and output Prometheus metrics for each security group 149 | ### 150 | compare() { 151 | local name="${1}" 152 | local region="${2}" 153 | local proto="${3}" 154 | local from_port="${4}" 155 | local ip4_cmd="${5}" 156 | local ip6_cmd="${6}" 157 | 158 | local sg_ids= 159 | local sg_ret= 160 | local error= 161 | local errno= 162 | 163 | local applied= 164 | local wanted= 165 | local version= 166 | 167 | if [ -n "${name}" ]; then 168 | 169 | # Get found security group id's 170 | set +e 171 | sg_ids="$( get_sg_ids "${name}" "${region}" )" 172 | sg_ret=$? 173 | set -e 174 | 175 | # Evalute possible errors on found/not-found security groups 176 | if [ "${sg_ret}" -eq "0" ]; then 177 | errno=0 178 | error="" 179 | elif [ "${sg_ret}" -eq "1" ]; then 180 | errno=1 181 | error="no sg found" 182 | >&2 printf "[ERR] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "aws-ec2-sg-exporter" "No sg found by name: ${name} in region: ${region}" 183 | elif [ "${sg_ret}" -gt "1" ]; then 184 | errno=2 185 | error="multiple sg found" 186 | sg_ids="$( echo "${sg_ids}" | paste -s -d, - )" 187 | >&2 printf "[ERR] %s (%s): %s\n" "$(date +"%Y-%m-%d %H:%M:%S")" "aws-ec2-sg-exporter" "Multiple sg found by name: ${name} in region: ${region}: ${sg_ids}" 188 | fi 189 | 190 | if [ -n "${ip4_cmd}" ]; then 191 | applied="$( get_applied_sg_ip4_addr "${name}" "${region}" "${from_port}" "${proto}" )" 192 | wanted="$( get_wanted_sg_ip4_addr "${ip4_cmd}" )" 193 | version="v4" 194 | else 195 | applied="$( get_applied_sg_ip6_addr "${name}" "${region}" "${from_port}" "${proto}" )" 196 | wanted="$( get_wanted_sg_ip6_addr "${ip6_cmd}" )" 197 | version="v6" 198 | fi 199 | 200 | for ip in ${wanted}; do 201 | # Exactly one security group found 202 | if [ "${errno}" -eq "0" ]; then 203 | if echo "${applied}" | dd obs=1M 2>/dev/null | grep -Eq "^${ip}\$"; then 204 | print_prom_metric "${name}" "${region}" "${proto}" "${from_port}" "${version}" "${ip}" "${sg_ids}" "${errno}" "${error}" "1" 205 | else 206 | print_prom_metric "${name}" "${region}" "${proto}" "${from_port}" "${version}" "${ip}" "${sg_ids}" "${errno}" "${error}" "0" 207 | fi 208 | # Zero or more than one security group found 209 | else 210 | print_prom_metric "${name}" "${region}" "${proto}" "${from_port}" "${version}" "${ip}" "" "${errno}" "${error}" "0" 211 | fi 212 | done 213 | fi 214 | } 215 | 216 | 217 | 218 | # -------------------------------------------------------------------------------------------------- 219 | # PRE-FLIGHT CHECK 220 | # -------------------------------------------------------------------------------------------------- 221 | 222 | ### 223 | ### Ensure 'aws' binary is available 224 | ### 225 | if ! command -v aws >/dev/null 2>&1; then 226 | >&2 "Error, 'aws' binary is required" 227 | exit 1 228 | fi 229 | 230 | if ! command -v jq >/dev/null 2>&1; then 231 | >&2 "Error, 'jq' binary is required" 232 | exit 1 233 | fi 234 | 235 | 236 | 237 | ### 238 | ### Copy environment variables to bash variables 239 | ### 240 | SG1_NAME="$( env | grep '^SG1_NAME=' | sed 's/^SG1_NAME=//g' || true )" 241 | SG1_REGION="$( env | grep '^SG1_REGION=' | sed 's/^SG1_REGION=//g' || true )" 242 | SG1_PROTO="$( env | grep '^SG1_PROTO=' | sed 's/^SG1_PROTO=//g' || true )" 243 | SG1_FROM_PORT="$( env | grep '^SG1_FROM_PORT=' | sed 's/^SG1_FROM_PORT=//g'|| true )" 244 | SG1_IP4_CMD="$( env | grep '^SG1_IP4_CMD=' | sed 's/^SG1_IP4_CMD=//g' || true )" 245 | SG1_IP6_CMD="$( env | grep '^SG1_IP6_CMD=' | sed 's/^SG1_IP6_CMD=//g' || true )" 246 | 247 | SG2_NAME="$( env | grep '^SG2_NAME=' | sed 's/^SG2_NAME=//g' || true )" 248 | SG2_REGION="$( env | grep '^SG2_REGION=' | sed 's/^SG2_REGION=//g' || true )" 249 | SG2_PROTO="$( env | grep '^SG2_PROTO=' | sed 's/^SG2_PROTO=//g' || true )" 250 | SG2_FROM_PORT="$( env | grep '^SG2_FROM_PORT=' | sed 's/^SG2_FROM_PORT=//g'|| true )" 251 | SG2_IP4_CMD="$( env | grep '^SG2_IP4_CMD=' | sed 's/^SG2_IP4_CMD=//g' || true )" 252 | SG2_IP6_CMD="$( env | grep '^SG2_IP6_CMD=' | sed 's/^SG2_IP6_CMD=//g' || true )" 253 | 254 | SG3_NAME="$( env | grep '^SG3_NAME=' | sed 's/^SG3_NAME=//g' || true )" 255 | SG3_REGION="$( env | grep '^SG3_REGION=' | sed 's/^SG3_REGION=//g' || true )" 256 | SG3_PROTO="$( env | grep '^SG3_PROTO=' | sed 's/^SG3_PROTO=//g' || true )" 257 | SG3_FROM_PORT="$( env | grep '^SG3_FROM_PORT=' | sed 's/^SG3_FROM_PORT=//g'|| true )" 258 | SG3_IP4_CMD="$( env | grep '^SG3_IP4_CMD=' | sed 's/^SG3_IP4_CMD=//g' || true )" 259 | SG3_IP6_CMD="$( env | grep '^SG3_IP6_CMD=' | sed 's/^SG3_IP6_CMD=//g' || true )" 260 | 261 | SG4_NAME="$( env | grep '^SG4_NAME=' | sed 's/^SG4_NAME=//g' || true )" 262 | SG4_REGION="$( env | grep '^SG4_REGION=' | sed 's/^SG4_REGION=//g' || true )" 263 | SG4_PROTO="$( env | grep '^SG4_PROTO=' | sed 's/^SG4_PROTO=//g' || true )" 264 | SG4_FROM_PORT="$( env | grep '^SG4_FROM_PORT=' | sed 's/^SG4_FROM_PORT=//g'|| true )" 265 | SG4_IP4_CMD="$( env | grep '^SG4_IP4_CMD=' | sed 's/^SG4_IP4_CMD=//g' || true )" 266 | SG4_IP6_CMD="$( env | grep '^SG4_IP6_CMD=' | sed 's/^SG4_IP6_CMD=//g' || true )" 267 | 268 | 269 | 270 | # ------------------------------------------------------------------------------------------------- 271 | # E N T R Y P O I N T 272 | # ------------------------------------------------------------------------------------------------- 273 | 274 | ### 275 | ### 276 | ### 277 | echo "# HELP ${METRIC_NAME} Determines If CIDR is applied to security group." 278 | echo "# TYPE ${METRIC_NAME} counter" 279 | 280 | 281 | compare "${SG1_NAME:-}" "${SG1_REGION:-}" "${SG1_PROTO:-}" "${SG1_FROM_PORT:-}" "${SG1_IP4_CMD:-}" "${SG1_IP6_CMD:-}" 282 | compare "${SG2_NAME:-}" "${SG2_REGION:-}" "${SG2_PROTO:-}" "${SG2_FROM_PORT:-}" "${SG2_IP4_CMD:-}" "${SG2_IP6_CMD:-}" 283 | compare "${SG3_NAME:-}" "${SG3_REGION:-}" "${SG3_PROTO:-}" "${SG3_FROM_PORT:-}" "${SG3_IP4_CMD:-}" "${SG3_IP6_CMD:-}" 284 | compare "${SG4_NAME:-}" "${SG4_REGION:-}" "${SG4_PROTO:-}" "${SG4_FROM_PORT:-}" "${SG4_IP4_CMD:-}" "${SG4_IP6_CMD:-}" 285 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS Security Group exporter for Prometheus 2 | 3 | **[Motivation](#motivation)** | 4 | **[How does it work](#how-does-it-work)** | 5 | **[Requirements](#requirements)** | 6 | **[Docker settings](#docker-settings)** | 7 | **[Metrics](#metrics)** | 8 | **[Examples](#examples)** | 9 | **[Grafana](#grafana-setup)** | 10 | **[Errors](#error-handling)** 11 | 12 | [![Build Status](https://travis-ci.com/cytopia/aws-ec2-sg-exporter.svg?branch=master)](https://travis-ci.com/cytopia/aws-ec2-sg-exporter) 13 | [![Tag](https://img.shields.io/github/tag/cytopia/aws-ec2-sg-exporter.svg)](https://github.com/cytopia/aws-ec2-sg-exporter/releases) 14 | [![](https://images.microbadger.com/badges/version/cytopia/aws-ec2-sg-exporter:latest.svg?&kill_cache=1)](https://microbadger.com/images/cytopia/aws-ec2-sg-exporter:latest "aws-ec2-sg-exporter") 15 | [![](https://images.microbadger.com/badges/image/cytopia/aws-ec2-sg-exporter:latest.svg?&kill_cache=1)](https://microbadger.com/images/cytopia/aws-ec2-sg-exporter:latest "aws-ec2-sg-exporter") 16 | [![](https://img.shields.io/docker/pulls/cytopia/aws-ec2-sg-exporter.svg)](https://hub.docker.com/r/cytopia/aws-ec2-sg-exporter) 17 | [![](https://img.shields.io/badge/github-cytopia%2Faws--ec2--sg--exporter-red.svg)](https://github.com/cytopia/aws-ec2-sg-exporter "github.com/cytopia/aws-ec2-sg-exporter") 18 | [![License](https://img.shields.io/badge/license-MIT-%233DA639.svg)](https://opensource.org/licenses/MIT) 19 | 20 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-dash-ok.png "Grafana Graph Example") 21 | 22 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-dash-err.png "Grafana Graph Example") 23 | 24 | A dockerized[1] Prometheus exporter that compares desired/wanted 25 | IPv4/IPv6 CIDR against currently applied inbound CIDR rules by protocol and port number in your AWS 26 | security group(s) per region. 27 | 28 | > [1]: If you want to use this exporter without Docker jump here: [Usage without Docker](#usage-without-docker) 29 | 30 | [![Docker hub](http://dockeri.co/image/cytopia/aws-ec2-sg-exporter?&kill_cache=1)](https://hub.docker.com/r/cytopia/aws-ec2-sg-exporter) 31 | 32 | 33 | ## Motivation 34 | 35 | Some IP addresses ranges such as Cloudfront edge nodes or SaaS hosts might change frequently and 36 | you possibly want to ensure that those are always in sync with what you have currently defined in 37 | your security group. 38 | This exporter does exactly this and can easily be hooked up with Alertmanager to trigger alerts in 39 | case you get out of sync. 40 | 41 | 42 | ## How does it work 43 | 44 | #### Desired/Wanted IP address CIDR 45 | 46 | You have to provide a command, which is parsable by bash's `eval` function and evalutes 47 | **newline-separated** to your desired/wanted IP address CIDR. As a few examples: 48 | ```bash 49 | # Note that for single IP addresses, AWS requires '/32' to be appended 50 | eval "dig +short nat.travisci.net | xargs -n1 -I% echo \"%/32\"" 51 | eval "printf \"10.13.23.23/32\n192.168.0.0/24\n\"" 52 | ``` 53 | 54 | #### Applied security Group CIDR 55 | 56 | You have to provide the following in order to fetch your currently applied sg rules: 57 | 58 | * Security group name (The `Name` tag)[1] 59 | * AWS region where the security group resides 60 | * Security group rule protocol (e.g.: `tcp`, `udp`, `icmp`, ...) 61 | * Security group rule from port (e.g.: `80`, `443`, ...) 62 | 63 | > [1]: The `*` wildcard is supported for the name, but you have to ensure to match exactly one security group 64 | 65 | #### Output 66 | 67 | The exporter will then output Prometheus readable information as such: 68 | ```bash 69 | # HELP aws_ec2_sg_compare Determines If CIDR is applied to security group. 70 | # TYPE aws_ec2_sg_compare counter 71 | aws_ec2_sg_compare{name="sg-name",region="us-east-1",proto="tcp",from_port="80",ip="v4",cidr="10.4.1.1/32",sg_id="sg-xxxxx",errno="0",error=""} 1 72 | aws_ec2_sg_compare{name="sg-name",region="us-east-1",proto="tcp",from_port="80",ip="v4",cidr="10.4.1.5/32",sg_id="sg-xxxxx",errno="0",error=""} 0 73 | ``` 74 | 75 | * A value of `1` means the desired/wanted IP CIDR is applied to the security group 76 | * A value of `0` means the desired/wanted IP CIDR is not applied to the security group 77 | 78 | See [Metrics](#metrics) for an indepth description. 79 | 80 | 81 | ## Requirements 82 | 83 | You will need AWS access key and secret with the following permission: 84 | ```yaml 85 | ec2:DescribeSecurityGroups 86 | ``` 87 | 88 | 89 | ## Docker settings 90 | 91 | ### Tagging 92 | 93 | Ensure to **use Docker image tags** (which are the same as git tags from this repository) to prevent 94 | any backwards incompatible changes. The `latest` tag should only be used for testing purposes. 95 | 96 | Additionally do not blindly update Docker image tags before having tested it. Security group rule 97 | checks are an important matter and you want to ensure your alerting is reliable. 98 | 99 | 100 | ### Environment variables 101 | 102 | You can specify up to 4 security group checks: `SG1_*`, `SG2_*`, `SG3_*` and `SG4_*`. 103 | 104 | | Variable | Description | 105 | |-------------------------|-------------| 106 | | `AWS_ACCESS_KEY_ID` | The AWS access key (required to connect to AWS to check the sg rules) | 107 | | `AWS_SECRET_ACCESS_KEY` | The AWS secret key (required to connect to AWS to check the sg rules) | 108 | | `AWS_SESSION_TOKEN` | (Optional) The AWS session token | 109 | | | | 110 | | `UPDATE_TIME` | Time interval in sec for how often to update metrics (default: `60`) | 111 | | | | 112 | | `SG1_NAME` | Name of the security group on AWS | 113 | | `SG1_REGION` | Region the security group resides in | 114 | | `SG1_PROTO` | Security group rule protocol: `tcp`, `udp`, `icmp` or a protocol number | 115 | | `SG1_FROM_PORT` | Security group rule from port | 116 | | `SG1_IP4_CMD` | The command that evaluates to newline-separated IPv4 IP address CIDR [1] | 117 | | `SG1_IP6_CMD` | The command that evaluates to newline-separated IPv6 IP address CDIR [1] | 118 | | | | 119 | | `SG2_NAME` | Name of the security group on AWS | 120 | | `SG2_REGION` | Region the security group resides in | 121 | | `SG2_PROTO` | Security group rule protocol: `tcp`, `udp`, `icmp` or a protocol number | 122 | | `SG2_FROM_PORT` | Security group rule from port | 123 | | `SG2_IP4_CMD` | The command that evaluates to newline-separated IPv4 IP address CIDR [1] | 124 | | `SG2_IP6_CMD` | The command that evaluates to newline-separated IPv6 IP address CDIR [1] | 125 | | | | 126 | | `SG3_NAME` | Name of the security group on AWS | 127 | | `SG3_REGION` | Region the security group resides in | 128 | | `SG3_PROTO` | Security group rule protocol: `tcp`, `udp`, `icmp` or a protocol number | 129 | | `SG3_FROM_PORT` | Security group rule from port | 130 | | `SG3_IP4_CMD` | The command that evaluates to newline-separated IPv4 IP address CIDR [1] | 131 | | `SG3_IP6_CMD` | The command that evaluates to newline-separated IPv6 IP address CDIR [1] | 132 | | | | 133 | | `SG4_NAME` | Name of the security group on AWS | 134 | | `SG4_REGION` | Region the security group resides in | 135 | | `SG4_PROTO` | Security group rule protocol: `tcp`, `udp`, `icmp` or a protocol number | 136 | | `SG4_FROM_PORT` | Security group rule from port | 137 | | `SG4_IP4_CMD` | The command that evaluates to newline-separated IPv4 IP address CIDR [1] | 138 | | `SG4_IP6_CMD` | The command that evaluates to newline-separated IPv6 IP address CDIR [1] | 139 | 140 | > [1]: `SG*_IP4_CMD` and `SG*_IP6_CMD` are mutually exclusive. Also note that evaluated 141 | IP address CIDR are only checked against security group rules that match the protocol (`SG*_PROTO`) 142 | and also match the from port (`SG*_FROM_PORT`). 143 | 144 | 145 | ### Mount points 146 | 147 | None - it's fully stateless 148 | 149 | 150 | ### Exposed ports 151 | 152 | | External | Internal | Description | 153 | |-----------|----------|-------------| 154 | | Up to you | `8080` | Where the `aws-ec2-sg-exporter` provides metrics via HTTP | 155 | 156 | 157 | ## Metrics 158 | 159 | This exporter outputs metrics in the following format: 160 | ```bash 161 | # HELP aws_ec2_sg_compare Determines If CIDR is applied to security group. 162 | # TYPE aws_ec2_sg_compare counter 163 | aws_ec2_sg_compare{name="sg-name",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="10.4.1.1/32",sg_id="sg-xxxxx",errno="0",error=""} 1 164 | ``` 165 | The following table describes each of the key/value paris: 166 | 167 | | Key | Value | 168 | |-------------|-------| 169 | | `name` | The security group name as specified by `SG*_NAME` | 170 | | `region` | The security group region as specified by `SG*_REGION` | 171 | | `proto` | The security group rule protocol as specified by `SG*_PROTO` | 172 | | `from_port` | The security group rule from port as specified by `SG*_FROM_PORT` | 173 | | `ip` | IP version of desired/wanted CIDR to be available in your security group by `proto` and `from_port` | 174 | | `cidr` | The desired/wanted IP to be available in your security group by `proto` and `from_port` | 175 | | `sg_id` | The security group ID found by `name` and `region`. If this is empty then either zero or more multiple security groups were found. | 176 | | `errno` | 0: One security group was found (OK)
1: No security group was found (ERR)
2: Multiple security groups were found (ERR) | 177 | | `error` | The corresponding error message for `errno` | 178 | 179 | * A value of `1` means the desired/wanted IP CIDR is applied to the security group 180 | * A value of `0` means the desired/wanted IP CIDR is not applied to the security group 181 | 182 | 183 | ## Examples 184 | 185 | ### Scenario 1 - Travis 186 | Check if your security group named `my-sg` (in us-east-1) allows all inbound IPv4 addresses from Travis-CI via `tcp` on `https`. 187 | 188 | #### Desired/wanted IP CIDR 189 | Ensure you have a working command which can be interpretated by `eval` and that outputs CIDR (with `/[0-9]+` appended) of your desired ranges: 190 | ```bash 191 | $ eval "dig +short nat.travisci.net | xargs -n1 -I% echo \"%/32\"" 192 | ``` 193 | ```bash 194 | 35.184.226.236/32 195 | 35.188.1.99/32 196 | 35.188.73.34/32 197 | 35.192.85.2/32 198 | 35.192.136.167/32 199 | ... 200 | ``` 201 | 202 | #### Run `aws-ec2-sg-exporter` 203 | ```bash 204 | docker run -it --rm \ 205 | -p 9000:8080 \ 206 | \ 207 | -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \ 208 | -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \ 209 | \ 210 | -e SG1_NAME="my-sg" \ 211 | -e SG1_REGION="us-east-1" \ 212 | -e SG1_PROTO="tcp" \ 213 | -e SG1_FROM_PORT="443" \ 214 | -e SG1_IP4_CMD="dig +short nat.travisci.net | xargs -n1 -I% echo \"%/32\"" \ 215 | cytopia/aws-ec2-sg-exporter 216 | ``` 217 | 218 | #### Check output 219 | Check the output via curl: 220 | ```bash 221 | $ curl localhost:9000` 222 | ``` 223 | ```bash 224 | # HELP aws_ec2_sg_compare Determines If CIDR is applied to security group. 225 | # TYPE aws_ec2_sg_compare counter 226 | aws_ec2_sg_compare{name="my-sg",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="35.184.226.236/32",sg_id="sg-xxxxx",errno="0",error=""} 1 227 | aws_ec2_sg_compare{name="my-sg",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="35.188.1.99/32",sg_id="sg-xxxxx",errno="0",error=""} 1 228 | aws_ec2_sg_compare{name="my-sg",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="35.188.73.34/32",sg_id="sg-xxxxx",errno="0",error=""} 1 229 | aws_ec2_sg_compare{name="my-sg",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="35.192.85.2/32",sg_id="sg-xxxxx",errno="0",error=""} 1 230 | aws_ec2_sg_compare{name="my-sg",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="35.192.136.167/32",sg_id="sg-xxxxx",errno="0",error=""} 0 231 | ... 232 | ``` 233 | 234 | As you can see, the last line returns a `0`, which means this IP CIDR is missing in your security group. 235 | 236 | 237 | ### Scenario 2 - Cloudfront 238 | 239 | * Check if your security group named `my-sg4` (in us-east-1) allows all inbound IPv4 addresses from Cloudfront edge-nodes via `tcp` on `https`. 240 | * Check if your security group named `my-sg6` (in us-east-1) allows all inbound IPv6 addresses from Cloudfront edge-nodes via `tcp` on `https`. 241 | 242 | #### Desired/wanted IP CIDR 243 | Ensure you have a working command which can be interpretated by `eval` and that outputs CIDR (with `/[0-9]+` appended) of your desired ranges: 244 | ```bash 245 | $ eval "curl -sS https://ip-ranges.amazonaws.com/ip-ranges.json \ 246 | | jq -r '.prefixes \ 247 | | sort_by(.ip_prefix)[] \ 248 | | select( .service | contains(\"CLOUDFRONT\")) \ 249 | | select ( .region | test(\"^(GLOBAL|us-|eu-)\")) \ 250 | | .ip_prefix'" 251 | ``` 252 | ```bash 253 | 13.224.0.0/14 254 | 13.249.0.0/16 255 | 13.32.0.0/15 256 | 13.35.0.0/16 257 | 13.52.204.0/23 258 | ... 259 | ``` 260 | ```bash 261 | $ eval "curl -sS https://ip-ranges.amazonaws.com/ip-ranges.json \ 262 | | jq -r '.ipv6_prefixes \ 263 | | sort_by(.ipv6_prefixes)[] \ 264 | | select( .service | contains(\"CLOUDFRONT\")) \ 265 | | select ( .region | test(\"^(GLOBAL|us-|eu-)\")) \ 266 | | .ipv6_prefix'" 267 | ``` 268 | ```bash 269 | 2600:9000:eee::/48 270 | 2600:9000:4000::/36 271 | 2600:9000:3000::/36 272 | 2600:9000:f000::/36 273 | 2600:9000:fff::/48 274 | ... 275 | ``` 276 | 277 | #### Run `aws-ec2-sg-exporter` 278 | ```bash 279 | docker run -it --rm \ 280 | -p 9000:8080 \ 281 | \ 282 | -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \ 283 | -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \ 284 | \ 285 | -e SG1_NAME="my-sg4" \ 286 | -e SG1_REGION="us-east-1" \ 287 | -e SG1_PROTO="tcp" \ 288 | -e SG1_FROM_PORT="443" \ 289 | -e SG1_IP4_CMD="curl -sS https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes | sort_by(.ip_prefix)[] | select( .service | contains(\"CLOUDFRONT\")) | select ( .region | test(\"^(GLOBAL|us-|eu-)\")) | .ip_prefix'" \ 290 | \ 291 | -e SG2_NAME="my-sg6" \ 292 | -e SG2_REGION="us-east-1" \ 293 | -e SG2_PROTO="tcp" \ 294 | -e SG2_FROM_PORT="443" \ 295 | -e SG2_IP6_CMD="curl -sS https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.ipv6_prefixes | sort_by(.ipv6_prefixes)[] | select( .service | contains(\"CLOUDFRONT\")) | select ( .region | test(\"^(GLOBAL|us-|eu-)\")) | .ipv6_prefix'" \ 296 | cytopia/aws-ec2-sg-exporter 297 | ``` 298 | 299 | #### Check output 300 | Check the output via curl: 301 | ```bash 302 | $ curl localhost:9000` 303 | ``` 304 | ```bash 305 | # HELP aws_ec2_sg_compare Determines If CIDR is applied to security group. 306 | # TYPE aws_ec2_sg_compare counter 307 | aws_ec2_sg_compare{name="my-sg4",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="13.224.0.0/14",sg_id="sg-xxxxx",errno="0",error=""} 1 308 | aws_ec2_sg_compare{name="my-sg4",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="13.249.0.0/16",sg_id="sg-xxxxx",errno="0",error=""} 0 309 | aws_ec2_sg_compare{name="my-sg4",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="13.32.0.0/15",sg_id="sg-xxxxx",errno="0",error=""} 1 310 | aws_ec2_sg_compare{name="my-sg4",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="13.35.0.0/16",sg_id="sg-xxxxx",errno="0",error=""} 1 311 | aws_ec2_sg_compare{name="my-sg4",region="us-east-1",proto="tcp",from_port="443",ip="v4",cidr="13.52.204.0/23",sg_id="sg-xxxxx",errno="0",error=""} 1 312 | ... 313 | aws_ec2_sg_compare{name="my-sg6",region="us-east-1",proto="tcp",from_port="443",ip="v6",cidr="2600:9000:eee::/48",sg_id="sg-yyyyy",errno="0",error=""} 1 314 | aws_ec2_sg_compare{name="my-sg6",region="us-east-1",proto="tcp",from_port="443",ip="v6",cidr="2600:9000:4000::/36",sg_id="sg-yyyyy",errno="0",error=""} 1 315 | aws_ec2_sg_compare{name="my-sg6",region="us-east-1",proto="tcp",from_port="443",ip="v6",cidr="2600:9000:3000::/36",sg_id="sg-yyyyy",errno="0",error=""} 1 316 | aws_ec2_sg_compare{name="my-sg6",region="us-east-1",proto="tcp",from_port="443",ip="v6",cidr="2600:9000:f000::/36",sg_id="sg-yyyyy",errno="0",error=""} 1 317 | aws_ec2_sg_compare{name="my-sg6",region="us-east-1",proto="tcp",from_port="443",ip="v6",cidr="2600:9000:fff::/48",sg_id="sg-yyyyy",errno="0",error=""} 0 318 | ... 319 | ``` 320 | 321 | As you can see, the second line ipv4 address returns a `0` and the last ipv6 address returns a `0`, which means these IP CIDR are missing in your security groups. 322 | 323 | 324 | ## Grafana setup 325 | 326 | ### Graphs 327 | 328 | * Align the `Min time interval` with what you have set `UPDATE_TIME` to. 329 | * Add you metrics by the name of your specified security group name 330 | * Set the legend to `{{ cidr }}` to have only the CIDR displayed 331 | 332 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-graph-setup.png "Grafana Graph Setup Example") 333 | 334 | Once this is done, your graph will look similar to this one: 335 | 336 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-graph.png "Grafana Graph Example") 337 | 338 | ### Single Stat 339 | 340 | * Align the `Min time interval` with what you have set `UPDATE_TIME` to. 341 | * Add you metrics by the name of your specified security group name 342 | * `sum()` gives your the total sum of values (`0` and `1`) and `count()` will give you the total number of available IP addresses 343 | 344 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-single-stat-setup.png "Grafana Single Stat Setup Example") 345 | 346 | Once this is done, your single stat will look similar to this one: 347 | 348 | ![Grafana](https://raw.githubusercontent.com/cytopia/aws-ec2-sg-exporter/master/doc/grafana-single-stat.png "Grafana Single Stat Example") 349 | 350 | 351 | ## Usage without Docker 352 | 353 | Docker is not necessarily required and you can simply use the exporter bash script: [aws-ec2-sg-exporter](data/src/aws-ec2-sg-exporter). 354 | 355 | By doing so, you need to ensure you have all requirements installed on your system (`aws` and `jq` binary as well as `bash` itself). 356 | 357 | Additionally you will have to make sure the script's `stdout` will somehow be served by a webserver. 358 | The recommended method is to have some mechanism which writes the script's output atomically to a static html file and a web server will simply serve it. 359 | 360 | [aws-ec2-sg-exporter](data/src/aws-ec2-sg-exporter) will read all configuration from the shell's environment, so in order to use this script you need to export 361 | all values to your env. See [Environment variables](#environment-variables) for possible values. 362 | 363 | 364 | ## Error handling 365 | 366 | The exporter writes all errors to `stderr` regardless of using Docker or the standalone [aws-ec2-sg-exporter](data/src/aws-ec2-sg-exporter) script. 367 | 368 | ### Expected errors 369 | 370 | **`An error occurred (RequestExpired) when calling the DescribeSecurityGroups operation: Request has expired.`** 371 | 372 | In case you are using IAM roles, your session has simply been expired and needs to be renewed. It 373 | is recommended to user IAM users instead without session. 374 | 375 | 376 | **`[ERR] 2019-08-18 10:55:11 (aws-ec2-sg-exporter): No sg found by name: sg-name22 in region: us-east-1`** 377 | 378 | A security group could not be found by name and region. The exporter will continue to run and output 379 | Prometheus metrics, but will mark all desired/wanted IP CIDR as not found in your security group. 380 | 381 | 382 | **`[ERR] 2019-08-18 10:56:17 (aws-ec2-sg-exporter): Multiple sg found by name: sg-name-* in region: us-east-1: sg-xxxxx,sg-yyyyy`** 383 | 384 | Multiple security groups have been found by the specified name and region. The exporter will continue to run and output 385 | Prometheus metrics, but will mark all desired/wanted IP CIDR as not found in your security group. 386 | 387 | 388 | ### Unexpected errors 389 | 390 | **`write error: Broken pipe`** 391 | 392 | This is a very rare condition and will most likely be caused by using broken shell pipes (`|`) 393 | in your commands specified via `SG*_IP4_CMD` or `SG*_IP6_CMD`. 394 | 395 | In case you are using something like this: 396 | ```bash 397 | curl http://some-page.tld | grep -E '^[.0-9]+/[0-9]+$'; 398 | ``` 399 | Consider to add a buffer in between: 400 | ``` 401 | curl http://some-page.tld | dd obs=1M 2>/dev/null | grep -E '^[.0-9]+/[0-9]+$'; 402 | ``` 403 | 404 | See here: https://superuser.com/a/642932/705357 405 | 406 | 407 | ## License 408 | 409 | **[MIT License](LICENSE)** 410 | 411 | Copyright (c) 2019 [cytopia](https://github.com/cytopia) 412 | --------------------------------------------------------------------------------