├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md ├── app ├── exporter.py └── requirements.txt └── example └── config.yml /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.pyc -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:2.7.13-alpine 2 | 3 | COPY app /opt/prometheus-jsonpath-exporter 4 | 5 | RUN pip install -r /opt/prometheus-jsonpath-exporter/requirements.txt 6 | 7 | EXPOSE 9158 8 | 9 | ENTRYPOINT ["python", "/opt/prometheus-jsonpath-exporter/exporter.py"] 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Project Sunbird 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # prometheus-jsonpath-exporter 2 | 3 | Converts json data from a http url into prometheus metrics using jsonpath 4 | 5 | 6 | ### Config 7 | 8 | ```yml 9 | exporter_port: 9158 # Port on which prometheus can call this exporter to get metrics 10 | log_level: info 11 | json_data_url: http://stubonweb.herokuapp.com/kong-cluster-status # Url to get json data used for fetching metric values 12 | metric_name_prefix: kong_cluster # All metric names will be prefixed with this value 13 | metrics: 14 | - name: total_nodes # Final metric name will be kong_cluster_total_nodes 15 | description: Total number of nodes in kong cluster 16 | path: $.total 17 | - name: alive_nodes # Final metric name will be kong_cluster_alive_nodes 18 | description: Number of live nodes in kong cluster 19 | path: count($.data[@.status is "alive"]) 20 | ``` 21 | 22 | See the example below to understand how the json data and metrics will look for this config 23 | 24 | ### Run 25 | 26 | #### Using code (local) 27 | 28 | ``` 29 | # Ensure python 2.x and pip installed 30 | pip install -r app/requirements.txt 31 | python app/exporter.py example/config.yml 32 | ``` 33 | 34 | #### Using docker 35 | 36 | ``` 37 | docker run -p 9158:9158 -v $(pwd)/example/config.yml:/etc/prometheus-jsonpath-exporter/config.yml sunbird/prometheus-jsonpath-exporter /etc/prometheus-jsonpath-exporter/config.yml 38 | ``` 39 | 40 | ### JsonPath Syntax 41 | 42 | This exporter uses [objectpath](http://objectpath.org) python library. The syntax is documented [here](http://objectpath.org/reference.html) 43 | 44 | ### Example 45 | 46 | For the above config, if the configured `json_data_url` returns 47 | 48 | ```json 49 | { 50 | "data": [ 51 | { 52 | "address": "x.x.x.15:7946", 53 | "status": "failed" 54 | }, 55 | { 56 | "address": "x.x.x.19:7946", 57 | "status": "alive" 58 | }, 59 | { 60 | "address": "x.x.x.12:7946", 61 | "status": "alive" 62 | } 63 | ], 64 | "total": 3 65 | } 66 | ``` 67 | 68 | Metrics will available in http://localhost:9158 69 | 70 | 71 | 72 | ``` 73 | $ curl -s localhost:9158 | grep ^kong 74 | kong_cluster_total_nodes 3.0 75 | kong_cluster_alive_nodes 2.0 76 | ``` 77 | 78 | -------------------------------------------------------------------------------- /app/exporter.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import json 4 | import time 5 | import urllib2 6 | from prometheus_client import start_http_server 7 | from prometheus_client.core import GaugeMetricFamily, REGISTRY 8 | import argparse 9 | import yaml 10 | from objectpath import Tree 11 | import logging 12 | 13 | DEFAULT_PORT=9158 14 | DEFAULT_LOG_LEVEL='info' 15 | 16 | class JsonPathCollector(object): 17 | def __init__(self, config): 18 | self._config = config 19 | 20 | def collect(self): 21 | config = self._config 22 | result = json.loads(urllib2.urlopen(config['json_data_url'], timeout=10).read()) 23 | result_tree = Tree(result) 24 | for metric_config in config['metrics']: 25 | metric_name = "{}_{}".format(config['metric_name_prefix'], metric_config['name']) 26 | metric_description = metric_config.get('description', '') 27 | metric_path = metric_config['path'] 28 | value = result_tree.execute(metric_path) 29 | logging.debug("metric_name: {}, value for '{}' : {}".format(metric_name, metric_path, value)) 30 | metric = GaugeMetricFamily(metric_name, metric_description, value=value) 31 | yield metric 32 | 33 | 34 | if __name__ == "__main__": 35 | parser = argparse.ArgumentParser(description='Expose metrics bu jsonpath for configured url') 36 | parser.add_argument('config_file_path', help='Path of the config file') 37 | args = parser.parse_args() 38 | with open(args.config_file_path) as config_file: 39 | config = yaml.load(config_file) 40 | log_level = config.get('log_level', DEFAULT_LOG_LEVEL) 41 | logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.getLevelName(log_level.upper())) 42 | exporter_port = config.get('exporter_port', DEFAULT_PORT) 43 | logging.debug("Config %s", config) 44 | logging.info('Starting server on port %s', exporter_port) 45 | start_http_server(exporter_port) 46 | REGISTRY.register(JsonPathCollector(config)) 47 | while True: time.sleep(1) -------------------------------------------------------------------------------- /app/requirements.txt: -------------------------------------------------------------------------------- 1 | pytz 2 | objectpath 3 | prometheus_client 4 | pyYAML -------------------------------------------------------------------------------- /example/config.yml: -------------------------------------------------------------------------------- 1 | 2 | exporter_port: 9158 # Port on which prometheu can call this exporter to get metrics 3 | log_level: info 4 | json_data_url: http://stubonweb.herokuapp.com/kong-cluster-status # Url to get json data used for fetching metric values 5 | metric_name_prefix: kong_cluster # All metric names will be prefixed with this value 6 | metrics: 7 | - name: total_nodes # Final metric name will be kong_cluster_total_nodes 8 | description: Total number of nodes in kong cluster 9 | path: $.total 10 | - name: alive_nodes # Final metric name will be kong_cluster_alive_nodes 11 | description: Number of live nodes in kong cluster 12 | path: count($.data[@.status is "alive"]) 13 | --------------------------------------------------------------------------------