├── .gitignore ├── LICENSE ├── README.md ├── content-pack └── content-pack.json ├── docker-compose.yml ├── elasticsearch └── elasticsearch.yml ├── graylog ├── Dockerfile └── server.conf ├── graylog_web ├── Dockerfile └── graylog-web-interface.conf ├── images ├── apply_cp.png └── upload_cp.png ├── mongo └── mongo.conf └── plugin ├── graylog-alarmcallback-pagerduty-1.1.0.jar ├── graylog-alarmcallback-slack-1.0.0.jar └── usage-statistics-1.0.7.jar /.gitignore: -------------------------------------------------------------------------------- 1 | # local files 2 | .DS_Store 3 | 4 | # VIM 5 | .*.swp 6 | 7 | graylog/node-id 8 | graylog_web/generated.keystore 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Micah Hausler 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Graylog in docker-compose 2 | This `docker-compose.yml` runs all the required processes for a Graylog setup on multiple docker containers. 3 | 4 | The following processes are run in their own docker containers 5 | 6 | * mongodb 3.0.4 7 | * elasticsearch 1.7.4 8 | * graylog 1.3.3 9 | * graylog-web 1.3.3 10 | 11 | ## Setup 12 | This setup assumes you already have docker-compose and docker (using boot2docker) installed. 13 | 14 | ``` 15 | git clone git@github.com:micahhausler/graylog-compose.git 16 | cd graylog-compose 17 | docker-compose build 18 | docker-compose up 19 | ``` 20 | 21 | ## Play 22 | Open [https://192.168.59.103:9443/](https://192.168.59.103:9443/) and use the login. (It may take a minute for the graylog server to come online) 23 | 24 | ``` 25 | username: admin 26 | password: password 27 | ``` 28 | 29 | Then go to the [Content Packs](https://192.168.59.103:9443/system/contentpacks) page, upload the provided content pack, and then click "Apply content". 30 | 31 | ![Upload Content Pack](images/upload_cp.png) 32 | 33 | ![Create Syslog UDP input](images/apply_cp.png) 34 | 35 | ### Input 36 | You can now go to the [Inputs page](https://192.168.59.103:9443/system/inputs) and see that the Docker GELF input has been entered to consume [logspout](https://github.com/gliderlabs/logspout) mesages from Docker (using the [GELF module](https://github.com/micahhausler/logspout-gelf)). 37 | 38 | ### Streams 39 | Go to the [Streams page](https://192.168.59.103:9443/streams#) to see the example streams that have been created. Clock on each one to see past messages. 40 | 41 | _[Hint: Open an incognito window and enter an invalid password in the Graylog login page. This will generate some content for you to see in your streams and dashboard.]_ 42 | 43 | ### Dashboards 44 | Go to the [Dashboards page](https://192.168.59.103:9443/dashboards) to see an example dashboard with graphs based on the 2 preconfigured streams. 45 | 46 | ### Plugins 47 | Go to the [Graylog Plugin page](https://www.graylog.org/resources/integrations/) to see available plugins. Simply drop them in the `plugin/` directory in the project, and they'll be loaded when you restart Graylog. 48 | 49 | ### API explorer 50 | Go to the Graylog API-Explorer [http://192.168.59.103:12900/api-browser](http://192.168.59.103:12900/api-browser) From here, you can play around with their [swagger](http://swagger.io/) api explorer. 51 | 52 | If you're using `docker-machine` instead of boot2docker, edit the line 53 | 54 | ``` 55 | rest_transport_uri= http://192.168.59.103:12900/ 56 | ``` 57 | in `graylog/server.conf` and change it to your docker machine's IP. You'll need to restart everything for this to take effect by running 58 | 59 | ``` 60 | docker-compose stop 61 | docker-compse rm -f 62 | docker-compose up 63 | ``` 64 | 65 | 66 | ## Security 67 | This is NOT a production-ready setup for graylog. You'll need add TLS to [Mongo](https://docs.mongodb.org/manual/reference/configuration-options/#net-ssl-options), [Elasticsearch](https://www.elastic.co/guide/en/shield/current/reference.html#ref-ssl-tls-settings), and the [graylog server](https://gist.github.com/micahhausler/e0b1b47738ee170c6caf#file-server-conf-L56-L68), as well as fine-tune each service for your own needs. This list of measures is not comprehensive. 68 | 69 | Be sure to: 70 | 71 | * change the `password_secret` in `graylog/server.conf` and also add it to `graylog_web/graylog-web-interface.conf`'s `application.secret` parameter 72 | * change the `root_password_sha2` in `graylog/server.conf` 73 | * Add authentication to mongo, enter the parameters in `graylog/server.conf` 74 | 75 | ## License 76 | MIT License 77 | -------------------------------------------------------------------------------- /content-pack/content-pack.json: -------------------------------------------------------------------------------- 1 | { 2 | "id" : null, 3 | "name" : "Demo Content Pack", 4 | "description" : "## DEMO\r\nThis content pack sets up an input, a few streams, and a dashboard.", 5 | "category" : "Demo", 6 | "inputs" : [ { 7 | "title" : "Docker GELF", 8 | "configuration" : { 9 | "port" : 12201, 10 | "override_source" : "", 11 | "bind_address" : "0.0.0.0", 12 | "recv_buffer_size" : 16384 13 | }, 14 | "type" : "org.graylog2.inputs.gelf.udp.GELFUDPInput", 15 | "global" : false, 16 | "extractors" : [ ], 17 | "static_fields" : { } 18 | } ], 19 | "streams" : [ { 20 | "id" : "559fc809e4b0bce97e8174ea", 21 | "title" : "Graylog Server Failed Authentication", 22 | "description" : "AuthenticationException detected", 23 | "disabled" : false, 24 | "outputs" : [ ], 25 | "stream_rules" : [ { 26 | "type" : "REGEX", 27 | "field" : "message", 28 | "value" : "org.apache.shiro.authc.AuthenticationException.*", 29 | "inverted" : false 30 | } ] 31 | }, { 32 | "id" : "559fc787e4b0bce97e81745b", 33 | "title" : "Graylog Server", 34 | "description" : "Graylog Server Output", 35 | "disabled" : false, 36 | "outputs" : [ ], 37 | "stream_rules" : [ { 38 | "type" : "REGEX", 39 | "field" : "docker.image", 40 | "value" : ".*graylog", 41 | "inverted" : false 42 | } ] 43 | } ], 44 | "outputs" : [ ], 45 | "dashboards" : [ { 46 | "title" : "Graylog Dashboard", 47 | "description" : "Detects Failed logins for graylog", 48 | "dashboard_widgets" : [ { 49 | "description" : "Failed Graylog Logins", 50 | "type" : "SEARCH_RESULT_CHART", 51 | "configuration" : { 52 | "interval" : "minute", 53 | "query" : "*", 54 | "timerange" : { 55 | "range" : 300, 56 | "type" : "relative" 57 | }, 58 | "stream_id" : "559fc809e4b0bce97e8174ea" 59 | }, 60 | "col" : 1, 61 | "row" : 1, 62 | "cache_time" : 10 63 | }, { 64 | "description" : "All Graylog Servre messages", 65 | "type" : "SEARCH_RESULT_CHART", 66 | "configuration" : { 67 | "interval" : "minute", 68 | "query" : "*", 69 | "timerange" : { 70 | "range" : 300, 71 | "type" : "relative" 72 | }, 73 | "stream_id" : "559fc787e4b0bce97e81745b" 74 | }, 75 | "col" : 1, 76 | "row" : 2, 77 | "cache_time" : 10 78 | } ] 79 | } ] 80 | } 81 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | # Docker-compose for Graylog 2 | 3 | persistdata: 4 | image: alpine 5 | command: /bin/true 6 | volumes: 7 | - "/data" 8 | - "/usr/share/elasticsearch/data" 9 | - "/opt/graylog/data" 10 | mongodb: 11 | image: "mongo:3.0.4" 12 | hostname: mongo 13 | command: "mongod --config /conf/mongo.conf" 14 | volumes_from: 15 | - persistdata 16 | volumes: 17 | - "./mongo:/conf/" 18 | elasticsearch: 19 | image: "elasticsearch:1.7.4" 20 | hostname: elasticsearch 21 | volumes_from: 22 | - persistdata 23 | volumes: 24 | - "./elasticsearch/:/usr/share/elasticsearch/config" 25 | environment: 26 | ES_HEAP_SIZE: 1g 27 | graylog: 28 | build: ./graylog 29 | hostname: graylog 30 | volumes_from: 31 | - persistdata 32 | volumes: 33 | - "./graylog/:/etc/graylog/server/" 34 | - "./plugin/:/opt/graylog/plugin/" 35 | links: 36 | - "elasticsearch:elasticsearch.service" 37 | - "mongodb:mongodb.service" 38 | ports: 39 | - "12900:12900" 40 | graylogweb: 41 | build: ./graylog_web 42 | volumes: 43 | - "./graylog_web/:/opt/graylog/conf/" 44 | links: 45 | - "graylog:graylog.service" 46 | ports: 47 | - "9443:9443" 48 | 49 | # Based on Logspout by Gliderlabs 50 | # https://github.com/gliderlabs/logspout 51 | # 52 | # I created a module to send Docker logs 53 | # from logspout to Graylog: 54 | # https://github.com/micahhausler/logspout-gelf 55 | logspout: 56 | image: "micahhausler/logspout:gelf" 57 | hostname: logspout 58 | command: "gelf://graylog:12201" 59 | environment: 60 | LOGSPOUT: ignore 61 | volumes: 62 | - "/var/run/docker.sock:/var/run/docker.sock" 63 | links: 64 | - "graylog:graylog" 65 | ports: 66 | - "8000:80" 67 | -------------------------------------------------------------------------------- /elasticsearch/elasticsearch.yml: -------------------------------------------------------------------------------- 1 | # Elasticsearch.conf 2 | # See https://www.elastic.co/guide/en/elasticsearch/reference/1.6/modules.html 3 | # for full documentation 4 | 5 | cluster.name: "graylog-server" 6 | discovery.zen.ping.multicast.enabled: false 7 | discovery.zen.ping.unicast.hosts: ["elasticsearch:9300"] 8 | path.repo: ["/usr/share/elasticsearch/data/"] 9 | -------------------------------------------------------------------------------- /graylog/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM debian:jessie 2 | 3 | MAINTAINER Micah Hausler, micah.hausler@ambition.com 4 | 5 | ENV DEBIAN_FRONTEND noninteractive 6 | ENV GRAYLOG_VERSION 1.3.3 7 | 8 | ENV JAVA_HOME /opt/graylog/embedded/jre 9 | 10 | RUN apt-get update && \ 11 | apt-get -y install \ 12 | curl \ 13 | dnsutils \ 14 | git \ 15 | htop \ 16 | jq \ 17 | less \ 18 | lsof \ 19 | man \ 20 | net-tools \ 21 | ntp \ 22 | openjdk-7-jdk \ 23 | pwgen \ 24 | tzdata \ 25 | unzip \ 26 | vim \ 27 | vim-common \ 28 | wget \ 29 | zip \ 30 | && rm -rf /var/lib/apt/lists/* 31 | 32 | ADD https://packages.graylog2.org/releases/graylog2-server/graylog-$GRAYLOG_VERSION.tgz /opt/ 33 | 34 | RUN cd /opt/ \ 35 | && tar xfz graylog-$GRAYLOG_VERSION.tgz \ 36 | && mv graylog-$GRAYLOG_VERSION/ graylog/ \ 37 | && rm graylog-$GRAYLOG_VERSION.tgz \ 38 | && mkdir -p /opt/graylog/embedded/jre/bin/ \ 39 | && ln -s /usr/bin/java /opt/graylog/embedded/jre/bin/java 40 | 41 | VOLUME /opt/graylog/plugin 42 | 43 | # gelf tcp/udp 44 | EXPOSE 12201 45 | EXPOSE 12201/udp 46 | 47 | # rest api 48 | EXPOSE 12900 49 | 50 | # syslog 51 | EXPOSE 514 52 | EXPOSE 514/udp 53 | 54 | ENV TERM xterm 55 | 56 | WORKDIR /opt/graylog 57 | CMD /opt/graylog/bin/graylogctl run 58 | -------------------------------------------------------------------------------- /graylog/server.conf: -------------------------------------------------------------------------------- 1 | # Graylog config 2 | # See the full example conf at https://gist.github.com/micahhausler/e0b1b47738ee170c6caf 3 | 4 | is_master = true 5 | node_id_file = /etc/graylog/server/node-id 6 | 7 | # Generate one by using for example: pwgen -N 1 -s 96 8 | # or even easier: 9 | # docker run --rm graylogcompose_graylog pwgen -N 1 -s 96 10 | password_secret = Bgw7mYROifuwMCojIcuQ0PYCiuwtOWNSIQBV6fbIfNIzpFFxKrVcRDOxJOPBmjHPhOA5uT928Ll9LlfpPEWxsy9a4VSgZebo 11 | 12 | # Create one by using for example: echo -n yourpassword | shasum -a 256 13 | # and put the resulting hash value into the following line 14 | # This one is just 'password' 15 | root_password_sha2 = 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8 16 | root_username = admin 17 | root_email = "admin@yourdomain.com" 18 | 19 | plugin_dir = plugin 20 | rest_listen_uri = http://0.0.0.0:12900/ 21 | rest_transport_uri= http://graylog:12900/ 22 | rest_enable_cors = true 23 | rotation_strategy = count 24 | 25 | # Elasticsearch 26 | elasticsearch_cluster_name = graylog-server 27 | elasticsearch_discovery_zen_ping_multicast_enabled = false 28 | elasticsearch_discovery_zen_ping_unicast_hosts = elasticsearch.service:9300 29 | elasticsearch_max_docs_per_index = 20000000 30 | elasticsearch_max_number_of_indices = 20 31 | retention_strategy = delete 32 | elasticsearch_shards = 4 33 | elasticsearch_replicas = 0 34 | elasticsearch_index_prefix = graylog2 35 | allow_leading_wildcard_searches = false 36 | allow_highlighting = false 37 | elasticsearch_analyzer = standard 38 | 39 | # Graylog 40 | output_batch_size = 500 41 | output_flush_interval = 1 42 | output_fault_count_threshold = 5 43 | output_fault_penalty_seconds = 30 44 | processbuffer_processors = 5 45 | outputbuffer_processors = 3 46 | processor_wait_strategy = blocking 47 | ring_size = 65536 48 | inputbuffer_ring_size = 65536 49 | inputbuffer_processors = 2 50 | inputbuffer_wait_strategy = blocking 51 | message_journal_enabled = true 52 | message_journal_dir = data/journal 53 | dead_letters_enabled = false 54 | lb_recognition_period_seconds = 3 55 | 56 | # Mongodb 57 | mongodb_uri = mongodb://mongodb.service/graylog2 58 | mongodb_max_connections = 100 59 | mongodb_threads_allowed_to_block_multiplier = 5 60 | -------------------------------------------------------------------------------- /graylog_web/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM debian:jessie 2 | 3 | MAINTAINER Micah Hausler, micah.hausler@ambition.com 4 | 5 | ENV DEBIAN_FRONTEND noninteractive 6 | ENV GRAYLOGWEB_VERSION 1.3.3 7 | 8 | RUN apt-get update && \ 9 | apt-get -y install \ 10 | curl \ 11 | dnsutils \ 12 | git \ 13 | htop \ 14 | jq \ 15 | less \ 16 | man \ 17 | net-tools \ 18 | ntp \ 19 | openjdk-7-jdk \ 20 | pwgen \ 21 | tzdata \ 22 | unzip \ 23 | vim \ 24 | vim-common \ 25 | wget \ 26 | zip \ 27 | && rm -rf /var/lib/apt/lists/* 28 | 29 | ADD https://packages.graylog2.org/releases/graylog2-web-interface/graylog-web-interface-$GRAYLOGWEB_VERSION.tgz /opt/ 30 | 31 | RUN cd /opt/ \ 32 | && tar xfz graylog-web-interface-$GRAYLOGWEB_VERSION.tgz \ 33 | && mv graylog-web-interface-$GRAYLOGWEB_VERSION graylog \ 34 | && rm graylog-web-interface-$GRAYLOGWEB_VERSION.tgz 35 | 36 | EXPOSE 9443 37 | 38 | WORKDIR /opt/graylog 39 | CMD /opt/graylog/bin/graylog-web-interface -Dhttps.port=9443 40 | -------------------------------------------------------------------------------- /graylog_web/graylog-web-interface.conf: -------------------------------------------------------------------------------- 1 | # graylog2-server REST URIs (one or more, comma separated) For example: "http://127.0.0.1:12900/,http://127.0.0.1:12910/" 2 | graylog2-server.uris="http://graylog.service:12900/" 3 | 4 | # Learn how to configure custom logging in the documentation: 5 | # http://support.torch.sh/help/kb/graylog2-web-interface/configuring-web-interface-logging 6 | 7 | # Secret key 8 | # ~~~~~ 9 | # The secret key is used to secure cryptographics functions. Set this to a long and randomly generated string. 10 | # If you deploy your application to several instances be sure to use the same key! 11 | # Generate for example with: pwgen -s 96 12 | application.secret="Bgw7mYROifuwMCojIcuQ0PYCiuwtOWNSIQBV6fbIfNIzpFFxKrVcRDOxJOPBmjHPhOA5uT928Ll9LlfpPEWxsy9a4VSgZebo" 13 | 14 | # Web interface timezone 15 | # Graylog2 stores all timestamps in UTC. To properly display times, set the default timezone of the interface. 16 | # If you leave this out, Graylog2 will pick your system default as the timezone. Usually you will want to configure it explicitly. 17 | # timezone="Europe/Berlin" 18 | 19 | # Message field limit 20 | # Your web interface can cause high load in your browser when you have a lot of different message fields. The default 21 | # limit of message fields is 100. Set it to 0 if you always want to get all fields. They are for example used in the 22 | # search result sidebar or for autocompletion of field names. 23 | field_list_limit=100 24 | 25 | # You usually do not want to change this. 26 | application.global=lib.Global 27 | -------------------------------------------------------------------------------- /images/apply_cp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micahhausler/graylog-compose/7614fb5c01f0a9402788f209d2a42f8696859627/images/apply_cp.png -------------------------------------------------------------------------------- /images/upload_cp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micahhausler/graylog-compose/7614fb5c01f0a9402788f209d2a42f8696859627/images/upload_cp.png -------------------------------------------------------------------------------- /mongo/mongo.conf: -------------------------------------------------------------------------------- 1 | # MongoDB Configuration 2 | # See https://docs.mongodb.org/manual/reference/configuration-options/ 3 | # for full configuration documentation 4 | 5 | storage: 6 | engine: wiredTiger 7 | dbPath: "/data/db" 8 | directoryPerDB: true 9 | journal: 10 | enabled: true 11 | systemLog: 12 | timeStampFormat: iso8601-utc 13 | processManagement: 14 | fork: false 15 | net: 16 | port: 27017 17 | wireObjectCheck : true 18 | unixDomainSocket: 19 | enabled : true 20 | -------------------------------------------------------------------------------- /plugin/graylog-alarmcallback-pagerduty-1.1.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micahhausler/graylog-compose/7614fb5c01f0a9402788f209d2a42f8696859627/plugin/graylog-alarmcallback-pagerduty-1.1.0.jar -------------------------------------------------------------------------------- /plugin/graylog-alarmcallback-slack-1.0.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micahhausler/graylog-compose/7614fb5c01f0a9402788f209d2a42f8696859627/plugin/graylog-alarmcallback-slack-1.0.0.jar -------------------------------------------------------------------------------- /plugin/usage-statistics-1.0.7.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micahhausler/graylog-compose/7614fb5c01f0a9402788f209d2a42f8696859627/plugin/usage-statistics-1.0.7.jar --------------------------------------------------------------------------------