├── .github └── FUNDING.yml ├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── defaults └── main.yml ├── files ├── README.md ├── logstash-forwarder-example.crt ├── logstash-forwarder.deb.init ├── logstash-forwarder.rpm.init └── logstashforwarder.repo ├── handlers └── main.yml ├── meta └── main.yml ├── tasks ├── main.yml ├── setup-Debian.yml └── setup-RedHat.yml ├── templates └── logstash-forwarder.j2 └── tests ├── inventory └── test.yml /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | --- 3 | github: geerlingguy 4 | patreon: geerlingguy 5 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | files/logstash-forwarder* 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sudo: required 3 | language: python 4 | python: "2.7" 5 | 6 | env: 7 | - SITE=test.yml 8 | 9 | before_install: 10 | - sudo apt-get update -qq 11 | - sudo apt-get install curl 12 | 13 | install: 14 | # Install Ansible. 15 | - pip install ansible 16 | 17 | # Add ansible.cfg to pick up roles path. 18 | - "{ echo '[defaults]'; echo 'roles_path = ../'; } >> ansible.cfg" 19 | 20 | script: 21 | # Check the role/playbook's syntax. 22 | - "ansible-playbook -i tests/inventory tests/$SITE --syntax-check" 23 | 24 | # Run the role/playbook with ansible-playbook. 25 | - "ansible-playbook -i tests/inventory tests/$SITE --connection=local --become" 26 | 27 | # Run the role/playbook again, checking to make sure it's idempotent. 28 | - > 29 | ansible-playbook -i tests/inventory tests/$SITE --connection=local --become 30 | | grep -q 'changed=0.*failed=0' 31 | && (echo 'Idempotence test: pass' && exit 0) 32 | || (echo 'Idempotence test: fail' && exit 1) 33 | 34 | # TODO: Test if logstash forwarder is working correctly. 35 | 36 | notifications: 37 | webhooks: https://galaxy.ansible.com/api/v1/notifications/ 38 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2017 Jeff Geerling 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so, 10 | subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 17 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 18 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 19 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 20 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Ansible Role: Logstash Forwarder 2 | 3 | **DEPRECATED**: This role is deprecated, as Logstash Forwarder has been deprecated and is replaced by Filebeat. Please use the [`geerlingguy.filebeat`](https://github.com/geerlingguy/ansible-role-filebeat) role instead. 4 | 5 | [![Build Status](https://travis-ci.org/geerlingguy/ansible-role-logstash-forwarder.svg?branch=master)](https://travis-ci.org/geerlingguy/ansible-role-logstash-forwarder) 6 | 7 | An Ansible Role that installs Logstash Forwarder on RedHat/CentOS or Debian/Ubuntu. 8 | 9 | **Note**: This role is well-tested on Debian/Ubuntu, but is still undergoing development for RedHat/CentOS. You've been warned! 10 | 11 | ## Requirements 12 | 13 | None. 14 | 15 | ## Role Variables 16 | 17 | Available variables are listed below, along with default values (see `defaults/main.yml`): 18 | 19 | logstash_forwarder_logstash_server: localhost 20 | logstash_forwarder_logstash_server_port: 5000 21 | 22 | The central Logstash server/port to which logstash-forwarder should connect. 23 | 24 | logstash_ssl_dir: /etc/pki/logstash 25 | logstash_forwarder_ssl_certificate_file: logstash-forwarder-example.crt 26 | 27 | The location and filename of the SSL certificate logstash-forwarder will use to authenticate to the logstash server. For the `logstash_forwarder_ssl_certificate_file`, you can provide a path relative to the role directory, or an absolute path to the file. 28 | 29 | logstash_forwarder_files: 30 | - paths: 31 | - /var/log/messages 32 | - /var/log/auth.log 33 | fields: 34 | type: syslog 35 | 36 | Configuration of files monitored by logstash-forwarder. You can add more sets of files by adding to the list with another set of files; see `defaults/main.yml` for an example. 37 | 38 | ## Dependencies 39 | 40 | None. 41 | 42 | ## Example Playbook 43 | 44 | - hosts: all 45 | roles: 46 | - { role: geerlingguy.logstash-forwarder } 47 | 48 | ## License 49 | 50 | MIT / BSD 51 | 52 | ## Author Information 53 | 54 | This role was created in 2014 by [Jeff Geerling](https://www.jeffgeerling.com/), author of [Ansible for DevOps](https://www.ansiblefordevops.com/). 55 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | logstash_forwarder_logstash_server: localhost 3 | logstash_forwarder_logstash_server_port: 5000 4 | 5 | logstash_ssl_dir: /etc/pki/logstash 6 | logstash_forwarder_ssl_certificate_file: logstash-forwarder-example.crt 7 | 8 | logstash_forwarder_files: 9 | - paths: 10 | - /var/log/messages 11 | - /var/log/auth.log 12 | fields: 13 | type: syslog 14 | # To add more sets of files... 15 | # - paths: 16 | # - /var/log/boot.log 17 | # fields: 18 | # type: syslog 19 | -------------------------------------------------------------------------------- /files/README.md: -------------------------------------------------------------------------------- 1 | Please use your own certificate instead of the example included in this folder, and be sure to update the `logstash_ssl_*` variables in your playbook to use your files instead of this example. 2 | 3 | To generate a self-signed certificate/key pair, you can use use the command: 4 | 5 | $ sudo openssl req -x509 -batch -nodes -days 3650 -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash-forwarder.crt 6 | -------------------------------------------------------------------------------- /files/logstash-forwarder-example.crt: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIDtTCCAp2gAwIBAgIJAPRAgobPBWQEMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV 3 | BAYTAkFVMRMwEQYDVQQIEwpTb21lLVN0YXRlMSEwHwYDVQQKExhJbnRlcm5ldCBX 4 | aWRnaXRzIFB0eSBMdGQwHhcNMTQwODI1MTcxNDQyWhcNMjQwODIyMTcxNDQyWjBF 5 | MQswCQYDVQQGEwJBVTETMBEGA1UECBMKU29tZS1TdGF0ZTEhMB8GA1UEChMYSW50 6 | ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB 7 | CgKCAQEAvrcDmeea73j9n7mL1uqMrOtX3ullLZ+I3W7KAbrJeRHTfPeD0O5RAXfJ 8 | +4cI4o2hHJlHj9RV+pDSXTjCNC5pgOnhjz1YJeIrDArqNHrXEDa9qj9JzVKtB3N2 9 | 5M5m0Ruh0hM7JtWplzXUIpsRUiLk4gu8OhDrfJ0WrRRQvihs+OZ2ha5ca2AMTSIg 10 | Ebx3ezq0nt1t6RvFgYULqP1j5waYdV8dXFCnzp3fVLgjN2/Np+CRyhZj1ZgWnN0/ 11 | /xTCCAxtSSxPXB7wwk9N5RJOYdAh+uHexCtIz3yi0No7RmZQvs6GwOuyn3hV658e 12 | qkhwhdVzekOzCuvV702xPWw5JMbSxQIDAQABo4GnMIGkMB0GA1UdDgQWBBRgLN2R 13 | V+t92I7wzAHzHZ614budpDB1BgNVHSMEbjBsgBRgLN2RV+t92I7wzAHzHZ614bud 14 | pKFJpEcwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgTClNvbWUtU3RhdGUxITAfBgNV 15 | BAoTGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZIIJAPRAgobPBWQEMAwGA1UdEwQF 16 | MAMBAf8wDQYJKoZIhvcNAQEFBQADggEBACRBgrkVcQlUw1I4IQQzxt/pXJdfowSx 17 | x+xWIOQ0ljYV7yRAvOOCNu9blRWX60mRXpsrufIYo4D1e+CJcQI7YyEUyciPmTgo 18 | AQS6EGgiIIHnDPR7C7Yaq0rhVp3LgUudzAMNRSdLopYXmDhqOJbgtDjXN8FPQdo5 19 | OmvgvEZigoZ1zcd0bK+O2LAu1ggVp9XEx/cYxHx6RSUwIFzmUk4MNxh+15W+wYj7 20 | +sTH1c2MAGUdT2ntYi+zN1Jx7+fD6osdYcv2vg/h4T4H9Llg2XPsv+dC3fqQCLDq 21 | r9d4sJDOjWvyZpgG5Tr45nU5FskdyhcoR7MJCo7BGPL2N1CikVnMtCE= 22 | -----END CERTIFICATE----- 23 | -------------------------------------------------------------------------------- /files/logstash-forwarder.deb.init: -------------------------------------------------------------------------------- 1 | #! /bin/sh 2 | ### BEGIN INIT INFO 3 | # Provides: skeleton 4 | # Required-Start: $remote_fs $syslog 5 | # Required-Stop: $remote_fs $syslog 6 | # Default-Start: 2 3 4 5 7 | # Default-Stop: 0 1 6 8 | # Short-Description: Example initscript 9 | # Description: This file should be used to construct scripts to be 10 | # placed in /etc/init.d. 11 | ### END INIT INFO 12 | 13 | # See: http://goo.gl/cz9Lnd 14 | # Once https://github.com/elasticsearch/logstash-forwarder/pull/196 is merged, 15 | # update this role to pull directly from the new .rpm.init file instead of 16 | # copying this file. 17 | 18 | # Author: Jordan Sissel 19 | 20 | # PATH should only include /usr/* if it runs after the mountnfs.sh script 21 | PATH=/sbin:/usr/sbin:/bin:/usr/bin 22 | DESC="log shipper" 23 | NAME=logstash-forwarder 24 | DAEMON=/opt/logstash-forwarder/bin/logstash-forwarder 25 | DAEMON_ARGS="-config /etc/logstash-forwarder -spool-size 100 -log-to-syslog" 26 | PIDFILE=/var/run/$NAME.pid 27 | SCRIPTNAME=/etc/init.d/$NAME 28 | 29 | [ -r /etc/default/$NAME ] && . /etc/default/$NAME 30 | . /lib/init/vars.sh 31 | . /lib/lsb/init-functions 32 | 33 | COMMAND="cd /var/run; exec $DAEMON $DAEMON_ARGS" 34 | 35 | do_start() { 36 | # Skip if it's already running 37 | start-stop-daemon --start --quiet --pidfile $PIDFILE --exec /bin/sh --test > /dev/null || return 1 38 | 39 | cd /var/run 40 | # Actually start it now. 41 | start-stop-daemon --start --quiet --make-pidfile --background \ 42 | --pidfile $PIDFILE --exec /bin/sh -- -c "$COMMAND" || return 2 43 | } 44 | 45 | do_stop() 46 | { 47 | start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE 48 | RETVAL="$?" 49 | [ "$RETVAL" = 2 ] && return 2 50 | start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON 51 | [ "$?" = 2 ] && return 2 52 | rm -f $PIDFILE 53 | return "$RETVAL" 54 | } 55 | 56 | case "$1" in 57 | start) 58 | [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" 59 | do_start 60 | case "$?" in 61 | 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 62 | 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; 63 | esac 64 | ;; 65 | stop) 66 | [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" 67 | do_stop 68 | case "$?" in 69 | 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 70 | 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; 71 | esac 72 | ;; 73 | status) 74 | status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $? 75 | ;; 76 | restart|force-reload) 77 | log_daemon_msg "Restarting $DESC" "$NAME" 78 | do_stop 79 | case "$?" in 80 | 0|1) 81 | do_start 82 | case "$?" in 83 | 0) log_end_msg 0 ;; 84 | 1) log_end_msg 1 ;; # Old process is still running 85 | *) log_end_msg 1 ;; # Failed to start 86 | esac 87 | ;; 88 | *) 89 | # Failed to stop 90 | log_end_msg 1 91 | ;; 92 | esac 93 | ;; 94 | *) 95 | echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2 96 | exit 3 97 | ;; 98 | esac 99 | 100 | : -------------------------------------------------------------------------------- /files/logstash-forwarder.rpm.init: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # chkconfig: 345 80 20 3 | # description: Logstash Forwarder 4 | # processname: logstash-forwarder 5 | # config: /etc/logstash-forwarder 6 | # pidfile: /var/run/logstash-forwarder.pid 7 | 8 | # See: http://goo.gl/cz9Lnd 9 | # Once https://github.com/elasticsearch/logstash-forwarder/pull/196 is merged, 10 | # update this role to pull directly from the new .rpm.init file instead of 11 | # copying this file. 12 | 13 | PATH=/sbin:/usr/sbin:/bin:/usr/bin 14 | NAME=logstash-forwarder 15 | DAEMON=/opt/logstash-forwarder/bin/logstash-forwarder 16 | PIDFILE=/var/run/$NAME.pid 17 | SCRIPTNAME=/etc/init.d/$NAME 18 | 19 | [ -r /etc/default/$NAME ] && . /etc/default/$NAME 20 | . /etc/init.d/functions 21 | 22 | [ -e /etc/sysconfig/logstash-forwarder ] && . /etc/sysconfig/logstash-forwarder 23 | DAEMON_ARGS="${DAEMON_ARGS:--config /etc/logstash-forwarder -spool-size 100 -log-to-syslog}" 24 | 25 | start() 26 | { 27 | echo -n $"Starting $NAME: " 28 | nohup $DAEMON $DAEMON_ARGS >/dev/null 2>&1 & 29 | RETVAL=$? 30 | PID=$! 31 | echo $PID > $PIDFILE 32 | echo 33 | [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$NAME 34 | } 35 | 36 | stop() 37 | { 38 | echo -n $"Stopping $NAME: " 39 | killproc -p "$PIDFILE" $DAEMON 40 | RETVAL=$? 41 | [ -f "$PIDFILE" ] && rm -f "$PIDFILE" 42 | echo 43 | [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$NAME 44 | } 45 | 46 | restart () { 47 | stop 48 | start 49 | } 50 | 51 | RETVAL=0 52 | 53 | case "$1" in 54 | start) 55 | status -p $PIDFILE >/dev/null 56 | RET=$? 57 | if [ $RET -ne 0 ];then 58 | start 59 | fi 60 | ;; 61 | stop) 62 | stop 63 | ;; 64 | restart|reload|force-reload) 65 | restart 66 | ;; 67 | condrestart) 68 | [ -f /var/lock/subsys/$NAME ] && restart || : 69 | ;; 70 | status) 71 | status -p $PIDFILE 72 | RETVAL=$? 73 | ;; 74 | *) 75 | echo "Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}" 76 | RETVAL=1 77 | esac 78 | 79 | exit $RETVAL 80 | -------------------------------------------------------------------------------- /files/logstashforwarder.repo: -------------------------------------------------------------------------------- 1 | [logstashforwarder] 2 | name=logstashforwarder repository 3 | baseurl=http://packages.elasticsearch.org/logstashforwarder/centos 4 | gpgcheck=1 5 | gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch 6 | enabled=1 7 | -------------------------------------------------------------------------------- /handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: restart logstash-forwarder 3 | service: name=logstash-forwarder state=restarted 4 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: [] 3 | 4 | galaxy_info: 5 | author: geerlingguy 6 | description: Logstash Forwarder for Linux. 7 | company: "Midwestern Mac, LLC" 8 | license: "license (BSD, MIT)" 9 | min_ansible_version: 1.8 10 | platforms: 11 | - name: EL 12 | versions: 13 | - all 14 | - name: Debian 15 | versions: 16 | - all 17 | - name: Ubuntu 18 | versions: 19 | - all 20 | galaxy_tags: 21 | - system 22 | - monitoring 23 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include: setup-RedHat.yml 3 | when: ansible_os_family == 'RedHat' 4 | 5 | - include: setup-Debian.yml 6 | when: ansible_os_family == 'Debian' 7 | 8 | # See: http://goo.gl/cz9Lnd 9 | # - name: Install logstash-forwarder init script. 10 | # get_url: 11 | # url: https://raw.github.com/elasticsearch/logstash-forwarder/master/logstash-forwarder.init 12 | # dest: /etc/init.d/logstash-forwarder 13 | # force: yes 14 | # mode: 0655 15 | 16 | - name: Create logstash-forwarder configuration file. 17 | template: 18 | src: logstash-forwarder.j2 19 | dest: /etc/logstash-forwarder 20 | mode: 0644 21 | notify: restart logstash-forwarder 22 | 23 | - name: Ensure Logstash SSL key pair directory exists. 24 | file: 25 | path: "{{ logstash_ssl_dir }}" 26 | state: directory 27 | 28 | - name: Copy SSL key and cert for logstash-forwarder. 29 | copy: 30 | src: "{{ logstash_forwarder_ssl_certificate_file }}" 31 | dest: "{{ logstash_ssl_dir }}/{{ logstash_forwarder_ssl_certificate_file | basename }}" 32 | mode: 0644 33 | notify: restart logstash-forwarder 34 | 35 | - name: Ensure logstash-forwarder is started and enabled on boot. 36 | service: name=logstash-forwarder state=started enabled=yes 37 | -------------------------------------------------------------------------------- /tasks/setup-Debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add Elasticsearch apt key. 3 | apt_key: 4 | url: http://packages.elasticsearch.org/GPG-KEY-elasticsearch 5 | state: present 6 | 7 | - name: Add logstash-forwarder repository. 8 | apt_repository: 9 | repo: 'deb http://packages.elasticsearch.org/logstashforwarder/debian stable main' 10 | state: present 11 | 12 | - name: Check if logstash-forwarder is already installed. 13 | stat: path=/etc/init.d/logstash-forwarder 14 | register: logstash_forwarder_installed 15 | 16 | - name: Update apt cache if repository just added. 17 | apt: update_cache=yes 18 | when: not logstash_forwarder_installed.stat.exists 19 | 20 | - name: Install logstash-forwarder. 21 | apt: pkg=logstash-forwarder state=present 22 | 23 | # See: http://goo.gl/cz9Lnd 24 | - name: Install logstash-forwarder init script. 25 | copy: 26 | src: logstash-forwarder.deb.init 27 | dest: /etc/init.d/logstash-forwarder 28 | mode: 0755 29 | -------------------------------------------------------------------------------- /tasks/setup-RedHat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add Elasticsearch GPG key. 3 | rpm_key: 4 | key: http://packages.elasticsearch.org/GPG-KEY-elasticsearch 5 | state: present 6 | 7 | - name: Add logstash-forwarder repository. 8 | copy: 9 | src: logstashforwarder.repo 10 | dest: /etc/yum.repos.d/logstashforwarder.repo 11 | mode: 0644 12 | 13 | - name: Install logstash-forwarder. 14 | yum: pkg=logstash-forwarder state=present 15 | 16 | # See: http://goo.gl/cz9Lnd 17 | - name: Install logstash-forwarder init script. 18 | copy: 19 | src: logstash-forwarder.rpm.init 20 | dest: /etc/init.d/logstash-forwarder 21 | mode: 0655 22 | -------------------------------------------------------------------------------- /templates/logstash-forwarder.j2: -------------------------------------------------------------------------------- 1 | { 2 | "network": { 3 | "servers": [ "{{ logstash_forwarder_logstash_server }}:{{ logstash_forwarder_logstash_server_port }}" ], 4 | "timeout": 15, 5 | "ssl ca": "{{ logstash_ssl_dir }}/{{ logstash_forwarder_ssl_certificate_file | basename }}" 6 | }, 7 | "files": {{ logstash_forwarder_files | to_json }} 8 | } 9 | -------------------------------------------------------------------------------- /tests/inventory: -------------------------------------------------------------------------------- 1 | localhost 2 | -------------------------------------------------------------------------------- /tests/test.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - hosts: localhost 3 | remote_user: root 4 | roles: 5 | - ansible-role-logstash-forwarder 6 | --------------------------------------------------------------------------------