├── .gitignore ├── LICENSE ├── README.md ├── Vagrantfile ├── bootstrap-final.sh ├── bootstrap-idx.sh ├── bootstrap-mn.sh ├── bootstrap-nix.sh ├── bootstrap-shc.sh ├── configs ├── hosts ├── idx │ └── APL-idxcluster │ │ └── default │ │ ├── app.conf │ │ ├── inputs.conf │ │ ├── server.conf │ │ └── web.conf ├── master │ └── APL-masternode │ │ └── default │ │ ├── app.conf │ │ ├── outputs.conf │ │ ├── server.conf │ │ └── web.conf └── shc │ └── APL-shcnode │ └── default │ ├── app.conf │ ├── server.conf │ └── web.conf ├── init.sh └── splunk-6.2.3-264376-linux-2.6-x86_64.rpm /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 aplura 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Let's use vagrant to auto-provision a Splunk Clustered Environment. 2 | 3 | * 1 Master Node 4 | * 3 Indexers 5 | * 3 Search Heads 6 | 7 | #How-To 8 | 9 | First, install Vagrant and Virtualbox (but this could probably use whatever Vagrant has hooks for). 10 | 11 | This was built with Vagrant 1.7.2 and Virtualbox 4.2.3. 12 | 13 | Simply run init.sh. This takes about 15 minutes on my machine. The virtual machines use an internal network to communicate with each other, but can also communicate to the outside if needed. The subnet is 10.10.50.0/24. You can access each individual machine using its corresponding port based by IP. In other terms, if the machine IP is 10.10.50.130, the local port to access it is 50130. Any number from 50130-5139 is reserved for any other port that may be needed on that machine. These systems are Cent OS 7 (x64) running Splunk 6.2.3. This can be changed by editing the local variables at the top of the Vagrantfile. Each VM can be ssh'd into using the command "vagrant ssh ". In the list below, each is the first bullet point. master, idx1, idx2, etc. 14 | 15 | The license is a Trial License, so License Master will not work until you grant a valid license. 16 | 17 | The Vagrantfile will stand up the entire Indexer Cluster, Search Head Cluster, and Master Node, and configure each as needed. 18 | 19 | The credentials for all systems are: admin/superS3cr3t. 20 | 21 | 22 | * master 23 | * VM IP: 10.10.50.130 24 | * Web: https://127.0.0.1:50130 25 | * Management: https://127.0.0.1:50131 26 | * Functions: Master Node, License Master (Trial license), Deployer. 27 | * idx1 28 | * VM IP: 10.10.50.100 29 | * Web: https://127.0.0.1:50100 30 | * Management: https://127.0.0.1:50101 31 | * Functions: Indexer 32 | * idx2 33 | * VM IP: 10.10.50.110 34 | * Web: https://127.0.0.1:50110 35 | * Management: https://127.0.0.1:50111 36 | * Functions: Indexer 37 | * idx3 38 | * VM IP: 10.10.50.120 39 | * Web: https://127.0.0.1:50120 40 | * Management: https://127.0.0.1:50121 41 | * Functions: Indexer 42 | * shc1 43 | * VM IP: 10.10.50.140 44 | * Web: https://127.0.0.1:50140 45 | * Management: https://127.0.0.1:50141 46 | * Functions: Search Head 47 | * shc1 48 | * VM IP: 10.10.50.150 49 | * Web: https://127.0.0.1:50150 50 | * Management: https://127.0.0.1:50151 51 | * Functions: Search Head 52 | * shc3 53 | * VM IP: 10.10.50.160 54 | * Web: https://127.0.0.1:50160 55 | * Management: https://127.0.0.1:50161 56 | * Functions: Search Head, Captain 57 | 58 | 59 | #REQUIREMENTS: 60 | * vagrant 61 | * virtualbox 62 | 63 | #Run time: 64 | real 14m25.687s 65 | user 0m22.346s 66 | sys 0m10.949s 67 | 68 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | ##### LOCAL VARIABLES ###### 2 | ##### Change these to match the environment you need to stand up. 3 | ##### CAUTION: NOT ALL SPLUNK VERSIONS SUPPORT CLUSTERING COMMANDS - USE AT YOUR OWN RISK 4 | 5 | _SPLUNK_VERSION = ["6.2.3","x86_64","rpm"] 6 | _VAGRANT_BOX = "chef/centos-7.0" 7 | 8 | ### END LOCAL VARIABLES ### 9 | 10 | Vagrant.configure("2") do |config| 11 | config.vm.box = _VAGRANT_BOX 12 | 13 | config.vm.define "master" do |master| 14 | master.vm.hostname = "vagrant-master" 15 | master.vm.network "private_network", ip: "10.10.50.130", 16 | virtualbox__intnet: true 17 | master.vm.network :forwarded_port, host: 50130, guest: 8000 18 | master.vm.network :forwarded_port, host: 50131, guest: 8089 19 | master.vm.network :forwarded_port, host: 50132, guest: 8080 20 | 21 | master.vm.provision "splunk_base", type: "shell" do |s| 22 | s.path = "bootstrap-mn.sh" 23 | s.args = _SPLUNK_VERSION 24 | end 25 | 26 | end 27 | 28 | config.vm.define "idx1" do |idx1| 29 | idx1.vm.hostname = "vagrant-idx1" 30 | idx1.vm.network "private_network", ip: "10.10.50.100", 31 | virtualbox__intnet: true 32 | idx1.vm.network :forwarded_port, host: 50100, guest: 8000 33 | idx1.vm.network :forwarded_port, host: 50101, guest: 8089 34 | 35 | idx1.vm.provision "splunk_base", type: "shell" do |s| 36 | s.path = "bootstrap-idx.sh" 37 | s.args = _SPLUNK_VERSION 38 | end 39 | 40 | end 41 | 42 | config.vm.define "idx2" do |idx2| 43 | idx2.vm.hostname = "vagrant-idx2" 44 | idx2.vm.network "private_network", ip: "10.10.50.110", 45 | virtualbox__intnet: true 46 | idx2.vm.network :forwarded_port, host: 50110, guest: 8000 47 | idx2.vm.network :forwarded_port, host: 50111, guest: 8089 48 | 49 | idx2.vm.provision "splunk_base", type: "shell" do |s| 50 | s.path = "bootstrap-idx.sh" 51 | s.args = _SPLUNK_VERSION 52 | end 53 | 54 | end 55 | 56 | 57 | config.vm.define "idx3" do |idx3| 58 | idx3.vm.hostname = "vagrant-idx3" 59 | idx3.vm.network "private_network", ip: "10.10.50.120", 60 | virtualbox__intnet: true 61 | idx3.vm.network :forwarded_port, host: 50120, guest: 8000 62 | idx3.vm.network :forwarded_port, host: 50121, guest: 8089 63 | 64 | idx3.vm.provision "splunk_base", type: "shell" do |s| 65 | s.path = "bootstrap-idx.sh" 66 | s.args = _SPLUNK_VERSION 67 | end 68 | 69 | end 70 | 71 | config.vm.define "shc1" do |shc1| 72 | shc1.vm.hostname = "vagrant-shc1" 73 | shc1.vm.network "private_network", ip: "10.10.50.140", 74 | virtualbox__intnet: true 75 | shc1.vm.network :forwarded_port, host: 50140, guest: 8000 76 | shc1.vm.network :forwarded_port, host: 50141, guest: 8089 77 | 78 | shc1.vm.provision "splunk_base", type: "shell" do |s| 79 | s.path = "bootstrap-shc.sh" 80 | s.args = _SPLUNK_VERSION 81 | end 82 | 83 | end 84 | 85 | config.vm.define "shc2" do |shc2| 86 | shc2.vm.hostname = "vagrant-shc2" 87 | shc2.vm.network "private_network", ip: "10.10.50.150", 88 | virtualbox__intnet: true 89 | shc2.vm.network :forwarded_port, host: 50150, guest: 8000 90 | shc2.vm.network :forwarded_port, host: 50151, guest: 8089 91 | 92 | shc2.vm.provision "splunk_base", type: "shell" do |s| 93 | s.path = "bootstrap-shc.sh" 94 | s.args = _SPLUNK_VERSION 95 | end 96 | 97 | end 98 | 99 | 100 | config.vm.define "shc3" do |shc3| 101 | shc3.vm.hostname = "vagrant-shc3" 102 | shc3.vm.network "private_network", ip: "10.10.50.160", 103 | virtualbox__intnet: true 104 | shc3.vm.network :forwarded_port, host: 50160, guest: 8000 105 | shc3.vm.network :forwarded_port, host: 50161, guest: 8089 106 | 107 | shc3.vm.provision "splunk_base", type: "shell" do |s| 108 | s.path = "bootstrap-shc.sh" 109 | s.args = _SPLUNK_VERSION 110 | end 111 | 112 | end 113 | end 114 | -------------------------------------------------------------------------------- /bootstrap-final.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | echo "Starting Final Configurations" 3 | echo "$@" > /tmp/final_passed_args 4 | /opt/splunk/bin/splunk edit user admin -password 'superS3cr3t' -roles admin -auth admin:changeme 5 | sudo touch /opt/splunk/etc/.ui_login 6 | sudo chown -R splunk: /opt/splunk 7 | sudo -u splunk splunk restart 8 | -------------------------------------------------------------------------------- /bootstrap-idx.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | #!/usr/bin/env bash 3 | echo "Starting Indexer Configuration" 4 | echo "$@" > /tmp/idx_passed_args 5 | /vagrant/bootstrap-nix.sh "$1" "$2" "$3" 6 | 7 | ## Copy over our Master App 8 | cp -r /vagrant/configs/idx/APL-idxcluster /opt/splunk/etc/apps 9 | 10 | ## Final Configuration Tasks 11 | /vagrant/bootstrap-final.sh 12 | 13 | 14 | -------------------------------------------------------------------------------- /bootstrap-mn.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | echo "Starting Master Node Configuration" 3 | echo "$@" > /tmp/mn_passed_args 4 | /vagrant/bootstrap-nix.sh "$1" "$2" "$3" 5 | 6 | ## Copy over our Master App 7 | cp -r /vagrant/configs/master/APL-masternode /opt/splunk/etc/apps 8 | 9 | ## Final Configuration Tasks 10 | /vagrant/bootstrap-final.sh 11 | 12 | ### HAPROXY CONFIG 13 | echo "Starting Haproxy Configuration" 14 | if [ ! -f /etc/haproxy/haproxy.cfg ]; then 15 | 16 | # Install haproxy 17 | sudo yum install haproxy -y 18 | 19 | # Configure haproxy 20 | cat > /etc/default/haproxy < /etc/haproxy/haproxy.cfg < /etc/rsyslog.d/haproxy.conf <> /etc/rsyslog.conf < /tmp/nix_passed_args 4 | FN="splunk-*$1*-*$2*.$3" 5 | echo "$FN" 6 | EX="rpm" 7 | if [ "$3" == "deb" ]; then 8 | EX="dpkg" 9 | fi 10 | sudo yum install vim-enhanced rsync -y 11 | sudo cat /vagrant/configs/hosts >> /etc/hosts 12 | FILE=$(find /vagrant/ -name "$FN") 13 | echo "$FILE" > /tmp/splunk_provisioned_version 14 | sudo $EX -i $FILE 15 | mkdir /root/.splunk 16 | chown -R splunk: /root/.splunk 17 | chown -R splunk: /opt/splunk 18 | sudo runuser -l splunk -c '/opt/splunk/bin/splunk start --accept-license' 19 | sudo /opt/splunk/bin/splunk enable boot-start -user splunk 20 | ln -s /opt/splunk/bin/splunk /usr/bin/splunk 21 | -------------------------------------------------------------------------------- /bootstrap-shc.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | echo "Starting Search Head Cluster Node Configuration" 3 | echo "$@" > /tmp/shc_passed_args 4 | /vagrant/bootstrap-nix.sh "$1" "$2" "$3" 5 | 6 | ## Copy over our SHC App 7 | cp -r /vagrant/configs/shc/APL-shcnode /opt/splunk/etc/apps 8 | 9 | HOSTNAME=$(hostname) 10 | IPADDR=$(ifconfig -a | grep 10.10.50. | awk -F' ' '{ print $2}') 11 | ## Setup Search Head Clustering Members 12 | echo "Running Search Head Init" 13 | splunk init shcluster-config -auth admin:changeme -mgmt_uri "https://$IPADDR:8089" -replication_port 8090 -conf_deploy_fetch_url https://10.10.50.130:8089 -secret vagrant 14 | splunk restart 15 | 16 | ## ELECT THE CAPTIAN, AYE! 17 | if [ "$HOSTNAME" == "shc3" ]; then 18 | echo "Electing SHC3 as Captain" 19 | splunk bootstrap shcluster-captain -servers_list "10.10.50.140:8089,10.10.50.150:8089,10.10.50.160:8089" -auth admin:changeme 20 | fi 21 | ## Setup Distributed search with the Index Cluster 22 | echo "Setting as part of index cluster" 23 | splunk edit cluster-config -mode searchhead -master_uri https://10.10.50.130:8089 -auth admin:changeme -secret vagrant 24 | 25 | ## Final Configuration Tasks 26 | /vagrant/bootstrap-final.sh 27 | -------------------------------------------------------------------------------- /configs/hosts: -------------------------------------------------------------------------------- 1 | 10.10.50.130 vagrant-master.local.domain 2 | 10.10.50.100 vagrant-idx1.local.domain 3 | 10.10.50.110 vagrant-idx2.local.domain 4 | 10.10.50.120 vagrant-idx3.local.domain 5 | 10.10.50.140 vagrant-shc1.local.domain 6 | 10.10.50.150 vagrant-shc2.local.domain 7 | 10.10.50.160 vagrant-shc3.local.domain 8 | -------------------------------------------------------------------------------- /configs/idx/APL-idxcluster/default/app.conf: -------------------------------------------------------------------------------- 1 | [install] 2 | is_configured = 0 3 | 4 | [ui] 5 | is_visible = 0 6 | label = Aplura Vagrant Indexer 7 | 8 | [launcher] 9 | version = 0.1 10 | description = Aplura, LLC Indexer Configuration for Vagrant 11 | author = Kyle Smith 12 | 13 | [package] 14 | id = APLIndexer 15 | 16 | -------------------------------------------------------------------------------- /configs/idx/APL-idxcluster/default/inputs.conf: -------------------------------------------------------------------------------- 1 | [splunktcp://9997] 2 | -------------------------------------------------------------------------------- /configs/idx/APL-idxcluster/default/server.conf: -------------------------------------------------------------------------------- 1 | [general] 2 | pass4SymmKey = vagrant 3 | 4 | [replication_port://9887] 5 | 6 | [clustering] 7 | mode = slave 8 | pass4SymmKey = vagrant 9 | master_uri = https://10.10.50.130:8089 10 | 11 | [license] 12 | master_uri = https://10.10.50.130:8089 13 | -------------------------------------------------------------------------------- /configs/idx/APL-idxcluster/default/web.conf: -------------------------------------------------------------------------------- 1 | [settings] 2 | enableSplunkWebSSL = true 3 | httpport = 8000 4 | -------------------------------------------------------------------------------- /configs/master/APL-masternode/default/app.conf: -------------------------------------------------------------------------------- 1 | [install] 2 | is_configured = 0 3 | 4 | [ui] 5 | is_visible = 0 6 | label = Aplura Vagrant Master 7 | 8 | [launcher] 9 | version = 0.1 10 | description = Aplura, LLC Master Node Configuration for Vagrant 11 | author = Kyle Smith 12 | 13 | [package] 14 | id = APLMaster 15 | 16 | -------------------------------------------------------------------------------- /configs/master/APL-masternode/default/outputs.conf: -------------------------------------------------------------------------------- 1 | [tcpout] 2 | defaultGroup = vagrant_indexers 3 | 4 | [tcpout:vagrant_indexers] 5 | server = 10.10.50.100:9997,10.10.50.110:9997,10.10.50.120:9997 6 | -------------------------------------------------------------------------------- /configs/master/APL-masternode/default/server.conf: -------------------------------------------------------------------------------- 1 | [general] 2 | pass4SymmKey = vagrant 3 | 4 | [shclustering] 5 | pass4SymmKey = vagrant 6 | 7 | [clustering] 8 | mode = master 9 | replication_factor = 1 10 | search_factor = 1 11 | pass4SymmKey = vagrant 12 | -------------------------------------------------------------------------------- /configs/master/APL-masternode/default/web.conf: -------------------------------------------------------------------------------- 1 | [settings] 2 | enableSplunkWebSSL = true 3 | httpport = 8000 4 | -------------------------------------------------------------------------------- /configs/shc/APL-shcnode/default/app.conf: -------------------------------------------------------------------------------- 1 | [install] 2 | is_configured = 0 3 | 4 | [ui] 5 | is_visible = 0 6 | label = Aplura Vagrant Search Head Cluster 7 | 8 | [launcher] 9 | version = 0.1 10 | description = Aplura, LLC SHC Configuration for Vagrant 11 | author = Kyle Smith 12 | 13 | [package] 14 | id = APLSHC 15 | 16 | -------------------------------------------------------------------------------- /configs/shc/APL-shcnode/default/server.conf: -------------------------------------------------------------------------------- 1 | [general] 2 | pass4SymmKey = vagrant 3 | 4 | [license] 5 | master_uri = https://10.10.50.130:8089 6 | -------------------------------------------------------------------------------- /configs/shc/APL-shcnode/default/web.conf: -------------------------------------------------------------------------------- 1 | [settings] 2 | enableSplunkWebSSL = true 3 | httpport = 8000 4 | -------------------------------------------------------------------------------- /init.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | vagrant up 3 | vagrant ssh master -c 'sudo splunk add search-server -host 10.10.50.140:8089 -auth admin:superS3cr3t -remoteUsername admin -remotePassword superS3cr3t' 4 | vagrant ssh master -c 'sudo splunk add search-server -host 10.10.50.150:8089 -auth admin:superS3cr3t -remoteUsername admin -remotePassword superS3cr3t' 5 | vagrant ssh master -c 'sudo splunk add search-server -host 10.10.50.160:8089 -auth admin:superS3cr3t -remoteUsername admin -remotePassword superS3cr3t' 6 | -------------------------------------------------------------------------------- /splunk-6.2.3-264376-linux-2.6-x86_64.rpm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aplura/Vagrant_Splunk_Cluster/f7e0768838b2b831290c455a2b58858092c2f292/splunk-6.2.3-264376-linux-2.6-x86_64.rpm --------------------------------------------------------------------------------