├── ChangeLog.txt ├── README.md ├── beholder └── beholder-1.08.000.sh.old /ChangeLog.txt: -------------------------------------------------------------------------------- 1 | Change Log 2 | 3 | ------------------ 4 | Beholder V1.10.001 5 | ------------------ 6 | 7 | Changes 8 | 9 | * bro_http fields fixed in the logstash config file. 10 | 11 | ------------------ 12 | Beholder V1.10.000 13 | ------------------ 14 | Current Versions 15 | 16 | * ElasticSearch: 5.4 - Upgraded 17 | * Logstash: 5.4 - Upgraded 18 | * Kibana: 5.4 - Upgraded 19 | * Curator - Added 20 | 21 | Changes 22 | 23 | * Upgraded to new ELK version. 24 | * Added Curator functionality after the 5.x upgrades. 25 | * Bro indices older than 30 days are now closed once a day via cron. 26 | 27 | ------------------ 28 | Beholder V1.09.000 29 | ------------------ 30 | Current Versions 31 | 32 | * ElasticSearch: 5.0 - Upgraded 33 | * Logstash: 5.0 - Upgraded 34 | * Kibana: 5.0 - Upgraded 35 | * Bro: 2.5 - Upgraded 36 | 37 | Changes 38 | 39 | * Upgraded to be used on Ubuntu 16! 40 | * Changed autostart functionality from upstart to systemd. 41 | * ELK and Bro upgraded to the newest version. 42 | * File is now executable. Install should be done as sudo ./beholder. 43 | * Now Proxy capable! 44 | * All input requirements are now upfront. 45 | * You can now select a different IP/interface between Bro and the ELK bound IP. 46 | * Curator is no longer auto-configured for closing out elk indexes. 47 | * Removed some of the app clutter from app-get install. 48 | 49 | ------------------ 50 | Beholder V1.08.000 51 | ------------------ 52 | Current Versions 53 | 54 | * ElasticSearch: 2.3.2 - Upgraded 55 | * Logstash: 2.3.2 - Upgraded 56 | * Kibana: 4.5.0 - Upgraded 57 | * Bro: 2.4.1 58 | 59 | Feature Changes 60 | 61 | * Added 30 other log sources from Bro which feeds into the ELK stack. This totals up to about 37. 62 | * A lot of fields have been normalized. Any "bytes" field has been changed to "bytes_(x)" to make it easier to poke through. 63 | 64 | Fixes 65 | 66 | * Changed the filter from Logstash which should remove any previously failed parses. 67 | 68 | ------------------ 69 | Beholder V1.07.000 70 | ------------------ 71 | Current Versions 72 | 73 | * ElasticSearch: 2.2.1 74 | * Logstash: 2.2.2 75 | * Kibana: 4.4.2 76 | * Bro: 2.4.1 77 | 78 | Feature Changes 79 | 80 | * Script now contains Apache and Kibana is served up via https. No longer required to hit 5601. 81 | * Apache sets itself up with a self-signed certificate. 82 | * Basic Auth now implemented to access Kibana. 83 | * UFW turned on with default allow and blocking kibana from external. 84 | * Special Message added to the script. 85 | 86 | Fixes 87 | 88 | * Beholder no longer added to sudo. 89 | * Bit of changes to the script so that there is less interaction required from the user. 90 | * User and Pass beholder/beholder for both basic auth and Linux user. 91 | 92 | ------------------ 93 | Beholder V1.06.000 94 | ------------------ 95 | Current Versions 96 | 97 | * ElasticSearch: 2.2.1 - Updated 98 | * Logstash: 2.2.2 - Updated 99 | * Kibana: 4.4.2 - Updated 100 | * Bro: 2.4.1 101 | 102 | Feature Changes 103 | 104 | NA 105 | 106 | Fixes 107 | 108 | * Added beholder HOME directory. 109 | * Tossed the ELK stack into a dropbox location as there tends to be frequent updates and this will allow the script to always be relevant. 110 | 111 | ------------------ 112 | Beholder V1.05.000 113 | ------------------ 114 | Current Versions 115 | 116 | * ElasticSearch: 2.1.1 - Updated 117 | * Logstash: 2.1.1 - Updated 118 | * Kibana: 4.3.1 - Updated 119 | * Bro: 2.4.1 120 | 121 | Feature Changes 122 | 123 | NA 124 | 125 | ------------------ 126 | Beholder V1.03.001 127 | ------------------ 128 | Current Versions 129 | 130 | * ElasticSearch: 1.7.2 131 | * Logstash: 1.5.4 132 | * Kibana: 4.1.2 133 | * Bro: 2.4.1 134 | 135 | Feature Changes 136 | 137 | * Ubuntu 15 is now supported. 138 | * Script will check your Ubuntu version and install necessary extra packages to get init running. 139 | * INIT 140 | * "Stoping" is now "stopping"... Yay! 141 | 142 | ------------------ 143 | Beholder V1.03.000 144 | ------------------ 145 | Current Versions 146 | 147 | * ElasticSearch: 1.7.2 - Updated 148 | * Logstash: 1.5.4 149 | * Kibana: 4.1.2 - Updated 150 | * Bro: 2.4.1 - Updated 151 | 152 | ------------------ 153 | Beholder V1.02.000 154 | ------------------ 155 | Current Versions 156 | 157 | * ElasticSearch: 1.7.1 - Updated 158 | * Logstash: 1.5.4 - Updated 159 | * Kibana: 4.1.1 160 | * Bro: 2.4 161 | 162 | Feature Changes 163 | 164 | * Cron 165 | * Fixed an issue with curator running from cron. 166 | 167 | ------------------ 168 | Beholder V1.01.000 169 | ------------------ 170 | Current Versions 171 | 172 | * ElasticSearch: 1.7.0 - Updated 173 | * Logstash: 1.5.3 - Updated 174 | * Kibana: 4.1.1 175 | * Bro: 2.4 176 | 177 | Feature Changes 178 | 179 | * Elasticsearch 180 | * Curator 181 | * Added "--prefix bro" to the cronjob so that custom indices will not be caught. 182 | 183 | ------------------ 184 | Beholder V1.00.001 185 | ------------------ 186 | Current Versions 187 | 188 | * ElasticSearch: 1.6.0 189 | * Logstash: 1.5.2 190 | * Kibana: 4.1.1 191 | * Bro: 2.4 192 | 193 | Feature Changes 194 | 195 | * Elasticsearch 196 | * Added Integer type fields for the duration and bytes fields from Bro. 197 | 198 | * Kibana 199 | * Because of the change in field types, SUM/Min/Max for Kibana dashboards can now properly function against the bytes and duration field. 200 | 201 | Bug Fix 202 | 203 | * Bro 204 | * Unable to compile Git code 205 | * Changed from getting Bro from git to pulling from their website. 206 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Beholder 2 | ``` 3 | Beholder V1.10.001 - ELK/BRO/Libtrace 4 | Created By: Jason Azzarella and Chris Pavan 5 | Problems or Feature Requests? 6 | E-mail Us: jmazzare@bechtel.com 7 | ``` 8 | Beholder is a shell script which installs and configures essentials to peer into your network activity. Monitor your network traffic with Bro IDS, build dashboards with Kibana to get a visual representation of your activity and obtain packet captures of the identified events with Tracesplit. 9 | 10 | ## Under the Hood 11 | 12 | - ELK Stack (https://www.elastic.co) 13 | - Elasticsearch 14 | - Curator 15 | - Logstash 16 | - Kibana 17 | - Bro IDS (https://www.bro.org/) 18 | - Libtrace (http://research.wand.net.nz/software/libtrace.php) 19 | 20 | ## Software Requirements 21 | 22 | - Ubuntu 16.04 x64 23 | 24 | ## Hardware Minimum Requirements 25 | 26 | - 64 Bit Processor 27 | - 3 GB RAM 28 | - 40 GB HDD 29 | 30 | ## Installation 31 | 32 | - Run the beholder script. Use sudo ./beholder. 33 | - Select the interface and IP address for monitoring and management. 34 | - At completion, the system will countdown and reboot. 35 | - Keep watch for the special message! 36 | 37 | ## Details 38 | 39 | - Linux User beholder 40 | - Pass beholder 41 | - Basic Auth User beholder 42 | - Pass beholder 43 | - Kibana Interface - https://{Your Kibana IP} 44 | 1. Use a web browser to access your Kibana instance. 45 | 2. On your first load, you will be required to insert the elasticsearch index. 46 | 3. Input bro* and select the @timestamp field from the timestamp dropdown. 47 | 4. You will be presented with the index fields that have been identified. Use the yellow refresh button in order to update the listed fields. You may need to hit this a few times on first setup. 48 | 5. You should get over 200 fields after all have been populated at least once. 49 | 50 | - Tracesplit Example 51 | - Capture data from interface eth0 - sudo /opt/libtrace/bin/tracesplit -z 6 -Z gzip int:eth0 erf:/pcaps/capture.gz 52 | 53 | ## What's in a Version Name? 54 | 55 | Version numbers get confusing so I'm adding an explanation as to what matters for versions of this script. 56 | 57 | EX 1.00.001 58 | 59 | - The first set of digits (1) represents major tool additions and new functionality. 60 | - The second set of digits (00) represents upgraded versions of tools such as Bro, Elasticsearch, Logstash. 61 | - The third set of digits (001) represents tweaks to the scripts, changes to the templates or all around fixes to the code. 62 | -------------------------------------------------------------------------------- /beholder: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | ############################################ 3 | #Beholder V1.10.001 - ELK/BRO/Libtrace 4 | #Created By: Jason Azzarella and Chris Pavan 5 | #Problems or Feature Requests? 6 | #E-mail Us: jmazzare@bechtel.com 7 | ############################################ 8 | ############## 9 | #Checking Sudo 10 | ############## 11 | clear 12 | dt() { 13 | date -u '+%m/%d/%Y %H:%M:%S' 14 | } 15 | rootcheck() { 16 | if [ $(id -u) != "0" ]; then 17 | echo -e "$(dt) Elevating your privileges..." 18 | sudo "sh" "$0" "$@" 19 | exit $? 20 | fi 21 | } 22 | rootcheck 23 | ######################## 24 | #Declaring URL Variables 25 | ######################## 26 | elasticsearchURL="https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.0.tar.gz" 27 | logstashURL="https://artifacts.elastic.co/downloads/logstash/logstash-5.4.0.tar.gz" 28 | kibanaURL="https://artifacts.elastic.co/downloads/kibana/kibana-5.4.0-linux-x86_64.tar.gz" 29 | broURL="https://www.bro.org/downloads/bro-2.5.tar.gz" 30 | libtraceURL="http://research.wand.net.nz/software/libtrace/libtrace-latest.tar.bz2" 31 | libtraceWandURL="http://research.wand.net.nz/software/wandio/wandio-1.0.4.tar.gz" 32 | ######################## 33 | #Declaring BRO Variables 34 | ######################## 35 | clear 36 | read -r -a broINT <<< $(awk 'NR>2 {print $1}' /proc/net/dev | sed 's/://g') 37 | broINTcount=$(echo ${broINT[@]} | wc | awk {'print $2'}) 38 | broinput=$(whiptail --title "Interface List" --radiolist "Select the interface you want the IDS to monitor." 20 80 $broINTcount "${broINT[0]}" "" ON "${broINT[1]}" "" OFF "${broINT[2]}" "" OFF "${broINT[3]}" "" OFF "${broINT[4]}" "" OFF "${broINT[5]}" "" OFF "${broINT[6]}" "" OFF 3>&1 1>&2 2>&3) 39 | ######################## 40 | #Declaring ELK Variables 41 | ######################## 42 | clear 43 | read -r -a elkIP <<< $(ifconfig | grep inet | grep 'Mask\|netmask' | awk {'print $2'} | sed 's/addr://g') 44 | elkIPcount=$(echo ${elkIP[@]} | wc | awk {'print $2'}) 45 | elkinput=$(whiptail --title "IP List" --radiolist "Select your Kibana management/search IP." 20 80 $elkIPcount "${elkIP[0]}" "" ON "${elkIP[1]}" "" OFF "${elkIP[2]}" "" OFF "${elkIP[3]}" "" OFF "${elkIP[4]}" "" OFF "${elkIP[5]}" "" OFF "${elkIP[6]}" "" OFF 3>&1 1>&2 2>&3) 46 | ################## 47 | #Identifying Proxy 48 | ################## 49 | clear 50 | if (whiptail --title "Proxy Check" --yesno "Do you have a proxy?" 10 60) then 51 | proxySetup=$(whiptail --inputbox "e.g. http://User:Password@AddYourDomainOrIP:Port/" 8 78 "" --title "Please enter your proxy information." 3>&1 1>&2 2>&3) 52 | cat < /etc/apt/apt.conf 53 | Acquire::http::Proxy "$proxySetup"; 54 | EOF 55 | sed -i "s~#https_proxy =.*~https_proxy=$proxySetup~" /etc/wgetrc 56 | sed -i "s~#http_proxy =.*~http_proxy=$proxySetup~" /etc/wgetrc 57 | sed -i "s~#use_proxy =.*~use_proxy = on~" /etc/wgetrc 58 | echo "Fixing proxy stuff!" 59 | else 60 | echo "No proxy selected." 61 | fi 62 | ##################### 63 | #Check Ubuntu Version 64 | ##################### 65 | echo "[+] Ubuntu Version Check." 66 | apt-get install -y lsb-core 67 | version=$(lsb_release -a | grep Release | awk '{print $2}' | sed 's/\..*//') 68 | systemUpdate() { 69 | if [ $version = "16" ]; then 70 | echo "You are on Ubuntu:" $version 71 | echo "Your Ubuntu version is supported. Installing requirements." 72 | echo "[+] Starting download and install. This WILL take a while. Be cool!" 73 | wget -qO - https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add - 74 | cat < /etc/apt/sources.list.d/curator.list 75 | deb http://packages.elasticsearch.org/curator/5/debian stable main 76 | EOF 77 | apt-get update 78 | apt-get install -y default-jre apache2 apache2-utils unzip lsb-core cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev git dh-autoreconf elasticsearch-curator 79 | else 80 | echo "Beholder does not support version" $version 81 | echo "Exiting Beholder." 82 | exit 83 | fi 84 | } 85 | systemUpdate 86 | clear 87 | ####################### 88 | #Creating Beholder User 89 | ####################### 90 | echo "[+] Creating beholder user." 91 | useradd beholder -m -d /home/beholder 92 | echo 'beholder:beholder' | chpasswd 93 | ##################### 94 | #Build File Structure 95 | ##################### 96 | echo "[+] Setting up the file system." 97 | mkdir /srv/logs 98 | mkdir /srv/logs/bro 99 | mkdir /srv/logs/elasticsearch 100 | mkdir /srv/logs/index 101 | mkdir /srv/logs/bro/spool 102 | mkdir /srv/logs/logstash 103 | mkdir /srv/pcaps/ 104 | mkdir /home/beholder 105 | chown beholder:beholder /home/beholder 106 | ####################### 107 | #Installing - ELK Stack 108 | ####################### 109 | echo "[+] Installing ELK Stack" 110 | cd /opt/ 111 | wget $elasticsearchURL 112 | tar -zxvf *.tar.gz && rm -rf *.tar.gz && mv elasticsearch-5.4.0 elasticsearch 113 | wget $logstashURL 114 | tar -zxvf *.tar.gz && rm -rf *.tar.gz && mv logstash* logstash 115 | wget $kibanaURL 116 | tar -zxvf *.tar.gz && rm -rf *.tar.gz && mv kibana-* kibana 117 | ################# 118 | #Installing - Bro 119 | ################# 120 | echo "[+] Installing Bro" 121 | cd /opt/ 122 | mkdir /opt/broinstall && mkdir /opt/bro 123 | cd /opt/broinstall 124 | wget --no-check-certificate $broURL 125 | tar -zxvf bro* 126 | cd /opt/broinstall/bro-2.5 127 | ./configure --prefix=/opt/bro 128 | make 129 | make install 130 | rm -rf /opt/broinst* 131 | ###################### 132 | #Installing - Libtrace 133 | ###################### 134 | echo "[+] Installing Libtrace" 135 | cd /opt/ 136 | wget $libtraceWandURL 137 | tar -zxvf wand* && rm -rf *.tar.gz && mv wand* /opt/wandinstall 138 | cd /opt/wandinstall 139 | ./configure && make && make install 140 | cd /opt/ 141 | rm -rf /opt/wandinstall 142 | wget $libtraceURL 143 | tar jxf libtrace-latest.tar.bz2 && rm -rf libtrace-latest.tar.bz2 && mv libtrace-* libtraceinstall 144 | cd /opt/libtraceinstall 145 | ./configure prefix=/opt/libtrace && make && make install 146 | rm -rf /opt/libtraceinstall 147 | cp /usr/local/lib /etc/ld.so.conf 148 | ldconfig 149 | ############## 150 | #Configuration 151 | ############## 152 | clear 153 | echo "[+] Beginning Configurations" 154 | ############################### 155 | #Configuration - Bro Node Setup 156 | ############################### 157 | sed -i "s/interface=.*/interface=$broinput/" /opt/bro/etc/node.cfg 158 | ############################## 159 | #Configuration - Bro Logs Path 160 | ############################## 161 | sed -i 's/LogDir\s=.*/LogDir = \/srv\/logs\/bro/' /opt/bro/etc/broctl.cfg 162 | sed -i 's/SpoolDir\s=.*/SpoolDir = \/srv\/logs\/bro\/spool/' /opt/bro/etc/broctl.cfg 163 | ############################ 164 | #Configuration - Bro Install 165 | ############################ 166 | /opt/bro/bin/broctl install 167 | /opt/bro/bin/broctl deploy 168 | ############################## 169 | #Configuration - Elasticsearch 170 | ############################## 171 | cd /opt/elasticsearch/config 172 | cat < elasticsearch.yml 173 | cluster.name: beholder 174 | node.name: beholder 175 | node.master: true 176 | node.data: true 177 | node.ingest: false 178 | path.data: /srv/logs/index 179 | path.logs: /srv/logs/elasticsearch 180 | network.host: $elkinput 181 | discovery.zen.ping.unicast.hosts: ["$elkinput"] 182 | EOF 183 | ####################### 184 | #Configuration - Kibana 185 | ####################### 186 | cd /opt/kibana/config 187 | cat < kibana.yml 188 | server.host: "$elkinput" 189 | server.name: "beholder" 190 | elasticsearch.url: "http://$elkinput:9200" 191 | kibana.index: ".kibana" 192 | kibana.defaultAppId: "discover" 193 | EOF 194 | ######################## 195 | #Configuration - Curator 196 | ######################## 197 | mkdir /opt/elasticsearch-curator/config 198 | cd /opt/elasticsearch-curator/config 199 | cat < curator-config.yml 200 | client: 201 | hosts: 202 | - $elkinput 203 | port: 9200 204 | url_prefix: 205 | use_ssl: False 206 | certificate: 207 | client_cert: 208 | client_key: 209 | ssl_no_validate: False 210 | http_auth: 211 | timeout: 30 212 | master_only: False 213 | 214 | logging: 215 | loglevel: INFO 216 | logfile: 217 | logformat: default 218 | blacklist: ['elasticsearch', 'urllib3'] 219 | EOF 220 | ########################## 221 | #Configuration - Curator 2 222 | ########################## 223 | cd /opt/elasticsearch-curator/config 224 | cat < curator.yml 225 | actions: 226 | 1: 227 | action: close 228 | options: 229 | delete_aliases: False 230 | disable_action: False 231 | filters: 232 | - filtertype: pattern 233 | kind: prefix 234 | value: bro- 235 | - filtertype: age 236 | source: name 237 | direction: older 238 | timestring: '%Y.%m.%d' 239 | unit: days 240 | unit_count: 30 241 | EOF 242 | ################################ 243 | #Configuration - Logstash Inputs 244 | ################################ 245 | cd /opt/logstash/config 246 | ##################################### 247 | #Configuration - Logstash ES Template 248 | ##################################### 249 | cat < /opt/logstash/config/bro.json 250 | { 251 | "template": "bro*", 252 | "settings": { 253 | "index.refresh_interval": "5s" 254 | }, 255 | "mappings": { 256 | "_default_": { 257 | "_all": { 258 | "enabled": true 259 | }, 260 | "dynamic_templates": [ 261 | { 262 | "string_fields": { 263 | "match": "*", 264 | "match_mapping_type": "string", 265 | "mapping": { 266 | "type": "string", 267 | "index": "analyzed", 268 | "omit_norms": true, 269 | "fields": { 270 | "raw": { 271 | "type": "string", 272 | "index": "not_analyzed", 273 | "ignore_above": 1024 274 | } 275 | } 276 | } 277 | } 278 | } 279 | ], 280 | "properties": { 281 | "@version": { 282 | "type": "string", 283 | "index": "not_analyzed" 284 | }, 285 | "bytes_seen": { 286 | "type": "integer", 287 | "ignore_malformed": true, 288 | "index": "analyzed" 289 | }, 290 | "bytes_total": { 291 | "type": "integer", 292 | "ignore_malformed": true, 293 | "index": "analyzed" 294 | }, 295 | "bytes_missing": { 296 | "type": "integer", 297 | "ignore_malformed": true, 298 | "index": "analyzed" 299 | }, 300 | "bytes_overflow": { 301 | "type": "integer", 302 | "ignore_malformed": true, 303 | "index": "analyzed" 304 | }, 305 | "bytes_origin": { 306 | "type": "integer", 307 | "ignore_malformed": true, 308 | "index": "analyzed" 309 | }, 310 | "bytes_response": { 311 | "type": "integer", 312 | "ignore_malformed": true, 313 | "index": "analyzed" 314 | }, 315 | "bytes_source_ip": { 316 | "type": "integer", 317 | "ignore_malformed": true, 318 | "index": "analyzed" 319 | }, 320 | "bytes_response_ip": { 321 | "type": "integer", 322 | "ignore_malformed": true, 323 | "index": "analyzed" 324 | }, 325 | "bytes_source": { 326 | "type": "integer", 327 | "ignore_malformed": true, 328 | "index": "analyzed" 329 | } 330 | } 331 | } 332 | } 333 | } 334 | EOF 335 | #################################### 336 | #Configuration - Logstash Bro Parser 337 | #################################### 338 | cat < /opt/logstash/config/bro.conf 339 | input { 340 | file { 341 | path => "/srv/logs/bro/spool/bro/files.log" 342 | type => "bro_files" 343 | sincedb_path => "/srv/logs/logstash/brofiles" 344 | } 345 | file { 346 | path => "/srv/logs/bro/spool/bro/dhcp.log" 347 | type => "bro_dhcp" 348 | sincedb_path => "/srv/logs/logstash/brodhcp" 349 | } 350 | file { 351 | path => "/srv/logs/bro/spool/bro/http.log" 352 | type => "bro_http" 353 | sincedb_path => "/srv/logs/logstash/brohttp" 354 | } 355 | file { 356 | path => "/srv/logs/bro/spool/bro/ssl.log" 357 | type => "bro_ssl" 358 | sincedb_path => "/srv/logs/logstash/brossl" 359 | } 360 | file { 361 | path => "/srv/logs/bro/spool/bro/dns.log" 362 | type => "bro_dns" 363 | sincedb_path => "/srv/logs/logstash/brodns" 364 | } 365 | file { 366 | path => "/srv/logs/bro/spool/bro/conn.log" 367 | type => "bro_conn" 368 | sincedb_path => "/srv/logs/logstash/broconn" 369 | } 370 | file { 371 | path => "/srv/logs/bro/spool/bro/smtp.log" 372 | type => "bro_smtp" 373 | sincedb_path => "/srv/logs/logstash/brosmtp" 374 | } 375 | file { 376 | path => "/srv/logs/bro/spool/bro/known_modbus.log" 377 | type => "bro_known_modbus" 378 | sincedb_path => "/srv/logs/logstash/knownmodbus" 379 | } 380 | file { 381 | path => "/srv/logs/bro/spool/bro/software.log" 382 | type => "bro_software" 383 | sincedb_path => "/srv/logs/logstash/software" 384 | } 385 | file { 386 | path => "/srv/logs/bro/spool/bro/known_certs.log" 387 | type => "bro_known_certs" 388 | sincedb_path => "/srv/logs/logstash/knowncerts" 389 | } 390 | file { 391 | path => "/srv/logs/bro/spool/bro/known_services.log" 392 | type => "bro_known_services" 393 | sincedb_path => "/srv/logs/logstash/knownservices" 394 | } 395 | file { 396 | path => "/srv/logs/bro/spool/bro/known_hosts.log" 397 | type => "bro_known_hosts" 398 | sincedb_path => "/srv/logs/logstash/knownhosts" 399 | } 400 | file { 401 | path => "/srv/logs/bro/spool/bro/x509.log" 402 | type => "bro_x509" 403 | sincedb_path => "/srv/logs/logstash/x509" 404 | } 405 | file { 406 | path => "/srv/logs/bro/spool/bro/pe.log" 407 | type => "bro_pe" 408 | sincedb_path => "/srv/logs/logstash/pe" 409 | } 410 | file { 411 | path => "/srv/logs/bro/spool/bro/known_devices.log" 412 | type => "bro_known_devices" 413 | sincedb_path => "/srv/logs/logstash/knowndevices" 414 | } 415 | file { 416 | path => "/srv/logs/bro/spool/bro/communication.log" 417 | type => "bro_communication" 418 | sincedb_path => "/srv/logs/logstash/communication" 419 | } 420 | file { 421 | path => "/srv/logs/bro/spool/bro/traceroute.log" 422 | type => "bro_traceroute" 423 | sincedb_path => "/srv/logs/logstash/traceroute" 424 | } 425 | file { 426 | path => "/srv/logs/bro/spool/bro/app_stats.log" 427 | type => "bro_app_stats" 428 | sincedb_path => "/srv/logs/logstash/appstats" 429 | } 430 | file { 431 | path => "/srv/logs/bro/spool/bro/dnp3.log" 432 | type => "bro_dnp3" 433 | sincedb_path => "/srv/logs/logstash/dnp3" 434 | } 435 | file { 436 | path => "/srv/logs/bro/spool/bro/intel.log" 437 | type => "bro_intel" 438 | sincedb_path => "/srv/logs/logstash/intel" 439 | } 440 | file { 441 | path => "/srv/logs/bro/spool/bro/modbus.log" 442 | type => "bro_modbus" 443 | sincedb_path => "/srv/logs/logstash/modbus" 444 | } 445 | file { 446 | path => "/srv/logs/bro/spool/bro/modbus_register_change.log" 447 | type => "bro_modbus_register_change" 448 | sincedb_path => "/srv/logs/logstash/modbusregisterchange" 449 | } 450 | file { 451 | path => "/srv/logs/bro/spool/bro/modbus_register_change.log" 452 | type => "bro_modbus_register_change" 453 | sincedb_path => "/srv/logs/logstash/modbusregisterchange" 454 | } 455 | file { 456 | path => "/srv/logs/bro/spool/bro/ftp.log" 457 | type => "bro_ftp" 458 | sincedb_path => "/srv/logs/logstash/ftp" 459 | } 460 | file { 461 | path => "/srv/logs/bro/spool/bro/irc.log" 462 | type => "bro_irc" 463 | sincedb_path => "/srv/logs/logstash/irc" 464 | } 465 | file { 466 | path => "/srv/logs/bro/spool/bro/kerberos.log" 467 | type => "bro_kerberos" 468 | sincedb_path => "/srv/logs/logstash/kerberos" 469 | } 470 | file { 471 | path => "/srv/logs/bro/spool/bro/mysql.log" 472 | type => "bro_mysql" 473 | sincedb_path => "/srv/logs/logstash/mysql" 474 | } 475 | file { 476 | path => "/srv/logs/bro/spool/bro/notice.log" 477 | type => "bro_notice" 478 | sincedb_path => "/srv/logs/logstash/notice" 479 | } 480 | file { 481 | path => "/srv/logs/bro/spool/bro/radius.log" 482 | type => "bro_radius" 483 | sincedb_path => "/srv/logs/logstash/radius" 484 | } 485 | file { 486 | path => "/srv/logs/bro/spool/bro/rdp.log" 487 | type => "bro_rdp" 488 | sincedb_path => "/srv/logs/logstash/rdp" 489 | } 490 | file { 491 | path => "/srv/logs/bro/spool/bro/sip.log" 492 | type => "bro_sip" 493 | sincedb_path => "/srv/logs/logstash/sip" 494 | } 495 | file { 496 | path => "/srv/logs/bro/spool/bro/snmp.log" 497 | type => "bro_snmp" 498 | sincedb_path => "/srv/logs/logstash/snmp" 499 | } 500 | file { 501 | path => "/srv/logs/bro/spool/bro/socks.log" 502 | type => "bro_socks" 503 | sincedb_path => "/srv/logs/logstash/socks" 504 | } 505 | file { 506 | path => "/srv/logs/bro/spool/bro/ssh.log" 507 | type => "bro_ssh" 508 | sincedb_path => "/srv/logs/logstash/ssh" 509 | } 510 | file { 511 | path => "/srv/logs/bro/spool/bro/syslog.log" 512 | type => "bro_syslog" 513 | sincedb_path => "/srv/logs/logstash/syslog" 514 | } 515 | file { 516 | path => "/srv/logs/bro/spool/bro/tunnel.log" 517 | type => "bro_tunnel" 518 | sincedb_path => "/srv/logs/logstash/tunnel" 519 | } 520 | file { 521 | path => "/srv/logs/bro/spool/bro/weird.log" 522 | type => "bro_weird" 523 | sincedb_path => "/srv/logs/logstash/weird" 524 | } 525 | file { 526 | path => "/srv/logs/bro/spool/bro/signatures.log" 527 | type => "bro_signatures" 528 | sincedb_path => "/srv/logs/logstash/signatures" 529 | } 530 | file { 531 | path => "/srv/logs/bro/spool/bro/smb_cmd.log" 532 | type => "bro_smb_cmd" 533 | sincedb_path => "/srv/logs/logstash/smb_cmd" 534 | } 535 | file { 536 | path => "/srv/logs/bro/spool/bro/smb_files.log" 537 | type => "bro_smb_files" 538 | sincedb_path => "/srv/logs/logstash/smb_files" 539 | } 540 | file { 541 | path => "/srv/logs/bro/spool/bro/smb_mapping.log" 542 | type => "bro_smb_mapping" 543 | sincedb_path => "/srv/logs/logstash/smb_mapping" 544 | } 545 | } 546 | filter { 547 | if ([message] =~ /^#/) { 548 | drop{} 549 | } 550 | else if [type] == "bro_files" { 551 | csv { 552 | columns => ["time","fuid","transmit","receive","conn_uids","bro_type","depth","analyzers","mime_type","filename","duration","local_orig","is_orig","bytes_seen","bytes_total","bytes_missing","bytes_overflow","timedout","parent_fuid","md5","sha1","sha256","extracted"] 553 | separator => " " 554 | } 555 | } 556 | else if [type] == "bro_smb_cmd" { 557 | csv { 558 | columns => ["time","uid","id","command","sub_command","argument","status","rtt","version","username","tree","tree_service","referenced_file","referenced_tree","smb1_offered_dialects","smb2_offered_dialects"] 559 | separator => " " 560 | } 561 | } 562 | else if [type] == "bro_smb_files" { 563 | csv { 564 | columns => ["time","uid","id","fuid","action","path","filename","size","prev_filename","mod_time","fid","uuid"] 565 | separator => " " 566 | } 567 | } 568 | else if [type] == "bro_smb_mapping" { 569 | csv { 570 | columns => ["time","uid","id","path","service","native_file_system","share_type"] 571 | separator => " " 572 | } 573 | } 574 | else if [type] == "bro_dhcp" { 575 | csv { 576 | columns => ["time","bro_id","source","source_port","destination","destination_port","mac","assigned_ip","lease_time","trans_id"] 577 | separator => " " 578 | } 579 | } 580 | else if [type] == "bro_http" { 581 | csv { 582 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","method","host","uri","referrer","bro_version","user_agent","request_body_len","response_body_len","status_code","status_msg","info_code","info_msg","tags","username","password","proxied","orig_fuids","orig_filenames","orig_mime_types","resp_fuids","resp_filenames","resp_mime_types"] 583 | separator => " " 584 | } 585 | } 586 | else if [type] == "bro_ssl" { 587 | csv { 588 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","cipher","curve","server_name","resumed","last_alert","next_protocol","established","cert_chain_fuids","client_cert_chain_fuids","subject","issuer","client_subject","client_issuer","validation_status"] 589 | separator => " " 590 | } 591 | } 592 | else if [type] == "bro_dns" { 593 | csv { 594 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","trans_id","query","qclass","qclass_name","qtype","qtype_name","rcode","rcode_name","AA","TC","RD","RA","Z","answers","TTLs","rejected"] 595 | separator => " " 596 | } 597 | } 598 | else if [type] == "bro_conn" { 599 | csv { 600 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","service","duration","bytes_origin","bytes_response","conn_state","local_orig","local_resp","bytes_missing","history","orig_pkts","bytes_source_ip","resp_pkts","bytes_response_ip","tunnel_parents"] 601 | separator => " " 602 | } 603 | } 604 | else if [type] == "bro_smtp" { 605 | csv { 606 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","helo","mailfrom","rcptto","date","from","to","reply_to","msg_id","in_reply_to","subject","x_originating_ip","first_received","second_received","last_reply","path","user_agent","tls","fuids","is_webmail"] 607 | separator => " " 608 | } 609 | } 610 | else if [type] == "bro_known_modbus" { 611 | csv { 612 | columns => ["time","source","device_type"] 613 | separator => " " 614 | } 615 | } 616 | else if [type] == "bro_software" { 617 | csv { 618 | columns => ["time","source","source_port","software_type","name","version.major","version.minor","version.minor2","version.minor3","version.addl","unparsed_version"] 619 | separator => " " 620 | } 621 | } 622 | else if [type] == "bro_known_certs" { 623 | csv { 624 | columns => ["time","source","source_port","subject","issuer_subject","serial"] 625 | separator => " " 626 | } 627 | } 628 | else if [type] == "bro_known_services" { 629 | csv { 630 | columns => ["time","source","source_port","port_proto","service"] 631 | separator => " " 632 | } 633 | } 634 | else if [type] == "bro_known_hosts" { 635 | csv { 636 | columns => ["time","source"] 637 | separator => " " 638 | } 639 | } 640 | else if [type] == "bro_x509" { 641 | csv { 642 | columns => ["time","source","certificate.version","certificate.serial","certificate.subject","certificate.issuer","certificate.not_valid_before","certificate.not_valid_after","certificate.key_alg","certificate.sig_alg","certificate.key_type","certificate.key_length","certificate.exponent","certificate.curve","san.dns","san.uri","san.email","san.ip","basic_constraints.ca","basic_constraints.path_len"] 643 | separator => " " 644 | } 645 | } 646 | else if [type] == "bro_pe" { 647 | csv { 648 | columns => ["time","source","machine","compile_ts","os","subsystem","is_exe","is_64bit","uses_aslr","uses_dep","uses_code_integrity","uses_seh","has_import_table","has_export_table","has_cert_table","has_debug_data","section_names"] 649 | separator => " " 650 | } 651 | } 652 | else if [type] == "bro_known_devices" { 653 | csv { 654 | columns => ["time","mac","dhcp_host_name"] 655 | separator => " " 656 | } 657 | } 658 | else if [type] == "bro_communication" { 659 | csv { 660 | columns => ["time","peer","source","connected_peer_desc","connected_peer_addr","connected_peer_port","level","bromessage"] 661 | separator => " " 662 | } 663 | } 664 | else if [type] == "bro_traceroute" { 665 | csv { 666 | columns => ["time","src","dst","proto"] 667 | separator => " " 668 | } 669 | } 670 | else if [type] == "bro_app_stats" { 671 | csv { 672 | columns => ["time","ts_delta","app","uniq_hosts","hits","bytes_source"] 673 | separator => " " 674 | } 675 | } 676 | else if [type] == "bro_dnp3" { 677 | csv { 678 | columns => ["time","bro_id","source","fc_request","fc_reply","iin"] 679 | separator => " " 680 | } 681 | } 682 | else if [type] == "bro_intel" { 683 | csv { 684 | columns => ["time","bro_id","source","fuid","file_mime_type","file_desc","seen","sources"] 685 | separator => " " 686 | } 687 | } 688 | else if [type] == "bro_modbus" { 689 | csv { 690 | columns => ["time","bro_id","source","func","exception","track_address"] 691 | separator => " " 692 | } 693 | } 694 | else if [type] == "bro_modbus_register_change" { 695 | csv { 696 | columns => ["time","bro_id","source","register","old_val","new_val","delta"] 697 | separator => " " 698 | } 699 | } 700 | else if [type] == "bro_ftp" { 701 | csv { 702 | columns => ["time","bro_id","source","source_port","destination","destination_port","user","password","command","arg","mime_type","file_size","reply_code","reply_msg","data_channel","cwd","cmdarg","pending_commands","passive","capture_password","fuid","File","unique","ID","last_auth_requested"] 703 | separator => " " 704 | } 705 | } 706 | else if [type] == "bro_irc" { 707 | csv { 708 | columns => ["time","bro_id","source","source_port","destination","destination_port","nick","user","command","value","addl","dcc_file_name","dcc_file_size","dcc_mime_type","fuid"] 709 | separator => " " 710 | } 711 | } 712 | else if [type] == "bro_kerberos" { 713 | csv { 714 | columns => ["time","bro_id","source","source_port","destination","destination_port","request_type","client","service","success","error_code","error_msg","from","till","cipher","forwardable","renewable","logged","client_cert","client_cert_subject","client_cert_fuid","server_cert","server_cert_subject","server_cert_fuid"] 715 | separator => " " 716 | } 717 | } 718 | else if [type] == "bro_mysql" { 719 | csv { 720 | columns => ["time","bro_id","source","source_port","destination","destination_port","cmd","arg","success","rows","response"] 721 | separator => " " 722 | } 723 | } 724 | else if [type] == "bro_notice" { 725 | csv { 726 | columns => ["time","bro_id","source","source_port","destination","destination_port","fuid","file_mime_type","file_desc","proto","note","msg","sub","src","dst","p","n","peer_descr","actions","suppress_for","dropped","remote_location.country_code","remote_location.region","remote_location.city","remote_location.latitude","remote_location.longitude"] 727 | separator => " " 728 | } 729 | } 730 | else if [type] == "bro_radius" { 731 | csv { 732 | columns => ["time","bro_id","source","source_port","destination","destination_port","username","mac","remote_ip","connect_info","result","logged"] 733 | separator => " " 734 | } 735 | } 736 | else if [type] == "bro_rdp" { 737 | csv { 738 | columns => ["time","bro_id","source","source_port","destination","destination_port","cookie","result","security_protocol","keyboard_layout","client_build","client_name","client_dig_product_id","desktop_width","desktop_height","requested_color_depth","cert_type","cert_count","cert_permanent","encryption_level","encryption_method","analyzer_id","done","ssl"] 739 | separator => " " 740 | } 741 | } 742 | else if [type] == "bro_sip" { 743 | csv { 744 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","method","uri","date","request_from","request_to","response_from","response_to","reply_to","call_id","seq","subject","request_path","response_path","user_agent","status_code","status_msg","warning","request_body_len","response_body_len","content_type"] 745 | separator => " " 746 | } 747 | } 748 | else if [type] == "bro_snmp" { 749 | csv { 750 | columns => ["time","bro_id","source","source_port","destination","destination_port","duration","version","community","get_requests","get_bulk_requests","get_responses","set_requests","display_string","up_since"] 751 | separator => " " 752 | } 753 | } 754 | else if [type] == "bro_socks" { 755 | csv { 756 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","user","password","status","request","request_p","bound","bound_p"] 757 | separator => " " 758 | } 759 | } 760 | else if [type] == "bro_ssh" { 761 | csv { 762 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","auth_success","direction","client","server","cipher_alg","mac_alg","compression_alg","kex_alg","host_key_alg","host_key","logged","num_failures","capabilities","remote_location"] 763 | separator => " " 764 | } 765 | } 766 | else if [type] == "bro_syslog" { 767 | csv { 768 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","facility","severity","bromessage"] 769 | separator => " " 770 | } 771 | } 772 | else if [type] == "bro_tunnel" { 773 | csv { 774 | columns => ["time","bro_id","source","source_port","destination","destination_port","tunnel_type","action"] 775 | separator => " " 776 | } 777 | } 778 | else if [type] == "bro_weird" { 779 | csv { 780 | columns => ["time","bro_id","source","source_port","destination","destination_port","name","addl","notice","peer"] 781 | separator => " " 782 | } 783 | } 784 | else if [type] == "bro_signatures" { 785 | csv { 786 | columns => ["time","bro_id","source","source_port","destination","destination_port","note","sig_id","event_msg","sub_msg","sig_count","host_count"] 787 | separator => " " 788 | } 789 | } 790 | } 791 | output { 792 | elasticsearch { 793 | hosts => ["$elkinput:9200"] 794 | index => "bro-%{+YYYY.MM.dd}" 795 | template => "/opt/logstash/config/bro.json" 796 | template_name => "bro*" 797 | } 798 | stdout { 799 | codec => rubydebug 800 | } 801 | } 802 | EOF 803 | ######################## 804 | #Prepping System for ELK 805 | ######################## 806 | cat <> /etc/sysctl.conf 807 | vm.max_map_count=262144 808 | EOF 809 | sed -i 's/.*pam_limits\.so/session required pam_limits\.so/' /etc/pam.d/su 810 | cat <> /etc/security/limits.conf 811 | * - nofile 65536 812 | EOF 813 | ######################### 814 | #Setup Initialize Scripts 815 | ######################### 816 | echo "[+] Setting up systemd" 817 | ################# 818 | #Logstash systemd 819 | ################# 820 | cd /lib/systemd/system/ 821 | cat < logstash.service 822 | [Unit] 823 | Description=Logstash 824 | [Service] 825 | ExecStart=/opt/logstash/bin/logstash -f /opt/logstash/config/bro.conf 826 | User=beholder 827 | LimitNOFILE=256000 828 | LimitMEMLOCK=infinity 829 | [Install] 830 | WantedBy=multi-user.target 831 | EOF 832 | systemctl enable logstash.service 833 | ###################### 834 | #Elasticsearch systemd 835 | ###################### 836 | cd /lib/systemd/system/ 837 | cat < elasticsearch.service 838 | [Unit] 839 | Description=Elasticsearch 840 | [Service] 841 | ExecStart=/opt/elasticsearch/bin/elasticsearch 842 | User=beholder 843 | LimitNOFILE=256000 844 | LimitMEMLOCK=infinity 845 | [Install] 846 | WantedBy=multi-user.target 847 | EOF 848 | systemctl enable elasticsearch.service 849 | ############ 850 | #Kibana Init 851 | ############ 852 | cd /lib/systemd/system/ 853 | cat < kibana.service 854 | [Unit] 855 | Description=Kibana 856 | [Service] 857 | ExecStart=/opt/kibana/bin/kibana 858 | User=beholder 859 | LimitNOFILE=256000 860 | LimitMEMLOCK=infinity 861 | [Install] 862 | WantedBy=multi-user.target 863 | EOF 864 | systemctl enable kibana.service 865 | ################# 866 | #Fixing the Crons 867 | ################# 868 | echo "[+] Fixin your crons!" 869 | cd /opt/ 870 | cat < cron 871 | 0-59/5 * * * * /opt/bro/bin/broctl cron 872 | 0 0 * * * /opt/elasticsearch-curator/curator --config /opt/elasticsearch-curator/config/curator-config.yml /opt/elasticsearch-curator/config/curator.yml > /tmp/closed-index.txt 873 | EOF 874 | crontab cron 875 | rm -rf cron 876 | ###################### 877 | #CHOWNing your system! 878 | ###################### 879 | chown -R beholder:beholder /srv/logs 880 | chown -R beholder:beholder /opt 881 | ###################### 882 | #Clearing Certificates 883 | ###################### 884 | update-ca-certificates -f 885 | ################### 886 | #Apache - a2e Setup 887 | ################### 888 | echo "[+] Setting up Apache." 889 | a2enmod proxy proxy_http ssl 890 | ################################## 891 | #Apache - Creating Basic Auth User 892 | ################################## 893 | htpasswd -cbm /etc/apache2/.htpasswd beholder beholder 894 | ################################### 895 | #Apache - Creating self-signed cert 896 | ################################### 897 | openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=DD/ST=Guarding/L=Caverns/O=beholder/CN=beholder" -keyout /etc/ssl/certs/beholder.key -out /etc/ssl/certs/beholder.crt 898 | ################################ 899 | #Apache - A2E Enable and Disable 900 | ################################ 901 | a2ensite default-ssl 902 | a2dissite 000-default 903 | ######################## 904 | #Apache - Port Listening 905 | ######################## 906 | cat < /etc/apache2/ports.conf 907 | Listen 443 908 | EOF 909 | ######################### 910 | #Apache - Sites-Available 911 | ######################### 912 | cat < /etc/apache2/sites-available/default-ssl.conf 913 | 914 | 915 | ServerAdmin webmaster@localhost 916 | 917 | Order deny,allow 918 | Allow from all 919 | AuthType Basic 920 | AuthName "Access Kibana" 921 | AuthUserFile /etc/apache2/.htpasswd 922 | Require valid-user 923 | 924 | ProxyPass / http://$elkinput:5601/ 925 | ProxyPassReverse / http://$elkinput:5601/ 926 | SSLEngine on 927 | SSLCertificateFile /etc/ssl/certs/beholder.crt 928 | SSLCertificateKeyFile /etc/ssl/certs/beholder.key 929 | 930 | SSLOptions +StdEnvVars 931 | 932 | 933 | SSLOptions +StdEnvVars 934 | 935 | BrowserMatch "MSIE [2-6]" \ 936 | nokeepalive ssl-unclean-shutdown \ 937 | downgrade-1.0 force-response-1.0 938 | # MSIE 7 and newer should be able to use keepalive 939 | BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown 940 | 941 | 942 | EOF 943 | ############# 944 | #Firewall Fix 945 | ############# 946 | echo "[+] Configuring the firewall." 947 | ufw deny 5601 948 | ufw default allow 949 | ufw enable 950 | ######### 951 | #Finished 952 | ######### 953 | clear 954 | echo 'Your installation has finished. We are rebooting your system.' 955 | seconds=10 956 | while [ $seconds -gt 0 ]; 957 | do 958 | echo "$seconds" 959 | sleep 1s 960 | seconds=$(($seconds - 1)) 961 | done 962 | echo 'VGhhbmsgeW91IGZvciB0cnlpbmcgdGhlIEJlaG9sZGVyIHNjcmlwdCEgVGhlIHBhbmNha2VzIGFyZSBub3QgYSBsaWUu' 963 | shutdown -r now 964 | -------------------------------------------------------------------------------- /beholder-1.08.000.sh.old: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | ############################################ 3 | #Beholder V1.08.000 - ELK/BRO/Libtrace 4 | #Created By: Jason Azzarella and Chris Pavan 5 | #Problems or Feature Requests? 6 | #E-mail Us: jmazzare@bechtel.com 7 | ############################################ 8 | clear 9 | dt() { 10 | date -u '+%m/%d/%Y %H:%M:%S' 11 | } 12 | rootcheck() { 13 | if [ $(id -u) != "0" ]; then 14 | echo -e "$(dt) Elevating your privileges..." 15 | sudo "sh" "$0" "$@" 16 | exit $? 17 | fi 18 | } 19 | rootcheck 20 | ##################### 21 | #Check Ubuntu Version 22 | ##################### 23 | echo "[+] Ubuntu Version Check." 24 | apt-get update 25 | apt-get install -y lsb-core 26 | version=$(lsb_release -a | grep Release | awk '{print $2}' | sed 's/\..*//') 27 | versioncheck() { 28 | if [ $version = "15" ]; then 29 | echo "You are on Ubuntu:" $version 30 | echo "Your Ubuntu version is supported. Installing init support." 31 | apt-get install -y upstart-sysv openjdk-8-jre 32 | update-initramfs -u 33 | elif [ $version = "14" ]; then 34 | sudo add-apt-repository ppa:webupd8team/java -y 35 | sudo apt-get update 36 | echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections 37 | sudo apt-get install oracle-java8-installer -y 38 | echo "You are on Ubuntu:" $version 39 | echo "Your Ubuntu version is supported." 40 | else 41 | echo "Beholder does not support version" $version 42 | echo "Exiting Beholder." 43 | exit 44 | fi 45 | } 46 | versioncheck 47 | clear 48 | ####################### 49 | #Creating Beholder User 50 | ####################### 51 | echo "[+] Creating beholder user." 52 | useradd beholder -m -d /home/beholder 53 | echo 'beholder:beholder' | chpasswd 54 | ##################### 55 | #Build File Structure 56 | ##################### 57 | echo "[+] Setting up the file system." 58 | mkdir /logs 59 | mkdir /logs/bro 60 | mkdir /logs/elasticsearch 61 | mkdir /logs/index 62 | mkdir /logs/bro/spool 63 | mkdir /logs/logstash 64 | mkdir /pcaps/ 65 | mkdir /home/beholder 66 | chown beholder:beholder /home/beholder 67 | ##################################### 68 | #Installing Updates and Dependencies. 69 | ##################################### 70 | echo "[+] Starting download and install. This WILL take a while. Be cool!" 71 | wget -qO - https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add - 72 | cd /etc/apt/sources.list.d/ 73 | cat < curator.list 74 | deb http://packages.elasticsearch.org/curator/3/debian stable main 75 | EOF 76 | apt-get update 77 | apt-get install -y apache2 apache2-utils unzip bless lsb-core cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev git dh-autoreconf python-elasticsearch-curator 78 | ##################### 79 | #Installing ELK Stack 80 | ##################### 81 | echo "[+] Installing ELK Stack" 82 | cd /opt/ 83 | wget https://www.dropbox.com/s/jikilywdcz3sryt/elasticsearch-2.3.2.tar.gz 84 | tar -zxvf *.tar.gz 85 | rm -rf *.tar.gz 86 | mv elastic* elasticsearch 87 | wget https://www.dropbox.com/s/c3ww7odx8dsnl9z/logstash-2.3.2.tar.gz 88 | tar -zxvf *.tar.gz 89 | rm -rf *.tar.gz 90 | mv logstash* logstash 91 | wget https://www.dropbox.com/s/h0bvni3nnpawdcg/kibana-4.5.0-linux-x64.tar.gz 92 | tar -zxvf *.tar.gz 93 | rm -rf *.tar.gz 94 | mv kibana-* kibana 95 | ############### 96 | #Installing Bro 97 | ############### 98 | echo "[+] Installing Bro" 99 | cd /opt/ 100 | mkdir /opt/broinstall 101 | mkdir /opt/bro 102 | cd /opt/broinstall 103 | wget https://www.bro.org/downloads/release/bro-2.4.1.tar.gz 104 | tar -zxvf bro* 105 | cd /opt/broinstall/bro-2.4.1 106 | ./configure --prefix=/opt/bro 107 | make 108 | make install 109 | rm -rf /opt/broinst* 110 | #################### 111 | #Installing Libtrace 112 | #################### 113 | echo "[+] Installing Libtrace" 114 | cd /opt/ 115 | wget http://research.wand.net.nz/software/libtrace/libtrace-latest.tar.bz2 116 | tar jxf libtrace-latest.tar.bz2 117 | rm -rf libtrace-latest.tar.bz2 118 | mv libtrace-* libtrace 119 | cd /opt/libtrace 120 | ./configure 121 | make 122 | make install 123 | ############### 124 | #Configurations 125 | ############### 126 | echo "[+] Beginning Configurations" 127 | ############################### 128 | #Configuration - Bro Node Setup 129 | ############################### 130 | clear 131 | echo "Bro is used to monitor traffic on an interface." 132 | echo "We have identified the following interfaces on your system;" 133 | echo "-----------------------------------------------------------" 134 | for line in $(awk 'NR>2 {print $1}' /proc/net/dev | sed 's/://g'); do 135 | echo "$line" 136 | done 137 | echo "-----------------------------------------------------------" 138 | read -p "Which interface above would you like to monitor: " broinput 139 | sed -i "s/interface=.*/interface=$broinput/" /opt/bro/etc/node.cfg 140 | ############################## 141 | #Configuration - Bro Logs Path 142 | ############################## 143 | sed -i 's/LogDir\s=.*/LogDir = \/logs\/bro/' /opt/bro/etc/broctl.cfg 144 | sed -i 's/SpoolDir\s=.*/SpoolDir = \/logs\/bro\/spool/' /opt/bro/etc/broctl.cfg 145 | ############################ 146 | #Configuration - Bro Install 147 | ############################ 148 | /opt/bro/bin/broctl install 149 | /opt/bro/bin/broctl deploy 150 | ############################## 151 | #Configuration - Elasticsearch 152 | ############################## 153 | cd /opt/elasticsearch/config 154 | cat < elasticsearch.yml 155 | cluster.name: beholder 156 | node.name: beholder 157 | path.data: /logs/index 158 | path.logs: /logs/elasticsearch 159 | EOF 160 | ################################ 161 | #Configuration - Logstash Inputs 162 | ################################ 163 | mkdir /opt/logstash/config 164 | cd /opt/logstash/config 165 | ##################################### 166 | #Configuration - Logstash ES Template 167 | ##################################### 168 | cat < /opt/logstash/config/bro.json 169 | { 170 | "template": "bro*", 171 | "settings": { 172 | "index.refresh_interval": "5s" 173 | }, 174 | "mappings": { 175 | "_default_": { 176 | "_all": { 177 | "enabled": true 178 | }, 179 | "dynamic_templates": [ 180 | { 181 | "string_fields": { 182 | "match": "*", 183 | "match_mapping_type": "string", 184 | "mapping": { 185 | "type": "string", 186 | "index": "analyzed", 187 | "omit_norms": true, 188 | "fields": { 189 | "raw": { 190 | "type": "string", 191 | "index": "not_analyzed", 192 | "ignore_above": 1024 193 | } 194 | } 195 | } 196 | } 197 | } 198 | ], 199 | "properties": { 200 | "@version": { 201 | "type": "string", 202 | "index": "not_analyzed" 203 | }, 204 | "bytes_seen": { 205 | "type": "integer", 206 | "ignore_malformed": true, 207 | "index": "analyzed" 208 | }, 209 | "bytes_total": { 210 | "type": "integer", 211 | "ignore_malformed": true, 212 | "index": "analyzed" 213 | }, 214 | "bytes_missing": { 215 | "type": "integer", 216 | "ignore_malformed": true, 217 | "index": "analyzed" 218 | }, 219 | "bytes_overflow": { 220 | "type": "integer", 221 | "ignore_malformed": true, 222 | "index": "analyzed" 223 | }, 224 | "bytes_origin": { 225 | "type": "integer", 226 | "ignore_malformed": true, 227 | "index": "analyzed" 228 | }, 229 | "bytes_response": { 230 | "type": "integer", 231 | "ignore_malformed": true, 232 | "index": "analyzed" 233 | }, 234 | "bytes_source_ip": { 235 | "type": "integer", 236 | "ignore_malformed": true, 237 | "index": "analyzed" 238 | }, 239 | "bytes_response_ip": { 240 | "type": "integer", 241 | "ignore_malformed": true, 242 | "index": "analyzed" 243 | }, 244 | "bytes_source": { 245 | "type": "integer", 246 | "ignore_malformed": true, 247 | "index": "analyzed" 248 | } 249 | } 250 | } 251 | } 252 | } 253 | EOF 254 | #################################### 255 | #Configuration - Logstash Bro Parser 256 | #################################### 257 | cat < /opt/logstash/config/bro.conf 258 | input { 259 | file { 260 | path => "/logs/bro/spool/bro/files.log" 261 | type => "bro_files" 262 | sincedb_path => "/logs/logstash/brofiles" 263 | } 264 | file { 265 | path => "/logs/bro/spool/bro/dhcp.log" 266 | type => "bro_dhcp" 267 | sincedb_path => "/logs/logstash/brodhcp" 268 | } 269 | file { 270 | path => "/logs/bro/spool/bro/http.log" 271 | type => "bro_http" 272 | sincedb_path => "/logs/logstash/brohttp" 273 | } 274 | file { 275 | path => "/logs/bro/spool/bro/ssl.log" 276 | type => "bro_ssl" 277 | sincedb_path => "/logs/logstash/brossl" 278 | } 279 | file { 280 | path => "/logs/bro/spool/bro/dns.log" 281 | type => "bro_dns" 282 | sincedb_path => "/logs/logstash/brodns" 283 | } 284 | file { 285 | path => "/logs/bro/spool/bro/conn.log" 286 | type => "bro_conn" 287 | sincedb_path => "/logs/logstash/broconn" 288 | } 289 | file { 290 | path => "/logs/bro/spool/bro/smtp.log" 291 | type => "bro_smtp" 292 | sincedb_path => "/logs/logstash/brosmtp" 293 | } 294 | file { 295 | path => "/logs/bro/spool/bro/known_modbus.log" 296 | type => "bro_known_modbus" 297 | sincedb_path => "/logs/logstash/knownmodbus" 298 | } 299 | file { 300 | path => "/logs/bro/spool/bro/software.log" 301 | type => "bro_software" 302 | sincedb_path => "/logs/logstash/software" 303 | } 304 | file { 305 | path => "/logs/bro/spool/bro/known_certs.log" 306 | type => "bro_known_certs" 307 | sincedb_path => "/logs/logstash/knowncerts" 308 | } 309 | file { 310 | path => "/logs/bro/spool/bro/known_services.log" 311 | type => "bro_known_services" 312 | sincedb_path => "/logs/logstash/knownservices" 313 | } 314 | file { 315 | path => "/logs/bro/spool/bro/known_hosts.log" 316 | type => "bro_known_hosts" 317 | sincedb_path => "/logs/logstash/knownhosts" 318 | } 319 | file { 320 | path => "/logs/bro/spool/bro/x509.log" 321 | type => "bro_x509" 322 | sincedb_path => "/logs/logstash/x509" 323 | } 324 | file { 325 | path => "/logs/bro/spool/bro/pe.log" 326 | type => "bro_pe" 327 | sincedb_path => "/logs/logstash/pe" 328 | } 329 | file { 330 | path => "/logs/bro/spool/bro/known_devices.log" 331 | type => "bro_known_devices" 332 | sincedb_path => "/logs/logstash/knowndevices" 333 | } 334 | file { 335 | path => "/logs/bro/spool/bro/communication.log" 336 | type => "bro_communication" 337 | sincedb_path => "/logs/logstash/communication" 338 | } 339 | file { 340 | path => "/logs/bro/spool/bro/traceroute.log" 341 | type => "bro_traceroute" 342 | sincedb_path => "/logs/logstash/traceroute" 343 | } 344 | file { 345 | path => "/logs/bro/spool/bro/app_stats.log" 346 | type => "bro_app_stats" 347 | sincedb_path => "/logs/logstash/appstats" 348 | } 349 | file { 350 | path => "/logs/bro/spool/bro/dnp3.log" 351 | type => "bro_dnp3" 352 | sincedb_path => "/logs/logstash/dnp3" 353 | } 354 | file { 355 | path => "/logs/bro/spool/bro/intel.log" 356 | type => "bro_intel" 357 | sincedb_path => "/logs/logstash/intel" 358 | } 359 | file { 360 | path => "/logs/bro/spool/bro/modbus.log" 361 | type => "bro_modbus" 362 | sincedb_path => "/logs/logstash/modbus" 363 | } 364 | file { 365 | path => "/logs/bro/spool/bro/modbus_register_change.log" 366 | type => "bro_modbus_register_change" 367 | sincedb_path => "/logs/logstash/modbusregisterchange" 368 | } 369 | file { 370 | path => "/logs/bro/spool/bro/modbus_register_change.log" 371 | type => "bro_modbus_register_change" 372 | sincedb_path => "/logs/logstash/modbusregisterchange" 373 | } 374 | file { 375 | path => "/logs/bro/spool/bro/ftp.log" 376 | type => "bro_ftp" 377 | sincedb_path => "/logs/logstash/ftp" 378 | } 379 | file { 380 | path => "/logs/bro/spool/bro/irc.log" 381 | type => "bro_irc" 382 | sincedb_path => "/logs/logstash/irc" 383 | } 384 | file { 385 | path => "/logs/bro/spool/bro/kerberos.log" 386 | type => "bro_kerberos" 387 | sincedb_path => "/logs/logstash/kerberos" 388 | } 389 | file { 390 | path => "/logs/bro/spool/bro/mysql.log" 391 | type => "bro_mysql" 392 | sincedb_path => "/logs/logstash/mysql" 393 | } 394 | file { 395 | path => "/logs/bro/spool/bro/notice.log" 396 | type => "bro_notice" 397 | sincedb_path => "/logs/logstash/notice" 398 | } 399 | file { 400 | path => "/logs/bro/spool/bro/radius.log" 401 | type => "bro_radius" 402 | sincedb_path => "/logs/logstash/radius" 403 | } 404 | file { 405 | path => "/logs/bro/spool/bro/rdp.log" 406 | type => "bro_rdp" 407 | sincedb_path => "/logs/logstash/rdp" 408 | } 409 | file { 410 | path => "/logs/bro/spool/bro/sip.log" 411 | type => "bro_sip" 412 | sincedb_path => "/logs/logstash/sip" 413 | } 414 | file { 415 | path => "/logs/bro/spool/bro/snmp.log" 416 | type => "bro_snmp" 417 | sincedb_path => "/logs/logstash/snmp" 418 | } 419 | file { 420 | path => "/logs/bro/spool/bro/socks.log" 421 | type => "bro_socks" 422 | sincedb_path => "/logs/logstash/socks" 423 | } 424 | file { 425 | path => "/logs/bro/spool/bro/ssh.log" 426 | type => "bro_ssh" 427 | sincedb_path => "/logs/logstash/ssh" 428 | } 429 | file { 430 | path => "/logs/bro/spool/bro/syslog.log" 431 | type => "bro_syslog" 432 | sincedb_path => "/logs/logstash/syslog" 433 | } 434 | file { 435 | path => "/logs/bro/spool/bro/tunnel.log" 436 | type => "bro_tunnel" 437 | sincedb_path => "/logs/logstash/tunnel" 438 | } 439 | file { 440 | path => "/logs/bro/spool/bro/weird.log" 441 | type => "bro_weird" 442 | sincedb_path => "/logs/logstash/weird" 443 | } 444 | file { 445 | path => "/logs/bro/spool/bro/signatures.log" 446 | type => "bro_signatures" 447 | sincedb_path => "/logs/logstash/signatures" 448 | } 449 | } 450 | filter { 451 | if ([message] =~ /^#/) { 452 | drop{} 453 | } 454 | else if [type] == "bro_files" { 455 | csv { 456 | columns => ["time","fuid","transmit","receive","conn_uids","bro_type","depth","analyzers","mime_type","filename","duration","local_orig","is_orig","bytes_seen","bytes_total","bytes_missing","bytes_overflow","timedout","parent_fuid","md5","sha1","sha256","extracted"] 457 | separator => " " 458 | } 459 | } 460 | else if [type] == "bro_dhcp" { 461 | csv { 462 | columns => ["time","bro_id","source","source_port","destination","destination_port","mac","assigned_ip","lease_time","trans_id"] 463 | separator => " " 464 | } 465 | } 466 | else if [type] == "bro_http" { 467 | csv { 468 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","method","host","uri","referrer","user_agent","request_body_len","response_body_len","status_code","status_msg","info_code","info_msg","filename","tags","username","password","proxied","orig_fuids","orig_mime_types","resp_fuids","resp_mime_types"] 469 | separator => " " 470 | } 471 | } 472 | else if [type] == "bro_ssl" { 473 | csv { 474 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","cipher","curve","server_name","resumed","last_alert","next_protocol","established","cert_chain_fuids","client_cert_chain_fuids","subject","issuer","client_subject","client_issuer","validation_status"] 475 | separator => " " 476 | } 477 | } 478 | else if [type] == "bro_dns" { 479 | csv { 480 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","trans_id","query","qclass","qclass_name","qtype","qtype_name","rcode","rcode_name","AA","TC","RD","RA","Z","answers","TTLs","rejected"] 481 | separator => " " 482 | } 483 | } 484 | else if [type] == "bro_conn" { 485 | csv { 486 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","service","duration","bytes_origin","bytes_response","conn_state","local_orig","local_resp","bytes_missing","history","orig_pkts","bytes_source_ip","resp_pkts","bytes_response_ip","tunnel_parents"] 487 | separator => " " 488 | } 489 | } 490 | else if [type] == "bro_smtp" { 491 | csv { 492 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","helo","mailfrom","rcptto","date","from","to","reply_to","msg_id","in_reply_to","subject","x_originating_ip","first_received","second_received","last_reply","path","user_agent","tls","fuids","is_webmail"] 493 | separator => " " 494 | } 495 | } 496 | else if [type] == "bro_known_modbus" { 497 | csv { 498 | columns => ["time","source","device_type"] 499 | separator => " " 500 | } 501 | } 502 | else if [type] == "bro_software" { 503 | csv { 504 | columns => ["time","source","source_port","software_type","name","version.major","version.minor","version.minor2","version.minor3","version.addl","unparsed_version"] 505 | separator => " " 506 | } 507 | } 508 | else if [type] == "bro_known_certs" { 509 | csv { 510 | columns => ["time","source","source_port","subject","issuer_subject","serial"] 511 | separator => " " 512 | } 513 | } 514 | else if [type] == "bro_known_services" { 515 | csv { 516 | columns => ["time","source","source_port","port_proto","service"] 517 | separator => " " 518 | } 519 | } 520 | else if [type] == "bro_known_hosts" { 521 | csv { 522 | columns => ["time","source"] 523 | separator => " " 524 | } 525 | } 526 | else if [type] == "bro_x509" { 527 | csv { 528 | columns => ["time","source","certificate.version","certificate.serial","certificate.subject","certificate.issuer","certificate.not_valid_before","certificate.not_valid_after","certificate.key_alg","certificate.sig_alg","certificate.key_type","certificate.key_length","certificate.exponent","certificate.curve","san.dns","san.uri","san.email","san.ip","basic_constraints.ca","basic_constraints.path_len"] 529 | separator => " " 530 | } 531 | } 532 | else if [type] == "bro_pe" { 533 | csv { 534 | columns => ["time","source","machine","compile_ts","os","subsystem","is_exe","is_64bit","uses_aslr","uses_dep","uses_code_integrity","uses_seh","has_import_table","has_export_table","has_cert_table","has_debug_data","section_names"] 535 | separator => " " 536 | } 537 | } 538 | else if [type] == "bro_known_devices" { 539 | csv { 540 | columns => ["time","mac","dhcp_host_name"] 541 | separator => " " 542 | } 543 | } 544 | else if [type] == "bro_communication" { 545 | csv { 546 | columns => ["time","peer","source","connected_peer_desc","connected_peer_addr","connected_peer_port","level","bromessage"] 547 | separator => " " 548 | } 549 | } 550 | else if [type] == "bro_traceroute" { 551 | csv { 552 | columns => ["time","src","dst","proto"] 553 | separator => " " 554 | } 555 | } 556 | else if [type] == "bro_app_stats" { 557 | csv { 558 | columns => ["time","ts_delta","app","uniq_hosts","hits","bytes_source"] 559 | separator => " " 560 | } 561 | } 562 | else if [type] == "bro_dnp3" { 563 | csv { 564 | columns => ["time","bro_id","source","fc_request","fc_reply","iin"] 565 | separator => " " 566 | } 567 | } 568 | else if [type] == "bro_intel" { 569 | csv { 570 | columns => ["time","bro_id","source","fuid","file_mime_type","file_desc","seen","sources"] 571 | separator => " " 572 | } 573 | } 574 | else if [type] == "bro_modbus" { 575 | csv { 576 | columns => ["time","bro_id","source","func","exception","track_address"] 577 | separator => " " 578 | } 579 | } 580 | else if [type] == "bro_modbus_register_change" { 581 | csv { 582 | columns => ["time","bro_id","source","register","old_val","new_val","delta"] 583 | separator => " " 584 | } 585 | } 586 | else if [type] == "bro_ftp" { 587 | csv { 588 | columns => ["time","bro_id","source","source_port","destination","destination_port","user","password","command","arg","mime_type","file_size","reply_code","reply_msg","data_channel","cwd","cmdarg","pending_commands","passive","capture_password","fuid","File","unique","ID","last_auth_requested"] 589 | separator => " " 590 | } 591 | } 592 | else if [type] == "bro_irc" { 593 | csv { 594 | columns => ["time","bro_id","source","source_port","destination","destination_port","nick","user","command","value","addl","dcc_file_name","dcc_file_size","dcc_mime_type","fuid"] 595 | separator => " " 596 | } 597 | } 598 | else if [type] == "bro_kerberos" { 599 | csv { 600 | columns => ["time","bro_id","source","source_port","destination","destination_port","request_type","client","service","success","error_code","error_msg","from","till","cipher","forwardable","renewable","logged","client_cert","client_cert_subject","client_cert_fuid","server_cert","server_cert_subject","server_cert_fuid"] 601 | separator => " " 602 | } 603 | } 604 | else if [type] == "bro_mysql" { 605 | csv { 606 | columns => ["time","bro_id","source","source_port","destination","destination_port","cmd","arg","success","rows","response"] 607 | separator => " " 608 | } 609 | } 610 | else if [type] == "bro_notice" { 611 | csv { 612 | columns => ["time","bro_id","source","source_port","destination","destination_port","fuid","file_mime_type","file_desc","proto","note","msg","sub","src","dst","p","n","peer_descr","actions","suppress_for","dropped","remote_location.country_code","remote_location.region","remote_location.city","remote_location.latitude","remote_location.longitude"] 613 | separator => " " 614 | } 615 | } 616 | else if [type] == "bro_radius" { 617 | csv { 618 | columns => ["time","bro_id","source","source_port","destination","destination_port","username","mac","remote_ip","connect_info","result","logged"] 619 | separator => " " 620 | } 621 | } 622 | else if [type] == "bro_rdp" { 623 | csv { 624 | columns => ["time","bro_id","source","source_port","destination","destination_port","cookie","result","security_protocol","keyboard_layout","client_build","client_name","client_dig_product_id","desktop_width","desktop_height","requested_color_depth","cert_type","cert_count","cert_permanent","encryption_level","encryption_method","analyzer_id","done","ssl"] 625 | separator => " " 626 | } 627 | } 628 | else if [type] == "bro_sip" { 629 | csv { 630 | columns => ["time","bro_id","source","source_port","destination","destination_port","trans_depth","method","uri","date","request_from","request_to","response_from","response_to","reply_to","call_id","seq","subject","request_path","response_path","user_agent","status_code","status_msg","warning","request_body_len","response_body_len","content_type"] 631 | separator => " " 632 | } 633 | } 634 | else if [type] == "bro_snmp" { 635 | csv { 636 | columns => ["time","bro_id","source","source_port","destination","destination_port","duration","version","community","get_requests","get_bulk_requests","get_responses","set_requests","display_string","up_since"] 637 | separator => " " 638 | } 639 | } 640 | else if [type] == "bro_socks" { 641 | csv { 642 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","user","password","status","request","request_p","bound","bound_p"] 643 | separator => " " 644 | } 645 | } 646 | else if [type] == "bro_ssh" { 647 | csv { 648 | columns => ["time","bro_id","source","source_port","destination","destination_port","version","auth_success","direction","client","server","cipher_alg","mac_alg","compression_alg","kex_alg","host_key_alg","host_key","logged","num_failures","capabilities","remote_location"] 649 | separator => " " 650 | } 651 | } 652 | else if [type] == "bro_syslog" { 653 | csv { 654 | columns => ["time","bro_id","source","source_port","destination","destination_port","proto","facility","severity","bromessage"] 655 | separator => " " 656 | } 657 | } 658 | else if [type] == "bro_tunnel" { 659 | csv { 660 | columns => ["time","bro_id","source","source_port","destination","destination_port","tunnel_type","action"] 661 | separator => " " 662 | } 663 | } 664 | else if [type] == "bro_weird" { 665 | csv { 666 | columns => ["time","bro_id","source","source_port","destination","destination_port","name","addl","notice","peer"] 667 | separator => " " 668 | } 669 | } 670 | else if [type] == "bro_signatures" { 671 | csv { 672 | columns => ["time","bro_id","source","source_port","destination","destination_port","note","sig_id","event_msg","sub_msg","sig_count","host_count"] 673 | separator => " " 674 | } 675 | } 676 | } 677 | output { 678 | elasticsearch { 679 | hosts => ["localhost:9200"] 680 | index => "bro-%{+YYYY.MM.dd}" 681 | template => "/opt/logstash/config/bro.json" 682 | template_name => "bro*" 683 | } 684 | stdout { 685 | codec => rubydebug 686 | } 687 | } 688 | EOF 689 | ######################### 690 | #Setup Initialize Scripts 691 | ######################### 692 | echo "[+] Setting up Init Scripts" 693 | ############## 694 | #Logstash Init 695 | ############## 696 | cd /etc/init.d 697 | cat <<'EOF' > logstash 698 | . /lib/lsb/init-functions 699 | name="logstash" 700 | logstash_bin="-- /opt/logstash/bin/logstash" 701 | logstash_conf="/opt/logstash/config/bro.conf" 702 | logstash_log="/logs/logstash/$name.log" 703 | pid_file="/var/run/$name.pid" 704 | NICE_LEVEL="-n 19" 705 | HOME=/home/beholder 706 | start () { 707 | command="/usr/bin/nice ${NICE_LEVEL} ${logstash_bin} agent -f $logstash_conf --log ${logstash_log} -- web" 708 | 709 | log_daemon_msg "Starting $name" 710 | if start-stop-daemon --start --chuid "beholder" --quiet --oknodo --pidfile "$pid_file" -b -m --exec $command; then 711 | log_end_msg 0 712 | else 713 | log_end_msg 1 714 | fi 715 | } 716 | stop () { 717 | echo "Stopping $name" 718 | start-stop-daemon --stop --quiet --oknodo --pidfile "$pid_file" 719 | echo "$name stopped" 720 | } 721 | 722 | status () { 723 | status_of_proc -p $pid_file "" "$name" 724 | } 725 | case $1 in 726 | start) 727 | if status; then exit 0; fi 728 | start 729 | ;; 730 | stop) 731 | stop 732 | ;; 733 | reload) 734 | stop 735 | start 736 | ;; 737 | restart) 738 | stop 739 | start 740 | ;; 741 | status) 742 | status && exit 0 || exit $? 743 | ;; 744 | *) 745 | echo "Usage: $0 {start|stop|restart|reload|status}" 746 | exit 1 747 | ;; 748 | esac 749 | exit 0 750 | EOF 751 | chmod +x logstash 752 | update-rc.d logstash defaults 753 | ################### 754 | #Elasticsearch Init 755 | ################### 756 | cd /etc/init.d 757 | cat <<'EOF' > elasticsearch 758 | . /lib/lsb/init-functions 759 | name="elasticsearch" 760 | elastic="-- /opt/elasticsearch/bin/elasticsearch" 761 | pid_file="/var/run/$name.pid" 762 | NICE_LEVEL="-n 19" 763 | start () { 764 | command="/usr/bin/nice ${NICE_LEVEL} ${elastic}" 765 | 766 | log_daemon_msg "Starting $mode" "$name" 767 | if start-stop-daemon --start --chuid "beholder" --quiet --oknodo --pidfile "$pid_file" -b -m --exec $command; then 768 | log_end_msg 0 769 | else 770 | log_end_msg 1 771 | fi 772 | } 773 | stop () { 774 | echo "Stopping $name" 775 | start-stop-daemon --stop --quiet --oknodo --pidfile "$pid_file" 776 | echo "$name stopped" 777 | } 778 | 779 | status () { 780 | status_of_proc -p $pid_file "" "$name" 781 | } 782 | case $1 in 783 | start) 784 | if status; then exit 0; fi 785 | start 786 | ;; 787 | stop) 788 | stop 789 | ;; 790 | reload) 791 | stop 792 | start 793 | ;; 794 | restart) 795 | stop 796 | start 797 | ;; 798 | status) 799 | status && exit 0 || exit $? 800 | ;; 801 | *) 802 | echo "Usage: $0 {start|stop|restart|reload|status}" 803 | exit 1 804 | ;; 805 | esac 806 | exit 0 807 | EOF 808 | chmod +x elasticsearch 809 | update-rc.d elasticsearch defaults 810 | ############ 811 | #Kibana Init 812 | ############ 813 | cd /etc/init.d 814 | cat <<'EOF' > kibana 815 | . /lib/lsb/init-functions 816 | name="kibana" 817 | kibana="-- /opt/kibana/bin/kibana" 818 | pid_file="/var/run/$name.pid" 819 | NICE_LEVEL="-n 19" 820 | start () { 821 | command="/usr/bin/nice ${NICE_LEVEL} ${kibana}" 822 | 823 | log_daemon_msg "Starting $mode" "$name" 824 | if start-stop-daemon --start --chuid "beholder" --quiet --oknodo --pidfile "$pid_file" -b -m --exec $command; then 825 | log_end_msg 0 826 | else 827 | log_end_msg 1 828 | fi 829 | } 830 | stop () { 831 | echo "Stopping $name" 832 | start-stop-daemon --stop --quiet --oknodo --pidfile "$pid_file" 833 | echo "$name stopped" 834 | } 835 | 836 | status () { 837 | status_of_proc -p $pid_file "" "$name" 838 | } 839 | case $1 in 840 | start) 841 | if status; then exit 0; fi 842 | start 843 | ;; 844 | stop) 845 | stop 846 | ;; 847 | reload) 848 | stop 849 | start 850 | ;; 851 | restart) 852 | stop 853 | start 854 | ;; 855 | status) 856 | status && exit 0 || exit $? 857 | ;; 858 | *) 859 | echo "Usage: $0 {start|stop|restart|reload|status}" 860 | exit 1 861 | ;; 862 | esac 863 | exit 0 864 | EOF 865 | chmod +x kibana 866 | update-rc.d kibana defaults 867 | ################# 868 | #Fixing the Crons 869 | ################# 870 | echo "[+] Fixin your crons!" 871 | cd /opt/ 872 | cat < cron 873 | 0-59/5 * * * * /opt/bro/bin/broctl cron 874 | 0 0 * * * /usr/local/bin/curator --host localhost --port 9200 close indices --older-than 30 --time-unit days --timestring '%Y.%m.%d' --prefix bro 875 | EOF 876 | crontab cron 877 | rm -rf cron 878 | ###################### 879 | #CHOWNing your system! 880 | ###################### 881 | chown -R beholder:beholder /logs 882 | chown -R beholder:beholder /opt 883 | ###################### 884 | #Clearing Certificates 885 | ###################### 886 | update-ca-certificates -f 887 | ################### 888 | #Apache - a2e Setup 889 | ################### 890 | echo "[+] Setting up Apache." 891 | a2enmod proxy proxy_http ssl 892 | ################################## 893 | #Apache - Creating Basic Auth User 894 | ################################## 895 | htpasswd -cbm /etc/apache2/.htpasswd beholder beholder 896 | ################################### 897 | #Apache - Creating self-signed cert 898 | ################################### 899 | openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=DD/ST=Guarding/L=Caverns/O=beholder/CN=beholder" -keyout /etc/ssl/certs/beholder.key -out /etc/ssl/certs/beholder.crt 900 | ################################ 901 | #Apache - A2E Enable and Disable 902 | ################################ 903 | a2ensite default-ssl 904 | a2dissite 000-default 905 | ######################## 906 | #Apache - Port Listening 907 | ######################## 908 | cat < /etc/apache2/ports.conf 909 | Listen 443 910 | EOF 911 | ######################### 912 | #Apache - Sites-Available 913 | ######################### 914 | cat < /etc/apache2/sites-available/default-ssl.conf 915 | 916 | 917 | ServerAdmin webmaster@localhost 918 | 919 | Order deny,allow 920 | Allow from all 921 | AuthType Basic 922 | AuthName "Access Kibana" 923 | AuthUserFile /etc/apache2/.htpasswd 924 | Require valid-user 925 | 926 | ProxyPass / http://localhost:5601/ 927 | ProxyPassReverse / http://localhost:5601/ 928 | SSLEngine on 929 | SSLCertificateFile /etc/ssl/certs/beholder.crt 930 | SSLCertificateKeyFile /etc/ssl/certs/beholder.key 931 | 932 | SSLOptions +StdEnvVars 933 | 934 | 935 | SSLOptions +StdEnvVars 936 | 937 | BrowserMatch "MSIE [2-6]" \ 938 | nokeepalive ssl-unclean-shutdown \ 939 | downgrade-1.0 force-response-1.0 940 | # MSIE 7 and newer should be able to use keepalive 941 | BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown 942 | 943 | 944 | EOF 945 | ############# 946 | #Firewall Fix 947 | ############# 948 | echo "[+] Configuring the firewall." 949 | ufw deny 5601 950 | ufw default allow 951 | ufw enable 952 | ######### 953 | #Finished 954 | ######### 955 | clear 956 | echo 'Your installation has finished. We are rebooting your system.' 957 | seconds=10 958 | while [ $seconds -gt 0 ]; 959 | do 960 | echo "$seconds" 961 | sleep 1s 962 | seconds=$(($seconds - 1)) 963 | done 964 | echo 'VGhhbmsgeW91IGZvciB0cnlpbmcgdGhlIEJlaG9sZGVyIHNjcmlwdCEgVGhlIHBhbmNha2VzIGFyZSBub3QgYSBsaWUu' 965 | shutdown -r now --------------------------------------------------------------------------------