├── .gitignore ├── CHANGELOG.md ├── LICENSE.md ├── README.md ├── docs ├── TODO.md ├── index.md ├── other.md ├── prepenv.md ├── prepscript.md ├── start.md └── templates.md ├── firestic.py ├── firestic_alert.py ├── fsconfig.py ├── geoip └── README.md ├── mkdocs.yml ├── mustache_templates ├── template.html └── template.txt ├── prep_files ├── firestic_ES_template.sh └── firestic_kibana.json ├── screenshots ├── examplealert_mal_cb.png └── examplealert_mal_obj.png └── testing └── fstest.py /.gitignore: -------------------------------------------------------------------------------- 1 | site/ 2 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | ###Change log for [FireStic](https://github.com/spcampbell/FireStic) 2 | 3 | **Master branch is the latest release** 4 | 5 | ####9/25/2017 6 | - changed time format so it is same for all alerts. Fireeye may have updated code which changed this so they are all the same. Previously, IPS events were different than all other alerts. Now they all match IPS events. 7 | 8 | ####3/9/2015 9 | - Added script in `testing` folder for uploading test json data to FireStic 10 | - Bug fix: malware-data was not being included fully if field was not an array 11 | - New screen shot of email alert added 12 | 13 | ####2/20/15 14 | - If [os-changes] exists, extracting [malicious-alert] from [os-changes] for each OS analyzed during this alert. This is sent both to Elasticsearch and also appended to the bottom of the HTML email notification. Shows the type of malicious activity that FireEye saw. This is very helpful for a quick context on what is going on. 15 | - Color for severity in HTML email notification is now red/orange/yellow for critical/major/minor levels. Mustache template tag added to HTML template in CSS section for color. 16 | - Documentation added that better explain how to handle errors installing premailer. See [Read the Docs](http://firestic.rtfd.org/) 17 | 18 | ####2/17/15 19 | - Bug fixes. Was not handling malicious-alert info correctly. Error has been corrected. 20 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 spcampbell 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## ATTENTION: FireStic is no longer being developed. I am leaving it here for informational purposes. 2 | 3 | ### **FireStic** 4 | 5 | #### A python script that accepts [FireEye](https://www.fireeye.com) alerts as json over http and indexes (puts) the data into [Elasticsearch](http://www.elasticsearch.org)...and notifies you too. 6 | 7 | #### Full documentation is located at: [Read The Docs](http://firestic.rtfd.org) 8 | 9 | > **DISCLAIMER:** This script is still being developed and may contain numerous 10 | > bugs, flaws and discrepencies. It may cause various types of damage and will 11 | > likely make your hair turn grey and fall out. I am not liable for any 12 | > situations which result from the use of this code. There is no stated or 13 | > implied warranty. It may chip, rip, rattle or run down the hill...all 14 | > without warning. Use at your own risk. I am not a professional programmer 15 | > and only offer this code as documentation of my humble attempt at solving a 16 | > problem for my own needs. 17 | 18 | #### The latest version is always in `master`. 19 | #### Last update to master: 9/25/2017 20 | 21 | See [changelog](https://github.com/spcampbell/FireStic/blob/master/CHANGELOG.md) for changes beginning 2/20/15 22 | 23 | #### Highlights 24 | 25 | **Concerning Elasticsearch:** 26 | 27 | - Put FireEye alerts into [Elasticsearch](http://www.elasticsearch.org) 28 | - Handles the variability of FireEye alert structure 29 | - Alternative to using [Logstash](http://logstash.net) which can get quite complex with FireEye alerts 30 | - Accepts alerts formatted in `JSON Extended` over http 31 | - Adds geoip information to alert using [pygeoip](https://github.com/appliedsec/pygeoip) 32 | - Allows for different geoip databases for internal vs. external ip addresses 33 | - Adds hostname information by performing a DNS lookup on both source and destination ip addresses 34 | 35 | **Concerning Notifications:** 36 | 37 | - Send notifications via email and SMS 38 | - Notifications can be turned on/off completely, by alert type, or by FireEye action 39 | - Customizable HTML email messages allow for easy reading and detail highlighting 40 | - SMS message are correctly split at 160 characters 41 | - Uses [mustache](http://mustache.github.io) templates (via [pystache](https://github.com/defunkt/pystache)) for designing notification messages 42 | - Automatically puts CSS inline before sending using [premailer](http://www.peterbe.com/plog/premailer.py) 43 | - Notifications include additional information like hostname and location 44 | 45 | #### Screenshots 46 | 47 | Current screenshots available in the `screenshots` directory: 48 | 49 | - [Malware-object HTML email alert](https://github.com/spcampbell/FireStic/blob/master/screenshots/examplealert_mal_obj.png) 50 | - [Malware-callback HTML email alert](https://github.com/spcampbell/FireStic/blob/master/screenshots/examplealert_mal_cb.png) 51 | 52 | #### License 53 | 54 | The MIT License (MIT) 55 | 56 | Copyright (c) 2014 57 | 58 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 59 | 60 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 61 | 62 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 63 | -------------------------------------------------------------------------------- /docs/TODO.md: -------------------------------------------------------------------------------- 1 | TODO 2 | ---- 3 | 4 | * Code to deal with os-changes data and index into Elasticsearch. Need to determine best relationship strategy (in code vs. nesting vs. parent-child) 5 | * Set up alerting config so type vs action is configurable for each possibility 6 | * Better logging with comprehensive exception handling to logs 7 | * Daemonize 8 | * Easy install via PyPI 9 | * Add option to use SSL 10 | * Add multi-thread capability (or refactor for use with NGINX/Apache) 11 | * Examine possibility of gathering IOCs from os-changes. Would put in STIX or openIOC format. Maybe hand off for scans. 12 | * Research other data gathering options for incorporating in alert. 13 | * Initiate Redline scan with winexe? 14 | * Kick off scan using another tool like Google's GRR? -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | ### **FireStic** 2 | 3 | #### A python script that accepts [FireEye](https://www.fireeye.com) alerts as json over http and indexes (puts) the data into [Elasticsearch](http://www.elasticsearch.org)...and notifies you too. 4 | 5 | > **DISCLAIMER:** This script is still being developed and may contain numerous 6 | > bugs, flaws and discrepencies. It may cause various types of damage and will 7 | > likely make your hair turn grey and fall out. I am not liable for any 8 | > situations which result from the use of this code. There is no stated or 9 | > implied warranty. It may chip, rip, rattle or run down the hill...all 10 | > without warning. Use at your own risk. I am not a professional programmer 11 | > and only offer this code as documentation of my humble attempt at solving a 12 | > problem for my own needs. 13 | 14 | #### The latest version is always in the [master branch on github](https://github.com/spcampbell/FireStic/tree/master). 15 | 16 | See [changelog](https://github.com/spcampbell/FireStic/blob/develop/CHANGELOG.md) for changes beginning 2/20/15 17 | 18 | 19 | #### Highlights 20 | 21 | **Concerning Elasticsearch:** 22 | 23 | - Put FireEye alerts into [Elasticsearch](http://www.elasticsearch.org) 24 | - Handles the variability of FireEye alert structure 25 | - Alternative to using [Logstash](http://logstash.net) which can get quite complex with FireEye alerts 26 | - Accepts alerts formatted in `JSON Extended` over http 27 | - Adds geoip information to alert using [pygeoip](https://github.com/appliedsec/pygeoip) 28 | - Allows for different geoip databases for internal vs. external ip addresses 29 | - Adds hostname information by performing a DNS lookup on both source and destination ip addresses 30 | 31 | **Concerning Notifications:** 32 | 33 | - Send notifications via email and SMS 34 | - Notifications can be turned on/off completely, by alert type, or by FireEye action 35 | - Customizable HTML email messages allow for easy reading and detail highlighting 36 | - SMS message are correctly split at 160 characters 37 | - Uses [mustache](http://mustache.github.io) templates (via [pystache](https://github.com/defunkt/pystache)) for designing notification messages 38 | - Automatically puts CSS inline before sending using [premailer](http://www.peterbe.com/plog/premailer.py) 39 | - Notifications include additional information like hostname and location 40 | 41 | > **NOTE:** Currently, the [os-changes] field is not included in the indexing 42 | > process as it can vary quite a bit and more examples are needed before 43 | > general patterns can be uncovered. Including this data is a top priority as 44 | > this field is a goldmine of information related to an incident. For now, if the 45 | > [os-changes] field is found in the json, the entire [os-changes] tree is saved 46 | > to a file called oschanges.json to use in developing the code. 47 | > 48 | > **UPDATE:** if [os-changes] exists and includes information in [malicious-alert], 49 | > then that data is now appended to the end of the email. There will be a section 50 | > for each OS analyzed. This is key information for understanding what happened. 51 | 52 | #### Documentation Quick Reference 53 | - [Prepare Your Environment](prepenv) 54 | - Server Environment 55 | - Elasticsearch Template 56 | - Configure FireEye to Send Notifications 57 | - [Prepare The Script and Dependencies](prepscript) 58 | - Python Module Dependencies 59 | - Geoip Setup 60 | - FireStic Script Configuration 61 | - firestic.py Settings 62 | - firestic_alert.py Settings 63 | - [Start the Show](start) 64 | - Running the Script 65 | - Send Some Test Alerts 66 | - [Customizing Notification Templates](templates) 67 | - [Other Stuff](other) 68 | - Kibana Template (optional) 69 | - [TODO](TODO) 70 | 71 | #### License 72 | 73 | The MIT License (MIT) 74 | 75 | Copyright (c) 2014 76 | 77 | Permission is hereby granted, free of charge, to any person obtaining a copy 78 | of this software and associated documentation files (the "Software"), to deal 79 | in the Software without restriction, including without limitation the rights 80 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 81 | copies of the Software, and to permit persons to whom the Software is 82 | furnished to do so, subject to the following conditions: 83 | 84 | The above copyright notice and this permission notice shall be included in all 85 | copies or substantial portions of the Software. 86 | 87 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 88 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 89 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 90 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 91 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 92 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 93 | SOFTWARE. 94 | -------------------------------------------------------------------------------- /docs/other.md: -------------------------------------------------------------------------------- 1 | Other Stuff 2 | ----------- 3 | 4 | #### Kibana Demo Template (optional) 5 | 6 | To get you up and running quickly, a dashboard template for Kibana is available in the file `firestic_kibana.json` located in the `prep_files` directory. It's pretty basic and I suspect you'll want to extend this further...or step past Kibana altogether and build your own uber-cool home-grown dashboard website to present the data. 7 | 8 | #### Q: FireStic? Really? Where did you get that stupid name? 9 | 10 | **A:** Because it indexes **FIRE**eye alerts into ela**STIC**search...and I'm not overly creative. 11 | -------------------------------------------------------------------------------- /docs/prepenv.md: -------------------------------------------------------------------------------- 1 | Prepare Your Environment 2 | ----------------------- 3 | 4 | #### Server Environment 5 | 6 | I am currently running this script under the following conditions: 7 | 8 | * [Ubuntu server 14.04.1 LTS](http://www.ubuntu.com/download/server) 9 | * Python 2.7.6 10 | * [Elasticsearch 1.4.x](http://www.elasticsearch.org) 11 | * [FireEye NX Series](http://www.fireeye.com) w/ optional IPS module enabled 12 | 13 | Other versions/variations have not been tested but it should relatively straightforward to work through any dependencies. 14 | 15 | For help setting up Elasticsearch: [here is a good tutorial](https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-and-visualize-logs-on-ubuntu-14-04). 16 | 17 | [elasticsearch.org](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html) also has a [helpful guide](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html) for installing from repositories. 18 | 19 | > **IMPORTANT:** Elasticsearch does not provide any security by default. 20 | > It can be secured but I leave that exercise up to you and Google. 21 | > Consider how you deploy this carefully as it will contain information 22 | > you probably do not want easily accessible and/or tampered with. 23 | 24 | My current test environment is two desktop machines with 4GB of RAM and i5 processors serving a two node cluster. The only change was to increase the heap memory size to 1GB. 25 | 26 | ####Elasticsearch Template 27 | 28 | If you use Logstash to ship logs to Elasticsearch, a template is created automatically that adds an additional .raw sub field to every string field. We need to do that here too otherwise multi-word strings will be broken up and you'll hate dealing with that in Kibana. 29 | 30 | The template is located in `firestic_ES_template.sh` located in the `prep_files` directory. You can run from a shell prompt...or, for your copy/paste convenience, here is the template and curl statement: 31 | 32 | ```Shell 33 | curl -XPUT localhost:9200/_template/firestic_1 -d ' 34 | { 35 | "template" : "firestic-*", 36 | "settings" : { 37 | "number_of_shards" : 5, 38 | "index.refresh_interval" : "5s" 39 | }, 40 | "mappings" : { 41 | "_default_" : { 42 | "_all" : {"enabled" : true}, 43 | "dynamic_templates" : [{ 44 | "string_fields" : { 45 | "match" : "*", 46 | "match_mapping_type" : "string", 47 | "mapping" : { 48 | "type" : "string", "index" : "analyzed", "omit_norms" : true, 49 | "fields" : { 50 | "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256} 51 | } 52 | } 53 | } 54 | }] 55 | } 56 | } 57 | }' 58 | ``` 59 | 60 | #### Configure FireEye to Send Notifications 61 | 62 | * In your FireEye appliance, click on "Settings" then "Notifications" 63 | * click "http" at the top of table 64 | * Add a new http server 65 | * The server url is the ip address or hostname plus port of the machine where the script will run. For example: `http://192.168.1.2:8888` 66 | * Set the following: 67 | * Notification: All Events 68 | * Delivery: Per Event 69 | * Default Provider: Generic 70 | * Message Format: JSON Extended 71 | * Make sure "enabled" is checked and click the "Update" button 72 | -------------------------------------------------------------------------------- /docs/prepscript.md: -------------------------------------------------------------------------------- 1 | Prepare The Script and Dependencies 2 | ----------------------------------- 3 | 4 | #### Python Module Dependencies: 5 | 6 | You will need to install the following python modules: 7 | 8 | - [pygeoip](https://github.com/appliedsec/pygeoip) 9 | - [pytz](http://pytz.sourceforge.net) 10 | - [elasticsearch](http://www.elasticsearch.org/guide/en/elasticsearch/client/python-api/current/) 11 | - [pystache](https://github.com/defunkt/pystache) 12 | - [premailer](http://www.peterbe.com/plog/premailer.py) 13 | 14 | `pip install ` should get you set up. 15 | 16 | (To install pip on Ubuntu, run: `sudo apt-get install python-pip`) 17 | 18 | > NOTE: If [premailer](http://www.peterbe.com/plog/premailer.py) throws errors on install, it is usually due to 19 | > package dependencies missing in the OS related to `lxml`. For Ubuntu, try this: 20 | > 21 | > - Uninstall premailer if you tried to install and got errors: `sudo pip uninstall premailer` 22 | > - Install dependencies : `sudo apt-get install -y libxml2-dev libxslt1-dev zlib1g-dev python2.7-dev` 23 | > - Reinstall premailer: `sudo pip install premailer` 24 | > 25 | > More help here: http://stackoverflow.com/questions/5178416/pip-install-lxml-error 26 | > 27 | 28 | For reference, here are the other modules: 29 | 30 | - smtplib 31 | - email.mime.multipart 32 | - email.mime.text 33 | - json 34 | - datetime 35 | - BaseHTTPServer 36 | - logging 37 | - socket 38 | 39 | #### Geoip Setup 40 | 41 | Download the free GeoLite City and GeoLite ASN databases (the binary versions) from [MaxMind](http://dev.maxmind.com/geoip/legacy/geolite/). 42 | 43 | Place the files in the `geoip` folder. 44 | 45 | The script accomodates different database files for internal vs. external devices. However, **you will need to create the geoip database for your internal addresses and locations**. Here is the best instruction I have found on how to do that: [Generate Local MaxMind Database](https://blog.vladionescu.com/geo-location-for-internal-networks/). It uses [mmutils](https://github.com/mteodoro/mmutils). The article describes adding your internal network locations and private ip addresses to the MaxMind CSV files. However, **you will want to make your own CSV files with only internal networks and locations then compile them to a new .dat for the best result.** The process is basically the same and fairly straightforward once you see how the two .csv files are organizing the data. 46 | 47 | > **TIP:** Do not open the csv files you create in Microsoft Excel as it will 48 | > completely wreck it out and will never compile. Use Open Office or a text 49 | > editor. Make sure the file is of type UNIX and not MAC or Windows. You 50 | > should be able to set this in most decent text editors (Notepad++, 51 | > TextWrangler, etc, etc) 52 | 53 | #### FireStic Script Configuration 54 | 55 | All configuration options are now located in `fsconfig.py`. Please configure the settings below in that file. 56 | 57 | ####firestic.py Settings: 58 | 59 | |setting name|example|description| 60 | |----------|------------|-------------| 61 | |esIndex|`'firestic'`|Elasticsearch index to use. `-YYYY.MM.DD` will be appended ala Logstash| 62 | |extGeoipDatabase|`'geoip/GeoLiteCity.dat'`|Geoip database for external (internet) addresses| 63 | |intGeoipDatabase|`'geoip/YourLocations.dat'`|Geoip database for internal (LAN) addresses. If you want to map these ip addresses to geo coordinates, you'll have to create the file. See the [Geoip Setup section above](#geoip-setup). You can use the same file as `extGeoipDatabase` but it will not resolve internal addresses| 64 | |ASNGeoipDatabase|`'geoip/GeoIPASNum.dat'`|Geoip database for external address ASN info| 65 | |localASN|`'your_org_name'`|ASN for internal ip addresses. Since internal addresses are private, this is used in the ASN field| 66 | |httpServerIP|`'192.168.1.2'`|ip address for http server to listen on| 67 | |httpServerPort|`8888`|Port for http server to listen on| 68 | |logfile|`'firestic_error.log'`|File for logging errors| 69 | |sendAlerts|`True`|Turn email/SMS alerts off `False` or on `True`| 70 | 71 | ####firestic_alert.py Settings 72 | 73 | |setting name|example|description| 74 | |-------------|---------------|---------------| 75 | |smtpServer|`'relayserver.yourdomain.org'`|Your email server FQDN or ip address| 76 | |smtpPort|`25`|Port on your email server| 77 | |fromEmail|`'Firestic@donotreply.yourdomain.org'`|Where the email alerts show to come from| 78 | |toEmail|`'securitydude@yourdomain.org'`|Who to send the email alerts to. Separate multiple addresses with commas| 79 | |emailTypeAlertOn|`['ips-event','malware-callback','malware-object']`|The types of alerts to send an email for. Possible types are: `ips-event`, `malware-callback`, `malware-object`, `infection-match`, `domain-match`, `web-infection`| 80 | |toSMS|`'aphonenumber@vtext.com'`|Who to send SMS alerts to. Format depends on the carrier. Separate multiple addresses with commas| 81 | |smsTypeAlertOn|`['malware-callback','malware-object']`|Possible types are the same as those for emailTypeAlertOn| 82 | |smsActionAlertOn|`['notified','alert']`|Only send SMS when these actions were reported by FireEye for this alert. Possible actions: `blocked`, `notified`, `alert`. Make this an empty array `[]` to not send SMS for anything| 83 | |myTimezone|`'US/Eastern'`|Local timezone for conversion (@timestamp is UTC). Common US TZ: `US/Central` `US/Eastern` `US/Mountain` `US/Pacific`. See [HERE](http://stackoverflow.com/questions/13866926/python-pytz-list-of-timezones) for a full list.| 84 | -------------------------------------------------------------------------------- /docs/start.md: -------------------------------------------------------------------------------- 1 | Start the Show 2 | -------------- 3 | 4 | #### Running the Script 5 | 6 | On one of your Elasticsearch nodes, run the script from the command line like this: 7 | 8 | `sudo python firestic.py` 9 | 10 | Check that it is running and accessible by going to the following in another computer's browser: 11 | 12 | `http://:/ping` 13 | 14 | for example: `http://192.168.1.2:8888/ping` 15 | 16 | At present, all exceptions are not being logged to file. You'll want to be able to see any exceptions that get thrown to stdout so running it from the console is preferred. Please report any issues you run across. 17 | 18 | I will daemonize the script later. For now, this will have to do. 19 | 20 | > **TIP:** Use [screen](http://www.gnu.org/software/screen/) to keep it running after log out. 21 | > Then you can ssh in from anywhere and check stdout as well as stop/start. This is a great 22 | > pseudo-service type of functionality and works very well. 23 | 24 | #### Send Some Test Alerts 25 | 26 | In your FireEye appliance, back on the notifications screen, look just under the table for a "Test-Fire" button. You can select the various types of alerts from the drop and down and click the button to send a test alert. I suggest doing a test notification for each one. If all is well, you'll see a single line per alert in the terminal where you are running the script (...and hopefully no exceptions are thrown) that looks like: 27 | 28 | ` - - [16/Dec/2014 08:08:09] "POST / HTTP/1.1" 200 -` 29 | 30 | Now check Elasticsearch to make sure the alerts made it. You can look in Kibana (preferred) or run the following: 31 | 32 | `curl -XGET localhost:9200/firestic*/_search?pretty` 33 | 34 | Finally, if you are using notifications, check your email and/or text messages and make sure they came through. 35 | 36 | Check for errors by looking for exceptions thrown to the screen and also in the log file (`firestic_error.log` by default). 37 | -------------------------------------------------------------------------------- /docs/templates.md: -------------------------------------------------------------------------------- 1 | Modifying Notification Templates 2 | -------------------------------- 3 | 4 | Two templates are provided for configuring the layout of notifications. One for HTML and one for text. Email notifications include both to accomodate all viewing requirements at the client. SMS text messages use the text template only. 5 | 6 | The templates are found in the `mustache_templates` directory: `template.html` and `template.txt` 7 | 8 | Mustache variable tags are used to place the data inside each template. These are the funny looking `{{somedata}}` parts scattered throughout the template. Currently, all available tags are being used in the notifications, but more will be added soon. The goal is to have any of the alert data available and easily inserted using these tags. 9 | 10 | The HTML template is a simple garden-variety HTML page. The CSS is at the top and the message body below. It can be opened in a browser as you play with the layout to allow for easy design work. The CSS is moved inline by the alert script automatically at run time to allow email clients to handle it gracefully. 11 | 12 | The text template is intentionally very basic so that SMS messages do not get out of hand. The alert script splits the final rendered message into 160 character sections and sends them individually. Be careful about putting too much in the text version and overwhelming your phone with massive SMS messages. 13 | 14 | **Currently available variable tags:** 15 | 16 | |tag |description | 17 | |---------------------------|-----------------------------------------------| 18 | |`{{alertid}}` |Alert id from FireEye system | 19 | |`{{alertname}}` |Alert type (e.g. malware-callback) | 20 | |`{{timestamp}}` |When event occurred | 21 | |`{{action}}` |Action taken by FireEye system (e.g. blocked) | 22 | |`{{severity}}` |Severity assigned by FireEye system | 23 | |`{{threatname}}` |If ips-event, this is the signature name. Otherwise, a list of malware names found| 24 | |`{{threatinfo}}` |If ips-event, this is a link to the CVE ID on cvedetails.com. Otherwise, a list of malware types found| 25 | |`{{sourceip}}` |Source ip address | 26 | |`{{sourcehostname}}` |Source host name | 27 | |`{{sourcecity}}` |Source city per geoip query | 28 | |`{{sourceregion}}` |Source state/region per geoip query | 29 | |`{{sourcecountry}}` |Source country per geoip query | 30 | |`{{sourceasn}}` |Source ASN per geoip query | 31 | |`{{destinationip}}` |Destination ip address | 32 | |`{{destinationhostname}}` |Destination host name | 33 | |`{{destinationcity}}` |Destination city per geoip query | 34 | |`{{destinationregion}}` |Destination state/region per geoip query | 35 | |`{{destinationcountry}}` |Destination country per geoip query | 36 | |`{{destinationasn}}` |Destination ASN per geoip query | -------------------------------------------------------------------------------- /firestic.py: -------------------------------------------------------------------------------- 1 | # FireStic - Python script for indexing FireEye json alerts 2 | # into Elasticsearch over http...and some alerting too 3 | # 4 | # Please see: https://github.com/spcampbell/firestic 5 | # 6 | from datetime import datetime 7 | from elasticsearch import Elasticsearch 8 | from BaseHTTPServer import HTTPServer 9 | from BaseHTTPServer import BaseHTTPRequestHandler 10 | from SocketServer import ThreadingMixIn 11 | import threading 12 | import json 13 | import logging 14 | import pygeoip # pip install pygeoip 15 | import socket 16 | import firestic_alert 17 | import fsconfig 18 | import socket 19 | 20 | 21 | class MyRequestHandler(BaseHTTPRequestHandler): 22 | 23 | # ---------- GET handler to check if httpserver up ---------- 24 | def do_GET(self): 25 | pingresponse = {"name": "Firestic is up"} 26 | if self.path == "/ping": 27 | self.send_response(200) 28 | self.send_header("Content-type:", "text/html") 29 | self.wfile.write("\n") 30 | json.dump(pingresponse, self.wfile) 31 | 32 | # -------------- POST handler: where the magic happens -------------- 33 | def do_POST(self): 34 | # get the posted data and remove newlines 35 | data = self.rfile.read(int(self.headers.getheader('Content-Length'))) 36 | clean = data.replace('\n', '') 37 | theJson = json.loads(clean) 38 | 39 | self.send_response(200) 40 | self.end_headers() 41 | 42 | # deal with multiple alerts embedded as an array 43 | if isinstance(theJson['alert'], list): 44 | # alertJson = theJson 45 | # del alertJson['alert'] 46 | for element in theJson['alert']: 47 | alertJson = {} # added for Issue #4 48 | alertJson['alert'] = element 49 | print "Processing FireEye Alert: " + str(alertJson['alert']['id']) 50 | processAlert(alertJson) 51 | else: 52 | print "Processing FireEye Alert: " + str(theJson['alert']['id']) 53 | processAlert(theJson) 54 | 55 | # ---------------- end class MyRequestHandler ---------------- 56 | 57 | 58 | # ---------------- Class handles requests in a separate thread. ---------------- 59 | 60 | class ThreadedHTTPServer(ThreadingMixIn, HTTPServer): 61 | pass 62 | 63 | # ---------------- end class ThreadedHTTPServer ---------------- 64 | 65 | def processAlert(theJson): 66 | # ---------- add geoip information ---------- 67 | alertInfo = {} 68 | alertInfo['srcIp'] = theJson['alert'].setdefault('src', {}).setdefault(u'ip', u'0.0.0.0') 69 | alertInfo['dstIp'] = theJson['alert'].setdefault('dst', {}).setdefault(u'ip', u'0.0.0.0') 70 | 71 | alertInfo['type'] = theJson['alert']['name'] 72 | if alertInfo['type'] == 'ips-event': 73 | alertInfo['mode'] = theJson['alert']['explanation']['ips-detected']['attack-mode'] 74 | 75 | geoInfo = queryGeoip(alertInfo) 76 | 77 | theJson['alert']['src']['geoip'] = geoInfo['src'] 78 | theJson['alert']['dst']['geoip'] = geoInfo['dst'] 79 | 80 | # ---------- add @timestamp ---------- 81 | # use alert.occurred for timestamp. It is different for IPS vs other alerts 82 | # ips-event alert.occurred format: 2014-12-11T03:28:08Z 83 | # all other alert.occurred format: 2014-12-11 03:28:33+00 84 | #if theJson['alert']['name'] == 'ips-event': 85 | # timeFormat = '%Y-%m-%dT%H:%M:%SZ' 86 | #else: 87 | # timeFormat = '%Y-%m-%d %H:%M:%S+00' 88 | # !! UPDATE 9/25/17 - time format is now same for all. Not sure exactly why. Likely a FE update. 89 | timeFormat = '%Y-%m-%dT%H:%M:%SZ' 90 | 91 | oc = datetime.strptime(theJson['alert']['occurred'], timeFormat) 92 | # Append YYYY.MM.DD to indexname like Logstash 93 | esIndexStamped = fsconfig.esIndex + oc.strftime('-%Y.%m.%d') 94 | # Put the formatted time into @timestamp 95 | theJson['@timestamp'] = oc.strftime('%Y-%m-%dT%H:%M:%S.%fZ') 96 | 97 | # ---------- Remove alert.explanation.os-changes ---------- 98 | # TODO: figure out a way to incorporate this info. 99 | # Doing this is complicated. Will require creative 100 | # Elasticsearch mapping (template?). Need to gather more json examples. 101 | # UPDATE -- For now, we will extract a few key bits of information from 102 | # os-changes: Operating system(s) targeted, application(s) targeted, 103 | # malicious activity found and listed in 'malicious-alert'. 104 | if 'os-changes' in theJson['alert']['explanation']: 105 | # Go extract some useful data to include in alert 106 | theJson['alert']['explanation']['summaryinfo'] = getSummaryInfo(theJson['alert']['explanation']['os-changes']) 107 | # DEV: save the os-changes field to file for later review 108 | with open('oschanges.json', 'a') as outfile: 109 | fileData = 'TIMESTAMP: ' + theJson['@timestamp'] + ' - ' 110 | fileData += theJson['alert']['name'] + ' - ' 111 | fileData += theJson['alert']['id'] + '\n' 112 | fileData += json.dumps(theJson['alert']['explanation']['os-changes']) 113 | fileData += '\n--------------------\n\n' 114 | outfile.write(fileData) 115 | del theJson['alert']['explanation']['os-changes'] 116 | print "[os-changes] deleted" 117 | 118 | # ---------- Index data into Elasticsearch ---------- 119 | try: 120 | es.index(index=esIndexStamped, 121 | doc_type=theJson['alert']['name'], body=theJson) 122 | except: 123 | logText = "\n-----------\nES POST ERROR\n-----------\nJSON: " 124 | logText += json.dumps(theJson) + "\n" 125 | # logText += "TIME: " + datetime.utcnow() + "\n" 126 | logging.exception(logText) 127 | 128 | # ---------- send email alerts ---------- 129 | if fsconfig.sendAlerts is True: 130 | try: 131 | firestic_alert.sendAlert(theJson, fsconfig) 132 | except: 133 | logText = "\n-----------\nEMAIL ERROR\n-----------\nJSON: " 134 | logText += json.dumps(theJson) + "\n" 135 | # logText += "TIME: " + datetime.utcnow() + "\n" 136 | logging.exception(logText) 137 | 138 | 139 | def queryGeoip(alertInfo): 140 | geoipInfo = {} 141 | 142 | if (alertInfo['type'] == 'ips-event') and (alertInfo['mode'] == 'server'): 143 | # ips-event mode is server so src = external, dest = internal 144 | geoipInfo['dst'] = getGeoipRecord(alertInfo['dstIp'], fsconfig.intGeoipDatabase, 'city') 145 | if geoipInfo['dst'] is not None: 146 | geoipInfo['dst']['asn'] = fsconfig.localASN 147 | geoipInfo['dst']['hostname'] = getHostname(alertInfo['dstIp']) 148 | geoipInfo['src'] = getGeoipRecord(alertInfo['srcIp'], fsconfig.extGeoipDatabase, 'city') 149 | if geoipInfo['src'] is not None: 150 | geoipInfo['src']['asn'] = getGeoipRecord(alertInfo['srcIp'], fsconfig.ASNGeoipDatabase, 'asn') 151 | geoipInfo['src']['hostname'] = getHostname(alertInfo['srcIp']) 152 | else: 153 | # treat all others as src = internal, dest = external 154 | geoipInfo['dst'] = getGeoipRecord(alertInfo['dstIp'], fsconfig.extGeoipDatabase, 'city') 155 | if geoipInfo['dst'] is not None: 156 | geoipInfo['dst']['asn'] = getGeoipRecord(alertInfo['dstIp'], fsconfig.ASNGeoipDatabase, 'asn') 157 | geoipInfo['dst']['hostname'] = getHostname(alertInfo['dstIp']) 158 | geoipInfo['src'] = getGeoipRecord(alertInfo['srcIp'], fsconfig.intGeoipDatabase, 'city') 159 | if geoipInfo['src'] is not None: 160 | geoipInfo['src']['asn'] = fsconfig.localASN 161 | geoipInfo['src']['hostname'] = getHostname(alertInfo['srcIp']) 162 | 163 | # add long,lat coordinate field...Kibana needs a field [long,lat] for "bettermap" 164 | if geoipInfo['dst'] is not None: 165 | geoipInfo['dst']['coordinates'] = [geoipInfo['dst']['longitude'], geoipInfo['dst']['latitude']] 166 | if geoipInfo['src'] is not None: 167 | geoipInfo['src']['coordinates'] = [geoipInfo['src']['longitude'], geoipInfo['src']['latitude']] 168 | 169 | return geoipInfo 170 | 171 | def getSummaryInfo(oschanges): 172 | summaryInfo = [] 173 | if isinstance(oschanges,list): 174 | for instance in oschanges: 175 | thisInfo = {} 176 | thisInfo['osinfo'] = instance['osinfo'] 177 | thisInfo['app-name'] = instance['application']['app-name'] 178 | thisInfo['malicious-alert'] = [] 179 | if ('malicious-alert' in instance): 180 | for eachma in instance['malicious-alert']: 181 | thisInfo['malicious-alert'].append(eachma) 182 | summaryInfo.append(thisInfo) 183 | else: 184 | thisInfo = {} 185 | thisInfo['osinfo'] = oschanges['osinfo'] 186 | thisInfo['app-name'] = oschanges['application']['app-name'] 187 | thisInfo['malicious-alert'] = [] 188 | if ('malicious-alert' in oschanges): 189 | for eachma in oschanges['malicious-alert']: 190 | thisInfo['malicious-alert'].append(eachma) 191 | summaryInfo.append(thisInfo) 192 | 193 | return summaryInfo 194 | 195 | def getHostname(ipaddress): 196 | try: 197 | lu = socket.gethostbyaddr(ipaddress) 198 | return lu[0] 199 | except: 200 | return None 201 | 202 | 203 | def getGeoipRecord(ipAddress, database, queryType): # queryType = asn or city 204 | gi = pygeoip.GeoIP(database) 205 | if queryType == 'city': 206 | return gi.record_by_addr(ipAddress) 207 | elif queryType == 'asn': 208 | return gi.org_by_addr(ipAddress) 209 | else: 210 | return None 211 | 212 | 213 | def main(): 214 | server = ThreadedHTTPServer((fsconfig.httpServerIP, fsconfig.httpServerPort), \ 215 | MyRequestHandler) 216 | print "\nStarting HTTP server...\n" 217 | try: 218 | server.serve_forever() 219 | except KeyboardInterrupt: 220 | print "\n\nHTTP server stopped.\n" 221 | 222 | 223 | if __name__ == "__main__": 224 | es = Elasticsearch() 225 | logging.basicConfig(level=logging.WARNING, 226 | filename=fsconfig.logFile, 227 | format='%(asctime)s - %(levelname)s - %(message)s') 228 | main() 229 | -------------------------------------------------------------------------------- /firestic_alert.py: -------------------------------------------------------------------------------- 1 | # FireStic - Python script for indexing FireEye json alerts 2 | # into Elasticsearch over http...and some alerting too 3 | # 4 | # Please see: https://github.com/spcampbell/firestic 5 | # 6 | import smtplib 7 | from email.mime.multipart import MIMEMultipart 8 | from email.mime.text import MIMEText 9 | from datetime import datetime 10 | import pytz # pip install pytz 11 | from premailer import transform # pip install premailer 12 | import pystache 13 | 14 | def sendAlert(theJson, fsconfig): 15 | 16 | # Prepare alert data to include in email 17 | emailData = gatherEmailData(theJson, fsconfig.myTimezone) 18 | 19 | # Build html version of message 20 | htmlEmail = buildHTMLMessage(emailData) 21 | 22 | # Build text version of message 23 | textEmail = buildTextMessage(emailData) 24 | 25 | # Email subject 26 | subjectLine = "FireStic Alert - " + emailData['alertname'] + " - " 27 | subjectLine += emailData['alertid'] + " - " + emailData['action'] 28 | 29 | # --------------------------------------- 30 | # Send SMS 31 | if emailData['alertname'] in fsconfig.smsTypeAlertOn: 32 | if emailData['action'] in fsconfig.smsActionAlertOn: 33 | txtMessages = splitForSMS(textEmail, emailData['alertid']) 34 | for txtMessage in txtMessages: 35 | msg = MIMEMultipart('alternative') 36 | msg['From'] = fsconfig.fromEmail 37 | msg['To'] = fsconfig.toSMS 38 | msg.attach(MIMEText(txtMessage, 'plain')) 39 | s = smtplib.SMTP(fsconfig.smtpServer, fsconfig.smtpPort) 40 | s.sendmail(fsconfig.fromEmail, fsconfig.toSMS, msg.as_string()) 41 | s.quit() 42 | 43 | # --------------------------------------- 44 | 45 | # Send email 46 | if emailData['alertname'] in fsconfig.emailTypeAlertOn: 47 | msg = MIMEMultipart('alternative') 48 | msg['Subject'] = subjectLine 49 | msg['From'] = fsconfig.fromEmail 50 | msg['To'] = fsconfig.toEmail 51 | part1 = MIMEText(textEmail, 'plain') 52 | part2 = MIMEText(htmlEmail, 'html') 53 | msg.attach(part1) 54 | msg.attach(part2) 55 | s = smtplib.SMTP(fsconfig.smtpServer, fsconfig.smtpPort) 56 | s.sendmail(fsconfig.fromEmail, fsconfig.toEmail, msg.as_string()) 57 | s.quit() 58 | 59 | # --------------------------------------- 60 | 61 | 62 | def buildHTMLMessage(emailData): 63 | # load HTML email template from file 64 | tf = open('mustache_templates/template.html', 'r') 65 | htmlTemplate = tf.read() 66 | tf.close() 67 | 68 | # render html message from mustache template 69 | htmlEmail = pystache.render(htmlTemplate, emailData) 70 | htmlEmail = encode_for_html(htmlEmail) 71 | # the following uses premailer to move css inline 72 | htmlEmail = transform(htmlEmail) 73 | 74 | return htmlEmail 75 | 76 | 77 | def buildTextMessage(emailData): 78 | # load text message template from file 79 | tf = open('mustache_templates/template.txt', 'r') 80 | textTemplate = tf.read() 81 | tf.close() 82 | 83 | # render text message 84 | textEmail = pystache.render(textTemplate, emailData) 85 | 86 | return textEmail 87 | 88 | 89 | def encode_for_html(unicode_data, encoding='ascii'): 90 | try: 91 | return unicode_data.encode(encoding, 'xmlcharrefreplace') 92 | except: 93 | print "encode_for_html failed for: " 94 | print unicode_data 95 | return " " 96 | 97 | 98 | def splitForSMS(fullText, alertID): 99 | texts = [] 100 | words = fullText.split(' ') 101 | curtext = '' 102 | for word in words: 103 | # for the first word, drop the space 104 | if len(curtext) == 0: 105 | curtext += word 106 | 107 | # check if there's enough space left in the current message 108 | elif len(curtext) <= 155 - (len(word) + 1): 109 | curtext += ' ' + word 110 | 111 | # not enough space. make a new message 112 | else: 113 | texts.append(curtext) 114 | curtext = '(' + alertID + '...) ' + word 115 | if curtext != '': 116 | texts.append(curtext) 117 | 118 | return texts 119 | 120 | 121 | def gatherEmailData(alertData, myTimezone): 122 | # TODO: refactor now that using mustache for template. 123 | # TODO: use get() 124 | 125 | # The process of gathering our data from the json sent over by FireEye is 126 | # complicated by the various permutations of how that data is presented. 127 | # It is likely that some of the following checks are redundant or 128 | # unnecessary. However, without a definitive answer on what all of the 129 | # possibilities are, the code tries to cover the worst case scenario. 130 | 131 | severityLevels = {u'crit': u'Critical', u'majr': u'Major', u'minr': u'Minor'} 132 | severityColors = {u'crit': u'red', u'majr':u'orange', u'minr':u'yellow'} 133 | emptyValue = u'N/A' 134 | emailData = {} 135 | 136 | emailData['timestamp'] = aslocaltimestr(alertData['@timestamp'], myTimezone) 137 | 138 | alertData = alertData.setdefault('alert', {}) 139 | 140 | emailData['alertname'] = alertData.setdefault('name', emptyValue) 141 | emailData['alertid'] = str(alertData['id']) 142 | emailData['action'] = alertData.setdefault('action', emptyValue) 143 | 144 | emailData['severity'] = alertData.setdefault('severity', emptyValue) 145 | if emailData['severity'] in severityLevels: 146 | emailData['severitycolor'] = severityColors[emailData['severity']] 147 | emailData['severity'] = severityLevels[emailData['severity']] 148 | 149 | emailData['alerturl'] = alertData.setdefault('alert-url', emptyValue) 150 | 151 | # ips-events provide a signature name and possible a CVE-ID 152 | # everything else tries to provide info on the malware 153 | if emailData['alertname'] == u'ips-event': 154 | emailData['threatname'] = alertData.setdefault('explanation', {}).setdefault('ips-detected', {}).setdefault('sig-name', emptyValue) 155 | emailData['threatinfo'] = alertData.setdefault('explanation', {}).setdefault('ips-detected', {}).setdefault('cve-id', emptyValue) 156 | if (emailData['threatinfo'] != emptyValue) and (emailData['threatinfo'] is not None) and (emailData['threatinfo'] != ''): 157 | emailData['threatinfo'] = u'' + emailData['threatinfo'] + u' on cvedetails.com' 158 | else: 159 | emailData['threatinfo'] = emptyValue 160 | emailData['action'] = alertData.setdefault('explanation', {}).setdefault('ips-detected', {}).setdefault('action-taken', emptyValue) 161 | else: 162 | # sometimes the malware field is an array, sometimes not 163 | mwNames = [] 164 | mwInfo = [] 165 | #mwURLs = [] 166 | urllist = [] 167 | if isinstance(alertData.setdefault('explanation', {}).setdefault('malware-detected', {}).setdefault('malware', {}), list): 168 | for element in alertData['explanation']['malware-detected']['malware']: 169 | thisURLrow = {} 170 | # if (element.has_key('name')) and (element['name'] is not None): 171 | if ('name' in element) and (element['name'] is not None): 172 | mwNames.append(element['name']) 173 | thisURLrow['name'] = element['name'] 174 | # if (element.has_key('original')) and (element['original'] is not None): 175 | if ('original' in element) and (element['original'] is not None): 176 | mwNames.append(element['original']) 177 | # if (element.has_key('stype')) and (element['stype'] is not None): 178 | if ('stype' in element) and (element['stype'] is not None): 179 | mwInfo.append(element['stype']) 180 | if ('url' in element) and (element['url'] is not None): 181 | #mwURLs.append(element['url']) 182 | thisURLrow['url'] = element['url'] 183 | if ('objurl' in element) and (element['objurl'] is not None): 184 | #mwURLs.append(element['objurl']) 185 | thisURLrow['url'] = element['objurl'] 186 | urllist.append(thisURLrow) 187 | else: 188 | element = alertData['explanation']['malware-detected']['malware'] 189 | thisURLrow = {} 190 | # if (element.has_key('name')) and (element['name'] is not None): 191 | if ('name' in element) and (element['name'] is not None): 192 | mwNames.append(element['name']) 193 | thisURLrow['name'] = element['name'] 194 | # if (element.has_key('original')) and (element['original'] is not None): 195 | if ('original' in element) and (element['original'] is not None): 196 | mwNames.append(element['original']) 197 | # if (element.has_key('stype')) and (element['stype'] is not None): 198 | if ('stype' in element) and (element['stype'] is not None): 199 | mwInfo.append(element['stype']) 200 | if ('url' in element) and (element['url'] is not None): 201 | #mwURLs.append(element['url']) 202 | thisURLrow['url'] = element['url'] 203 | if ('objurl' in element) and (element['objurl'] is not None): 204 | #mwURLs.append(element['objurl']) 205 | thisURLrow['url'] = element['objurl'] 206 | 207 | #emailData['threatname'] = alertData['explanation']['malware-detected']['malware'].setdefault('name', emptyValue) 208 | #emailData['threatinfo'] = alertData['explanation']['malware-detected']['malware'].setdefault('stype', emptyValue) 209 | 210 | if len(mwNames): 211 | emailData['threatname'] = ', '.join(mwNames) 212 | else: 213 | emailData['threatname'] = emptyValue 214 | if len(mwInfo): 215 | emailData['threatinfo'] = ', '.join(mwInfo) 216 | else: 217 | emailData['threatinfo'] = emptyValue 218 | 219 | emailData['threatURLs'] = [] 220 | if len(urllist): 221 | emailData['threatURLs'].append({'urllist':urllist}) 222 | #for url in mwURLs: 223 | # thisDict = {'url':url} 224 | # emailData['threatURLs']['urllist'].append(thisDict) 225 | 226 | 227 | 228 | emailData['sourceip'] = alertData.setdefault('src', {}).setdefault('ip', emptyValue) 229 | emailData['destinationip'] = alertData.setdefault('dst', {}).setdefault('ip', emptyValue) 230 | 231 | # source detail 232 | if alertData['src'].setdefault('geoip', {}) is None: 233 | alertData['src']['geoip'] = {} 234 | if alertData['src']['geoip'].setdefault('city', emptyValue) is None: 235 | alertData['src']['geoip']['city'] = emptyValue 236 | emailData['sourcecity'] = alertData['src']['geoip']['city'] 237 | if alertData['src']['geoip'].setdefault('region_code', emptyValue) is None: 238 | alertData['src']['geoip']['region_code'] = emptyValue 239 | emailData['sourceregion'] = alertData['src']['geoip']['region_code'] 240 | if alertData['src']['geoip'].setdefault('country_name', emptyValue) is None: 241 | alertData['src']['geoip']['country_name'] = emptyValue 242 | emailData['sourcecountry'] = alertData['src']['geoip']['country_name'] 243 | if alertData['src']['geoip'].setdefault('asn', emptyValue) is None: 244 | alertData['src']['geoip']['asn'] = emptyValue 245 | emailData['sourceasn'] = alertData['src']['geoip']['asn'] 246 | if alertData['src']['geoip'].setdefault('hostname', emptyValue) is None: 247 | alertData['src']['geoip']['hostname'] = emptyValue 248 | emailData['sourcehostname'] = alertData['src']['geoip']['hostname'] 249 | 250 | # destination detail 251 | if alertData['dst'].setdefault('geoip', {}) is None: 252 | alertData['dst']['geoip'] = {} 253 | if alertData['dst']['geoip'].setdefault('city', emptyValue) is None: 254 | alertData['dst']['geoip']['city'] = emptyValue 255 | emailData['destinationcity'] = alertData['dst']['geoip']['city'] 256 | if alertData['dst']['geoip'].setdefault('region_code', emptyValue) is None: 257 | alertData['dst']['geoip']['region_code'] = emptyValue 258 | emailData['destinationregion'] = alertData['dst']['geoip']['region_code'] 259 | if alertData['dst']['geoip'].setdefault('country_name', emptyValue) is None: 260 | alertData['dst']['geoip']['country_name'] = emptyValue 261 | emailData['destinationcountry'] = alertData['dst']['geoip']['country_name'] 262 | if alertData['dst']['geoip'].setdefault('asn', emptyValue) is None: 263 | alertData['dst']['geoip']['asn'] = emptyValue 264 | emailData['destinationasn'] = alertData['dst']['geoip']['asn'] 265 | if alertData['dst']['geoip'].setdefault('hostname', emptyValue) is None: 266 | alertData['dst']['geoip']['hostname'] = emptyValue 267 | emailData['destinationhostname'] = alertData['dst']['geoip']['hostname'] 268 | 269 | # instance['osinfo'] --> name of OS 270 | # instance['application']['app-name'] --> targeted application 271 | # instance['malicious-alert'] --> list of malicious activity 272 | # example of one element in list: 273 | # 'msg':'Process is registering a hook to monitor...' 274 | # 'classtype':'Keylogging-Activity' 275 | # 'display-msg':'High-level keyboard hook registered' 276 | 277 | #{{ #summaryinfo }} 278 | # {{ osinfo }} 279 | # {{ app-name }} 280 | # {{ #malicious-alert }} 281 | # {{ classtype }} 282 | # {{ msg }} 283 | # {{ display-msg }} 284 | # {{ /malicious-alert }} 285 | #{{ /summaryinfo }} 286 | 287 | # basic information concerning malicious activity from os-changes 288 | if (len(alertData['explanation'].setdefault('summaryinfo',[]))): 289 | emailData['summaryinfo'] = alertData['explanation']['summaryinfo'] 290 | 291 | return emailData 292 | 293 | 294 | def aslocaltimestr(utc_str, myTimezone): 295 | utc_dt = datetime.strptime(utc_str, '%Y-%m-%dT%H:%M:%S.%fZ') 296 | local_tz = pytz.timezone(myTimezone) 297 | local_dt = utc_dt.replace(tzinfo=pytz.utc).astimezone(local_tz) 298 | return local_tz.normalize(local_dt).strftime('%m-%d-%Y %H:%M:%S %Z%z') 299 | -------------------------------------------------------------------------------- /fsconfig.py: -------------------------------------------------------------------------------- 1 | # --------------------------------------------- 2 | # --- firestic.py configuration 3 | 4 | # Elasticsearch index to use - YYYY.MM.DD will be appended 5 | esIndex = 'firestic' 6 | 7 | # Geoip database for external (internet) addresses 8 | extGeoipDatabase = 'geoip/GeoLiteCity.dat' 9 | 10 | # Geoip database for internal (LAN) addresses (see README) 11 | intGeoipDatabase = 'geoip/GeoLiteCity.dat' 12 | 13 | # Geoip database for external address ASN info 14 | ASNGeoipDatabase = 'geoip/GeoIPASNum.dat' 15 | 16 | # ASN for internal addresses 17 | localASN = 'your_org_name' 18 | 19 | # IP for http server to listen on 20 | httpServerIP = 'ipa.dd.re.ss' 21 | 22 | # Port for http server to listen on 23 | httpServerPort = 8888 24 | 25 | # File for logging errors 26 | logFile = 'firestic_error.log' 27 | 28 | # Send email/SMS alerts - see firestic_alert.py 29 | sendAlerts = True 30 | 31 | # --------------------------------------------- 32 | # --- firestic_alert.py configuration 33 | 34 | # email server FQDN or ip address 35 | smtpServer = "your.relay.server.org" 36 | smtpPort = 25 37 | 38 | # From address 39 | fromEmail = "FireStic@donotreply.yourdomain.org" 40 | 41 | # Email Recipients 42 | # Comma delimited string of email addresses 43 | toEmail = "securitydude@yourdomain.org" 44 | 45 | # Possible types: ips-event, malware-callback, malware-object, infection-match, domain-match, web-infection 46 | emailTypeAlertOn = ['ips-event', 'malware-callback', 'malware-object', 'infection-match', 'domain-match', 'web-infection'] 47 | 48 | # SMS Recipients 49 | # Comma delimted string. Format depends on carrier. You'll have to look it up. 50 | toSMS = "aphonenumber@vtext.com" 51 | 52 | # Possible types: ips-event, malware-callback, malware-object, infection-match, domain-match, web-infection 53 | smsTypeAlertOn = ['malware-callback', 'malware-object', 'infection-match', 'domain-match', 'web-infection'] 54 | 55 | # Possible actions: blocked, notified, alert. 56 | # Make this an empty array [] to not alert on anything. 57 | smsActionAlertOn = ['notified', 'alert'] 58 | 59 | # Local timezone for conversion (@timestamp is UTC) - see pytz.all_timezones 60 | # http://stackoverflow.com/questions/13866926/python-pytz-list-of-timezones 61 | # Common US TZ: US/Central US/Eastern US/Mountain US/Pacific - see link above for more 62 | myTimezone = 'US/Eastern' 63 | -------------------------------------------------------------------------------- /geoip/README.md: -------------------------------------------------------------------------------- 1 | Put geoip .dat files here 2 | -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: FireStic 2 | pages: 3 | - [index.md, Home] 4 | - [prepenv.md, Prepare Your Environment] 5 | - [prepscript.md, Prepare The Script and Dependencies] 6 | - [start.md, Start the Show] 7 | - [templates.md, Modifying Notification Templates] 8 | - [other.md, Other Stuff] 9 | - [TODO.md, TODO] 10 | theme: readthedocs 11 | -------------------------------------------------------------------------------- /mustache_templates/template.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 72 | 85 | 86 | 87 | 109 | 131 | 132 |
{{ alertname }} - {{ alertid }}
58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 |
Occurred: {{ timestamp }}
Severity:{{ severity }}
Action:{{ action }}
71 | 		73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 |
Threat Name: {{ threatname }}
Threat Info: {{ threatinfo }}
FireEye Alert Link
84 |
88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 |
Source
{{ sourceip }}
Hostname: {{ sourcehostname }}
{{ sourcecity }}, {{ sourceregion }}
{{ sourcecountry }}
ASN: {{ sourceasn }}
108 | 		110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 |
Destination
{{ destinationip }}
Hostname: {{ destinationhostname }}
{{ destinationcity }}, {{ destinationregion }}
{{ destinationcountry }}
ASN: {{ destinationasn }}
130 |
133 | 134 | {{ #threatURLs }} 135 | 136 |
137 | 138 | {{ #urllist }} 139 | 140 | 141 | 142 | {{ /urllist }} 143 |
Malicious URLs Found
{{ name }}{{ url }}
144 | 145 | {{ /threatURLs }} 146 | 147 | {{ #summaryinfo }} 148 | 149 |
150 | 151 | 152 | {{ #malicious-alert }} 153 | 154 | 155 | 156 | {{ /malicious-alert }} 157 |
Operating system: {{ osinfo }}
Targeted application: {{ app-name }}
{{ classtype }}{{ msg }}{{ display-msg }}
158 | 159 | {{ /summaryinfo }} 160 | 161 | 162 | 163 | -------------------------------------------------------------------------------- /mustache_templates/template.txt: -------------------------------------------------------------------------------- 1 | ID: {{ alertid }} 2 | Type: {{ alertname }} 3 | Action: {{ action }} 4 | Severity: {{ severity }} 5 | Threat: {{ threatname }} 6 | Source: {{ sourceip }} 7 | {{ sourcecity }}, {{ sourceregion }} 8 | {{ sourcecountry }} 9 | {{ sourceasn }} 10 | Destination: {{ destinationip }} 11 | {{ destinationcity}}, {{ destinationregion }} 12 | {{ destinationcountry }} 13 | {{ destinationasn }} -------------------------------------------------------------------------------- /prep_files/firestic_ES_template.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | curl -XPUT localhost:9200/_template/firestic_1 -d ' 4 | { 5 | "template" : "firestic-*", 6 | "settings" : { 7 | "number_of_shards" : 5, 8 | "index.refresh_interval" : "5s" 9 | }, 10 | "mappings" : { 11 | "_default_" : { 12 | "_all" : {"enabled" : true}, 13 | "dynamic_templates" : [ { 14 | "string_fields" : { 15 | "match" : "*", 16 | "match_mapping_type" : "string", 17 | "mapping" : { 18 | "type" : "string", "index" : "analyzed", "omit_norms" : true, 19 | "fields" : { 20 | "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256} 21 | } 22 | } 23 | } 24 | } ] 25 | } 26 | } 27 | }' 28 | -------------------------------------------------------------------------------- /prep_files/firestic_kibana.json: -------------------------------------------------------------------------------- 1 | { 2 | "title": "FireStic_demo", 3 | "services": { 4 | "query": { 5 | "list": { 6 | "0": { 7 | "query": "*", 8 | "alias": "", 9 | "color": "#7EB26D", 10 | "id": 0, 11 | "pin": true, 12 | "type": "lucene", 13 | "enable": true 14 | }, 15 | "1": { 16 | "id": 1, 17 | "color": "#806EB7", 18 | "alias": "IPS", 19 | "pin": true, 20 | "type": "lucene", 21 | "enable": true, 22 | "query": "_type:ips-event" 23 | }, 24 | "2": { 25 | "id": 2, 26 | "color": "#6ED0E0", 27 | "alias": "Infection Match", 28 | "pin": true, 29 | "type": "lucene", 30 | "enable": true, 31 | "query": "_type:infection-match" 32 | }, 33 | "3": { 34 | "id": 3, 35 | "color": "#EAB839", 36 | "alias": "Malware Object", 37 | "pin": true, 38 | "type": "lucene", 39 | "enable": true, 40 | "query": "_type:malware-object" 41 | }, 42 | "4": { 43 | "id": 4, 44 | "color": "#E24D42", 45 | "alias": "Malware Callback", 46 | "pin": true, 47 | "type": "lucene", 48 | "enable": true, 49 | "query": "_type:malware-callback" 50 | }, 51 | "5": { 52 | "id": 5, 53 | "color": "#1F78C1", 54 | "alias": "Domain Match", 55 | "pin": true, 56 | "type": "lucene", 57 | "enable": true, 58 | "query": "_type:domain-match" 59 | }, 60 | "6": { 61 | "id": 6, 62 | "color": "#BF1B00", 63 | "alias": "Critical", 64 | "pin": true, 65 | "type": "lucene", 66 | "enable": true, 67 | "query": "-_type:ips-event AND alert.severity:crit" 68 | }, 69 | "7": { 70 | "id": 7, 71 | "color": "#E0752D", 72 | "alias": "Major", 73 | "pin": true, 74 | "type": "lucene", 75 | "enable": true, 76 | "query": "-_type:ips-event AND alert.severity:majr" 77 | }, 78 | "8": { 79 | "id": 8, 80 | "color": "#E5AC0E", 81 | "alias": "Minor", 82 | "pin": true, 83 | "type": "lucene", 84 | "enable": true, 85 | "query": "-_type:ips-event AND alert.severity:minr" 86 | }, 87 | "9": { 88 | "id": 9, 89 | "color": "#F29191", 90 | "alias": "Web Infection", 91 | "pin": true, 92 | "type": "lucene", 93 | "enable": true, 94 | "query": "_type:web-infection" 95 | } 96 | }, 97 | "ids": [ 98 | 0, 99 | 1, 100 | 2, 101 | 3, 102 | 4, 103 | 5, 104 | 6, 105 | 7, 106 | 8, 107 | 9 108 | ] 109 | }, 110 | "filter": { 111 | "list": { 112 | "0": { 113 | "type": "time", 114 | "field": "@timestamp", 115 | "from": "now-2d", 116 | "to": "now", 117 | "mandate": "must", 118 | "active": true, 119 | "alias": "", 120 | "id": 0 121 | } 122 | }, 123 | "ids": [ 124 | 0 125 | ] 126 | } 127 | }, 128 | "rows": [ 129 | { 130 | "title": "Graph", 131 | "height": "200", 132 | "editable": true, 133 | "collapse": false, 134 | "collapsable": true, 135 | "panels": [ 136 | { 137 | "span": 9, 138 | "editable": true, 139 | "group": [ 140 | "default" 141 | ], 142 | "type": "histogram", 143 | "mode": "count", 144 | "time_field": "@timestamp", 145 | "value_field": null, 146 | "auto_int": true, 147 | "resolution": 100, 148 | "interval": "30m", 149 | "fill": 4, 150 | "linewidth": 1, 151 | "timezone": "browser", 152 | "spyable": true, 153 | "zoomlinks": true, 154 | "bars": false, 155 | "stack": false, 156 | "points": false, 157 | "lines": true, 158 | "legend": true, 159 | "x-axis": true, 160 | "y-axis": true, 161 | "percentage": false, 162 | "interactive": true, 163 | "queries": { 164 | "mode": "selected", 165 | "ids": [ 166 | 2, 167 | 3, 168 | 4, 169 | 5, 170 | 9 171 | ] 172 | }, 173 | "title": "Alert Events over time", 174 | "intervals": [ 175 | "auto", 176 | "1s", 177 | "1m", 178 | "5m", 179 | "10m", 180 | "30m", 181 | "1h", 182 | "3h", 183 | "12h", 184 | "1d", 185 | "1w", 186 | "1M", 187 | "1y" 188 | ], 189 | "options": true, 190 | "tooltip": { 191 | "value_type": "cumulative", 192 | "query_as_alias": true 193 | }, 194 | "scale": 1, 195 | "y_format": "none", 196 | "grid": { 197 | "max": null, 198 | "min": 0 199 | }, 200 | "annotate": { 201 | "enable": false, 202 | "query": "*", 203 | "size": 20, 204 | "field": "_type", 205 | "sort": [ 206 | "_score", 207 | "desc" 208 | ] 209 | }, 210 | "pointradius": 5, 211 | "show_query": true, 212 | "legend_counts": true, 213 | "zerofill": true, 214 | "derivative": false 215 | }, 216 | { 217 | "span": 3, 218 | "editable": true, 219 | "type": "hits", 220 | "loadingEditor": false, 221 | "style": { 222 | "font-size": "10pt" 223 | }, 224 | "arrangement": "horizontal", 225 | "chart": "pie", 226 | "counter_pos": "below", 227 | "donut": true, 228 | "tilt": false, 229 | "labels": true, 230 | "spyable": true, 231 | "queries": { 232 | "mode": "selected", 233 | "ids": [ 234 | 1, 235 | 2, 236 | 3, 237 | 4, 238 | 5, 239 | 9 240 | ] 241 | }, 242 | "title": "Alert Types" 243 | } 244 | ], 245 | "notice": false 246 | }, 247 | { 248 | "title": "", 249 | "height": "200", 250 | "editable": true, 251 | "collapse": false, 252 | "collapsable": true, 253 | "panels": [ 254 | { 255 | "span": 9, 256 | "editable": true, 257 | "group": [ 258 | "default" 259 | ], 260 | "type": "histogram", 261 | "mode": "count", 262 | "time_field": "@timestamp", 263 | "value_field": null, 264 | "auto_int": true, 265 | "resolution": 100, 266 | "interval": "30m", 267 | "fill": 4, 268 | "linewidth": 1, 269 | "timezone": "browser", 270 | "spyable": true, 271 | "zoomlinks": true, 272 | "bars": false, 273 | "stack": false, 274 | "points": false, 275 | "lines": true, 276 | "legend": true, 277 | "x-axis": true, 278 | "y-axis": true, 279 | "percentage": false, 280 | "interactive": true, 281 | "queries": { 282 | "mode": "selected", 283 | "ids": [ 284 | 1 285 | ] 286 | }, 287 | "title": "Critical IPS Events over time", 288 | "intervals": [ 289 | "auto", 290 | "1s", 291 | "1m", 292 | "5m", 293 | "10m", 294 | "30m", 295 | "1h", 296 | "3h", 297 | "12h", 298 | "1d", 299 | "1w", 300 | "1M", 301 | "1y" 302 | ], 303 | "options": true, 304 | "tooltip": { 305 | "value_type": "cumulative", 306 | "query_as_alias": true 307 | }, 308 | "scale": 1, 309 | "y_format": "none", 310 | "grid": { 311 | "max": null, 312 | "min": 0 313 | }, 314 | "annotate": { 315 | "enable": false, 316 | "query": "*", 317 | "size": 20, 318 | "field": "_type", 319 | "sort": [ 320 | "_score", 321 | "desc" 322 | ] 323 | }, 324 | "pointradius": 5, 325 | "show_query": true, 326 | "legend_counts": true, 327 | "zerofill": true, 328 | "derivative": false 329 | }, 330 | { 331 | "span": 3, 332 | "editable": true, 333 | "type": "hits", 334 | "loadingEditor": false, 335 | "style": { 336 | "font-size": "10pt" 337 | }, 338 | "arrangement": "horizontal", 339 | "chart": "pie", 340 | "counter_pos": "below", 341 | "donut": true, 342 | "tilt": false, 343 | "labels": true, 344 | "spyable": true, 345 | "queries": { 346 | "mode": "selected", 347 | "ids": [ 348 | 6, 349 | 7, 350 | 8 351 | ] 352 | }, 353 | "title": "Alert Severity (non-ips)" 354 | } 355 | ], 356 | "notice": false 357 | }, 358 | { 359 | "title": "Events 2", 360 | "height": "350", 361 | "editable": true, 362 | "collapse": false, 363 | "collapsable": true, 364 | "panels": [ 365 | { 366 | "error": false, 367 | "span": 6, 368 | "editable": true, 369 | "type": "bettermap", 370 | "loadingEditor": false, 371 | "field": "alert.dst.geoip.coordinates", 372 | "size": 1000, 373 | "spyable": true, 374 | "tooltip": "_id", 375 | "queries": { 376 | "mode": "selected", 377 | "ids": [ 378 | 0 379 | ] 380 | }, 381 | "title": "Destinations" 382 | }, 383 | { 384 | "error": false, 385 | "span": 6, 386 | "editable": true, 387 | "type": "bettermap", 388 | "loadingEditor": false, 389 | "field": "alert.src.geoip.coordinates", 390 | "size": 1000, 391 | "spyable": true, 392 | "tooltip": "_id", 393 | "queries": { 394 | "mode": "selected", 395 | "ids": [ 396 | 0 397 | ] 398 | }, 399 | "title": "Sources" 400 | } 401 | ], 402 | "notice": false 403 | }, 404 | { 405 | "title": "", 406 | "height": "200", 407 | "editable": true, 408 | "collapse": false, 409 | "collapsable": true, 410 | "panels": [ 411 | { 412 | "error": false, 413 | "span": 3, 414 | "editable": true, 415 | "type": "terms", 416 | "loadingEditor": false, 417 | "field": "alert.dst.geoip.country_name.raw", 418 | "exclude": [], 419 | "missing": false, 420 | "other": false, 421 | "size": 10, 422 | "order": "count", 423 | "style": { 424 | "font-size": "9pt" 425 | }, 426 | "donut": false, 427 | "tilt": false, 428 | "labels": true, 429 | "arrangement": "horizontal", 430 | "chart": "table", 431 | "counter_pos": "above", 432 | "spyable": true, 433 | "queries": { 434 | "mode": "all", 435 | "ids": [ 436 | 0, 437 | 1, 438 | 2, 439 | 3, 440 | 4, 441 | 5, 442 | 6, 443 | 7, 444 | 8, 445 | 9 446 | ] 447 | }, 448 | "tmode": "terms", 449 | "tstat": "total", 450 | "valuefield": "", 451 | "title": "Destination Countries" 452 | }, 453 | { 454 | "error": false, 455 | "span": 3, 456 | "editable": true, 457 | "type": "terms", 458 | "loadingEditor": false, 459 | "field": "alert.src.geoip.city.raw", 460 | "exclude": [], 461 | "missing": false, 462 | "other": false, 463 | "size": 10, 464 | "order": "count", 465 | "style": { 466 | "font-size": "9pt" 467 | }, 468 | "donut": false, 469 | "tilt": false, 470 | "labels": true, 471 | "arrangement": "horizontal", 472 | "chart": "table", 473 | "counter_pos": "above", 474 | "spyable": true, 475 | "queries": { 476 | "mode": "selected", 477 | "ids": [ 478 | 0 479 | ] 480 | }, 481 | "tmode": "terms", 482 | "tstat": "total", 483 | "valuefield": "", 484 | "title": "Top 10 Source Locations" 485 | }, 486 | { 487 | "error": false, 488 | "span": 3, 489 | "editable": true, 490 | "type": "terms", 491 | "loadingEditor": false, 492 | "field": "alert.src.geoip.hostname.raw", 493 | "exclude": [], 494 | "missing": false, 495 | "other": false, 496 | "size": 10, 497 | "order": "count", 498 | "style": { 499 | "font-size": "9pt" 500 | }, 501 | "donut": false, 502 | "tilt": false, 503 | "labels": true, 504 | "arrangement": "horizontal", 505 | "chart": "table", 506 | "counter_pos": "above", 507 | "spyable": true, 508 | "queries": { 509 | "mode": "selected", 510 | "ids": [ 511 | 0 512 | ] 513 | }, 514 | "tmode": "terms", 515 | "tstat": "total", 516 | "valuefield": "", 517 | "title": "Top 10 Source Hosts" 518 | }, 519 | { 520 | "error": false, 521 | "span": 3, 522 | "editable": true, 523 | "type": "terms", 524 | "loadingEditor": false, 525 | "field": "alert.explanation.malware-detected.malware.name.raw", 526 | "exclude": [], 527 | "missing": false, 528 | "other": false, 529 | "size": 10, 530 | "order": "count", 531 | "style": { 532 | "font-size": "9pt" 533 | }, 534 | "donut": false, 535 | "tilt": false, 536 | "labels": true, 537 | "arrangement": "horizontal", 538 | "chart": "table", 539 | "counter_pos": "above", 540 | "spyable": true, 541 | "queries": { 542 | "mode": "selected", 543 | "ids": [ 544 | 0 545 | ] 546 | }, 547 | "tmode": "terms", 548 | "tstat": "total", 549 | "valuefield": "", 550 | "title": "Top 10 Malware" 551 | } 552 | ], 553 | "notice": false 554 | }, 555 | { 556 | "title": "Events", 557 | "height": "350px", 558 | "editable": true, 559 | "collapse": false, 560 | "collapsable": true, 561 | "panels": [ 562 | { 563 | "title": "All events", 564 | "error": false, 565 | "span": 12, 566 | "editable": true, 567 | "group": [ 568 | "default" 569 | ], 570 | "type": "table", 571 | "size": 100, 572 | "pages": 5, 573 | "offset": 0, 574 | "sort": [ 575 | "@timestamp", 576 | "desc" 577 | ], 578 | "style": { 579 | "font-size": "9pt" 580 | }, 581 | "overflow": "min-height", 582 | "fields": [ 583 | "@timestamp", 584 | "alert.src.ip", 585 | "alert.src.geoip.hostname", 586 | "alert.src.geoip.city", 587 | "alert.dst.ip", 588 | "alert.dst.geoip.hostname", 589 | "alert.dst.geoip.city", 590 | "alert.dst.geoip.country_name", 591 | "_type", 592 | "alert.action", 593 | "alert.explanation.ips-detected.sig-name", 594 | "alert.explanation.malware-detected.malware.name" 595 | ], 596 | "localTime": true, 597 | "timeField": "@timestamp", 598 | "highlight": [], 599 | "sortable": true, 600 | "header": true, 601 | "paging": true, 602 | "spyable": true, 603 | "queries": { 604 | "mode": "selected", 605 | "ids": [ 606 | 0 607 | ] 608 | }, 609 | "field_list": false, 610 | "status": "Stable", 611 | "trimFactor": 300, 612 | "normTimes": true, 613 | "all_fields": false 614 | } 615 | ], 616 | "notice": false 617 | } 618 | ], 619 | "editable": true, 620 | "failover": false, 621 | "index": { 622 | "interval": "day", 623 | "pattern": "[firestic-]YYYY.MM.DD", 624 | "default": "NO_TIME_FILTER_OR_INDEX_PATTERN_NOT_MATCHED", 625 | "warm_fields": true 626 | }, 627 | "style": "dark", 628 | "panel_hints": true, 629 | "pulldowns": [ 630 | { 631 | "type": "query", 632 | "collapse": true, 633 | "notice": false, 634 | "query": "*", 635 | "pinned": true, 636 | "history": [ 637 | "-_type:ips-event AND alert.severity:minr", 638 | "-_type:ips-event AND alert.severity:majr", 639 | "-_type:ips-event AND alert.severity:crit", 640 | "_type:domain-match", 641 | "_type:malware-callback", 642 | "_type:malware-object", 643 | "_type:infection-match", 644 | "_type:ips-event", 645 | "*" ], 646 | "remember": 10, 647 | "enable": true 648 | }, 649 | { 650 | "type": "filtering", 651 | "collapse": true, 652 | "notice": false, 653 | "enable": true 654 | } 655 | ], 656 | "nav": [ 657 | { 658 | "type": "timepicker", 659 | "collapse": false, 660 | "notice": false, 661 | "status": "Stable", 662 | "time_options": [ 663 | "5m", 664 | "15m", 665 | "1h", 666 | "6h", 667 | "12h", 668 | "24h", 669 | "2d", 670 | "4d", 671 | "7d", 672 | "30d" 673 | ], 674 | "refresh_intervals": [ 675 | "5s", 676 | "10s", 677 | "30s", 678 | "1m", 679 | "5m", 680 | "15m", 681 | "30m", 682 | "1h", 683 | "2h", 684 | "1d" 685 | ], 686 | "timefield": "@timestamp", 687 | "now": true, 688 | "filter_id": 0, 689 | "enable": true 690 | } 691 | ], 692 | "loader": { 693 | "save_gist": false, 694 | "save_elasticsearch": true, 695 | "save_local": true, 696 | "save_default": true, 697 | "save_temp": true, 698 | "save_temp_ttl_enable": false, 699 | "save_temp_ttl": "30d", 700 | "load_gist": true, 701 | "load_elasticsearch": true, 702 | "load_elasticsearch_size": 20, 703 | "load_local": true, 704 | "hide": false 705 | }, 706 | "refresh": "15m" 707 | } 708 | -------------------------------------------------------------------------------- /screenshots/examplealert_mal_cb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/spcampbell/FireStic/050133d1b76ddc3a19a7cc6e734fb47a439f1d3f/screenshots/examplealert_mal_cb.png -------------------------------------------------------------------------------- /screenshots/examplealert_mal_obj.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/spcampbell/FireStic/050133d1b76ddc3a19a7cc6e734fb47a439f1d3f/screenshots/examplealert_mal_obj.png -------------------------------------------------------------------------------- /testing/fstest.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Script to send FireEye alerts saved as json files to FireStic for testing. 3 | 4 | Option to send a single file or to read a directory and send all .json files. 5 | ''' 6 | 7 | import requests 8 | import json 9 | import sys 10 | import getopt 11 | import glob 12 | import time 13 | 14 | # parameters 15 | # -f --file = a specific json file to send 16 | # -d --dir = all the json files in a directory 17 | # -t --timeout = delay in seconds between sends. Default 1 second 18 | # -u --url = url/ip address of server 19 | # -p --port = port server is listening on 20 | 21 | def processFile(inputfile,serverurl): 22 | headers = {'content-type': 'application/json'} 23 | try: 24 | with open(inputfile) as json_file: 25 | file_data = json_file.read() 26 | try: 27 | r = requests.post(serverurl, data=file_data, headers=headers, timeout=5) 28 | except Exception, e: 29 | print " " 30 | print "COMMUNICATION ERROR : " + str(e) 31 | print " " 32 | sys.exit(2) 33 | except Exception, e: 34 | print " " 35 | print "FILE ERROR : " + str(e) 36 | print " " 37 | 38 | print inputfile + " sent to " + serverurl + ". Status code: " + str(r.status_code) + "." 39 | 40 | return 41 | 42 | def printopts(): 43 | print ''' 44 | USAGE: 45 | -f --file a specific json file to send 46 | -d --dir directory of json files to send. Use ./ for current directory 47 | ** must include either -f or -d but not both ** 48 | -t --timeout (optional) seconds delay between multiple sends. Default = 1 49 | -u --url url/ip address to send to 50 | -p --port port server is listening on 51 | 52 | EXAMPLES: 53 | fstest.py -f ./testalert.json -u localhost -p 8080 54 | fstest.py -d ./alerts -t 2 -u 192.168.1.2 -p 8080 55 | fstest.py -d ./ -u localhost -p 8888 56 | ''' 57 | 58 | def main(argv): 59 | inputfile = '' 60 | inputdir = '' 61 | timeout = 1 62 | url = '' 63 | port = '' 64 | mode = '' 65 | 66 | try: 67 | opts, args = getopt.getopt(argv,"hf:d:t:u:p:",["help=","file=","dir=","timeout=","url=","port="]) 68 | except getopt.GetoptError: 69 | printopts() 70 | sys.exit(2) 71 | 72 | if not len(opts): 73 | print 'No options specified:' 74 | printopts() 75 | sys.exit(2) 76 | for opt, arg in opts: 77 | if opt in ("-h","--help"): 78 | printopts() 79 | sys.exit() 80 | elif opt in ("-f", "--file"): 81 | inputfile = arg 82 | mode = 'file' 83 | elif opt in ("-d", "--dir"): 84 | inputdir = arg 85 | mode = 'directory' 86 | elif opt in ("-t", "--timeout"): 87 | timeout = arg 88 | elif opt in ("-u", "--url"): 89 | url = arg 90 | elif opt in ("-p", "--port"): 91 | port = arg 92 | 93 | # if no url or port --> error 94 | if (url == '') or (port == ''): 95 | print "ERROR: url and port are required" 96 | printopts() 97 | sys.exit(2) 98 | 99 | serverurl = 'http://' + url + ':' + port 100 | 101 | # go try to read file and send 102 | if (mode == 'file'): 103 | processFile(inputfile,serverurl) 104 | elif (mode == 'directory'): 105 | filelist = glob.glob(inputdir + '*.json') 106 | if len(filelist): 107 | for afile in filelist: 108 | processFile(afile,serverurl) 109 | time.sleep(float(timeout)) 110 | else: 111 | print "No files of type .json found in directory: " + inputdir 112 | else: 113 | print "unknown mode" 114 | 115 | if __name__ == "__main__": 116 | main(sys.argv[1:]) --------------------------------------------------------------------------------