├── LICENSE.txt ├── README.md ├── bin ├── capture-and-analyze.py ├── trigger-and-analyze.py └── upload-and-analyze.py ├── honeybot ├── __init__.py └── lib │ ├── __init__.py │ ├── const.py │ ├── interfaces.py │ └── utils.py ├── img ├── capture-analyze.gif ├── list-pcaps.gif ├── packettotal.png └── upload-analyze.gif ├── packettotal_sdk ├── __init__.py ├── packettotal_api.py ├── packettotal_sdk.rst └── search_tools.py ├── requirements.txt └── setup.py /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 PacketTotal 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # HoneyBot 2 | ### Cloud-based PCAP analysis; powered by PacketTotal.com 3 | ![PacketTotal Labs](https://raw.githubusercontent.com/PacketTotal/HoneyBot/master/img/packettotal.png) 4 | 5 | ## Description 6 | HoneyBot is a set of scripts and libraries for capturing and analyzing packet captures with PacketTotal.com. 7 | Currently this library provides three scripts: 8 | 9 | - `capture-and-analyze.py` - Capture on an interface for some period of time, and upload capture for analysis. 10 | - `upload-and-analyze.py` - Upload and analyze multiple packet captures to PacketTotal.com. 11 | - `trigger-and-analyze.py` - Listen for unknown connections, and begin capturing when one is made. Captures are automatically uploaded and analyzed. 12 | 13 | 14 | ## Warning 15 | > Any packet capture uploaded to becomes publicly available upon completed analysis. 16 | 17 | ## Limitations 18 | - Only .pcap and .pcapng files supported. 19 | - 6 MB analysis max. 20 | 21 | For more information visit [PacketTotal.com](https://packettotal.com/about.html). 22 | 23 | ## Use Cases 24 | 1. Set your honeypot up to stream network traffic directly to PacketTotal.com for analysis. 25 | 2. Analyze a personal repository of malicious PCAPs. 26 | 3. Determine the benignity of hundreds of packet captures. 27 | 4. Automate analyzing (and sharing) honeypot packet captures. 28 | 5. Automate preliminary malware analysis/triage. 29 | 30 | 31 | 32 | ## Prerequisites: 33 | - [WireShark](https://www.wireshark.org/download.html) must be installed. 34 | - If you are on a linux based operating system you can just install t-shark 35 | - `apt-get install tshark` 36 | - [Python 3.5](https://www.python.org/downloads/) or later is required. 37 | - You must [request an api key](https://packettotal.com/api.html), before you can leverage these scripts. 38 | 39 | 40 | ## Installation 41 | - `pip install -r requirements.txt` 42 | - `python setup.py install` 43 | 44 | 45 | ## Usage 46 | 47 | ### capture-and-analyze.py 48 | 49 | ``` 50 | usage: capture-and-analyze.py [-h] [--seconds SECONDS] [--interface INTERFACE] 51 | [--analyze] [--list-interfaces] [--list-pcaps] 52 | [--export-pcaps] 53 | 54 | Capture, upload and analyze network traffic; powered by PacketTotal.com. 55 | 56 | optional arguments: 57 | -h, --help show this help message and exit 58 | --seconds SECONDS The number of seconds to capture traffic for. 59 | --interface INTERFACE 60 | The name of the interface (--list-interfaces to show 61 | available) 62 | --analyze If included, capture will be uploaded for analysis to 63 | PacketTotal.com. 64 | --list-interfaces Lists the available interfaces. 65 | --list-pcaps Lists pcaps submitted to PacketTotal.com for analysis. 66 | --export-pcaps Writes pcaps submitted to PacketTotal.com for analysis 67 | to a csv file. 68 | ``` 69 | 70 | 71 | ### upload-and-analyze.py 72 | 73 | ``` 74 | usage: upload-and-analyze.py [-h] [--path PATH [PATH ...]] [--analyze] 75 | [--list-pcaps] [--export-pcaps] 76 | 77 | Upload and analyze .pcap/.pcapng files in bulk; powered by PacketTotal.com. 78 | 79 | optional arguments: 80 | -h, --help show this help message and exit 81 | --path PATH [PATH ...] 82 | One or more paths to pcap or directory of pcaps. 83 | --analyze If included, capture will be uploaded for analysis to 84 | PacketTotal.com. 85 | --list-pcaps Lists pcaps submitted to PacketTotal.com for analysis. 86 | --export-pcaps Writes pcaps submitted to PacketTotal.com for analysis 87 | to a csv file. 88 | ``` 89 | 90 | 91 | ### trigger-and-analyze.py 92 | ``` 93 | usage: trigger-and-analyze.py [-h] [--interface INTERFACE] [--learn LEARN] 94 | [--listen] [--capture-seconds CAPTURE_SECONDS] 95 | [--list-interfaces] [--list-pcaps] 96 | [--export-pcaps] 97 | 98 | Listen for unknown connections, and begin capturing when one is made. Captures 99 | are automatically uploaded and analyzed; powered by PacketTotal.com 100 | 101 | optional arguments: 102 | -h, --help show this help message and exit 103 | --interface INTERFACE 104 | The name of the interface (--list-interfaces to show 105 | available) 106 | --learn LEARN The number of seconds from which to build the known 107 | connections whitelist. Connections in this whitelist 108 | will be ignored. 109 | --listen If included, we will begin listening for unknown 110 | connections, and immediately starting a packet capture 111 | and uploading to PacketTotal.com for analysis. 112 | --capture-seconds CAPTURE_SECONDS 113 | The number of seconds worth of network traffic to 114 | capture and analyze after a trigger has fired. 115 | --list-interfaces Lists the available interfaces. 116 | --list-pcaps Lists pcaps submitted to PacketTotal.com for analysis. 117 | --export-pcaps Writes pcaps submitted to PacketTotal.com for analysis 118 | to a csv file. 119 | ``` 120 | 121 | -------------------------------------------------------------------------------- /bin/capture-and-analyze.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | 3 | import argparse 4 | import sys 5 | from time import sleep 6 | 7 | from honeybot.lib import interfaces, utils, const 8 | 9 | 10 | def parse_commandline(): 11 | args = argparse.ArgumentParser(description=const.CAPTURE_AND_ANALYZE_DESC) 12 | args.add_argument( 13 | "--seconds", 14 | help="The number of seconds to capture traffic for.", 15 | type=int, 16 | required='--analyze' in sys.argv 17 | ) 18 | args.add_argument( 19 | "--interface", 20 | help="The name of the interface (--list-interfaces to show available)", 21 | type=str, 22 | required='--analyze' in sys.argv 23 | ) 24 | args.add_argument( 25 | '--analyze', 26 | help="If included, capture will be uploaded for analysis to PacketTotal.com.", 27 | action="store_true" 28 | ) 29 | args.add_argument( 30 | "--list-interfaces", 31 | help="Lists the available interfaces.", 32 | action="store_true" 33 | ) 34 | args.add_argument( 35 | "--list-pcaps", 36 | help="Lists pcaps submitted to PacketTotal.com for analysis.", 37 | action="store_true" 38 | ) 39 | args.add_argument( 40 | '--export-pcaps', 41 | help="Writes pcaps submitted to PacketTotal.com for analysis to a csv file.", 42 | action="store_true" 43 | ) 44 | return args.parse_args() 45 | 46 | 47 | if __name__ == '__main__': 48 | args = parse_commandline() 49 | if len(sys.argv) == 1: 50 | utils.print_pt_ascii_logo() 51 | if args.list_interfaces: 52 | utils.print_network_interaces() 53 | elif args.list_pcaps: 54 | interfaces.print_submission_status() 55 | elif args.export_pcaps: 56 | interfaces.export_submissions_status() 57 | elif args.analyze: 58 | utils.print_analysis_disclaimer() 59 | interfaces.Database().initialize_database() 60 | pcap = interfaces.Capture(args.interface, timeout=args.seconds) 61 | print("Beginning packet capture for {} seconds. Max PacketTotal upload size is 50MB; " 62 | "will terminate if this is reached.".format(args.seconds)) 63 | sleep(2) 64 | pcap.capture() 65 | print('Uploading {} ({} bytes)'.format(pcap.name, pcap.size)) 66 | try: 67 | if pcap.upload(): 68 | pcap.save() 69 | print('Upload complete. Check analysis status with --list-pcaps option') 70 | except Exception: 71 | print("Upload failed!") 72 | sys.exit(1) 73 | sys.exit(0) 74 | -------------------------------------------------------------------------------- /bin/trigger-and-analyze.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | 3 | import argparse 4 | import sys 5 | 6 | from honeybot.lib import interfaces, utils, const 7 | 8 | 9 | def parse_commandline(): 10 | args = argparse.ArgumentParser(description=const.TRIGGER_AND_ANALYZE_DESC) 11 | args.add_argument( 12 | "--interface", 13 | help="The name of the interface (--list-interfaces to show available)", 14 | type=str, 15 | required='--learn' in sys.argv or '--listen' in sys.argv 16 | ) 17 | args.add_argument( 18 | '--learn', 19 | help="The number of seconds from which to build the known connections whitelist. " 20 | "Connections in this whitelist will be ignored.", 21 | type=int 22 | ) 23 | args.add_argument( 24 | '--listen', 25 | help="If included, we will begin listening for unknown connections, " 26 | "and immediately starting a packet capture and uploading to PacketTotal.com for analysis.", 27 | action="store_true" 28 | ) 29 | args.add_argument( 30 | '--capture-seconds', 31 | type=int, 32 | help='The number of seconds worth of network traffic to capture and analyze after a trigger has fired.', 33 | required='--listen' in sys.argv 34 | ) 35 | args.add_argument( 36 | "--list-interfaces", 37 | help="Lists the available interfaces.", 38 | action="store_true" 39 | ) 40 | args.add_argument( 41 | "--list-pcaps", 42 | help="Lists pcaps submitted to PacketTotal.com for analysis.", 43 | action="store_true" 44 | ) 45 | args.add_argument( 46 | '--export-pcaps', 47 | help="Writes pcaps submitted to PacketTotal.com for analysis to a csv file.", 48 | action="store_true" 49 | ) 50 | return args.parse_args() 51 | 52 | 53 | if __name__ == '__main__': 54 | args = parse_commandline() 55 | if len(sys.argv) == 1: 56 | utils.print_pt_ascii_logo() 57 | if args.list_interfaces: 58 | utils.print_network_interaces() 59 | elif args.list_pcaps: 60 | interfaces.print_submission_status() 61 | elif args.export_pcaps: 62 | interfaces.export_submissions_status() 63 | elif args.learn: 64 | interfaces.Trigger(args.interface, capture_period_after_trigger=args.learn).learn(args.learn) 65 | elif args.listen: 66 | interfaces.Database().initialize_database() 67 | interfaces.Trigger(args.interface, capture_period_after_trigger=args.capture_seconds).listen_and_trigger() 68 | sys.exit(0) 69 | -------------------------------------------------------------------------------- /bin/upload-and-analyze.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | 3 | """ 4 | __author__: Jamin Becker (jamin@packettotal.com) 5 | """ 6 | 7 | 8 | import argparse 9 | import os 10 | import sys 11 | 12 | import progressbar 13 | 14 | from honeybot.lib import interfaces, utils, const 15 | 16 | 17 | def parse_commandline(): 18 | args = argparse.ArgumentParser(description=const.UPLOAD_AND_ANALYZE_DESC) 19 | args.add_argument( 20 | '--path', 21 | help='One or more paths to pcap or directory of pcaps.', 22 | nargs='+', 23 | required='--analyze' in sys.argv 24 | ) 25 | args.add_argument( 26 | '--analyze', 27 | help="If included, capture will be uploaded for analysis to PacketTotal.com.", 28 | action="store_true", 29 | required='--path' in sys.argv 30 | ) 31 | args.add_argument( 32 | "--list-pcaps", 33 | help="Lists pcaps submitted to PacketTotal.com for analysis.", 34 | action="store_true" 35 | ) 36 | args.add_argument( 37 | '--export-pcaps', 38 | help="Writes pcaps submitted to PacketTotal.com for analysis to a csv file.", 39 | action="store_true" 40 | ) 41 | return args.parse_args() 42 | 43 | 44 | if __name__ == '__main__': 45 | if len(sys.argv) == 1: 46 | utils.print_pt_ascii_logo() 47 | args = parse_commandline() 48 | analyze_paths = [] 49 | if args.list_pcaps: 50 | interfaces.print_submission_status() 51 | elif args.export_pcaps: 52 | interfaces.export_submissions_status() 53 | if not args.path: 54 | sys.exit(0) 55 | utils.print_analysis_disclaimer() 56 | for path in args.path: 57 | if os.path.isdir(path): 58 | for f in os.listdir(path): 59 | f = os.path.join(path, f) 60 | if not os.path.isfile(f): 61 | continue 62 | with open(f, 'rb') as fh: 63 | if not utils.is_packet_capture(fh.read()): 64 | continue 65 | fh.seek(0) 66 | if len(fh.read(6000000)) > const.PT_MAX_BYTES: 67 | continue 68 | analyze_paths.append(f) 69 | elif os.path.isfile(path): 70 | with open(path, 'rb') as f: 71 | if not utils.is_packet_capture(f.read()): 72 | print('Skipping {} as PCAP not a valid packet capture.'.format(path)) 73 | continue 74 | f.seek(0) 75 | if len(f.read(6000001)) > const.PT_MAX_BYTES: 76 | print('Skipping {} as PCAP is too large to be processed by PacketTotal API.'.format(path)) 77 | continue 78 | analyze_paths.append(path) 79 | interfaces.Database().initialize_database() 80 | for path in progressbar.progressbar(analyze_paths): 81 | pcap = interfaces.Capture(filepath=path) 82 | try: 83 | if pcap.upload(): 84 | pcap.save() 85 | except Exception: 86 | print("Upload failed!") 87 | sys.exit(1) 88 | sys.exit(0) 89 | 90 | 91 | 92 | -------------------------------------------------------------------------------- /honeybot/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/honeybot/__init__.py -------------------------------------------------------------------------------- /honeybot/lib/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/honeybot/lib/__init__.py -------------------------------------------------------------------------------- /honeybot/lib/const.py: -------------------------------------------------------------------------------- 1 | VERSION = '0.7.0' 2 | PT_MAX_BYTES = 5900000 # 5.9 MB (API limit) 3 | CAPTURE_AND_ANALYZE_DESC = 'Capture on an interface for some period of time, and upload capture for analysis.' 4 | TRIGGER_AND_ANALYZE_DESC = 'Listen for unknown connections, and begin capturing when one is made. ' \ 5 | 'Captures are automatically uploaded and analyzed; powered by PacketTotal.com' 6 | UPLOAD_AND_ANALYZE_DESC = 'Upload and analyze .pcap/.pcapng files in bulk; powered by PacketTotal.com.' -------------------------------------------------------------------------------- /honeybot/lib/interfaces.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import csv 4 | import json 5 | import logging 6 | import sqlite3 7 | import warnings 8 | from time import sleep 9 | from datetime import datetime 10 | 11 | import progressbar 12 | import requests 13 | import terminaltables 14 | from IPy import IP 15 | 16 | 17 | from honeybot.lib import const 18 | from honeybot.lib.utils import check_auth 19 | from honeybot.lib.utils import gen_unique_id 20 | from honeybot.lib.utils import listen_on_interface 21 | from honeybot.lib.utils import capture_on_interface 22 | from honeybot.lib.utils import get_filepath_md5_hash 23 | from packettotal_sdk.packettotal_api import PacketTotalApi 24 | 25 | logger = logging.getLogger('honeybot.interfaces') 26 | logger.setLevel(logging.DEBUG) 27 | fh = logging.FileHandler('logs/honeybot.log') 28 | ch = logging.StreamHandler() 29 | ch.setLevel(logging.DEBUG) 30 | fh.setLevel(logging.DEBUG) 31 | formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') 32 | fh.setFormatter(formatter) 33 | ch.setFormatter(formatter) 34 | logger.addHandler(fh) 35 | logger.addHandler(ch) 36 | 37 | 38 | class Capture: 39 | """ 40 | Provides the ability to capture, upload, and save packet captures 41 | """ 42 | 43 | def __init__(self, interface='lo', timeout=60, filepath=None): 44 | ts = datetime.now() 45 | self.interface = interface 46 | self.timeout = timeout 47 | self.path = filepath 48 | self.capture_start = None 49 | self.capture_end = None 50 | self.upload_start = None 51 | self.upload_end = None 52 | self.md5 = None 53 | self.size = None 54 | self.researcher_id = gen_unique_id(interface) 55 | 56 | self.name = 'sc_%s%s%s%s.pcap' % (ts.day, ts.hour, ts.min, ts.second) 57 | 58 | self.auth = check_auth() 59 | if self.path and os.path.isfile(str(self.path)): 60 | self.size = os.path.getsize(self.path) 61 | self.md5 = get_filepath_md5_hash(self.path) 62 | self.name = os.path.basename(self.path) 63 | 64 | def capture(self): 65 | """ 66 | Begin a packet capture 67 | """ 68 | 69 | try: 70 | logger.info("Beginning packet capture for {} seconds.".format(self.timeout)) 71 | self.capture_start = datetime.utcnow() 72 | self.size = capture_on_interface(self.interface, self.name, timeout=self.timeout) 73 | if self.size: 74 | self.capture_end = datetime.utcnow() 75 | self.md5 = get_filepath_md5_hash('tmp/{}'.format(self.name)) 76 | self.path = 'tmp/{}'.format(self.name) 77 | except Exception as e: 78 | logger.error("An error was encountered while capturing on {} - {}".format(self.interface, e), exc_info=True) 79 | 80 | def upload(self): 81 | """ 82 | Begin an upload to public bucket 83 | """ 84 | if self.size == 0: 85 | logger.error("Will not upload PCAP of 0 bytes. {} ({})".format(self.md5, self.name)) 86 | return False 87 | try: 88 | logger.info("Beginning upload to PacketTotal API repo {}".format(self.path)) 89 | 90 | self.upload_start = datetime.utcnow() 91 | if os.path.getsize(self.path) <= const.PT_MAX_BYTES: 92 | PacketTotalApi(api_key=self.auth).analyze(pcap_file_obj=open(self.path, 'rb')) 93 | else: 94 | logger.error('Skipping {} as PCAP is too large to be processed by PacketTotal API.'.format(self.path)) 95 | self.upload_end = datetime.utcnow() 96 | except Exception as e: 97 | logger.error("Failed to complete analysis through PacketTotal API {} - {}".format(self.name, e), 98 | exc_info=True) 99 | raise e 100 | return True 101 | 102 | def save(self): 103 | """ 104 | Save the submission for analysis later 105 | """ 106 | 107 | try: 108 | Database().insert_pcap([ 109 | self.md5, 110 | self.name, 111 | self.capture_start, 112 | self.capture_end, 113 | self.upload_start, 114 | self.upload_end, 115 | self.size 116 | ]) 117 | logger.info('{} saved.'.format(self.md5)) 118 | 119 | return True 120 | except Exception as e: 121 | if 'UNIQUE' in str(e): 122 | logger.warning('Skipping save, we\'ve analyzed this pcap before: {} ({}).'.format(self.md5, 123 | self.name)) 124 | else: 125 | logger.error("Failed to complete database write of {} ({}) - {}".format(self.md5, 126 | self.name, e), exc_info=True) 127 | return False 128 | 129 | def cleanup(self): 130 | """ 131 | Remove the temporary file from your tmp directory 132 | """ 133 | 134 | os.remove('tmp/{}'.format(self.name)) 135 | 136 | 137 | class Database: 138 | """ 139 | Provides a basic CRUD interface for storing submission metadata 140 | """ 141 | 142 | def __init__(self): 143 | self.conn = sqlite3.connect('database.db') 144 | 145 | def initialize_database(self): 146 | """ 147 | Creates a new database instance and default pcaps table (if one does not already exist) 148 | """ 149 | c = self.conn.cursor() 150 | c.execute('''CREATE TABLE IF NOT EXISTS pcaps ( 151 | id VARCHAR(32) PRIMARY KEY, 152 | name VARCHAR(50), 153 | capture_start TEXT, 154 | capture_end TEXT, 155 | upload_start TEXT, 156 | upload_end TEXT, 157 | size INTEGER); 158 | ''') 159 | 160 | c.execute('''CREATE TABLE IF NOT EXISTS completed ( 161 | id VARCHAR(32) PRIMARY KEY, 162 | data TEXT, 163 | FOREIGN KEY(id) REFERENCES pcaps(id) 164 | ); 165 | ''') 166 | self.conn.commit() 167 | 168 | def insert_pcap(self, row): 169 | """ 170 | :param row: A list containing all the items of a completed analysis 171 | [id, name, capture_start, capture_end, upload_start, upload_end, size] 172 | """ 173 | 174 | c = self.conn.cursor() 175 | c.execute('''INSERT INTO pcaps(id, name, capture_start, capture_end, upload_start, upload_end, size) 176 | VALUES(?,?,?,?,?,?,?);''', row) 177 | self.conn.commit() 178 | 179 | def insert_completed(self, row): 180 | c = self.conn.cursor() 181 | c.execute('''INSERT INTO completed(id, data) 182 | VALUES(?,?);''', row) 183 | self.conn.commit() 184 | 185 | def select_pcaps(self): 186 | c = self.conn.cursor() 187 | try: 188 | res = c.execute('SELECT * FROM pcaps;') 189 | except Exception: 190 | logger.info('No PCAPs analyzed yet. Submit your first pcap before using this option.') 191 | exit(0) 192 | return res 193 | 194 | def select_completed(self, _id): 195 | c = self.conn.cursor() 196 | res = c.execute("SELECT * FROM completed WHERE id='{}';".format(_id)) 197 | return res 198 | 199 | 200 | class PTClient: 201 | """ 202 | Provides a simple interface for retrieving information about a submission 203 | """ 204 | 205 | def __init__(self): 206 | self.base = "https://packettotal.com" 207 | self.useragent = 'honeybot Client Version {}'.format(const.VERSION) 208 | self.session = requests.session() 209 | 210 | def get_pcap_status(self, _id): 211 | """ 212 | :param _id: The md5 has associated with the pcap file 213 | :return: a submission object if found, None otherwise 214 | """ 215 | url = self.base + '/app/submission/status?id={}'.format(_id) 216 | res = requests.get(url, headers={ 217 | 'User-Agent': self.useragent, 218 | 'Accept': 'text/json' 219 | }) 220 | suc, data = res.json() 221 | if suc: 222 | return data 223 | return None 224 | 225 | 226 | def export_submissions_status(): 227 | """ 228 | #Exports the results (analysis statuses) of all submissions to a csv 229 | """ 230 | with open('pcap-statuses.csv', 'w') as f: 231 | writer = csv.writer(f, dialect='excel') 232 | table = [['Capture MD5', 233 | 'Capture Name', 234 | 'Capture Start', 235 | 'Capture End', 236 | 'Upload Start', 237 | 'Upload End', 238 | 'Size', 239 | 'Queued', 240 | 'Analysis Started', 241 | 'Analysis Completed', 242 | 'Malicious', 243 | 'Link']] 244 | table.extend(get_submissions_status()) 245 | print('\n=== Written to {} ==='.format('pcap-statuses.csv')) 246 | writer.writerows(table) 247 | 248 | 249 | def get_submissions_status(): 250 | """ 251 | :return: A list of statuses of all the submissions in the database 252 | """ 253 | results = [] 254 | database = Database() 255 | print("Fetching analysis statuses...Please wait.") 256 | for row in progressbar.progressbar(Database().select_pcaps()): 257 | _id, name, capture_start, capture_end, upload_start, upload_end, size = row 258 | try: 259 | raw_result = next(database.select_completed(_id)) 260 | res = json.loads(raw_result[1]) 261 | except StopIteration: 262 | res = PTClient().get_pcap_status(_id) 263 | if res and res.get('analysisCompleted'): 264 | try: 265 | database.insert_completed([_id, json.dumps(res)]) 266 | except Exception as e: 267 | logger.warning('Could not cache status for {} - {}'.format(_id, e)) 268 | queued, analysis_started, analysis_completed = False, False, False 269 | link = None 270 | malicious = None 271 | if res: 272 | submission = res.get('submission', {}) 273 | if submission.get('queuedTimestamp'): 274 | queued = True 275 | if submission.get('analysisStarted'): 276 | analysis_started = True 277 | if submission.get('analysisCompleted'): 278 | analysis_completed = True 279 | if 'signature_alerts' in submission.get('logsTransmitted'): 280 | malicious = True 281 | else: 282 | malicious = False 283 | if analysis_completed: 284 | link = "https://packettotal.com/app/analysis?id={}".format(_id) 285 | results.append([_id, name, capture_start, capture_end, upload_start, upload_end, size, queued, analysis_started, 286 | analysis_completed, malicious, link]) 287 | return results 288 | 289 | 290 | def print_submission_status(): 291 | """ 292 | Prints a formatted table of submitted PCAPs 293 | """ 294 | table = [['Capture MD5', 295 | 'Capture Name', 296 | 'Capture Start', 297 | 'Capture End', 298 | 'Upload Start', 299 | 'Upload End', 300 | 'Size', 301 | 'Queued', 302 | 'Analysis Started', 303 | 'Analysis Completed', 304 | 'Malicious', 305 | 'Link']] 306 | table.extend(get_submissions_status()) 307 | print(terminaltables.AsciiTable(table).table) 308 | 309 | 310 | class Trigger: 311 | """ 312 | Provides a simple interface which listens for pcaps and performs a capture, when an unknown connection is made. 313 | """ 314 | 315 | def __init__(self, interface, capture_period_after_trigger=60): 316 | self.interface = interface 317 | self.capture_period_after_trigger = capture_period_after_trigger 318 | self.whitelisted_ips = [] 319 | self._open_whitelist() 320 | 321 | def _open_whitelist(self): 322 | """ 323 | Open the ip.whitelist file 324 | """ 325 | try: 326 | with open('ip.whitelist', 'r') as f: 327 | self.whitelisted_ips = [line.strip() for line in f.readlines() if line.strip() != ''] 328 | except FileNotFoundError: 329 | self.whitelisted_ips = [] 330 | 331 | def learn(self, timeout=60): 332 | """ 333 | Builds a whitelist of IP addresses for every connection captured during this time-period 334 | 335 | :param timeout: The number of seconds to capture traffic 336 | """ 337 | 338 | src_ips = set() 339 | dst_ips = set() 340 | 341 | with open('ip.whitelist', 'w') as f: 342 | if not sys.warnoptions: 343 | warnings.simplefilter("ignore") 344 | print('Generating whitelist of IP addresses based on traffic from the next {} seconds.'.format(timeout)) 345 | bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength) 346 | for conn in self.listener(timeout=timeout): 347 | try: 348 | src, dst, proto = conn 349 | if IP(src).iptype() == 'PUBLIC': 350 | src_ips.add(src) 351 | bar.update(len(src_ips) + len(dst_ips)) 352 | if IP(dst).iptype() == 'PUBLIC': 353 | dst_ips.add(dst) 354 | bar.update(len(src_ips) + len(dst_ips)) 355 | except AttributeError: 356 | pass 357 | all_ips = list(src_ips) 358 | all_ips.extend(dst_ips) 359 | all_ips = set(all_ips) 360 | for ip in all_ips: 361 | f.write(ip + '\n') 362 | 363 | def listener(self, timeout=None): 364 | for packet in listen_on_interface(interface=self.interface, timeout=timeout): 365 | try: 366 | yield packet.ip.src, packet.ip.dst, packet.transport_layer 367 | except AttributeError: 368 | continue 369 | 370 | def listen_and_trigger(self): 371 | """ 372 | Begin listening for unknown connections, 373 | start a capture and upload for analysis if one is detected 374 | """ 375 | suppress = None # We don't want to trigger on the same IP twice in a row 376 | while True: 377 | for conn in self.listener(timeout=None): 378 | src, dst, _ = conn 379 | trigger = None 380 | if src not in self.whitelisted_ips: 381 | if src == suppress: 382 | break 383 | if IP(src).iptype() == 'PUBLIC': 384 | logger.info("Trigger [Source: {} not in whitelist] Capturing for {} seconds" 385 | .format(src, self.capture_period_after_trigger)) 386 | trigger = src 387 | elif dst not in self.whitelisted_ips: 388 | if dst == suppress: 389 | break 390 | if IP(dst).iptype() == 'PUBLIC': 391 | logger.info("Trigger [Destination {} not in whitelist] Capturing for {} seconds" 392 | .format(src, self.capture_period_after_trigger)) 393 | trigger = dst 394 | if trigger: 395 | capture = Capture(self.interface, timeout=self.capture_period_after_trigger) 396 | capture.capture() 397 | suppress = trigger 398 | try: 399 | if capture.upload(): 400 | capture.save() 401 | logger.info('Upload complete') 402 | except Exception: 403 | logger.error('Upload Failed') 404 | # We don't want to upload packets that we already captured (and analyzed), 405 | # so we break out of this inner loop 406 | break 407 | # We don't want to trigger on API upload, buffer tends to be a bit backed up 408 | sleep(30) 409 | -------------------------------------------------------------------------------- /honeybot/lib/utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | __author__: Jamin Becker (jamin@packettotal.com) 3 | """ 4 | 5 | import os 6 | import sys 7 | import time 8 | import socket 9 | import logging 10 | import pathlib 11 | import warnings 12 | from hashlib import md5 13 | 14 | 15 | import psutil 16 | import pyshark 17 | import progressbar 18 | from magic import from_buffer 19 | from terminaltables import AsciiTable 20 | from packettotal_sdk.packettotal_api import PacketTotalApi 21 | 22 | 23 | from honeybot.lib import const 24 | 25 | 26 | def capture_on_interface(interface, name, timeout=60): 27 | """ 28 | :param interface: The name of the interface on which to capture traffic 29 | :param name: The name of the capture file 30 | :param timeout: A limit in seconds specifying how long to capture traffic 31 | """ 32 | 33 | if timeout < 15: 34 | logger.error("Timeout must be over 15 seconds.") 35 | return 36 | if not sys.warnoptions: 37 | warnings.simplefilter("ignore") 38 | start = time.time() 39 | widgets = [ 40 | progressbar.Bar(marker=progressbar.RotatingMarker()), 41 | ' ', 42 | progressbar.FormatLabel('Packets Captured: %(value)d'), 43 | ' ', 44 | progressbar.Timer(), 45 | ] 46 | progress = progressbar.ProgressBar(widgets=widgets) 47 | capture = pyshark.LiveCapture(interface=interface, output_file=os.path.join('tmp', name)) 48 | pcap_size = 0 49 | for i, packet in enumerate(capture.sniff_continuously()): 50 | progress.update(i) 51 | if os.path.getsize(os.path.join('tmp', name)) != pcap_size: 52 | pcap_size = os.path.getsize(os.path.join('tmp', name)) 53 | if not isinstance(packet, pyshark.packet.packet.Packet): 54 | continue 55 | if time.time() - start > timeout: 56 | break 57 | if pcap_size > const.PT_MAX_BYTES: 58 | break 59 | capture.clear() 60 | capture.close() 61 | return pcap_size 62 | 63 | 64 | def check_auth(): 65 | home = str(pathlib.Path.home()) 66 | key = '' 67 | auth_path = os.path.join(home, 'honeybot.auth') 68 | if not os.path.exists(auth_path): 69 | print('HoneyBot requires a PacketTotal API key.') 70 | print('Signup at: \n\t: https://packettotal.com/api.html\n') 71 | else: 72 | key = open(auth_path, 'r').read() 73 | while PacketTotalApi(key).usage().status_code == 403: 74 | print('Invalid API Key. Try again.') 75 | key = input('API Key: ') 76 | open(auth_path, 'w').write(key) 77 | 78 | return open(auth_path, 'r').read() 79 | 80 | 81 | def get_filepath_md5_hash(file_path): 82 | """ 83 | :param file_path: path to the file being hashed 84 | :return: the md5 hash of a file 85 | """ 86 | with open(file_path, 'rb') as afile: 87 | return get_file_md5_hash(afile) 88 | 89 | 90 | def get_str_md5_hash(s): 91 | return md5(str(s).encode('utf-8')).hexdigest() 92 | 93 | 94 | def get_mac_address_of_interface(interface): 95 | """ 96 | :param interface: The friendly name of a network interface 97 | :return: the MAC address associated with that interface 98 | """ 99 | for k,v in psutil.net_if_addrs().items(): 100 | if interface == k: 101 | for item in v: 102 | try: 103 | if item.family == socket.AF_LINK: 104 | return item.address 105 | except AttributeError: 106 | # Linux 107 | if item.family == socket.AF_PACKET: 108 | return item.address 109 | return None 110 | 111 | 112 | def gen_unique_id(interface): 113 | """ 114 | Generates a unique ID based on your MAC address that will be used to tag all PCAPs uploaded to PacketTotal.com 115 | This ID can be used to search and view PCAPs you have uploaded. 116 | 117 | :param interface: The friendly name of a network interface 118 | :return: A unique id 119 | """ 120 | mac_address = get_mac_address_of_interface(interface) 121 | if mac_address: 122 | return get_str_md5_hash(get_str_md5_hash(mac_address))[0:15] 123 | return None 124 | 125 | 126 | def get_file_md5_hash(fh): 127 | """ 128 | :param fh: file handle 129 | :return: the md5 hash of the file 130 | """ 131 | block_size = 65536 132 | md5_hasher = md5() 133 | buf = fh.read(block_size) 134 | while len(buf) > 0: 135 | md5_hasher.update(buf) 136 | buf = fh.read(block_size) 137 | return md5_hasher.hexdigest() 138 | 139 | 140 | def get_network_interfaces(): 141 | """ 142 | :return: A list of valid interfaces and their addresses 143 | """ 144 | return psutil.net_if_addrs().items() 145 | 146 | 147 | def is_packet_capture(bytes): 148 | """ 149 | :param bytes: raw bytes 150 | :return: True is valid pcap or pcapng file 151 | """ 152 | result = from_buffer(bytes) 153 | valid = "pcap-ng" in result or "tcpdump" in result or "NetMon" in result or 'pcap capture file' in result 154 | return valid 155 | 156 | 157 | def mkdir_p(path): 158 | """ 159 | :param path: Path to the new directory to create 160 | """ 161 | 162 | pathlib.Path(path).mkdir(parents=True, exist_ok=True) 163 | 164 | 165 | def listen_on_interface(interface, timeout=60): 166 | """ 167 | :param interface: The name of the interface on which to capture traffic 168 | :return: generator containing live packets 169 | """ 170 | 171 | start = time.time() 172 | capture = pyshark.LiveCapture(interface=interface) 173 | 174 | for item in capture.sniff_continuously(): 175 | if timeout and time.time() - start > timeout: 176 | break 177 | yield item 178 | 179 | 180 | def print_network_interaces(): 181 | """ 182 | :return: Prints a human readable representation of the available network interfaces 183 | """ 184 | 185 | for intf, items in get_network_interfaces(): 186 | table = [["family", "address", "netmask", "broadcast", "ptp"]] 187 | for item in items: 188 | family, address, netmask, broadcast, ptp = item 189 | table.append([str(family), str(address), str(netmask), str(broadcast), str(ptp)]) 190 | print(AsciiTable(table_data=table, title=intf).table) 191 | print('\n') 192 | 193 | 194 | def print_analysis_disclaimer(): 195 | print(""" 196 | WARNING: Analysis will result in the network traffic becoming public at https://packettotal.com. 197 | 198 | ADVERTENCIA: El análisis hará que el tráfico de la red se haga público en https://packettotal.com. 199 | 200 | WARNUNG: Die Analyse führt dazu, dass der Netzwerkverkehr unter https://packettotal.com öffentlich wird. 201 | 202 | ПРЕДУПРЕЖДЕНИЕ. Анализ приведет к тому, что сетевой трафик станет общедоступным на https://packettotal.com. 203 | 204 | चेतावनी: विश्लेषण का परिणाम नेटवर्क ट्रैफिक https://packettotal.com पर सार्वजनिक हो जाएगा 205 | 206 | 警告：分析将导致网络流量在https://packettotal.com上公开 207 | 208 | 警告：分析により、ネットワークトラフィックはhttps://packettotal.comで公開されます。 209 | 210 | tahdhir: sayuadiy altahlil 'iilaa 'an tusbih harakat murur alshabakat eamat ealaa https://packettotal.com 211 | 212 | """) 213 | 214 | answer = input('Continue? [Y/n]: ') 215 | if answer.lower() == 'n': 216 | exit(0) 217 | 218 | 219 | def print_pt_ascii_logo(): 220 | """ 221 | :return: Prints a PacketTotal.com (visit https://PacketTotal.com!) 222 | """ 223 | 224 | logo = """ 225 | ,,*****, ,****, 226 | ****/**, / ,*//*, 227 | ****/**, ,,**********,. ****** 228 | .. .,*****, .. ******** ************,. . 229 | . .,,***, ,******, .,***,,.. *****////***, ,***, 230 | ... ,******, ,******, .,******** *****////***,. ****, 231 | .,,****, ,******, ,******, ,******** ************, ./// 232 | /////* // ////// /( ////// /( *///// ************ 233 | ****,. ,,******. ,******. ,******, .******,,.,******,. * 234 | *****, ********, ,******, ,******, .,********.,******** ,*******, 235 | *****, ********, ,******, ,******, .,********.,******** ****//**, 236 | *****, /*******, ,******* *//////* ********/ ,******** ****//**, 237 | ////* / .////// ((.(((((( ## (((((( (& /((((( % ////// ,******** . 238 | ,,,,,. ,,,,,,,,. ,,*****, ,******, ,*****,,,..,,,,,,,. .,,,,,, 239 | *****, /*******, *//////* *//////* .*//////*/.*********,,*****, 240 | *****, /*******, *//////* (. ( *////////.*********,,****** 241 | *****, /******** */////(* ..,,,,,,.. (/////// ********* ******* 242 | ////* / ,/((((/ #@,####*..,**********,. /####. & /(((((. @ *//// 243 | ,,,,,. ..,,,,*,, ,**, ,*****////*****,./***,. ,,,,,,,.. .,,,,, 244 | ,***, /*****//* */// .,***//(##((/****, /////,*//******,,***, 245 | ****, /*****//* *//* ,****/(%&&%(//**,, ////(,*//******,,**** 246 | ****, /*****//* *//* ,***//(##(//***** *///( *//******.***, 247 | *** ( **///// #@//((. ******////****** /(((* @ //////* **, 248 | ,,. ..,,,,,,, ,****,. ************ .,****,. ,,,,,,,.. ,,* 249 | *, /******** *//////, ,****, ,////////.*********,,* 250 | * /*******, *//////* ,******, .*////////.*********,, 251 | /******** *//////* *//////* *//////*/ ********* 252 | *******,# ////////# ////////# //////* /****** 253 | ,,,,,..((.,,,,,,.(#.,,,,,,.(#.,,,,,,..(..,,,,, 254 | * *****, ,******, *//////* .,*******/.,***** 255 | ***, ,******, ,******, .,********.,*** / 256 | * ,*, ,******, ,******, ,********.,* . 257 | *******/ *******/ ,******* , 258 | ,,,,,..// .,,,,..// .,,,,, 259 | / .****, ,******, .,****, / 260 | / *** ,******, *** 261 | ******** # 262 | 263 | - honeybot: Capture and analyze network traffic; powered by PacketTotal.com 264 | : VERSION: {} 265 | """.format(const.VERSION) 266 | print(logo) 267 | 268 | 269 | 270 | # Setup Logging 271 | 272 | mkdir_p('logs/') 273 | mkdir_p('tmp/') 274 | logger = logging.getLogger('honeybot.utils') 275 | logger.setLevel(logging.DEBUG) 276 | fh = logging.FileHandler('logs/honeybot.log') 277 | ch = logging.StreamHandler() 278 | ch.setLevel(logging.DEBUG) 279 | fh.setLevel(logging.DEBUG) 280 | formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') 281 | fh.setFormatter(formatter) 282 | ch.setFormatter(formatter) 283 | logger.addHandler(fh) 284 | logger.addHandler(ch) -------------------------------------------------------------------------------- /img/capture-analyze.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/img/capture-analyze.gif -------------------------------------------------------------------------------- /img/list-pcaps.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/img/list-pcaps.gif -------------------------------------------------------------------------------- /img/packettotal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/img/packettotal.png -------------------------------------------------------------------------------- /img/upload-analyze.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/img/upload-analyze.gif -------------------------------------------------------------------------------- /packettotal_sdk/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacketTotal/HoneyBot/3843ec6d684786091ced053857d1718ef1fa495c/packettotal_sdk/__init__.py -------------------------------------------------------------------------------- /packettotal_sdk/packettotal_api.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import typing 4 | import base64 5 | import requests 6 | 7 | PT_API_BASE = os.environ.get('PACKETTOTAL_API_BASE_URL') 8 | PT_API_VERSION_STRING = os.environ.get('PACKETTOTAL_API_VERSION_STRING') 9 | 10 | 11 | class PacketTotalApi: 12 | """ 13 | Client that provides search/analysis capabilities 14 | 15 | Sign up here: https://www.packettotal.com/api.html 16 | """ 17 | 18 | def __init__(self, api_key: str): 19 | """ 20 | :param api_key: An API authentication token 21 | """ 22 | self.headers = { 23 | 'x-api-key': api_key 24 | } 25 | self.base_url = 'https://api.packettotal.com/' 26 | self.version = 'v1' 27 | if PT_API_BASE: 28 | self.base_url = PT_API_BASE 29 | if PT_API_VERSION_STRING: 30 | self.version = PT_API_VERSION_STRING 31 | 32 | def analyze(self, pcap_file_obj: typing.BinaryIO, pcap_name=None, pcap_sources=None) -> requests.Response: 33 | """ 34 | Publicly analyze a PCAP file 35 | 36 | :param pcap_file_obj: A file like object that provides a .read() interface (E.G open('path_to_pcap.pcap, 'rb') ) 37 | :param pcap_name: The optional name of the pcap file, if none is given the md5 hash of the PCAP is used 38 | :param pcap_sources: The optional list of URLs referencing making reference to the PCAP file 39 | :return: A request.Response instance, containing information such as where the finished analysis can be found 40 | """ 41 | pcap_base64 = base64.b64encode(pcap_file_obj.read()) 42 | pcap_base64 = pcap_base64.decode('utf-8') 43 | 44 | self.headers['Content-Type'] = 'application/json' 45 | body = { 46 | 'pcap_base64': pcap_base64 47 | } 48 | 49 | if pcap_name: 50 | body['pcap_name'] = pcap_name 51 | 52 | if isinstance(pcap_sources, list): 53 | body['sources'] = pcap_sources 54 | 55 | response = requests.post( 56 | self.base_url + self.version + '/analyze/base64', 57 | headers=self.headers, 58 | data=json.dumps(body) 59 | ) 60 | return response 61 | 62 | def search(self, query: str, pretty=False) -> requests.Response: 63 | """ 64 | Search with term or with a valid Lucene query. 65 | https://www.packettotal.com/api-docs/#/search/get_search 66 | 67 | :param query: A search term, such as an IP address or file hash. 68 | :param pretty: True, if you wish the response.text to be human readable 69 | :return: A request.Response instance, containing the results of the search query 70 | """ 71 | pretty_str = '&pretty={}'.format(str(pretty).lower()) 72 | response = requests.get( 73 | self.base_url + self.version + '/search?query={0}{1}"'.format(query, pretty_str), 74 | headers=self.headers 75 | ) 76 | 77 | return response 78 | 79 | def deep_search_create(self, query: str) -> requests.Response: 80 | """ 81 | Create a new deep search task. Search for a term or with a Lucene query. 82 | Deep searches run longer and can return more results than a normal search. 83 | The are executed asyncronously, and results must be fetched later using the resulting search_id 84 | https://www.packettotal.com/api-docs/#/search/post_search_deep 85 | 86 | :param query: A search term, such as an IP address or file hash. 87 | :return: A request.Response instance, containing the corresponding search_id, which can be used to retrieve 88 | results at a later point 89 | """ 90 | body = { 91 | 'query': query 92 | } 93 | response = requests.post( 94 | self.base_url + self.version + '/search/deep', 95 | headers=self.headers, 96 | data=json.dumps(body) 97 | ) 98 | 99 | return response 100 | 101 | def deep_search_get(self, search_id: str, pretty=False) -> requests.Response: 102 | """ 103 | Get the results from a deep search task. 104 | https://www.packettotal.com/api-docs/#/search/get_search_deep_results__search_id_ 105 | 106 | :param search_id: An id corresponding to the search you previously created. 107 | :param pretty: True, if you wish the response.text to be human readable 108 | :return: A request.Response instance, containing the results of the initial deep search query 109 | """ 110 | pretty_str = '?pretty={}'.format(str(pretty).lower()) 111 | response = requests.get( 112 | self.base_url + self.version + '/search/deep/results/{0}{1}'.format(search_id, pretty_str), 113 | headers=self.headers 114 | ) 115 | 116 | return response 117 | 118 | def pcap_analysis(self, pcap_md5: str) -> requests.Response: 119 | """ 120 | Get a detailed report of PCAP traffic, carved files, signatures, and top-talkers. 121 | https://www.packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id__analysis 122 | 123 | :param pcap_md5: An md5 hash corresponding to the PCAP file submission on PacketTotal.com 124 | :return: A request.Response instance, containing more detailed information about the contents of a PCAP file 125 | """ 126 | response = requests.get( 127 | self.base_url + self.version + '/pcaps/{0}/analysis'.format(pcap_md5), 128 | headers=self.headers 129 | ) 130 | 131 | return response 132 | 133 | def pcap_download(self, pcap_md5: str) -> requests.Response: 134 | """ 135 | Download a PCAP analysis archive. The result is a zip archive containing the PCAP itself, CSVs representing 136 | various analysis results, and all carved files. 137 | https://www.packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id__download 138 | 139 | :param pcap_md5: An md5 hash corresponding to the PCAP file submission on PacketTotal.com 140 | :return: A request.Response instance, containing the download zip archive 141 | """ 142 | response = requests.get( 143 | self.base_url + self.version + '/pcaps/{0}/download'.format(pcap_md5), 144 | headers=self.headers 145 | ) 146 | 147 | return response 148 | 149 | def pcap_info(self, pcap_md5: str) -> requests.Response: 150 | """ 151 | Get high-level information about a specific PCAP file. 152 | https://www.packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id_ 153 | 154 | :param pcap_md5: An md5 hash corresponding to the PCAP file submission on PacketTotal.com 155 | :return: A request.Response instance, containing high-level metadata about a PCAP submission 156 | """ 157 | response = requests.get( 158 | self.base_url + self.version + '/pcaps/{0}'.format(pcap_md5), 159 | headers=self.headers 160 | ) 161 | 162 | return response 163 | 164 | def pcap_similar(self, pcap_md5: str, intensity='low', weighting_mode='behavior', 165 | prioritize_uncommon_fields=False, pretty=False) -> requests.Response: 166 | """ 167 | Get a similarity graph relative to the current PCAP file. 168 | https://www.packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id__similar 169 | 170 | :param pcap_md5: An md5 hash corresponding to the PCAP file submission on PacketTotal.com 171 | :param intensity: [minimal|low|medium|high] The scope of the search, basically translates to the maximum number 172 | of aggregations to exhaust. 173 | :param weighting_mode: [behavior|content] Weight search results either based on their similarity to the 174 | behaviors exhibited or contents contained within the current PCAP file. 175 | :param prioritize_uncommon_fields: By default, the most common values are used to seed the initial similarity 176 | search. Enabling this parameter, seeds the initial search with the least common values instead. 177 | :param pretty: True, if you wish the response.text to be human readable 178 | :return: A request.Response instance, containing a graph of similar pcaps with matched terms 179 | """ 180 | intensity_str = '?intensity={}'.format(intensity) 181 | weighting_mode_str = '&weighting_mode={}'.format(weighting_mode) 182 | prioritize_uncommon_fields_str = '&prioritize_uncommon_fields={}'.format( 183 | str(prioritize_uncommon_fields).lower()) 184 | pretty_str = '&pretty={}'.format(str(pretty).lower()) 185 | 186 | response = requests.get( 187 | self.base_url + self.version + '/pcaps/{0}/similar{1}{2}{3}{4}'.format( 188 | pcap_md5, intensity_str, weighting_mode_str, prioritize_uncommon_fields_str, pretty_str), 189 | headers=self.headers 190 | ) 191 | 192 | return response 193 | 194 | def usage(self) -> requests.Response: 195 | """ 196 | Retrieve API usage and subscription plan information. 197 | https://www.packettotal.com/api-docs/#/usage/get_usage 198 | 199 | :return: A request.Response instance, containing information about requests made, and your current subscription 200 | """ 201 | response = requests.get( 202 | self.base_url + self.version + '/usage', 203 | headers=self.headers 204 | ) 205 | 206 | return response 207 | 208 | def set_version(self, version: str) -> None: 209 | """ 210 | Set the API version 211 | 212 | :param version: The version prefix for the API (E.G v1) 213 | """ 214 | self.version = version 215 | 216 | def set_api_key(self, api_key) -> None: 217 | """ 218 | Set the API key 219 | 220 | :param api_key: An API authentication token 221 | """ 222 | self.headers = { 223 | 'x-api-key': api_key 224 | } 225 | -------------------------------------------------------------------------------- /packettotal_sdk/packettotal_sdk.rst: -------------------------------------------------------------------------------- 1 | PacketTotal API Module 2 | ====================== 3 | 4 | This module provides a basic interface for working with the PacketTotal API. 5 | 6 | .. automodule:: packettotal_sdk.packettotal_api 7 | :members: 8 | -------------------------------------------------------------------------------- /packettotal_sdk/search_tools.py: -------------------------------------------------------------------------------- 1 | import time 2 | import typing 3 | import requests 4 | from sys import stderr 5 | from datetime import datetime 6 | 7 | 8 | from packettotal_sdk import packettotal_api 9 | 10 | 11 | class SearchTools(packettotal_api.PacketTotalApi): 12 | 13 | def __init__(self, api_key: str): 14 | """ 15 | :param api_key: An API authentication token 16 | """ 17 | super().__init__(api_key) 18 | 19 | def search_by_pcap(self, pcap_file_obj: typing.BinaryIO) -> requests.Response: 20 | """ 21 | Search by a pcap/pcapng file, get list list of similar packet captures 22 | 23 | :param pcap_file_obj: A file like object that provides a .read() interface (E.G open('path_to_pcap.pcap, 'rb') ) 24 | :return: A request.Response instance, containing a graph of similar pcaps with matched terms 25 | """ 26 | response = super().analyze(pcap_file_obj) 27 | if response.status_code == 200: 28 | sim_response = super().pcap_similar(response.json()['pcap_metadata']['md5']) 29 | elif response.status_code == 202: 30 | pcap_id = response.json()['id'] 31 | info_response = super().pcap_info(pcap_id) 32 | while info_response.status_code == 404: 33 | print('[{}] Waiting for {} to finish analyzing.'.format(datetime.utcnow(), pcap_id)) 34 | info_response = super().pcap_info(response.json()['id']) 35 | time.sleep(10) 36 | print('[{}] Fetching results for {}.'.format(datetime.utcnow(), pcap_id)) 37 | time.sleep(5) 38 | sim_response = super().pcap_similar(response.json()['id']) 39 | else: 40 | return response 41 | return sim_response 42 | 43 | def search_by_iocs(self, ioc_file: typing.TextIO) -> requests.Response: 44 | """ 45 | Search up to 100 IOC terms at once, and get matching packet captures 46 | 47 | :param ioc_file: A file like object that provides a .read() interface (E.G open('path_to_iocs.txt, 'r') 48 | contents are line delim 49 | :return: A request.Response instance containing the search results containing at least one matching IOC 50 | """ 51 | text = ioc_file.read() 52 | delim = '\n' 53 | if '\r\n' in text[0:2048]: 54 | delim = '\r\n' 55 | elif '\r' in text[0:2048]: 56 | delim = '\r' 57 | elif ',' in text[0:2048]: 58 | delim = ',' 59 | elif '\t' in text[0:2048]: 60 | delim = '\t' 61 | text_delimd = text.split(delim) 62 | search_str = '' 63 | for i, ioc in enumerate(text_delimd[0: -2]): 64 | search_str += '"{}" OR '.format(ioc.strip()) 65 | if i > 100: 66 | print('Warning searching only the first 100 IOC terms of {}.'.format(len(text_delimd)), file=stderr) 67 | break 68 | search_str += '"{}"'.format(text_delimd[-1].strip()) 69 | response = super().search(search_str) 70 | return response 71 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | IPy==0.83 2 | progressbar2==3.38.0 3 | psutil==5.4.7 4 | python_magic==0.4.15 5 | pyshark==0.4.1 6 | requests==2.20.0 7 | terminaltables==3.1.0 -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from distutils.core import setup 2 | 3 | 4 | def parse_requirements(): 5 | """ 6 | load requirements from a pip requirements file 7 | """ 8 | 9 | lineiter = (line.strip() for line in open('requirements.txt')) 10 | return [line for line in lineiter if line and not line.startswith("#")] 11 | 12 | 13 | install_reqs = parse_requirements() 14 | 15 | setup( 16 | name='honeybot', 17 | version='0.5.0', 18 | packages=['honeybot.lib', 'packettotal_sdk'], 19 | scripts=['bin/capture-and-analyze.py', 'bin/trigger-and-analyze.py', 'bin/upload-and-analyze.py'], 20 | url='https://packettotal.com', 21 | license='MIT', 22 | author='Jamin Becker', 23 | author_email='jamin@packettotal.com', 24 | description='A suite of utilities providing the ability to do bulk network analysis with PacketTotal.com', 25 | install_reqires=install_reqs 26 | ) 27 | --------------------------------------------------------------------------------