├── LICENSE ├── README.md ├── firstorder.py └── firstorder_slide.pdf /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2018, Utku Sen 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ``` 2 | 3 | .==. A traffic analyzer to evade Empire's communication 4 | ()''()-. __ _ _ _ 5 | .---. ;--; / / _(_) | | | | 6 | .'_:___'. _..'. __'. | |_ _ _ __ ___| |_ ___ _ __ __| | ___ _ __ 7 | |__ --==|'-''' /'...; | _| | '__/ __| __/ _ \| '__/ _` |/ _ \ '__| 8 | [ ] :[| |---/ | | | | | \__ \ || (_) | | | (_| | __/ | 9 | |__| =[| .' '. |_| |_|_| |___/\__\___/|_| \__,_|\___|_| 10 | / / ____| : '._ 11 | |-/.____.' | : : by Utku Sen, Gozde Sinturk 12 | /___\ /___\ '-'._----' TEAR Security 13 | 14 | 15 | ``` 16 | 17 |

18 | 19 |

20 | 21 | Slides from DEF CON 26 - Packet Hacking Village: https://www.slideshare.net/utkusen/normalizing-empires-traffic-to-evade-anomalybased-ids 22 | 23 | A more detailed blog post: https://utkusen.com/blog/bypassing-anomaly-based-nids-with-empire.html 24 | 25 | ## Abstract 26 | 27 | firstorder is designed to evade Empire's C2-Agent communication from anomaly-based intrusion detection systems. It takes a traffic capture file (pcap) of the network and tries to identify normal traffic profile. According to results, it creates an Empire HTTP listener with appropriate options. 28 | 29 | ## The Idea 30 | 31 | Anomaly-based NIDS refers to building a statistical model describing the normal network traffic, and flagging the abnormal traffic. Anomaly-based systems have the advantage of detecting zero-day attacks, since these attacks will be distinguishable from normal traffic. On the other hand, anomaly-based systems require a training phase in order to identify normal network traffic. It is possible to mislead learning algorithm of an anomaly-based system by poisoning the initial data. However in a real-world scenario, it's hard for an attacker to know, when the network is being trained for anomaly detection purposes. Because of that, we have to guess the normal traffic profile. 32 | 33 | Empire is a PowerShell and Python based post-exploitation framework which is designed for "assume breach" type of activities. We can describe Empire's workflow in two parts: Agent and listener. Agent states the infected machine on the network which takes and executes given tasks on there. Listener is described as a communication server (C2) in which agent connects there and gets it's task, sends output of the tasks. 34 | 35 | We can list following options of the listener which can be insight of an anomaly-based NIDS: 36 | 37 | 1) Request URI: Agent makes it's connection to the C2 server with a GET request to a specific URI (for example:"/read.php") If only html or aspx pages are in use in the local network, this "php" extension may flagged by the anomaly detection system 38 | 39 | 2) User-agent value: User-agent value defines the operating system and browser choice of the agent. For example if all users on the 40 | network uses Microsoft Windows with Chrome, setting user-agent value to macOS with Safari may flagged by the anomaly detection system. 41 | 42 | 3) Server header: Server header value defines the web server type of the C2. For example if all the servers on the network are using 43 | Linux, setting server header as "Microsoft-IIS" may flagged by the anomaly detection system 44 | 45 | 4) Port: If only common ports like 80, 443, 8080 are used in the network, selecting communication port as 5839 may flagged by the anomaly 46 | detection system 47 | 48 | 5) Connection Interval (DefaultDelay): By default, agent will send heartbeat request to the C2 server in every 5 seconds. If regular 49 | users are not connecting to a local server in every 5 seconds, this will be likely to flagged by anomaly detection system. 50 | 51 | Our goal is configuring these options in order to normalize Empire's C2-agent communication. 52 | 53 | ## Usage 54 | 55 | firstorder requires Python 2.7 with scapy and requests libraries to work. You can install them with pip: 56 | 57 | `pip install scapy requests` 58 | 59 | It extracts following information from a pcap file: 60 | 61 | -Most used ports 62 | 63 | -Most used server headers 64 | 65 | -Most used user-agents 66 | 67 | -Most used URIs 68 | 69 | -How many different machines broadcasted ARP packets (for determining network size) 70 | 71 | -How many different machines executed LDAP queries (for determining network size) 72 | 73 | If you only pass pcap file as an argument with -f parameter, it analyzes and extracts information from the pcap file but doesn't create an Empire listener. 74 | 75 | Command: `python firstorder.py -f file.pcap` 76 | 77 | To create an Empire listener according to analyzed data, you need to start the Empire in REST API mode with username and password. For example: 78 | 79 | `python empire --rest --username empireadmin --password Password123` 80 | 81 | Now, you can start firstorder with following command: 82 | 83 | `python firstorder.py -f file.pcap -u empireadmin -p Password123` 84 | 85 | It automatically creates a listener named "firstorder" with appropriate options. 86 | 87 | Example Output: 88 | 89 | ``` 90 | === Top 10 Port Statistics === 91 | 92 | Port 443: 1677/5937 (28.25%) 93 | Port 58471: 1107/5937 (18.65%) 94 | Port 80: 536/5937 (9.03%) 95 | Port 58457: 454/5937 (7.65%) 96 | Port 54674: 341/5937 (5.74%) 97 | Port 57859: 228/5937 (3.84%) 98 | Port 54119: 157/5937 (2.64%) 99 | Port 58408: 155/5937 (2.61%) 100 | Port 53: 124/5937 (2.09%) 101 | Port 58403: 80/5937 (1.35%) 102 | 103 | === Top 10 Server Headers === 104 | 105 | Server: PWS/8.3.1.0.4: 9/36 (25.00%) 106 | Server: RocketCache/2.2: 5/36 (13.89%) 107 | Server: nginx: 5/36 (13.89%) 108 | Server: NetDNA-cache/2.2: 4/36 (11.11%) 109 | Server: None: 3/36 (8.33%) 110 | Server: nginx/1.8.1: 2/36 (5.56%) 111 | Server: cafe: 1/36 (2.78%) 112 | Server: Microsoft-IIS/7.5: 1/36 (2.78%) 113 | Server: cloudflare-nginx: 1/36 (2.78%) 114 | Server: Microsoft-IIS/10.0: 1/36 (2.78%) 115 | 116 | === Top 10 User-Agent Headers === 117 | 118 | User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36: 29/32 (90.62%) 119 | User-Agent: Google Chrome/63.0.3239.84 Mac OS X: 3/32 (9.38%) 120 | 121 | === Top 10 GET Request URI === 122 | 123 | GET request URI: /api/country/: 3/29 (10.34%) 124 | GET request URI: /request: 2/29 (6.90%) 125 | GET request URI: /_1515920640761/redot.js: 1/29 (3.45%) 126 | GET request URI: /widgets/assets/widget/scripts.min.js: 1/29 (3.45%) 127 | GET request URI: /widgets/assets/widget/style.min.css?v=1: 1/29 (3.45%) 128 | GET request URI: /gtm.js?id=GTM-NVDWP6: 1/29 (3.45%) 129 | GET request URI: /static/912a9b7effbdc65cceea05635536577c8b0665f7.js: 1/29 (3.45%) 130 | GET request URI: /widgets/YYN-000399-20160616.html: 1/29 (3.45%) 131 | GET request URI: /async/CreateCookieSSO_Gb: 1/29 (3.45%) 132 | 133 | === Number of Unique IP addresses (ARP)=== 134 | 16 135 | 136 | === Number of Unique Computer Names(LDAP)=== 137 | 15 138 | 139 | ``` 140 | -------------------------------------------------------------------------------- /firstorder.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import operator 3 | import threading 4 | import requests 5 | from requests import ConnectionError 6 | import logging 7 | logging.getLogger("scapy.runtime").setLevel(logging.ERROR) 8 | from scapy.all import * 9 | from scapy_http import http 10 | from time import sleep 11 | 12 | banner = """ 13 | .==. A traffic analyzer to evade Empire's communication 14 | ()''()-. __ _ _ _ 15 | .---. ;--; / / _(_) | | | | 16 | .'_:___'. _..'. __'. | |_ _ _ __ ___| |_ ___ _ __ __| | ___ _ __ 17 | |__ --==|'-''' /'...; | _| | '__/ __| __/ _ \| '__/ _` |/ _ \ '__| 18 | [ ] :[| |---/ | | | | | \__ \ || (_) | | | (_| | __/ | 19 | |__| =[| .' '. |_| |_|_| |___/\__\___/|_| \__,_|\___|_| 20 | / / ____| : '._ 21 | |-/.____.' | : : by Utku Sen, Gozde Sinturk 22 | /___\ /___\ '-'._----' TEAR Security 23 | """ 24 | 25 | progress_current = 0 26 | progress_last_updated = 0 27 | progress_update_timeout = 1 28 | scan_result = { 29 | "arp_broadcasts": {}, 30 | "ldap_users": {}, 31 | "ports": {"src": {}, "dst": {}}, 32 | "protocols": {"list": set(), "proto": {}, "total": 0}, 33 | "server": {}, 34 | "request": {"path": {}, "user-agent": {}} 35 | } 36 | scan_settings_verbose = False 37 | watchdog_last_packet_parsed_at = 0 38 | watchdog_stop_flag = False 39 | headers = {'Content-Type': 'application/json'} 40 | requests.packages.urllib3.disable_warnings() 41 | 42 | def add_to_scan_result(data, result): 43 | 44 | if type(data) is list: 45 | # Add protocols stack 46 | s = result 47 | s["total"] += 1 48 | proto_prev_name = None 49 | for proto_name in data: 50 | if proto_name is "Padding": 51 | continue 52 | if proto_name == proto_prev_name: 53 | continue 54 | if proto_name not in s["proto"]: 55 | s["proto"][proto_name] = {"proto": {}, "total": 0} 56 | s = s["proto"][proto_name] 57 | s["total"] += 1 58 | result["list"].update({proto_name}) 59 | proto_prev_name = proto_name 60 | else: 61 | # Add a single value 62 | if data in result: 63 | result[data] += 1 64 | else: 65 | result[data] = 1 66 | 67 | 68 | def error(message): 69 | """Outputs error message to STDERR.""" 70 | 71 | sys.stderr.write("{prog}: error: {msg}".format(prog=os.path.basename(sys.argv[0]), msg=message+os.linesep)) 72 | 73 | 74 | def parse_raw(load, word): 75 | result = [] 76 | lines = load.split("\r\n") 77 | for line in lines: 78 | if word in line: 79 | result.append(line.split(":")[1]) 80 | return result 81 | 82 | def parse_packet(pkt): 83 | 84 | global scan_result 85 | global watchdog_last_packet_parsed_at 86 | 87 | if scan_settings_verbose: 88 | progress_update(1) 89 | 90 | if pkt.haslayer("ARP"): 91 | if pkt.dst == 'ff:ff:ff:ff:ff:ff': 92 | add_to_scan_result(pkt.psrc, scan_result["arp_broadcasts"]) 93 | 94 | if pkt.haslayer("TCP") or pkt.haslayer("UDP"): 95 | if pkt.dport == 389 or pkt.sport == 389 or pkt.dport == 636 or pkt.sport == 636: 96 | add_to_scan_result(pkt["IP"].dst, scan_result["ldap_users"]) 97 | add_to_scan_result(pkt["IP"].src, scan_result["ldap_users"]) 98 | 99 | if hasattr(pkt, "dport"): 100 | add_to_scan_result(pkt.dport, scan_result["ports"]["dst"]) 101 | if hasattr(pkt, "sport"): 102 | add_to_scan_result(pkt.sport, scan_result["ports"]["src"]) 103 | 104 | if pkt.haslayer(http.HTTPResponse): 105 | if hasattr(pkt, "Server"): 106 | add_to_scan_result(pkt.Server, scan_result["server"]) 107 | 108 | if pkt.haslayer(http.HTTPRequest): 109 | if hasattr(pkt, "User-Agent"): 110 | add_to_scan_result(getattr(pkt, "User-Agent"), scan_result["request"]["user-agent"]) 111 | 112 | if pkt.haslayer(http.HTTPRequest): 113 | if hasattr(pkt, "Path"): 114 | add_to_scan_result(getattr(pkt, "Path"), scan_result["request"]["path"]) 115 | 116 | if pkt.haslayer("UDP") and pkt["UDP"].dport == 1900: 117 | for ua in parse_raw(pkt["Raw"].load, "USER-AGENT"): 118 | add_to_scan_result(ua, scan_result["request"]["user-agent"]) 119 | for srv in parse_raw(pkt["Raw"].load, "Server"): 120 | add_to_scan_result(srv, scan_result["server"]) 121 | 122 | watchdog_last_packet_parsed_at = time.time() 123 | 124 | 125 | def port_stats(data, port_type): 126 | if port_type != "src" and port_type != "dst": 127 | print "port_type must be 'src' or 'dst'" 128 | return None 129 | print "=== Top 10 Port Statistics ===" 130 | print 131 | if len(port_type) <= 0: 132 | return None 133 | 134 | records = sorted(data[port_type].items(), key=operator.itemgetter(1), reverse=True) 135 | total = sum(data[port_type].values()) 136 | for record in records[:10]: 137 | templ = "Port {port}: {num}/{ttl} ({shr:.2f}%)" 138 | share = float(record[1]) / total * 100 139 | print templ.format(port=record[0], num=record[1], ttl=total, shr=share) 140 | print 141 | return records[0][0] 142 | 143 | 144 | def server_stats(data): 145 | print "=== Top 10 Server Headers ===" 146 | print 147 | if len(data) <= 0: 148 | return None 149 | 150 | records = sorted(data.items(), key=operator.itemgetter(1), reverse=True) 151 | total = sum(data.values()) 152 | for record in records[:10]: 153 | templ = "Server: {name}: {num}/{ttl} ({shr:.2f}%)" 154 | share = float(record[1]) / total * 100 155 | print templ.format(name=record[0], num=record[1], ttl=total, shr=share) 156 | print 157 | return records[0][0] 158 | 159 | 160 | def useragent_stats(data): 161 | print "=== Top 10 User-Agent Headers ===" 162 | print 163 | if len(data) <= 0: 164 | return None 165 | 166 | records = sorted(data.items(), key=operator.itemgetter(1), reverse=True) 167 | total = sum(data.values()) 168 | for record in records[:10]: 169 | templ = "User-Agent: {name}: {num}/{ttl} ({shr:.2f}%)" 170 | share = float(record[1]) / total * 100 171 | print templ.format(name=record[0], num=record[1], ttl=total, shr=share) 172 | print 173 | return records[0][0] 174 | 175 | 176 | def uri_stats(data): 177 | print "=== Top 10 GET Request URI ===" 178 | print 179 | if len(data) <= 0: 180 | return None 181 | 182 | records = sorted(data.items(), key=operator.itemgetter(1), reverse=True) 183 | total = sum(data.values()) 184 | for record in records[:10]: 185 | templ = "GET request URI: {name}: {num}/{ttl} ({shr:.2f}%)" 186 | share = float(record[1]) / total * 100 187 | print templ.format(name=record[0], num=record[1], ttl=total, shr=share) 188 | print 189 | uri_list = [] 190 | uri_list.append(records[0][0].split('?')[0]) 191 | uri_list.append(records[1][0].split('?')[0]) 192 | uri_list.append(records[2][0].split('?')[0]) 193 | return uri_list 194 | 195 | 196 | def arp_stats(data): 197 | print "=== Number of Unique IP addresses (ARP)===" 198 | print 199 | if len(data) <= 0: 200 | return 0 201 | 202 | records = sorted(data.items(), key=operator.itemgetter(1), reverse=True) 203 | print len(records) 204 | return len(records) 205 | 206 | 207 | def ldap_stats(data): 208 | print "=== Number of Unique Computer Names (LDAP)===" 209 | print 210 | if len(data) <= 0: 211 | return 0 212 | 213 | records = sorted(data.items(), key=operator.itemgetter(1), reverse=True) 214 | print len(records) 215 | return len(records) 216 | 217 | 218 | def progress_reset(): 219 | 220 | global progress_current 221 | global progress_last_updated 222 | 223 | progress_current = 0 224 | progress_last_updated = 0 225 | 226 | 227 | def progress_update(increment=0, force_print=False): 228 | 229 | global progress_current 230 | global progress_last_updated 231 | global progress_update_timeout 232 | 233 | progress_current += increment 234 | if force_print or time.time() > (progress_last_updated + progress_update_timeout): 235 | print "{} packets scanned...".format(progress_current) 236 | progress_last_updated = time.time() 237 | 238 | 239 | def scan_files(files): 240 | """Initiates pcap files scan for statistics.""" 241 | 242 | for f in files: 243 | if scan_settings_verbose: 244 | progress_reset() 245 | print "Scanning '{}'...".format(f) 246 | progress_update() 247 | start_watchdog(True) 248 | sniff(offline=f, store=0, prn=parse_packet) 249 | stop_watchdog() 250 | if scan_settings_verbose: 251 | progress_update(force_print=True) 252 | print "Scanning '{}' complete!".format(f) 253 | if scan_settings_verbose: 254 | print 255 | 256 | 257 | def start_watchdog(init=False): 258 | 259 | global watchdog_last_packet_parsed_at 260 | global watchdog_stop_flag 261 | 262 | if init: 263 | watchdog_last_packet_parsed_at = time.time() 264 | watchdog_stop_flag = False 265 | if not watchdog_stop_flag: 266 | if watchdog_last_packet_parsed_at + 3 < time.time(): 267 | thread.interrupt_main() 268 | else: 269 | threading.Timer(3, start_watchdog).start() 270 | 271 | 272 | def stop_watchdog(): 273 | 274 | global watchdog_stop_flag 275 | watchdog_stop_flag = True 276 | 277 | def login(username,password): 278 | creds = {'username': username,'password': password} 279 | try: 280 | r = requests.post('https://127.0.0.1:1337/api/admin/login', json=creds, headers=headers, verify=False) 281 | 282 | if r.status_code == 200: 283 | token = r.json()['token'] 284 | return token 285 | else: 286 | print "Login failed" 287 | except ConnectionError: 288 | print 'Connection Error' 289 | 290 | def start_http_listener(options,token): 291 | r = requests.post('https://127.0.0.1:1337/api/listeners/http?token='+token, headers=headers, json=options, verify=False) 292 | if r.status_code == 200: 293 | print "Listener created" 294 | else: 295 | print "Error occured on listener creation" 296 | 297 | def action(filename,empuser,emppass): 298 | scan_files(filename) 299 | global scan_result 300 | port = port_stats(scan_result["ports"], "dst") 301 | server = server_stats(scan_result["server"]) 302 | agent = useragent_stats(scan_result["request"]["user-agent"]) 303 | path = uri_stats(scan_result["request"]["path"]) 304 | arp = arp_stats(scan_result["arp_broadcasts"]) 305 | ldap = ldap_stats(scan_result["ldap_users"]) 306 | if empuser is None or emppass is None: 307 | pass 308 | else: 309 | token = login(empuser,emppass) 310 | try: 311 | default_profile = path[0] + "," + path[1] + "," + path[2] + "|" + agent 312 | except: 313 | default_profile = None 314 | if arp > ldap: 315 | number_of_computers = arp 316 | else: 317 | number_of_computers = ldap 318 | 319 | if int(number_of_computers) < 25: 320 | default_delay = 25 321 | elif int(number_of_computers) > 25 and int(number_of_computers) < 50: 322 | default_delay = 20 323 | elif int(number_of_computers) > 50 and int(number_of_computers) < 75: 324 | default_delay = 15 325 | elif int(number_of_computers) > 75 and int(number_of_computers) < 100: 326 | default_delay = 10 327 | elif int(number_of_computers) > 100: 328 | default_delay = 5 329 | 330 | 331 | if default_profile is not None: 332 | options = {'Name':'firstorder','Port':port,'ServerVersion':server,'DefaultProfile':default_profile,'DefaultDelay':default_delay} 333 | else: 334 | options = {'Name':'firstorder','Port':port,'ServerVersion':server,'DefaultDelay':default_delay} 335 | 336 | start_http_listener(options,token) 337 | 338 | 339 | 340 | parser = argparse.ArgumentParser() 341 | parser.add_argument('-f', nargs='+', action='store', dest='filename', help='Location of the pcap file', required=True) 342 | parser.add_argument('-u', action='store', dest='empuser', help='Username of the Empire REST API', required=False) 343 | parser.add_argument('-p', action='store', dest='emppass', help='Password of the Empire REST API', required=False) 344 | argv = parser.parse_args() 345 | scan_settings_verbose = True 346 | print banner 347 | action(argv.filename,argv.empuser,argv.emppass) 348 | 349 | 350 | 351 | 352 | 353 | -------------------------------------------------------------------------------- /firstorder_slide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/utkusen/firstorder/107eb6a91edcbf3af9b42f3f36c255ede04d7923/firstorder_slide.pdf --------------------------------------------------------------------------------