├── img ├── help.png ├── logo.png └── usage.png ├── LICENSE ├── README.md └── EventTranscriptParser.py /img/help.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stuxnet999/EventTranscriptParser/HEAD/img/help.png -------------------------------------------------------------------------------- /img/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stuxnet999/EventTranscriptParser/HEAD/img/logo.png -------------------------------------------------------------------------------- /img/usage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stuxnet999/EventTranscriptParser/HEAD/img/usage.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 P. Abhiram Kumar 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![logo](./img/logo.png) 2 | 3 | ### About 4 | 5 | **EventTranscriptParser** is python based tool to extract forensically useful details from EventTranscript.db (Windows Diagnostic Database). 6 | 7 | The database is found in Windows 10 systems and present at `C:\ProgramData\Microsoft\Diagnosis\EventTranscript\EventTranscript.db`. 8 | 9 | The tool currently supports the following features. 10 | 11 | + Extracts Microsoft Edge browsing history 12 | + Extracts application inventory 13 | + Extracts Wireless scan results. 14 | + Extracts successful WiFi connection events 15 | + Extracts User's default preferences (Video player, default browser etc...) 16 | + Extracts SRUM information 17 | + Application execution 18 | + Application network usage 19 | + Extracts Application execution activity 20 | 21 | ### Requirements 22 | 23 | Python 3.8 or above. The older versions of Python 3.x should work fine as well. 24 | 25 | #### Dependencies 26 | 27 | These are the required python libraries/modules needed to run the script 28 | 29 | + json 30 | + os 31 | + sqlalchemy 32 | + csv 33 | + argparse 34 | 35 | All the above modules are available by default in python3. Incase one or the other is missing, you can install by 36 | 37 | ``` 38 | pip install 39 | ``` 40 | 41 | ### Usage 42 | 43 | **Tip**: Before running the tool against the database, make sure that the **-wal (Write Ahead Log)** file data is merged with the original database. Because you might miss out on crucial/juicy data. 44 | 45 | The tool is completely CLI based and there are 2 ways to use it. 46 | 47 | #### Using Python 48 | 49 | ```python 50 | python3 EventTranscriptParser.py -f -o 51 | ``` 52 | ![usage](./img/usage.png) 53 | 54 | 55 | To view help, 56 | ``` 57 | python3 EventTranscriptParser.py -h 58 | ``` 59 | 60 | ![help](./img/help.png) 61 | 62 | #### Using Executable 63 | 64 | If you do not have python pre-installed in you system or have issues with the running the script, you can use the compiled executable. The executable is also CLI based. 65 | 66 | Download the executable from https://github.com/stuxnet999/EventTranscriptParser/releases 67 | 68 | ```sh 69 | .\EventTranscriptParser.exe -f .\EventTranscript.db -o .\CSV-Output\ 70 | ``` 71 | 72 | The executable was compiled using `pyinstaller`. 73 | 74 | #### Compiling on your own 75 | 76 | If you wish to compile on your own, use the commands below in any command prompt/terminal window. 77 | 78 | ```sh 79 | pip install pyinstaller 80 | pyinstaller --onefile EventTranscriptParser.py 81 | ``` 82 | 83 | You will find the compiled executable in the `dist` directory. 84 | 85 | ### Demo video 86 | 87 | Here is a demo video of the usage of the tool. 88 | 89 | https://github.com/stuxnet999/EventTranscriptParser/assets/31406812/005aa288-0250-4143-aabf-7d711090ef83 90 | 91 | ### Acknowledgements 92 | 93 | This tool wouldn't have been possible without the excellent research & hard work put in by my colleagues [Andrew Rathbun](https://twitter.com/bunsofwrath12) & [Josh Mitchell](https://www.linkedin.com/in/josh-mitchell-0990ba6a/) in investigating the Windows Diagnostic Data. 94 | 95 | Read more about their research here - https://github.com/rathbuna/EventTranscript.db-Research 96 | 97 | Follow the investigative series at Kroll on EventTranscript.db - https://www.kroll.com/en/insights/publications/cyber/forensically-unpacking-eventtranscript 98 | 99 | ### Author 100 | 101 | Abhiram Kumar 102 | 103 | + Twitter: [@_abhiramkumar](https://www.twitter.com/_abhiramkumar) 104 | + Personal blog: https://stuxnet999.github.io 105 | -------------------------------------------------------------------------------- /EventTranscriptParser.py: -------------------------------------------------------------------------------- 1 | __author__ = "Abhiram Kumar P (stuxnet999)" 2 | 3 | import json 4 | import os 5 | from unittest import result 6 | import sqlalchemy as sql 7 | import sqlalchemy.orm as sql_orm 8 | import csv 9 | import argparse 10 | 11 | def EdgeBrowsingHistory(session, output_directory): 12 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = "1" and events_persisted.full_event_name LIKE "%Aria.218d658af29e41b6bc37144bd03f018d.Microsoft.WebBrowser.HistoryJournal%")')) 13 | history_events = result.fetchall() 14 | 15 | if len(history_events) == 0: 16 | print("Microsoft Edge browsing history events not recorded in this database. Will not create CSV") 17 | return 18 | else: 19 | print("{} events related to browsing history from MS Edge found. Extracting & writing to CSV. Note - Some events might not contain URLs and will be skipped.".format(len(history_events))) 20 | browsinghistory_csv = open(os.path.join(output_directory, "Edge Browsing History.csv"), "w", newline='') 21 | browsinghistory_csv_writer = csv.writer(browsinghistory_csv, dialect='excel') 22 | browsinghistory_csv_writer.writerow(["Visited URL", "Visit Timestamp (UTC)", "Refer URL", "SID"]) 23 | 24 | for events in history_events: 25 | row_list = [] 26 | temp_json = json.loads(events[1]) 27 | 28 | if 'navigationUrl' in temp_json['data']: 29 | row_list.append(temp_json['data']['navigationUrl']) 30 | row_list.append(temp_json['data']['Timestamp'].replace("T"," ").replace("Z","")) 31 | 32 | if 'referUrl' in temp_json['data']: 33 | row_list.append(temp_json['data']['referUrl']) 34 | else: 35 | row_list.append("") 36 | 37 | row_list.append(events[0]) 38 | browsinghistory_csv_writer.writerow(row_list) 39 | browsinghistory_csv.close() 40 | return 41 | 42 | def ApplicationInventory(session, output_directory): 43 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 31 and events_persisted.full_event_name="Microsoft.Windows.Inventory.Core.InventoryApplicationAdd")')) 44 | application_inventory = result.fetchall() 45 | 46 | if len(application_inventory) == 0: 47 | print("Application inventory events not recorded in this database. Will not create CSV") 48 | else: 49 | print("{} events related to application inventory found. Extracting & writing to CSV".format(len(application_inventory))) 50 | application_inventory_csv = open(os.path.join(output_directory,"Application Inventory.csv"),"w", newline='') 51 | application_inventory_csv_writer = csv.writer(application_inventory_csv, dialect='excel') 52 | application_inventory_csv_writer.writerow(["Application Name", "Installation Directory", "Installation Timestamp (UTC)", "Publisher", "Application Version", "SID"]) 53 | 54 | for apps in application_inventory: 55 | row_list = [] 56 | temp_json = json.loads(apps[1]) 57 | row_list.append(temp_json['data']['Name']) 58 | row_list.append(temp_json['data']['RootDirPath']) 59 | row_list.append(temp_json['data']['InstallDate']) 60 | row_list.append(temp_json['data']['Publisher']) 61 | row_list.append(temp_json['data']['Version']) 62 | row_list.append(apps[0]) 63 | if len(set(row_list)) != 1: 64 | application_inventory_csv_writer.writerow(row_list) 65 | application_inventory_csv.close() 66 | return 67 | 68 | def ApplicationExecution(session, output_directory): 69 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 25 and events_persisted.full_event_name="Win32kTraceLogging.AppInteractivitySummary")')) 70 | execution_list = result.fetchall() 71 | 72 | if len(execution_list) == 0: 73 | print("Win32k.TraceLogging.AppInteractivitySummary not recorded in this database. Will not create CSV") 74 | else: 75 | print("{} events related to application execution found. Extracting & writing to CSV".format(len(execution_list))) 76 | execution_list_csv = open(os.path.join(output_directory, "Application Execution.csv"), "w", newline='') 77 | execution_list_csv_writer = csv.writer(execution_list_csv, dialect='excel') 78 | execution_list_csv_writer.writerow(["Binary Name", "Execution Timestamp (UTC)", "SHA1 Hash", "Compiler Timestamp (UTC)", "SID"]) 79 | 80 | for binaries in execution_list: 81 | row_list = [] 82 | temp_json = json.loads(binaries[1]) 83 | temp_binary_list = temp_json['data']['AppId'].split('!') 84 | 85 | if temp_binary_list[0][0] == "W": 86 | binary_hash = temp_binary_list[1][4:] 87 | compiler_timestamp = temp_json['data']['AppVersion'].split('!')[0].replace("/","-").replace(":", " ", 1) 88 | binary_name = temp_json['data']['AppVersion'].split('!')[2] 89 | elif temp_binary_list[0][0] == "U": 90 | binary_hash = "" 91 | compiler_timestamp = temp_json['data']['AppVersion'].split('!')[1].replace("/","-").replace(":", " ", 1) 92 | binary_name = temp_json['data']['AppVersion'].split('!')[3] 93 | 94 | row_list.append(binary_name) 95 | row_list.append(temp_json['time'].replace("T"," ").replace("Z","")) 96 | row_list.append(binary_hash) 97 | row_list.append(compiler_timestamp) 98 | row_list.append(binaries[0]) 99 | execution_list_csv_writer.writerow(row_list) 100 | execution_list_csv.close() 101 | return 102 | 103 | def UserDefaults(session, txt_dir): 104 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 11 and events_persisted.full_event_name = "Census.Userdefault")')) 105 | defaults_list = result.fetchall() 106 | if len(defaults_list) == 0: 107 | print("Device census events relating to user default settings not recorded in database. Will not create text file") 108 | else: 109 | print("{} events related to user default app preferences found. Extracting and writing to text file".format(len(defaults_list))) 110 | userdefaults_file = open(os.path.join(txt_dir, "UserDefaults.txt"), "w") 111 | for defaults in defaults_list: 112 | temp_json = json.loads(defaults[1]) 113 | userdefaults_file.write("====Record Start====\n") 114 | userdefaults_file.write("Recorded at: " + temp_json['time'].replace("T", " ").replace("Z", "") + "\n") 115 | userdefaults_file.write("Default browser: " + temp_json['data']['DefaultBrowserProgId'] + "\n") 116 | userdefaults_file.write("---Default Apps---") 117 | temp_list = temp_json['data']['DefaultApp'].split('|') 118 | for apps in temp_list: 119 | userdefaults_file.write(apps + "\n") 120 | userdefaults_file.write("====Record End====\n") 121 | userdefaults_file.close() 122 | return 123 | 124 | def WiFiConnectedEvents(session, csv_dir): 125 | result = session.execute(sql.text('SELECT events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 11 and events_persisted.full_event_name = "Microsoft.OneCore.NetworkingTriage.GetConnected.WiFiConnectedEvent")')) 126 | wifi_connections_list = result.fetchall() 127 | 128 | if len(wifi_connections_list) == 0: 129 | print("WiFi connection events have not been recorded in the database. Will not create CSV") 130 | else: 131 | print("{} events associated with successful WiFi connections found. Extracting and writing to CSV".format(len(wifi_connections_list))) 132 | 133 | wifi_connections_file = open(os.path.join(csv_dir, "WiFi Successful Connections.csv"), "w", newline='') 134 | wifi_connections_csv_writer = csv.writer(wifi_connections_file, dialect='excel') 135 | wifi_connections_csv_writer.writerow(["WiFi SSID", "WiFi BSSID", "WiFi Connection Time (UTC)", "AP Manufacturer", "AP Model Name", "AP Model No.", "Authentication Algorithm", "Cipher Algo"]) 136 | 137 | for wifi in wifi_connections_list: 138 | row_list = [] 139 | temp_json = json.loads(wifi[0]) 140 | row_list.append(temp_json['data']['ssid']) 141 | row_list.append(temp_json['data']['bssid']) 142 | row_list.append(temp_json['time'].replace('T', " ").replace('Z', "")) 143 | row_list.append(temp_json['data']['apManufacturer']) 144 | row_list.append(temp_json['data']['apModelName']) 145 | row_list.append(temp_json['data']['apModelNum']) 146 | row_list.append(temp_json['data']['authAlgo']) 147 | row_list.append(temp_json['data']['cipherAlgo']) 148 | wifi_connections_csv_writer.writerow(row_list) 149 | wifi_connections_file.close() 150 | return 151 | 152 | def SRUMAppActivity(session, csv_dir): 153 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 24 and events_persisted.full_event_name = "Microsoft.Windows.SRUM.Telemetry.AppTimelines")')) 154 | srum_app_activity_list = result.fetchall() 155 | 156 | if (len(srum_app_activity_list) == 0): 157 | print("Application activity fetched from SRUM not recorded in database. Will not create CSV") 158 | else: 159 | print("{} events associated with application activity within SRUM found. Extracting and writing to CSV".format(len(srum_app_activity_list))) 160 | SRUMAppActivity_file = open(os.path.join(csv_dir, "SRUM Application Execution Activity.csv"), "w", newline='') 161 | SRUMAppActivity_csv_writer = csv.writer(SRUMAppActivity_file, dialect='excel') 162 | 163 | SRUMAppActivity_csv_writer.writerow(["SID", "EventTranscriptDB Record Time (UTC)", "Application Start Time (UTC)", "Application Name", "Compiler Timestamp (UTC)"]) 164 | 165 | for event in srum_app_activity_list: 166 | temp_json = json.loads(event[1]) 167 | 168 | for apps in temp_json['data']['records']: 169 | row_list = [] 170 | row_list.append(event[0]) 171 | row_list.append(temp_json['time'].replace("T", " ").replace("Z","")) 172 | row_list.append(apps['startTime'].replace("T", " ").replace("z", "")) 173 | 174 | if "W:" in apps['appId']: 175 | row_list.append(apps['appId'][4:]) 176 | row_list.append(apps['appVer'].split('!', 1)[0].replace("/", "-").replace(":", " ", 1)) 177 | elif "U:" in apps['appId']: 178 | row_list.append(apps['appId'][2:]) 179 | row_list.append(apps['appVer'].split('!')[1].replace("/", "-").replace(":", " ", 1)) 180 | else: 181 | row_list.append(apps['appId']) 182 | row_list.append("N/A") 183 | SRUMAppActivity_csv_writer.writerow(row_list) 184 | SRUMAppActivity_file.close() 185 | return 186 | 187 | def WLANScanResults(session, csv_dir): 188 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 11 and events_persisted.full_event_name = "WlanMSM.WirelessScanResults")')) 189 | wlan_scan_list = result.fetchall() 190 | 191 | if len(wlan_scan_list) == 0: 192 | print("Events associated with WLAN (WiFi) scan not recorded in database. Will not create CSV") 193 | else: 194 | print("{} events associated to WLAN scan found in database. Extracting and writing to CSV".format((len(wlan_scan_list)))) 195 | 196 | wlan_scan_file = open(os.path.join(csv_dir, "WLAN Scan Results.csv"), "w", newline='') 197 | wlan_scan_csv_writer = csv.writer(wlan_scan_file, dialect='excel') 198 | 199 | wlan_scan_csv_writer.writerow(["SSID", "MAC Address", "Scan Record Timestamp (UTC)", "Interface GUID"]) 200 | 201 | for scan in wlan_scan_list: 202 | temp_json = json.loads(scan[1]) 203 | 204 | for devices in temp_json['data']['ScanResults'].split('\n'): 205 | row_list = [] 206 | wlan_scan_entry = devices.split('\t') 207 | if wlan_scan_entry[0] != '': 208 | row_list.append(wlan_scan_entry[0]) 209 | row_list.append(wlan_scan_entry[2]) 210 | row_list.append(temp_json['time'].replace("T", " ").replace("Z", "")) 211 | row_list.append(temp_json['data']['InterfaceGuid']) 212 | wlan_scan_csv_writer.writerow(row_list) 213 | else: 214 | continue 215 | wlan_scan_file.close() 216 | return 217 | 218 | def SRUMNetworkUsageActivity(session, csv_dir): 219 | result = session.execute(sql.text('SELECT events_persisted.sid, events_persisted.payload from events_persisted inner join event_tags on events_persisted.full_event_name_hash = event_tags.full_event_name_hash inner join tag_descriptions on event_tags.tag_id = tag_descriptions.tag_id where (tag_descriptions.tag_id = 24 and events_persisted.full_event_name = "Microsoft.Windows.SrumSvc.DataUsageAggregateTimer")')) 220 | network_usage_list = result.fetchall() 221 | 222 | if len(network_usage_list) == 0: 223 | print("Events associated with application network usage (from SRUM) not recorded in database. Will not create CSV.") 224 | else: 225 | print("{} events associated to App network usage found. Extracting and writing to CSV".format((len(network_usage_list)))) 226 | 227 | network_usage_file = open(os.path.join(csv_dir, "SRUM Application Network Usage.csv"), "w", newline='') 228 | net_usage_csv_writer = csv.writer(network_usage_file, dialect='excel') 229 | 230 | net_usage_csv_writer.writerow(["Event Recorded Timestamp (UTC)", "Application Name", "Bytes Sent", "Bytes Received", "Interface GUID", "SID"]) 231 | 232 | for event in network_usage_list: 233 | row_list = [] 234 | temp_json = json.loads(event[1]) 235 | row_list.append(temp_json['time'].replace("T"," ").replace("Z", "")) 236 | row_list.append(temp_json['data']['applicationName']) 237 | row_list.append(temp_json['data']['bytesSent']) 238 | row_list.append(temp_json['data']['bytesRecieved']) 239 | row_list.append(temp_json['data']['interfaceGuid']) 240 | row_list.append(event[0]) 241 | net_usage_csv_writer.writerow(row_list) 242 | network_usage_file.close() 243 | return 244 | 245 | if __name__=="__main__": 246 | event_transcript_parser=argparse.ArgumentParser( 247 | description='''EventTranscript.db parser by Abhiram Kumar.''', 248 | epilog= '''For any queries, please reach out to me via Twitter - @_abhiramkumar''') 249 | 250 | event_transcript_parser.add_argument('-f','--file', required=True, help="Please specify the path to EventTranscript.db") 251 | event_transcript_parser.add_argument('-o','--output-dir', required=True, help="Please specify the output directory") 252 | 253 | args = event_transcript_parser.parse_args() 254 | 255 | print("Windows Diagnostic Data - EventTranscript.db Parser\n") 256 | 257 | print("Author: Abhiram Kumar (Twitter: @_abhiramkumar)\nGitHub: https://github.com/stuxnet999/EventTranscriptParser") 258 | print("-"*50 + "\n") 259 | 260 | if os.path.exists(args.file): 261 | if not os.path.isdir(args.output_dir): 262 | os.makedirs(args.output_dir) 263 | 264 | db_path = args.file 265 | out_dir = args.output_dir 266 | engine = sql.create_engine('sqlite:///{}'.format(db_path)) 267 | Session = sql_orm.sessionmaker(engine) 268 | session = Session() 269 | 270 | EdgeBrowsingHistory(session, out_dir) 271 | ApplicationInventory(session, out_dir) 272 | ApplicationExecution(session, out_dir) 273 | UserDefaults(session, out_dir) 274 | WiFiConnectedEvents(session, out_dir) 275 | SRUMAppActivity(session, out_dir) 276 | WLANScanResults(session, out_dir) 277 | SRUMNetworkUsageActivity(session, out_dir) --------------------------------------------------------------------------------