├── .gitignore ├── AutoAnalysis.bat ├── CSV_parser ├── basic.csv ├── config │ ├── fullFilePathIgnore.txt │ ├── fullPathRegistryKeyIgnore.txt │ ├── processIgnore.txt │ ├── regexFilePathIgnore.txt │ └── regexRegistryKeyIgnore.txt ├── log.docx ├── log.txt ├── parse.bat ├── parse.py └── test.csv ├── LICENSE ├── logs └── .keep ├── readme.md └── tools └── .keep /.gitignore: -------------------------------------------------------------------------------- 1 | # General 2 | .DS_Store 3 | .AppleDouble 4 | .LSOverride 5 | 6 | # Icon must end with two \r 7 | Icon 8 | 9 | 10 | # Thumbnails 11 | ._* 12 | 13 | # Files that might appear in the root of a volume 14 | .DocumentRevisions-V100 15 | .fseventsd 16 | .Spotlight-V100 17 | .TemporaryItems 18 | .Trashes 19 | .VolumeIcon.icns 20 | .com.apple.timemachine.donotpresent 21 | 22 | # Directories potentially created on remote AFP share 23 | .AppleDB 24 | .AppleDesktop 25 | Network Trash Folder 26 | Temporary Items 27 | .apdisk 28 | 29 | # Byte-compiled / optimized / DLL files 30 | __pycache__/ 31 | *.py[cod] 32 | *$py.class 33 | 34 | # C extensions 35 | *.so 36 | 37 | # Distribution / packaging 38 | .Python 39 | env/ 40 | build/ 41 | develop-eggs/ 42 | dist/ 43 | downloads/ 44 | eggs/ 45 | .eggs/ 46 | lib/ 47 | lib64/ 48 | parts/ 49 | sdist/ 50 | var/ 51 | wheels/ 52 | *.egg-info/ 53 | .installed.cfg 54 | *.egg 55 | 56 | # PyInstaller 57 | # Usually these files are written by a python script from a template 58 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 59 | *.manifest 60 | *.spec 61 | 62 | # Installer logs 63 | pip-log.txt 64 | pip-delete-this-directory.txt 65 | 66 | # Unit test / coverage reports 67 | htmlcov/ 68 | .tox/ 69 | .coverage 70 | .coverage.* 71 | .cache 72 | nosetests.xml 73 | coverage.xml 74 | *.cover 75 | .hypothesis/ 76 | 77 | # Translations 78 | *.mo 79 | *.pot 80 | 81 | # Django stuff: 82 | *.log 83 | local_settings.py 84 | 85 | # Flask stuff: 86 | instance/ 87 | .webassets-cache 88 | 89 | # Scrapy stuff: 90 | .scrapy 91 | 92 | # Sphinx documentation 93 | docs/_build/ 94 | 95 | # PyBuilder 96 | target/ 97 | 98 | # Jupyter Notebook 99 | .ipynb_checkpoints 100 | 101 | # pyenv 102 | .python-version 103 | 104 | # celery beat schedule file 105 | celerybeat-schedule 106 | 107 | # SageMath parsed files 108 | *.sage.py 109 | 110 | # dotenv 111 | .env 112 | 113 | # virtualenv 114 | .venv 115 | venv/ 116 | ENV/ 117 | 118 | # Spyder project settings 119 | .spyderproject 120 | .spyproject 121 | 122 | # Rope project settings 123 | .ropeproject 124 | 125 | # mkdocs documentation 126 | /site 127 | 128 | # mypy 129 | .mypy_cache/ -------------------------------------------------------------------------------- /AutoAnalysis.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | :: ########################### 4 | :: TOOLS 5 | :: windump.exe 6 | :: procmon.exe 7 | :: graphviz 8 | :: ProcDot (post processing) 9 | :: Windows PSR ( Event driven screen capture tool ) 10 | :: ########################### 11 | :: SET WORKING DIRECTORY 12 | set basePath=C:\Users\IEUser\Desktop\autoanalysis 13 | :: ########################### 14 | :: Variables 15 | :: Get TimeStamp 16 | :: See http://stackoverflow.com/q/1642677/1143274 17 | FOR /f %%a IN ('WMIC OS GET LocalDateTime ^| FIND "."') DO SET DTS=%%a 18 | SET DateTime=%DTS:~0,4%-%DTS:~4,2%-%DTS:~6,2%_%DTS:~8,2%-%DTS:~10,2%-%DTS:~12,2% 19 | set logPath=%basePath%\logs\%DateTime% 20 | set toolPath=%basePath%\tools 21 | :: ########################### 22 | 23 | echo ----------------- 24 | echo Starting Analysis 25 | echo ----------------- 26 | 27 | cd %basePath% 28 | mkdir %logPath% 29 | 30 | echo Starting Windump... 31 | start /min %toolPath%\windump\windump.exe -s 0 -w %logPath%\capture.pcap 32 | 33 | echo Starting Procmon... 34 | start /min %toolPath%\SysinternalsSuite\procmon.exe /AcceptEula /Quiet /Minimized /backingfile %logPath%\capture.pml 35 | start /min %toolPath%\SysinternalsSuite\procmon.exe /AcceptEula /WaitForIdle 36 | 37 | echo Starting PSR... 38 | start psr /start /output %logPath%\psr.zip /sc 1 /gui 0 39 | 40 | echo Sleeping 5 seconds... 41 | ping -n 5 localhost > nul 42 | 43 | echo Capturing Data... 44 | echo Execute test plan and close when done executing processes. 45 | pause 46 | 47 | start /min %toolPath%\SysinternalsSuite\procmon.exe /AcceptEula /Terminate 48 | start /min %toolPath%\SysinternalsSuite\procmon.exe /AcceptEula /PagingFile /NoConnect /Minimized /Quiet 49 | 50 | echo Killing windump 51 | taskkill /im windump.exe /f 52 | 53 | echo Sleeping 10 seconds... 54 | ping -n 10 localhost > nul 55 | 56 | echo Terminating Capture 57 | %toolPath%\SysinternalsSuite\procmon.exe /Terminate 58 | 59 | echo Sleeping 5 seconds... 60 | ping -n 5 localhost > nul 61 | psr /stop 62 | 63 | echo Sleeping 5 seconds... 64 | ping -n 5 localhost > nul 65 | 66 | echo Saving as .csv 67 | %toolPath%\SysinternalsSuite\procmon.exe /AcceptEula /OpenLog %logPath%\capture.pml /SaveAs %logPath%\capture.csv 68 | 69 | echo. 70 | echo Logs Saved - %logPath% 71 | 72 | pause 73 | -------------------------------------------------------------------------------- /CSV_parser/config/fullFilePathIgnore.txt: -------------------------------------------------------------------------------- 1 | c: 2 | c:\ 3 | c:\system volume information 4 | c:\$bitmap 5 | c:\$converttononresident 6 | c:\$directory 7 | c:\$logfile 8 | c:\$mft 9 | c:\$mftmirr 10 | c:\pagefile.sys 11 | c:\windows\system32\config\software 12 | c:\windows\system32\config\software.log 13 | c:\windows\system32\config\system 14 | c:\windows\system32\config\system.log 15 | c:\windows\system32\wbem\logs\wmiprov.log 16 | c:\windows\system32\wbem\repository\fs\index.btr 17 | c:\windows\system32\wbem\repository\fs\index.map 18 | c:\windows\system32\wbem\repository\fs\mapping.ver 19 | c:\windows\system32\wbem\repository\fs\mapping1.map 20 | c:\windows\system32\wbem\repository\fs\mapping2.map 21 | c:\windows\system32\wbem\repository\fs\objects.data 22 | c:\windows\system32\wbem\repository\fs\objects.map 23 | c:\windows\system32\wbem\performance\wmiaprpl_new.ini 24 | c:\windows\system32\perfstringbackup.ini 25 | c:\windows\system32\perfstringbackup.tmp 26 | c:\windows\system32\wbem\performance\wmiaprpl_new.h 27 | c:\windows\system32\wbem\performance\wmiaprpl.hew.h 28 | c:\windows\system32\wbem\performance\wmiaprpl.ini.ini 29 | c:\windows\system32\config\appevent.evt 30 | c:\windows\system32\config\sysevent.evt 31 | c:\windows\system32\d3d9caps.dat 32 | c:\windows\system32\d3d9caps.tmp 33 | -------------------------------------------------------------------------------- /CSV_parser/config/fullPathRegistryKeyIgnore.txt: -------------------------------------------------------------------------------- 1 | *\cachelimit 2 | *captureprocessmonitor* 3 | *captureregistrymonitor* 4 | *procmon23* 5 | hkcu\sessioninformation\programcount 6 | hkcu\software\microsoft\direct3d\mostrecentapplication\name 7 | hkcu\software\microsoft\windows\currentversion\explorer\shell folders\cache 8 | hkcu\software\microsoft\windows\currentversion\explorer\shell folders\cookies 9 | hkcu\software\microsoft\windows\currentversion\explorer\startpage\programscache 10 | hklm\software\microsoft\cryptography\rng\seed 11 | hklm\software\microsoft\directdraw\mostrecentapplication\id 12 | hklm\software\microsoft\directdraw\mostrecentapplication\name 13 | hklm\software\microsoft\windows nt\currentversion\prefetcher\lasttracefailure 14 | hklm\software\microsoft\windows nt\currentversion\prefetcher\tracesprocessed 15 | hklm\software\microsoft\windows nt\currentversion\prefetcher\tracessuccessful 16 | hklm\software\microsoft\windows\currentversion\internet settings\cache\paths\path2\cachepath 17 | hklm\software\microsoft\windows\currentversion\internet settings\cache\paths\path3\cachepath 18 | hklm\system\currentcontrolset\services\kmixer\enum\nextinstance 19 | hklm\system\currentcontrolset\services\kmixer\enum\count 20 | hklm\system\currentcontrolset\services\wmiaprpl\performance\last counter 21 | hklm\system\currentcontrolset\services\wmiaprpl\performance\last help 22 | hklm\system\currentcontrolset\services\wmiaprpl\performance\first counter 23 | hklm\system\currentcontrolset\services\wmiaprpl\performance\object list 24 | hklm\system\currentcontrolset\services\wmiaprpl\performance\first help 25 | hklm\software\microsoft\wbem\providers\performance\performance refresh 26 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\advapi32.dll[mofresourcename] 27 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\mssmbios.sys[mofresource] 28 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\pcntpci5.sys[ndismofresource] 29 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\acpi.sys[acpimofresource] 30 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\ipnat.sys[ipnatmofresource] 31 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\http.sys[ulmofresource] 32 | hklm\software\microsoft\wbem\wdm\dredge\c:\windows\system32\drivers\ac97intc.sys[mofresourcename] 33 | hklm\software\microsoft\wbem\providers\performance\performance refreshed 34 | hklm\software\microsoft\windows nt\currentversion\perflib\last counter 35 | hklm\software\microsoft\windows nt\currentversion\perflib\last help 36 | hklm\software\microsoft\windows nt\currentversion\perflib\updating 37 | hklm\system\currentcontrolset\services\kmixer\enum\0 38 | hkcr\libraryfolder\docobject -------------------------------------------------------------------------------- /CSV_parser/config/processIgnore.txt: -------------------------------------------------------------------------------- 1 | WinDump.exe 2 | Procmon.exe 3 | Procmon64.exe 4 | psr.exe 5 | procdot.exe 6 | SearchProtocolHost.exe 7 | SearchIndexer.exe 8 | VBoxService.exe 9 | csrss.exe 10 | --ITEMS BELOW THIS LINE MAY FILTER OUT GOOD DATA-- 11 | #explorer.exe 12 | #svchost.exe 13 | #mmc.exe 14 | #conhost.exe -------------------------------------------------------------------------------- /CSV_parser/config/regexFilePathIgnore.txt: -------------------------------------------------------------------------------- 1 | ^.*capture.pml$ 2 | ^.*capture.csv$ 3 | ^.*capture.pcap$ 4 | ^c\:\\.*?\\.*?\\.*?\\.*?\\history\.ie5\\mshist[0-9]*\\index\.dat$ 5 | ^c\:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows\\usrclass\.dat$ 6 | ^c\:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows\\usrclass\.dat\.log$ 7 | ^c\:\\.*?\\.*?\\.*?\\sun\\java\\deployment\\cache\\6\.0\\[0-9]+\\[0-9a-f\-]+-temp$ 8 | ^c\:\\.*?\\.*?\\.*?\\sun\\java\\deployment\\cache\\6\.0\\[0-9]+\\[0-9a-f\-]+\.idx$ 9 | ^c\:\\.*?\\.*?\\.*?\\sun\\java\\deployment\\cache\\6\.0\\lastaccessed$ 10 | ^c\:\\.*?\\.*?\\.*?\\sun\\java\\deployment\\deployment\.properties$ 11 | ^c\:\\.*?\\.*?\\.*?\\temp\\jar_cache[0-9]+\.tmp$ 12 | ^c\:\\.*?\\.*?\\.*?\\temp\\minibis-cpp-transfer\.bat$ 13 | ^c\:\\.*?\\.*?\\.*?\\temp\\minibis-cpp\.out\.[0-9]+$ 14 | ^c\:\\.*?\\.*?\\.*?\\temp\\windump\.pcap$ 15 | ^c\:\\.*?\\.*?\\.*?\\temporary internet files\\content\.ie5$ 16 | ^c\:\\.*?\\.*?\\.*?\\temporary internet files\\content\.ie5\\cache[^\\]*$ 17 | ^c\:\\.*?\\.*?\\.*?\\temporary internet files\\content\.ie5\\index\.dat$ 18 | ^c\:\\.*?\\.*?\\.*\\temp\\minibis\.bat$ 19 | ^c\:\\.*?\\.*?\\ntuser\.dat$ 20 | ^c\:\\.*?\\.*?\\ntuser\.dat\.log$ 21 | ^c\:\\.*?\\capture\\capture_.*?_.*?\.zip$ 22 | ^c\:\\.*?\\capture\\logs\\.*?\.pcap$ 23 | ^c\:\\windows\\prefetch\\.*?\-.*?\.pf$ 24 | ^c\:\\.*?\\.*?\\.*?\\macromedia\\flash player\\macromedia\.com\\support\\flashplayer\\sys\\settings.sxx$ 25 | ^c\:\\.*?\\.*?\\.*?\\macromedia\\flash player\\macromedia\.com\\support\\flashplayer\\sys\\settings.sxx$ 26 | ^c\:\\.*?\\.*?\\.*?\\macromedia\\flash player\\macromedia\.com\\support\\flashplayer\\sys\\settings.sol$ 27 | ^c:\\windows\\system32\\perf.*?\.dat 28 | ^[a-z]:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows media\\.*?\\wmsdkns\.xml$ 29 | ^[a-z]:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows media\\.*?\\wmsdknsd\.xml$ 30 | ^[a-z]:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows media\\.*?\\wmsdknsr\.xml$ 31 | ^[a-z]:\\.*?\\.*?\\.*?\\.*?\\microsoft\\windows media\\.*?\\wmsdkns\.dtd$ 32 | -------------------------------------------------------------------------------- /CSV_parser/config/regexRegistryKeyIgnore.txt: -------------------------------------------------------------------------------- 1 | ^hkcu\\software\\microsoft\\windows\\currentversion\\ext\\stats\\\{.*?\}\\iexplore\\count$ 2 | ^hkcu\\software\\microsoft\\windows\\currentversion\\ext\\stats\\\{.*?\}\\iexplore\\time$ 3 | ^hkcu\\software\\microsoft\\windows\\currentversion\\ext\\stats\\\{.*?\}\\iexplore\\type$ 4 | ^hkcu\\software\\microsoft\\windows\\currentversion\\internet settings\\5\.0\\cache\\extensible cache\\mshist[0-9]*\\cache[^\\]*$ 5 | ^hkcu\\software\\microsoft\\windows\\currentversion\\runonce\\flashplayerupdate$ 6 | ^hklm\\software\\microsoft\\systemcertificates\\authroot\\certificates\\[0-9a-z]*\\blob$ 7 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\dhcpdefaultgateway$ 8 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\dhcpipaddress$ 9 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\dhcpserver$ 10 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\dhcpsubnetmask$ 11 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\dhcpsubnetmaskopt$ 12 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\lease$ 13 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\leaseobtainedtime$ 14 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\leaseterminatestime$ 15 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\t1$ 16 | ^hklm\\system\\currentcontrolset\\services\\\{.*?\}\\parameters\\tcpip\\t2$ 17 | ^hklm\\system\\currentcontrolset\\services\\dhcp\\parameters\\\{.*?\}$ 18 | ^hklm\\system\\currentcontrolset\\services\\sharedaccess\\epoch\\epoch$ 19 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\dhcpnameserver$ 20 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\addresstype$ 21 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpdefaultgateway$ 22 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpipaddress$ 23 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpnameserver$ 24 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpretrystatus$ 25 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpretrytime$ 26 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpserver$ 27 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpsubnetmask$ 28 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\dhcpsubnetmaskopt$ 29 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\ipautoconfigurationaddress$ 30 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\ipautoconfigurationmask$ 31 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\ipautoconfigurationseed$ 32 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\lease$ 33 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\leaseobtainedtime$ 34 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\leaseterminatestime$ 35 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\t1$ 36 | ^hklm\\system\\currentcontrolset\\services\\tcpip\\parameters\\interfaces\\\{.*?\}\\t2$ 37 | ^hkcu\\software\\classes\\clsid\\\{.*?\}\\\(default\)$ 38 | ^hkcu\\software\\classes\\clsid\\\{.*?\}\\inprocserver32\\\(default\)$ 39 | ^hkcu\\software\\classes\\clsid\\\{.*?\}\\inprocserver32\\threadingmodel$ 40 | ^hkcr\\.*$ 41 | -------------------------------------------------------------------------------- /CSV_parser/log.docx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/threatexpress/procdot_sandbox/a95ed7c90af7ab09a397ad5f32be47e9c5172210/CSV_parser/log.docx -------------------------------------------------------------------------------- /CSV_parser/parse.bat: -------------------------------------------------------------------------------- 1 | c:\python27\python.exe parse.py .\basic.csv 2 | 3 | pause -------------------------------------------------------------------------------- /CSV_parser/parse.py: -------------------------------------------------------------------------------- 1 | import csv 2 | import sys 3 | import re 4 | from docx import Document 5 | 6 | ############################################ 7 | # Reference 8 | ############################################ 9 | # CSV Parsing - http://pymotw.com/2/csv/ 10 | 11 | ############################################ 12 | # Initialize Variables 13 | ############################################ 14 | # Open CSV and files 15 | f = open(sys.argv[1], 'rt') 16 | logText = open('log.txt','w') 17 | logWord = 'log.docx' 18 | document = Document() 19 | 20 | # Row Count 21 | totalrows = 0 22 | # priorRow Holder 23 | priorRow = {} 24 | priorRow['\xef\xbb\xbf"Time of Day"'] = '' 25 | priorRow['Process Name'] = '' 26 | priorRow['PID'] = '' 27 | priorRow['Operation'] = '' 28 | priorRow['Path'] = '' 29 | priorRow['Result'] = '' 30 | priorRow['Detail'] = '' 31 | priorRow['TID'] = '' 32 | 33 | # Open Config files 34 | processIgnore = [line.strip() for line in open('./config/processIgnore.txt')] 35 | fullFilePathIgnore = [line.strip() for line in open('./config/fullFilePathIgnore.txt')] 36 | fullPathRegistryKeyIgnore = [line.strip() for line in open('./config/fullPathRegistryKeyIgnore.txt')] 37 | regexFilePathIgnore = [line.strip() for line in open('./config/regexFilePathIgnore.txt')] 38 | regexRegistryKeyIgnore = [line.strip() for line in open('./config/regexRegistryKeyIgnore.txt')] 39 | 40 | # Parser Counters 41 | fullFilePathIgnoreCOUNT = 0 42 | fullPathRegistryKeyIgnoreCOUNT = 0 43 | regexFilePathIgnoreCOUNT = 0 44 | regexRegistryKeyIgnoreCOUNT = 0 45 | 46 | ############################################ 47 | # Functions 48 | ############################################ 49 | def printStatus(i): 50 | ''' Print current progress ''' 51 | print "Progress: {0}% ({1} of {2})".format(int(float(i) / float(totalrows) * 100),i,totalrows) 52 | 53 | def updatepriorRow(row): 54 | ''' Store the current row for repetition comparison ''' 55 | 56 | #priorRow['\xef\xbb\xbf"Time of Day"'] = row['\xef\xbb\xbf"Time of Day"'] 57 | priorRow['Process Name'] = row['Process Name'] 58 | priorRow['PID'] = row['PID'] 59 | priorRow['Operation'] = row['Operation'] 60 | priorRow['Path'] = row['Path'] 61 | priorRow['Result'] = row['Result'] 62 | priorRow['Detail'] = ['Detail'] 63 | priorRow['TID'] = ['TID'] 64 | 65 | def ignoreRepeat(row): 66 | ''' Check to see if the prior Operation is the same as the current. 67 | Do not logText if it was the same.''' 68 | #logText.write("{} {} {} {}".format(priorRow['Operation'], row['Operation'], priorRow['Path'], row['Path'])+ "\n") 69 | if (priorRow['Operation'] == row['Operation']) and (priorRow['Path'] == row['Path']): 70 | return True 71 | else: 72 | return False 73 | 74 | def ignoreProcess(processName): 75 | ''' Compares process with a list of process to ignore and not logText. ''' 76 | 77 | if processName in processIgnore: 78 | return True 79 | else: 80 | return False 81 | 82 | def ignoreFullFilePath(path): 83 | ''' Compares file path with a list of file paths to ignore and not logText. ''' 84 | global fullFilePathIgnoreCOUNT 85 | 86 | if path.lower() in fullFilePathIgnore: 87 | fullFilePathIgnoreCOUNT += 1 88 | return True 89 | else: 90 | return False 91 | 92 | def ignoreRegexFilePath(path): 93 | ''' Compares file path using regex with a list of file paths to ignore and not logText. ''' 94 | global regexFilePathIgnoreCOUNT 95 | 96 | pathFound = False 97 | for search in regexFilePathIgnore: 98 | p = re.compile(search, re.IGNORECASE) 99 | found = p.match(path.lower()) 100 | if found: 101 | pathFound = True 102 | break 103 | 104 | if pathFound: 105 | regexFilePathIgnoreCOUNT += 1 106 | return True 107 | else: 108 | return False 109 | 110 | def ignoreFullPathRegistryKey(path): 111 | ''' Compares registy path with a list of paths to ignore and not logText. ''' 112 | global fullPathRegistryKeyIgnoreCOUNT 113 | 114 | if path.lower() in fullPathRegistryKeyIgnore: 115 | fullPathRegistryKeyIgnoreCOUNT += 1 116 | return True 117 | else: 118 | return False 119 | 120 | def ignoreRegexRegistryKey(path): 121 | ''' Compares registry key using regex with a list of key values to ignore and not logText. ''' 122 | global regexRegistryKeyIgnoreCOUNT 123 | 124 | pathFound = False 125 | for search in regexRegistryKeyIgnore: 126 | p = re.compile(search, re.IGNORECASE) 127 | 128 | found = p.match(path.lower()) 129 | if found: 130 | pathFound = True 131 | break 132 | 133 | if pathFound: 134 | regexRegistryKeyIgnoreCOUNT += 1 135 | return True 136 | else: 137 | return False 138 | 139 | def processRow(row): 140 | ''' Process row for data 141 | Row Header Reference 142 | "Time of Day","Process Name","PID","Operation","Path","Result","Detail","TID" 143 | ''' 144 | # Capture row values 145 | #TOD = row['\xef\xbb\xbf"Time of Day"'] 146 | PRN = row['Process Name'] 147 | PID = row['PID'] 148 | OPN = row['Operation'] 149 | PTH = row['Path'] 150 | RST = row['Result'] 151 | DTL = row['Detail'] 152 | TID = row['TID'] 153 | 154 | # Check for rows to Ignore - START 155 | if ignoreProcess(PRN): 156 | return 157 | 158 | if ignoreFullFilePath(PTH): 159 | return 160 | 161 | if ignoreFullPathRegistryKey(PTH): 162 | return 163 | 164 | if ignoreRegexFilePath(PTH): 165 | return 166 | 167 | if ignoreRegexRegistryKey(PTH): 168 | return 169 | # Check for rows to Ignore - END 170 | 171 | elif OPN == "WriteFile": 172 | if ignoreRepeat(row): 173 | return 174 | else: 175 | line = "Thread {0} of process {1} (PID: {2}) wrote to file {3}.".format(TID, PRN, PID, PTH) 176 | logText.write(line + "\n") 177 | line = "Thread {0} of process {1} (PID: {2}) wrote to file.".format(TID, PRN, PID) 178 | document.add_paragraph(line) 179 | #document.add_paragraph("{0}ACTION: {1}".format("\t",OPN)) 180 | #document.add_paragraph("{0}THREAD: {1}".format("\t\t",TID)) 181 | #document.add_paragraph("{0}PATH: {1}".format("\t\t",PTH)) 182 | #document.add_paragraph("{0}VALUE: {1}".format("\t\t",DTL)) 183 | updatepriorRow(row) 184 | return 185 | 186 | elif OPN == "RegQueryValue": 187 | if ignoreRepeat(row): 188 | return 189 | else: 190 | line = "Thread {0} of process {1} (PID: {2}) queried registy key {3}. Result: {4}".format(TID, PRN, PID, PTH, DTL) 191 | logText.write(line + "\n") 192 | line = "Thread {0} of process {1} (PID: {2}) queried registy key.".format(TID, PRN, PID) 193 | #document.add_paragraph(line) 194 | #document.add_paragraph("{0}ACTION: {1}".format("\t",OPN)) 195 | #document.add_paragraph("{0}THREAD: {1}".format("\t\t",TID)) 196 | #ocument.add_paragraph("{0}PATH: {1}".format("\t\t",PTH)) 197 | #ocument.add_paragraph("{0}VALUE: {1}".format("\t\t",DTL)) 198 | updatepriorRow(row) 199 | return 200 | 201 | elif OPN == "RegSetValue": 202 | if ignoreRepeat(row): 203 | return 204 | else: 205 | line = "Thread {0} of process {1} (PID: {2}) set registy key {3}. Result: {4}".format(TID, PRN, PID, PTH, DTL) 206 | logText.write(line + "\n") 207 | line = "Thread {0} of process {1} (PID: {2}) set registy key.".format(TID, PRN, PID) 208 | #document.add_paragraph(line) 209 | #document.add_paragraph("{0}ACTION: {1}".format("\t",OPN)) 210 | #ocument.add_paragraph("{0}THREAD: {1}".format("\t\t",TID)) 211 | #document.add_paragraph("{0}PATH: {1}".format("\t\t",PTH)) 212 | #document.add_paragraph("{0}VALUE: {1}".format("\t\t",DTL)) 213 | updatepriorRow(row) 214 | return 215 | 216 | elif OPN == "SetDispositionInformationFile": 217 | if ignoreRepeat(row): 218 | return 219 | else: 220 | line = "Thread {0} of process {1} (PID: {2}) deleted file {3}. Result: {4}".format(TID, PRN, PID, PTH, DTL) 221 | logText.write(line + "\n") 222 | line = "Thread {0} of process {1} (PID: {2}) deleted file.".format(TID, PRN, PID) 223 | #document.add_paragraph(line) 224 | #document.add_paragraph("{0}ACTION: {1}".format("\t",OPN)) 225 | #document.add_paragraph("{0}THREAD: {1}".format("\t\t",TID)) 226 | #document.add_paragraph("{0}PATH: {1}".format("\t\t",PTH)) 227 | #document.add_paragraph("{0}VALUE: {1}".format("\t\t",DTL)) 228 | #document.add_paragraph(line) 229 | updatepriorRow(row) 230 | return 231 | else: 232 | # Ignore Row 233 | return 234 | ############################################ 235 | # Main 236 | ############################################ 237 | try: 238 | 239 | # Count Rows 240 | readerCntr = csv.DictReader(f) 241 | print "Reading CSV..." 242 | for row in readerCntr: 243 | totalrows += 1 244 | f.seek(0) # Reset rile position 245 | 246 | # Process Rows 247 | print "Processing CSV..." 248 | reader = csv.DictReader(f) 249 | i = 1 250 | for row in reader: 251 | if i%10000 == 0: 252 | printStatus(i) 253 | processRow(row) 254 | 255 | i += 1 256 | print "" 257 | print "--------------------------------------------------------" 258 | print " Summary" 259 | print "--------------------------------------------------------" 260 | print "Total Items Processed: {0}".format(i,totalrows) 261 | print "--------------------------------------------------------" 262 | print "Items Filtered" 263 | print "---------------" 264 | print " fullFilePathIgnore {0}".format(fullFilePathIgnoreCOUNT) 265 | print " fullPathRegistryKeyIgnore {0}".format(fullPathRegistryKeyIgnoreCOUNT) 266 | print " regexFilePathIgnore {0}".format(regexFilePathIgnoreCOUNT) 267 | print " regexRegistryKeyIgnore {0}".format(regexRegistryKeyIgnoreCOUNT) 268 | print "--------------------------------------------------------" 269 | 270 | finally: 271 | f.close() 272 | logText.close() 273 | document.save(logWord) 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | -------------------------------------------------------------------------------- /CSV_parser/test.csv: -------------------------------------------------------------------------------- 1 | "Time of Day","Process Name","PID","Operation","Path","Result","Detail","TID" 2 | "10:56:43.5431413 AM","Procmon64.exe","1512","RegQueryValue","HKLM\System\CurrentControlSet\Control\WMI\Security\9e814aad-3204-11d2-9a82-006008a86939","BUFFER OVERFLOW","Length: 524","3212" 3 | "10:56:43.5431752 AM","SomeProcess.exe","1512","WriteFile","c:\windows\system32\perf.log\.dat","SUCCESS","YES","3212" 4 | "10:56:43.5432343 AM","System","4","Thread Create","","SUCCESS","Thread ID: 4004","3212" 5 | "10:56:43.5433597 AM","DFSRs.exe","1380","FileSystemControl","C:","SUCCESS","Control: FSCTL_READ_USN_JOURNAL","1556" 6 | "10:56:43.5433978 AM","DFSRs.exe","1380","FileSystemControl","C:","SUCCESS","Control: FSCTL_READ_USN_JOURNAL","1556" 7 | "10:56:43.5434115 AM","DFSRs.exe","1380","FileSystemControl","C:","SUCCESS","Control: FSCTL_READ_USN_JOURNAL","1556" 8 | "10:56:43.5437641 AM","VBoxTray.exe","1964","Thread Exit","","SUCCESS","Thread ID: 4032, User Time: 0.0000000, Kernel Time: 0.0000000","4032" 9 | "10:56:43.5440682 AM","VBoxService.exe","708","RegQueryKey","HKLM","SUCCESS","Query: HandleTags, HandleTags: 0x0","740" 10 | "10:56:43.5440876 AM","VBoxService.exe","708","RegOpenKey","HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\Interfaces","REPARSE","Desired Access: Read","740" 11 | "10:56:43.5441063 AM","VBoxService.exe","708","RegOpenKey","HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\Interfaces","SUCCESS","Desired Access: Read","740" 12 | "10:56:43.5441357 AM","VBoxService.exe","708","RegQueryKey","HKLM\System\CurrentControlSet\services\Tcpip\Parameters\Interfaces","SUCCESS","Query: HandleTags, HandleTags: 0x0","740" 13 | "10:56:43.5441441 AM","VBoxService.exe","708","RegOpenKey","HKLM\System\CurrentControlSet\services\Tcpip\Parameters\Interfaces\{12eb7006-935c-46d9-ad27-209a987bd800}","SUCCESS","Desired Access: Query Value","740" 14 | "10:57:24.0231715 AM","secretsdump.exe","3816","TCP Receive","2k8DC.test.local:58843 -> 2k8DC.test.local:microsoft-ds","SUCCESS","Length: 1460, seqnum: 0, connid: 0","0" 15 | "10:57:24.0231768 AM","secretsdump.exe","3816","TCP Receive","2k8DC.test.local:58843 -> 2k8DC.test.local:microsoft-ds","SUCCESS","Length: 1460, seqnum: 0, connid: 0","0" 16 | "10:57:24.0231818 AM","secretsdump.exe","3816","TCP Receive","2k8DC.test.local:58843 -> 2k8DC.test.local:microsoft-ds","SUCCESS","Length: 1460, seqnum: 0, connid: 0","0" 17 | "10:57:24.0231871 AM","secretsdump.exe","3816","TCP Receive","2k8DC.test.local:58843 -> 2k8DC.test.local:microsoft-ds","SUCCESS","Length: 976, seqnum: 0, connid: 0","0" 18 | "10:57:24.0231959 AM","System","4","TCP Send","2k8DC.test.local:microsoft-ds -> 2k8DC.test.local:58843","SUCCESS","Length: 8276, startime: 329105, endtime: 329105, seqnum: 0, connid: 0","0" 19 | "10:57:24.0232820 AM","secretsdump.exe","3816","FASTIO_MDL_READ_COMPLETE","C:\Windows\Temp\auizwMsu.tmp","SUCCESS","MDL: 0xfffffa8004b434c0","780" 20 | "10:57:24.0278931 AM","secretsdump.exe","3816","WriteFile","C:\Users\Administrator\Desktop\secretsdump\consoleOutput.txt","SUCCESS","Offset: 18,693, Length: 64","780" 21 | "10:57:24.0351738 AM","secretsdump.exe","3816","WriteFile","C:\Users\Administrator\Desktop\secretsdump\consoleOutput.txt","SUCCESS","Offset: 18,757, Length: 64","780" 22 | "10:57:24.0423477 AM","secretsdump.exe","3816","WriteFile","C:\Users\Administrator\Desktop\secretsdump\consoleOutput.txt","SUCCESS","Offset: 18,821, Length: 64","780" 23 | "10:57:24.0497698 AM","secretsdump.exe","3816","WriteFile","C:\Users\Administrator\Desktop\secretsdump\consoleOutput.txt","SUCCESS","Offset: 18,885, Length: 64","780" 24 | "10:57:24.0577068 AM","secretsdump.exe","3816","WriteFile","C:\Users\Administrator\Desktop\secretsdump\consoleOutput.txt","SUCCESS","Offset: 18,949, Length: 64","780" -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Threat Express 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /logs/.keep: -------------------------------------------------------------------------------- 1 | keep -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # ProDot Sandbox - AutoAnalysis 2 | 3 | Quick Malware Analysis Toolkit. This repository contains quick setup notes to setup a malware analysis sandbox using a variety of tools and uses ProcDot to perform the analysis. 4 | 5 | These instructions are very highlevel. You will need to adjust to work in your lab. 6 | 7 | [ProcDot](http://www.procdot.com/) 8 | 9 | ## Requirements 10 | 11 | - Target OS (Windows 10/7) 12 | - graphviz - http://www.graphviz.org/ 13 | - ProcDOT - http://www.procdot.com/ 14 | - WinPcap - https://www.winpcap.org/ 15 | - Windump - https://www.winpcap.org/windump/default.htm 16 | - ProcMon [Sysinternals Suite https://docs.microsoft.com/en-us/sysinternals/](https://docs.microsoft.com/en-us/sysinternals/) 17 | - PSR - Problem Step Recorder (Built in Windows tool) 18 | 19 | OPTIONAL: Python to run CSV_parser 20 | 21 | The CSV_parser directory contains a python script that can help filter noise from the procmon CSV logs. 22 | 23 | -------------- 24 | ## Installation 25 | 26 | - Download/extract tools to a common directory 27 | + This example uses C:\Users\IEUser\Desktop\autoanalysis\tools\ 28 | - Install WinPcap 29 | 30 | ## Configuration 31 | 32 | ### ProcDOT 33 | 34 | Open ProcDOT and configure the following options 35 | 36 | **Note:** More detailed installation information can be found here [ProcDot](http://www.procdot.com/) 37 | 38 | __Path to windump/tcpdump__ 39 | 40 | C:\Users\IEUser\Desktop\autoanalysis\tools\windump\WinDump.exe 41 | 42 | __Path to dot (Graphviz)__ 43 | 44 | C:\Users\IEUser\Desktop\autoanalysis\tools\graphviz-2.38\release\bin\dot.exe 45 | 46 | ### ProcMon 47 | 48 | You need to adjust Procmon's configuration to be compatible with ProcDOT. 49 | 50 | __In Procmon__ 51 | 52 | - disable (uncheck) "Show Resolved Network Addresses" (Options) 53 | - disable (uncheck) "Enable Advanced Output" (Filter) 54 | - adjust the displayed columns (Options > Select Columns ...) 55 | + to not show the "Sequence" column 56 | + to show the "Thread ID" column 57 | 58 | -------------- 59 | ## Quick Start 60 | 61 | 1. Run AutoAnalysis.bat as Administrator 62 | 2. Execute Malware 63 | 3. Stop AutoAnalysis 64 | 4. Analyze Results 65 | 66 | ## Analyze with ProcDOT 67 | 68 | 1. Open procdot.exe 69 | 70 | __Monitoring Logs__ 71 | 72 | Procmon: browse to procmon capture.csv 73 | Procmon: browse to pcap capture.pcap 74 | 75 | 2. Click ... in the Launcher button to analyze logs 76 | 77 | 3. Select the first relavant process 78 | 79 | 4. Click Refresh to build the graph 80 | 81 | 5. Proceed to analyze results 82 | 83 | -------------- 84 | ## Analyst Tips 85 | 86 | __Tune logs__ 87 | 88 | - Consider filtering out unnecessary data from PCAP 89 | - Consider removing unnecessary procmon logs from the report 90 | + CSV_parser contains a python script that can help filter the procmon CSV logs -------------------------------------------------------------------------------- /tools/.keep: -------------------------------------------------------------------------------- 1 | keep --------------------------------------------------------------------------------