├── .gitignore ├── README.md ├── WordlistSmith.sln └── WordlistSmith ├── App.config ├── FodyWeavers.xml ├── FodyWeavers.xsd ├── Program.cs ├── Properties └── AssemblyInfo.cs ├── Smithy.cs ├── WordlistSmith.csproj ├── WordlistSmith.csproj.DotSettings └── packages.config /.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore 5 | 6 | # User-specific files 7 | *.rsuser 8 | *.suo 9 | *.user 10 | *.userosscache 11 | *.sln.docstates 12 | 13 | # User-specific files (MonoDevelop/Xamarin Studio) 14 | *.userprefs 15 | 16 | # Mono auto generated files 17 | mono_crash.* 18 | 19 | # Build results 20 | [Dd]ebug/ 21 | [Dd]ebugPublic/ 22 | [Rr]elease/ 23 | [Rr]eleases/ 24 | x64/ 25 | x86/ 26 | [Aa][Rr][Mm]/ 27 | [Aa][Rr][Mm]64/ 28 | bld/ 29 | [Bb]in/ 30 | [Oo]bj/ 31 | [Ll]og/ 32 | [Ll]ogs/ 33 | 34 | # Visual Studio 2015/2017 cache/options directory 35 | .vs/ 36 | # Uncomment if you have tasks that create the project's static files in wwwroot 37 | #wwwroot/ 38 | 39 | # Visual Studio 2017 auto generated files 40 | Generated\ Files/ 41 | 42 | # MSTest test Results 43 | [Tt]est[Rr]esult*/ 44 | [Bb]uild[Ll]og.* 45 | 46 | # NUnit 47 | *.VisualState.xml 48 | TestResult.xml 49 | nunit-*.xml 50 | 51 | # Build Results of an ATL Project 52 | [Dd]ebugPS/ 53 | [Rr]eleasePS/ 54 | dlldata.c 55 | 56 | # Benchmark Results 57 | BenchmarkDotNet.Artifacts/ 58 | 59 | # .NET Core 60 | project.lock.json 61 | project.fragment.lock.json 62 | artifacts/ 63 | 64 | # StyleCop 65 | StyleCopReport.xml 66 | 67 | # Files built by Visual Studio 68 | *_i.c 69 | *_p.c 70 | *_h.h 71 | *.ilk 72 | *.meta 73 | *.obj 74 | *.iobj 75 | *.pch 76 | *.pdb 77 | *.ipdb 78 | *.pgc 79 | *.pgd 80 | *.rsp 81 | *.sbr 82 | *.tlb 83 | *.tli 84 | *.tlh 85 | *.tmp 86 | *.tmp_proj 87 | *_wpftmp.csproj 88 | *.log 89 | *.vspscc 90 | *.vssscc 91 | .builds 92 | *.pidb 93 | *.svclog 94 | *.scc 95 | 96 | # Chutzpah Test files 97 | _Chutzpah* 98 | 99 | # Visual C++ cache files 100 | ipch/ 101 | *.aps 102 | *.ncb 103 | *.opendb 104 | *.opensdf 105 | *.sdf 106 | *.cachefile 107 | *.VC.db 108 | *.VC.VC.opendb 109 | 110 | # Visual Studio profiler 111 | *.psess 112 | *.vsp 113 | *.vspx 114 | *.sap 115 | 116 | # Visual Studio Trace Files 117 | *.e2e 118 | 119 | # TFS 2012 Local Workspace 120 | $tf/ 121 | 122 | # Guidance Automation Toolkit 123 | *.gpState 124 | 125 | # ReSharper is a .NET coding add-in 126 | _ReSharper*/ 127 | *.[Rr]e[Ss]harper 128 | *.DotSettings.user 129 | 130 | # TeamCity is a build add-in 131 | _TeamCity* 132 | 133 | # DotCover is a Code Coverage Tool 134 | *.dotCover 135 | 136 | # AxoCover is a Code Coverage Tool 137 | .axoCover/* 138 | !.axoCover/settings.json 139 | 140 | # Visual Studio code coverage results 141 | *.coverage 142 | *.coveragexml 143 | 144 | # NCrunch 145 | _NCrunch_* 146 | .*crunch*.local.xml 147 | nCrunchTemp_* 148 | 149 | # MightyMoose 150 | *.mm.* 151 | AutoTest.Net/ 152 | 153 | # Web workbench (sass) 154 | .sass-cache/ 155 | 156 | # Installshield output folder 157 | [Ee]xpress/ 158 | 159 | # DocProject is a documentation generator add-in 160 | DocProject/buildhelp/ 161 | DocProject/Help/*.HxT 162 | DocProject/Help/*.HxC 163 | DocProject/Help/*.hhc 164 | DocProject/Help/*.hhk 165 | DocProject/Help/*.hhp 166 | DocProject/Help/Html2 167 | DocProject/Help/html 168 | 169 | # Click-Once directory 170 | publish/ 171 | 172 | # Publish Web Output 173 | *.[Pp]ublish.xml 174 | *.azurePubxml 175 | # Note: Comment the next line if you want to checkin your web deploy settings, 176 | # but database connection strings (with potential passwords) will be unencrypted 177 | *.pubxml 178 | *.publishproj 179 | 180 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 181 | # checkin your Azure Web App publish settings, but sensitive information contained 182 | # in these scripts will be unencrypted 183 | PublishScripts/ 184 | 185 | # NuGet Packages 186 | *.nupkg 187 | # NuGet Symbol Packages 188 | *.snupkg 189 | # The packages folder can be ignored because of Package Restore 190 | **/[Pp]ackages/* 191 | # except build/, which is used as an MSBuild target. 192 | !**/[Pp]ackages/build/ 193 | # Uncomment if necessary however generally it will be regenerated when needed 194 | #!**/[Pp]ackages/repositories.config 195 | # NuGet v3's project.json files produces more ignorable files 196 | *.nuget.props 197 | *.nuget.targets 198 | 199 | # Microsoft Azure Build Output 200 | csx/ 201 | *.build.csdef 202 | 203 | # Microsoft Azure Emulator 204 | ecf/ 205 | rcf/ 206 | 207 | # Windows Store app package directories and files 208 | AppPackages/ 209 | BundleArtifacts/ 210 | Package.StoreAssociation.xml 211 | _pkginfo.txt 212 | *.appx 213 | *.appxbundle 214 | *.appxupload 215 | 216 | # Visual Studio cache files 217 | # files ending in .cache can be ignored 218 | *.[Cc]ache 219 | # but keep track of directories ending in .cache 220 | !?*.[Cc]ache/ 221 | 222 | # Others 223 | ClientBin/ 224 | ~$* 225 | *~ 226 | *.dbmdl 227 | *.dbproj.schemaview 228 | *.jfm 229 | *.pfx 230 | *.publishsettings 231 | orleans.codegen.cs 232 | 233 | # Including strong name files can present a security risk 234 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 235 | #*.snk 236 | 237 | # Since there are multiple workflows, uncomment next line to ignore bower_components 238 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 239 | #bower_components/ 240 | 241 | # RIA/Silverlight projects 242 | Generated_Code/ 243 | 244 | # Backup & report files from converting an old project file 245 | # to a newer Visual Studio version. Backup files are not needed, 246 | # because we have git ;-) 247 | _UpgradeReport_Files/ 248 | Backup*/ 249 | UpgradeLog*.XML 250 | UpgradeLog*.htm 251 | ServiceFabricBackup/ 252 | *.rptproj.bak 253 | 254 | # SQL Server files 255 | *.mdf 256 | *.ldf 257 | *.ndf 258 | 259 | # Business Intelligence projects 260 | *.rdl.data 261 | *.bim.layout 262 | *.bim_*.settings 263 | *.rptproj.rsuser 264 | *- [Bb]ackup.rdl 265 | *- [Bb]ackup ([0-9]).rdl 266 | *- [Bb]ackup ([0-9][0-9]).rdl 267 | 268 | # Microsoft Fakes 269 | FakesAssemblies/ 270 | 271 | # GhostDoc plugin setting file 272 | *.GhostDoc.xml 273 | 274 | # Node.js Tools for Visual Studio 275 | .ntvs_analysis.dat 276 | node_modules/ 277 | 278 | # Visual Studio 6 build log 279 | *.plg 280 | 281 | # Visual Studio 6 workspace options file 282 | *.opt 283 | 284 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 285 | *.vbw 286 | 287 | # Visual Studio LightSwitch build output 288 | **/*.HTMLClient/GeneratedArtifacts 289 | **/*.DesktopClient/GeneratedArtifacts 290 | **/*.DesktopClient/ModelManifest.xml 291 | **/*.Server/GeneratedArtifacts 292 | **/*.Server/ModelManifest.xml 293 | _Pvt_Extensions 294 | 295 | # Paket dependency manager 296 | .paket/paket.exe 297 | paket-files/ 298 | 299 | # FAKE - F# Make 300 | .fake/ 301 | 302 | # CodeRush personal settings 303 | .cr/personal 304 | 305 | # Python Tools for Visual Studio (PTVS) 306 | __pycache__/ 307 | *.pyc 308 | 309 | # Cake - Uncomment if you are using it 310 | # tools/** 311 | # !tools/packages.config 312 | 313 | # Tabs Studio 314 | *.tss 315 | 316 | # Telerik's JustMock configuration file 317 | *.jmconfig 318 | 319 | # BizTalk build output 320 | *.btp.cs 321 | *.btm.cs 322 | *.odx.cs 323 | *.xsd.cs 324 | 325 | # OpenCover UI analysis results 326 | OpenCover/ 327 | 328 | # Azure Stream Analytics local run output 329 | ASALocalRun/ 330 | 331 | # MSBuild Binary and Structured Log 332 | *.binlog 333 | 334 | # NVidia Nsight GPU debugger configuration file 335 | *.nvuser 336 | 337 | # MFractors (Xamarin productivity tool) working folder 338 | .mfractor/ 339 | 340 | # Local History for Visual Studio 341 | .localhistory/ 342 | 343 | # BeatPulse healthcheck temp database 344 | healthchecksdb 345 | 346 | # Backup folder for Package Reference Convert tool in Visual Studio 2017 347 | MigrationBackup/ 348 | 349 | # Ionide (cross platform F# VS Code tools) working folder 350 | .ionide/ 351 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # WordlistSmith 2 | A tool to quickly scrape a website and generate a wordlist. Multithreading capable. 3 | 4 | ## Introduction 5 | 6 | WordlistSmith (working title) is a C# implimentation of [CeWL](https://github.com/digininja/CeWL), which is still an awesome tool. But, when running it recently on an engagement I noticed it took a large amount of time to run against a particular website with a ton of sub-pages at a default depth (think a mile wide and an inch deep application). This pushed me to create WordlistSmith to speed up the process and set a limit on the number of pages spidered. 7 | 8 | ## Setup: 9 | 10 | It's probably easiest to use the built version under Releases, just note that it is compiled in Debug mode. If you want to build the solution yourself, follow the steps below. 11 | 12 | 1. Load WordlistSmith.sln into Visual Studio 2019 13 | 2. Go to Build at the top and then Build Solution if no modifications are wanted 14 | 15 | ## Usage 16 | 17 | I decided to have the verbose option enabled by default due to the good amount of information it shows. If you don't want this, use the `-q` or `--quiet` flag. 18 | 19 | - Help menu
20 | `WordlistSmith.exe --help` 21 | 22 | - Default run (min word length = 3, max length = 10, delay = 100 seconds, timeout = 15 seconds, threads = 10, max depth = 3)
23 | `WordlistSmith.exe -u https://www.example.com` 24 | 25 | - Setting a max number of pages regardless of depth to 1000
26 | `WordlistSmith.exe -u https://www.example.com --max-pages 1000` 27 | 28 | - Quick run (may have issues if site is slow)
29 | `WordlistSmith.exe -u https://www.example.com --max-pages 1000 --threads 20 --delay 0 -o outputWordlist.txt` 30 | 31 | ## Props 32 | Thanks to Robin ([@digininja](https://twitter.com/digininja)) for the great work and inspiration on CeWL. 33 | -------------------------------------------------------------------------------- /WordlistSmith.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 15 4 | VisualStudioVersion = 15.0.28307.1401 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "WordlistSmith", "WordlistSmith\WordlistSmith.csproj", "{CBBB4C6C-90AF-4256-BC47-8ACD56EB385B}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Any CPU = Debug|Any CPU 11 | Release|Any CPU = Release|Any CPU 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {CBBB4C6C-90AF-4256-BC47-8ACD56EB385B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 15 | {CBBB4C6C-90AF-4256-BC47-8ACD56EB385B}.Debug|Any CPU.Build.0 = Debug|Any CPU 16 | {CBBB4C6C-90AF-4256-BC47-8ACD56EB385B}.Release|Any CPU.ActiveCfg = Release|Any CPU 17 | {CBBB4C6C-90AF-4256-BC47-8ACD56EB385B}.Release|Any CPU.Build.0 = Release|Any CPU 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | GlobalSection(ExtensibilityGlobals) = postSolution 23 | SolutionGuid = {E8541B07-20F9-4FE8-90C4-DCC75A7E4196} 24 | EndGlobalSection 25 | EndGlobal 26 | -------------------------------------------------------------------------------- /WordlistSmith/App.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /WordlistSmith/FodyWeavers.xml: -------------------------------------------------------------------------------- 1 |  2 | 3 | -------------------------------------------------------------------------------- /WordlistSmith/FodyWeavers.xsd: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | A list of assembly names to exclude from the default action of "embed all Copy Local references", delimited with line breaks 13 | 14 | 15 | 16 | 17 | A list of assembly names to include from the default action of "embed all Copy Local references", delimited with line breaks. 18 | 19 | 20 | 21 | 22 | A list of unmanaged 32 bit assembly names to include, delimited with line breaks. 23 | 24 | 25 | 26 | 27 | A list of unmanaged 64 bit assembly names to include, delimited with line breaks. 28 | 29 | 30 | 31 | 32 | The order of preloaded assemblies, delimited with line breaks. 33 | 34 | 35 | 36 | 37 | 38 | This will copy embedded files to disk before loading them into memory. This is helpful for some scenarios that expected an assembly to be loaded from a physical file. 39 | 40 | 41 | 42 | 43 | Controls if .pdbs for reference assemblies are also embedded. 44 | 45 | 46 | 47 | 48 | Embedded assemblies are compressed by default, and uncompressed when they are loaded. You can turn compression off with this option. 49 | 50 | 51 | 52 | 53 | As part of Costura, embedded assemblies are no longer included as part of the build. This cleanup can be turned off. 54 | 55 | 56 | 57 | 58 | Costura by default will load as part of the module initialization. This flag disables that behavior. Make sure you call CosturaUtility.Initialize() somewhere in your code. 59 | 60 | 61 | 62 | 63 | Costura will by default use assemblies with a name like 'resources.dll' as a satellite resource and prepend the output path. This flag disables that behavior. 64 | 65 | 66 | 67 | 68 | A list of assembly names to exclude from the default action of "embed all Copy Local references", delimited with | 69 | 70 | 71 | 72 | 73 | A list of assembly names to include from the default action of "embed all Copy Local references", delimited with |. 74 | 75 | 76 | 77 | 78 | A list of unmanaged 32 bit assembly names to include, delimited with |. 79 | 80 | 81 | 82 | 83 | A list of unmanaged 64 bit assembly names to include, delimited with |. 84 | 85 | 86 | 87 | 88 | The order of preloaded assemblies, delimited with |. 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 'true' to run assembly verification (PEVerify) on the target assembly after all weavers have been executed. 97 | 98 | 99 | 100 | 101 | A comma-separated list of error codes that can be safely ignored in assembly verification. 102 | 103 | 104 | 105 | 106 | 'false' to turn off automatic generation of the XML Schema file. 107 | 108 | 109 | 110 | 111 | -------------------------------------------------------------------------------- /WordlistSmith/Program.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Diagnostics; 4 | using System.IO; 5 | using System.Net; 6 | using System.Threading.Tasks; 7 | using Abot2.Core; 8 | using Abot2.Crawler; 9 | using Abot2.Poco; 10 | using CommandLine; 11 | using CommandLine.Text; 12 | using NUglify; 13 | using Serilog; 14 | 15 | namespace WordlistSmith 16 | { 17 | class Program 18 | { 19 | private static HashSet wordlist = new HashSet(); 20 | public static Smithy smithy; 21 | 22 | static async Task Main(string[] args) 23 | { 24 | Stopwatch watch = new Stopwatch(); 25 | watch.Start(); 26 | 27 | // Parse arguments passed 28 | Parser parser = new Parser(with => 29 | { 30 | with.CaseInsensitiveEnumValues = true; 31 | with.CaseSensitive = false; 32 | with.HelpWriter = null; 33 | }); 34 | 35 | ParserResult parserResult = parser.ParseArguments(args); 36 | parserResult.WithParsed(o => { Options.Instance = o; }) 37 | .WithNotParsed(errs => DisplayHelp(parserResult, errs)); 38 | Options options = Options.Instance; 39 | 40 | try 41 | { 42 | smithy = new Smithy(); 43 | 44 | if (options.Quiet) 45 | { 46 | Log.Logger = new LoggerConfiguration() 47 | .MinimumLevel.Error() 48 | .WriteTo.Console() 49 | .CreateLogger(); 50 | } 51 | 52 | else 53 | { 54 | Log.Logger = new LoggerConfiguration() 55 | .MinimumLevel.Information() 56 | .WriteTo.Console() 57 | .CreateLogger(); 58 | } 59 | 60 | await DoCrawl(smithy); 61 | 62 | Uri uri = new Uri(smithy.Url); 63 | string safeUri = uri.Authority; 64 | safeUri = safeUri.Replace('.', '_'); 65 | 66 | if (string.IsNullOrEmpty(smithy.Output)) 67 | { 68 | smithy.Output = "wordlist_" + safeUri + DateTime.Now.ToString("_HH-mm-ss") + ".txt"; 69 | 70 | if (smithy.Output.Length > 250) 71 | smithy.Output = "wordlist_" + DateTime.Now.ToString("M-dd-yyyy_HH-mm-ss") + ".txt"; 72 | } 73 | 74 | Console.WriteLine($"\n[+] Crawl finished, writing to file: {smithy.Output}\n"); 75 | 76 | using (StreamWriter outfile = new StreamWriter(smithy.Output)) 77 | { 78 | foreach (var word in wordlist) 79 | await outfile.WriteLineAsync(word); 80 | } 81 | 82 | watch.Stop(); 83 | Console.WriteLine("Execution time: " + watch.ElapsedMilliseconds / 1000 + " Seconds"); 84 | } 85 | 86 | catch (Exception e) 87 | { 88 | Console.WriteLine(e); 89 | } 90 | } 91 | 92 | private static async Task DoCrawl(Smithy smithy) 93 | { 94 | var config = new CrawlConfiguration(); 95 | if (!string.IsNullOrEmpty(smithy.User) && !string.IsNullOrEmpty(smithy.Pass)) 96 | { 97 | config.MaxConcurrentThreads = smithy.Threads; 98 | config.MaxCrawlDepth = smithy.Depth; 99 | config.MinCrawlDelayPerDomainMilliSeconds = smithy.Delay; 100 | config.MaxPagesToCrawl = smithy.MaxPages; 101 | config.MaxRetryCount = 1; 102 | //HttpServicePointConnectionLimit = 2000, 103 | config.HttpRequestTimeoutInSeconds = smithy.Timeout; 104 | config.LoginUser = smithy.User; 105 | config.LoginPassword = smithy.Pass; 106 | } 107 | 108 | if (!string.IsNullOrEmpty(smithy.User) || !string.IsNullOrEmpty(smithy.Pass)) 109 | { 110 | if (string.IsNullOrEmpty(smithy.Pass) || string.IsNullOrEmpty(smithy.User)) 111 | { 112 | Console.WriteLine("Please specify both a username and a password if using basic auth"); 113 | System.Environment.Exit(1); 114 | } 115 | } 116 | 117 | else 118 | { 119 | config.MaxConcurrentThreads = smithy.Threads; 120 | config.MaxCrawlDepth = smithy.Depth; 121 | config.MinCrawlDelayPerDomainMilliSeconds = smithy.Delay; 122 | config.MaxPagesToCrawl = smithy.MaxPages; 123 | config.MaxRetryCount = 1; 124 | //HttpServicePointConnectionLimit = 2000, 125 | config.HttpRequestTimeoutInSeconds = smithy.Timeout; 126 | } 127 | 128 | var crawler = new PoliteWebCrawler(config); 129 | 130 | crawler.PageCrawlCompleted += PageCrawlCompleted; 131 | 132 | var crawlResult = await crawler.CrawlAsync(new Uri(smithy.Url)); 133 | } 134 | 135 | private static async Task DemoSinglePageRequest() 136 | { 137 | var pageRequester = new PageRequester(new CrawlConfiguration(), new WebContentExtractor()); 138 | 139 | var crawledPage = await pageRequester.MakeRequestAsync(new Uri("http://msn.com")); 140 | Log.Logger.Information("{result}", new 141 | { 142 | url = crawledPage.Uri, 143 | status = Convert.ToInt32(crawledPage.HttpResponseMessage.StatusCode) 144 | }); 145 | } 146 | 147 | private static void PageCrawlCompleted(object sender, PageCrawlCompletedArgs e) 148 | { 149 | var httpStatus = e.CrawledPage.HttpResponseMessage?.StatusCode; 150 | var rawPageText = e.CrawledPage.Content.Text; 151 | char[] charsToRemove = 152 | { 153 | ' ', ';', '.', '!', ':', '=', '@', '#', '$', '"', '%', '^', '&', '*', '(', ')', '<', '>', '?', '\'', 154 | ',', '-' 155 | }; 156 | 157 | if (httpStatus == HttpStatusCode.OK) 158 | { 159 | //Log.Logger.Information("{result}", new 160 | //{ 161 | // url = e.CrawledPage.Uri, 162 | // status = Convert.ToInt32(e.CrawledPage.HttpResponseMessage.StatusCode) 163 | //}); 164 | 165 | //lock (rawPageText) 166 | //{ 167 | List tempList = new List(); 168 | 169 | var result = Uglify.HtmlToText(rawPageText).ToString(); 170 | foreach (var word in result.Split()) 171 | { 172 | string cleanWord = word.ToLower().Trim(charsToRemove); 173 | 174 | if (cleanWord.Length > smithy.Min && cleanWord.Length < 256) 175 | { 176 | if (!wordlist.Contains(cleanWord)) 177 | tempList.Add(cleanWord); 178 | } 179 | } 180 | 181 | foreach (var word in tempList) 182 | wordlist.Add(word); 183 | //} 184 | } 185 | } 186 | 187 | public class Options 188 | { 189 | public static Options Instance { get; set; } 190 | 191 | // Command line options 192 | [Option('q', "quiet", Required = false, HelpText = "Do not log anything to the screen")] 193 | public bool Quiet { get; set; } 194 | 195 | [Option('u', "url", Required = true, HelpText = "Specify a URL to scrape words from")] 196 | public string Url { get; set; } 197 | 198 | [Option('o', "output", Required = false, HelpText = "Specify a file to output the wordlist to", 199 | Default = null)] 200 | public string Output { get; set; } 201 | 202 | [Option("depth", Required = false, HelpText = "Specify the depth to crawl", Default = 3)] 203 | public int Depth { get; set; } 204 | 205 | [Option("max-pages", Required = false, HelpText = "Specify the maximum pages to crawl", Default = null)] 206 | public int MaxPages { get; set; } 207 | 208 | [Option('t', "threads", Required = false, HelpText = "Specify the number of concurrent threads", 209 | Default = 10)] 210 | public int Threads { get; set; } 211 | 212 | [Option("min-length", Required = false, HelpText = "Specify a minimum word length to save", Default = 3)] 213 | public int Minimum { get; set; } 214 | 215 | [Option("max-length", Required = false, HelpText = "Specify a maximum word length to save", Default = null)] 216 | public int Maximum { get; set; } 217 | 218 | [Option("delay", Required = false, HelpText = "Specify a delay between requests", Default = 100)] 219 | public int Delay { get; set; } 220 | 221 | [Option("timeout", Required = false, HelpText = "Specify a timeout for each request", Default = 15)] 222 | public int Timeout { get; set; } 223 | 224 | [Option("user", Required = false, HelpText = "Specify a username for basic auth")] 225 | public string User { get; set; } 226 | 227 | [Option("pass", Required = false, HelpText = "Specify a password for basic auth")] 228 | public string Pass { get; set; } 229 | 230 | [Option('a', "agent", Required = false, HelpText = "Specify a User Agent to use", 231 | Default = 232 | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36")] 233 | public string Agent { get; set; } 234 | } 235 | 236 | private static void DisplayHelp(ParserResult result, IEnumerable errs) 237 | { 238 | HelpText helpText = HelpText.AutoBuild(result, h => 239 | { 240 | h.AdditionalNewLineAfterOption = false; 241 | h.Heading = "WordSmith C# Version 0.1"; //change header 242 | h.Copyright = ""; //change copyright text 243 | h.AutoVersion = false; 244 | return HelpText.DefaultParsingErrorsHandler(result, h); 245 | }, e => e); 246 | Console.WriteLine(helpText); 247 | System.Environment.Exit(1); 248 | } 249 | } 250 | } -------------------------------------------------------------------------------- /WordlistSmith/Properties/AssemblyInfo.cs: -------------------------------------------------------------------------------- 1 | using System.Reflection; 2 | using System.Runtime.CompilerServices; 3 | using System.Runtime.InteropServices; 4 | 5 | // General Information about an assembly is controlled through the following 6 | // set of attributes. Change these attribute values to modify the information 7 | // associated with an assembly. 8 | [assembly: AssemblyTitle("WordlistSmith")] 9 | [assembly: AssemblyDescription("")] 10 | [assembly: AssemblyConfiguration("")] 11 | [assembly: AssemblyCompany("")] 12 | [assembly: AssemblyProduct("WordlistSmith")] 13 | [assembly: AssemblyCopyright("Copyright © 2021")] 14 | [assembly: AssemblyTrademark("")] 15 | [assembly: AssemblyCulture("")] 16 | 17 | // Setting ComVisible to false makes the types in this assembly not visible 18 | // to COM components. If you need to access a type in this assembly from 19 | // COM, set the ComVisible attribute to true on that type. 20 | [assembly: ComVisible(false)] 21 | 22 | // The following GUID is for the ID of the typelib if this project is exposed to COM 23 | [assembly: Guid("cbbb4c6c-90af-4256-bc47-8acd56eb385b")] 24 | 25 | // Version information for an assembly consists of the following four values: 26 | // 27 | // Major Version 28 | // Minor Version 29 | // Build Number 30 | // Revision 31 | // 32 | // You can specify all the values or you can default the Build and Revision Numbers 33 | // by using the '*' as shown below: 34 | // [assembly: AssemblyVersion("1.0.*")] 35 | [assembly: AssemblyVersion("1.0.0.0")] 36 | [assembly: AssemblyFileVersion("1.0.0.0")] 37 | -------------------------------------------------------------------------------- /WordlistSmith/Smithy.cs: -------------------------------------------------------------------------------- 1 |  2 | namespace WordlistSmith 3 | { 4 | public class Smithy 5 | { 6 | public string Url, Output, Agent, User, Pass; 7 | public int Min, Max, Depth, Delay, Threads, MaxPages, Timeout; 8 | 9 | public Smithy() 10 | { 11 | this.Url = Program.Options.Instance.Url; 12 | this.Min = Program.Options.Instance.Minimum; 13 | this.Max = Program.Options.Instance.Maximum; 14 | this.Output = Program.Options.Instance.Output; 15 | this.Depth = Program.Options.Instance.Depth; 16 | this.Threads = Program.Options.Instance.Threads; 17 | this.Delay = Program.Options.Instance.Delay; 18 | this.Agent = Program.Options.Instance.Agent; 19 | this.MaxPages = Program.Options.Instance.MaxPages; 20 | this.Timeout = Program.Options.Instance.Timeout; 21 | this.User = Program.Options.Instance.User; 22 | this.Pass = Program.Options.Instance.Pass; 23 | } 24 | } 25 | } -------------------------------------------------------------------------------- /WordlistSmith/WordlistSmith.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | Debug 7 | AnyCPU 8 | {CBBB4C6C-90AF-4256-BC47-8ACD56EB385B} 9 | Exe 10 | WordlistSmith 11 | WordlistSmith 12 | v4.7.2 13 | 512 14 | true 15 | true 16 | 17 | 18 | 19 | 20 | 21 | AnyCPU 22 | true 23 | full 24 | false 25 | bin\Debug\ 26 | DEBUG;TRACE 27 | prompt 28 | 4 29 | latest 30 | 31 | 32 | AnyCPU 33 | pdbonly 34 | true 35 | bin\Release\ 36 | TRACE 37 | prompt 38 | 4 39 | latest 40 | 41 | 42 | 43 | ..\packages\Abot.2.0.69\lib\netstandard2.0\Abot2.dll 44 | 45 | 46 | ..\packages\AngleSharp.0.14.0\lib\net472\AngleSharp.dll 47 | 48 | 49 | ..\packages\CommandLineParser.2.8.0\lib\net461\CommandLine.dll 50 | 51 | 52 | ..\packages\Costura.Fody.4.1.0\lib\net40\Costura.dll 53 | 54 | 55 | 56 | ..\packages\NRobotsCore.1.1.3\lib\net451\NRobotsCore.dll 57 | 58 | 59 | ..\packages\NUglify.1.13.8\lib\net40\NUglify.dll 60 | 61 | 62 | ..\packages\Serilog.2.10.0\lib\net46\Serilog.dll 63 | 64 | 65 | ..\packages\Serilog.Enrichers.Thread.3.1.0\lib\net45\Serilog.Enrichers.Thread.dll 66 | 67 | 68 | ..\packages\Serilog.Sinks.Console.3.1.1\lib\net45\Serilog.Sinks.Console.dll 69 | 70 | 71 | 72 | 73 | 74 | ..\packages\System.IO.4.3.0\lib\net462\System.IO.dll 75 | True 76 | True 77 | 78 | 79 | ..\packages\System.Net.Http.4.3.4\lib\net46\System.Net.Http.dll 80 | True 81 | True 82 | 83 | 84 | ..\packages\System.Runtime.4.3.1\lib\net462\System.Runtime.dll 85 | True 86 | True 87 | 88 | 89 | ..\packages\System.Runtime.CompilerServices.Unsafe.5.0.0\lib\net45\System.Runtime.CompilerServices.Unsafe.dll 90 | 91 | 92 | ..\packages\System.Security.Cryptography.Algorithms.4.3.1\lib\net463\System.Security.Cryptography.Algorithms.dll 93 | True 94 | True 95 | 96 | 97 | ..\packages\System.Security.Cryptography.Encoding.4.3.0\lib\net46\System.Security.Cryptography.Encoding.dll 98 | True 99 | True 100 | 101 | 102 | ..\packages\System.Security.Cryptography.Primitives.4.3.0\lib\net46\System.Security.Cryptography.Primitives.dll 103 | True 104 | True 105 | 106 | 107 | ..\packages\System.Security.Cryptography.X509Certificates.4.3.2\lib\net461\System.Security.Cryptography.X509Certificates.dll 108 | True 109 | True 110 | 111 | 112 | ..\packages\System.Text.Encoding.CodePages.5.0.0\lib\net461\System.Text.Encoding.CodePages.dll 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. 129 | 130 | 131 | 132 | 133 | 134 | -------------------------------------------------------------------------------- /WordlistSmith/WordlistSmith.csproj.DotSettings: -------------------------------------------------------------------------------- 1 |  2 | CSharp80 -------------------------------------------------------------------------------- /WordlistSmith/packages.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | --------------------------------------------------------------------------------