├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Optiv Security 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | # OSINT Encyclopedia 4 | #### Credit: [Cham423](https://github.com/cham423) 5 | 6 | This checklist is designed to increase the success of your open-source intelligence (OSINT) operations by collecting a comprehensive list of information about your target. Understanding the fundamentals of OSINT is a prerequisite to using this checklist, as detailed technical operations will not be captured here. This list will be a working document that is driven by the community and maintained by Optiv. 7 | 8 | 9 | ## OSINT Checklist for ALL Engagements 10 | 11 | - [ ] Social Media 12 | - [ ] Corporate/Busniess Controlled Content 13 | - [ ] LinkedIn 14 | - [ ] Facebook 15 | - [ ] Instagram 16 | - [ ] Employee Controlled Content 17 | - [ ] Instagram facility analysis 18 | - [ ] Instagram hashtag review 19 | - [ ] Office 365 20 | - [ ] getuserrealm.srf 21 | - [ ] DNS 22 | - [ ] dnsdumpster 23 | - [ ] amass 24 | - [ ] horizontal (other domains owned by the same entity) and vertical (subdomain) domain enumeration 25 | - [ ] viewdns 26 | - [ ] whoisxmlapi domain research suite 27 | - [ ] riskiq 28 | - [ ] Host Enumeration 29 | - [ ] WHOIS 30 | - [ ] shodan 31 | - [ ] censys 32 | - [ ] spyse 33 | - [ ] Domain flyovers 34 | - [ ] aquatone 35 | - [ ] Document Metadata Analysis 36 | - [ ] pull large sites from google/aquatone report 37 | - [ ] pymeta 38 | - [ ] pull down manually 39 | 40 | 41 | 42 | ## Meta Sites 43 | 44 | The following links are additional lists and frameworks that can assist while performing OSINT. 45 | 46 | - [https://osintframework.com/](https://osintframework.com/) 47 | - Provides an interactive chart of actions with links for each action 48 | - Links to tools and third party sites 49 | 50 | 51 | ## Mail Blacklist Check 52 | 53 | The following services allow you to check whether a domain or IP address is present on several blacklists. Additionally, this can help troubleshoot email delivery issues while performing phishing campaigns. 54 | 55 | - [https://mxtoolbox.com/blacklists.aspx](https://mxtoolbox.com/blacklists.aspx) 56 | 57 | ## WHOIS 58 | 59 | [https://whois.arin.net/ui/advanced.jsp](https://whois.arin.net/ui/advanced.jsp) 60 | 61 | - Primary source 62 | - Manual web browsing 63 | 64 | https://viewdns.info/ 65 | 66 | - Multiple tools 67 | 68 | [https://domainbigdata.com/](https://domainbigdata.com/) 69 | 70 | - Allows host correlation based on site registrant 71 | - Third-party 72 | 73 | [https://whoisology.com/#advanced](https://whoisology.com/#advanced) 74 | 75 | - Reverse WHOIS search based on multiple parameters 76 | - Third-party 77 | 78 | [https://whoisfreaks.com/pricing/whois-database.html](https://whoisfreaks.com/pricing/whois-database.html) 79 | 80 | [https://www.whoisxmlapi.com/](https://www.whoisxmlapi.com/) 81 | 82 | - Largest dataset available (800M+ domains) 83 | - $24,000 per year for full access to current and historical WHOIS data (for commercial license) 84 | - Has an API with many functions that is more affordable than the commercial license 85 | - Free license allows for 500 queries per month 86 | 87 | ## Domains 88 | 89 | [https://domains-monitor.com/](https://domains-monitor.com/) 90 | 91 | - Allows downloading a raw list of all registered domains in all zones 92 | - Updates quarterly with updated/deleted domains 93 | - Provides list of registration emails 94 | - $90 per year for access 95 | 96 | [https://networksdb.io/](https://networksdb.io/) 97 | 98 | [https://www.expireddomains.net/](https://www.expireddomains.net/) 99 | 100 | - Monitors and lists domains that are expiring 101 | - Includes alexa rank and [archive.org](http://archive.org) details for domains, allowing users to select valuable domains 102 | - Free to signup 103 | 104 | ## DNS 105 | 106 | 107 | https://dnsdumpster.com/ 108 | 109 | [https://www.robtex.com/](https://www.robtex.com/dns-lookup/optiv.com) 110 | 111 | ## Website Lookup 112 | 113 | - [https://website.informer.com/](https://website.informer.com/optiv.com) 114 | - Gives generalized information about a website and a screenshot of the homepage. daily visitors, hosting info, alexa ranking 115 | - Paywall: no 116 | - Bot Detection: unknown 117 | - [https://archive.ph/](https://archive.ph/) 118 | - Allows snapshotting of a webpage by providing a URL. also allows retrieving screenshots and text data from previously archived sites 119 | - Similar to wayback machine 120 | - Paywall: no 121 | - Bot Detection: unknown 122 | - [https://www.page2images.com/URL-Live-Website-Screenshot-Generator](https://www.page2images.com/URL-Live-Website-Screenshot-Generator) 123 | - Generates screenshots of urls, 15 seconds or more per url 124 | - No cost solution 125 | - Bot detection: unknown 126 | 127 | ## Phishing Site Lookup 128 | 129 | - [https://www.phishtank.com/](https://www.phishtank.com/) 130 | - Crowdsourced link submission and verification allows the community to determine phish validity 131 | - Limited reliability and visibility into anything more than the URL of a potential phishing site 132 | - Indicates whether site is online or offline 133 | - No cost solutions 134 | - API: yes, email verification required. commercial use allowed, has per hour request limit 135 | - Bot detection: hCaptcha (website) 136 | - [https://openphish.com/](https://openphish.com/) 137 | - Raw feed of phishing urls 138 | - Free version updates every 12 hours, in text file format 139 | - Paid version updates more quickly and allows multiple formats (CSV or JSON) 140 | - Has IP address listing of recent phishing site 141 | - Provides global statistics of phishing attacks 142 | - What brands are being spoofed 143 | - What ASNs are most commonly hosting phishing attacks 144 | 145 | ## Twitter 146 | 147 | [https://tinfoleak.com/](https://tinfoleak.com/) 148 | 149 | - Shows devices, locations, etc. for a given Twitter handle 150 | - Requires email registration 151 | - Slow and requires capcha submitted for each request 152 | - No bulk capabilities 153 | 154 | ## Phone Number Validation 155 | 156 | [https://phonevalidator.com/phone-validator-api.aspx](https://phonevalidator.com/phone-validator-api.aspx) 157 | 158 | - Shows phone number type (CELL PHONE, LANDLINE, VOIP, TOLL-FREE or UNKNOWN) 159 | - 0.004 per number pricing ($4 per 1000 phone numbers) 160 | - Useful for smishing to confirm that you can text a phone number 161 | 162 | ## Corporate Databases 163 | 164 | [https://opencorporates.com/](https://opencorporates.com/) 165 | 166 | - Registration/incorporation articles for corporate entities 167 | - Shows registered trademarks, logos, and historical data 168 | - Shows branch locations 169 | - Can search by officer (person) as well to expand based on company involvement 170 | 171 | ## Github 172 | 173 | [https://github.com/BishopFox/GitGot](https://github.com/BishopFox/GitGot) 174 | 175 | - Searches github for potentially sensitive info 176 | - Semi-interactive, prompts user to manually review then enumerates based on feedback 177 | - Python, last commit Sep 2020 178 | 179 | ## Mobile Emulators 180 | 181 | [https://www.genymotion.com/](https://www.genymotion.com/) 182 | 183 | - SaaS based mobile emulator 184 | - Pay as you go 185 | - Focused around app testing 186 | 187 | ## Paywalled 188 | 189 | - http://www.domaincrawler.com/ 190 | 191 | 192 | ## Search engines: 193 | 194 | - Yandex - Russian google 195 | - Baidu - Chinese google 196 | - Goo - Japanese google 197 | - 2lingual.com - Can query search engines in two languages at a time, results are displayed side-by-side 198 | 199 | --------------------------------------------------------------------------------