├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md ├── LICENSE ├── README.md ├── SubDomainizer.py └── requirements.txt /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | labels: 5 | 6 | --- 7 | 8 | **Describe the bug** 9 | A clear and concise description of what the bug is. 10 | 11 | **To Reproduce** 12 | Steps to reproduce the behavior: 13 | 1. Go to '...' 14 | 2. Click on '....' 15 | 3. Scroll down to '....' 16 | 4. See error 17 | 18 | **Expected behavior** 19 | A clear and concise description of what you expected to happen. 20 | 21 | **Screenshots** 22 | If applicable, add screenshots to help explain your problem. 23 | 24 | **Desktop (please complete the following information):** 25 | - OS: [e.g. iOS] 26 | - Browser [e.g. chrome, safari] 27 | - Version [e.g. 22] 28 | 29 | **Smartphone (please complete the following information):** 30 | - Device: [e.g. iPhone6] 31 | - OS: [e.g. iOS8.1] 32 | - Browser [e.g. stock browser, safari] 33 | - Version [e.g. 22] 34 | 35 | **Additional context** 36 | Add any other context about the problem here. 37 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | labels: 5 | 6 | --- 7 | 8 | **Is your feature request related to a problem? Please describe.** 9 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 10 | 11 | **Describe the solution you'd like** 12 | A clear and concise description of what you want to happen. 13 | 14 | **Describe alternatives you've considered** 15 | A clear and concise description of any alternative solutions or features you've considered. 16 | 17 | **Additional context** 18 | Add any other context or screenshots about the feature request here. 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Neeraj Sonaniya 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Python 3.x](https://img.shields.io/badge/python-%3E3.5-yellow.svg)](https://www.python.org/) 2 | [![Twitter](https://img.shields.io/badge/twitter-@neeraj_sonaniya-blue.svg)](https://twitter.com/neeraj_sonaniya) 3 | 4 | ## Buy Me A [Coffee](https://www.buymeacoffee.com/neerajson) 5 | 6 | ## SubDomainizer 7 | 8 | SubDomainizer is a tool designed to find hidden subdomains and secrets present is either webpage, Github, and external javascripts present in the given URL. 9 | This tool also finds S3 buckets, cloudfront URL's and more from those JS files which could be interesting like S3 bucket is open to read/write, or subdomain takeover and similar case for cloudfront. 10 | It also scans inside given folder which contains your files. 11 | 12 | ## Cloud Storage Services Supported: 13 | SubDomainizer can find URL's for following cloud storage services: 14 | ``` 15 | 1. Amazon AWS services (cloudfront and S3 buckets) 16 | 2. Digitalocean spaces 17 | 3. Microsoft Azure 18 | 4. Google Cloud Services 19 | 5. Dreamhost 20 | 6. RackCDN. 21 | ``` 22 | ## Secret Key's Searching: (beta) 23 | SubDomainizer will also find secrets present in content of the page and javascripts files. 24 | Those secret finding depends on some specific keywords and *Shannon Entropy* formula. 25 | It might be possible that some secrets which searched by tool will be false positive. 26 | This secret key searching is in beta and later version might have increased accuracy for search results. 27 | 28 | ## Screenshots: 29 | 30 | ![SubDomainizer](https://i.imgur.com/x3XSamk.png) 31 | 32 | ![Sub2.0](https://i.imgur.com/TvVKabs.png) 33 | 34 | ## Installation Steps 35 | 36 | 1. Clone SubDomainzer from git: 37 | ``` 38 | git clone https://github.com/nsonaniya2010/SubDomainizer.git 39 | ``` 40 | 2. Change the directory: 41 | ``` 42 | cd SubDomainizer 43 | ``` 44 | 45 | 3. Install the requirements: 46 | 47 | ``` 48 | pip3 install -r requirements.txt 49 | ``` 50 | 4. Enjoy the Tool. 51 | 52 | ## Update to latest version: 53 | 54 | Use following command to update to latest version: 55 | 56 | ``` 57 | git pull 58 | ``` 59 | 60 | ## Usage 61 | 62 | Short Form | Long Form | Description 63 | ------------- | ------------- |------------- 64 | -u | --url | URL in which you want to find (sub)domains. 65 | -l | --listfile | File which contain list of URL's needs to be scanned. 66 | -o | --output | Output file name in which you need to save the results. 67 | -c | --cookie | Cookies which needs to be sent with request. 68 | -h | --help | show the help message and exit. 69 | -cop | --cloudop | Give file name in which you need to store cloud services results. 70 | -d | --domains | Give TLD (eg. for www.example.com you have to give example.com) to find subdomain for given TLD seperated by comma (no spaces b/w comma). 71 | -g | --gitscan | Needed if you want to get things via Github too. 72 | -gt | --gittoken | Github API token is needed, if want to scan (also needed -g also). 73 | -gop | --gitsecretop | Saving secrets to a file found in github. 74 | -k | --nossl | Use this to bypass the verification of SSL certificate. 75 | -f | --folder | Root folder which contains files/folder. 76 | -san | --subject_alt_name | Find Subject Alternative Names for all found subdomains, Options: 'all', 'same'. 77 | 78 | ## SAN options description: 79 | * all - This option will find all domains and subdomains. 80 | * same - This will only find subdomains for specific subdomains. 81 | 82 | ## Examples 83 | 84 | * To list help about the tool: 85 | ``` 86 | python3 SubDomainizer.py -h 87 | ``` 88 | * To find subdomains, s3 buckets, and cloudfront URL's for given single URL: 89 | ``` 90 | python3 SubDomainizer.py -u http://www.example.com 91 | ``` 92 | * To find subdomains from given list of URL (file given): 93 | ``` 94 | python3 SubDomainizer.py -l list.txt 95 | ``` 96 | 97 | * To save the results in (output.txt) file: 98 | ``` 99 | python3 SubDomainizer.py -u https://www.example.com -o output.txt 100 | ``` 101 | * To give cookies: 102 | ``` 103 | python3 SubDomainizer.py -u https://www.example.com -c "test=1; test=2" 104 | ``` 105 | * To scan via github: 106 | ``` 107 | python3 SubDomainizer.py -u https://www.example.com -o output.txt -gt -g 108 | ``` 109 | * No SSL Certificate Verification: 110 | ``` 111 | python3 SubDomainizer.py -u https://www.example.com -o output.txt -gt -g -k 112 | ``` 113 | * Folder Scanning: 114 | ``` 115 | python3 SubDomainizer.py -f /path/to/root/folder/having/files/and/folders/ -d example.com -gt -g -k 116 | ``` 117 | * Subject Alternative Names: 118 | ``` 119 | python3 SubDomainizer.py -u https://www.example -san all 120 | ``` 121 | * Saving secrets to a file scan found in github: 122 | ``` 123 | python3 SubDomainizer.py -u https://www.example.com -o output.txt -gt -g -gop filename_to_save 124 | ``` 125 | 126 | 127 | ## Difference in results (with cookies and without cookies on facebook.com): 128 | 129 | Results before using facebook cookies in SubDomainizer: 130 | 131 | ![BeforeCookies](https://i.imgur.com/v7igAId.png) 132 | 133 | Results after using facebook cookies in SubDomainizer: 134 | 135 | ![AfterCookies](https://i.imgur.com/QKY09mx.png) 136 | 137 | 138 | ## Changes: 139 | In the latest version (2.0) following important features are added: 140 | 1. Find Subject Alternative Names for the found subdomains. 141 | 2. Added where the secrets were found. 142 | 143 | ## License 144 | This tools is licensed under the MIT license. take a look at the [LICENSE](https://github.com/nsonaniya2010/SubDomainizer/blob/master/LICENSE) for information about it. 145 | 146 | ## Want to Help? 147 | Want to help if you like features and tools? or Liked this tool? 148 | [Help Here](https://paypal.me/BugsByNeeraj) 149 | -------------------------------------------------------------------------------- /SubDomainizer.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | ####################################### 4 | # Author: Neeraj Sonaniya # 5 | # Twitter: neeraj_sonaniya # 6 | # Linkedin: neerajsonaniya # 7 | # Facebook: neeraj.sonaniya # 8 | # Medium: neerajedwards # 9 | # Email: nsonaniya2010@gmail.com # 10 | ####################################### 11 | 12 | 13 | import termcolor 14 | import base64 15 | import json 16 | import argparse 17 | from bs4 import BeautifulSoup 18 | import requests 19 | import re 20 | import socket 21 | import ssl 22 | import htmlmin 23 | from urllib.parse import * 24 | import tldextract 25 | import sys 26 | from multiprocessing.dummy import Pool as ThreadPool 27 | from itertools import repeat 28 | from collections import Counter 29 | from math import log2 30 | import urllib3 31 | import queue 32 | import glob 33 | import os 34 | import time 35 | import warnings 36 | import colorama 37 | colorama.init() 38 | 39 | parse = argparse.ArgumentParser() 40 | parse.add_argument('-c', '--cookie', help="Cookies which needs to be sent with request. User double quotes if have more than one.") 41 | parse.add_argument('-cop', '--cloudop', help="Enter the file name in which you want to save results of cloud services finding.") 42 | parse.add_argument('-sop', '--secretop', help="Enter the file name in which you want to save results of secrets found.") 43 | parse.add_argument('-gop', '--gitsecretop', help="Enter the file name in which you want to save results of secrets found in github.") 44 | parse.add_argument('-d', '--domains', help="Enter the top-level-domain(s) seperated with comma (no spaces after comma) to extract all the subdomain of those domains") 45 | parse.add_argument('-f', '--folder', help="Folder in which files needs to be scanned.") 46 | parse.add_argument('-g', '--gitscan', help="Give this option if you wants to search for subdomain from github", action='store_true') 47 | parse.add_argument('-gt', '--gittoken', help="Finding subdomains from github") 48 | parse.add_argument('-k', '--nossl', help="Use it when SSL certiheadsficate is not verified.", action='store_true') 49 | parse.add_argument('-l', '--listfile', help="List file which contain list of URLs to be scanned for subdomains") 50 | parse.add_argument('-o', '--output', help="Enter the file name to which you want to save the results of subdomains found.") 51 | parse.add_argument('-san', '--subject_alt_name', help="Get Subject Alternative Names, Options: 'all', 'same'") 52 | parse.add_argument('-u', '--url', help="Enter the URL in which you want to find (sub)domains.") 53 | 54 | args = parse.parse_args() 55 | url = args.url 56 | listfile = args.listfile 57 | cloudop = args.cloudop 58 | secretop = args.secretop 59 | gitToken = args.gittoken 60 | isGit = args.gitscan 61 | isSSL = args.nossl 62 | folderName = args.folder 63 | is_san = args.subject_alt_name 64 | githubsc_out = args.gitsecretop 65 | 66 | jsLinkList = list() 67 | jsname = list() 68 | finalset = set() 69 | new_final_dict = dict() 70 | secret_dict = dict() 71 | git_data = dict() 72 | cloudurlset = set() 73 | finallist = list() 74 | github_secrets = set() 75 | 76 | if args.cookie: 77 | heads = {"Cookie": args.cookie, 78 | "User-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/70.0"} 79 | else: 80 | heads = {"User-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/70.0"} 81 | 82 | 83 | def argerror(urls, listfile): 84 | """ 85 | 86 | This function will get all the files path (including filename) recursively, given root folder. 87 | 88 | Parameters 89 | ---------- 90 | urls : str 91 | URL to scan for information. 92 | listfile: str 93 | Path of file which contains urls seperated by newline. 94 | """ 95 | if (urls is None and listfile is None) or (urls is not None and listfile is not None): 96 | print("Atmost one of -u/--url or -l/--listfile or -f/--folder argument is required. Exiting...") 97 | sys.exit(1) 98 | else: 99 | pass 100 | 101 | 102 | def gitArgError(gitToken, isGit): 103 | """ 104 | 105 | This function will check if both -g and -gt arguments were provided or not, required for GitHub scanning. 106 | 107 | Parameters 108 | ---------- 109 | gitToken : str 110 | Authtoken provided by github. 111 | isGit : None 112 | This argument will be used to tell the program to scan GitHub for information. 113 | """ 114 | if (gitToken is None and isGit is not None) or (gitToken is not None and isGit is None): 115 | print("Either both '-g' and '-gt' arguments are required or none required. Exiting...") 116 | sys.exit(1) 117 | else: 118 | pass 119 | 120 | 121 | def getRecursiveFolderData(rootfolder): 122 | """ 123 | 124 | This function will get all the files path (including filename) recursively, given root folder. 125 | 126 | Parameters 127 | ---------- 128 | rootfolder : str 129 | Root folder in which all files are present. 130 | 131 | Returns 132 | ---------- 133 | dict 134 | dict of files path and their data. 135 | int 136 | total number of (only) files within the root folder. 137 | """ 138 | folderData = dict() 139 | for filename in glob.iglob(rootfolder + '**/**', recursive=True): 140 | if os.path.isfile(filename): 141 | with open(filename, 'r') as file: 142 | try: 143 | folderData[filename] = file.read() 144 | except UnicodeDecodeError: 145 | pass 146 | return folderData, len(folderData) 147 | 148 | 149 | def getUrlsFromFile(): 150 | """ 151 | 152 | Getting urls from file provided in input, file contains url seperated by newline. 153 | 154 | Returns 155 | --------- 156 | list 157 | It returns list of urls from file. 158 | """ 159 | with open(args.listfile, 'rt') as f: 160 | urllst = f.readlines() 161 | urllst = [x.strip() for x in urllst if x != ''] 162 | urllst = set(urllst) 163 | return urllst 164 | 165 | 166 | class JsExtract: 167 | """ 168 | This class contain the methods to get data from Internal (Inline) and External Javascript files (Present in