├── grepfile.txt ├── charset.txt ├── requirements.txt ├── testwordlist.txt ├── interesting_Keywords.txt ├── generateBucketNames.py ├── .gitignore ├── README.md └── bucketkicker.py /grepfile.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /charset.txt: -------------------------------------------------------------------------------- 1 | s3 = [abcdefghijklmnopqrstuvwxyz0123456789-.] -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | certifi==2017.7.27.1 2 | chardet==3.0.4 3 | idna==2.6 4 | requests>=2.20.0 5 | urllib3==1.22 6 | xmltodict==0.11.0 7 | -------------------------------------------------------------------------------- /testwordlist.txt: -------------------------------------------------------------------------------- 1 | testbucket 2 | bucket 3 | imcraighays 4 | zabbix 5 | production-config 6 | settings 7 | fitbit 8 | archive 9 | archive1 10 | archive01 11 | archive-old 12 | com.companyname 13 | -------------------------------------------------------------------------------- /interesting_Keywords.txt: -------------------------------------------------------------------------------- 1 | password 2 | passwd 3 | token 4 | key 5 | user 6 | wp-config 7 | bak 8 | back 9 | backup 10 | old 11 | bank 12 | secret 13 | vpn 14 | xls 15 | csv 16 | confidential 17 | invoice 18 | ssn 19 | zip -------------------------------------------------------------------------------- /generateBucketNames.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import sys 4 | 5 | if len(sys.argv) != 3: 6 | print "Usage: python generateBucketNames.py wordlistfile domainname" 7 | exit(1); 8 | 9 | with open(sys.argv[1]) as f: 10 | for line in f: 11 | print line.strip()+"."+sys.argv[2].strip() -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # General 2 | .DS_Store 3 | .AppleDouble 4 | .LSOverride 5 | 6 | # Icon must end with two \r 7 | Icon 8 | 9 | 10 | # Thumbnails 11 | ._* 12 | 13 | # Files that might appear in the root of a volume 14 | .DocumentRevisions-V100 15 | .fseventsd 16 | .Spotlight-V100 17 | .TemporaryItems 18 | .Trashes 19 | .VolumeIcon.icns 20 | .com.apple.timemachine.donotpresent 21 | 22 | # Directories potentially created on remote AFP share 23 | .AppleDB 24 | .AppleDesktop 25 | Network Trash Folder 26 | Temporary Items 27 | .apdisk 28 | 29 | venv/ 30 | *char.txt 31 | bucket404.txt 32 | bucket403.txt 33 | openbucket.txt 34 | *char-*.txt 35 | 36 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # bucketkicker 2 | 3 | #### bucketkicker is a tool to quickly enumerate AWS S3 buckets verify whether or not they exist and to look for loot. It's similar to a subdomain bruteforcer but is made specifically for S3 buckets and also has some extra features that allow you to grep for delicious files as well as download interesting files if you're not afraid to quickly fill up your hard drive. 4 | 5 | [Further reading](https://medium.com/bugbountyhunting/bug-bounty-hunting-tips-3-kicking-s3-buckets-84c231939066) 6 | 7 | This is a hard fork based on the original by [jordanpotti](https://github.com/jordanpotti/AWSBucketDump) with some changes specifically around efficient listing of buckets based on found/not found/access denied HTTP responses 8 | #### [@CraigHays](https://twitter.com/craighays) 9 | 10 | ## Pre-Requisites 11 | Non-Standard Python Libraries: 12 | 13 | * xmltodict 14 | * requests 15 | * argparse 16 | 17 | Created with Python 3.6 18 | 19 | ### Install with virtualenv 20 | ```virtualenv-3.6 venv 21 | source venv/bin/activate 22 | pip install -r requirements.txt 23 | ``` 24 | 25 | ## General 26 | 27 | This is a tool that enumerates Amazon S3 buckets to confirm whether or not they exist, are open to the webm and can also look for interesting files. 28 | 29 | I have included a wordlist generater which I will improve to add more rules as time goes by. 30 | 31 | https://github.com/danielmiessler/SecLists will have all the word lists you need. If you are targeting a specific company, you will likely want to use massdns to ennumerate a list of DNS subdomains to feed into this. 32 | 33 | Original wording around grepping stuff from [jordanpotti](https://github.com/jordanpotti/AWSBucketDump) 34 | 35 | As far as word lists for grepping interesting files, that is completely up to you. The one I provided has some basics and yes, those word lists are based on files that I personally have found with this tool. 36 | 37 | Using the download feature might fill your hard drive up, you can provide a max file size for each download at the command line when you run the tool. Keep in mind that it is in bytes. 38 | 39 | I honestly don't know if Amazon rate limits this, I am guessing they do to some point but I haven't gotten around to figuring out what that limit is. By default there are two threads for checking buckets and two buckets for downloading. 40 | 41 | After building this tool, I did find an [interesting article](https://community.rapid7.com/community/infosec/blog/2013/03/27/1951-open-s3-buckets) from Rapid7 regarding this research. 42 | 43 | ## Usage: 44 | 45 | usage: bucketkicker.py [-h] [-D] [-t THREADS] -l HOSTLIST [-g GREPWORDS] [-m MAXSIZE] 46 | 47 | optional arguments: 48 | -b Write list of open buckets to openbuckets.txt only - don't look at contents - fastest way to ennumerate buckets 49 | -h, --help show this help message and exit 50 | -D Download files. This requires significant diskspace 51 | -d If set to 1 or True, create directories for each host w/ results 52 | -t THREADS number of threads 53 | -l HOSTLIST 54 | -g GREPWORDS Provide a wordlist to grep for 55 | -m MAXSIZE Maximum file size to download. 56 | 57 | python3 bucketkicker.py -d False -l -g grepfile.txt -b 58 | 59 | ## Generating random wordlists via crunch 60 | 61 | ```kali > crunch -f charset s3 -o ``` 62 | 63 | stripping out invalid line starts and ends 64 | 65 | sed '/^\./ d' | sed "/\.$/d" | sed '/^-/ d' | sed "/-$/d" 66 | 67 | 68 | ## Naming rules 69 | S3 URLs use the format http://bucketname.s3.amazonaws.com . If we replace bucketname with random strings that meet the following rules it is possible to see if buckets exist. Feed this tool with a wordlist of your choice and you're good to go. 70 | 71 | Rules from https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html 72 | - Bucket names must be at least 3 and no more than 63 characters long. 73 | - Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.). Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number. 74 | - Bucket names must not be formatted as an IP address (for example, 192.168.5.4). 75 | - When using virtual hosted–style buckets with SSL, the SSL wildcard certificate only matches buckets that do not contain periods. To work around this, use HTTP or write your own certificate verification logic. We recommend that you do not use periods (".") in bucket names. 76 | 77 | ## Outputs 78 | 79 | The script will produce the following three output files in the directory you run the script. 80 | 81 | bucket404.txt - buckets not found 82 | bucket403.txt - buckets found but access denied 83 | openbucket.txt - buckets found and content listing enabled 84 | 85 | ### Contributors 86 | 87 | [craighays](https://github.com/craighays) 88 | 89 | [jordanpotti](https://github.com/jordanpotti) 90 | 91 | [grogsaxle](https://github.com/grogsaxle) 92 | 93 | [codingo](https://github.com/codingo) 94 | 95 | [aarongorka](https://github.com/aarongorka) 96 | 97 | [BHaFSec](https://github.com/BHaFSec) 98 | 99 | [paralax](https://github.com/paralax) 100 | 101 | [fzzo](https://github.com/fzzo) 102 | 103 | [rypb](https://github.com/rypb) 104 | 105 | -------------------------------------------------------------------------------- /bucketkicker.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # AWSBucketDump is a tool to quickly enumerate AWS S3 buckets to look for loot. 4 | # It's similar to a subdomain bruteforcer but is made specifically to S3 5 | # buckets and also has some extra features that allow you to grep for 6 | # delicous files as well as download interesting files if you're not 7 | # afraid to quickly fill up your hard drive. 8 | 9 | # by Jordan Potti 10 | # @ok_bye_now 11 | 12 | from argparse import ArgumentParser 13 | import codecs 14 | import requests 15 | import xmltodict 16 | import sys 17 | import os 18 | import shutil 19 | import traceback 20 | from queue import Queue 21 | from threading import Thread, Lock 22 | 23 | 24 | bucket_q = Queue() 25 | download_q = Queue() 26 | 27 | grep_list = None 28 | 29 | arguments = None 30 | 31 | def fetch(url): 32 | print('Fetching ' + url + '...') 33 | response = requests.get(url) 34 | if response.status_code == 404: 35 | write_bucket404_file(url) 36 | if response.status_code == 403: 37 | write_bucket403_file(url) 38 | #status403(url) 39 | if response.status_code == 200: 40 | if "Content" in response.text: 41 | write_openbucket_file(url) 42 | if not arguments.bucketonly: 43 | returnedList=status200(response,grep_list,url) 44 | 45 | 46 | def bucket_worker(): 47 | while True: 48 | item = bucket_q.get() 49 | try: 50 | fetch(item) 51 | except Exception as e: 52 | traceback.print_exc(file=sys.stdout) 53 | print(e) 54 | bucket_q.task_done() 55 | 56 | def downloadWorker(): 57 | print('Download worker running...') 58 | while True: 59 | item = download_q.get() 60 | try: 61 | downloadFile(item) 62 | except Exception as e: 63 | traceback.print_exc(file=sys.stdout) 64 | print(e) 65 | download_q.task_done() 66 | 67 | directory_lock = Lock() 68 | 69 | def get_directory_lock(): 70 | directory_lock.acquire() 71 | 72 | def release_directory_lock(): 73 | directory_lock.release() 74 | 75 | 76 | def get_make_directory_return_filename_path(url): 77 | global arguments 78 | bits = url.split('/') 79 | directory = arguments.savedir 80 | for i in range(2,len(bits)-1): 81 | directory = os.path.join(directory, bits[i]) 82 | try: 83 | get_directory_lock() 84 | if not os.path.isdir(directory): 85 | os.makedirs(directory) 86 | except Exception as e: 87 | traceback.print_exc(file=sys.stdout) 88 | print(e) 89 | finally: 90 | release_directory_lock() 91 | 92 | return os.path.join(directory, bits[-1]).rstrip() 93 | 94 | interesting_file_lock = Lock() 95 | def get_interesting_file_lock(): 96 | interesting_file_lock.acquire() 97 | 98 | def release_interesting_file_lock(): 99 | interesting_file_lock.release() 100 | 101 | 102 | def write_interesting_file(filepath): 103 | try: 104 | get_interesting_file_lock() 105 | with open('interesting_file.txt', 'ab+') as interesting_file: 106 | interesting_file.write(filepath.encode('utf-8')) 107 | interesting_file.write('\n'.encode('utf-8')) 108 | finally: 109 | release_interesting_file_lock() 110 | 111 | openbucket_file_lock = Lock() 112 | def get_openbucket_file_lock(): 113 | openbucket_file_lock.acquire() 114 | 115 | def release_openbucket_file_lock(): 116 | openbucket_file_lock.release() 117 | 118 | 119 | def write_openbucket_file(bucketname): 120 | try: 121 | get_openbucket_file_lock() 122 | with open('openbucket.txt', 'ab+') as openbucket_file: 123 | openbucket_file.write(bucketname.encode('utf-8')) 124 | openbucket_file.write('\n'.encode('utf-8')) 125 | finally: 126 | release_openbucket_file_lock() 127 | 128 | 129 | bucket404_file_lock = Lock() 130 | def get_bucket404_file_lock(): 131 | bucket404_file_lock.acquire() 132 | 133 | def release_bucket404_file_lock(): 134 | bucket404_file_lock.release() 135 | 136 | 137 | def write_bucket404_file(bucketname): 138 | try: 139 | get_bucket404_file_lock() 140 | with open('bucket404.txt', 'ab+') as bucket404_file: 141 | bucket404_file.write(bucketname.encode('utf-8')) 142 | bucket404_file.write('\n'.encode('utf-8')) 143 | finally: 144 | release_bucket404_file_lock() 145 | 146 | bucket403_file_lock = Lock() 147 | def get_bucket403_file_lock(): 148 | bucket403_file_lock.acquire() 149 | 150 | def release_bucket403_file_lock(): 151 | bucket403_file_lock.release() 152 | 153 | 154 | def write_bucket403_file(bucketname): 155 | try: 156 | get_bucket403_file_lock() 157 | with open('bucket403.txt', 'ab+') as bucket403_file: 158 | bucket403_file.write(bucketname.encode('utf-8')) 159 | bucket403_file.write('\n'.encode('utf-8')) 160 | finally: 161 | release_bucket403_file_lock() 162 | 163 | 164 | 165 | def downloadFile(filename): 166 | global arguments 167 | print('Downloading {}'.format(filename) + '...') 168 | local_path = get_make_directory_return_filename_path(filename) 169 | local_filename = (filename.split('/')[-1]).rstrip() 170 | print('local {}'.format(local_path)) 171 | if local_filename =="": 172 | print("Directory..\n") 173 | else: 174 | r = requests.get(filename.rstrip(), stream=True) 175 | if 'Content-Length' in r.headers: 176 | if int(r.headers['Content-Length']) > arguments.maxsize: 177 | print("This file is greater than the specified max size... skipping...\n") 178 | else: 179 | with open(local_path, 'wb') as f: 180 | shutil.copyfileobj(r.raw, f) 181 | r.close() 182 | 183 | 184 | 185 | def print_banner(): 186 | print('''\nDescription: 187 | bucketkicker is a tool to quickly enumerate AWS S3 buckets verify whether 188 | or not they exist and to look for loot. It's similar to a subdomain 189 | bruteforcer but is made specifically to S3 buckets and also has some 190 | extra features that allow you to grep for delicous files as well as 191 | download interesting files if you're not afraid to quickly fill up 192 | your hard drive. 193 | 194 | by @CraigHays, based on AWSBucketDump by Jordan Potti 195 | @ok_bye_now''' 196 | ) 197 | 198 | 199 | def cleanUp(): 200 | print("Cleaning up files...") 201 | 202 | def status403(line): 203 | print(line.rstrip() + " is not accessible.") 204 | 205 | 206 | def queue_up_download(filepath): 207 | download_q.put(filepath) 208 | print('Collectable: {}'.format(filepath)) 209 | write_interesting_file(filepath) 210 | 211 | 212 | def status200(response,grep_list,line): 213 | print("Pilfering "+line.rstrip() + '...') 214 | objects=xmltodict.parse(response.text) 215 | Keys = [] 216 | interest=[] 217 | try: 218 | for child in objects['ListBucketResult']['Contents']: 219 | Keys.append(child['Key']) 220 | except: 221 | pass 222 | hit = False 223 | for words in Keys: 224 | words = (str(words)).rstrip() 225 | collectable = line+'/'+words 226 | if grep_list != None and len(grep_list) > 0: 227 | for grep_line in grep_list: 228 | grep_line = (str(grep_line)).rstrip() 229 | if grep_line in words: 230 | queue_up_download(collectable) 231 | break 232 | else: 233 | queue_up_download(collectable) 234 | 235 | def main(): 236 | global arguments 237 | global grep_list 238 | parser = ArgumentParser() 239 | parser.add_argument("-D", dest="download", required=False, action="store_true", default=False, help="Download files. This requires significant disk space.") 240 | parser.add_argument("-d", dest="savedir", required=False, default='', help="If -D, then -d 1 to create save directories for each bucket with results.") 241 | parser.add_argument("-l", dest="hostlist", required=True, help="") 242 | parser.add_argument("-g", dest="grepwords", required=False, help="Provide a wordlist to grep for.") 243 | parser.add_argument("-m", dest="maxsize", type=int, required=False, default=1024, help="Maximum file size to download.") 244 | parser.add_argument("-t", dest="threads", type=int, required=False, default=1, help="Number of threads.") 245 | parser.add_argument("-b", dest="bucketonly", required=False, action="store_true", default=False, help="List open buckets only") 246 | 247 | if len(sys.argv) == 1: 248 | print_banner() 249 | parser.error("No arguments given.") 250 | parser.print_usage 251 | sys.exit() 252 | 253 | 254 | # output parsed arguments into a usable object 255 | arguments = parser.parse_args() 256 | 257 | # specify primary variables 258 | with open(arguments.grepwords, "r") as grep_file: 259 | grep_content = grep_file.readlines() 260 | grep_list = [ g.strip() for g in grep_content ] 261 | 262 | if arguments.download and arguments.savedir: 263 | print("Downloads enabled (-D), save directories (-d) for each host will be created/used.") 264 | elif arguments.download and not arguments.savedir: 265 | print("Downloads enabled (-D), will be saved to current directory.") 266 | else: 267 | print("Downloads were not enabled (-D), not saving results locally.") 268 | 269 | # start up bucket workers 270 | for i in range(0,arguments.threads): 271 | print('Starting thread...') 272 | t = Thread(target=bucket_worker) 273 | t.daemon = True 274 | t.start() 275 | 276 | # start download workers 277 | for i in range(1, arguments.threads): 278 | t = Thread(target=downloadWorker) 279 | t.daemon = True 280 | t.start() 281 | 282 | with open(arguments.hostlist) as f: 283 | for line in f: 284 | bucket = 'http://'+line.rstrip()+'.s3.amazonaws.com' 285 | print('Queuing {}'.format(bucket) + '...') 286 | bucket_q.put(bucket) 287 | 288 | bucket_q.join() 289 | if arguments.download: 290 | download_q.join() 291 | 292 | cleanUp() 293 | 294 | if __name__ == "__main__": 295 | main() 296 | 297 | --------------------------------------------------------------------------------