├── .gitignore ├── .travis.yml ├── code-of-conduct.md ├── license ├── onioff.py ├── readme.md ├── reports └── readme.md └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | env/ 2 | build/ 3 | develop-eggs/ 4 | dist/ 5 | downloads/ 6 | eggs/ 7 | .eggs/ 8 | lib/ 9 | lib64/ 10 | parts/ 11 | sdist/ 12 | var/ 13 | *.egg-info/ 14 | *.egg 15 | *.manifest 16 | *.spec 17 | pip-log.txt 18 | pip-delete-this-directory.txt 19 | .scrapy 20 | target/ 21 | .python-version 22 | venv/ 23 | ENV/ 24 | *.pyc 25 | *.tmp 26 | *.bak 27 | *.cfg 28 | reports/onioff* 29 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "3.4" 4 | - "3.5" 5 | - "3.6" 6 | matrix: 7 | include: 8 | - python: "3.7" 9 | dist: xenial 10 | sudo: true 11 | install: 12 | - pip3 install -r requirements.txt 13 | script: 14 | - python -c "import onioff;" -------------------------------------------------------------------------------- /code-of-conduct.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of experience, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at nikolaskam@gmail.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at [http://contributor-covenant.org/version/1/4][version] 72 | 73 | [homepage]: http://contributor-covenant.org 74 | [version]: http://contributor-covenant.org/version/1/4/ 75 | -------------------------------------------------------------------------------- /license: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016-2018 Nikolaos Kamarinakis 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /onioff.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -.- coding: utf-8 -.- 3 | # onioff.py 4 | 5 | """ 6 | Copyright (C) 2016-2018 Nikolaos Kamarinakis (nikolaskam@gmail.com) 7 | See License at nikolaskama.me (https://nikolaskama.me/onioffproject) 8 | """ 9 | 10 | import socket, socks, requests, sys, os, optparse, datetime, re 11 | from urllib.request import urlopen 12 | from termcolor import colored 13 | from bs4 import BeautifulSoup 14 | from time import process_time, sleep 15 | from threading import Thread 16 | import queue as queue 17 | 18 | 19 | BLUE, RED, WHITE, YELLOW, GREEN, END = '\33[94m', '\033[91m', '\33[97m', '\33[93m', '\033[32m', '\033[0m' 20 | sys.stdout.write(RED + """ 21 | ██████╗ ███╗ ██╗██╗ ██████╗ ███████╗███████╗ 22 | ██╔═══██╗████╗ ██║██║██╔═══██╗██╔════╝██╔════╝ 23 | ██║ ██║██╔██╗ ██║██║██║ ██║█████╗ █████╗ 24 | ██║ ██║██║╚██╗██║██║██║ ██║██╔══╝ ██╔══╝ 25 | ╚██████╔╝██║ ╚████║██║╚██████╔╝██║ ██║ 26 | ╚═════╝ ╚═╝ ╚═══╝╚═╝ ╚═════╝ ╚═╝ ╚═╝ v2.1 27 | """ + END + BLUE + 28 | '\n' + '{}Onion URL Inspector ({}ONIOFF{}){}'.format(YELLOW, RED, YELLOW, BLUE).center(67) + 29 | '\n' + 'Made with <3 by: {}Nikolaos Kamarinakis ({}k4m4{}){}'.format(YELLOW, RED, YELLOW, BLUE).center(67) + 30 | '\n' + 'Version: {}2.1{}'.format(YELLOW, END).center(57) + '\n') 31 | 32 | 33 | def nowPrint(msg, error=False, ext=False, heavy=False): 34 | if ext: 35 | msg, msg_e = msg.split(' --> ') 36 | msg += ' --> ' 37 | 38 | if error: 39 | sys.stdout.write(colored(msg, 'red')) 40 | if ext: 41 | sys.stdout.write(colored(msg_e, 'red', attrs = ['bold'])) 42 | elif heavy: 43 | sys.stdout.write(colored(msg, 'yellow')) 44 | if ext: 45 | sys.stdout.write(colored(msg_e, 'yellow', attrs = ['bold'])) 46 | else: 47 | sys.stdout.write(colored(msg, 'green')) 48 | if ext: 49 | sys.stdout.write(colored(msg_e, 'green', attrs = ['bold'])) 50 | 51 | sleep(0.1) 52 | 53 | 54 | 55 | # Create TOR connection 56 | def connectTor(): 57 | global pure_ip 58 | 59 | ipcheck_url = 'https://locate.now.sh/ip' 60 | pure_ip = requests.get(ipcheck_url).text.replace('\n','') 61 | 62 | socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', 9050) 63 | socket.socket = socks.socksocket 64 | def create_connection(address, timeout=None, source_address=None): 65 | sock = socks.socksocket() 66 | sock.connect(address) 67 | return sock 68 | socket.create_connection = create_connection 69 | 70 | tor_ip = requests.get(ipcheck_url).text.replace('\n','') 71 | if pure_ip == tor_ip: 72 | nowPrint("[-] Unsuccessful Tor connection", True) 73 | nowPrint("\n[-] Exiting...\n", True) 74 | os._exit(1) 75 | else: 76 | nowPrint("\n[+] Tor running normally\n") 77 | 78 | 79 | 80 | # Perform onion status & title inspection 81 | def checkOnion(onion): 82 | global gathered, response, outFile 83 | 84 | ipcheck_url = 'https://locate.now.sh/ip' 85 | check_ip = requests.get(ipcheck_url).text.replace('\n','') 86 | if check_ip != pure_ip: 87 | try: 88 | response = urlopen(onion).getcode() 89 | except Exception as e: 90 | response = e 91 | 92 | if response == 200: 93 | try: 94 | soup = BeautifulSoup(urlopen(onion), 'lxml') 95 | response2 = soup.title.string 96 | except: 97 | response2 = 'UNAVAILABLE' 98 | 99 | show = ("[O] {} ({}ACTIVE{}) ==> '{}'").format(onion, GREEN, END, response2) 100 | gathered[onion] = 'ACTIVE', response2 101 | else: 102 | response = str(response).strip().replace(':','') 103 | response2 = 'UNAVAILABLE (Onion Inactive)' 104 | if len(response) > 2: 105 | show = ("[O] {} ({}INACTIVE{}) - {}").format(onion, RED, END, str(response).strip()) 106 | else: 107 | show = ("[O] {} ({}INACTIVE{})").format(onion, RED, END) 108 | if not options.active: 109 | gathered[onion] = 'INACTIVE', response2 110 | 111 | return show 112 | 113 | else: 114 | nowPrint("[-] Lost Tor connection", True) 115 | nowPrint("\n[-] Exiting...\n", True) 116 | os._exit(1) 117 | 118 | 119 | 120 | # Extract onion URLs from file 121 | def readFile(file): 122 | try: 123 | with open(file, 'r') as myFile: 124 | if os.path.getsize(file) > 0: 125 | onions = myFile.readlines() 126 | for onion in re.findall(r'(?:https?://)?(?:www)?\S*?\.onion', '\n'.join(onions)): 127 | onion = onion.replace('\n', '') 128 | if not len(onion) > len('.onion'): 129 | pass 130 | else: 131 | if not onion.startswith('http') and not onion.startswith('https'): 132 | onion = 'http://'+str(onion) 133 | q.put(onion) 134 | 135 | else: 136 | nowPrint("[-] Onion file is empty --> Please enter a valid file", True) 137 | nowPrint("\n[-] Exiting...\n", True) 138 | os._exit(1) 139 | 140 | q.join() 141 | myFile.close() 142 | 143 | except IOError: 144 | nowPrint("[-] Invalid onion file --> Please enter a valid file path", True) 145 | nowPrint("\n[-] Exiting...\n", True) 146 | os._exit(1) 147 | 148 | 149 | 150 | # Unique output filename generation 151 | def uniqueOutFile(checkFile): 152 | f = checkFile.split('.') 153 | if len(f) < 2: 154 | checkFile += '.txt' 155 | if os.path.exists(checkFile): 156 | fName, fType = checkFile.split('.') 157 | fList = list(fName) 158 | exists = True 159 | fName += '-{}' 160 | i = 1 161 | while exists: 162 | tempFile = (str(fName)+'.'+str(fType)) 163 | tempFile = tempFile.format(i) 164 | if os.path.exists(tempFile): 165 | i += 1 166 | else: 167 | outFile = tempFile 168 | exists = False 169 | else: 170 | outFile = checkFile 171 | 172 | return outFile 173 | 174 | 175 | 176 | def main(): 177 | 178 | if len(sys.argv[1:]) > 0: 179 | 180 | if (len(sys.argv[1:]) == 2 and sys.argv[1] == '--output') or (len(sys.argv[1:]) == 1 and sys.argv[1] == '--active') or \ 181 | (len(sys.argv[1:]) == 2 and sys.argv[1] == '--output'): 182 | nowPrint("\n[!] Invalid Options --> Use '-h' or '--help' for usage options\n", False, False, True) 183 | os._exit(1) 184 | 185 | nowPrint("\n[+] Commencing onion inspection") 186 | try: 187 | connectTor() 188 | except KeyboardInterrupt: 189 | print('\nHave a great day! :)') 190 | os._exit(1) 191 | except: 192 | nowPrint("\n[-] Tor offline --> Please make sure Tor is running", True) 193 | nowPrint("\n[-] Exiting...\n", True) 194 | os._exit(1) 195 | 196 | 197 | def inspect(): 198 | while True: 199 | onion = q.get() 200 | response = checkOnion(onion) 201 | sys.stdout.write(response+'\n') 202 | sleep(0.1) 203 | q.task_done() 204 | 205 | for i in range(concurrent): 206 | t = Thread(target=inspect) 207 | t.daemon = True 208 | t.start() 209 | 210 | try: 211 | for onion in argv: 212 | if not onion.startswith('http') and not onion.startswith('https'): 213 | nowPrint("[-] No onion URL found --> Please enter a valid URL", True) 214 | nowPrint("\n[-] Exiting...\n", True) 215 | os._exit(1) 216 | else: 217 | q.put(onion) 218 | q.join() 219 | except KeyboardInterrupt: 220 | print('\nHave a great day! :)') 221 | os._exit(1) 222 | 223 | 224 | if options.file != None: 225 | file = options.file 226 | readFile(file) 227 | 228 | 229 | try: 230 | outFile = uniqueOutFile(options.output_file) 231 | with open(outFile, 'a') as OutFile: 232 | for k, v in gathered.items(): 233 | # output format: {some_link.onion} - {page_title} 234 | if 'ACTIVE' in v[0]: 235 | OutFile.write('{} - {}'.format(k, v[1]) + '\n') 236 | else: 237 | OutFile.write('{} - {}'.format(k, v[0]) + '\n') 238 | except IOError: 239 | nowPrint("[-] Invalid path to out file given --> Please enter a valid path", True) 240 | nowPrint("\n[-] Exiting...\n", True) 241 | os._exit(1) 242 | except KeyboardInterrupt: 243 | print('\nHave a great day! :)') 244 | os._exit(1) 245 | 246 | 247 | nowPrint("[!] Onion inspection successfully complete", False, False, True) 248 | saved_msg = "\n[!] Inspection report saved as --> " + str(outFile) 249 | nowPrint(saved_msg, False, True, True) 250 | print("\nComp/tional time elapsed:", (process_time() - start)) 251 | 252 | else: 253 | nowPrint("\n[!] Use '-h' or '--help' for usage options\n", False, False, True) 254 | 255 | 256 | 257 | if __name__ == '__main__': 258 | 259 | start = process_time() 260 | 261 | optparse.OptionParser.format_epilog = lambda self, formatter: self.epilog 262 | 263 | info = 'ONIOFF v3.0 Nikolaos Kamarinakis (nikolaskama.me)' 264 | 265 | examples = ('\nExamples:\n'+ 266 | ' python3 onioff.py http://xmh57jrzrnw6insl.onion/\n'+ 267 | ' python3 onioff.py -f ~/onions.txt -o ~/report.txt\n'+ 268 | ' python3 onioff.py https://facebookcorewwwi.onion/ -o ~/report.txt\n') 269 | 270 | parser = optparse.OptionParser(epilog=examples, 271 | usage='python3 %prog {onion} [options]', 272 | prog='onioff.py', version=('ONIOFF v2.1')) 273 | 274 | parser.add_option('-f', '--file', action='store', 275 | dest='file', help='name of onion file') 276 | 277 | default = 'reports/onioff_{}'.format(str(datetime.datetime.now())[:-7].replace(' ', '_')) 278 | parser.add_option('-o', '--output', action='store', default=default, 279 | dest='output_file', help='output filename') 280 | 281 | parser.add_option('-a', '--active', action='store_true', default=False, 282 | dest='active', help='log active onions only to output file') 283 | 284 | (options, argv) = parser.parse_args() 285 | 286 | gathered = {} 287 | 288 | concurrent = 200 289 | q = queue.Queue(concurrent * 2) 290 | 291 | main() -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | 4 | 5 | ``` 6 | ██████╗ ███╗ ██╗██╗ ██████╗ ███████╗███████╗ 7 | ██╔═══██╗████╗ ██║██║██╔═══██╗██╔════╝██╔════╝ 8 | ██║ ██║██╔██╗ ██║██║██║ ██║█████╗ █████╗ 9 | ██║ ██║██║╚██╗██║██║██║ ██║██╔══╝ ██╔══╝ 10 | ╚██████╔╝██║ ╚████║██║╚██████╔╝██║ ██║ 11 | ╚═════╝ ╚═╝ ╚═══╝╚═╝ ╚═════╝ ╚═╝ ╚═╝ v2.0 12 | ``` 13 | 14 | [![Build Status](https://travis-ci.org/k4m4/onioff.svg?branch=master)](https://travis-ci.org/k4m4/onioff) 15 | [![License Badge](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/k4m4/onioff/blob/master/license) 16 | [![Compatibility](https://img.shields.io/badge/python-3-brightgreen.svg)](https://github.com/k4m4/onioff) 17 | [![Say Thanks](https://img.shields.io/badge/say-thanks-ff69b4.svg)](https://saythanks.io/to/k4m4) 18 | [![GitHub Stars](https://img.shields.io/github/stars/k4m4/onioff.svg)](https://github.com/k4m4/onioff/stargazers) 19 | 20 | > A simple tool - written in pure python - for inspecting Deep Web URLs (or onions). 21 | 22 | - Compatible with Python 3🎉. 23 | - Author: [Nikolaos Kamarinakis](mailto:nikolaskam@gmail.com) ([nikolaskama.me](https://nikolaskama.me/)) 24 | 25 | --- 26 | 27 |

✨Read my latest post: Inspecting Deep Web Links.✨

28 | 29 | --- 30 | 31 |

32 | Visit nikolaskama.me/onioffproject for more information. Check out my blog and follow me on Twitter. 33 |

34 | 35 |
36 | 37 | # Installation 38 | 39 | You can download ONIOFF by cloning the [Git Repo](https://github.com/k4m4/onioff) and simply installing its requirements: 40 | 41 | ``` 42 | ~ ❯❯❯ git clone https://github.com/k4m4/onioff.git 43 | ~ ❯❯❯ cd onioff 44 | ~/onioff ❯❯❯ pip3 install -r requirements.txt 45 | ~/onioff ❯❯❯ python3 onioff.py 46 | ``` 47 | 48 | **NOTE**: In order for ONIOFF to work, Tor must be correctly configured and running. 49 | 50 |
51 | 52 | # Usage 53 | 54 | ``` 55 | Usage: python3 onioff.py {onion} [options] 56 | 57 | Options: 58 | --version show program's version number and exit 59 | -h, --help show this help message and exit 60 | -f FILE, --file=FILE name of onion file 61 | -o OUTPUT_FILE, --output=OUTPUT_FILE 62 | output filename 63 | -a, --active log active onions only to output file 64 | 65 | Examples: 66 | python3 onioff.py http://xmh57jrzrnw6insl.onion/ 67 | python3 onioff.py -f ~/onions.txt -o ~/report.txt -a 68 | python3 onioff.py https://facebookcorewwwi.onion/ -o ~/report.txt 69 | ``` 70 | 71 | To view all available options run: 72 | 73 | ``` 74 | ~/onioff ❯❯❯ python3 onioff.py -h 75 | ``` 76 | 77 |
78 | 79 | # Demo 80 | 81 | Here's a short demo: 82 | 83 | [![ONIOFF Demo](https://nikolaskama.me/content/images/2016/09/onioff_demo.png)](https://asciinema.org/a/87557?autoplay=1) 84 | 85 | (For more demos click [here](https://asciinema.org/~k4m4)) 86 | 87 |
88 | 89 | # Developer 90 | 91 | - **Nikolaos Kamarinakis** (k4m4) - [@nikolaskama](https://twitter.com/nikolaskama) 92 | 93 |
94 | 95 | # License 96 | 97 | Copyright 2016-2017 by [Nikolaos Kamarinakis](mailto:nikolaskam@gmail.com). Some rights reserved. 98 | 99 | ONIOFF is under the terms of the [MIT License](https://www.tldrlegal.com/l/mit), following all clarifications stated in the [license file](https://raw.githubusercontent.com/k4m4/onioff/master/license). 100 | 101 |
102 | 103 | For more information head over to the [official project page](https://nikolaskama.me/onioffproject/). 104 | You can also go ahead and email me anytime at **nikolaskam{at}gmail{dot}com**. 105 | -------------------------------------------------------------------------------- /reports/readme.md: -------------------------------------------------------------------------------- 1 | # ONIOFF Reports 2 | 3 | - Author: Nikolaos Kamarinakis ([nikolaskama.me](https://nikolaskama.me/)) 4 | 5 | DO NOT DELETE THIS DIRECTORY. 6 | INSPECTIONS WITHOUT SPECIFIED OUTPUT FILENAMES ARE SAVED HERE. 7 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests 2 | beautifulsoup4 3 | pysocks 4 | termcolor 5 | lxml --------------------------------------------------------------------------------