├── README.md ├── database ├── additional_tools │ └── domains_db.py ├── db_test.py ├── init_db.py └── queries │ ├── crlf_create.sql │ ├── directories_create.sql │ ├── domains_create.sql │ ├── errors_create.sql │ ├── ports_create.sql │ └── scans_screate.sql ├── install.sh ├── main.py ├── requirements.txt ├── run.py ├── telegram ├── bot.py ├── notify.py └── setup.py ├── tools └── online.py └── web └── setup.sh /README.md: -------------------------------------------------------------------------------- 1 | ## 📌 Description 2 | ICU is a tool to constantly keep an updated database of domains and subdomains, by regularly scanning domains for subdomains with the most common subdomain scanners. 3 | 4 | ICU works by creating a database with domains and a crontask to launch the subdomain scanners script. You can launch this script manually as well. You can also keep control of your domains and subdomains with the main.py script or with the telegram bot. There is also a simple web application that is meant for a quick view of your domains. This web application is not meant yet for a large number of domains. 5 | 6 | 7 | # Install 8 | ``` 9 | git clone https://github.com/003random/ICU 10 | cd ICU 11 | ./install.sh 12 | ``` 13 | The installation script asks for various things, including your MySQL database username and password. These will be saved in credentials.py. You can always change these credentials later on. 14 | 15 | ## Optional (recommended) 16 | ICU also uses [Subfinder]("https://github.com/Ice3man543/subfinder") and [Amass]("https://github.com/caffix/amass/"). 17 | You need to install those as well. You need to have GO for those tools. [Here]("https://www.digitalocean.com/community/tutorials/how-to-install-go-on-debian-8") you can find how to install GO. 18 | After you've installed GO; Execute the following commands to install Amass and Subfinder: 19 | ``` 20 | go get github.com/caffix/amass 21 | go get github.com/Ice3man543/subfinder 22 | ``` 23 | 24 | ## Setting up the MySQL server 25 | 26 | ``` 27 | 28 | $sudo mysql_secure_installation 29 | 30 | Would you like to setup VALIDATE PASSWORD plugin? 31 | 32 | Press y|Y for Yes, any other key for No: n 33 | 34 | Remove anonymous users? (Press y|Y for Yes, any other key for No) : y 35 | 36 | Success. 37 | 38 | Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y 39 | 40 | Success. 41 | 42 | Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y 43 | 44 | - Dropping test database... 45 | 46 | Success. 47 | 48 | - Removing privileges on test database... 49 | 50 | Success. 51 | 52 | Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y 53 | 54 | Success. 55 | 56 | All done! 57 | 58 | ``` 59 | 60 | # Telegram 61 | ICU also includes a telegram bot and notifications part. If you want to use this, you will have to include your telegram bot token in credentials.py. You can get a telegram bot token [here]("https://core.telegram.org/bots#3-how-do-i-create-a-bot"). Next off, you need to run setup.py in /telegram, and then send /start to the bot. This will save your chat_id to credentials.py so it can be used for authentication with the bot, and to send the notifications to. 62 | 63 | # Modules 64 | The following modules are used: MySQLdb, telegram, random, sys, os, datetime, logging, time. 65 | 66 | The install script offers an option to install the modules from requirements.txt. This requires pip to be installed. If, for some reason, some modules are still missing. Then install these modules. The most important one is MySQLdb. [here]("https://stackoverflow.com/questions/25865270/how-to-install-python-mysqldb-module-using-pip") you can read how to install MySQLdb. 67 | 68 | # Extra 69 | To get ICU up and running, requires some simple skills. If you need serious help, you can contact me via twitter. 70 | 71 | # Credits 72 | Credits to: 73 | [Subfinder]("https://github.com/Ice3man543/subfinder"), [Amass]("https://github.com/caffix/amass/"), [Sublist3r]("https://github.com/aboul3la/Sublist3r")! 74 | 75 | # Images 76 | ![main.py](https://poc-server.com/github/ICU/main.py.png) 77 | ___ 78 | ![Telegram Bot](https://poc-server.com/github/ICU/telegrambot.png) 79 | 80 | 81 | *Created by [003random](http://hackerone.com/003random) - [@003random](https://twitter.com/rub003) - [003random.com](https://poc-server.com/blog/)* 82 | 83 | 84 | 85 | -------------------------------------------------------------------------------- /database/additional_tools/domains_db.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import os, sys, MySQLdb, time 3 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../../") 4 | import credentials 5 | 6 | try: 7 | domain = sys.argv[1].strip() 8 | scanId = sys.argv[2] 9 | 10 | if not os.path.exists("/tmp/ICU"): 11 | os.makedirs("/tmp/ICU") 12 | 13 | if not os.path.exists("/tmp/ICU/"+domain+"/"): 14 | os.makedirs("/tmp/ICU/"+domain+"/") 15 | 16 | #Add new subdomain scanners here. Make sure to let them save the output to /tmp/ICU/{domain}/doains-all.txt 17 | os.system(os.path.dirname(os.path.abspath(__file__)) + "/../../tools/dependencies/sublister/sublist3r.py -o /tmp/ICU/"+domain+"/domains-all.txt -d "+domain) 18 | time.sleep(2) 19 | 20 | try: 21 | #Subfinder 22 | os.system("subfinder -d " + domain + " -v -o /tmp/ICU/"+domain+"/domains-subfinder.txt --timeout 6") 23 | time.sleep(2) 24 | 25 | #Amass 26 | os.system("amass -o /tmp/ICU/"+domain+"/domains-amass.txt -d " + domain) 27 | time.sleep(2) 28 | except Exception as e: 29 | print "An error occured; You probably dont have either subfinder or amass installed. Check the README.md to see you how to install them. Error: " 30 | print str(e) 31 | 32 | 33 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 34 | cursor = connection.cursor() 35 | 36 | #Retrieve all info from a top domain and its subdomains, so we can use this data instead of opening new db connections later on 37 | cursor.execute("select Domain, TopDomainID, Active, Program, DomainID, scan_Id from domains where TopDomainID = (select DomainID from domains where Domain = %s) or Domain = %s", (domain, domain)) 38 | database_data = cursor.fetchall() 39 | database_domains = [d[0] for d in database_data] 40 | 41 | non_active_subdomains = [x[0] for x in database_data if ord(x[2]) == False] 42 | program = [x[3] for x in database_data if x[0] == domain][0] 43 | topDomainID = [x[4] for x in database_data if x[0] == domain][0] 44 | 45 | #All the domains from the subdomain scanners 46 | domains_all = open("/tmp/ICU/"+domain+"/domains-all.txt",'r').read().split('\n') 47 | 48 | try: 49 | #Domains from subfinder 50 | domains_subfinder = open("/tmp/ICU/"+domain+"/domains-subfinder.txt",'r').read().split('\n') 51 | 52 | #Domains from amass 53 | domains_amass = open("/tmp/ICU/"+domain+"/domains-amass.txt",'r').read().split('\n') 54 | 55 | #Add the subfinder domains 56 | domains_all.extend(x for x in domains_subfinder if x not in domains_all) 57 | 58 | #unique 59 | domains_all = list(set(domains_all)) 60 | 61 | #Add the amass domains 62 | domains_all.extend(x for x in domains_amass if x not in domains_all) 63 | 64 | #unique 65 | domains_all = list(set(domains_all)) 66 | except Exception as e: 67 | print "An error occured; You probably dont have either subfinder or amass installed. Check the README.md to see you how to install them. Error: " 68 | print str(e) 69 | 70 | #Add all the database subdomain to it 71 | domains_all.extend(x for x in database_domains if x not in domains_all) 72 | 73 | #unique -- Unique each time after adding a new list, to limit ram usage 74 | domains_all = list(set(domains_all)) 75 | 76 | #Put all the online domains in a domains-online.txt 77 | os.system(os.path.dirname(os.path.abspath(__file__)) + "/../../tools/online.py /tmp/ICU/"+domain+"/domains-all.txt /tmp/ICU/"+domain+"/domains-online.txt") 78 | 79 | #Convert online domains to array 80 | domains_online = open("/tmp/ICU/"+domain+"/domains-online.txt",'r').read().split('\n') 81 | 82 | #Loop through every subdomain 83 | for sub_domain in domains_all: 84 | #Get the scanID to insert. If the domains was already in the db and isnt changed, then keep the old scanID. otherwise use the scanID of the current scan 85 | insertScanId = scanId if not [x[5] for x in database_data if x[0] == sub_domain] else [x[5] for x in database_data if x[0] == sub_domain][0] 86 | 87 | #If the subdomain is online 88 | if sub_domain in domains_online: 89 | active=True 90 | #If the subdomain used to be offline, give it the current scanID 91 | if sub_domain in non_active_subdomains: 92 | insertScanId = scanId 93 | else: 94 | active=False 95 | 96 | 97 | if sub_domain: 98 | #Insert the new values, or update them if they already existed 99 | cursor.execute("INSERT INTO domains (Program, TopDomainID, Active, InScope, Domain, scan_Id) VALUES (%s, %s, %s, %s, %s, %s) ON DUPLICATE KEY UPDATE Active = %s, LastModified = now(), scan_Id = %s", (program, topDomainID, active, 1, sub_domain, insertScanId, active, insertScanId)) 100 | connection.commit() 101 | 102 | cursor.close () 103 | connection.close () 104 | except Exception as e: 105 | #Handle the errors, and save them to the database 106 | print "error in domains_db.py with main domain; " + domain 107 | if scanId == "NULL": 108 | scanId = None 109 | cursor.execute("INSERT INTO errors (Domain, ErrorDescription, Error, Script, scan_Id) VALUES (%s, %s, %s, %s, %s) ", (domain, "error in domains_db.py with main domain; "+domain, e, "sublister_to_db.py", scanId)) 110 | connection.commit() 111 | cursor.close() 112 | connection.close() 113 | print e 114 | sys.exit() 115 | -------------------------------------------------------------------------------- /database/db_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import MySQLdb 3 | import sys 4 | import os 5 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../") 6 | import credentials 7 | 8 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 9 | 10 | cursor = connection.cursor () 11 | cursor.execute (""" 12 | SELECT COUNT(*) 13 | FROM information_schema.tables 14 | WHERE table_name = '{0}' 15 | """.format("domains")) 16 | 17 | if cursor.fetchone()[0] == 1: 18 | print " [+] Table domains found" 19 | else: 20 | print " [-] Hmm... No table 'domains' was found in the database recon. Did you run the initialize script?" 21 | 22 | 23 | cursor.execute (""" 24 | SELECT COUNT(*) 25 | FROM information_schema.tables 26 | WHERE table_name = '{0}' 27 | """.format("errors")) 28 | 29 | if cursor.fetchone()[0] == 1: 30 | print " [+] Table errors found" 31 | else: 32 | print " [-] Hmm... No table 'errors' was found in the database recon. Did you run the initialize script?" 33 | 34 | cursor.execute (""" 35 | SELECT COUNT(*) 36 | FROM information_schema.tables 37 | WHERE table_name = '{0}' 38 | """.format("scans")) 39 | 40 | if cursor.fetchone()[0] == 1: 41 | print " [+] Table scans found" 42 | else: 43 | print " [-] Hmm... No table 'scans' was found in the database recon. Did you run the initialize script?" 44 | 45 | 46 | cursor.close () 47 | connection.close () 48 | sys.exit() 49 | 50 | -------------------------------------------------------------------------------- /database/init_db.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import MySQLdb 4 | import sys 5 | import os 6 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../") 7 | import credentials 8 | 9 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password) 10 | cursor = connection.cursor () 11 | 12 | cursor.execute (""" 13 | CREATE DATABASE recon; 14 | """) 15 | 16 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 17 | cursor = connection.cursor () 18 | 19 | cursor.execute (""" 20 | CREATE TABLE domains 21 | ( 22 | DomainID int NOT NULL AUTO_INCREMENT, 23 | Program varchar(255) not null, 24 | TopDomainID int default null, 25 | Active bit NOT NULL DEFAULT 0, 26 | InScope bit NOT NULL DEFAULT 0, 27 | Domain varchar(100) not null, 28 | LastModified timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL, 29 | scan_Id int, 30 | PRIMARY KEY (DomainID), 31 | FOREIGN KEY (TopDomainID) REFERENCES domains(DomainID), 32 | UNIQUE (Domain) 33 | ); 34 | """) 35 | 36 | cursor.execute (""" 37 | CREATE TABLE errors 38 | ( 39 | ErrorID int NOT NULL AUTO_INCREMENT, 40 | Domain varchar(255), 41 | ErrorDescription varchar(255), 42 | Error varchar(255), 43 | Script varchar(255), 44 | ErrorDate timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL, 45 | scan_ID int, 46 | PRIMARY KEY (ErrorID) 47 | ); 48 | """) 49 | 50 | 51 | cursor.execute (""" 52 | CREATE TABLE scans 53 | ( 54 | ScanID int NOT NULL AUTO_INCREMENT, 55 | StartDate datetime NOT NULL, 56 | EndDate datetime, 57 | primary key (ScanID) 58 | ); 59 | """) 60 | 61 | cursor.close () 62 | connection.close () 63 | sys.exit() 64 | -------------------------------------------------------------------------------- /database/queries/crlf_create.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE crlf 2 | ( 3 | CRLFID int NOT NULL AUTO_INCREMENT, 4 | DomainID int NOT NULL, 5 | Payload varchar(255) not null, 6 | Active bit NOT NULL DEFAULT 0, 7 | PRIMARY KEY (CRLFID), 8 | FOREIGN KEY (DomainID) REFERENCES domains(DomainID) 9 | ); 10 | 11 | -------------------------------------------------------------------------------- /database/queries/directories_create.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE directories 2 | ( 3 | DirectoryID int NOT NULL AUTO_INCREMENT, 4 | DomainID int NOT NULL, 5 | Directory varchar(255) not null, 6 | File bit NOT NULL DEFAULT 0, 7 | Active bit NOT NULL DEFAULT 0, 8 | PRIMARY KEY (DirectoryID), 9 | FOREIGN KEY (DomainID) REFERENCES domains(DomainID) 10 | ); 11 | -------------------------------------------------------------------------------- /database/queries/domains_create.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE domains 2 | ( 3 | DomainID int NOT NULL AUTO_INCREMENT, 4 | Program varchar(255) not null, 5 | TopDomainID int default null, 6 | Active bit NOT NULL DEFAULT 0, 7 | InScope bit NOT NULL DEFAULT 0, 8 | Domain varchar(100) not null, 9 | PRIMARY KEY (DomainID), 10 | FOREIGN KEY (TopDomainID) REFERENCES domains(DomainID), 11 | UNIQUE (Domain) 12 | ); 13 | 14 | -------------------------------------------------------------------------------- /database/queries/errors_create.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE errors 2 | ( 3 | ErrorID int NOT NULL AUTO_INCREMENT, 4 | Domain varchar(255), 5 | ErrorDescription varchar(255), 6 | Error varchar(255), 7 | Script varchar(255), 8 | ErrorDate datetime DEFAULT CURRENT_TIMESTAMP NOT NULL, 9 | PRIMARY KEY (ErrorID) 10 | ); 11 | -------------------------------------------------------------------------------- /database/queries/ports_create.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE ports 2 | ( 3 | PortID int NOT NULL AUTO_INCREMENT, 4 | DomainID int NOT NULL, 5 | Port int NOT NULL, 6 | PortInfo varchar(255), 7 | Active bit NOT NULL DEFAULT 0, 8 | PRIMARY KEY (PortID), 9 | FOREIGN KEY (DomainID) REFERENCES domains(DomainID) 10 | ); 11 | 12 | -------------------------------------------------------------------------------- /database/queries/scans_screate.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE scans ( 2 | ScanID int NOT NULL AUTO_INCREMENT, 3 | StartDate datetime NOT NULL, 4 | EndDate datetime, 5 | primary key (ScanID) 6 | ); 7 | 8 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | echo "[+] Installing sublist3r" 4 | git clone https://github.com/aboul3la/Sublist3r.git tools/dependencies/sublister 5 | 6 | echo "----------------------------------------------------------------" 7 | 8 | echo "[?] Do you want to pip install the requirements.txt?" 9 | echo "[Y/n]" 10 | read choice_pip_requirements 11 | 12 | if [ "$choice_pip_requirements" == "Y" ] || [ "$choice_pip_requirements" == "y" ] || [ -z "$choice_pip_requirements" ]; then 13 | pip install -r requirements.txt 14 | sudo apt-get install python-mysqldb 15 | else 16 | echo "[!] Make sure you have the right modules installed. You can check which modules are used in requirements.txt" 17 | fi 18 | 19 | echo "----------------------------------------------------------------" 20 | 21 | echo "[?] What is the database username?" 22 | read database_username 23 | 24 | echo "[?] What is the database password?" 25 | read database_password 26 | 27 | echo "[?] What is the database server? e.g. localhost (most common), if it runs on the same server." 28 | read database_server 29 | 30 | echo " 31 | database_username = \"$database_username\" 32 | database_password = \"$database_password\" 33 | database_server = \"$database_server\" 34 | database_name = \"recon\" 35 | 36 | telegram_bot_token = \"\" 37 | telegram_chat_id = \"\" 38 | " > "credentials.py" 39 | 40 | echo "[+] Creating database 'recon' with tables 'domains', scans and 'errors'" 41 | python database/init_db.py 42 | 43 | echo "[+] Checking if the database was created successfully" 44 | python database/db_test.py 45 | 46 | echo "----------------------------------------------------------------" 47 | 48 | echo "[+] Adding a cron task to run 'run.py' every 12 hours. You can edit this with the command 'crontab -e'" 49 | echo "[?] Adding the path to crontab. If this isn't the right path to the file, please edit this with the command 'crontab -e'" 50 | #write out current crontab 51 | crontab -l > mycron 52 | #echo new cron into cron file 53 | echo "0 */12 * * * python $(pwd)/run.py" >> mycron 54 | #install new cron file 55 | crontab mycron 56 | rm mycron 57 | echo "[+] Crontab task created!" 58 | 59 | echo "----------------------------------------------------------------" 60 | 61 | echo "[?] Do you want to create ICU.php? A simple web interface for the domains." 62 | echo "[Y/n]" 63 | read choice_web 64 | 65 | if [ "$choice_web" == "Y" ] || [ "$choice_web" == "y" ] || [ -z "$choice_web" ]; then 66 | ./web/setup.sh "$database_username" "$database_password" "$database_server" 67 | echo "[+] ICU.php was added." 68 | fi 69 | 70 | echo "----------------------------------------------------------------" 71 | 72 | echo "[!] All set!" 73 | 74 | echo "----------------------------------------------------------------" 75 | 76 | echo "[+] If you want to use the telegram options, please add your telegram bot token in credentials.py." 77 | echo "[?] Do you want to run main.py? This script lets you manage your domains." 78 | echo "[Y/n]" 79 | read choice_main 80 | 81 | if [ $choice_main = "Y" ] || [ $choice_main = "y" ] || [ -z $choice_main ]; then 82 | python main.py 83 | fi 84 | -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import os, sys, MySQLdb, time 4 | import credentials 5 | 6 | class bcolors: 7 | HEADER = '\033[95m' 8 | OKBLUE = '\033[94m' 9 | OKGREEN = '\033[92m' 10 | WARNING = '\033[93m' 11 | FAIL = '\033[91m' 12 | ENDC = '\033[0m' 13 | BOLD = '\033[1m' 14 | UNDERLINE = '\033[4m' 15 | 16 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 17 | cursor = connection.cursor() 18 | 19 | def exit_program(): 20 | sys.exit() 21 | 22 | 23 | def list_scan_domains(): 24 | scan_id = raw_input('[Scan ID] > ') 25 | cursor.execute ("select Domain, Program from domains where scan_Id = %s", (scan_id,)) 26 | data = cursor.fetchall() 27 | 28 | for row in data: 29 | print bcolors.BOLD + row[0] + bcolors.ENDC + " - " + bcolors.OKGREEN + row[1] + bcolors.ENDC 30 | 31 | raw_input("\nPress any key to go back...") 32 | start() 33 | 34 | 35 | 36 | def list_subdomains(): 37 | sub_domain = raw_input('[Domain] > ') 38 | cursor.execute ("select Domain, Active from domains where TopDomainID = (select DomainID from domains where Domain = %s) order by Domain", (sub_domain,)) 39 | data = cursor.fetchall() 40 | 41 | for row in data: 42 | if ord(row[1]): 43 | print bcolors.OKGREEN + row[0] + bcolors.ENDC 44 | else: 45 | print bcolors.WARNING + row[0] + bcolors.ENDC 46 | 47 | raw_input("\nPress any key to go back...") 48 | start() 49 | 50 | 51 | def run_scan(): 52 | print "[!] Running scan..." 53 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/run.py") 54 | 55 | 56 | 57 | def run_subdomain_scan_on_target(top_domain_par = None): 58 | if top_domain_par is None: 59 | print "What is the domain?" 60 | top_domain = raw_input('[Domain] > ') 61 | else: 62 | top_domain = top_domain_par 63 | 64 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/database/additional_tools/domains_db.py " + top_domain + " NULL") 65 | 66 | def delete_top_domain(): 67 | print "What is the domain? " 68 | top_domain = raw_input('[Domain] > ') 69 | 70 | cursor.execute ("select * from domains where Domain = %s", (top_domain,)) 71 | data = cursor.fetchone() 72 | 73 | top_domain = int(data[0]) 74 | 75 | cursor.execute("delete from domains where topDomainID = %s", (top_domain,)) 76 | connection.commit() 77 | 78 | cursor.execute("delete from domains where DomainID = %s", (top_domain,)) 79 | connection.commit() 80 | 81 | print bcolors.OKGREEN + "Domain with its subdomains deleted" + bcolors.ENDC 82 | raw_input("\nPress any key to go back...") 83 | start() 84 | 85 | 86 | def insert_topdomain(top_domain_par = None): 87 | if top_domain_par is None: 88 | print "What is the domain? " 89 | top_domain = raw_input('[Domain] > ') 90 | else: 91 | top_domain = top_domain_par 92 | 93 | cursor.execute ("select * from domains where Domain = %s", (top_domain,)) 94 | data = cursor.fetchone() 95 | 96 | if data: 97 | print bcolors.WARNING + "Domains already exists; program: " + str(data[1]) + ", last modified: " + str(data[6]) + bcolors.ENDC 98 | raw_input("Press any key to go back...") 99 | start() 100 | 101 | 102 | print "What is the program? " 103 | program = raw_input('[program] > ') 104 | 105 | print "Inscope? " 106 | inscope = raw_input('[Y/n] > ') 107 | if "n" not in inscope.lower(): 108 | inscope = 1 109 | else: 110 | inscope = 0 111 | 112 | 113 | cursor.execute("INSERT INTO domains (Program, InScope, Domain, Active) VALUES (%s, %s, %s, %s)", (program, inscope, top_domain, 1)) 114 | connection.commit() 115 | 116 | print bcolors.OKGREEN + "Domain added" + bcolors.ENDC 117 | raw_input("\nPress any key to go back...") 118 | start() 119 | 120 | 121 | 122 | 123 | def insert_subdomain(top_domain_par = None): 124 | if top_domain_par is None: 125 | print "What is the (top)domain? " 126 | top_domain = raw_input('[(Top)Domain] > ') 127 | else: 128 | top_domain = top_domain_par 129 | 130 | cursor.execute ("select * from domains where Domain = %s", (top_domain,)) 131 | data = cursor.fetchone() 132 | 133 | if not data: 134 | print bcolors.WARNING + "Domain not found!" + bcolors.ENDC 135 | print "Do you want to add it? " 136 | answer = raw_input('[Y/n] > ') 137 | if "n" not in answer.lower(): 138 | insert_topdomain(top_domain) 139 | else: 140 | raw_input("Press any key to go back...") 141 | start() 142 | 143 | program = str(data[1]) 144 | topdomainid = int(data[0]) 145 | 146 | 147 | 148 | print "What is the subdomain? " 149 | sub_domain = raw_input('[(Subdomain] > ') 150 | 151 | cursor.execute ("select * from domains where Domain = %s", (sub_domain,)) 152 | data = cursor.fetchone() 153 | 154 | if data: 155 | print bcolors.WARNING + "Subdomains already exists; program: " + str(data[1]) + ", last modified: " + str(data[6]) + bcolors.ENDC 156 | does_not_exist = False 157 | else: 158 | does_not_exist = True 159 | 160 | if does_not_exist: 161 | print "Inscope? " 162 | inscope = raw_input('[Y/n] > ') 163 | if "n" not in inscope.lower(): 164 | inscope = 1 165 | else: 166 | inscope = 0 167 | 168 | cursor.execute("INSERT INTO domains (Program, InScope, Domain, Active, TopDomainID) VALUES (%s, %s, %s, %s, %s)", (program, inscope, sub_domain, 1, topdomainid)) 169 | connection.commit() 170 | 171 | print bcolors.OKGREEN + "Domain added" + bcolors.ENDC 172 | 173 | print "Do you want to add another one?? " 174 | another_one = raw_input('[Y/n] > ') 175 | if "n" not in another_one.lower(): 176 | 177 | insert_subdomain(top_domain) 178 | 179 | raw_input("\nPress any key to go back...") 180 | start() 181 | 182 | 183 | 184 | def list_domains(): 185 | cursor.execute ("select Domain, Active from domains where TopDomainID is NULL") 186 | data = cursor.fetchall() 187 | 188 | for row in data: 189 | if ord(row[1]): 190 | print bcolors.OKGREEN + row[0] + bcolors.ENDC 191 | else: 192 | print bcolors.WARNING + row[0] + bcolors.ENDC 193 | 194 | raw_input("\nPress any key to go back...") 195 | start() 196 | 197 | 198 | 199 | 200 | options = {1 : insert_topdomain, 201 | 2 : list_subdomains, 202 | 3 : list_domains, 203 | 4 : insert_subdomain, 204 | 5 : run_subdomain_scan_on_target, 205 | 6 : delete_top_domain, 206 | 7 : list_scan_domains, 207 | 8 : run_scan, 208 | 9 : exit_program 209 | } 210 | 211 | 212 | def start(): 213 | os.system('clear') 214 | cursor.execute ("select count(*) from domains where TopDomainID is NULL;") 215 | data = cursor.fetchall() 216 | sub_domains = data[0][0] 217 | 218 | cursor.execute ("select count(TopDomainID) from domains") 219 | data = cursor.fetchall() 220 | top_domains = data[0][0] 221 | 222 | domains = top_domains + sub_domains 223 | 224 | banner = """ {0}---------------+------- 225 | Domains | {1} 226 | Top-domains | {2} 227 | Sudomains | {3} 228 | ---------------+-------{4}""" 229 | 230 | choices = """ 231 | 1. Add domain 232 | 2. List subdomains from domain 233 | 3. List all (top)domains 234 | 4. Add subdomain 235 | 5. Run subdomain scan on top domain 236 | 6. Delete a (top)domain 237 | 7. Get domains from scan ID 238 | 8. Run scan on all domains 239 | 9. Exit 240 | """ 241 | 242 | print banner.format(bcolors.HEADER, domains, sub_domains, top_domains, bcolors.ENDC) 243 | 244 | print choices.format() 245 | 246 | try: 247 | num = raw_input('> ') 248 | num = int(num) 249 | except: 250 | print "Provide a number please," 251 | raw_input("\nPress any key to go back...") 252 | start() 253 | start() 254 | 255 | options[num]() 256 | 257 | 258 | start() 259 | cursor.close () 260 | connection.close () 261 | sys.exit() 262 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | mysql-python==1.2.3 2 | telegram==0.0.1 3 | python-telegram-bot==10.0.1 4 | 5 | 6 | # All the modules used are: 7 | 8 | # MySQLdb 9 | # telegram 10 | # random 11 | # sys 12 | # os 13 | # datetime 14 | # logging 15 | # time 16 | -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | try: 3 | import sys, os, MySQLdb, datetime, credentials 4 | 5 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 6 | cursor = connection.cursor () 7 | 8 | cursor.execute ("insert into scans (StartDate) values (CURRENT_TIMESTAMP)") 9 | connection.commit() 10 | scanId = cursor.lastrowid 11 | cursor.execute ("select Domain from domains where TopDomainID is NULL order by Domain") 12 | data = cursor.fetchall () 13 | connection.close() 14 | 15 | for row in data: 16 | print "Starting subdomain scans on " + row[0] 17 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/database/additional_tools/domains_db.py " + row[0] + " " + str(scanId)) 18 | 19 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 20 | cursor = connection.cursor () 21 | cursor.execute ("update scans set EndDate = CURRENT_TIMESTAMP where ScanID = %s", (scanId)) 22 | connection.commit() 23 | connection.close() 24 | 25 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/telegram/notify.py " + str(scanId)) 26 | except Exception, e: 27 | print "error: " + str(e) 28 | 29 | if not os.path.exists(os.path.dirname(os.path.abspath(__file__)) + "/logs"): 30 | os.makedirs(os.path.dirname(os.path.abspath(__file__)) + "/logs") 31 | 32 | with open(os.path.dirname(os.path.abspath(__file__)) + '/logs/run_logs.txt', 'w+') as the_file: 33 | the_file.write(str(e) + "\n") 34 | -------------------------------------------------------------------------------- /telegram/bot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import logging, datetime, MySQLdb, os, telegram, sys 5 | from telegram import (InlineKeyboardMarkup, ReplyKeyboardMarkup, ReplyKeyboardRemove, InlineKeyboardButton) 6 | from telegram.ext import Updater, CommandHandler, CallbackQueryHandler, ConversationHandler, MessageHandler, Filters 7 | from random import randint 8 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../") 9 | import credentials 10 | 11 | 12 | # Enable logging 13 | logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', 14 | level=logging.INFO) 15 | 16 | telegram_bot_token = credentials.telegram_bot_token 17 | 18 | logger = logging.getLogger(__name__) 19 | 20 | BUTTON, CUSTOM_SCAN_ID_INPUT, ADD_DOMAIN, EDIT_DOMAIN, GET_DOMAINS, CONTAINS = range(6) 21 | 22 | def start(bot, update): 23 | print "---------- Start -----------" 24 | if str(update.message.chat_id) != str(credentials.telegram_chat_id): 25 | update.message.reply_text("Not authorized! If you think this is a mistake, please check if your chat_id is in credentials.py. You can also run setup.py in /telegram to setup the right credentials for your telegram.") 26 | return 27 | 28 | user = update.message.from_user 29 | hour = datetime.datetime.now().hour 30 | greeting = "Good morning " + str(user['first_name']) if 5<=hour<12 else "Good afternoon " + str(user['first_name']) if hour<18 else "Good evening " + str(user['first_name']) 31 | 32 | keyboard = [[InlineKeyboardButton("Data", callback_data='data-' + str(randint(0, 999))), 33 | InlineKeyboardButton("Scans", callback_data='scan-' + str(randint(0, 999)))], 34 | [InlineKeyboardButton("✘ Close", callback_data='close-' + str(randint(0, 999)))]] 35 | 36 | reply_markup = InlineKeyboardMarkup(keyboard) 37 | update.message.reply_text(greeting, reply_markup=reply_markup) 38 | print "button before" 39 | return BUTTON 40 | 41 | 42 | def button(bot, update): 43 | query = update.callback_query 44 | #each callback_data attr has a random int after the '-' to make the button unique each time so the spinning loading circle goes away after returning to an excisting button 45 | choice = query.data.split('-')[0] 46 | r = str(randint(0, 99)) 47 | 48 | header_1 = "Catagory:" 49 | keyboard_1 = [[InlineKeyboardButton("Data", callback_data='data-' + r), 50 | InlineKeyboardButton("Scans", callback_data='scan-' + r)], 51 | [InlineKeyboardButton("✘ Close", callback_data='close-' + r)]] 52 | 53 | header_2 = "Action:" 54 | keyboard_2 = [[InlineKeyboardButton("Latest", callback_data='latest-' + r), 55 | InlineKeyboardButton("Custom", callback_data='custom-' + r), 56 | InlineKeyboardButton("Run", callback_data='run-' + r)], 57 | [InlineKeyboardButton("« Back to catagories", callback_data='back-' + r)]] 58 | 59 | header_3 = "Action:" 60 | keyboard_3 = [[InlineKeyboardButton("Add", callback_data='add-' + r), 61 | InlineKeyboardButton("Edit", callback_data='edit-' + r), 62 | InlineKeyboardButton("Get", callback_data='get-' + r)], 63 | [InlineKeyboardButton("« Back to catagories", callback_data='back-' + r)]] 64 | 65 | header_4 = "It looks like a scan is already running. Want to start a new one?" 66 | keyboard_4 = [[InlineKeyboardButton("Yes", callback_data='yes_scan-' + r), 67 | InlineKeyboardButton("No", callback_data='no_scan-' + r)], 68 | [InlineKeyboardButton("« Back to scans", callback_data='back_scan-' + r)]] 69 | 70 | header_5 = "Action:" 71 | keyboard_5 = [[InlineKeyboardButton("(top)Domains", callback_data='topdomains-' + r), 72 | InlineKeyboardButton("Subdomains", callback_data='subdomains-' + r), 73 | InlineKeyboardButton("Contains", callback_data='contains-' + r)], 74 | [InlineKeyboardButton("« Back to data", callback_data='back_data-' + r)]] 75 | 76 | header_6 = "Which type of domains?" 77 | keyboard_6 = [[InlineKeyboardButton("Active", callback_data='active-' + r), 78 | InlineKeyboardButton("All", callback_data='all-' + r)], 79 | [InlineKeyboardButton("« Back to actions", callback_data='back_get-' + r)]] 80 | 81 | header_7 = "How many domains?" 82 | keyboard_7 = [[InlineKeyboardButton("Top 20", callback_data='limit-' + r), 83 | InlineKeyboardButton("All", callback_data='nolimit-' + r)], 84 | [InlineKeyboardButton("« Back to actions", callback_data='back_data-' + r)]] 85 | 86 | 87 | #ToDO: Transform into a swtich 88 | if choice == "back": 89 | bot.edit_message_text(header_1, reply_markup=InlineKeyboardMarkup(keyboard_1), chat_id=query.message.chat_id, message_id=query.message.message_id) 90 | return BUTTON 91 | elif choice == "close": 92 | bot.edit_message_text("/start", chat_id=query.message.chat_id, message_id=query.message.message_id) 93 | return ConversationHandler.END 94 | elif choice == "scan": 95 | bot.edit_message_text(header_2, reply_markup=InlineKeyboardMarkup(keyboard_2), chat_id=query.message.chat_id, message_id=query.message.message_id) 96 | return BUTTON 97 | elif choice == "data": 98 | bot.edit_message_text(header_3, reply_markup=InlineKeyboardMarkup(keyboard_3), chat_id=query.message.chat_id, message_id=query.message.message_id) 99 | return BUTTON 100 | 101 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 102 | cursor = connection.cursor () 103 | 104 | if choice == "latest": 105 | get_latest_scan(bot, update, cursor) 106 | cursor.close() 107 | connection.close() 108 | elif choice == "custom": 109 | get_custom_scan(bot, update, cursor) 110 | cursor.close() 111 | connection.close() 112 | return CUSTOM_SCAN_ID_INPUT 113 | elif choice == "run": 114 | run_scan(bot, update, cursor) 115 | cursor.close() 116 | connection.close() 117 | 118 | if choice == "add": 119 | cursor.close() 120 | connection.close() 121 | bot.send_message(text="Coming soon, pr's are welcome...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 122 | return BUTTON 123 | #return ADD_DOMAIN 124 | elif choice == "edit": 125 | cursor.close() 126 | connection.close() 127 | bot.send_message(text="Coming soon...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 128 | return BUTTON 129 | #return EDIT_DOMAIN 130 | elif choice == "get": 131 | cursor.close() 132 | connection.close() 133 | bot.edit_message_text(header_5, reply_markup=InlineKeyboardMarkup(keyboard_5), chat_id=query.message.chat_id, message_id=query.message.message_id) 134 | return BUTTON 135 | 136 | if choice == "topdomains": 137 | bot.edit_message_text(header_7, reply_markup=InlineKeyboardMarkup(keyboard_7), chat_id=query.message.chat_id, message_id=query.message.message_id) 138 | global subdomains 139 | subdomains = False 140 | print "choice = topdomains" 141 | return BUTTON 142 | elif choice == "subdomains": 143 | bot.edit_message_text(header_6, reply_markup=InlineKeyboardMarkup(keyboard_6), chat_id=query.message.chat_id, message_id=query.message.message_id) 144 | global subdomains 145 | subdomains = True 146 | return BUTTON 147 | elif choice == "contains": 148 | bot.send_message(text="Coming soon...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 149 | return BUTTON 150 | #return CONTAINS 151 | elif choice == "back_data": 152 | bot.edit_message_text(header_3, reply_markup=InlineKeyboardMarkup(keyboard_3), chat_id=query.message.chat_id, message_id=query.message.message_id) 153 | return BUTTON 154 | elif choice == "back_get": 155 | bot.edit_message_text(header_5, reply_markup=InlineKeyboardMarkup(keyboard_5), chat_id=query.message.chat_id, message_id=query.message.message_id) 156 | return BUTTON 157 | 158 | if choice == "active": 159 | global active 160 | active = True 161 | bot.edit_message_text(header_7, reply_markup=InlineKeyboardMarkup(keyboard_7), chat_id=query.message.chat_id, message_id=query.message.message_id) 162 | return BUTTON 163 | elif choice == "all": 164 | global active 165 | active = False 166 | bot.edit_message_text(header_7, reply_markup=InlineKeyboardMarkup(keyboard_7), chat_id=query.message.chat_id, message_id=query.message.message_id) 167 | return BUTTON 168 | 169 | if choice == "nolimit": 170 | print "Nolimit" 171 | global subdomains 172 | if subdomains: 173 | global limit 174 | limit = False 175 | bot.send_message(text="What is the (top)domain?", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 176 | return GET_DOMAINS 177 | else: 178 | print "topdomain" 179 | global limit 180 | limit = False 181 | get_topdomains(bot, update) 182 | 183 | elif choice == "limit": 184 | global subdomains 185 | print "Limit" 186 | if subdomains: 187 | global limit 188 | limit = True 189 | bot.send_message(text="What is the (top)domain?", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 190 | return GET_DOMAINS 191 | else: 192 | print "Topdomain" 193 | global limit 194 | limit = True 195 | get_topdomains(bot, update) 196 | 197 | if choice == "yes_scan": 198 | bot.send_message(text="Starting a new scan...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 199 | #os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/../run.py") 200 | elif choice == "back_scan" or choice == "no_scan": 201 | bot.edit_message_text(header_2, reply_markup=InlineKeyboardMarkup(keyboard_2), chat_id=query.message.chat_id, message_id=query.message.message_id) 202 | return BUTTON 203 | 204 | 205 | def get_latest_scan(bot, update, cursor): 206 | cursor.execute("select max(ScanID) from scans where EndDate is not null") 207 | data = cursor.fetchall() 208 | if data: 209 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/notify.py " + str(data[0][0]) + " true") 210 | else: 211 | bot.send_message(text="No completed scans found!", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 212 | 213 | 214 | def get_custom_scan(bot, update, cursor): 215 | query = update.callback_query 216 | 217 | cursor.execute ("SELECT ScanID FROM scans where EndDate is not null ORDER BY ScanID DESC LIMIT 10") 218 | data = cursor.fetchall() 219 | if data: 220 | latestScanIds = sorted([str(x[0]) for x in data]) 221 | firstRow = latestScanIds[:len(latestScanIds)/2] 222 | secondRow = latestScanIds[len(latestScanIds)/2:] 223 | custom_keyboard = [firstRow, secondRow] 224 | reply_markup = telegram.ReplyKeyboardMarkup(custom_keyboard) 225 | 226 | if data: 227 | bot.send_message(chat_id=query.message.chat_id, 228 | text="Which scan ID? \n/cancel to cancel", 229 | reply_markup=reply_markup, ForceReply = True) 230 | else: 231 | bot.send_message(chat_id=query.message.chat_id, 232 | text="Which scan ID? \n/cancel to cancel") 233 | 234 | 235 | def run_scan(bot, update, cursor): 236 | query = update.callback_query 237 | cursor.execute ("SELECT * FROM scans ORDER BY ScanID DESC LIMIT 1") 238 | data = cursor.fetchall() 239 | try: 240 | if data[0][2] != None: 241 | bot.send_message(text="Starting a new scan...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 242 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/../run.py") 243 | else: 244 | r = str(randint(0, 99)) 245 | header_4 = "It looks like a scan is already running. Want to start a new one?" 246 | keyboard_4 = [[InlineKeyboardButton("Yes", callback_data='yes_scan-' + r), 247 | InlineKeyboardButton("No", callback_data='no_scan-' + r)], 248 | [InlineKeyboardButton("« Back to scans", callback_data='back_scan-' + r)]] 249 | bot.edit_message_text(header_4, reply_markup=InlineKeyboardMarkup(keyboard_4), chat_id=query.message.chat_id, message_id=query.message.message_id) 250 | return BUTTON 251 | except Exception, e: 252 | print "error: " + str(e) 253 | bot.send_message(text="Starting a new scan...", chat_id=query.message.chat_id, parse_mode=telegram.ParseMode.MARKDOWN) 254 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/../run.py") 255 | 256 | 257 | def custom_scan_id_input(bot, update): 258 | print "inside input method with data: " + update.message.text 259 | customId = update.message.text 260 | try: 261 | int(customId) 262 | except ValueError: 263 | print "invalid number" 264 | update.message.reply_text("Not a valid number") 265 | else: 266 | print "valid number" 267 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 268 | cursor = connection.cursor () 269 | cursor.execute ("SELECT EndDate FROM scans where scanID = %s", (customId)) 270 | data = cursor.fetchall() 271 | cursor.close() 272 | connection.close() 273 | if data: 274 | print "scan ID was found in the db" 275 | print "data[0][0] is: " + str(data[0][0]) 276 | if data[0][0] == None: 277 | print "EndDate of scan is empty" 278 | update.message.reply_text("This scan hasn't finished yet") 279 | else: 280 | print "Valid scan found" 281 | reply_markup = telegram.ReplyKeyboardRemove() 282 | update.message.reply_text("Showing scan from scan " + str(customId), reply_markup=reply_markup) 283 | os.system("python " + os.path.dirname(os.path.abspath(__file__)) + "/notify.py " + str(customId) + " true") 284 | return ConversationHandler.END 285 | else: 286 | print "Scan ID not found in db" 287 | update.message.reply_text("This scan ID doesn't exist") 288 | 289 | 290 | def add_domain(bot, update): 291 | print update.message.text 292 | return BUTTON 293 | 294 | 295 | def edit_domain(bot, update): 296 | print update.message.text 297 | return BUTTON 298 | 299 | 300 | def get_topdomains(bot, update): 301 | global limit 302 | print "Inside get_topdomains" 303 | 304 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 305 | cursor = connection.cursor () 306 | 307 | if limit == True: 308 | cursor.execute ("select Domain from domains where TopDomainID is NULL order by Domain limit 20") 309 | else: 310 | cursor.execute ("select Domain from domains where TopDomainID is NULL order by Domain") 311 | 312 | 313 | data = cursor.fetchall() 314 | cursor.close() 315 | connection.close() 316 | 317 | domains_message = "" 318 | 319 | if not data: 320 | domains_message = "No domains found" 321 | 322 | for row in data: 323 | domains_message += "\n" + row[0] 324 | 325 | bot.send_message(chat_id=credentials.telegram_chat_id, text=domains_message, parse_mode=telegram.ParseMode.MARKDOWN) 326 | keyboard = [[InlineKeyboardButton("Data", callback_data='data-' + str(randint(0, 999))), 327 | InlineKeyboardButton("Scans", callback_data='scan-' + str(randint(0, 999)))], 328 | [InlineKeyboardButton("✘ Close", callback_data='close-' + str(randint(0, 999)))]] 329 | 330 | reply_markup = InlineKeyboardMarkup(keyboard) 331 | bot.send_message(chat_id=credentials.telegram_chat_id, text="Hi again!", reply_markup=reply_markup) 332 | return BUTTON 333 | 334 | 335 | 336 | def get_domains(bot, update): 337 | global active 338 | global limit 339 | print "active: " + str(active) 340 | print "limit: " + str(limit) 341 | print "domain: " + update.message.text 342 | 343 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 344 | cursor = connection.cursor () 345 | 346 | cursor.execute ("select Domain, Active from domains where TopDomainID = (select DomainID from domains where Domain = %s) order by Domain", (update.message.text,)) 347 | 348 | data = cursor.fetchall() 349 | cursor.close() 350 | connection.close() 351 | 352 | subdomains_message = "" 353 | 354 | if not data: 355 | subdomains_message = "No subdomains found for " + str(update.message.text) 356 | 357 | if active == True: 358 | data = [x for x in data if ord(x[1]) == True] 359 | 360 | if limit == True: 361 | print "Limit is True if" 362 | data = data[:20] 363 | 364 | for row in data: 365 | subdomains_message += "\n" + row[0] 366 | 367 | update.message.reply_text(subdomains_message) 368 | 369 | keyboard = [[InlineKeyboardButton("Data", callback_data='data-' + str(randint(0, 999))), 370 | InlineKeyboardButton("Scans", callback_data='scan-' + str(randint(0, 999)))], 371 | [InlineKeyboardButton("✘ Close", callback_data='close-' + str(randint(0, 999)))]] 372 | 373 | reply_markup = InlineKeyboardMarkup(keyboard) 374 | update.message.reply_text("Hi again :wave:", reply_markup=reply_markup) 375 | return BUTTON 376 | 377 | 378 | 379 | 380 | 381 | def domains_contain(bot, update): 382 | print update.message.text 383 | return BUTTON 384 | 385 | 386 | def help(bot, update): 387 | update.message.reply_text("click /start to start :)") 388 | 389 | 390 | def error(bot, update, error): 391 | logger.warning('Update "%s" caused error "%s"', update, error) 392 | 393 | 394 | def cancel(bot, update): 395 | print "canceled" 396 | reply_markup = telegram.ReplyKeyboardRemove() 397 | update.message.reply_text("canceled! Click or type /start to start again", reply_markup=reply_markup) 398 | return ConversationHandler.END 399 | 400 | 401 | def main(): 402 | # Create the EventHandler and pass it your bot's token. 403 | updater = Updater(telegram_bot_token) 404 | 405 | # Get the dispatcher to register handlers 406 | dp = updater.dispatcher 407 | 408 | # Add conversation handler with the states GENDER, PHOTO, LOCATION and BIO 409 | conv_handler = ConversationHandler( 410 | entry_points=[CommandHandler('start', start)], 411 | 412 | states={ 413 | BUTTON: [CallbackQueryHandler(button), 414 | CommandHandler('cancel', cancel)], 415 | CUSTOM_SCAN_ID_INPUT: [MessageHandler(Filters.text, custom_scan_id_input), 416 | CommandHandler('cancel', cancel)], 417 | ADD_DOMAIN: [MessageHandler(Filters.text, add_domain), 418 | CommandHandler('cancel', cancel)], 419 | EDIT_DOMAIN: [MessageHandler(Filters.text, edit_domain), 420 | CommandHandler('cancel', cancel)], 421 | GET_DOMAINS: [MessageHandler(Filters.text, get_domains), 422 | CommandHandler('cancel', cancel)], 423 | CONTAINS: [MessageHandler(Filters.text, domains_contain), 424 | CommandHandler('cancel', cancel)] 425 | 426 | }, 427 | 428 | fallbacks=[CommandHandler('cancel', cancel)] 429 | ) 430 | 431 | dp.add_handler(conv_handler) 432 | 433 | # log all errors 434 | dp.add_error_handler(error) 435 | 436 | # Start the Bot 437 | updater.start_polling() 438 | 439 | # Run the bot until you press Ctrl-C or the process receives SIGINT, 440 | # SIGTERM or SIGABRT. This should be used most of the time, since 441 | # start_polling() is non-blocking and will stop the bot gracefully. 442 | updater.idle() 443 | 444 | 445 | if __name__ == '__main__': 446 | main() 447 | -------------------------------------------------------------------------------- /telegram/notify.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import sys, datetime, MySQLdb, telegram, os 4 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../") 5 | import credentials 6 | 7 | if credentials.telegram_bot_token == "" or credentials.telegram_chat_id == "": 8 | print "[+] No telegram bot token and/or telegram chat id set in credentials.py" 9 | sys.exit() 10 | 11 | bot = telegram.Bot(credentials.telegram_bot_token) 12 | scanId = sys.argv[1] 13 | message = "*" + str(datetime.datetime.now().replace(microsecond=0)) + "*" 14 | 15 | connection = MySQLdb.connect (host = credentials.database_server, user = credentials.database_username, passwd = credentials.database_password, db = credentials.database_name) 16 | cursor = connection.cursor() 17 | 18 | cursor.execute ("select Domain from domains where scan_Id = %s and Active order by TopDomainID", (scanId)) 19 | newSubDomains = cursor.fetchall() 20 | 21 | cursor.execute ("select * from errors where scan_Id = %s order by ErrorDate", (scanId)) 22 | errors = cursor.fetchall() 23 | 24 | connection.close() 25 | 26 | display_all = False 27 | 28 | try: 29 | if sys.argv[2] == "true": 30 | display_all = True 31 | except: 32 | print "Not called from bot" 33 | 34 | message += "\n_Scan " + str(scanId) + "_" 35 | 36 | message += "\n" 37 | 38 | if len(errors) > 1: 39 | message += "\n(" + str(len(errors)) + " Errors)" 40 | elif len(errors) == 1: 41 | message += "\n(" + str(len(errors)) + " Error)" 42 | 43 | if len(errors) > 0: 44 | message += "\n--------------" 45 | 46 | if (len(newSubDomains) < 15 and display_all == False) or (len(newSubDomains) < 100 and display_all == True): 47 | message += "\n\[+] " + str(len(newSubDomains)) + " New subdomains:" 48 | for domain in newSubDomains: 49 | message += "\n" + str(domain[0]) 50 | else: 51 | message += "\n\[+] " + str(len(newSubDomains)) + " New subdomains" 52 | 53 | message += "" 54 | 55 | bot.send_message(chat_id=credentials.telegram_chat_id, text=message, parse_mode=telegram.ParseMode.MARKDOWN) 56 | -------------------------------------------------------------------------------- /telegram/setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | import telegram, sys, os, logging, datetime 4 | from telegram import (InlineKeyboardMarkup, ReplyKeyboardMarkup, ReplyKeyboardRemove, InlineKeyboardButton) 5 | from telegram.ext import Updater, CommandHandler, CallbackQueryHandler, ConversationHandler, MessageHandler, Filters 6 | sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/../") 7 | import credentials 8 | 9 | telegram_bot_token = credentials.telegram_bot_token 10 | 11 | 12 | # Enable logging 13 | logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', 14 | level=logging.INFO) 15 | 16 | logger = logging.getLogger(__name__) 17 | 18 | def start(bot, update): 19 | user = update.message.from_user 20 | chat_id = str(update.message.chat_id) 21 | 22 | with open(os.path.dirname(os.path.abspath(__file__)) + "/../credentials.py") as f: 23 | var_lines = f.readlines() 24 | var_lines = [x.strip() for x in var_lines] 25 | f.close() 26 | 27 | new_lines = [] 28 | 29 | for line in var_lines: 30 | if line.startswith("telegram_chat_id"): 31 | new_lines.append("telegram_chat_id = \"" + str(chat_id) + "\"") 32 | else: 33 | new_lines.append(line) 34 | 35 | creds = open(os.path.dirname(os.path.abspath(__file__)) + "/../credentials.py", 'w') 36 | for item in new_lines: 37 | if item: 38 | creds.write("%s\n" % item) 39 | creds.close() 40 | update.message.reply_text("Welcome " + str(user['first_name']) + "! The telegram setup is successfully finished.") 41 | print "Success! You can now close the setup with ctr + c" 42 | return ConversationHandler.END 43 | 44 | def main(): 45 | updater = Updater(telegram_bot_token) 46 | dp = updater.dispatcher 47 | 48 | print "[!] Send /start to the bot." 49 | 50 | conv_handler = ConversationHandler( 51 | entry_points=[CommandHandler('start', start)], 52 | 53 | states={ 54 | }, 55 | 56 | fallbacks=[CommandHandler('cancel', cancel)] 57 | ) 58 | 59 | dp.add_handler(conv_handler) 60 | updater.start_polling() 61 | updater.idle() 62 | 63 | def cancel(bot, update): 64 | print "canceled" 65 | return ConversationHandler.END 66 | 67 | if __name__ == '__main__': 68 | main() 69 | -------------------------------------------------------------------------------- /tools/online.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import sys, requests 4 | 5 | input_file = sys.argv[1] 6 | output_file = sys.argv[2] 7 | 8 | input_file_open = open(input_file, 'r') 9 | output_file_open = open(output_file, 'w+') 10 | 11 | domains = input_file_open.readlines() 12 | 13 | print("\n-- Writing online hosts in "+input_file+" to "+output_file+" --\n") 14 | 15 | 16 | def available(domain): 17 | try: 18 | r = requests.get(domain, timeout=3) 19 | return True 20 | except: 21 | return False 22 | 23 | for domain in domains: 24 | domain = domain.strip() 25 | 26 | http = available("http://" + domain) 27 | https = available("https://" + domain) 28 | 29 | if http == True or https == True: 30 | print("[+]" + domain.strip()) 31 | output_file_open.write(domain+"\n") 32 | else: 33 | print("[-]" + domain.strip()) 34 | 35 | 36 | input_file_open.close() 37 | output_file_open.close() 38 | print("\n-- Done --") 39 | -------------------------------------------------------------------------------- /web/setup.sh: -------------------------------------------------------------------------------- 1 | echo " 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
11 | connect_error) { 20 | die(\"Connection failed: \" . \$conn->connect_error); 21 | } 22 | 23 | \$sql = \"SELECT * FROM domains ORDER BY Active\"; 24 | 25 | \$result = \$conn->query(\$sql); 26 | 27 | if (\$result->num_rows > 0) { 28 | while(\$row = \$result->fetch_assoc()) { 29 | \$rows[] = \$row; 30 | } 31 | } else { 32 | echo \"0 results\"; 33 | } 34 | \$conn->close(); 35 | 36 | \$domains = array(); 37 | 38 | foreach (\$rows as \$r) { 39 | if(empty(\$r[\"TopDomainID\"])){ 40 | \$subdomains = array(); 41 | foreach (\$rows as \$r2) { 42 | if(\$r2[\"TopDomainID\"] == \$r[\"DomainID\"]){ 43 | array_push(\$subdomains, \$r2); 44 | } 45 | } 46 | array_push(\$domains, array(\$r, \$subdomains)); 47 | } 48 | 49 | } 50 | ?> 51 | 52 | 78 | 79 | 224 | 225 | 226 | 227 |
228 |
229 | 230 | 231 |
232 |
233 | 234 |

Info

235 |
236 |
237 |

body text

238 |
239 |
240 | 241 |
242 |
243 | 244 |
245 |
246 | 247 | 248 | 314 |
315 | 316 | 317 | " > /var/www/html/ICU.php 318 | --------------------------------------------------------------------------------