├── .gitignore ├── README.md ├── htb ├── __init__.py ├── __main__.py ├── connection.py ├── exceptions.py ├── machine.py ├── notification.py ├── scanner │ ├── __init__.py │ ├── enum4linux.py │ ├── gobuster.py │ ├── nikto.py │ └── scanner.py ├── util.py └── vpn.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | env/ 2 | **/__pycache__ 3 | **/*.pyc 4 | **/*.swp 5 | **/*.egg-info/ 6 | dist/ 7 | build/ 8 | .idea/ 9 | 10 | .ghimages/* 11 | .ghimages 12 | 13 | docs/ 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hack the Box Python API 2 | 3 | A Python3 API for interacting with the Hack the Box platform. 4 | 5 | ## Fancy Showcase 6 | 7 | Because a README doesn't do it justice, I recorded an `asciinema` of a small 8 | subset of the functionality. You can see the output of the `machine info` 9 | command and the automatic enumeration command `machine init`. The graphs don't 10 | quite render properly in asciinema, but it should give you an idea of how the 11 | tool works :) 12 | 13 | [Asciinema Showcase](https://asciinema.org/a/hQbKBl3zbAYlNtpWQa2czyrff) 14 | 15 | ## Features 16 | 17 | - Connect to hack the box with `api_token` with an optional connection with 18 | E-Mail/Password 19 | - List machines (active/retired/running/assigned/etc) 20 | - Query VPN status (Labs and Fortress) 21 | - Switch VPN assignment (only for labs VPN) 22 | - Grab OVPN configuration (requires E-mail/password credentials) 23 | - Start and stop machines (w/ VIP this works for all machines, only active 24 | machines are supported for free labs) 25 | - Cancel termination or reset of machines 26 | - Send messages to the Shoutbox (including `/{command}` commands) 27 | - Command line interface 28 | - Automatically build analysis directory structure and start basic 29 | enumeration/scans. 30 | - Two Factor Authentication Support 31 | 32 | ## Example Configuration File 33 | 34 | The `htb` command line application utilizes the `ConfigParser` module in Python 35 | to read a configuration file from `~/.htbrc`. This file contains your 36 | authentication information as well as any configuration items which may be 37 | available. Here's an example of the configuration file: 38 | 39 | ```ini 40 | [htb] 41 | api_token = your_api_token 42 | email = your_email 43 | password = your_password 44 | session = session_token 45 | analysis_path = ~/htb 46 | 47 | [lab] 48 | connection = NetworkManager-Connection-UUID 49 | ``` 50 | 51 | The `connection` and `session` options are filled automatically on running to 52 | track sessions between running `htb` and the connection which `htb lab` is able 53 | to create with Network Manager. 54 | 55 | This configuration is also passed to all scanners, allowing scanner specific 56 | options to be specified. At this time, only one scanner utilizes the 57 | configuraiton: `gobuster`. You can specify the worldist path under the 58 | `gobuster` section. The default wordlist is the `dirbuster` small wordlist in 59 | the Kali default wordlists directory. As an example, you can specify an 60 | alternate like: 61 | 62 | ```ini 63 | [gobuster] 64 | wordlist = /usr/share/dirbuster/directory-list-lowercase-2.3-medium.txt 65 | ``` 66 | 67 | ## Example Command Line Usage 68 | 69 | The Command Line Interface provides two methods for invocation. The first 70 | simply runs a single command and exits. This is the type of invocation you 71 | can expect from a shellscript. By default, the configuration information 72 | is read from a file located at `$HOME/.htbrc`, but can also be specified 73 | with the environment variable `HTBRC`. 74 | 75 | To get a list of valid commands, you can use the `help` command: 76 | 77 | ``` 78 | python-htb on  master [!] via python-htb took 2s 79 | ➜ python -m htb help -v 80 | 81 | Documented commands (use 'help -v' for verbose/'help ' for details): 82 | 83 | Hack the Box 84 | ================================================================================ 85 | invalidate Invalidate API cache 86 | lab View and manage lab VPN connection 87 | machine View and manage active and retired machines 88 | 89 | Uncategorized 90 | ================================================================================ 91 | alias Manage aliases 92 | edit Run a text editor and optionally open a file with it 93 | help List available commands or provide detailed help for a specific command 94 | history View, run, edit, save, or clear previously entered commands 95 | macro Manage macros 96 | py Invoke Python command or shell 97 | quit Exit this application 98 | run_pyscript Run a Python script file inside the console 99 | run_script Run commands in script file that is encoded as either ASCII or UTF-8 text 100 | set Set a settable parameter or show current settings of parameters 101 | shell Execute a command as if at the OS prompt 102 | shortcuts List available shortcuts 103 | ``` 104 | 105 | To run a command, simply append it to the command line when invoking the module. 106 | This is the first method of invocation: 107 | 108 | ![List Active Machines](https://user-images.githubusercontent.com/7529189/76907462-a7487b00-687c-11ea-852f-87d566fcefd4.png) 109 | 110 | Next, you can enter an interactive Hack the Box interpreter by 111 | ommitting the command: 112 | 113 | ![Show Currently Assigned Machine Details](https://user-images.githubusercontent.com/7529189/76907463-a7487b00-687c-11ea-81cf-7c1efd3e0817.png) 114 | 115 | ## Available Commands 116 | 117 | ### `machine list` 118 | 119 | List available machines on the Hack the Box platform. Results are paged if too 120 | numerous to fit on screen and not redirected. Currently assigned machine is 121 | highlighted by an asterics following the machine ID. 122 | 123 | ``` 124 | htb ➜ machine list --help 125 | Usage: machine list [-h] [--inactive] [--active] [--owned] [--unowned] [--todo] 126 | 127 | optional arguments: 128 | -h, --help show this help message and exit 129 | --inactive, -i 130 | --active, -a 131 | --owned, -o 132 | --unowned, -u 133 | --todo, -t 134 | ``` 135 | 136 | ### `machine info` 137 | 138 | Display detailed machine information. This includes difficulty graph and rating 139 | matrix (both user and maker). 140 | 141 | ``` 142 | htb ➜ machine info --help 143 | Usage: machine info [-h] (--assigned | machine) 144 | 145 | positional arguments: 146 | machine A name regex, IP address or machine ID 147 | 148 | optional arguments: 149 | -h, --help show this help message and exit 150 | --assigned, -a Perform action on the currently assigned machine 151 | ``` 152 | 153 | ### `machine up` 154 | 155 | Start a machine instance in your current lab. This command is only valid for 156 | VIP users, and will fail if another machine is already assigned to your 157 | account. 158 | 159 | ``` 160 | htb ➜ machine up --help 161 | Usage: machine up [-h] machine 162 | 163 | positional arguments: 164 | machine A name regex, IP address or machine ID to start 165 | 166 | optional arguments: 167 | -h, --help show this help message and exit 168 | ``` 169 | 170 | ### `machine reset` 171 | 172 | Issue a reset for the given machine. Resets happen after two minutes and can be 173 | cancelled by other users in your lab. Check the `info` or `list` output for this 174 | machine periodically after issuing to see if another user cancelled your reset. 175 | 176 | ``` 177 | htb ➜ machine reset --help 178 | Usage: machine reset [-h] machine 179 | 180 | positional arguments: 181 | machine A name regex, IP address or machine ID 182 | 183 | optional arguments: 184 | -h, --help show this help message and exit 185 | ``` 186 | 187 | ### `machine own` 188 | 189 | Submit a user or root flag for a given machine. If no rating is specified, a rating 190 | of `0` is submitted (same as default on website). 191 | 192 | ``` 193 | htb ➜ machine own --help 194 | Usage: machine own [-h] 195 | [--rate {1-100}] 196 | [--assigned] 197 | [machine] flag 198 | 199 | positional arguments: 200 | machine A name regex, IP address or machine ID 201 | flag The user or root flag 202 | 203 | optional arguments: 204 | -h, --help show this help message and exit 205 | --rate, -r {1-100} 206 | Difficulty Rating (1-100) 207 | --assigned, -a Perform action on the currently assigned machine 208 | ``` 209 | 210 | ### `machine enum` 211 | 212 | Perform initial enumeration for the given machine. This will perform an 213 | all-ports scan with `masscan`, and use the results to do an in-depth scan with 214 | `nmap`. The results are saved under the `scans` directory for this machine. 215 | Also, individual service results are parsed and saved in the `machine.json` file 216 | at the root of the analysis directory. Future invocations of `htb` will be able 217 | to read this and skip the initial enumeration phase. 218 | 219 | ``` 220 | htb ➜ machine enum --help 221 | Usage: htb enum [-h] (--assigned | machine) 222 | 223 | positional arguments: 224 | machine A name regex, IP address or machine ID to start 225 | 226 | optional arguments: 227 | -h, --help show this help message and exit 228 | --assigned, -a Perform action on the currently assigned machine 229 | ``` 230 | 231 | ### `machine scan` 232 | 233 | Perform basic scans which are applicable to enumerated services running on the 234 | machine. You must complete the `enum` command first, or no matching services 235 | will be located (because `htb` doesn't know what services are available). If a 236 | scan is started in the foreground, you can background the scan with `C-z`. 237 | Background jobs can be managed with the `jobs` command. 238 | 239 | ``` 240 | htb ➜ machine scan --help 241 | Usage: machine scan [-h] [--service SERVICE] [--scanner SCANNER] [--recommended RECOMMENDED] [--background] 242 | [--assigned] 243 | [machine] 244 | 245 | positional arguments: 246 | machine A name regex, IP address or machine ID to start 247 | 248 | optional arguments: 249 | -h, --help show this help message and exit 250 | --service, -v SERVICE 251 | Only run scans for this service (format: `{PORT}/{PROTOCOL}`) 252 | --scanner, -s SCANNER 253 | Only run scans for this scanner 254 | --recommended, -r RECOMMENDED 255 | Run all recommended scans 256 | --background, -b Run scans in the background 257 | --assigned, -a Perform action on the currently assigned machine 258 | ``` 259 | 260 | ### `jobs list` 261 | 262 | List all background jobs. This includes completed and running jobs, and will 263 | output the status if any is available from the individual scanner. 264 | 265 | ``` 266 | htb ➜ jobs list --help 267 | Usage: jobs list [-h] 268 | 269 | List background scanner jobs and their status 270 | 271 | optional arguments: 272 | -h, --help show this help message and exit 273 | ``` 274 | 275 | ### `jobs kill` 276 | 277 | Kill the specified job ID. The job ID can be retrieved from the `jobs list` 278 | command. 279 | 280 | ``` 281 | htb ➜ jobs kill --help 282 | Usage: jobs kill [-h] job_id 283 | 284 | Stop a running background scanner job 285 | 286 | positional arguments: 287 | job_id Kill the identified job 288 | 289 | optional arguments: 290 | -h, --help show this help message and exit 291 | ``` 292 | 293 | ### `lab status` 294 | 295 | Display the current status of the lab VPN connection. 296 | 297 | ### `lab switch` 298 | 299 | Change VPN servers. 300 | 301 | ``` 302 | htb ➜ lab switch --help 303 | Usage: lab switch [-h] {usfree, usvip, eufree, euvip, aufree} 304 | 305 | Show the connection status of the currently assigned lab VPN 306 | 307 | positional arguments: 308 | {usfree, usvip, eufree, euvip, aufree} 309 | The lab to switch to 310 | 311 | optional arguments: 312 | -h, --help show this help message and exit 313 | ``` 314 | 315 | ### `lab config` 316 | 317 | This command retrieves and outputs the contents of your OVPN configuration 318 | file. An E-mail and password must be set in your configuration file for 319 | this call to work (`api_token` alone is **not** enough). 320 | 321 | ### `lab import` 322 | 323 | This command will retrieve you lab configuration and import it into 324 | NetworkManager. Obviously, you need to use Network Manager to manager your 325 | network cards for this to work properly. You also need the Network Manager 326 | OpenVPN plugin. The connection is managed by `htb` and the UUID is saved in your 327 | configuration file. 328 | 329 | ``` 330 | htb ➜ lab import --help 331 | Usage: lab import [-h] [--reload] [--name NAME] 332 | 333 | Import your OpenVPN configuration into Network Manager 334 | 335 | optional arguments: 336 | -h, --help show this help message and exit 337 | --reload, -r Reload configuration from Hack the Box 338 | --name, -n NAME NetworkManager Connection ID 339 | ``` 340 | 341 | ### `lab connect` 342 | 343 | This command will attempt to connect with Network Manager to the Hack the Box 344 | VPN. If the connection has not been imported, it will automoatically import the 345 | configuration. It looks for the connection specified by UUID in your 346 | configuration file. 347 | 348 | ``` 349 | htb ➜ lab connect --help 350 | Usage: lab connect [-h] [--update] 351 | 352 | Connect to the Hack the Box VPN. If no previous configuration has been created in NetworkManager, it attempts to download it and import it. 353 | 354 | optional arguments: 355 | -h, --help show this help message and exit 356 | --update, -u Force a redownload/import of the OpenVPN configuration 357 | ``` 358 | 359 | ### `lab disconnect` 360 | 361 | Disconnect the Network Manager connection referring to the Hack the Box 362 | connection (specified in your configuration file). 363 | 364 | ``` 365 | htb ➜ lab connect --help 366 | Usage: lab connect [-h] [--update] 367 | 368 | Connect to the Hack the Box VPN. If no previous configuration has been created in NetworkManager, it attempts to download it and import it. 369 | 370 | optional arguments: 371 | -h, --help show this help message and exit 372 | --update, -u Force a redownload/import of the OpenVPN configuration 373 | ``` 374 | 375 | ### `invalidate` 376 | 377 | The connection object maintains an API response cache by default for up to 378 | one minute. This command will flush/invalidate the cache in order to force 379 | a refresh of the data in the connection object. If you notice stale 380 | information or require the most up to date machine status, then use this 381 | command. It is not useful from the CLI interface. It only has relevance 382 | from a long-running REPL context. 383 | 384 | ## Example Module Usage 385 | 386 | ```python 387 | import htb 388 | 389 | # Connect to hack the box 390 | cnxn = htb.Connection( 391 | api_token="YOUR_API_TOKEN" 392 | email="YOUR_EMAIL", 393 | password="YOUR_PASSWORD", 394 | ) 395 | 396 | # Switch to the US VIP lab 397 | cnxn.lab.switch(htb.VPN.US_VIP) 398 | 399 | # Save your OVPN configuration (requires email/password) 400 | with open("htb.ovpn", "wb") as f: 401 | f.write(cnxn.lab.config) 402 | 403 | # Grab the mango box by name and start it 404 | cnxn["mango"].spawned = True 405 | 406 | # Cancel a reset on Bastion (ip 10.10.10.137) 407 | cnxn["10.10.10.137"].resetting = False 408 | 409 | # Schedule termination on Registry (id 213) 410 | cnxn[213].terminating = True 411 | 412 | # Cancel all machine resets (probably shouldn't do this...) 413 | for m in filter(lambda m: m.resetting, cnxn.machines): 414 | m.resetting = False 415 | ``` 416 | -------------------------------------------------------------------------------- /htb/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from htb.connection import Connection 4 | from htb.machine import Machine 5 | from htb.vpn import VPN 6 | -------------------------------------------------------------------------------- /htb/__main__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Any, List, Dict, Union 3 | import cmd2 4 | from cmd2 import Cmd 5 | from cmd2.argparse_custom import Cmd2ArgumentParser 6 | import configparser 7 | import NetworkManager 8 | import subprocess 9 | from colorama import Fore, Style, Back 10 | import functools 11 | import queue 12 | import argparse 13 | import os.path 14 | import tempfile 15 | import signal 16 | import shlex 17 | import time 18 | import dbus 19 | import sys 20 | import os 21 | 22 | from htb import util 23 | from htb import Connection, Machine, VPN 24 | from htb.exceptions import * 25 | from htb.scanner.scanner import Tracker, Scanner, Service 26 | from htb.scanner import AVAILABLE_SCANNERS 27 | import htb.scanner 28 | 29 | 30 | class HackTheBox(Cmd): 31 | """ Hack the Box Command Line Interface """ 32 | 33 | OS_ICONS = { 34 | "freebsd": "\uf3a4", 35 | "windows": "\uf17a", 36 | "linux": "\uf17c", 37 | "android": "\uf17b", 38 | "solaris": "\uf185", 39 | "other": "\uf233", 40 | } 41 | 42 | _singleton = None 43 | ASSIGNED = "\x00assigned\x00" 44 | 45 | max_completion_items = 50 46 | case_insensitive = True 47 | 48 | def __init__(self, resource="~/.htbrc", *args, **kwargs): 49 | super(HackTheBox, self).__init__(*args, **kwargs) 50 | 51 | # Find file location referencing "~" 52 | path_resource = os.path.expanduser(resource) 53 | if not os.path.isfile(path_resource): 54 | raise RuntimeError(f"{resource}: no such file or directory") 55 | 56 | # Read configuration 57 | parser = configparser.ConfigParser(interpolation=None) 58 | parser.read(path_resource) 59 | 60 | # Save the configuration for later 61 | self.config_path: str = path_resource 62 | self.config = parser 63 | 64 | # Extract relevant information 65 | email = parser["htb"].get("email", None) 66 | password = parser["htb"].get("password", None) 67 | api_token = parser["htb"].get("api_token", None) 68 | session = parser["htb"].get("session", None) 69 | 70 | # if session is not None: 71 | # self.pwarning("attempting to use existing session") 72 | 73 | # Ensure we have an API token 74 | if api_token is None: 75 | raise RuntimeError("no api token provided!") 76 | 77 | # Construct the connection object 78 | self.cnxn: Connection = Connection( 79 | api_token=api_token, 80 | email=email, 81 | password=password, 82 | existing_session=session, 83 | analysis_path=self.config["htb"].get("analysis_path", "~/htb"), 84 | twofactor_prompt=self.twofactor_prompt, 85 | subscribe=True, 86 | config=self.config, 87 | ) 88 | 89 | self.prompt = ( 90 | f"{Fore.CYAN}htb{Fore.RESET} {Style.BRIGHT+Fore.GREEN}➜{Style.RESET_ALL} " 91 | ) 92 | 93 | # List of job trackers 94 | self.jobs: List[Tracker] = [] 95 | self.job_events: queue.Queue = queue.Queue() 96 | 97 | # Enable self in python 98 | self.self_in_py = True 99 | 100 | # Aliases 101 | self.aliases["exit"] = "quit" 102 | 103 | self.cnxn.subscribe("repl", self._on_notification) 104 | 105 | @classmethod 106 | def get(cls, *args, **kwargs) -> "HackTheBox": 107 | """ Get the singleton object for the HackTheBox REPL 108 | 109 | :param args: Positional arguments passed directly to __init__ 110 | :param kwargs: Keyword arguments passed directly to __init__ 111 | :return: The singleton HackTheBox instance 112 | """ 113 | 114 | if cls._singleton is None: 115 | cls._singleton = HackTheBox(*args, **kwargs) 116 | 117 | return cls._singleton 118 | 119 | def _on_notification(self, message): 120 | """ Display notification information from a asynchronous update """ 121 | 122 | server = "-".join(self.cnxn.lab.hostname.split(".")[0].split("-")[1:]) 123 | 124 | # Ignore messages for other servers 125 | if message["server"] != server: 126 | return True 127 | 128 | with self.terminal_lock: 129 | self.async_alert(message["title"].lower().split("]")[1]) 130 | 131 | return True 132 | 133 | def twofactor_prompt(self) -> str: 134 | self.pwarning("One Time Password: ", end="") 135 | sys.stderr.flush() 136 | return self.read_input("") 137 | 138 | def poutput(self, msg: Any = "", end: str = "\n", apply_style: bool = True) -> None: 139 | if apply_style: 140 | msg = f"[{Style.BRIGHT+Fore.BLUE}-{Style.RESET_ALL}] {msg}" 141 | super(HackTheBox, self).poutput(msg, end=end) 142 | 143 | def psuccess( 144 | self, msg: Any = "", end: str = "\n", apply_style: bool = True 145 | ) -> None: 146 | if apply_style: 147 | msg = f"[{Style.BRIGHT+Fore.GREEN}+{Style.RESET_ALL}] {msg}" 148 | super(HackTheBox, self).poutput(msg, end=end) 149 | 150 | def pwarning( 151 | self, msg: Any = "", end: str = "\n", apply_style: bool = True 152 | ) -> None: 153 | if apply_style: 154 | msg = f"[{Style.BRIGHT+Fore.YELLOW}?{Style.RESET_ALL}] {msg}" 155 | super(HackTheBox, self).pwarning( 156 | msg, end=end, apply_style=False, 157 | ) 158 | 159 | def async_alert(self, msg: str, new_prompt: str = None): 160 | """ Asynchronous alert, and redraw prompt """ 161 | msg = f"[{Style.BRIGHT+Fore.YELLOW}!{Style.RESET_ALL}] {msg}" 162 | super(HackTheBox, self).async_alert(msg, new_prompt) 163 | 164 | def perror(self, msg: Any = "", end: str = "\n", apply_style: bool = True) -> None: 165 | if apply_style: 166 | msg = f"[{Style.BRIGHT+Fore.RED}!{Style.RESET_ALL}] {msg}" 167 | super(HackTheBox, self).perror( 168 | msg, end=end, apply_style=False, 169 | ) 170 | 171 | jobs_parser = Cmd2ArgumentParser(description="Manage background scanner jobs") 172 | 173 | @cmd2.with_argparser(jobs_parser) 174 | @cmd2.with_category("Management") 175 | def do_jobs(self, args: argparse.Namespace) -> bool: 176 | """ Manage running background scanner jobs """ 177 | 178 | actions = {"list": self._jobs_list, "kill": self._jobs_kill} 179 | actions[args.action](args) 180 | return False 181 | 182 | def _jobs_list(self, args: argparse.Namespace) -> None: 183 | """ List the background scanner jobs """ 184 | 185 | # Grab any pending events 186 | try: 187 | while True: 188 | t = self.job_events.get_nowait() 189 | t.thread.join() 190 | t.thread = None 191 | # t.status = "completed" 192 | except queue.Empty: 193 | pass 194 | 195 | table = [["", "Host", "Service", "Scanner", "Status"]] 196 | for ident, job in enumerate(self.jobs): 197 | style = Style.DIM if job.thread is None else "" 198 | table.append( 199 | [ 200 | ">" + style + str(ident), 201 | job.machine.name, 202 | f"{job.service.port}/{job.service.protocol} ({job.service.name})", 203 | job.scanner.name, 204 | job.status, 205 | ] 206 | ) 207 | 208 | self.ppaged("\n".join(util.build_table(table))) 209 | 210 | def _jobs_kill(self, args: argparse.Namespace) -> None: 211 | """ Stop a running background scanner job """ 212 | 213 | # Ensure the job exists 214 | if args.job_id < 0 or args.job_id >= len(self.jobs): 215 | self.perror(f"{args.job_id}: no such job") 216 | return 217 | 218 | job = self.jobs[args.job_id] 219 | if job.thread is None: 220 | self.pwarning(f"{args.job_id}: already completed") 221 | return 222 | 223 | # Inform it should stop 224 | self.poutput(f"killing job {args.job_id}") 225 | job.stop = True 226 | 227 | # Argument parser for `machine` command 228 | machine_parser = Cmd2ArgumentParser( 229 | description="View and manage active and retired machines" 230 | ) 231 | 232 | @cmd2.with_argparser(machine_parser) 233 | @cmd2.with_category("Hack the Box") 234 | def do_machine(self, args: argparse.Namespace) -> bool: 235 | """ View and manage active and retired machines """ 236 | actions = { 237 | "list": self._machine_list, 238 | "start": self._machine_start, 239 | "stop": self._machine_stop, 240 | "own": self._machine_own, 241 | "info": self._machine_info, 242 | "cancel": self._machine_cancel, 243 | "reset": self._machine_reset, 244 | "scan": self._machine_scan, 245 | "enum": self._machine_enum, 246 | } 247 | actions[args.action](args) 248 | return False 249 | 250 | def _machine_list(self, args: argparse.Namespace) -> None: 251 | """ List machines on hack the box """ 252 | 253 | # Grab all machines 254 | machines = self.cnxn.machines 255 | 256 | if args.state != "all": 257 | machines = [m for m in machines if m.retired == (args.state != "active")] 258 | if args.owned != "all": 259 | machines = [ 260 | m 261 | for m in machines 262 | if (m.owned_root and m.owned_user) == (args.owned == "owned") 263 | ] 264 | if args.todo: 265 | machines = [m for m in machines if m.todo] 266 | 267 | # Pre-calculate column widths to output correctly formatted header 268 | name_width = max([len(m.name) for m in machines]) + 2 269 | ip_width = max([len(m.ip) for m in machines]) + 2 270 | id_width = max([len(str(m.id)) for m in machines]) + 1 271 | diff_width = 12 272 | rating_width = 5 273 | owned_width = 7 274 | state_width = 13 275 | 276 | # Lookup tables for creating the difficulty ratings 277 | rating_char = [ 278 | "\u2581", 279 | "\u2582", 280 | "\u2583", 281 | "\u2584", 282 | "\u2585", 283 | "\u2586", 284 | "\u2587", 285 | "\u2588", 286 | ] 287 | rating_color = [*([Fore.GREEN] * 3), *([Fore.YELLOW] * 4), *([Fore.RED] * 3)] 288 | 289 | # Build initial table with headers 290 | table = [ 291 | ["", "", "OS", "Name", "Address", "Difficulty", "Rate", "Owned", "State"] 292 | ] 293 | 294 | # Create the individual machine rows 295 | for m in machines: 296 | style = Style.DIM if m.owned_user and m.owned_root else "" 297 | 298 | # Grab OS Icon 299 | try: 300 | os_icon = HackTheBox.OS_ICONS[m.os.lower()] 301 | except KeyError: 302 | os_icon = HackTheBox.OS_ICONS["other"] 303 | 304 | # Create scaled difficulty rating. Highest rated is full. Everything 305 | # else is scaled appropriately. 306 | max_ratings = max(m.ratings) 307 | if max_ratings == 0: 308 | ratings = m.ratings 309 | else: 310 | ratings = [float(r) / max_ratings for r in m.ratings] 311 | difficulty = "" 312 | for i, r in enumerate(ratings): 313 | difficulty += rating_color[i] + rating_char[round(r * 6)] 314 | difficulty += Style.RESET_ALL + style 315 | 316 | # "$" for user and "#" for root 317 | owned = f"^{'$' if m.owned_user else ' '} {'#' if m.owned_root else ' '}" 318 | 319 | # Display time left/terminating/resetting/off etc 320 | if m.spawned and not m.terminating and not m.resetting: 321 | state = m.expires 322 | elif m.terminating: 323 | state = "terminating" 324 | elif m.resetting: 325 | state = "resetting" 326 | else: 327 | state = "off" 328 | 329 | # Show an astrics and highlight state in blue for assigned machine 330 | if m.assigned: 331 | state = Fore.BLUE + state + Fore.RESET 332 | assigned = f"{Fore.BLUE}*{Style.RESET_ALL} " 333 | else: 334 | assigned = " " 335 | 336 | table.append( 337 | [ 338 | f"{style}{m.id}", 339 | assigned, 340 | f"{os_icon} {m.os}", 341 | m.name, 342 | m.ip, 343 | difficulty, 344 | f"{m.rating:.1f}", 345 | owned, 346 | state, 347 | ] 348 | ) 349 | 350 | # print data 351 | self.ppaged("\n".join(util.build_table(table))) 352 | 353 | def _machine_start(self, args: argparse.Namespace): 354 | """ Start a machine """ 355 | 356 | m = args.machine 357 | a = self.cnxn.assigned 358 | 359 | if m.spawned and self.cnxn.assigned is None: 360 | self.poutput( 361 | f"{m.name}: machine already running, transferring ownership..." 362 | ) 363 | m.assigned = True 364 | return 365 | elif m.spawned and self.cnxn.assigned is not None: 366 | self.poutput( 367 | f"unable to transfer machine. {self.cnxn.assigned.name} is currently assigned." 368 | ) 369 | return 370 | 371 | self.psuccess(f"starting {m.name}") 372 | try: 373 | m.spawned = True 374 | except RequestFailed as e: 375 | self.perror(f"request failed: {e}") 376 | 377 | def _machine_reset(self, args: argparse.Namespace) -> None: 378 | """ Stop an active machine """ 379 | 380 | m = args.machine 381 | 382 | if not m.spawned: 383 | self.poutput(f"{m.name}: not running") 384 | return 385 | 386 | self.psuccess(f"{m.name}: scheduling reset") 387 | m.resetting = True 388 | 389 | def _machine_stop(self, args: argparse.Namespace) -> None: 390 | """ Stop an active machine """ 391 | 392 | if not args.machine.spawned: 393 | self.poutput(f"{args.machine.name} is not running") 394 | return 395 | 396 | self.psuccess(f"scheduling termination for {args.machine.name}") 397 | args.machine.spawned = False 398 | 399 | def _machine_info(self, args: argparse.Namespace) -> None: 400 | """ Show detailed machine information 401 | 402 | NOTE: This function is gross. I'm not sure of a cleaner way to build 403 | the pretty graphs and tables than manually like this. I need to 404 | research some other python modules that may be able to help 405 | """ 406 | 407 | # Shorthand for args.machine 408 | m = args.machine 409 | 410 | if m.spawned and not m.resetting and not m.terminating: 411 | state = f"{Fore.GREEN}up{Fore.RESET} for {m.expires}" 412 | elif m.terminating: 413 | state = f"{Fore.RED}terminating{Fore.RESET}" 414 | elif m.resetting: 415 | state = f"{Fore.YELLOW}resetting{Fore.RESET}" 416 | else: 417 | state = f"{Fore.RED}off{Fore.RESET}" 418 | 419 | if m.retired: 420 | retiree = f"{Fore.YELLOW}retired{Fore.YELLOW}" 421 | else: 422 | retiree = f"{Fore.GREEN}active{Fore.RESET}" 423 | 424 | try: 425 | os_icon = HackTheBox.OS_ICONS[m.os.lower()] 426 | except KeyError: 427 | os_icon = HackTheBox.OS_ICONS["other"] 428 | 429 | output = [] 430 | output.append( 431 | f"{Style.BRIGHT}{Fore.GREEN}{m.name}{Fore.RESET} - {Style.RESET_ALL}{m.ip}{Style.BRIGHT} - {Style.RESET_ALL}{os_icon}{Style.BRIGHT} {Fore.CYAN}{m.os}{Fore.RESET} - {Fore.MAGENTA}{m.points}{Fore.RESET} points - {state}" 432 | ) 433 | 434 | output.append("") 435 | output.append(f"{Style.BRIGHT}Difficulty{Style.RESET_ALL}") 436 | output.extend(["", "", "", "", ""]) 437 | 438 | # Lookup tables for creating the difficulty ratings 439 | rating_char = [ 440 | "\u2581", 441 | "\u2582", 442 | "\u2583", 443 | "\u2584", 444 | "\u2585", 445 | "\u2586", 446 | "\u2587", 447 | "\u2588", 448 | ] 449 | rating_color = [*([Fore.GREEN] * 3), *([Fore.YELLOW] * 4), *([Fore.RED] * 3)] 450 | 451 | # Create scaled difficulty rating. Highest rated is full. Everything 452 | # else is scaled appropriately. 453 | max_ratings = max(m.ratings) 454 | ratings = [round((float(r) / max_ratings) * 40) for r in m.ratings] 455 | difficulty = "" 456 | for i, r in enumerate(ratings): 457 | for row in range(1, 6): 458 | if r > (5 - row) * 8 and r <= (5 - row + 1) * 8: 459 | output[-6 + row] += rating_color[i] + rating_char[r % 8] * 3 460 | elif r > (5 - row) * 8: 461 | output[-6 + row] += rating_color[i] + rating_char[7] * 3 462 | else: 463 | output[-6 + row] += " " 464 | 465 | output.append( 466 | f"{Fore.GREEN}Easy {Fore.YELLOW} Medium {Fore.RED} Hard{Style.RESET_ALL}" 467 | ) 468 | output.append("") 469 | output.append( 470 | f"{Style.BRIGHT}Rating Matrix ({Fore.CYAN}maker{Fore.RESET}, {Style.DIM}{Fore.GREEN}user{Style.RESET_ALL}{Style.BRIGHT}){Style.RESET_ALL}" 471 | ) 472 | output.extend(["", "", "", "", ""]) 473 | column_widths = [6, 8, 6, 8, 6] 474 | 475 | for i in range(5): 476 | for row in range(5): 477 | content = f"{'MMAA':^{column_widths[i]}}" 478 | if (m.matrix["maker"][i] * 4) >= ((row + 1) * 8): 479 | content = content.replace("MM", f"{Fore.CYAN}{rating_char[7]*2}") 480 | elif (m.matrix["maker"][i] * 4) > (row * 8): 481 | content = content.replace( 482 | "MM", 483 | f"{Fore.CYAN}{rating_char[round(m.matrix['maker'][i]*4) % 8]*2}", 484 | ) 485 | else: 486 | content = content.replace("MM", " " * 2) 487 | if (m.matrix["aggregate"][i] * 4) >= ((row + 1) * 8): 488 | content = content.replace( 489 | "AA", 490 | f"{Style.DIM}{Fore.GREEN}{rating_char[7]*2}{Style.RESET_ALL}", 491 | ) 492 | elif (m.matrix["aggregate"][i] * 4) > (row * 8): 493 | content = content.replace( 494 | "AA", 495 | f"{Style.DIM}{Fore.GREEN}{rating_char[round(m.matrix['maker'][i]*4) % 8]*2}{Style.RESET_ALL}", 496 | ) 497 | else: 498 | content = content.replace("AA", " " * 2) 499 | output[-1 - row] += content 500 | 501 | output.append( 502 | f"{'Enum':^{column_widths[0]}}{'R-Life':^{column_widths[1]}}{'CVE':^{column_widths[2]}}{'Custom':^{column_widths[3]}}{'CTF':^{column_widths[4]}}" 503 | ) 504 | 505 | output.append("") 506 | 507 | user_width = max([6, len(m.blood["user"]["name"]) + 2]) 508 | 509 | output.append( 510 | f"{Style.BRIGHT} {'User':<{user_width}}Root{Style.RESET_ALL}" 511 | ) 512 | output.append( 513 | f"{Style.BRIGHT}Owns {Style.RESET_ALL}{Fore.YELLOW}{m.user_owns:<{user_width}}{Fore.RED}{m.root_owns}{Style.RESET_ALL}" 514 | ) 515 | output.append( 516 | f"{Style.BRIGHT}{'Blood':<6}{Style.RESET_ALL}{m.blood['user']['name']:<{user_width}}{m.blood['root']['name']}" 517 | ) 518 | 519 | if len(m.services): 520 | output.append("") 521 | 522 | table = [["Port", "Protocol", "Name", "Version"]] 523 | for service in m.services: 524 | table.append( 525 | [f"{service.port}", f"{service.protocol}", service.name, ""] 526 | ) 527 | 528 | output.extend(util.build_table(table)) 529 | else: 530 | output.append("") 531 | output.append(f"{Style.BRIGHT}No enumerated services.{Style.RESET_ALL}") 532 | 533 | self.ppaged("\n".join(output)) 534 | 535 | def _machine_own(self, args: argparse.Namespace) -> None: 536 | """ Submit a machine own (user or root) """ 537 | 538 | if args.machine.submit(args.flag, difficulty=args.rate): 539 | self.psuccess(f"correct flag for {args.machine.Name}!") 540 | else: 541 | self.perror(f"incorrect flag") 542 | 543 | def _machine_cancel(self, args: argparse.Namespace) -> None: 544 | """ Cancel pending termination or reset """ 545 | 546 | if len(args.cancel) == 0 or "t" in args.cancel: 547 | if args.machine.terminating: 548 | args.machine.terminating = False 549 | self.psuccess(f"{args.machine.name}: pending termination cancelled") 550 | if len(args.cancel) == 0 or "r" in args.cancel: 551 | if args.machine.resetting: 552 | args.machine.resetting = False 553 | self.psuccess(f"{args.machine.name}: pending reset cancelled") 554 | 555 | def wait_for_machine(self, machine: Machine) -> bool: 556 | 557 | # Start the machine if we don't have a machine assigned 558 | self.pwarning(f"starting {machine.name}") 559 | machine.spawned = True 560 | 561 | # Ensure the device is up before we scan 562 | self.pwarning(f"waiting for machine to respond to ping...") 563 | try: 564 | # Wait for a positive ping response 565 | while True: 566 | if ( 567 | subprocess.call( 568 | ["ping", "-c", "1", machine.ip], 569 | stdout=subprocess.DEVNULL, 570 | stderr=subprocess.DEVNULL, 571 | ) 572 | == 0 573 | ): 574 | self.psuccess(f"received ping response!") 575 | break 576 | time.sleep(5) 577 | except KeyboardInterrupt: 578 | self.perror("no ping response received. cancelling enumeration.") 579 | return False 580 | else: 581 | # Give the machine some time to start services after networking comes up 582 | self.poutput(f"waiting for services to start...") 583 | time.sleep(10) 584 | 585 | # Ensure we grab the newest machine status 586 | self.cnxn.invalidate_cache() 587 | 588 | return True 589 | 590 | def _machine_enum(self, args: argparse.Namespace) -> None: 591 | """ Perform initial service enumeration """ 592 | 593 | if not args.machine.spawned: 594 | if self.cnxn.assigned is not None: 595 | # We can't assign this machine, if we already have a machine assigned. 596 | self.pwarning( 597 | f"unable to start {args.machine.name}. {self.cnxn.assigned.name} is already assigned." 598 | ) 599 | return 600 | elif not self.wait_for_machine(args.machine): 601 | return 602 | 603 | if args.machine.analysis_path is None: 604 | self.pwarning("initializing analysis structure") 605 | 606 | try: 607 | args.machine.init(self.cnxn.analysis_path) 608 | # except OSError as e: 609 | # self.perror(f"failed to create directory structure: {e}") 610 | # return 611 | except EtcHostsFailed: 612 | self.perror("failed to add host to /etc/hosts") 613 | return 614 | 615 | if len(args.machine.services) == 0 or args.force: 616 | self.poutput("enumerating machine services") 617 | try: 618 | args.machine.enumerate() 619 | except MasscanFailed: 620 | self.perror("masscan failed") 621 | except NmapFailed: 622 | self.perror("nmap failed") 623 | else: 624 | self.poutput( 625 | f"{args.machine.name} already enumerated ({len(args.machine.services)} service(s) detected)" 626 | ) 627 | 628 | def _machine_scan(self, args: argparse.Namespace) -> None: 629 | """ Scan the open service for the given machine """ 630 | 631 | # args.machine shorthand 632 | m = args.machine 633 | 634 | if not m.spawned: 635 | if self.cnxn.assigned is not None: 636 | self.perror( 637 | f"unable to start {m.name}. {self.cnxn.assigned.name} is currently assigned." 638 | ) 639 | return 640 | elif not self.wait_for_machine(m): 641 | return 642 | 643 | if args.recommended: 644 | scanners = [s for s in AVAILABLE_SCANNERS if s.recommended and s.match(m)] 645 | elif args.scanner: 646 | scanners = [s for s in AVAILABLE_SCANNERS if s.name == args.scanner] 647 | else: 648 | scanners = [s for s in AVAILABLE_SCANNERS if s.match(m)] 649 | 650 | if args.service: 651 | port = int(args.service.split("/")[0]) 652 | protocol = args.service.split("/")[1] 653 | services = [ 654 | s for s in m.services if s.port == port and s.protocol == protocol 655 | ] 656 | else: 657 | services = m.services 658 | 659 | if len(services) == 0: 660 | self.perror("no matching services found") 661 | return 662 | 663 | # Get scanners that match a service specified/present 664 | if args.recommended: 665 | scanners = [ 666 | s 667 | for s in scanners 668 | if s.recommended and any([s.match_service(svc) for svc in services]) 669 | ] 670 | elif args.scanner: 671 | scanners = [ 672 | s 673 | for s in scanners 674 | if s.name == args.scanner 675 | and any([s.match_service(svc) for svc in services]) 676 | ] 677 | else: 678 | scanners = [ 679 | s for s in scanners if any([s.match_service(svc) for svc in services]) 680 | ] 681 | 682 | if len(scanners) == 0: 683 | self.perror(f"no matching scanners found") 684 | return 685 | 686 | # Iterate over all scanners and services to run the correct scans 687 | for service in services: 688 | for scanner in scanners: 689 | if not scanner.match_service(service): 690 | continue 691 | 692 | self.poutput( 693 | f"beginning {scanner.name} scan on {service.port}/{service.protocol} ({service.name})" 694 | ) 695 | tracker = m.scan(scanner, service, silent=args.background) 696 | if args.background: 697 | # Transfer control of the scan to the `jobs` command 698 | tracker.events = self.job_events 699 | tracker.lock.release() 700 | self.jobs.append(tracker) 701 | else: 702 | # Monitor the scan progress in the forground, and give 703 | # options to cancel or background the scan 704 | self.monitor_scan(tracker) 705 | 706 | def monitor_scan(self, tracker: Tracker) -> None: 707 | """ Monitor a foreground scan """ 708 | 709 | # Setup local event queue for completion 710 | events = queue.Queue() 711 | tracker.events = events 712 | tracker.lock.release() 713 | 714 | # Exception used when C-z is pressed to background a task 715 | class GoToSleep(Exception): 716 | pass 717 | 718 | def background_me(signo, stack): 719 | """ Transfer running task to background thread """ 720 | # Turn off the signal handler 721 | signal.signal(signal.SIGTSTP, signal.SIG_DFL) 722 | raise GoToSleep 723 | 724 | try: 725 | # Register C-z handler 726 | signal.signal(signal.SIGTSTP, background_me) 727 | 728 | try: 729 | tracker = tracker.events.get() 730 | except KeyboardInterrupt: 731 | self.pwarning( 732 | f"cancelling {tracker.scanner.name} for {tracker.service.port}/{tracker.service.protocol}" 733 | ) 734 | tracker.stop = True 735 | 736 | # Restore previous signal 737 | signal.signal(signal.SIGTSTP, signal.SIG_DFL) 738 | except GoToSleep: 739 | with tracker.lock: 740 | self.pwarning( 741 | f"backgrounding {tracker.scanner.name} for {tracker.service.port}/{tracker.service.protocol}" 742 | ) 743 | tracker.silent = True 744 | tracker.events = self.job_events 745 | self.jobs.append(tracker) 746 | 747 | # Argument parser for `machine` command 748 | lab_parser = Cmd2ArgumentParser(description="View and manage lab VPN connection") 749 | 750 | @cmd2.with_argparser(lab_parser) 751 | @cmd2.with_category("Hack the Box") 752 | def do_lab(self, args: argparse.Namespace) -> bool: 753 | """ Execute the various lab sub-commands """ 754 | actions = { 755 | "status": self._lab_status, 756 | "switch": self._lab_switch, 757 | "config": self._lab_config, 758 | "connect": self._lab_connect, 759 | "disconnect": self._lab_disconnect, 760 | "import": self._lab_import, 761 | } 762 | actions[args.action](args) 763 | return False 764 | 765 | def _lab_status(self, args: argparse.Namespace) -> None: 766 | """ Print the lab VPN status """ 767 | 768 | lab = self.cnxn.lab 769 | 770 | output = [] 771 | 772 | output.append( 773 | f"{Style.BRIGHT}Server: {Style.RESET_ALL}{Fore.CYAN}{lab.name}{Fore.RESET} ({lab.hostname}:{lab.port})" 774 | ) 775 | 776 | if lab.active: 777 | output.append( 778 | f"{Style.BRIGHT}Status: {Style.RESET_ALL}{Fore.GREEN}Connected{Fore.RESET}" 779 | ) 780 | output.append( 781 | f"{Style.BRIGHT}IPv4 Address: {Style.RESET_ALL}{Style.DIM+Fore.GREEN}{lab.ipv4}{Style.RESET_ALL}" 782 | ) 783 | output.append( 784 | f"{Style.BRIGHT}IPv6 Address: {Style.RESET_ALL}{Style.DIM+Fore.MAGENTA}{lab.ipv6}{Style.RESET_ALL}" 785 | ) 786 | output.append( 787 | f"{Style.BRIGHT}Traffic: {Style.RESET_ALL}{Fore.GREEN}{lab.rate_up}{Fore.RESET} up, {Fore.CYAN}{lab.rate_down}{Fore.RESET} down" 788 | ) 789 | else: 790 | output.append( 791 | f"{Style.BRIGHT}Status: {Style.RESET_ALL}{Fore.RED}Disconnected{Fore.RESET}" 792 | ) 793 | 794 | self.poutput("\n".join(output)) 795 | 796 | def _lab_switch(self, args: argparse.Namespace) -> None: 797 | """ Switch labs """ 798 | try: 799 | self.cnxn.lab.switch(args.lab) 800 | except RequestFailed as e: 801 | self.perror(f"failed to switch: {e}") 802 | else: 803 | self.psuccess(f"lab switched to {args.lab}") 804 | 805 | def _lab_config(self, args: argparse.Namespace) -> None: 806 | """ Download OVPN configuration file """ 807 | try: 808 | self.poutput(self.cnxn.lab.config.decode("utf-8"), apply_style=False) 809 | except AuthFailure: 810 | self.perror("authentication failure (did you supply email/password?)") 811 | 812 | def _lab_connect(self, args: argparse.Namespace) -> None: 813 | """ Connect to the Hack the Box VPN using NetworkManager """ 814 | 815 | # Attempt to grab the VPN if it exists, and import it if it doesn't 816 | connection, uuid = self._nm_import_vpn(name="python-htb", force=False) 817 | if connection is None: 818 | # nm_import_vpn handles error output 819 | return 820 | 821 | # Check if this connection is active on any devices 822 | for active_connection in NetworkManager.NetworkManager.ActiveConnections: 823 | if active_connection.Uuid == uuid: 824 | self.poutput(f"vpn connection already active") 825 | return 826 | 827 | # Activate the connection 828 | for device in NetworkManager.NetworkManager.GetDevices(): 829 | # Attempt to activate the VPN on each wired and wireless device... 830 | # I couldn't find a good way to do this intelligently other than 831 | # trying them until one worked... 832 | if ( 833 | device.DeviceType == NetworkManager.NM_DEVICE_TYPE_ETHERNET 834 | or device.DeviceType == NetworkManager.NM_DEVICE_TYPE_WIFI 835 | ): 836 | try: 837 | active_connection = NetworkManager.NetworkManager.ActivateConnection( 838 | connection, device, "/" 839 | ) 840 | if active_connection is None: 841 | self.perror("failed to activate vpn connection") 842 | return 843 | except dbus.exceptions.DBusException: 844 | continue 845 | else: 846 | break 847 | else: 848 | self.perror("vpn connection failed") 849 | return 850 | 851 | # Wait for VPN to become active or transition to failed 852 | while ( 853 | active_connection.VpnState 854 | < NetworkManager.NM_VPN_CONNECTION_STATE_ACTIVATED 855 | ): 856 | time.sleep(0.5) 857 | 858 | if ( 859 | active_connection.VpnState 860 | != NetworkManager.NM_VPN_CONNECTION_STATE_ACTIVATED 861 | ): 862 | self.perror("vpn connection failed") 863 | return 864 | 865 | self.psuccess( 866 | f"connected w/ ipv4 address: {active_connection.Ip4Config.Addresses[0][0]}/{active_connection.Ip4Config.Addresses[0][1]}" 867 | ) 868 | 869 | def _lab_disconnect(self, args: argparse.Namespace) -> None: 870 | """ Disconnect from Hack the Box VPN via Network Manager """ 871 | 872 | if "lab" not in self.config or "connection" not in self.config["lab"]: 873 | self.perror('lab vpn configuration not imported (hint: use "lab import")') 874 | return 875 | 876 | for c in NetworkManager.NetworkManager.ActiveConnections: 877 | if c.Uuid == self.config["lab"]["connection"]: 878 | NetworkManager.NetworkManager.DeactivateConnection(c) 879 | self.psuccess("vpn connection deactivated") 880 | break 881 | else: 882 | self.poutput("vpn connection not active or not found") 883 | 884 | def _lab_import(self, args: argparse.Namespace) -> None: 885 | """ Import OpenVPN configuration into NetworkManager """ 886 | 887 | # Import the connection 888 | c, uuid = self._nm_import_vpn(args.name, force=args.reload) 889 | 890 | # "nm_import_vpn" handles error/warning output 891 | if c is None: 892 | return 893 | 894 | self.psuccess(f"imported vpn configuration w/ uuid {uuid}") 895 | 896 | def _nm_import_vpn(self, name, force=True) -> NetworkManager.Connection: 897 | """ Import the VPN configuration with the specified name """ 898 | 899 | # Ensure we aren't already managing a connection 900 | try: 901 | c, uuid = self._nm_get_vpn_connection() 902 | if force: 903 | c.Delete() 904 | else: 905 | return c, uuid 906 | except ConnectionNotFound: 907 | pass 908 | except InvalidConnectionID: 909 | self.pwarning("invalid connection id found in configuration; removing.") 910 | 911 | # We need to download and import the OVPN configuration file 912 | with tempfile.NamedTemporaryFile() as ovpn: 913 | # Write the configuration to a file 914 | ovpn.write(self.cnxn.lab.config) 915 | 916 | # Import the connection w/ Network Manager CLI 917 | p = subprocess.run( 918 | ["nmcli", "c", "import", "type", "openvpn", "file", ovpn.name], 919 | stdout=subprocess.PIPE, 920 | stderr=subprocess.PIPE, 921 | ) 922 | if p.returncode != 0: 923 | self.perror("failed to import vpn configuration") 924 | self.perror( 925 | "tip: try importing the config manually and fixing any network manager issues:\n\tnmcli connection import type openvpn file {your-ovpn-file}" 926 | ) 927 | self.perror("nmcli stderr output:\n" + p.stderr.decode("utf-8")) 928 | return None, None 929 | 930 | # Parse the UUID out of the output 931 | try: 932 | uuid = p.stdout.split(b"(")[1].split(b")")[0].decode("utf-8") 933 | except: 934 | self.perror("unexpected output from nmcli") 935 | self.perror( 936 | "tip: try importing the config manually and fixing any network manager issues:\n\tnmcli connection import type openvpn file {your-ovpn-file}" 937 | ) 938 | self.perror("nmcli stderr output:\n" + p.stderr.decode("utf-8")) 939 | self.perror("nmcli stdout output:\n" + p.stdout.decode("utf-8")) 940 | return None, None 941 | 942 | try: 943 | # Grab the connection object 944 | connection = NetworkManager.Settings.GetConnectionByUuid(uuid) 945 | 946 | # Ensure the routing settings are correct 947 | connection_settings = connection.GetSettings() 948 | connection_settings["connection"]["id"] = name 949 | connection_settings["ipv4"]["never-default"] = True 950 | connection_settings["ipv6"]["never-default"] = True 951 | connection.Update(connection_settings) 952 | except dbus.exceptions.DBusException as e: 953 | self.perror(f"dbus error during connection lookup: {e}") 954 | return None, None 955 | 956 | # Save the uuid in our configuration file 957 | self.config["lab"] = {} 958 | self.config["lab"]["connection"] = uuid 959 | with open(self.config_path, "w") as f: 960 | self.config.write(f) 961 | 962 | return connection, uuid 963 | 964 | def _nm_get_vpn_connection(self) -> NetworkManager.Connection: 965 | """ Grab the NetworkManager VPN configuration object """ 966 | 967 | if "lab" not in self.config or "connection" not in self.config["lab"]: 968 | raise ConnectionNotFound 969 | 970 | try: 971 | # Grab the connection 972 | c = NetworkManager.Settings.GetConnectionByUuid( 973 | self.config["lab"]["connection"] 974 | ) 975 | except dbus.exceptions.DBusException as e: 976 | raise InvalidConnectionID(str(e)) 977 | 978 | return c, self.config["lab"]["connection"] 979 | 980 | @cmd2.with_category("Hack the Box") 981 | def do_invalidate(self, args) -> None: 982 | """ Invalidate API cache """ 983 | self.cnxn.invalidate_cache() 984 | 985 | 986 | def complete_machine( 987 | self, running=None, active=None, term_or_reset=None 988 | ) -> List[cmd2.argparse_custom.CompletionItem]: 989 | """ Return a list of CompletionItems for machines """ 990 | result = [] 991 | for m in self.cnxn.machines: 992 | # Match active 993 | if active is not None and m.expired == active: 994 | continue 995 | # Match running 996 | if running is not None and m.spawned != running: 997 | continue 998 | # Match terminating or resetting 999 | if term_or_reset is not None and not m.terminating and not m.resetting: 1000 | continue 1001 | 1002 | if m.resetting: 1003 | state = "resetting" 1004 | elif m.terminating: 1005 | state = "terminating" 1006 | elif m.spawned: 1007 | state = m.expires 1008 | else: 1009 | state = "stopped" 1010 | 1011 | os = f"{HackTheBox.OS_ICONS.get(m.os.lower(), HackTheBox.OS_ICONS['other'])} {m.os}" 1012 | result.append(cmd2.CompletionItem(m.name.lower(), f"{os:<13}{m.ip:<13}{state}")) 1013 | 1014 | return result 1015 | 1016 | 1017 | MACHINE_DESCRIPTION = f"{'OS':<13}{'IP':<13}State" 1018 | 1019 | 1020 | def ArgparseMachineType(arg: str) -> Machine: 1021 | """ 1022 | 1023 | :param arg: The argument passed at the command line 1024 | :type arg: str 1025 | :return: A machine object or raises invalid argument error 1026 | :raises: argparse.ArgumentTypeError 1027 | """ 1028 | 1029 | # We know it has already been configured, no params needed 1030 | self = HackTheBox.get() 1031 | 1032 | if arg == HackTheBox.ASSIGNED: 1033 | m = self.cnxn.assigned 1034 | if m is None: 1035 | raise argparse.ArgumentTypeError(f"no currently assigned machine") 1036 | else: 1037 | # Convert to integer, if possible. Otherwise pass as-is 1038 | try: 1039 | machine_id = int(arg) 1040 | except ValueError: 1041 | machine_id = arg 1042 | 1043 | try: 1044 | m = self.cnxn[machine_id] 1045 | except KeyError: 1046 | raise argparse.ArgumentTypeError(f"{machine_id}: no such machine") 1047 | 1048 | return m 1049 | 1050 | 1051 | def main(): 1052 | 1053 | if "HTBRC" in os.environ: 1054 | config = os.environ["HTBRC"] 1055 | else: 1056 | config = "~/.htbrc" 1057 | 1058 | # Setup the job parser for the cmd2 object 1059 | HackTheBox.jobs_parser.set_defaults(action="list") 1060 | jobs_subparsers = HackTheBox.jobs_parser.add_subparsers( 1061 | help="Actions", dest="_action" 1062 | ) 1063 | 1064 | # "job kill" parser 1065 | jobs_kill_parser = jobs_subparsers.add_parser( 1066 | "kill", 1067 | aliases=["rm", "stop"], 1068 | description="Stop a running background scanner job", 1069 | prog="jobs kill", 1070 | ) 1071 | jobs_kill_parser.add_argument("job_id", type=int, help="Kill the identified job") 1072 | jobs_kill_parser.set_defaults(action="kill") 1073 | 1074 | # "job list" parser 1075 | jobs_list_parser = jobs_subparsers.add_parser( 1076 | "list", 1077 | aliases=["ls"], 1078 | description="List background scanner jobs and their status", 1079 | prog="jobs list", 1080 | ) 1081 | jobs_list_parser.set_defaults(action="list") 1082 | 1083 | # "machine" argument parser 1084 | HackTheBox.machine_parser.set_defaults( 1085 | action="list", state="all", owned="all", todo=None 1086 | ) 1087 | machine_subparsers = HackTheBox.machine_parser.add_subparsers( 1088 | help="Actions", dest="_action" 1089 | ) 1090 | 1091 | # "machine list" argument parser 1092 | machine_list_parser = machine_subparsers.add_parser( 1093 | "list", aliases=["ls"], help="List machines", prog="machine list" 1094 | ) 1095 | machine_list_parser.set_defaults(action="list") 1096 | machine_list_parser.add_argument( 1097 | "--inactive", "-i", action="store_const", const="inactive", dest="state" 1098 | ) 1099 | machine_list_parser.add_argument( 1100 | "--active", 1101 | "-a", 1102 | action="store_const", 1103 | const="active", 1104 | dest="state", 1105 | default="all", 1106 | ) 1107 | machine_list_parser.add_argument( 1108 | "--owned", "-o", action="store_const", const="owned", default="all" 1109 | ) 1110 | machine_list_parser.add_argument( 1111 | "--unowned", "-u", action="store_const", const="unowned", dest="owned" 1112 | ) 1113 | machine_list_parser.add_argument("--todo", "-t", action="store_true") 1114 | machine_list_parser.set_defaults(state="all", owned="all") 1115 | 1116 | # "machine start" argument parser 1117 | machine_start_parser = machine_subparsers.add_parser( 1118 | "start", aliases=["up", "spawn"], help="Start a machine", prog="machine up" 1119 | ) 1120 | machine_start_parser.add_argument( 1121 | "machine", 1122 | help="A name regex, IP address or machine ID to start", 1123 | type=ArgparseMachineType, 1124 | choices_method=functools.partial(complete_machine, running=False), 1125 | descriptive_header=MACHINE_DESCRIPTION, 1126 | ) 1127 | machine_start_parser.set_defaults(action="start") 1128 | 1129 | # "machine reset" argument parser 1130 | machine_reset_parser = machine_subparsers.add_parser( 1131 | "reset", 1132 | aliases=["restart"], 1133 | help="Schedule a machine reset", 1134 | prog="machine reset", 1135 | ) 1136 | machine_reset_parser.add_argument( 1137 | "machine", 1138 | help="A name regex, IP address or machine ID", 1139 | type=ArgparseMachineType, 1140 | choices_method=functools.partial(complete_machine, running=True), 1141 | descriptive_header=MACHINE_DESCRIPTION, 1142 | ) 1143 | machine_reset_parser.set_defaults(action="reset") 1144 | 1145 | # "machine stop" argument parser 1146 | machine_stop_parser = machine_subparsers.add_parser( 1147 | "stop", aliases=["down", "shutdown"], help="Stop a machine", prog="machine down" 1148 | ) 1149 | machine_stop_parser.add_argument( 1150 | "machine", 1151 | nargs="?", 1152 | help="A name regex, IP address or machine ID to start (default: assigned)", 1153 | default=HackTheBox.ASSIGNED, 1154 | type=ArgparseMachineType, 1155 | choices_method=functools.partial(complete_machine, running=False), 1156 | descriptive_header=MACHINE_DESCRIPTION, 1157 | ) 1158 | machine_stop_parser.set_defaults(action="stop") 1159 | 1160 | # "machine info" argument parser 1161 | machine_info_parser = machine_subparsers.add_parser( 1162 | "info", 1163 | aliases=["cat", "show"], 1164 | help="Show detailed machine information", 1165 | prog="machine info", 1166 | ) 1167 | machine_info_parser.add_argument( 1168 | "machine", 1169 | nargs="?", 1170 | help="A name regex, IP address or machine ID (default: assigned)", 1171 | default=HackTheBox.ASSIGNED, 1172 | type=ArgparseMachineType, 1173 | choices_method=complete_machine, 1174 | descriptive_header=MACHINE_DESCRIPTION, 1175 | ) 1176 | machine_info_parser.set_defaults(action="info") 1177 | 1178 | # "machine own" argument parser 1179 | machine_own_parser = machine_subparsers.add_parser( 1180 | "own", 1181 | aliases=["submit", "shutdown"], 1182 | help="Submit a root or user flag", 1183 | prog="machine own", 1184 | ) 1185 | machine_own_parser.add_argument( 1186 | "--rate", 1187 | "-r", 1188 | type=int, 1189 | default=0, 1190 | choices=range(1, 100), 1191 | help="Difficulty Rating (1-100)", 1192 | ) 1193 | machine_own_parser.add_argument( 1194 | "machine", 1195 | nargs="?", 1196 | help="A name regex, IP address or machine ID (default: assigned)", 1197 | default=HackTheBox.ASSIGNED, 1198 | type=ArgparseMachineType, 1199 | choices_method=complete_machine, 1200 | descriptive_header=MACHINE_DESCRIPTION, 1201 | ) 1202 | machine_own_parser.add_argument("flag", help="The user or root flag") 1203 | machine_own_parser.set_defaults(action="own") 1204 | 1205 | # "machine cancel" argument parser 1206 | machine_cancel_parser = machine_subparsers.add_parser( 1207 | "cancel", 1208 | description="Cancel a pending termination or reset for a machine", 1209 | prog="machine cancel", 1210 | ) 1211 | machine_cancel_parser.add_argument( 1212 | "--termination", 1213 | "-t", 1214 | action="append_const", 1215 | const="t", 1216 | dest="cancel", 1217 | help="Cancel a machine termination", 1218 | ) 1219 | machine_cancel_parser.add_argument( 1220 | "--reset", 1221 | "-r", 1222 | action="append_const", 1223 | const="r", 1224 | dest="cancel", 1225 | help="Cancel a machine reset", 1226 | ) 1227 | machine_cancel_parser.add_argument( 1228 | "--both", 1229 | "-b", 1230 | action="store_const", 1231 | const=[], 1232 | dest="cancel", 1233 | help="Cancel machine reset and termination", 1234 | ) 1235 | machine_cancel_parser.add_argument( 1236 | "machine", 1237 | nargs="?", 1238 | help="A name regex, IP address or machine ID (default: assigned)", 1239 | default=HackTheBox.ASSIGNED, 1240 | type=ArgparseMachineType, 1241 | choices_method=functools.partial(complete_machine, term_or_reset=True), 1242 | descriptive_header=MACHINE_DESCRIPTION, 1243 | ) 1244 | machine_cancel_parser.set_defaults(action="cancel", cancel=[]) 1245 | 1246 | # "machine enum" argument parser 1247 | machine_enum_parser = machine_subparsers.add_parser( 1248 | "enum", aliases=["enumerate"], help="Perform initial service enumeration" 1249 | ) 1250 | machine_enum_parser.add_argument( 1251 | "--force", "-f", action="store_true", help="Force re-enumeration", default=False 1252 | ) 1253 | machine_enum_parser.add_argument( 1254 | "machine", 1255 | nargs="?", 1256 | help="A name regex, IP address or machine ID to start (default: assigned)", 1257 | default=HackTheBox.ASSIGNED, 1258 | type=ArgparseMachineType, 1259 | choices_method=complete_machine, 1260 | descriptive_header=MACHINE_DESCRIPTION, 1261 | ) 1262 | machine_enum_parser.set_defaults(action="enum") 1263 | 1264 | # "machine scan" argument parser 1265 | machine_scan_parser = machine_subparsers.add_parser( 1266 | "scan", 1267 | help="Perform prepared applicable scans against this host", 1268 | prog="machine scan", 1269 | ) 1270 | machine_scan_parser.add_argument( 1271 | "--service", 1272 | "-v", 1273 | help="Only run scans for this service (format: `{PORT}/{PROTOCOL}`)", 1274 | ) 1275 | machine_scan_parser.add_argument( 1276 | "--scanner", "-s", help="Only run scans for this scanner" 1277 | ) 1278 | machine_scan_parser.add_argument( 1279 | "--recommended", "-r", help="Run all recommended scans" 1280 | ) 1281 | machine_scan_parser.add_argument( 1282 | "--background", 1283 | "-b", 1284 | help="Run scans in the background", 1285 | action="store_true", 1286 | default=False, 1287 | ) 1288 | machine_scan_parser.add_argument( 1289 | "machine", 1290 | nargs="?", 1291 | help="A name regex, IP address or machine ID to start (default: assigned)", 1292 | default=HackTheBox.ASSIGNED, 1293 | type=ArgparseMachineType, 1294 | choices_method=complete_machine, 1295 | descriptive_header=MACHINE_DESCRIPTION, 1296 | ) 1297 | machine_scan_parser.set_defaults(action="scan") 1298 | 1299 | # "lab" argument parser setup 1300 | HackTheBox.lab_parser.set_defaults(action="status") 1301 | lab_subparsers = HackTheBox.lab_parser.add_subparsers( 1302 | help="Actions", dest="_action" 1303 | ) 1304 | 1305 | # "lab status" argument parser 1306 | lab_status_parser = lab_subparsers.add_parser( 1307 | "status", 1308 | description="Show the connection status of the currently assigned lab VPN", 1309 | prog="lab status", 1310 | ) 1311 | lab_status_parser.set_defaults(action="status") 1312 | 1313 | # "lab switch" argument parser 1314 | lab_switch_parser = lab_subparsers.add_parser( 1315 | "switch", 1316 | description="Show the connection status of the currently assigned lab VPN", 1317 | prog="lab switch", 1318 | ) 1319 | lab_switch_parser.add_argument( 1320 | "lab", choices=VPN.VALID_LABS, type=str, help="The lab to switch to" 1321 | ) 1322 | lab_switch_parser.set_defaults(action="switch") 1323 | 1324 | # "lab config" argument parser 1325 | lab_config_parser = lab_subparsers.add_parser( 1326 | "config", description="Download OVPN configuration file", prog="lab config", 1327 | ) 1328 | lab_config_parser.set_defaults(action="config") 1329 | 1330 | # "lab connect" argument parser 1331 | lab_connect_parser = lab_subparsers.add_parser( 1332 | "connect", 1333 | description="Connect to the Hack the Box VPN. If no previous configuration has been created in NetworkManager, it attempts to download it and import it.", 1334 | prog="lab connect", 1335 | ) 1336 | lab_connect_parser.add_argument( 1337 | "--update", 1338 | "-u", 1339 | action="store_true", 1340 | help="Force a redownload/import of the OpenVPN configuration", 1341 | ) 1342 | lab_connect_parser.set_defaults(action="connect") 1343 | 1344 | # "lab disconnect" argument parser 1345 | lab_disconnect_parser = lab_subparsers.add_parser( 1346 | "disconnect", 1347 | description="Disconnect from the Hack the Box lab VPN", 1348 | prog="lab disconnect", 1349 | ) 1350 | lab_disconnect_parser.set_defaults(action="disconnect") 1351 | 1352 | # "lab import" argument parser 1353 | lab_import_parser = lab_subparsers.add_parser( 1354 | "import", 1355 | description="Import your OpenVPN configuration into Network Manager", 1356 | prog="lab import", 1357 | ) 1358 | lab_import_parser.add_argument( 1359 | "--reload", 1360 | "-r", 1361 | action="store_true", 1362 | help="Reload configuration from Hack the Box", 1363 | ) 1364 | lab_import_parser.add_argument( 1365 | "--name", "-n", default="python-htb", help="NetworkManager Connection ID" 1366 | ) 1367 | lab_import_parser.set_defaults(action="import") 1368 | 1369 | # Build REPL object 1370 | cmd = HackTheBox.get(resource=config, allow_cli_args=False) 1371 | 1372 | # Run remaning arguments as a command 1373 | if len(sys.argv) > 1: 1374 | try: 1375 | cmd.onecmd(" ".join([shlex.quote(x) for x in sys.argv[1:]])) 1376 | except cmd2.exceptions.Cmd2ArgparseError: 1377 | sys.exit(1) 1378 | except RequestFailed as r: 1379 | cmd.perror(f"request failed: {r}") 1380 | if len([j for j in cmd.jobs if j.thread is not None]): 1381 | cmd.pwarning("background jobs active. staring interpreter...") 1382 | result = cmd.cmdloop() 1383 | else: 1384 | result = 0 1385 | else: 1386 | result = cmd.cmdloop() 1387 | 1388 | try: 1389 | if len([j for j in cmd.jobs if j.thread is not None]): 1390 | cmd.poutput("waiting for background jobs to complete") 1391 | while len([j for j in cmd.jobs if j.thread is not None]): 1392 | tracker = cmd.job_events.get() 1393 | tracker.status = "completed" 1394 | tracker.thread = None 1395 | except KeyboardInterrupt: 1396 | cmd.pwarning("cancelling background jobs") 1397 | for j in cmd.jobs: 1398 | j.stop = True 1399 | 1400 | try: 1401 | while len([j for j in cmd.jobs if j.thread is not None]): 1402 | tracker = cmd.job_events.get() 1403 | tracker.status = "completed" 1404 | tracker.thread = None 1405 | except KeyboardInterrupt: 1406 | cmd.pwarning("forcing background job exit!") 1407 | for j in [j for j in cmd.jobs if j.thread is not None]: 1408 | j.thread.daemon = True 1409 | 1410 | cmd.config["htb"]["session"] = cmd.cnxn.session.cookies.get( 1411 | "hackthebox_session", default="", domain="www.hackthebox.eu" 1412 | ) 1413 | with open(os.path.expanduser(config), "w") as f: 1414 | cmd.config.write(f) 1415 | 1416 | for m in cmd.cnxn.machines: 1417 | m.dump() 1418 | 1419 | 1420 | if __name__ == "__main__": 1421 | main() 1422 | -------------------------------------------------------------------------------- /htb/connection.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Any, Dict, List, Union, Callable 3 | from configparser import ConfigParser 4 | import threading 5 | import requests 6 | import time 7 | import json 8 | import re 9 | 10 | from htb.exceptions import * 11 | from htb.vpn import VPN 12 | from htb.machine import Machine 13 | 14 | 15 | class Connection(object): 16 | """ Server Connection Object """ 17 | 18 | BASE_URL = "https://www.hackthebox.eu" 19 | 20 | def __init__( 21 | self, 22 | api_token: str, 23 | email=None, 24 | password=None, 25 | existing_session=None, 26 | analysis_path=None, 27 | twofactor_prompt: Callable = None, 28 | subscribe: bool = False, 29 | config: ConfigParser = ConfigParser(), 30 | ): 31 | """ Construct a connection with the specified API key """ 32 | 33 | # Save configuration info 34 | self.config = config 35 | 36 | # Save the API key 37 | self.api_token: str = api_token 38 | 39 | # Save authentication information, if we were given any 40 | self.email: str = email 41 | self.password: str = password 42 | 43 | # API result cache 44 | self._cache: Dict[str, Any] = {} 45 | self.cache_timeout: float = 60 46 | 47 | # Callback to get two factor prompt 48 | self.twofactor_prompt = twofactor_prompt 49 | 50 | # Ongoing session for standard authentication 51 | self.session = requests.Session() 52 | self.session.cookies.update({"hackthebox_session": existing_session}) 53 | 54 | # List of tracked machines 55 | self._machines: Dict[int, Machine] = {} 56 | 57 | # Path where machine analysis is kept 58 | self.analysis_path = analysis_path 59 | 60 | # Subscribe the asynchronous messages via Pusher (WebSockets) 61 | if subscribe: 62 | # If you don't subscribe, you don't need pysher 63 | import pysher 64 | 65 | self.subcribed: bool = True 66 | self.subscriber_lock: threading.Lock = threading.RLock() 67 | self.subscribers: Dict[str, Callable] = {} 68 | self.pusher = pysher.Pusher("97608bf7532e6f0fe898", cluster="eu") 69 | 70 | def _on_connect(data): 71 | channel = self.pusher.subscribe("notifications-channel") 72 | channel.bind("display-notification", self._on_notification) 73 | 74 | self.pusher.connection.bind("pusher:connection_established", _on_connect) 75 | self.pusher.connect() 76 | 77 | def _on_notification(self, *args, **kwargs) -> None: 78 | """ Receive notifications from Hack the Box and distribute them to 79 | subscribers """ 80 | message = json.loads(args[0]) 81 | subscribers = {} 82 | for name, callback in self.subscribers.items(): 83 | if callback(message): 84 | subscribers[name] = callback 85 | 86 | def subscribe(self, name: str, subscriber: Callable) -> None: 87 | """ Subscribe to notification messages """ 88 | 89 | with self.subscriber_lock: 90 | if name in self.subscribers: 91 | raise ValueError(f"{name}: already registered subscriber") 92 | 93 | self.subscribers[name] = subscriber 94 | 95 | def unsubscribe(self, name: str) -> None: 96 | """ Unsubscribe from notification messages """ 97 | 98 | with self.subscriber_lock: 99 | if name not in self.subscribers: 100 | raise KeyError(f"{name} not a registered subscriber") 101 | 102 | del self.subscribers[name] 103 | 104 | def invalidate_cache(self, endpoint: str = None, method: str = None) -> None: 105 | """ Invalidate the cache of one endpoint, endpoint/method or all entries """ 106 | 107 | if endpoint is None: 108 | self._cache = {} 109 | elif method is None and endpoint in self._cache: 110 | self._cache[endpoint] = {} 111 | elif endpoint in self._cache and method in self._cache[endpoint]: 112 | self._cache[endpoint][method] = (0, None) 113 | 114 | def _api(self, endpoint, args={}, method="post", cache=False, **kwargs) -> Dict: 115 | """ Send an API requests with the stored API key """ 116 | 117 | # If requested, attempt to cache the response for up to `self.cache_timeout` seconds 118 | if cache and endpoint in self._cache and method in self._cache[endpoint]: 119 | if (time.time() - self._cache[endpoint][method][0]) < self.cache_timeout: 120 | return self._cache[endpoint][method][1] 121 | 122 | # Construct necessary parameters for request 123 | url = f"{Connection.BASE_URL}/api/{endpoint.lstrip('/')}" 124 | headers = { 125 | "User-Agent": "https://github.com/calebstewart/python-htb", 126 | "Authorization": f"Bearer {self.api_token}", 127 | } 128 | methods = {"post": requests.post, "get": requests.get} 129 | args.update({"api_token": self.api_token}) 130 | 131 | # Request failed 132 | r = methods[method.lower()]( 133 | url, params=args, headers=headers, allow_redirects=False, **kwargs 134 | ) 135 | if r.status_code != 200: 136 | raise AuthFailure 137 | 138 | # Grab response data 139 | response = r.json() 140 | 141 | # It's an integer but they always send it as a string :( 142 | if "success" in response: 143 | if isinstance(response["success"], str): 144 | response["success"] = int(response["success"]) 145 | 146 | # Save the response for future cache reuse 147 | if cache: 148 | if endpoint not in self._cache: 149 | self._cache[endpoint] = {} 150 | self._cache[endpoint][method] = (time.time(), response) 151 | 152 | return response 153 | 154 | def _request( 155 | self, endpoint, method, _retry_auth=True, **kwargs 156 | ) -> requests.Response: 157 | """ Make a standard (non-api) request. May require authentication prior, 158 | but in order to authenticate, the connection must have been given 159 | credentials beyond the required auth token. """ 160 | 161 | # Easy lookup table for request method 162 | methods = {"get": self.session.get, "post": self.session.post} 163 | headers = {"User-Agent": "https://github.com/calebstewart/python-htb"} 164 | 165 | if "headers" in kwargs: 166 | kwargs["headers"].update(headers) 167 | else: 168 | kwargs["headers"] = headers 169 | 170 | # Send request 171 | r = methods[method]( 172 | f"{Connection.BASE_URL}/{endpoint.lstrip('/')}", 173 | allow_redirects=False, 174 | **kwargs, 175 | ) 176 | 177 | if r.status_code == 302: 178 | if _retry_auth: 179 | self._authenticate() 180 | return self._request(endpoint, method, _retry_auth=False, **kwargs) 181 | else: 182 | raise AuthFailure 183 | 184 | return r 185 | 186 | def _authenticate(self) -> None: 187 | """ Check that the provided API key is valid and query user details """ 188 | 189 | # Test email/password auth as well 190 | if self.email is not None and self.password is not None: 191 | 192 | # Build session object 193 | self.session = requests.Session() 194 | headers = {"User-Agent": "https://github.com/calebstewart/python-htb"} 195 | 196 | # Grab CSRF Token 197 | r = self.session.get(f"{Connection.BASE_URL}/login", headers=headers) 198 | data = r.text.split('id="loginForm"')[1] 199 | token = data.split('_token" value="')[1].split('"')[0] 200 | 201 | # Authenticate 202 | r = self.session.post( 203 | f"{Connection.BASE_URL}/login", 204 | data={"_token": token, "email": self.email, "password": self.password}, 205 | allow_redirects=False, 206 | headers=headers, 207 | ) 208 | 209 | # Ensure we succeeded 210 | if ( 211 | r.status_code != 302 212 | or r.headers["location"] != "https://www.hackthebox.eu/home" 213 | ): 214 | raise AuthFailure 215 | 216 | # Check for Two Factor Authentication 217 | r = self.session.get(r.headers["location"], headers=headers) 218 | if "One Time Password" not in r.text: 219 | return 220 | 221 | # Prompt for the one time password 222 | token = ( 223 | r.text.split('id="loginForm"')[1] 224 | .split('_token" value="')[1] 225 | .split('"')[0] 226 | ) 227 | 228 | # Request the two-factor one time passcode 229 | otp = self.twofactor_prompt() 230 | 231 | r = self.session.post( 232 | f"{Connection.BASE_URL}/2fa", 233 | data={"_token": token, "one_time_password": otp, "backup_code": ""}, 234 | allow_redirects=False, 235 | headers=headers, 236 | ) 237 | if ( 238 | r.status_code != 302 239 | or r.headers["location"] != "https://www.hackthebox.eu/home" 240 | ): 241 | raise TwoFactorAuthRequired 242 | 243 | @property 244 | def lab(self) -> VPN: 245 | """ Grab the Lab VPN object """ 246 | r = self._api("/users/htb/connection/status") 247 | return VPN(self, r) 248 | 249 | @property 250 | def fortress(self) -> VPN: 251 | """ Grab the Fortress VPN object """ 252 | r = self._api("/users/htb/fortress/connection/status") 253 | return VPN(self, r) 254 | 255 | @property 256 | def machines(self) -> List[Machine]: 257 | """ Grab the list of active machines """ 258 | 259 | # Request all the machine information 260 | data = self._api("/machines/get/all", method="get", cache=True) 261 | 262 | # Build internal machine list 263 | for datum in data: 264 | if int(datum["id"]) in self._machines: 265 | self._machines[int(datum["id"])].update(datum) 266 | else: 267 | self._machines[int(datum["id"])] = Machine(self, datum) 268 | try: 269 | self._machines[int(datum["id"])].load(self.analysis_path) 270 | except NoAnalysisPath: 271 | pass 272 | 273 | # Create machine objects for all the machine information 274 | return [m for _, m in self._machines.items()] 275 | 276 | def get_machine(self, ident: int) -> Machine: 277 | """ Lookup a machine by ID """ 278 | 279 | # request the machine 280 | machine = self._api(f"/machines/get/{ident}", method="get", cache=True) 281 | 282 | if ident in self._machines: 283 | self._machines[ident].update(machine) 284 | else: 285 | self._machines[ident] = Machine(self, machine) 286 | try: 287 | self._machines[ident].load(self.analysis_path) 288 | except NoAnalysisPath: 289 | pass 290 | 291 | return self._machines[ident] 292 | 293 | @property 294 | def active(self) -> List[Machine]: 295 | """ Grab all active machines """ 296 | return [m for m in self.machines if not m.retired] 297 | 298 | @property 299 | def retired(self) -> List[Machine]: 300 | """ Grab all retired machines """ 301 | return [m for m in self.machines if m.retired] 302 | 303 | @property 304 | def todo(self) -> List[Machine]: 305 | """ List of machines marked as "todo" """ 306 | return [m for m in self.machines if m.todo] 307 | 308 | @property 309 | def assigned(self) -> Machine: 310 | """ Return the machine assigned to your or None """ 311 | for m in self.machines: 312 | if m.assigned: 313 | return m 314 | return None 315 | 316 | @property 317 | def spawned(self) -> List[Machine]: 318 | """ All spawned/running machines """ 319 | return [m for m in self.machines if m.spawned] 320 | 321 | def shout(self, message) -> None: 322 | """ Send a message on the shoutbox """ 323 | r = self._api("/shouts/new/", data={"text": message}) 324 | return r 325 | 326 | def __getitem__(self, value: Union[str, int]): 327 | """ Lookup a machine based on either its integer ID or a regular 328 | expression matching its name or IP address """ 329 | 330 | # Find the machine based on name regex 331 | if isinstance(value, str): 332 | m = [ 333 | m 334 | for m in self.machines 335 | if re.match(value, m.name, flags=re.IGNORECASE) 336 | or re.match(value, m.ip, flags=re.IGNORECASE) 337 | ] 338 | # Find machine based on ID 339 | elif isinstance(value, int): 340 | m = [self.get_machine(value)] 341 | else: 342 | # Invalid search 343 | raise ValueError("expected machine id or name regex") 344 | 345 | # Machine does not exist 346 | if len(m) == 0: 347 | raise KeyError("no matching machine found") 348 | 349 | # Return first match 350 | return m[0] 351 | -------------------------------------------------------------------------------- /htb/exceptions.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | 4 | class AlreadyEnumerated(Exception): 5 | """ The machine was already enumerated """ 6 | 7 | pass 8 | 9 | 10 | class NotApplicable(Exception): 11 | """ The given service was not applicable to the scanner """ 12 | 13 | pass 14 | 15 | 16 | class ConnectionNotFound(Exception): 17 | """ No NetworkManager connection found in the configuration file """ 18 | 19 | pass 20 | 21 | 22 | class TwoFactorAuthRequired(Exception): 23 | """ During authentication, the user was prompted for 2FA. """ 24 | 25 | pass 26 | 27 | 28 | class InvalidConnectionID(Exception): 29 | """ The specified NetworkManager connection UUID doesn't exist """ 30 | 31 | pass 32 | 33 | 34 | class AuthFailure(Exception): 35 | """ Authentication Failure """ 36 | 37 | pass 38 | 39 | 40 | class RequestFailed(Exception): 41 | """ A request recieved a negative response from the server """ 42 | 43 | pass 44 | 45 | 46 | class NotRunning(Exception): 47 | """ The requested machine is not running """ 48 | 49 | pass 50 | 51 | 52 | class Terminating(Exception): 53 | """ The requested machine is currently terminating """ 54 | 55 | pass 56 | 57 | 58 | class NoAnalysisPath(Exception): 59 | """ The requested machine does not have an active analysis directory """ 60 | 61 | pass 62 | 63 | 64 | class EtcHostsFailed(Exception): 65 | """ Adding the machine to /etc/hosts failed """ 66 | 67 | pass 68 | 69 | 70 | class MasscanFailed(Exception): 71 | """ Masscan attempt failed """ 72 | 73 | pass 74 | 75 | 76 | class NmapFailed(Exception): 77 | """ Nmap attempt failed """ 78 | 79 | pass 80 | -------------------------------------------------------------------------------- /htb/machine.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Dict, Any, List 3 | from io import StringIO 4 | import subprocess 5 | import threading 6 | import json 7 | import os 8 | import re 9 | 10 | from htb.scanner import Service, Scanner, Tracker, AVAILABLE_SCANNERS 11 | from htb.exceptions import * 12 | 13 | 14 | class Machine(object): 15 | """ Interact with a Hack the Box machine """ 16 | 17 | def __init__(self, connection: Any, data: Dict[str, Any]): 18 | """ Build a machine object from API data """ 19 | 20 | self.connection = connection 21 | 22 | # Standard data (should always exist) 23 | self.id: int = None 24 | self.name: str = None 25 | self.os: str = None 26 | self.ip: str = None 27 | self.avatar: str = None 28 | self.points: str = None 29 | self.release_date: str = None 30 | self.retire_date: str = None 31 | self.makers: List[Dict] = None 32 | self.rating: float = None 33 | self.user_owns: int = None 34 | self.root_owns: int = None 35 | self.free: bool = None 36 | self.analysis_path: str = None 37 | self.services: List[Service] = [] 38 | self.knowns: Dict[str, Any] = {} 39 | 40 | self.update(data) 41 | 42 | def __repr__(self) -> str: 43 | return f"""""" 44 | 45 | def update(self, data: Dict[str, Any]): 46 | """ Update internal machine state from recent request """ 47 | 48 | # Standard data (should always exist) 49 | self.id: int = data["id"] 50 | self.name: str = data["name"].lower() # We don't like capitals :( 51 | self.os: str = data["os"] 52 | self.ip: str = data["ip"] 53 | self.avatar: str = data["avatar_thumb"] 54 | self.points: str = data["points"] 55 | self.release_date: str = data["release"] 56 | self.retire_date: str = data["retired_date"] 57 | self.makers: List[Dict] = [data["maker"]] 58 | self.rating: float = 0.0 59 | self.user_owns: int = data["user_owns"] 60 | self.root_owns: int = data["root_owns"] 61 | self.free: bool = False 62 | 63 | # May exist 64 | if "maker2" in data and data["maker2"] is not None: 65 | self.makers.append(data["maker2"]) 66 | 67 | @property 68 | def hostname(self) -> str: 69 | return f"{self.name.lower()}.htb" 70 | 71 | @property 72 | def todo(self) -> bool: 73 | """ Whether this machine on the todo list """ 74 | todos = self.connection._api("/machines/todo", method="get", cache=True) 75 | return any([t["id"] == self.id for t in todos]) 76 | 77 | @property 78 | def expires(self) -> str: 79 | """ The time until this machine expires """ 80 | 81 | # Grab expiration information for all machines 82 | expiry = self.connection._api("/machines/expiry", method="get", cache=True) 83 | 84 | try: 85 | # Return the expire time for this machine 86 | return [t["expires_at"] for t in expiry if t["id"] == self.id][0] 87 | except IndexError: 88 | # Or return none if it is not running/has no expiration time 89 | return None 90 | 91 | @property 92 | def spawned(self) -> bool: 93 | """ Whether this machine has been spawned """ 94 | 95 | spawned = self.connection._api("/machines/spawned", method="get", cache=True) 96 | 97 | return any([s["id"] == self.id for s in spawned]) 98 | 99 | @property 100 | def terminating(self) -> bool: 101 | """ Whether this machine has been spawned """ 102 | 103 | terminating = self.connection._api( 104 | "/machines/terminating", method="get", cache=True 105 | ) 106 | 107 | return any([s["id"] == self.id for s in terminating]) 108 | 109 | @property 110 | def assigned(self) -> bool: 111 | """ Whether this machine is currently assigned to the logged in user """ 112 | 113 | machines = self.connection._api("/machines/assigned", method="get", cache=True) 114 | 115 | return any([m["id"] == self.id for m in machines]) 116 | 117 | @property 118 | def retired(self) -> bool: 119 | """ Whether this machine is currently assigned to the logged in user """ 120 | 121 | machines = self.connection._api("/machines/get/all", method="get", cache=True) 122 | 123 | try: 124 | return [m["retired"] for m in machines if m["id"] == self.id][0] 125 | except IndexError: 126 | return False 127 | 128 | @property 129 | def resetting(self) -> bool: 130 | """ Whether this machine has been requested to reset """ 131 | 132 | machines = self.connection._api("/machines/resetting", method="get", cache=True) 133 | 134 | return any([m["id"] == self.id for m in machines]) 135 | 136 | @property 137 | def owned_user(self) -> bool: 138 | """ Whether you have owned user on this machine """ 139 | 140 | machines = self.connection._api("/machines/owns", method="get", cache=True) 141 | 142 | return any([m["id"] == self.id and m["owned_user"] for m in machines]) 143 | 144 | @property 145 | def owned_root(self) -> bool: 146 | """ Whether you have owned root on this machine """ 147 | 148 | machines = self.connection._api("/machines/owns", method="get", cache=True) 149 | 150 | return any([m["id"] == self.id and m["owned_root"] for m in machines]) 151 | 152 | @property 153 | def ratings(self) -> bool: 154 | """ The difficulty rating for this machine """ 155 | 156 | machines = self.connection._api( 157 | "/machines/difficulty", method="get", cache=True 158 | ) 159 | 160 | try: 161 | return [m["difficulty_ratings"] for m in machines if m["id"] == self.id][0] 162 | except IndexError: 163 | return [0 for i in range(10)] 164 | 165 | @property 166 | def matrix(self) -> Dict[str, List[int]]: 167 | """ Get the rating matrix for this machine """ 168 | r = self.connection._api( 169 | f"/machines/get/matrix/{self.id}", method="get", cache=True 170 | ) 171 | if r["success"] != 1: 172 | return {"aggregate": [0] * 5, "maker": [0] * 5} 173 | 174 | return {"aggregate": r["aggregate"], "maker": r["maker"]} 175 | 176 | @property 177 | def blood(self) -> Dict[str, str]: 178 | """ Grab machine blood information """ 179 | r = self.connection._api(f"/machines/get/{self.id}", method="get", cache=True) 180 | return {"user": r["user_blood"], "root": r["root_blood"]} 181 | 182 | @todo.setter 183 | def todo(self, value: bool) -> None: 184 | """ Change the current todo status """ 185 | 186 | # Don't do anything if it's already right 187 | if self.todo == value: 188 | return 189 | 190 | # Attempt to update todo 191 | r = self.connection._api(f"/machines/todo/update/{self.id}", method="post", ) 192 | 193 | @spawned.setter 194 | def spawned(self, value: bool) -> None: 195 | """ Start or stop the machine """ 196 | 197 | if value: 198 | action = "assign" 199 | else: 200 | action = "remove" 201 | 202 | # Attempt to start/stop the VM 203 | r = self.connection._api(f"/vm/vip/{action}/{self.id}", method="post", ) 204 | 205 | if r["success"] != 1: 206 | raise RequestFailed(r["status"]) 207 | 208 | @assigned.setter 209 | def assigned(self, value: bool) -> bool: 210 | 211 | # We don't want to be the owner anymore 212 | if not value: 213 | # Trigger setter to remove the VM 214 | self.spawned = False 215 | return 216 | 217 | # We want to transfer ownership to ourselves 218 | r = self.connection._api(f"/vm/vip/assign/{self.id}", method="post", ) 219 | 220 | if r["success"] != 1: 221 | raise RequestFailed(r["status"]) 222 | 223 | @terminating.setter 224 | def terminating(self, value: bool) -> None: 225 | """ Terminate a machine or cancel termination """ 226 | 227 | # Make request 228 | action = "remove" if value else "cancel" 229 | r = self.connection._api(f"/vm/vip/{action}/{self.id}", method="post") 230 | if r["success"] != 1: 231 | raise RequestFailed(r["status"]) 232 | 233 | @resetting.setter 234 | def resetting(self, value: bool) -> None: 235 | """ Reset a machine """ 236 | 237 | # Attempt the reset 238 | action = "/vm/reset" if value else "/machines/reset/cancel" 239 | r = self.connection._api(f"{action}/{self.id}", method="post") 240 | 241 | # Raise exception on failure 242 | if r["success"] != 1: 243 | raise RequestFailed(r["status"]) 244 | 245 | def submit(self, flag: str, difficulty: str = 50): 246 | """ Submit a flag for this machine """ 247 | 248 | r = self.connection._api( 249 | "/machines/own", 250 | method="post", 251 | json={"flag": flag, "difficulty": int(difficulty), "id": self.id}, 252 | ) 253 | 254 | if r["success"] == 0: 255 | raise RequestFailed(r["status"]) 256 | 257 | return True 258 | 259 | def extend(self) -> bool: 260 | """ Extend machine uptime """ 261 | 262 | # Machine isn't up 263 | if not self.spawned: 264 | return False 265 | 266 | # https://www.hackthebox.eu/api/vm/vip/extend/213 267 | r = self.connection._api(f"/vm/vip/extend/{self.id}", method="post") 268 | if r["success"] != 1: 269 | return RequestFailed(r["status"]) 270 | 271 | return True 272 | 273 | def review(self, stars: int, message: str) -> None: 274 | """ Submit a review for a machine """ 275 | 276 | r = self.connection._api( 277 | f"/machines/review", 278 | method="post", 279 | json={"stars": stars, "message": message}, 280 | ) 281 | 282 | def init(self, base_path="./") -> None: 283 | """ Initialize analysis directory and load an previous enumerations """ 284 | 285 | # Check if we already initialized the directory tree 286 | try: 287 | self.load(base_path) 288 | except NoAnalysisPath: 289 | # We didn't, pass to this function to do initialization 290 | pass 291 | else: 292 | # We did, our job is done 293 | return 294 | 295 | # Create analysis path and check if it's currently a file 296 | self.analysis_path = os.path.abspath( 297 | os.path.expanduser(os.path.join(base_path, self.name.lower())) 298 | ) 299 | 300 | # Create analysis structure 301 | os.makedirs(os.path.join(self.analysis_path, "scans"), exist_ok=True) 302 | os.makedirs(os.path.join(self.analysis_path, "artifacts"), exist_ok=True) 303 | os.makedirs(os.path.join(self.analysis_path, "exploits"), exist_ok=True) 304 | os.makedirs(os.path.join(self.analysis_path, "img"), exist_ok=True) 305 | 306 | # Create initial readme 307 | with open(os.path.join(self.analysis_path, "README.md"), "w") as f: 308 | f.write(f"# Hack the Box - {self.name} - {self.ip}\n") 309 | 310 | # Build hostname 311 | hostname = f"{self.name.lower()}.htb" 312 | 313 | # Check if we are already in /etc/hosts 314 | with open("/etc/hosts", "r") as f: 315 | in_hosts = any( 316 | [ 317 | re.fullmatch(f"^{self.ip}.*\\s+{self.hostname}.*$", line) 318 | is not None 319 | for line in f 320 | ] 321 | ) 322 | 323 | # Add our host to /etc/hosts if needed 324 | if not in_hosts: 325 | code = subprocess.run( 326 | ["sudo", "tee", "-a", "/etc/hosts"], 327 | input=bytes(f"\n{self.ip}\t{hostname}", "utf-8"), 328 | stdout=subprocess.DEVNULL, 329 | ) 330 | if code.returncode != 0: 331 | raise EtcHostsFailed 332 | 333 | def dump(self) -> bool: 334 | """ Dump our current findings and services to a state file in the 335 | anaylsis directory. If this machine has not been initialized, then don't 336 | do anything. """ 337 | 338 | if self.analysis_path is None: 339 | return False 340 | 341 | with open(os.path.join(self.analysis_path, "machine.json"), "w") as fh: 342 | json.dump( 343 | {"services": [s.json() for s in self.services], "knowns": self.knowns}, 344 | fh, 345 | ) 346 | 347 | return True 348 | 349 | def load(self, base_path: str = "./") -> None: 350 | """ Load saved machine information from `machine.json` in the analysis 351 | directory. """ 352 | 353 | # Ensure the directory exists 354 | analysis_path = os.path.expanduser( 355 | os.path.join(base_path, f"{self.name.lower()}") 356 | ) 357 | if not os.path.isdir(analysis_path): 358 | raise NoAnalysisPath 359 | 360 | try: 361 | with open(os.path.join(analysis_path, "machine.json"), "r") as fh: 362 | data = json.load(fh) 363 | self.services = [Service.from_json(s) for s in data["services"]] 364 | self.knowns = data["knowns"] 365 | self.analysis_path = analysis_path 366 | except OSError as e: 367 | # No machine.json file 368 | print(f"oserror: {e}") 369 | raise NoAnalysisPath 370 | except KeyError as e: 371 | print(f"keyerror: {e}") 372 | # Invalid machine json format 373 | raise NoAnalysisPath 374 | 375 | def enumerate(self, force: bool = False) -> None: 376 | """ Enumerate running services on the machine 377 | 378 | :param force: Force enumeration if it is already completed 379 | :type force: bool 380 | """ 381 | 382 | # The machine has to be running 383 | if not self.spawned: 384 | raise NotRunning 385 | 386 | # It also needs to not be actively terminating 387 | if self.terminating: 388 | raise Terminating 389 | 390 | # We already enumerated this machine 391 | if not force and len(self.services): 392 | return 393 | 394 | masscan_path = os.path.join(self.analysis_path, "scans", "masscan.grep") 395 | code = subprocess.call( 396 | [ 397 | "sudo", 398 | "masscan", 399 | self.ip, 400 | "-p", 401 | "1-65535", 402 | "--max-rate", 403 | "1000", 404 | "-oG", 405 | masscan_path, 406 | "-e", 407 | "tun0", 408 | ] 409 | ) 410 | 411 | # Ensure masscan succeeded 412 | if code != 0: 413 | raise MasscanFailed 414 | 415 | # Read all open port lines 416 | with open(masscan_path, "r") as f: 417 | ports = [ 418 | int(line.split(" ")[-1].split("/")[0]) 419 | for line in f.read().split("\n") 420 | if line != "" and line[0] != "#" and "open" in line 421 | ] 422 | 423 | # Run an in-depth nmap scan for the open ports 424 | nmap_path = os.path.join(self.analysis_path, "scans", "open-tcp") 425 | code = subprocess.call( 426 | [ 427 | "nmap", 428 | "-Pn", 429 | "-T5", 430 | "-sV", 431 | "-A", 432 | "-p", 433 | ",".join([str(p) for p in ports]), 434 | "-oA", 435 | nmap_path, 436 | self.hostname, 437 | ] 438 | ) 439 | 440 | # Check nmap result 441 | if code != 0: 442 | raise NmapFailed 443 | 444 | # Read greppable nmap output and extract open services 445 | with open(nmap_path + ".gnmap", "r") as f: 446 | services_list = [ 447 | line.split("Ports: ")[1] 448 | for line in f.read().split("\n") 449 | if line != "" and line[0] != "#" and "Ports:" in line 450 | ] 451 | 452 | self.services = [] 453 | for l in services_list: 454 | for s in l.split("/, "): 455 | self.services.append(Service.from_nmap((s + "/").strip())) 456 | 457 | # Ensure we write the services out 458 | self.dump() 459 | 460 | def scan(self, scanner: Scanner, service: Service, silent=False) -> Tracker: 461 | """ Start a scan for the given service. A tracker is allocated with the 462 | lock held and the `job_events` field set to None. """ 463 | 464 | if not scanner.match_service(service): 465 | raise NotApplicable 466 | 467 | # Construct a tracker object 468 | tracker = Tracker( 469 | silent=silent, 470 | machine=self, 471 | service=service, 472 | scanner=scanner, 473 | status="", 474 | events=None, 475 | thread=None, 476 | stop=False, 477 | data={}, 478 | lock=threading.Lock(), 479 | ) 480 | 481 | # Acquire the lock so the scanner doesn't modify the event queue before 482 | # initialization 483 | tracker.lock.acquire() 484 | 485 | # Start the background scan 486 | tracker.thread = scanner.background( 487 | tracker, self.analysis_path, self.hostname, self, service 488 | ) 489 | 490 | return tracker 491 | -------------------------------------------------------------------------------- /htb/notification.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | -------------------------------------------------------------------------------- /htb/scanner/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from htb.scanner.scanner import Service, Scanner, Tracker 3 | from htb.scanner.nikto import NiktoScanner 4 | from htb.scanner.enum4linux import Enum4LinuxScanner 5 | from htb.scanner.gobuster import GobusterScanner 6 | 7 | AVAILABLE_SCANNERS = [NiktoScanner(), Enum4LinuxScanner(), GobusterScanner()] 8 | -------------------------------------------------------------------------------- /htb/scanner/enum4linux.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import subprocess 3 | import shlex 4 | import time 5 | import sys 6 | import os 7 | 8 | # from htb.machine import Machine 9 | from htb.scanner.scanner import Scanner, Service, Tracker 10 | 11 | 12 | class Enum4LinuxScanner(Scanner): 13 | """ Scan a web server with nikto """ 14 | 15 | def __init__(self): 16 | super(Enum4LinuxScanner, self).__init__( 17 | name="enum4linux", 18 | ports=[445], 19 | regex=[r".*smb.*", r".*microsoft-ds.*"], 20 | protocol=["tcp"], 21 | ) 22 | 23 | def scan( 24 | self, 25 | tracker: Tracker, 26 | path: str, 27 | hostname: str, 28 | machine: "htb.machine.Machine", 29 | service: Service, 30 | ) -> None: 31 | """ Scan the host with nikto """ 32 | 33 | output_path = os.path.join(path, "scans", f"{self.ident(service)}.txt") 34 | 35 | # If we are backgrounded, ignore stdout and stderr 36 | output = open(output_path, "w") 37 | 38 | # Call enum4linux 39 | tracker.data["popen"] = subprocess.Popen( 40 | ["enum4linux", "-a", hostname], 41 | stdout=subprocess.PIPE, 42 | stderr=subprocess.DEVNULL, 43 | ) 44 | 45 | while tracker.data["popen"].poll() is None: 46 | line = tracker.data["popen"].stdout.readline() 47 | 48 | # Output if not silent 49 | if not tracker.silent: 50 | sys.stdout.write(line.decode("utf-8")) 51 | 52 | output.write(line.decode("utf-8")) 53 | 54 | # Set status 55 | if line.startswith(b"|"): 56 | yield line.split(b"|")[1].decode("utf-8").strip() 57 | 58 | time.sleep(0.1) 59 | 60 | output.close() 61 | 62 | yield "completed" 63 | 64 | def cancel(self, tracker: Tracker) -> None: 65 | """ Ensure the running process dies """ 66 | 67 | tracker.data["popen"].terminate() 68 | try: 69 | tracker.data["popen"].wait(timeout=1) 70 | except subprocess.TimeoutExpired: 71 | tracker.data["popen"].kill() 72 | tracker.data["popen"].wait() 73 | -------------------------------------------------------------------------------- /htb/scanner/gobuster.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Union 3 | import subprocess 4 | import signal 5 | import shlex 6 | import time 7 | import sys 8 | import os 9 | 10 | # from htb.machine import Machine 11 | from htb.scanner.scanner import ExternalScanner, Scanner, Service, Tracker 12 | from htb import util 13 | 14 | 15 | class GobusterScanner(ExternalScanner): 16 | """ Scan a web server with for directories/files with Gobuster """ 17 | 18 | LINE_DELIM = [b"\n", b"\r"] 19 | 20 | def __init__(self): 21 | super(GobusterScanner, self).__init__( 22 | name="gobuster", 23 | ports=[80, 443, 8080, 8443, 8888], 24 | regex=[r".*http.*", r".*web.*"], 25 | protocol=["tcp"], 26 | ) 27 | 28 | def scan( 29 | self, 30 | tracker: Tracker, 31 | path: str, 32 | hostname: str, 33 | machine: "htb.machine.Machine", 34 | service: Service, 35 | ) -> None: 36 | """ Scan the host with gobuster """ 37 | 38 | output_path = os.path.join(path, "scans", f"{self.ident(service)}.txt") 39 | 40 | wordlist = machine.connection.config.get( 41 | "gobuster", 42 | "wordlist", 43 | fallback="/usr/share/wordlists/dirbuster/directory-list-2.3-small.txt", 44 | ) 45 | url = f"{hostname}:{service.port}" 46 | 47 | yield from super(GobusterScanner, self).scan( 48 | tracker, 49 | path, 50 | hostname, 51 | machine, 52 | service, 53 | [ 54 | "gobuster", 55 | "dir", 56 | "-w", 57 | wordlist, 58 | "-f", 59 | "-k", 60 | "-o", 61 | output_path, 62 | "-u", 63 | url, 64 | ], 65 | ) 66 | 67 | def do_line( 68 | self, tracker: Tracker, scanner: Scanner, line: bytes 69 | ) -> Union[None, str]: 70 | if line.startswith(b"Progress:"): 71 | return line.split(b"Progress:")[1].decode("utf-8").strip() 72 | return None 73 | 74 | def cancel(self, tracker: Tracker) -> None: 75 | """ Ensure the running process dies """ 76 | 77 | tracker.data["popen"].terminate() 78 | try: 79 | tracker.data["popen"].wait(timeout=1) 80 | except subprocess.TimeoutExpired: 81 | tracker.data["popen"].kill() 82 | tracker.data["popen"].wait() 83 | -------------------------------------------------------------------------------- /htb/scanner/nikto.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Union 3 | import subprocess 4 | import shlex 5 | import time 6 | import sys 7 | import os 8 | 9 | # from htb.machine import Machine 10 | from htb.scanner.scanner import ExternalScanner, Service, Tracker, Scanner 11 | 12 | 13 | class NiktoScanner(ExternalScanner): 14 | """ Scan a web server with nikto """ 15 | 16 | def __init__(self): 17 | super(NiktoScanner, self).__init__( 18 | name="nikto", 19 | ports=[80, 443, 8080, 8443, 8000], 20 | regex=[r".*http.*", r".*web.*"], 21 | protocol=["tcp"], 22 | ) 23 | 24 | def scan( 25 | self, 26 | tracker: Tracker, 27 | path: str, 28 | hostname: str, 29 | machine: "htb.machine.Machine", 30 | service: Service, 31 | ) -> None: 32 | """ Scan the host with nikto """ 33 | 34 | output_path = os.path.join(path, "scans", f"{self.ident(service)}.txt") 35 | 36 | # If we are backgrounded, ignore stdout and stderr 37 | output = open(output_path, "w") 38 | 39 | url = f"http://{hostname}:{service.port}" 40 | 41 | return super(NiktoScanner, self).scan( 42 | tracker, 43 | path, 44 | hostname, 45 | machine, 46 | service, 47 | argv=["nikto", "-ask", "no", "-output", output_path, "-host", url], 48 | ) 49 | 50 | def do_line( 51 | self, tracker: Tracker, scanner: Scanner, line: bytes 52 | ) -> Union[None, str]: 53 | """ I have no useful progress info for Nikto :( """ 54 | return None 55 | -------------------------------------------------------------------------------- /htb/scanner/scanner.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import List, Dict, Any, Generator 3 | from dataclasses import dataclass 4 | import subprocess 5 | import threading 6 | import datetime 7 | import signal 8 | import queue 9 | import time 10 | import sys 11 | import re 12 | 13 | # from htb.machine import Machine 14 | 15 | 16 | class Service(object): 17 | """ Holds service information """ 18 | 19 | def __init__(self): 20 | """ Initialize blank service """ 21 | self.host: str = "" 22 | self.port: int = 0 23 | self.name: str = "blank" 24 | self.state: str = "closed" 25 | self.protocol: str = "none" 26 | 27 | @classmethod 28 | def from_masscan(cls, data: str): 29 | """ Build service from a line of greppable masscan results """ 30 | 31 | # Grab the last column 32 | service_data = data.split(" ")[-1].split("/") 33 | 34 | self = Service() 35 | self.port = int(service_data[0]) 36 | self.state = service_data[1] 37 | self.protocol = service_data[2] 38 | self.name = service_data[4] 39 | self.host = data.split("Host: ")[1].split(" ")[0] 40 | 41 | return self 42 | 43 | @classmethod 44 | def from_nmap(cls, data: str): 45 | """ Build service from a line of greppable masscan results """ 46 | 47 | # Grab the last column 48 | service_data = data.split("/") 49 | 50 | self = Service() 51 | self.port = int(service_data[0]) 52 | self.state = service_data[1] 53 | self.protocol = service_data[2] 54 | self.name = service_data[4] 55 | self.host = None # data.split("Host: ")[1].split(" ")[0] 56 | 57 | return self 58 | 59 | def json(self) -> Dict[str, Any]: 60 | """ Converts this object to a dictionary appropriate for JSON output """ 61 | return { 62 | "port": self.port, 63 | "protocol": self.protocol, 64 | "state": self.state, 65 | "name": self.name, 66 | } 67 | 68 | @classmethod 69 | def from_json(cls, data): 70 | self = Service() 71 | self.port = data["port"] 72 | self.protocol = data["protocol"] 73 | self.state = data["state"] 74 | self.name = data["name"] 75 | self.host = None 76 | return self 77 | 78 | 79 | @dataclass 80 | class Tracker(object): 81 | silent: bool 82 | machine: Any 83 | service: Service 84 | scanner: "Scanner" 85 | status: str 86 | events: queue.Queue 87 | thread: threading.Thread 88 | stop: bool 89 | data: Dict[str, Any] 90 | lock: threading.Lock = None 91 | 92 | 93 | class Scanner(object): 94 | """ Generic service/port scanner """ 95 | 96 | def __init__( 97 | self, 98 | name: str, 99 | ports: List[int], 100 | regex: List[str], 101 | protocol: List[str], 102 | recommended=False, 103 | ): 104 | super(Scanner, self).__init__() 105 | 106 | self.name: str = name 107 | self.ports: List[int] = ports 108 | self.regex: List[re.Pattern] = [ 109 | re.compile(p, re.IGNORECASE) for p in regex if isinstance(p, str) 110 | ] 111 | self.protocol: List[str] = protocol 112 | self.recommended: bool = recommended 113 | 114 | def ident(self, service) -> str: 115 | """ Get unique identifier for this service/scanner combo """ 116 | return f"{self.name}-{service.port}-{service.protocol}" 117 | 118 | def match(self, machine: "htb.machine.Machine") -> List[Service]: 119 | """ Match this scanner to a service. Returns true if it matches """ 120 | return [service for service in machine.services if self.match_service(service)] 121 | 122 | def match_service(self, service: Service) -> bool: 123 | return service.protocol in self.protocol and ( 124 | service.port in self.ports 125 | or any([r.match(service.name) for r in self.regex]) 126 | ) 127 | 128 | def background( 129 | self, tracker: Tracker, path: str, hostname: str, machine, service: Service, 130 | ) -> threading.Thread: 131 | """ Start the scan in the background """ 132 | 133 | # Ensure we run silently 134 | # tracker.silent = True 135 | 136 | # Create and start the thread 137 | thread = threading.Thread( 138 | target=self._do_background_scan, 139 | args=(tracker, path, hostname, machine, service), 140 | ) 141 | thread.start() 142 | 143 | # Return thread handle 144 | return thread 145 | 146 | def continue_background( 147 | self, tracker: Tracker, generator: Generator[str, None, None] 148 | ): 149 | """ Transfer control of a running scan to a background task """ 150 | 151 | tracker.silent = True 152 | 153 | thread = threading.Thread( 154 | target=self._do_continue_background, args=(tracker, generator) 155 | ) 156 | thread.start() 157 | 158 | return thread 159 | 160 | def _do_continue_background( 161 | self, tracker: Tracker, generator: Generator[str, None, None] 162 | ): 163 | """ Continue the scan in the background """ 164 | 165 | # This ensures the main thread doesn't trample us 166 | tracker.lock.acquire() 167 | 168 | for status in generator: 169 | tracker.status = status 170 | if tracker.stop: 171 | self.cancel(tracker) 172 | 173 | tracker.events.put(tracker) 174 | 175 | def _do_background_scan( 176 | self, tracker: Tracker, path: str, hostname: str, machine, service: Service, 177 | ) -> None: 178 | """ Start the scan in the background and notify the queue when it is complete """ 179 | for status in self.scan(tracker, path, hostname, machine, service): 180 | # Set status 181 | tracker.status = status 182 | 183 | # The job was killed 184 | if tracker.stop: 185 | # Perform shutdown needed 186 | self.cancel(tracker) 187 | break 188 | 189 | with tracker.lock: 190 | tracker.events.put(tracker) 191 | 192 | def cancel(self, tracker: Tracker) -> None: 193 | """ Shutdown any recurring things (like killing processes) """ 194 | return 195 | 196 | def scan( 197 | self, tracker: Tracker, path: str, hostname: str, machine, service: Service, 198 | ) -> None: 199 | """ Scan the service on this host """ 200 | yield "running" 201 | 202 | 203 | class ExternalScanner(Scanner): 204 | 205 | LINE_DELIM = [b"\n"] 206 | 207 | def __init__(self, *args, **kwargs): 208 | super(ExternalScanner, self).__init__(*args, **kwargs) 209 | 210 | def scan( 211 | self, 212 | tracker: Tracker, 213 | path: str, 214 | hostname: str, 215 | machine: "htb.machine.Machine", 216 | service: Service, 217 | argv: List[str], 218 | ): 219 | """ Start the external application (specified by argv) and monitor output """ 220 | 221 | # Call gobuster 222 | tracker.data["popen"] = subprocess.Popen( 223 | argv, 224 | stdout=subprocess.PIPE, 225 | stderr=subprocess.STDOUT, 226 | preexec_fn=lambda: signal.signal(signal.SIGTSTP, signal.SIG_IGN), 227 | ) 228 | 229 | line = b"" 230 | 231 | # Track start time 232 | start_time = time.time() 233 | 234 | while tracker.data["popen"].poll() is None: 235 | 236 | # Grab next byte 237 | data = tracker.data["popen"].stdout.read(1) 238 | 239 | # Not silent, output 240 | if not tracker.silent: 241 | sys.stdout.write(data.decode("utf-8")) 242 | 243 | if data in self.LINE_DELIM: 244 | 245 | # Set status 246 | status = self.do_line(tracker, service, line) 247 | if status is not None: 248 | yield status 249 | 250 | line = b"" 251 | 252 | # We don't want a busy loop. Sleep after every line 253 | if tracker.silent: 254 | time.sleep(0.1) 255 | else: 256 | line += data 257 | 258 | yield f"completed in {datetime.timedelta(seconds=time.time()-start_time)}" 259 | 260 | def do_line(self, tracker: Tracker, service: Service, line: bytes): 261 | """ Process a line of output from the subprocess """ 262 | 263 | pass 264 | 265 | def cancel(self, tracker: Tracker) -> None: 266 | """ Ensure the running process dies """ 267 | 268 | tracker.data["popen"].terminate() 269 | try: 270 | tracker.data["popen"].wait(timeout=1) 271 | except subprocess.TimeoutExpired: 272 | tracker.data["popen"].kill() 273 | tracker.data["popen"].wait() 274 | -------------------------------------------------------------------------------- /htb/util.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import List, Dict 3 | from colorama import Style, Fore, Back 4 | from cmd2.ansi import strip_style 5 | 6 | 7 | def readuntil(f, delim: List[bytes]): 8 | result = [] 9 | while True: 10 | b = f.read(1) 11 | if b == b"" or b in delim: 12 | break 13 | if b in delim: 14 | break 15 | return b"".join(result) 16 | 17 | 18 | def build_table(data: List[List[str]], highlight=True) -> List[str]: 19 | """ Build an ASCII table for the terminal. Each item in headers and data can 20 | can start with "<", ">", or "^" to control justification. Column justification 21 | propogates to all cells in the column unless overridden. If highlight is true, 22 | column headers will use the `Style.BRIGHT` colorama style. """ 23 | 24 | # Find number of rows and columns 25 | rows = len(data) 26 | columns = len(data[0]) 27 | 28 | # Find widths of columns 29 | if columns > 1: 30 | padding = [1] + [2] * (columns - 1) + [1] 31 | else: 32 | padding = [1] 33 | 34 | width = [ 35 | max([len(strip_style(data[r][c])) for r in range(rows)]) for c in range(columns) 36 | ] 37 | column_justify = [] 38 | 39 | # Find column justification 40 | for c in range(columns): 41 | if len(data[0][c]) == 0 or data[0][c][0] not in "<>^": 42 | column_justify.append("<") 43 | else: 44 | column_justify.append(data[0][c][0]) 45 | data[0][c] = data[0][c][1:] 46 | 47 | # Initialize output 48 | output = [] 49 | 50 | # Build table 51 | for r in range(rows): 52 | row = [] 53 | for c in range(columns): 54 | # Find correct justification 55 | if len(data[r][c]) > 0 and data[r][c][0] in "<>^": 56 | justify = data[r][c][0] 57 | data[r][c] = data[r][c][1:] 58 | else: 59 | justify = column_justify[c] 60 | 61 | # Highlight the headers if requested 62 | if highlight and r == 0: 63 | style = Style.BRIGHT 64 | else: 65 | style = "" 66 | 67 | w = width[c] 68 | placeholder = "A" * len(strip_style(data[r][c])) 69 | 70 | # Justify fake input to avoid issues with formatting 71 | row.append(f"{placeholder:{justify}{w}}") 72 | # Insert correct input after justification 73 | row[-1] = style + row[-1].replace(placeholder, data[r][c]) 74 | 75 | if highlight and r == 0: 76 | row[-1] += Style.RESET_ALL 77 | 78 | # Build this row 79 | output.append(" ".join(row)) 80 | 81 | return output 82 | -------------------------------------------------------------------------------- /htb/vpn.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | from typing import Any, Dict, List 3 | 4 | from htb.exceptions import RequestFailed 5 | 6 | 7 | class VPN(object): 8 | """ Represents the VPN server you're currently attached to """ 9 | 10 | US_FREE = "usfree" 11 | US_VIP = "usvip" 12 | EU_FREE = "eufree" 13 | EU_VIP = "euvip" 14 | AU_FREE = "aufree" 15 | VALID_LABS = [US_FREE, US_VIP, EU_FREE, EU_VIP, AU_FREE] 16 | 17 | def __init__(self, connection: Any, data: Dict[str, Any]): 18 | """ Create a new VPN object from status information """ 19 | 20 | # Certain data is only available when connected 21 | if data["success"] == 0: 22 | self.ipv4: str = None 23 | self.ipv6: str = None 24 | self.rate_up: float = 0 25 | self.rate_down: float = 0 26 | self.user: str = None 27 | self.active: bool = False 28 | else: 29 | self.active: bool = True 30 | self.user: str = data["connection"]["name"] 31 | self.ipv4: str = data["connection"]["ip4"] 32 | self.ipv6: str = data["connection"]["ip6"] 33 | self.rate_up: float = data["connection"]["up"] 34 | self.rate_down: float = data["connection"]["down"] 35 | 36 | # Server information is always available 37 | self.hostname = data["server"]["serverHostname"] 38 | self.port = data["server"]["serverPort"] 39 | self.connection = connection 40 | 41 | def switch(self, lab: str) -> None: 42 | """ Switch to a different lab. **NOTE** regenerates keys! """ 43 | 44 | if "-fort-" in self.hostname: 45 | raise ValueError("Fortress labs cannot be switched") 46 | 47 | if lab not in VPN.VALID_LABS: 48 | raise ValueError(f"unknown lab name: {lab}") 49 | 50 | r = self.connection._api(f"/labs/switch/{lab}", method="post") 51 | if int(r["status"]) != 1: 52 | raise RequestFailed(r["error"]) 53 | 54 | @property 55 | def name(self) -> str: 56 | name_map = { 57 | "us-free": VPN.US_FREE, 58 | "us-vip": VPN.US_VIP, 59 | "eu-free": VPN.EU_FREE, 60 | "eu-vip": VPN.EU_VIP, 61 | "au-free": VPN.AU_FREE, 62 | "-fort-": "fortress", 63 | } 64 | 65 | for piece in name_map: 66 | if piece in self.hostname: 67 | return name_map[piece] 68 | 69 | @property 70 | def config(self) -> bytes: 71 | """ Get the ovpn configuration. This is only possible if you provided 72 | Hack the Box username and password. """ 73 | 74 | r = self.connection._request(f"/home/htb/access/ovpnfile", method="get") 75 | if r.status_code != 200: 76 | raise RequestFailed("unknown error") 77 | 78 | return bytes(r.text, "utf-8") 79 | 80 | def __repr__(self) -> str: 81 | if self.active: 82 | return f"""""" 83 | else: 84 | return f"""""" 85 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from setuptools import find_packages 4 | from setuptools import setup 5 | 6 | dependencies = [ 7 | "wheel", 8 | "requests", 9 | "argparse", 10 | "cmd2", 11 | "pygments", 12 | "regex", 13 | "dbus-python", 14 | ] 15 | 16 | dependency_links = [ 17 | "https://github.com/calebstewart/python-networkmanager/tarball/master#egg=python-networkmanager" 18 | ] 19 | 20 | # Setup 21 | setup( 22 | name="htb", 23 | version="0.1", 24 | description="Hack the Box Platform API", 25 | author="Caleb Stewart", 26 | url="https://github.com/calebstewart/python-htb", 27 | packages=find_packages(), 28 | package_data={"htb": []}, 29 | entry_points={"console_scripts": ["htb=htb.__main__:main"]}, 30 | install_requires=dependencies, 31 | dependency_links=dependency_links, 32 | ) 33 | --------------------------------------------------------------------------------