├── .gitignore ├── .travis.yml ├── AUTHORS ├── CHANGELOG.rst ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.rst ├── ISSUE_TEMPLATE.md ├── LICENSE ├── MANIFEST.in ├── PULL_REQUEST_TEMPLATE.md ├── README.md ├── TODO.md ├── docs ├── CHANGELOG.rst ├── CONTRIBUTING.rst ├── LICENSE.rst ├── README.rst ├── conf.py ├── html │ └── index.html └── index.rst ├── img └── pdlist_demo.gif ├── pdlist ├── __init__.py ├── __main__.py ├── main.py ├── source │ ├── __init__.py │ ├── certspotter.py │ ├── crtsh.py │ ├── dnsdumpster.py │ ├── hackertarget.py │ ├── threatcrowd.py │ └── urlscan.py └── utils.py ├── release ├── requirements.txt ├── setup.cfg ├── setup.py └── tests ├── __init__.py ├── test_procedure_1.py └── test_sanity.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | __pycache__ 3 | 4 | 5 | list.txt 6 | 7 | # Virtual Environment 8 | env/ 9 | venv/ 10 | 11 | # C extensions 12 | *.so 13 | 14 | # Vim 15 | [._]*.s[a-w][a-z] 16 | [._]s[a-w][a-z] 17 | *.un~ 18 | Session.vim 19 | 20 | # Packages 21 | *.egg 22 | *.egg-info 23 | dist 24 | build 25 | eggs 26 | parts 27 | var 28 | sdist 29 | develop-eggs 30 | .installed.cfg 31 | lib 32 | lib64 33 | 34 | # Installer logs 35 | pip-log.txt 36 | 37 | # Unit test / coverage reports 38 | .coverage 39 | .tox 40 | nosetests.xml 41 | 42 | # Mr Developer 43 | .mr.developer.cfg 44 | .project 45 | .pydevproject 46 | 47 | # GitHub token file 48 | .pypt/gh-token 49 | 50 | #  51 | .DS_Store 52 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | sudo: false 3 | python: 4 | - "3.4" 5 | - "3.5" 6 | - "3.6" 7 | - "3.7" 8 | - "pypy" 9 | 10 | install: 11 | - "pip install -r requirements.txt" 12 | - "pip install ." 13 | 14 | script: 15 | - "py.test --cov pdlist --cov-report term-missing tests/" 16 | 17 | notifications: 18 | email: 19 | on_success: change 20 | on_failure: always 21 | -------------------------------------------------------------------------------- /AUTHORS: -------------------------------------------------------------------------------- 1 | gnc 2 | -------------------------------------------------------------------------------- /CHANGELOG.rst: -------------------------------------------------------------------------------- 1 | ===================== 2 | Appendix C. Changelog 3 | ===================== 4 | :Info: This is the changelog for pdlist. 5 | :Author: gnc 6 | :Copyright: © 2019, gnc. 7 | :License: BSD (see /LICENSE or :doc:`Appendix B `.) 8 | :Date: 2019-07-25 9 | :Version: 0.1.0 10 | 11 | .. index:: CHANGELOG 12 | 13 | GitHub holds releases, too 14 | ========================== 15 | 16 | More information can be found on GitHub in the `releases section 17 | `_. 18 | 19 | Version History 20 | =============== 21 | 22 | 0.1.0 23 | Initial release. 24 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at nebbionegiuseppe@gmail.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============================== 2 | Appendix A. Contribution rules 3 | ============================== 4 | :Info: Those are the contribution rules for pdlist. 5 | :Copyright: © 2012-2018, Chris Warrick. 6 | :License: 3-clause BSD 7 | 8 | .. index:: contributing 9 | 10 | Do you want to contribute to this project? Great! I’d love to see some help, 11 | but you must comply with some rules. 12 | 13 | The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL 14 | NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and 15 | “OPTIONAL” in this document are to be interpreted as described in 16 | RFC 2119. 17 | 18 | --------------- 19 | Issue reporting 20 | --------------- 21 | 22 | .. index:: issues 23 | 24 | GitHub Issues are the recommended way to report an issue. If you do not have an 25 | account there, get one or mail me. 26 | 27 | When pasting console sessions, you must paste them fully, *prompt-to-prompt*, 28 | to see all the messages and your input. Trim only stuff that you are 1000% 29 | sure that is not related to the project in question. 30 | 31 | -------------------------------------------- 32 | General preparations, rules and pull process 33 | -------------------------------------------- 34 | 35 | Prepare 36 | ======= 37 | 38 | A GitHub account is recommended. Patches by mail are accepted, but I’d prefer 39 | to work via GitHub. 40 | 41 | .. _Rules: 42 | 43 | Rules 44 | ===== 45 | 46 | 1. Commits must have short, informative and logical messages. Signoffs and 47 | long messages are recommended. “Fix #xxx” is required if an issue 48 | exists. 49 | 2. The following fancy Unicode characters should be used when 50 | needed: ``— “ ” ‘ ’``. ``…`` should not appear in console output, but may 51 | appear elsewhere. 52 | 3. For Python code, use the PEP 8 coding style and PEP 257 documentation style. 53 | For other languages, K&R style applies. Braces are mandatory in all blocks 54 | (even one-line blocks). Braces are on the same lines as class names and 55 | function signatures. Use 4-space indents. 56 | 57 | Request a Pull 58 | ============== 59 | 60 | Done? Go hit the **Pull Request** button over on GitHub! And if you don’t 61 | use GitHub, ``git format-patch``. Other formats are not accepted. 62 | 63 | Your commit should be pulled up in a (longer) while. If I like it. Because 64 | some commits may be bad. So, do your best not to do those bad commits. 65 | -------------------------------------------------------------------------------- /ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | ## Expected Behavior 2 | 3 | 4 | ## Actual Behavior 5 | 6 | 7 | ## Steps to Reproduce the Problem 8 | 9 | 1. 10 | 1. 11 | 1. 12 | 13 | ## Specifications 14 | 15 | - Version: 16 | - Platform: 17 | - Subsystem: 18 | 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright © 2019, gnc. 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are 6 | met: 7 | 8 | 1. Redistributions of source code must retain the above copyright 9 | notice, this list of conditions, and the following disclaimer. 10 | 11 | 2. Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions, and the following disclaimer in the 13 | documentation and/or other materials provided with the distribution. 14 | 15 | 3. Neither the name of the author of this software nor the names of 16 | contributors to this software may be used to endorse or promote 17 | products derived from this software without specific prior written 18 | consent. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 21 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 22 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 23 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 24 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 25 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 26 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 27 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 28 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 30 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 31 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | graft pdlist 2 | graft docs 3 | graft tests 4 | include README.rst AUTHORS LICENSE CHANGELOG.rst setup.py setup.cfg requirements.txt 5 | global-exclude __pycache__ *.pyc 6 | -------------------------------------------------------------------------------- /PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | What does this implement/fix? Explain your changes. 2 | --------------------------------------------------- 3 | ... 4 | 5 | Does this close any currently open issues? 6 | ------------------------------------------ 7 | ... 8 | 9 | 10 | Any relevant logs, error output, etc? 11 | ------------------------------------- 12 | (If it’s long, please paste to https://ghostbin.com/ or https://bpaste.net and insert the link here.) 13 | 14 | Any other comments? 15 | ------------------- 16 | ... 17 | 18 | Where has this been tested? 19 | --------------------------- 20 | 21 | **Operating System:** ... 22 | 23 | **Platform:** ... 24 | 25 | **Target Platform:** ... 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # pdlist. A passive subdomain finder 2 | 3 | 4 | Author: gnc 5 | 6 | Copyright: © 2019, gnc. 7 | 8 | Date: 2019-07-25 9 | 10 | Version: 0.1.0 11 | 12 | 13 | ## PURPOSE 14 | 15 | pdlist is a passive subdomain finder written in python3. This tool can be used 16 | effectively to collect information about a domain without ever sending a 17 | single packet to any of its hosts. 18 | Given a domain like "example.com" it will find all the hosts which 19 | have a hostname `.example.com` or URLs strictly related to example.com. 20 | 21 | In order to collect informations about subdomains the software queries different 22 | publicly available websites, which at the moment are: 23 | - [threatcrowd](https://www.threatcrowd.org/) 24 | - [urlscan](https://urlscan.io/) 25 | - [hackertarget](https://hackertarget.com/) 26 | - [dnsdumpster](https://dnsdumpster.com/) 27 | - [crt.sh](https://crt.sh/) 28 | - [certspotter](https://certspotter.com) 29 | 30 | 31 | pdlist is very user-friendly and lightweight since the only dependencies are 32 | the following python modules: 33 | - requests 34 | - BeautifulSoup4 35 | 36 | 37 | 38 | ## INSTALLATION 39 | 40 | We can install pdlist simply by doing: 41 | ```sh 42 | git clone https://github.com/gnebbia/pdlist 43 | cd pdlist 44 | pip install -r requirements.txt 45 | python setup.py install 46 | ``` 47 | 48 | Notice that both pip and python should refer to version 3, so if you are not 49 | using pyenv as I am doing you should probably substitute `pip` with `pip3` and 50 | `python` with `python3`. 51 | 52 | 53 | ## USAGE 54 | 55 | To have a list of subdomains passively of for example 56 | [example.com](https://example.com/) we can do: 57 | 58 | ```sh 59 | pdlist example.com 60 | ``` 61 | 62 | we can also specify multiple domains, e.g.,; 63 | 64 | ```sh 65 | pdlist example1.com example2.com 66 | ``` 67 | 68 | We can save the output in a text file by doing: 69 | ```sh 70 | pdlist example.com -o example-list.txt 71 | ``` 72 | 73 | Notice that by default pdlist will also output hostnames which may not really 74 | be proper subdomains of the specified domains, and this happens because those 75 | subdomains are still related to the specified domains. 76 | 77 | If we want to only output proper subdomains we can enable the strict mode by 78 | doing: 79 | ```shs 80 | pdlist example.com --strict 81 | ``` 82 | 83 | A usage example in the gif below: 84 | ![](img/pdlist_demo.gif) 85 | 86 | 87 | 88 | ## NOTES 89 | 90 | This is a minimalist passive domain finder, the aim of this project is to have 91 | few dependencies, small code footprint and easily extensible. 92 | 93 | If you want to extend the code it is enough to add a module in the `source` 94 | package with a `def parse(domains)` method. 95 | 96 | 97 | 98 | ## TODO 99 | 100 | * Add more passive recon sources 101 | * Modify the code to work in asynchrounous mode 102 | * Generate fancy html reports 103 | 104 | ## COPYRIGHT 105 | 106 | Copyright © 2019, gnc. 107 | All rights reserved. 108 | 109 | Redistribution and use in source and binary forms, with or without 110 | modification, are permitted provided that the following conditions are 111 | met: 112 | 113 | 1. Redistributions of source code must retain the above copyright 114 | notice, this list of conditions, and the following disclaimer. 115 | 116 | 2. Redistributions in binary form must reproduce the above copyright 117 | notice, this list of conditions, and the following disclaimer in the 118 | documentation and/or other materials provided with the distribution. 119 | 120 | 3. Neither the name of the author of this software nor the names of 121 | contributors to this software may be used to endorse or promote 122 | products derived from this software without specific prior written 123 | consent. 124 | 125 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 126 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 127 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 128 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 129 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 130 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 131 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 132 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 133 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 134 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 135 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 136 | -------------------------------------------------------------------------------- /TODO.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnebbia/pdlist/90e5e9ca16182b9554d010e6f6dabd43238287fe/TODO.md -------------------------------------------------------------------------------- /docs/CHANGELOG.rst: -------------------------------------------------------------------------------- 1 | ===================== 2 | Appendix C. Changelog 3 | ===================== 4 | :Info: This is the changelog for pdlist. 5 | :Author: gnc 6 | :Copyright: © 2019, gnc. 7 | :License: BSD (see /LICENSE or :doc:`Appendix B `.) 8 | :Date: 2019-07-25 9 | :Version: 0.1.0 10 | 11 | .. index:: CHANGELOG 12 | 13 | GitHub holds releases, too 14 | ========================== 15 | 16 | More information can be found on GitHub in the `releases section 17 | `_. 18 | 19 | Version History 20 | =============== 21 | 22 | 0.1.0 23 | Initial release. 24 | -------------------------------------------------------------------------------- /docs/CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============================== 2 | Appendix A. Contribution rules 3 | ============================== 4 | :Info: Those are the contribution rules for pdlist. 5 | :Copyright: © 2012-2018, Chris Warrick. 6 | :License: 3-clause BSD 7 | 8 | .. index:: contributing 9 | 10 | Do you want to contribute to this project? Great! I’d love to see some help, 11 | but you must comply with some rules. 12 | 13 | The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL 14 | NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and 15 | “OPTIONAL” in this document are to be interpreted as described in 16 | RFC 2119. 17 | 18 | --------------- 19 | Issue reporting 20 | --------------- 21 | 22 | .. index:: issues 23 | 24 | GitHub Issues are the recommended way to report an issue. If you do not have an 25 | account there, get one or mail me. 26 | 27 | When pasting console sessions, you must paste them fully, *prompt-to-prompt*, 28 | to see all the messages and your input. Trim only stuff that you are 1000% 29 | sure that is not related to the project in question. 30 | 31 | -------------------------------------------- 32 | General preparations, rules and pull process 33 | -------------------------------------------- 34 | 35 | Prepare 36 | ======= 37 | 38 | A GitHub account is recommended. Patches by mail are accepted, but I’d prefer 39 | to work via GitHub. 40 | 41 | .. _Rules: 42 | 43 | Rules 44 | ===== 45 | 46 | 1. Commits must have short, informative and logical messages. Signoffs and 47 | long messages are recommended. “Fix #xxx” is required if an issue 48 | exists. 49 | 2. The following fancy Unicode characters should be used when 50 | needed: ``— “ ” ‘ ’``. ``…`` should not appear in console output, but may 51 | appear elsewhere. 52 | 3. For Python code, use the PEP 8 coding style and PEP 257 documentation style. 53 | For other languages, K&R style applies. Braces are mandatory in all blocks 54 | (even one-line blocks). Braces are on the same lines as class names and 55 | function signatures. Use 4-space indents. 56 | 57 | Request a Pull 58 | ============== 59 | 60 | Done? Go hit the **Pull Request** button over on GitHub! And if you don’t 61 | use GitHub, ``git format-patch``. Other formats are not accepted. 62 | 63 | Your commit should be pulled up in a (longer) while. If I like it. Because 64 | some commits may be bad. So, do your best not to do those bad commits. 65 | -------------------------------------------------------------------------------- /docs/LICENSE.rst: -------------------------------------------------------------------------------- 1 | ======================================================= 2 | Appendix B. License for pdlist 3 | ======================================================= 4 | :Info: This is the license for pdlist. 5 | :Author: gnc 6 | :Copyright: © 2019, gnc. 7 | :License: BSD (see /LICENSE or :doc:`Appendix B `.) 8 | :Date: 2019-07-25 9 | :Version: 0.1.0 10 | 11 | .. index:: LICENSE 12 | 13 | Copyright © 2019, gnc. 14 | All rights reserved. 15 | 16 | Redistribution and use in source and binary forms, with or without 17 | modification, are permitted provided that the following conditions are 18 | met: 19 | 20 | 1. Redistributions of source code must retain the above copyright 21 | notice, this list of conditions, and the following disclaimer. 22 | 23 | 2. Redistributions in binary form must reproduce the above copyright 24 | notice, this list of conditions, and the following disclaimer in the 25 | documentation and/or other materials provided with the distribution. 26 | 27 | 3. Neither the name of the author of this software nor the names of 28 | contributors to this software may be used to endorse or promote 29 | products derived from this software without specific prior written 30 | consent. 31 | 32 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 33 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 34 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 35 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 36 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 37 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 38 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 39 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 40 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 41 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 42 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 43 | -------------------------------------------------------------------------------- /docs/README.rst: -------------------------------------------------------------------------------- 1 | ============================================================================== 2 | pdlist. A passive subdomain finder 3 | ============================================================================== 4 | :Info: This is the README file for pdlist. 5 | :Author: gnc 6 | :Copyright: © 2019, gnc. 7 | :Date: 2019-07-25 8 | :Version: 0.1.0 9 | 10 | .. index: README 11 | .. image:: https://travis-ci.org/gnebbia/pdlist.svg?branch=master 12 | :target: https://travis-ci.org/gnebbia/pdlist 13 | 14 | PURPOSE 15 | ------- 16 | 17 | INSTALLATION 18 | ------------ 19 | 20 | USAGE 21 | ----- 22 | 23 | NOTES 24 | ----- 25 | 26 | COPYRIGHT 27 | --------- 28 | Copyright © 2019, gnc. 29 | All rights reserved. 30 | 31 | Redistribution and use in source and binary forms, with or without 32 | modification, are permitted provided that the following conditions are 33 | met: 34 | 35 | 1. Redistributions of source code must retain the above copyright 36 | notice, this list of conditions, and the following disclaimer. 37 | 38 | 2. Redistributions in binary form must reproduce the above copyright 39 | notice, this list of conditions, and the following disclaimer in the 40 | documentation and/or other materials provided with the distribution. 41 | 42 | 3. Neither the name of the author of this software nor the names of 43 | contributors to this software may be used to endorse or promote 44 | products derived from this software without specific prior written 45 | consent. 46 | 47 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 48 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 49 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 50 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 51 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 52 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 53 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 54 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 55 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 56 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 57 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 58 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # pdlist documentation build configuration file, created by 4 | # sphinx-quickstart on Fri Dec 14 21:02:58 2012. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | #sys.path.insert(0, os.path.abspath('.')) 20 | 21 | # -- General configuration ----------------------------------------------------- 22 | 23 | # If your documentation needs a minimal Sphinx version, state it here. 24 | #needs_sphinx = '1.0' 25 | 26 | # Add any Sphinx extension module names here, as strings. They can be extensions 27 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 28 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode'] 29 | 30 | # Add any paths that contain templates here, relative to this directory. 31 | templates_path = ['_templates'] 32 | 33 | # The suffix of source filenames. 34 | source_suffix = '.rst' 35 | 36 | # The encoding of source files. 37 | #source_encoding = 'utf-8-sig' 38 | 39 | # The master toctree document. 40 | master_doc = 'index' 41 | 42 | # General information about the project. 43 | project = u'pdlist' 44 | copyright = u'2019, gnc' 45 | 46 | # The version info for the project you're documenting, acts as replacement for 47 | # |version| and |release|, also used in various other places throughout the 48 | # built documents. 49 | # 50 | # The short X.Y version. 51 | version = '0.1.0' 52 | # The full version, including alpha/beta/rc tags. 53 | release = '0.1.0' 54 | 55 | # The language for content autogenerated by Sphinx. Refer to documentation 56 | # for a list of supported languages. 57 | #language = None 58 | 59 | # There are two options for replacing |today|: either, you set today to some 60 | # non-false value, then it is used: 61 | #today = '' 62 | # Else, today_fmt is used as the format for a strftime call. 63 | #today_fmt = '%B %d, %Y' 64 | 65 | # List of patterns, relative to source directory, that match files and 66 | # directories to ignore when looking for source files. 67 | exclude_patterns = ['_build'] 68 | 69 | # The reST default role (used for this markup: `text`) to use for all documents. 70 | #default_role = None 71 | 72 | # If true, '()' will be appended to :func: etc. cross-reference text. 73 | #add_function_parentheses = True 74 | 75 | # If true, the current module name will be prepended to all description 76 | # unit titles (such as .. function::). 77 | #add_module_names = True 78 | 79 | # If true, sectionauthor and moduleauthor directives will be shown in the 80 | # output. They are ignored by default. 81 | #show_authors = False 82 | 83 | # The name of the Pygments (syntax highlighting) style to use. 84 | pygments_style = 'sphinx' 85 | 86 | # A list of ignored prefixes for module index sorting. 87 | #modindex_common_prefix = [] 88 | 89 | 90 | # -- Options for HTML output --------------------------------------------------- 91 | 92 | # The theme to use for HTML and HTML Help pages. See the documentation for 93 | # a list of builtin themes. 94 | html_theme = 'default' 95 | 96 | # Theme options are theme-specific and customize the look and feel of a theme 97 | # further. For a list of options available for each theme, see the 98 | # documentation. 99 | #html_theme_options = {} 100 | 101 | # Add any paths that contain custom themes here, relative to this directory. 102 | #html_theme_path = [] 103 | 104 | # The name for this set of Sphinx documents. If None, it defaults to 105 | # " v documentation". 106 | #html_title = None 107 | 108 | # A shorter title for the navigation bar. Default is the same as html_title. 109 | #html_short_title = None 110 | 111 | # The name of an image file (relative to this directory) to place at the top 112 | # of the sidebar. 113 | #html_logo = None 114 | 115 | # The name of an image file (within the static path) to use as favicon of the 116 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 117 | # pixels large. 118 | #html_favicon = None 119 | 120 | # Add any paths that contain custom static files (such as style sheets) here, 121 | # relative to this directory. They are copied after the builtin static files, 122 | # so a file named "default.css" will overwrite the builtin "default.css". 123 | html_static_path = ['_static'] 124 | 125 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 126 | # using the given strftime format. 127 | #html_last_updated_fmt = '%b %d, %Y' 128 | 129 | # If true, SmartyPants will be used to convert quotes and dashes to 130 | # typographically correct entities. 131 | #html_use_smartypants = True 132 | 133 | # Custom sidebar templates, maps document names to template names. 134 | #html_sidebars = {} 135 | 136 | # Additional templates that should be rendered to pages, maps page names to 137 | # template names. 138 | #html_additional_pages = {} 139 | 140 | # If false, no module index is generated. 141 | #html_domain_indices = True 142 | 143 | # If false, no index is generated. 144 | #html_use_index = True 145 | 146 | # If true, the index is split into individual pages for each letter. 147 | #html_split_index = False 148 | 149 | # If true, links to the reST sources are added to the pages. 150 | #html_show_sourcelink = True 151 | 152 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 153 | #html_show_sphinx = True 154 | 155 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 156 | #html_show_copyright = True 157 | 158 | # If true, an OpenSearch description file will be output, and all pages will 159 | # contain a tag referring to it. The value of this option must be the 160 | # base URL from which the finished HTML is served. 161 | #html_use_opensearch = '' 162 | 163 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 164 | #html_file_suffix = None 165 | 166 | # Output file base name for HTML help builder. 167 | htmlhelp_basename = 'pdlistdoc' 168 | 169 | 170 | # -- Options for LaTeX output -------------------------------------------------- 171 | 172 | # The paper size ('letter' or 'a4'). 173 | #latex_paper_size = 'letter' 174 | 175 | # The font size ('10pt', '11pt' or '12pt'). 176 | #latex_font_size = '10pt' 177 | 178 | # Grouping the document tree into LaTeX files. List of tuples 179 | # (source start file, target name, title, author, documentclass [howto/manual]). 180 | latex_documents = [ 181 | ('index', 'pdlist.tex', u'pdlist Documentation', 182 | u'gnc', 'manual'), 183 | ] 184 | 185 | latex_elements = {'papersize': 'a4paper', 'fontpkg': '\\usepackage{tgheros}', 186 | 'fncychap': '\\usepackage[Sonny]{fncychap}'} 187 | 188 | # The name of an image file (relative to this directory) to place at the top of 189 | # the title page. 190 | #latex_logo = None 191 | 192 | # For "manual" documents, if this is true, then toplevel headings are parts, 193 | # not chapters. 194 | #latex_use_parts = False 195 | 196 | # If true, show page references after internal links. 197 | #latex_show_pagerefs = False 198 | 199 | # If true, show URL addresses after external links. 200 | #latex_show_urls = False 201 | 202 | # Additional stuff for the LaTeX preamble. 203 | #latex_preamble = '' 204 | 205 | # Documents to append as an appendix to all manuals. 206 | #latex_appendices = [] 207 | 208 | # If false, no module index is generated. 209 | #latex_domain_indices = True 210 | 211 | 212 | # -- Options for manual page output -------------------------------------------- 213 | 214 | # One entry per manual page. List of tuples 215 | # (source start file, name, description, authors, manual section). 216 | man_pages = [ 217 | ('index', 'pdlist', u'pdlist Documentation', 218 | [u'gnc'], 1) 219 | ] 220 | 221 | 222 | # Example configuration for intersphinx: refer to the Python standard library. 223 | intersphinx_mapping = {'http://docs.python.org/': None} 224 | -------------------------------------------------------------------------------- /docs/html/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | Redirecting to Read The Docs 4 | 5 | The docs are at pdlist.readthedocs.org. 6 | You will be redirected there in a while. If not, click the link above. 7 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | =============================== 2 | pdlist 3 | =============================== 4 | 5 | .. toctree:: 6 | :maxdepth: 2 7 | 8 | README for pdlist 9 | CONTRIBUTING 10 | LICENSE 11 | CHANGELOG 12 | 13 | Indices and tables 14 | ================== 15 | 16 | * :ref:`genindex` 17 | * :ref:`modindex` 18 | * :ref:`search` 19 | -------------------------------------------------------------------------------- /img/pdlist_demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnebbia/pdlist/90e5e9ca16182b9554d010e6f6dabd43238287fe/img/pdlist_demo.gif -------------------------------------------------------------------------------- /pdlist/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # pdlist v0.1.0 3 | # A passive subdomain finder 4 | # Copyright © 2019, gnc. 5 | # All rights reserved. 6 | # 7 | # Redistribution and use in source and binary forms, with or without 8 | # modification, are permitted provided that the following conditions are 9 | # met: 10 | # 11 | # 1. Redistributions of source code must retain the above copyright 12 | # notice, this list of conditions, and the following disclaimer. 13 | # 14 | # 2. Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions, and the following disclaimer in the 16 | # documentation and/or other materials provided with the distribution. 17 | # 18 | # 3. Neither the name of the author of this software nor the names of 19 | # contributors to this software may be used to endorse or promote 20 | # products derived from this software without specific prior written 21 | # consent. 22 | # 23 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 24 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 25 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 26 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 27 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 28 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 29 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 30 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 31 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 32 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 33 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 34 | 35 | """ 36 | A passive subdomain finder 37 | 38 | :Copyright: © 2019, gnc. 39 | :License: BSD (see /LICENSE). 40 | """ 41 | 42 | __title__ = 'pdlist' 43 | __version__ = '0.1.0' 44 | __author__ = 'gnc' 45 | __license__ = '3-clause BSD' 46 | __docformat__ = 'restructuredtext en' 47 | 48 | __all__ = () 49 | 50 | # import gettext 51 | # G = gettext.translation('pdlist', '/usr/share/locale', fallback='C') 52 | # _ = G.gettext 53 | -------------------------------------------------------------------------------- /pdlist/__main__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # pdlist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | Main routine of pdlist. 9 | 10 | :Copyright: © 2019, gnc. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | from pdlist.main import main 14 | 15 | 16 | 17 | 18 | if __name__ == '__main__': 19 | main() 20 | -------------------------------------------------------------------------------- /pdlist/main.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # pdlist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | Main routine of pdlist. 9 | 10 | :Copyright: © 2019, gnc. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import re 14 | import argparse 15 | import pdlist.source.crtsh as cr 16 | import pdlist.source.urlscan as us 17 | import pdlist.source.threatcrowd as tc 18 | import pdlist.source.dnsdumpster as dd 19 | import pdlist.source.certspotter as cs 20 | import pdlist.source.hackertarget as ht 21 | from pdlist.utils import (polish_subdomain_strings, 22 | remove_unrelated_domains, 23 | clean_domain_strings, 24 | sort_domains) 25 | 26 | __all__ = ('main',) 27 | 28 | 29 | def show_banner(): 30 | print(""" 31 | _____ __ 32 | ____ ____/ / (_)____/ /_ 33 | / __ \/ __ / / / ___/ __/ 34 | / /_/ / /_/ / / (__ ) /_ 35 | / .___/\__,_/_/_/____/\__/ 36 | /_/ 37 | 38 | A passive domain sublister 39 | Developed by gnc 40 | """) 41 | 42 | 43 | def main(): 44 | """Main routine of pdlist.""" 45 | show_banner() 46 | parser = argparse.ArgumentParser( 47 | prog='pdlist', description='A passive subdomain enumerator') 48 | 49 | parser.add_argument( 50 | "domains", 51 | help="Specify the domain to enumerate", 52 | default=[], 53 | type=str, 54 | nargs='+', 55 | ) 56 | parser.add_argument( 57 | "-s", "--strict", 58 | dest='is_strict', 59 | action='store_true', 60 | help="Enables strict mode, where only proper (and not also related)\ 61 | subdomains will be saved", 62 | default=False, 63 | ) 64 | parser.add_argument( 65 | "-o", "--output", 66 | dest='outputfile', 67 | help="Save results to the specified file", 68 | default=None, 69 | nargs='?', 70 | type=argparse.FileType(mode='wt',encoding='utf-8'), 71 | ) 72 | 73 | args = parser.parse_args() 74 | subdomains = [] 75 | domains = clean_domain_strings(args.domains) 76 | 77 | print( 78 | '\033[94m[*] \033[0m The analyzed domains will be: ' + 79 | ' '.join(domains)) 80 | 81 | found_subdomains = [] 82 | sources = [tc, ht, us, dd, cr, cs] 83 | for source in sources: 84 | name = re.sub(r' parser$', '', source.__doc__.strip().split('\n')[0]) 85 | print('\033[32m[+] \033[0m Searching on {}...'.format(name)) 86 | # Decomment here if you want to get the number of extracted 87 | # subdomains ... 88 | 89 | # found_subdomains += source.parse(domains) 90 | # print('\033[32m[+] \033[0m Found {} subdomains from {}' 91 | # .format(len(found_subdomains), source)) 92 | # subdomains += found_subdomains 93 | try: 94 | subdomains += source.parse(domains) 95 | except Exception: 96 | next 97 | 98 | print() 99 | print('\033[32m[+] \033[0m Printing domain list...') 100 | print() 101 | 102 | subdomains = polish_subdomain_strings(subdomains) 103 | 104 | if args.is_strict: 105 | subdomains = remove_unrelated_domains(subdomains, domains) 106 | 107 | # only get unique subdomains strings to avoid duplicates 108 | subdomains = list(set(subdomains)) 109 | 110 | # sort subdomains 111 | subdomains = sort_domains(subdomains) 112 | 113 | print() 114 | 115 | if args.outputfile is not None: 116 | args.outputfile.write('\n'.join(subdomains)) 117 | for subdomain in subdomains: 118 | print(subdomain) 119 | -------------------------------------------------------------------------------- /pdlist/source/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnebbia/pdlist/90e5e9ca16182b9554d010e6f6dabd43238287fe/pdlist/source/__init__.py -------------------------------------------------------------------------------- /pdlist/source/certspotter.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | certspotter parser 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import json 14 | import requests 15 | 16 | 17 | def parse(domains): 18 | """ 19 | This function performs a request to certspotter and after having 20 | parsed its output returns a cleaned list of unique domains 21 | 22 | Args: 23 | domains -- the list of input domain to query 24 | 25 | Returns: 26 | a cleaned list of unique subdomains obtained after querying certspotter 27 | """ 28 | subdomains = [] 29 | for domain in domains: 30 | url = 'https://certspotter.com/api/v0/certs?domain={}'.format(domain) 31 | json_resp = json.loads(requests.get(url).text) 32 | try: 33 | doms = [e['dns_names'] for e in json_resp] 34 | for subs in doms: 35 | subdomains += subs 36 | except TypeError: 37 | print("\033[93m[*] \033[0m Cert Spotter API: Number of allowed requests exceeded") 38 | return list(set(subdomains)) 39 | -------------------------------------------------------------------------------- /pdlist/source/crtsh.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | crt.sh parser 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import sys 14 | import json 15 | import requests 16 | 17 | 18 | def parse(domains): 19 | """ 20 | This function performs a request to crt.sh and after having 21 | parsed its output returns a cleaned list of unique domains 22 | 23 | Args: 24 | domains -- the list of input domain to query 25 | 26 | Returns: 27 | a cleaned list of unique subdomains obtained after querying crt.sh 28 | """ 29 | ua = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1' 30 | subdomains = [] 31 | crt_namevalues = [] 32 | for domain in domains: 33 | url = "https://crt.sh/?q={}&output=json".format(domain) 34 | req = requests.get(url, headers={'User-Agent':ua}) 35 | if req.ok: 36 | try: 37 | json_resp = json.loads(req.content.decode('utf-8')) 38 | crt_namevalues = [e['name_value'] for e in json_resp] 39 | subdomains += [x for s in crt_namevalues for x in s.split()] 40 | except json.decoder.JSONDecodeError: 41 | print("ERROR: crt.sh response may be too long or badly formatted", 42 | file=sys.stderr) 43 | return list(set(subdomains)) 44 | -------------------------------------------------------------------------------- /pdlist/source/dnsdumpster.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | 3 | # dplist v0.1.0 4 | # A passive sudomain lister 5 | # Copyright © 2019, Giuseppe Nebbione. 6 | # See /LICENSE for licensing information. 7 | 8 | """ 9 | DnsDumpster parser 10 | 11 | :Copyright: © 2019, Giuseppe Nebbione. 12 | :License: BSD (see /LICENSE). 13 | """ 14 | import re 15 | import requests 16 | import sys 17 | from pdlist.utils import find 18 | from bs4 import BeautifulSoup 19 | 20 | 21 | 22 | def retrieve_results(table): 23 | """ 24 | Function copied from DNSDumpster API to cancel 25 | errors related to index list out of range 26 | """ 27 | 28 | res = [] 29 | trs = table.findAll('tr') 30 | for tr in trs: 31 | tds = tr.findAll('td') 32 | pattern_ip = r'([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})' 33 | try: 34 | ip = re.findall(pattern_ip, tds[1].text)[0] 35 | domain = str(tds[0]).split('
')[0].split('>')[1] 36 | header = ' '.join(tds[0].text.replace('\n', '').split(' ')[1:]) 37 | reverse_dns = tds[1].find('span', attrs={}).text 38 | 39 | additional_info = tds[2].text 40 | country = tds[2].find('span', attrs={}).text 41 | autonomous_system = additional_info.split(' ')[0] 42 | provider = ' '.join(additional_info.split(' ')[1:]) 43 | provider = provider.replace(country, '') 44 | data = {'domain': domain, 45 | 'ip': ip, 46 | 'reverse_dns': reverse_dns, 47 | 'as': autonomous_system, 48 | 'provider': provider, 49 | 'country': country, 50 | 'header': header} 51 | res.append(data) 52 | except: 53 | pass 54 | return res 55 | 56 | 57 | def retrieve_txt_record(table): 58 | """ 59 | Function copied from DNSDumpster API to cancel 60 | errors related to index list out of range 61 | """ 62 | res = [] 63 | for td in table.findAll('td'): 64 | res.append(td.text) 65 | return res 66 | 67 | def search(domain): 68 | """ 69 | Function copied from DNSDumpster API to cancel 70 | errors related to index list out of range 71 | """ 72 | dnsdumpster_url = 'https://dnsdumpster.com/' 73 | 74 | session = requests.Session() 75 | req = session.get(dnsdumpster_url) 76 | soup = BeautifulSoup(req.content, 'html.parser') 77 | csrf_middleware = soup.findAll('input', attrs={'name': 'csrfmiddlewaretoken'})[0]['value'] 78 | 79 | cookies = {'csrftoken': csrf_middleware} 80 | headers = {'Referer': dnsdumpster_url} 81 | data = {'csrfmiddlewaretoken': csrf_middleware, 'targetip': domain} 82 | req = session.post(dnsdumpster_url, cookies=cookies, data=data, headers=headers) 83 | 84 | if req.status_code != 200: 85 | print( 86 | "Unexpected status code from {url}: {code}".format( 87 | url=dnsdumpster_url, code=req.status_code), 88 | file=sys.stderr, 89 | ) 90 | return [] 91 | 92 | if 'error' in req.content.decode('utf-8'): 93 | print("There was an error getting results", file=sys.stderr) 94 | return [] 95 | 96 | soup = BeautifulSoup(req.content, 'html.parser') 97 | tables = soup.findAll('table') 98 | 99 | res = {} 100 | res['domain'] = domain 101 | res['dns_records'] = {} 102 | res['dns_records']['dns'] = retrieve_results(tables[0]) 103 | res['dns_records']['mx'] = retrieve_results(tables[1]) 104 | res['dns_records']['txt'] = retrieve_txt_record(tables[2]) 105 | res['dns_records']['host'] = retrieve_results(tables[3]) 106 | 107 | # Network mapping image 108 | try: 109 | tmp_url = 'https://dnsdumpster.com/static/map/{}.png'.format(domain) 110 | image_data = base64.b64encode(session.get(tmp_url).content) 111 | except: 112 | image_data = None 113 | finally: 114 | res['image_data'] = image_data 115 | 116 | # XLS hosts. 117 | # eg. tsebo.com-201606131255.xlsx 118 | try: 119 | pattern = r'https://dnsdumpster.com/static/xls/' + domain + '-[0-9]{12}\.xlsx' 120 | xls_url = re.findall(pattern, req.content.decode('utf-8'))[0] 121 | xls_data = base64.b64encode(session.get(xls_url).content) 122 | except Exception as err: 123 | xls_data = None 124 | finally: 125 | res['xls_data'] = xls_data 126 | 127 | return res 128 | 129 | 130 | 131 | def parse(domains): 132 | """ 133 | This function performs a request to dnsdumpster and after having 134 | parsed its output returns a cleaned list of unique domains 135 | 136 | Args: 137 | domains -- the list of input domain to query 138 | 139 | Returns: 140 | a cleaned list of unique subdomains obtained after querying dnsdumpster 141 | """ 142 | subdomains = [] 143 | for domain in domains: 144 | results = search(domain) 145 | if not results: 146 | continue 147 | subdomains += list(set(find('domain', results['dns_records']))) 148 | subdomains += list(set(find('reverse_dns', results['dns_records']))) 149 | return subdomains 150 | -------------------------------------------------------------------------------- /pdlist/source/hackertarget.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | Hackertarget parser 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import requests 14 | 15 | 16 | def parse(domains): 17 | """ 18 | This function performs a request to hackertarget and after having 19 | parsed its output returns a cleaned list of unique domains 20 | 21 | Args: 22 | domains -- the list of input domain to query 23 | 24 | Returns: 25 | a cleaned list of unique subdomains obtained after querying hackertarget 26 | """ 27 | subdomains = [] 28 | for domain in domains: 29 | url = 'https://api.hackertarget.com/hostsearch/?q={}'.format(domain) 30 | resp = requests.get(url) 31 | lines = resp.text.split() 32 | for line in lines: 33 | subdomains.append(line.split(',')[0]) 34 | return subdomains 35 | -------------------------------------------------------------------------------- /pdlist/source/threatcrowd.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | Threatcrowd parser 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import json 14 | import requests 15 | 16 | 17 | def parse(domains): 18 | """ 19 | This function performs a request to threatcrowd and after having 20 | parsed its output returns a cleaned list of unique domains 21 | 22 | Args: 23 | domains -- the list of input domain to query 24 | 25 | Returns: 26 | a cleaned list of unique subdomains obtained after querying threatcrowd 27 | """ 28 | subdomains = [] 29 | for domain in domains: 30 | url = 'https://www.threatcrowd.org/searchApi/v2/domain/report/?domain={}'.format(domain) 31 | resp = requests.get(url) 32 | json_resp = json.loads(resp.text) 33 | if 'subdomains' in json_resp.keys(): 34 | subdomains += json_resp['subdomains'] 35 | return subdomains 36 | -------------------------------------------------------------------------------- /pdlist/source/urlscan.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | UrlScan parser 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import json 14 | import requests 15 | from pdlist.utils import find 16 | 17 | 18 | def parse(domains): 19 | """ 20 | This function performs a request to urlscan and after having 21 | parsed its output returns a cleaned list of unique domains 22 | 23 | Args: 24 | domains -- the list of input domain to query 25 | 26 | Returns: 27 | a cleaned list of unique subdomains obtained after querying urlscan 28 | """ 29 | subdomains = [] 30 | for domain in domains: 31 | url = 'https://urlscan.io/api/v1/search/?q=domain:{}'.format(domain) 32 | json_resp = json.loads(requests.get(url).text) 33 | subdomains += list(set(find('domain', json_resp))) 34 | return list(set(subdomains)) 35 | -------------------------------------------------------------------------------- /pdlist/utils.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # dplist v0.1.0 3 | # A passive sudomain lister 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | """ 8 | utils functions for pdlist 9 | 10 | :Copyright: © 2019, Giuseppe Nebbione. 11 | :License: BSD (see /LICENSE). 12 | """ 13 | import re 14 | from collections import defaultdict 15 | 16 | 17 | def remove_unrelated_domains(subdomains, domains): 18 | """ 19 | This function removes from the entire set hostnames found, the ones who 20 | do not end with the target provided domain name. 21 | So if in the list of domains we have example.com and our target/scope 22 | is example.it, then example.com will be removed because falls out of the 23 | scope. 24 | 25 | Args: 26 | domains -- the list of input target domains 27 | 28 | Returns: 29 | subdomains -- the set of subomains strictly respecting the scope 30 | """ 31 | subdomains = [s for s in subdomains if s.endswith(tuple(domains))] 32 | return subdomains 33 | 34 | def polish_subdomain_strings(subdomains): 35 | """ 36 | This function is used to polish subdomain strings which may come out 37 | from some query service. Indeed some services provide hostnames with 38 | spaces or starting with dots. 39 | Hence this function returns a polished list of subdomains which are legal 40 | and can be queried further. 41 | Polishing in this context means: 42 | - removing spaces 43 | - removing initial or ending dots 44 | 45 | Args: 46 | subdomains -- the list of subdomains to polish 47 | 48 | Returns: 49 | subdomains -- the list of polished subdomains 50 | """ 51 | # remove initial leading/trailing spaces 52 | subdomains = [item.strip() for item in subdomains] 53 | # remove initial dots if present 54 | subdomains = [item.rstrip('.') for item in subdomains] 55 | # remove strings before space, e.g., 5 mail.example.com -> mail.example.com 56 | subdomains = [re.sub(r'^.* ', '', item) for item in subdomains] 57 | # remove strings starting with a dot or a star 58 | subdomains = [re.sub(r'^[\.\*]\.?', '', item) for item in subdomains] 59 | # remove leading/traling spaces 60 | subdomains = [item.strip() for item in subdomains] 61 | # remove empty strings 62 | subdomains = list(filter(lambda x: x != "", subdomains)) 63 | 64 | return subdomains 65 | 66 | def find(key, dictionary): 67 | """ 68 | This function is used to find the value associated to a key in an 69 | arbitrarily nested dictionary. 70 | 71 | Args: 72 | key -- the key associated to the value we are interested in 73 | dictionary -- the (arbitrarily nested) dictionary in which to search for the 74 | key 75 | 76 | Returns: 77 | value -- returns the value associated to the key we searched for 78 | """ 79 | if isinstance(dictionary, dict): 80 | for k, v in dictionary.items(): 81 | if k == key: 82 | yield v 83 | elif isinstance(v, dict): 84 | for result in find(key, v): 85 | yield result 86 | elif isinstance(v, list): 87 | for d in v: 88 | for result in find(key, d): 89 | yield result 90 | 91 | def clean_domain_strings(domains): 92 | """ 93 | This function is used to normalize input domain passed from the command line 94 | by the user. Basically we want to be able to handle both domains passed in 95 | the form example.com but also domains passed erroneously with a scheme, such 96 | as https://example.com 97 | 98 | Args: 99 | domains -- the list of input domains 100 | 101 | Returns: 102 | subdomains -- the list of normalized domains 103 | """ 104 | domains = [item.rstrip('/') for item in domains] 105 | domains = [re.sub(r'(http://|https://)', '', item) for item in domains] 106 | return domains 107 | 108 | 109 | def sort_domains(domains): 110 | """ 111 | This function is used to sort the domains in a hierarchical order for 112 | readability. 113 | 114 | Args: 115 | domains -- the list of input domains 116 | 117 | Returns: 118 | domains -- hierarchically sorted list of domains 119 | """ 120 | domainbits = [domain.lower().split('.') for domain in domains] 121 | for domain in domainbits: 122 | domain.reverse() 123 | sorted(domainbits) 124 | domains = [('.'.join(reversed(domain))) for domain in sorted(domainbits)] 125 | return domains 126 | -------------------------------------------------------------------------------- /release: -------------------------------------------------------------------------------- 1 | #!/bin/zsh 2 | # The Release Script 3 | # Part of the Python Project Template. 4 | # Copyright © 2013-2018, Chris Warrick. 5 | # All rights reserved. 6 | # 7 | # Redistribution and use in source and binary forms, with or without 8 | # modification, are permitted provided that the following conditions are 9 | # met: 10 | # 11 | # 1. Redistributions of source code must retain the above copyright 12 | # notice, this list of conditions, and the following disclaimer. 13 | # 14 | # 2. Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions, and the following disclaimer in the 16 | # documentation and/or other materials provided with the distribution. 17 | # 18 | # 3. Neither the name of the author of this software nor the names of 19 | # contributors to this software may be used to endorse or promote 20 | # products derived from this software without specific prior written 21 | # consent. 22 | # 23 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 24 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 25 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 26 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 27 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 28 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 29 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 30 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 31 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 32 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 33 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 34 | 35 | . .pypt/config 36 | function status { 37 | echo $@ 38 | } 39 | 40 | function warning { 41 | echo 'WARNING: '$@ 42 | } 43 | 44 | function error { 45 | echo 'ERROR: '$@ 46 | } 47 | 48 | function cleanup { 49 | rm -rf $PROJECTLC.egg-info build || true 50 | rm -rf **/__pycache__ || true 51 | } 52 | 53 | function cleanup_cmfn { 54 | [[ -e $cmfn ]] && rm $cmfn || true 55 | [[ -e $cmfn2 ]] && rm $cmfn2 || true 56 | } 57 | 58 | status '*** Chris Warrick’s Release Scripts (PyPT)' 59 | 60 | echo -n "Version number: " 61 | read version 62 | 63 | echo $version | grep '^[0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}$' > /dev/null 64 | 65 | if [[ $? != 0 ]]; then 66 | echo $version | grep '^[0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\-[0-9A-Za-z-]\{1,\}$' > /dev/null 67 | if [[ $? != 0 ]]; then 68 | warning 'version number is not compliant with versioning scheme (Semantic Versioning 2.0)' 69 | echo -n 'Continue? [y/N] ' 70 | read vercont 71 | if [[ $vercont == 'y' || $vercont == 'Y' ]]; then 72 | echo 'Continuing.' 73 | else 74 | exit 2 75 | fi 76 | else 77 | status 'NOTICE: pre-release version number in use.' 78 | echo -n 'Continue? [Y/n] ' 79 | read vercont 80 | if [[ $vercont == 'n' || $vercont == 'N' ]]; then 81 | exit 2 82 | else 83 | echo 'Continuing.' 84 | fi 85 | fi 86 | fi 87 | 88 | # Creating all the dates at the exact same time. 89 | date=$(date '+%Y-%m-%d') 90 | datel=$(date '+%Y-%m-%d %H:%M%z') 91 | #datep=$(date '+%Y%m%d') 92 | dates=$(date '+%s') 93 | 94 | cmfn=$PWD/.git/kwrs-$dates 95 | cmfn2=$cmfn"-commit" 96 | 97 | cleanup 98 | cleanup_cmfn 99 | git add -A --ignore-errors . 100 | 101 | cat > $cmfn <> $cmfn 121 | 122 | if [[ "$EDITOR" == 'vim' || "$EDITOR" == '/usr/bin/vim' ]]; then 123 | $EDITOR -c 'set filetype=gitcommit' $cmfn 124 | elif [[ $EDITOR == '' ]]; then 125 | vim -c 'set filetype=gitcommit' $cmfn 126 | else 127 | $EDITOR $cmfn 128 | fi 129 | 130 | .pypt/commitlog $cmfn $PWD $version 131 | 132 | status 'Replacing versions and dates in files...' 133 | sed "s/version=.*/version='$version',/g" setup.py -i 134 | sed "s/version = .*/version = '$version'/g" docs/conf.py -i 135 | sed "s/release = .*/release = '$version'/g" docs/conf.py -i 136 | sed "s/:Version: .*/:Version: $version/g" docs/**/*.rst -i 137 | sed "s/# $PROJECT v.*/# $PROJECT v$version/" $PROJECTLC/**/*.py -i 138 | sed "s/__version__ = .*/__version__ = '$version'/g" $PROJECTLC/__init__.py -i 139 | sed "s/:Date: .*/:Date: $date/g" docs/*.rst -i 140 | 141 | [[ -e "PKGBUILD" ]] && sed "s/pkgver=.*/pkgver=$version/g" PKGBUILD -i || true 142 | [[ -e "PKGBUILD-git" ]] && sed "s/pkgver=.*/pkgver=$version/g" PKGBUILD-git -i || true 143 | 144 | cp docs/README.rst docs/CHANGELOG.rst docs/CONTRIBUTING.rst . 145 | cp docs/README.rst README 146 | 147 | status 'Generating locales...' 148 | ./.pypt/localegen 149 | 150 | status 'Importing...' 151 | python -c "import $PROJECTLC" 152 | if [[ $? != 0 ]]; then 153 | error "Import failed. Fix your code or don't come back." 154 | exit 1 155 | fi 156 | 157 | if [[ -e tests ]]; then 158 | status 'Running tests...' 159 | pytest tests/ 160 | if [[ $? != 0 ]]; then 161 | error "Tests failed. Fix your code or don't come back." 162 | exit 1 163 | fi 164 | fi 165 | 166 | status 'Running pre-sdist.hook...' 167 | 168 | . .pypt/hooks/pre-sdist.hook 169 | 170 | status 'This is the last chance to quit. Hit ^C now if you want to.' 171 | read bailout 172 | 173 | ./setup.py sdist bdist_wheel || exit $? 174 | twine upload -s dist/$PROJECTLC-$version.tar.gz dist/$PROJECTLC-$version*.whl || exit $? 175 | 176 | if [[ -e PKGBUILD ]]; then 177 | status 'Updating AUR PKGBUILDs...' 178 | md5out=$(md5sum 'dist/'$PROJECTLC'-'$version'.tar.gz'|awk '{print $1}') 179 | sed "s/md5sums=.*/md5sums=('$md5out')/" PKGBUILD -i 180 | fi 181 | 182 | cleanup 183 | 184 | git add -A --ignore-errors . || exit $? 185 | 186 | git commit -S -asF $cmfn2 || exit $? 187 | git tag -sm "Version $version" v$version || exit $? 188 | git push --follow-tags origin master || exit $? 189 | 190 | .pypt/ghrel $cmfn $PWD $GITUSER/$GITREPO v$version 191 | 192 | cleanup_cmfn 193 | 194 | status "Done!" 195 | . .pypt/hooks/post-release.hook 196 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests 2 | dnsdumpster 3 | BeautifulSoup4 4 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.md 3 | 4 | [flake8] 5 | ignore = E741 6 | 7 | [wheel] 8 | universal = 1 9 | 10 | [tool:pytest] 11 | norecursedirs = .git 12 | addopts = --cov=pdlist --cov-report term-missing 13 | 14 | [coverage:run] 15 | branch = True 16 | omit = tests/* 17 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- encoding: utf-8 -*- 3 | import io 4 | from setuptools import setup, find_packages 5 | 6 | 7 | setup(name='pdlist', 8 | version='0.1.0', 9 | description='A passive subdomain finder', 10 | keywords='pdlist', 11 | author='gnc', 12 | author_email='nebbionegiuseppe@gmail.com', 13 | url='https://github.com/gnebbia/pdlist', 14 | license='3-clause BSD', 15 | long_description=io.open( 16 | './docs/README.rst', 'r', encoding='utf-8').read(), 17 | platforms='any', 18 | zip_safe=False, 19 | # http://pypi.python.org/pypi?%3Aaction=list_classifiers 20 | classifiers=['Development Status :: 1 - Planning', 21 | 'Programming Language :: Python :: 3', 22 | 'Programming Language :: Python :: 3.3', 23 | 'Programming Language :: Python :: 3.4', 24 | 'Programming Language :: Python :: 3.5', 25 | 'Programming Language :: Python :: 3.6', 26 | 'Programming Language :: Python :: 3.7', 27 | ], 28 | packages=find_packages(exclude=('tests',)), 29 | include_package_data=True, 30 | install_requires=['dnsdumpster','requests', 'BeautifulSoup4'], 31 | entry_points={ 32 | 'console_scripts':[ 33 | 'pdlist = pdlist.main:main', 34 | ] 35 | }, 36 | ) 37 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | # pdlist test suite 3 | # Copyright © 2019, Giuseppe Nebbione. 4 | # All rights reserved. 5 | # 6 | # Redistribution and use in source and binary forms, with or without 7 | # modification, are permitted provided that the following conditions are 8 | # met: 9 | # 10 | # 1. Redistributions of source code must retain the above copyright 11 | # notice, this list of conditions, and the following disclaimer. 12 | # 13 | # 2. Redistributions in binary form must reproduce the above copyright 14 | # notice, this list of conditions, and the following disclaimer in the 15 | # documentation and/or other materials provided with the distribution. 16 | # 17 | # 3. Neither the name of the author of this software nor the names of 18 | # contributors to this software may be used to endorse or promote 19 | # products derived from this software without specific prior written 20 | # consent. 21 | # 22 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | -------------------------------------------------------------------------------- /tests/test_procedure_1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- encoding: utf-8 -*- 3 | # pdlist test suite 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | 8 | import pdlist 9 | import pdlist.source.threatcrowd as tc 10 | import pdlist.source.hackertarget as ht 11 | import pdlist.source.dnsdumpster as dd 12 | import pdlist.source.crtsh as cr 13 | import pdlist.source.certspotter as cs 14 | import pdlist.source.urlscan as us 15 | 16 | 17 | 18 | def test_main(): 19 | """Test adding source.""" 20 | domain = 'localhost' 21 | subdomains = [] 22 | 23 | print('[+] Searching on threatcrowd...') 24 | subdomains += tc.parse(['localhost']) 25 | print(subdomains) 26 | 27 | print('[+] Searching on hackertarget...') 28 | subdomains += ht.parse(['localhost']) 29 | print(subdomains) 30 | 31 | print('[+] Searching on dnsdumpster...') 32 | subdomains += dd.parse(['localhost']) 33 | print(subdomains) 34 | 35 | print('[+] Searching on cert.sh...') 36 | subdomains += cr.parse(['localhost']) 37 | print(subdomains) 38 | 39 | print('[+] Searching on certspotter...') 40 | subdomains += cs.parse(['localhost']) 41 | print(subdomains) 42 | 43 | 44 | def test_import(): 45 | """Test imports.""" 46 | pdlist 47 | pdlist.source 48 | -------------------------------------------------------------------------------- /tests/test_sanity.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- encoding: utf-8 -*- 3 | # pdlist test suite 4 | # Copyright © 2019, Giuseppe Nebbione. 5 | # See /LICENSE for licensing information. 6 | 7 | 8 | import pdlist 9 | import pdlist.source 10 | 11 | 12 | def test_true(): 13 | """Test if True is truthy.""" 14 | assert True 15 | 16 | 17 | def test_false(): 18 | """Test if False is falsey.""" 19 | assert not False 20 | 21 | 22 | def test_trueexpr(): 23 | """Test if this evaluates as True.""" 24 | assert 1 == 1 25 | 26 | 27 | def test_falseexpr(): 28 | """Test if this evaluates as False.""" 29 | assert 1 != 2 30 | 31 | 32 | def test_math(): 33 | """Test basic arithmetic skills of Python.""" 34 | assert 2 + 2 == 4 35 | assert 2 - 2 == 0 36 | assert 2 * 2 == 4 37 | assert 2 / 2 == 1 38 | assert 3 % 2 == 1 39 | 40 | 41 | def test_bitwise(): 42 | """Test bitwise operators.""" 43 | assert 0b11 ^ 0b10 == 0b01 44 | assert 0b100 | 0b010 == 0b110 45 | assert 0b101 & 0b011 == 0b001 46 | assert 0b10 << 2 == 0b1000 47 | assert 0b1111 >> 2 == 0b11 48 | 49 | 50 | def test_import(): 51 | """Test imports.""" 52 | pdlist 53 | pdlist.source 54 | --------------------------------------------------------------------------------