├── .gitmodules ├── LICENSE ├── README.md ├── black.css ├── imgs ├── accenture.jpg ├── appsecco.png ├── appsecco_logo.png ├── cdn.png ├── censys.gif ├── censys.png ├── censys_script.png ├── cheatsheet_full.png ├── commit.jpg ├── comparision.png ├── cover.png ├── crtsh.png ├── crtsh_script.jpg ├── csp.gif ├── csp.png ├── delo.jpg ├── domain_types.png ├── example_nsec3_1.png ├── example_nsec3_2.png ├── facebook_ct.png ├── fdns_enum.png ├── insecure_aws.jpg ├── insecure_aws2.jpg ├── issues.jpg ├── maps_tint.jpeg ├── massdns.png ├── nsec_zone_walk.png ├── private_space.jpeg ├── public_space.jpeg ├── reversewhois.png ├── spaces_finder.png ├── spaces_pattern.jpeg ├── speakerdeck.png ├── spf.gif ├── spf.png ├── spf_sample.png ├── video.png └── vt.png ├── practical_recon.md └── practical_recon.pdf /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "the-art-of-subdomain-enumeration"] 2 | path = the-art-of-subdomain-enumeration 3 | url = git@github.com:appsecco/the-art-of-subdomain-enumeration.git 4 | [submodule "censys-enumeration"] 5 | path = censys-enumeration 6 | url = git@github.com:yamakira/censys-enumeration.git 7 | [submodule "domains-from-csp"] 8 | path = domains-from-csp 9 | url = git@github.com:yamakira/domains-from-csp.git 10 | [submodule "assets-from-spf"] 11 | path = assets-from-spf 12 | url = git@github.com:yamakira/assets-from-spf.git 13 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Practical recon techniques for bug hunters & pentesters 2 | 3 | This repository contains all the material from the talk "Practical recon techniques for bug hunters & pentesters" given at Bugcrowd LevelUp 0x02 virtual conference 4 | 5 | All the scripts are maintained as part of different repos and are included in this repo as git submodules so ideally if you want to clone along with the submodules, use the following command 6 | 7 | 8 | ``` 9 | git clone --recurse-submodules -j8 git@github.com:appsecco/practical-recon-levelup0x02.git 10 | ``` 11 | 12 | ## Slide deck 13 | 14 | [![Practical recon techniques for bug hunters & pentesters](imgs/speakerdeck.png)](https://speakerdeck.com/0xbharath/practical-recon-techniques-for-bug-hunters-and-pentesters) 15 | 16 | ## Talk video 17 | 18 | [![Practical recon techniques for bug hunters & pentesters](imgs/video.png)](https://www.youtube.com/watch?v=McLdm4c1oLs) 19 | 20 | ## Rendering the presentation 21 | 22 | The presentation is built using [reveal-md](https://github.com/webpro/reveal-md). If you simply want to run the presentation, follow the below steps: 23 | 24 | 1. Clone the repo `git clone git@github.com:appsecco/practical-recon-levelup0x02.git` 25 | 2. Install reveal-md `npm install -g reveal-md` 26 | 3. Once reveal-md is installed, browse to the cloned directory and run the following command `reveal-md -t black.css --separator "\n\n\n\n" --vertical-separator "\n\n\n" practical_recon.md` 27 | -------------------------------------------------------------------------------- /black.css: -------------------------------------------------------------------------------- 1 | /** 2 | * Black theme for reveal.js. This is the opposite of the 'white' theme. 3 | * 4 | * By Hakim El Hattab, http://hakim.se 5 | */ 6 | @import url(../../lib/font/source-sans-pro/source-sans-pro.css); 7 | section.has-light-background, section.has-light-background h1, section.has-light-background h2, section.has-light-background h3, section.has-light-background h4, section.has-light-background h5, section.has-light-background h6 { 8 | color: #222; } 9 | 10 | /********************************************* 11 | * GLOBAL STYLES 12 | *********************************************/ 13 | body { 14 | background: #222; 15 | background-color: #222; } 16 | 17 | .reveal { 18 | font-family: "Source Sans Pro", Helvetica, sans-serif; 19 | font-size: 42px; 20 | font-weight: normal; 21 | color: #fff; } 22 | 23 | ::selection { 24 | color: #fff; 25 | background: #bee4fd; 26 | text-shadow: none; } 27 | 28 | ::-moz-selection { 29 | color: #fff; 30 | background: #bee4fd; 31 | text-shadow: none; } 32 | 33 | .reveal .slides > section, 34 | .reveal .slides > section > section { 35 | line-height: 1.3; 36 | font-weight: inherit; } 37 | 38 | /********************************************* 39 | * HEADERS 40 | *********************************************/ 41 | .reveal h1, 42 | .reveal h2, 43 | .reveal h3, 44 | .reveal h4, 45 | .reveal h5, 46 | .reveal h6 { 47 | margin: 0 0 20px 0; 48 | color: #fff; 49 | font-family: "Source Sans Pro", Helvetica, sans-serif; 50 | font-weight: 600; 51 | line-height: 1.2; 52 | letter-spacing: normal; 53 | text-transform: uppercase; 54 | text-shadow: none; 55 | word-wrap: break-word; } 56 | 57 | .reveal h1 { 58 | font-size: 2.5em; } 59 | 60 | .reveal h2 { 61 | font-size: 1.6em; } 62 | 63 | .reveal h3 { 64 | font-size: 1.3em; } 65 | 66 | .reveal h4 { 67 | font-size: 1em; } 68 | 69 | .reveal h1 { 70 | text-shadow: none; } 71 | 72 | /********************************************* 73 | * OTHER 74 | *********************************************/ 75 | .reveal p { 76 | margin: 20px 0; 77 | line-height: 1.3; } 78 | 79 | /* Ensure certain elements are never larger than the slide itself */ 80 | .reveal img, 81 | .reveal video, 82 | .reveal iframe { 83 | max-width: 95%; 84 | max-height: 95%; } 85 | 86 | .reveal strong, 87 | .reveal b { 88 | font-weight: bold; } 89 | 90 | .reveal em { 91 | font-style: italic; } 92 | 93 | .reveal ol, 94 | .reveal dl, 95 | .reveal ul { 96 | display: inline-block; 97 | text-align: left; 98 | margin: 0 0 0 1em; } 99 | 100 | .reveal ol { 101 | list-style-type: decimal; } 102 | 103 | .reveal ul { 104 | list-style-type: disc; } 105 | 106 | .reveal ul ul { 107 | list-style-type: square; } 108 | 109 | .reveal ul ul ul { 110 | list-style-type: circle; } 111 | 112 | .reveal ul ul, 113 | .reveal ul ol, 114 | .reveal ol ol, 115 | .reveal ol ul { 116 | display: block; 117 | margin-left: 40px; } 118 | 119 | .reveal dt { 120 | font-weight: bold; } 121 | 122 | .reveal dd { 123 | margin-left: 40px; } 124 | 125 | .reveal blockquote { 126 | display: block; 127 | position: relative; 128 | width: 70%; 129 | margin: 20px auto; 130 | padding: 5px; 131 | font-style: italic; 132 | background: rgba(255, 255, 255, 0.05); 133 | box-shadow: 0px 0px 2px rgba(0, 0, 0, 0.2); } 134 | 135 | .reveal blockquote p:first-child, 136 | .reveal blockquote p:last-child { 137 | display: inline-block; } 138 | 139 | .reveal q { 140 | font-style: italic; } 141 | 142 | .reveal pre { 143 | display: block; 144 | position: relative; 145 | width: 90%; 146 | margin: 20px auto; 147 | text-align: left; 148 | font-size: 0.55em; 149 | font-family: monospace; 150 | line-height: 1.2em; 151 | word-wrap: break-word; 152 | box-shadow: 0px 0px 6px rgba(0, 0, 0, 0.3); } 153 | 154 | .reveal code { 155 | font-family: monospace; 156 | text-transform: none; } 157 | 158 | .reveal pre code { 159 | display: block; 160 | padding: 5px; 161 | overflow: auto; 162 | max-height: 400px; 163 | word-wrap: normal; } 164 | 165 | .reveal table { 166 | margin: auto; 167 | border-collapse: collapse; 168 | border-spacing: 0; } 169 | 170 | .reveal table th { 171 | font-weight: bold; } 172 | 173 | .reveal table th, 174 | .reveal table td { 175 | text-align: left; 176 | padding: 0.2em 0.5em 0.2em 0.5em; 177 | border-bottom: 1px solid; } 178 | 179 | .reveal table th[align="center"], 180 | .reveal table td[align="center"] { 181 | text-align: center; } 182 | 183 | .reveal table th[align="right"], 184 | .reveal table td[align="right"] { 185 | text-align: right; } 186 | 187 | .reveal table tbody tr:last-child th, 188 | .reveal table tbody tr:last-child td { 189 | border-bottom: none; } 190 | 191 | .reveal sup { 192 | vertical-align: super; } 193 | 194 | .reveal sub { 195 | vertical-align: sub; } 196 | 197 | .reveal small { 198 | display: inline-block; 199 | font-size: 0.6em; 200 | line-height: 1.2em; 201 | vertical-align: top; } 202 | 203 | .reveal small * { 204 | vertical-align: top; } 205 | 206 | /********************************************* 207 | * LINKS 208 | *********************************************/ 209 | .reveal a { 210 | color: #42affa; 211 | text-decoration: none; 212 | -webkit-transition: color .15s ease; 213 | -moz-transition: color .15s ease; 214 | transition: color .15s ease; } 215 | 216 | .reveal a:hover { 217 | color: #8dcffc; 218 | text-shadow: none; 219 | border: none; } 220 | 221 | .reveal .roll span:after { 222 | color: #fff; 223 | background: #068de9; } 224 | 225 | /********************************************* 226 | * IMAGES 227 | *********************************************/ 228 | .reveal section img { 229 | margin: 15px 0px; 230 | box-shadow: 0 0 10px rgba(0, 0, 0, 0.15); } 231 | 232 | .reveal section img.plain { 233 | border: 0; 234 | box-shadow: none; } 235 | 236 | .reveal a img { 237 | -webkit-transition: all .15s linear; 238 | -moz-transition: all .15s linear; 239 | transition: all .15s linear; } 240 | 241 | .reveal a:hover img { 242 | background: rgba(255, 255, 255, 0.2); 243 | border-color: #42affa; 244 | box-shadow: 0 0 20px rgba(0, 0, 0, 0.55); } 245 | 246 | /********************************************* 247 | * NAVIGATION CONTROLS 248 | *********************************************/ 249 | .reveal .controls { 250 | color: #42affa; } 251 | 252 | /********************************************* 253 | * PROGRESS BAR 254 | *********************************************/ 255 | .reveal .progress { 256 | background: rgba(0, 0, 0, 0.2); 257 | color: #42affa; } 258 | 259 | .reveal .progress span { 260 | -webkit-transition: width 800ms cubic-bezier(0.26, 0.86, 0.44, 0.985); 261 | -moz-transition: width 800ms cubic-bezier(0.26, 0.86, 0.44, 0.985); 262 | transition: width 800ms cubic-bezier(0.26, 0.86, 0.44, 0.985); } 263 | 264 | .reveal .footer{ 265 | background-image: url("imgs/appsecco-logo.png"); 266 | position:absolute; 267 | bottom: 50%; 268 | left: -50%; } 269 | -------------------------------------------------------------------------------- /imgs/accenture.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/accenture.jpg -------------------------------------------------------------------------------- /imgs/appsecco.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/appsecco.png -------------------------------------------------------------------------------- /imgs/appsecco_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/appsecco_logo.png -------------------------------------------------------------------------------- /imgs/cdn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/cdn.png -------------------------------------------------------------------------------- /imgs/censys.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/censys.gif -------------------------------------------------------------------------------- /imgs/censys.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/censys.png -------------------------------------------------------------------------------- /imgs/censys_script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/censys_script.png -------------------------------------------------------------------------------- /imgs/cheatsheet_full.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/cheatsheet_full.png -------------------------------------------------------------------------------- /imgs/commit.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/commit.jpg -------------------------------------------------------------------------------- /imgs/comparision.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/comparision.png -------------------------------------------------------------------------------- /imgs/cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/cover.png -------------------------------------------------------------------------------- /imgs/crtsh.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/crtsh.png -------------------------------------------------------------------------------- /imgs/crtsh_script.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/crtsh_script.jpg -------------------------------------------------------------------------------- /imgs/csp.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/csp.gif -------------------------------------------------------------------------------- /imgs/csp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/csp.png -------------------------------------------------------------------------------- /imgs/delo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/delo.jpg -------------------------------------------------------------------------------- /imgs/domain_types.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/domain_types.png -------------------------------------------------------------------------------- /imgs/example_nsec3_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/example_nsec3_1.png -------------------------------------------------------------------------------- /imgs/example_nsec3_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/example_nsec3_2.png -------------------------------------------------------------------------------- /imgs/facebook_ct.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/facebook_ct.png -------------------------------------------------------------------------------- /imgs/fdns_enum.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/fdns_enum.png -------------------------------------------------------------------------------- /imgs/insecure_aws.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/insecure_aws.jpg -------------------------------------------------------------------------------- /imgs/insecure_aws2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/insecure_aws2.jpg -------------------------------------------------------------------------------- /imgs/issues.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/issues.jpg -------------------------------------------------------------------------------- /imgs/maps_tint.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/maps_tint.jpeg -------------------------------------------------------------------------------- /imgs/massdns.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/massdns.png -------------------------------------------------------------------------------- /imgs/nsec_zone_walk.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/nsec_zone_walk.png -------------------------------------------------------------------------------- /imgs/private_space.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/private_space.jpeg -------------------------------------------------------------------------------- /imgs/public_space.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/public_space.jpeg -------------------------------------------------------------------------------- /imgs/reversewhois.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/reversewhois.png -------------------------------------------------------------------------------- /imgs/spaces_finder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/spaces_finder.png -------------------------------------------------------------------------------- /imgs/spaces_pattern.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/spaces_pattern.jpeg -------------------------------------------------------------------------------- /imgs/speakerdeck.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/speakerdeck.png -------------------------------------------------------------------------------- /imgs/spf.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/spf.gif -------------------------------------------------------------------------------- /imgs/spf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/spf.png -------------------------------------------------------------------------------- /imgs/spf_sample.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/spf_sample.png -------------------------------------------------------------------------------- /imgs/video.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/video.png -------------------------------------------------------------------------------- /imgs/vt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/imgs/vt.png -------------------------------------------------------------------------------- /practical_recon.md: -------------------------------------------------------------------------------- 1 | 2 | ## Practical recon techniques for bug hunters & pen testers 3 | 4 |
5 | ![](imgs/appsecco_logo.png) 6 | #### Bharath Kumar 7 | 8 | #### LevelUp 0x02 | May 26th 2018 9 | 10 | 11 | 12 | 13 | 14 | ## About me 15 | 16 | - Bharath Kumar 17 | - Live from Bangalore, India 18 | - Security Engineer @[Appsecco](https://appsecco.com) 19 | - **O**ffensive **S**ecurity **C**ertified **P**rofessional(OSCP) 20 | 21 | 22 | 23 | ## Demo environment 24 | 25 | - Feel free to run the DNS & DNSSEC attacks **mentioned in this talk** against the following nameservers and domain names 26 | 27 | 28 |
**Nameservers**
29 | 30 | - **ns1.insecuredns.com** 31 | - **ns2.insecuredns.com** 32 | 33 |
**Domains**
34 | - **totallylegit.in** 35 | - **insecuredns.com** 36 | 37 | 38 | 39 | 40 | 41 | ## What is this talk about? 42 | 43 | - This talk is about practical recon techniques that are useful for bug bounty hunters and penetration testers 44 | - The objective of this talk is to cover exhaustive number of practical recon techniques, tools of trade and tips/tricks 45 | 46 | Note: By practical I mean that these techniques covered can actually be used during a security assessment. 47 | The talk will be crisp and concise. We demonstrate quick and effective ways to apply a technique in such a way that the audience can use them in their assessments right away 48 | 49 | 50 | 51 | ## WHAT IS RECONNAISSANCE? 52 | 53 | > **Reconnaissance is the act of gathering preliminary data or intelligence on your target.** The data is gathered in order to better plan for your attack. Reconnaissance can be performed actively or passively. 54 | 55 | 56 | ## What do we look for during recon? 57 | 58 | 1. Info to increase attack surface(domains, net blocks) 59 | 2. Credentials(email, passwords, API keys) 60 | 3. Sensitive information 61 | 4. Infrastructure details 62 | 63 | 64 | 65 | ## Enumerating domains 66 | 67 | > The objective is to find/correlate all domain names owned by a single entity of our interest. 68 | 69 | 70 | ## Types of domain correlation 71 | 72 | ![test](imgs/domain_types.png) 73 | 74 | 75 | https://0xpatrik.com/asset-discovery/ 76 | 77 | 78 | 79 | ## What is sub-domain enumeration? 80 | 81 | > Sub-domain enumeration is the process of finding subdomains for one or more domain(s). 82 | 83 | 84 | ## Using popular search engines 85 | 86 | - Search engines like Google and Bing supports various advanced search operators to refine search queries. 87 | - `site:` is helpful in doing vertical domain correlation(sub-domains) 88 | - `ip:` is helpful in doing horizontal domain correlation 89 | 90 | 91 | ## Using 3rd party information aggregators 92 | 93 | - **VirusTotal** runs its own passive DNS replication service, built by storing DNS resolutions performed when visiting URLs submitted by users. 94 | 95 | https://www.virustotal.com/#/home/search 96 | 97 | 98 | **A script that uses VirusTotal to find sub-domains** 99 | 100 | ![vt](imgs/vt.png) 101 | 102 | Script - https://git.io/vhqBF 103 | 104 | 105 | ## Quick tip 106 | 107 | - I like using shell functions to quickly perform some recon tasks 108 | 109 | ```bash 110 | find-subdomains-vt() 111 | { curl -s https://www.virustotal.com/ui/domains/$1/subdomains\?limit\=$2 | jq .data[].id; } 112 | ``` 113 | 114 | 115 | ## Using 3rd party information aggregators 116 | 117 | - [viewdns.info](https://viewdns.info) is a handy service for all the DNS and WHOIS related recon 118 | 119 | ![vt](imgs/reversewhois.png) 120 | 121 | 122 | ## Certificate Transparency 123 | 124 | - Under CT, a Certificate Authority(CA) will have to publish all SSL/TLS certificates they issue in a public log 125 | - Anyone can look through the CT logs and find certificates issued for a domain 126 | - Details of known CT log files - https://www.certificate-transparency.org/known-logs 127 | 128 | https://blog.appsecco.com/certificate-transparency-part-2-the-bright-side-c0b99ebf31a8 129 | 130 | 131 | ### Certificate Transparency - side effect 132 | 133 | - CT logs by design contain all the certificates issued by a participating CA for any given domain 134 | - By looking through the logs, **an attacker can gather a lot of information** about an organization’s infrastructure i.e. internal domains, email addresses in a **completely passive manner** 135 | 136 | https://blog.appsecco.com/certificate-transparency-part-3-the-dark-side-9d401809b025 137 | 138 | 139 | ### Searching through CT logs 140 | 141 | - There are various search engines that collect the CT logs and let’s anyone search through them 142 | 1. https://crt.sh/ 143 | 2. https://censys.io/ 144 | 3. https://developers.facebook.com/tools/ct/ 145 | 4. https://google.com/transparencyreport/https/ct/ 146 | 147 | 148 | **A script that searches SSL/TLS certificates issued for a domain using crt.sh** 149 | 150 | ![certificate-lookup](imgs/crtsh.png) 151 | 152 | Script - https://git.io/vhqRd 153 | 154 | 155 | 156 | ### Keeping track of an organisation's sub-domains 157 | 158 | ![fb-subscribe](imgs/facebook_ct.png) 159 | 160 | https://developers.facebook.com/tools/ct/ 161 | 162 | 163 | 164 | ### Downside of CT for recon 165 | 166 | - CT logs are append-only. There is no way to delete an existing entry 167 | - The domain names found in the CT logs may not exist anymore and thus they can’t be resolved to an IP address 168 | 169 | https://blog.appsecco.com/a-penetration-testers-guide-to-sub-domain-enumeration-7d842d5570f6 170 | 171 | 172 | ### CT logs + massdns 173 | 174 | - You can use tools like [massdns](https://github.com/blechschmidt/massdns) along with CT logs script to quickly identify resolvable domain names. 175 | 176 | ```bash 177 | python3 ct.py example.com | ./bin/massdns -r resolvers.txt -t A -a -o -w results.txt - 178 | ``` 179 | 180 | ![ctlogs-massdns](imgs/massdns.png) 181 | 182 | 183 | ## Using certspotter 184 | 185 | ``` 186 | find-cert() 187 | { curl -s https://certspotter.com/api/v0/certs?domain=$1 | jq -c '.[].dns_names' | grep -o '"[^"]\+"'; } 188 | ``` 189 | 190 | 191 | ## Using certdb.com 192 | 193 | - While `crt.sh` gets the data from CT logs only where "legit" CA submit the certs to a log; CertDB is based on the scanning the IPv4 segment, domains and "finding & analyzing" all the certificates 194 | 195 | ``` 196 | curl -L -sd "api_key=API-KEY&q=Organization:\"tesla\"&response_type=3" -X POST https://certdb.com/api | tr , '\n' 197 | ``` 198 | https://certdb.com 199 | 200 | 201 | ### Finding vulnerable CMS using CT 202 | 203 | - When setting up some CMSs like Wordpress, Joomla and others, there is a window of time where the installer has no form of authentication 204 | - If the domain supports HTTPS it will end up on a CT log(sometimes in near real time) 205 | - If an attacker can search through CT Logs and find such a web application without authentication then he/she can take over the server 206 | 207 | 208 | ### Finding vulnerable CMS using CT 209 | 210 | - This attack has been demonstrated by [Hanno Böck at Defcon 25](https://www.golem.de/news/certificate-transparency-hacking-web-applications-before-they-are-installed-1707-129172.html) 211 | - He claimed to have found 5,000 WordPress installations using CT logs over a period of 3 months that he could have potentially taken over 212 | - HD Moore also discussed this technique in his [talk at BSidesLV 2017](https://github.com/fathom6/2017-BSidesLV-Modern-Recon) 213 | 214 | 215 | ## Censys.io 216 | 217 | - Censys aggregates SSL certificates that are a result of SSL scans on IPv4 address space and also from Certificate Transparency (CT) logs 218 | - This is a good source of domains and also email addresses 219 | 220 | https://0xpatrik.com/censys-guide/ 221 | 222 | 223 | **Extracting domains/emails from SSL/TLS certs using censys** 224 | 225 | ![censys](imgs/censys.gif) 226 | https://github.com/0xbharath/censys-enumeration 227 | 228 | 229 | ## Content Security Policy(CSP) 230 | 231 | - Content Security Policy(CSP) defines the `Content-Security-Policy` HTTP header, which allows us to create a whitelist of sources of trusted content, and instructs the browser to only execute or render resources from those sources 232 | - So basically, Content-Security-Policy header will list a bunch of sources(domains) that might be of interest to us as an attackers. 233 | 234 | 235 | **Extract domains from CSP headers** 236 | ![csp-extract](imgs/csp.gif) 237 | https://github.com/0xbharath/domains-from-csp 238 | 239 | 240 | ![cdn](imgs/cdn.png) 241 | 242 | https://justi.cz/security/2018/05/23/cdn-tar-oops.html 243 | 244 | 245 | ## Sender Policy Framework 246 | 247 | - A Sender Policy Framework(SPF) record and is used to indicate to recieving mail exchanges which hosts are authorized to send mail for a given domain 248 | - Simply put, an SPF record lists all the hosts that are authorised send emails on behalf of a domain 249 | 250 | ![spf](imgs/spf_sample.png) 251 | 252 | 253 | **Extract net blocks/domains from SPF record** 254 | 255 | ![csp-extract](imgs/spf.gif) 256 | https://github.com/0xbharath/assets-from-spf 257 | 258 | 259 | ### Domain enumeration in DNSSEC 260 | 261 | **Authenticated Denial of Existence(RFC 7129)** 262 | 263 | > In DNS, when client queries for a non-existent domain, the server must deny the existence of that domain. It is harder to do that in DNSSEC due to cryptographic signing. 264 | 265 | 266 | ### Zone walking NSEC - LDNS 267 | 268 | - The `ldns-walk`(part of `ldnsutils`) can be used to zone walk DNSSEC signed zone that uses NSEC. 269 | 270 | ```bash 271 | # zone walking with ldnsutils 272 | $ ldns-walk iana.org 273 | iana.org. iana.org. A NS SOA MX TXT AAAA RRSIG NSEC DNSKEY 274 | api.iana.org. CNAME RRSIG NSEC 275 | app.iana.org. CNAME RRSIG NSEC 276 | autodiscover.iana.org. CNAME RRSIG NSEC 277 | beta.iana.org. CNAME RRSIG NSEC 278 | data.iana.org. CNAME RRSIG NSEC 279 | dev.iana.org. CNAME RRSIG NSEC 280 | ftp.iana.org. CNAME RRSIG NSEC 281 | ^C 282 | ``` 283 | 284 | 285 | ### Installing ldnsutils 286 | 287 | ```bash 288 | # On Debian/Ubuntu 289 | $ sudo apt-get install ldnsutils 290 | ``` 291 | 292 | ```bash 293 | # On Redhat/CentOS 294 | $ sudo yum install ldns 295 | # You may need to do 296 | $ sudo yum install -y epel-release 297 | ``` 298 | 299 | 300 | ### NSEC3 301 | 302 | - The NSEC3 record is like an NSEC record, but, NSEC3 provides a signed gap of **hashes of domain names**. 303 | - Returning hashes was intended to prevent zone enumeration(or make it expensive). 304 | 305 | ``` 306 | 231SPNAMH63428R68U7BV359PFPJI2FC.example.com. NSEC3 1 0 3 ABCDEF 307 | NKDO8UKT2STOL6EJRD1EKVD1BQ2688DM A NS SOA TXT AAAA RRSIG DNSKEY NSEC3PARAM 308 | ``` 309 | 310 | ``` 311 | NKDO8UKT2STOL6EJRD1EKVD1BQ2688DM.example.com. NSEC3 1 0 3 ABCDEF 312 | 231SPNAMH63428R68U7BV359PFPJI2FC A TXT AAAA RRSIG 313 | ``` 314 | 315 | 316 | ### Zone walking NSEC3 317 | 318 | - An attacker can collect all the sub-domain hashes and crack the hashes offline 319 | - Tools like *[nsec3walker](https://dnscurve.org/nsec3walker.html)*, *[nsec3map](https://github.com/anonion0/nsec3map)* help us automate collecting NSEC3 hases and cracking the hashes 320 | 321 | 322 | ### Zone walking NSEC3 323 | 324 | **Zone walking NSEC3 protected zone using *nsec3walker*:** 325 | 326 | ```bash 327 | # Collect NSEC3 hashes of a domain 328 | $ ./collect insecuredns.com > insecuredns.com.collect 329 | ``` 330 | 331 | ```bash 332 | # Undo the hashing, expose the sub-domain information. 333 | $ ./unhash < insecuredns.com.collect > insecuredns.com.unhash 334 | ``` 335 | 336 | 337 | ### Zone walking NSEC3 338 | 339 | ```bash 340 | # Checking the number of sucessfully cracked sub-domain hashes 341 | $ cat icann.org.unhash | grep "icann" | wc -l 342 | 182 343 | ``` 344 | 345 | ```bash 346 | # Listing only the sub-domain part from the unhashed data 347 | $ cat icann.org.unhash | grep "icann" | awk '{print $2;}' 348 | del.icann.org. 349 | access.icann.org. 350 | charts.icann.org. 351 | communications.icann.org. 352 | fellowship.icann.org. 353 | files.icann.org. 354 | forms.icann.org. 355 | mail.icann.org. 356 | maintenance.icann.org. 357 | new.icann.org. 358 | public.icann.org. 359 | research.icann.org. 360 | rs.icann.org. 361 | stream.icann.org. 362 | tally.icann.org. 363 | video.icann.org. 364 | mm.icann.org. 365 | ns.icann.org. 366 | qa.icann.org. 367 | ist.icann.org. 368 | aso.icann.org. 369 | cai.icann.org. 370 | dev.icann.org. 371 | exc.icann.org. 372 | jss.icann.org. 373 | mex.icann.org. 374 | rrs.icann.org. 375 | syd.icann.org. 376 | upk.icann.org. 377 | vip.icann.org. 378 | crm.icann.org. 379 | dns.icann.org. 380 | liao.icann.org. 381 | redis.icann.org. 382 | svn.icann.org. 383 | admin.icann.org. 384 | orbis.icann.org. 385 | jira.icann.org. 386 | omblog.icann.org. 387 | pptr.icann.org. 388 | splunk.icann.org. 389 | nomcom.icann.org. 390 | rssac.icann.org. 391 | sftp.icann.org. 392 | netscan.icann.org. 393 | ``` 394 | 395 | 396 | ## Installing nsec3walker 397 | 398 | - Installation instructions are available at https://dnscurve.org/nsec3walker.html 399 | - I used following commands to install `nsec3walker` on Ubuntu 16.04. 400 | - `build-essential` package is a prerequisite. 401 | 402 | ```bash 403 | # Installing nsec3walker 404 | $ wget https://dnscurve.org/nsec3walker-20101223.tar.gz 405 | $ tar -xzf nsec3walker-20101223.tar.gz 406 | $ cd nsec3walker-20101223 407 | $ make 408 | ``` 409 | 410 | 411 | 412 | ### Few things that changed with the advent of DevOps 413 | 414 | 1. Storage 415 | 2. Authentication 416 | 3. More and more code 417 | 4. CI/CD pipelines 418 | 419 | 420 | ## Cloud storage 421 | 422 | - Cloud storage has gotten inexpensive, easy to setup and gained popularity 423 | - Especially object/block storage 424 | - Object storage is ideal for storing static, unstructured data like audio, video, documents, images and logs as well as large amounts of text. 425 | 426 | 1. AWS S3 buckets 427 | 2. Digital Ocean Spaces 428 | 429 | 430 | ## What's the catch with object storage? 431 | 432 | - Due to the nature of object storage, it is a treasure trove of information from an attacker/penetration tester perspective. 433 | - In our experience, given an chance, users will store anything on third-party services, from their passwords in plain text files to pictures of their pets. 434 | 435 | 436 | ## Amazon S3 buckets 437 | 438 | - AWS S3 is an object storage service by Amazon 439 | - Buckets allow users to store and serve large amounts of data. 440 | 441 | 442 | **Attack on Accenture(Sep, 2017)- AWS S3 buckets as attack surface** 443 | 444 | 445 | ![accenture-attack](imgs/accenture.jpg) 446 | 447 | https://www.upguard.com/breaches/cloud-leak-accenture 448 | 449 | 450 | **AWS S3 buckets as attack surface - The trend** 451 | 452 | 453 | ![insecure-aws](imgs/insecure_aws.jpg) 454 | 455 | 456 | **AWS S3 buckets as attack surface - The trend** 457 | 458 | 459 | ![insecure-aws](imgs/insecure_aws2.jpg) 460 | 461 | 462 | ### Hunting for publicly accessible S3 buckets 463 | 464 | - Users can store Files(Objects) in a Bucket 465 | - Each Bucket will get an unique, predictable URL and each file in a Bucket will get an unique URL as well 466 | - There are Access controls mechanisms available at both Bucket and Object level. 467 | 468 | 469 | ### Hunting for publicly accessible S3 buckets 470 | 471 | - Good old Google dorks 472 | 473 | ``` 474 | site:s3.amazonaws.com file:pdf 475 | ``` 476 | 477 | ``` 478 | site:s3.amazonaws.com password 479 | ``` 480 | 481 | 482 | ### Hunting for publicly accessible S3 buckets 483 | 484 | - As buckets have predictable URL it is trivial to do a dictionary based attack 485 | - Following tools help run a dictionary attack to identify S3 buckets 486 | 487 | 1. [AWSBucketDump](https://github.com/jordanpotti/AWSBucketDump) 488 | 2. [Slurp](https://github.com/bbb31/slurp) 489 | 490 | 491 | ### Digital Ocean Spaces 492 | 493 | - Spaces is an object storage service by DigitalOcean 494 | - It is similar to AWS S3 buckets 495 | - Spaces API aims to be interoperable with Amazon’s AWS S3 API. 496 | 497 | 498 | ### Spaces URL pattern 499 | 500 | - Users can store Files in a “Space” 501 | - Each Space will get an unique, predictable URL 502 | - Each file in a Space will get an unique URL as well. 503 | - Access controls mechanisms are available at Space and file level. 504 | 505 | ![spaces-pattern](imgs/spaces_pattern.jpeg) 506 | 507 | 508 | ### Hunting for publicly accessible S3 buckets 509 | 510 | 511 | A Space is typically considered “public” if any user can list the contents of the Space 512 | 513 | 514 |
public_space
515 | 516 | 517 | 518 | A Space is typically considered “private” if the Space’s contents can only be listed or written by certain users 519 | 520 | 521 | ![private-space](imgs/private_space.jpeg) 522 | 523 | 524 | ## Spaces finder 525 | 526 | - Spaces API is interoperable with Amazon’s S3 API, we tweaked [AWSBucketDump](https://github.com/jordanpotti/AWSBucketDump) to work with DO Spaces 527 | - *Spaces finder* is a tool that can look for publicly accessible DO Spaces using a wordlist, list all the accessible files on a public Space and download the files. 528 | 529 | https://github.com/appsecco/spaces-finder 530 | 531 | 532 | ## Spaces finder in action 533 | 534 | ![spaces-finder](imgs/spaces_finder.png) 535 | https://github.com/appsecco/spaces-finder 536 | 537 | 538 | 539 | ## Authentication 540 | 541 | - With almost every service exposing an API, keys have become critical in authenticating 542 | - API keys are treated as keys to the kingdom 543 | - For applications, API keys tend to be achilles heel 544 | 545 | 546 | https://danielmiessler.com/blog/apis-2fas-achilles-heel/ 547 | 548 | 549 | 550 | ## Code repos for recon 551 | 552 | - Code repos are a treasure trove during recon 553 | - Code repos can reveal a lot from credentials, potential vulnerabilities to infrastructure details 554 | 555 | ![del-github](imgs/delo.jpg) 556 | 557 | 558 | ## Github for recon 559 | 560 | - GitHub is an extremely popular version control and collaboration platform 561 | - Code repos on github tend to have all sorts of sensitive information 562 | - Github also has a powerful search feature with advanced operators 563 | - Github has a very well designed REST API 564 | - [edoverflow](https://twitter.com/edoverflow) has a neat little guide on [GitHub for Bug Bounty Hunters](https://gist.github.com/EdOverflow/922549f610b258f459b219a32f92d10b) 565 | 566 | 567 | ## Things to focus on in Github 568 | 569 | There are 4 main sections to look out for here. 570 | 571 | - Repositories 572 | - Code 573 | - Commits(My fav!) 574 | - Issues 575 | 576 | 577 | ![commit-leak](imgs/commit.jpg) 578 | 579 | 580 | ![issues-leak](imgs/issues.jpg) 581 | 582 | 583 | ## Mass Cloning on Github 584 | 585 | - You can ideally clone all the target organization's repos and analyze them locally 586 | - [GitHubCloner](https://github.com/mazen160/GithubCloner) by @mazen160 comes very handy to automate the process 587 | 588 | ``` 589 | $ python githubcloner.py --org organization -o /tmp/output 590 | ``` 591 | 592 | 593 | https://gist.github.com/EdOverflow/922549f610b258f459b219a32f92d10b 594 | 595 | 596 | 597 | ## Static code analysis 598 | 599 | - Once the repos are cloned, you can do a static code analysis 600 | - There are language specific tools to speed up and automate the process 601 | 602 | 1. [Brakeman](https://brakemanscanner.org/) for Ruby 603 | 2. [Bandit](https://github.com/openstack/bandit) for Python 604 | 605 | 606 | ## Finding secrets in code manually 607 | 608 | - Once you have the repos cloned. You can understand the code, language used and architecture 609 | - Start looking for keywords or patterns 610 | 611 | - API and key. (Get some more endpoints and find API keys.) 612 | - token 613 | - secret 614 | - vulnerable 615 | - http:// 616 | 617 | 618 | ## Finding secrets in code in automated fashion 619 | 620 | There are various tools available to find juicy information in source code. 621 | 622 | 1. [Truffle Hog](https://github.com/dxa4481/truffleHog) 623 | 2. [git-all-secrets](https://github.com/anshumanbh/git-all-secrets) 624 | 625 | 626 | ## Github dorks 627 | 628 | - Github dorks are the new Google dorks 629 | - Github search is quite powerful feature & can be used to find sensitive data on the repos 630 | 631 | - **A collection of Github dorks** https://github.com/techgaun/github-dorks/blob/master/github-dorks.txt 632 | 633 | - **Tool to run Github dorks against a repo** https://github.com/techgaun/github-dorks 634 | 635 | 636 | 637 | ### Passive recon using public datasets 638 | 639 | - There are various projects that gather Internet wide scan data and make it available to researchers and the security community. 640 | - This data includes port scans, DNS data, SSL/TLS cert data and even data breach dumps that they can find. 641 | - Find your needle in the haystack. 642 | 643 | 644 | ## Why use public data sets for recon? 645 | 646 | - To reduce dependency on 3rd party APIs and services 647 | - To reduce active probing of target infrastructure 648 | - More the sources better the coverage 649 | - Build your own recon platforms 650 | 651 | 652 | ## Let's look at some public datasets 653 | 654 | | Name | Description | Price | 655 | | ------------- |:-------------:| -----:| 656 | | [Sonar](https://scans.io) | FDNS, RDNS, UDP, TCP, TLS, HTTP, HTTPS scan data | FREE | 657 | | [Censys.io](https://www.censys.io/)| TCP, TLS, HTTP, HTTPS scan data | FREE | 658 | | [CT](https://www.certificate-transparency.org/)| TLS | FREE | 659 | 660 |
661 | 662 | https://github.com/fathom6/inetdata 663 | 664 | 665 | 666 | ## Let's look at some public datasets 667 | 668 | | Name | Description | Price | 669 | | ------------- |:-------------:| -----:| 670 | | [CZDS](https://czds.icann.org/) | zone files for "new" global TLDs | FREE | 671 | | [ARIN](https://www.arin.net) | American IP registry information | FREE | 672 | | [CAIDA PFX2AS IPv4](http://data.caida.org/datasets/routing/routeviews-prefix2as) | Daily snapshots of ASN to IPv4 mappings | FREE | 673 | 674 | 675 | ## Let's look at some public datasets 676 | 677 | | Name | Description | Price | 678 | | ------------- |:-------------:| -----:| 679 | | [US Gov](https://raw.githubusercontent.com/GSA/data/gh-pages/dotgov-domains/current-full.csv) | US government domain names | FREE | 680 | | [UK Gov](https://www.gov.uk/government/publications/list-of-gov-uk-domain-names) | UK government domain names | FREE | 681 | | [RIR Delegations](http://ftp.arin.net/pub/stats/) | Regional IP allocations | FREE | 682 | 683 |
684 | 685 | https://github.com/fathom6/inetdata 686 | 687 | 688 | 689 | ## Let's look at some public datasets 690 | 691 | | Name | Description | Price | 692 | | ------------- |:-------------:| -----:| 693 | | [PremiumDrops](http://premiumdrops.com/) | DNS zone files for com/net/info/org/biz/xxx/sk/us TLDs | $24.95/mo | 694 | | [WWWS.io](https://wwws.io/) | Domains across many TLDs (~198m) | $9/mo | 695 | | [WhoisXMLAPI.com](https://WhoisXMLAPI.com/) | New domain whois data | $109/mo | 696 | 697 | 698 | https://github.com/fathom6/inetdata 699 | 700 | 701 | 702 | ### Rapid7 Forward DNS dataset 703 | 704 | - Rapid7 publishes its Forward DNS study/dataset on `scans.io` project(it's a massive dataset, 20+ GB compressed & 300+ GB uncompressed) 705 | - This dataset aims to discover all domains found on the Internet 706 | 707 | 708 | ## Hunting sub-domain in FDNS dataset 709 | 710 | - The data format is a gzip-compressed JSON file so we can use `jq` utility to extract sub-domains of a specific domain: 711 | 712 | ``` 713 | curl --silent -L https://opendata.rapid7.com/sonar.fdns_v2/2018-04-21-1524297601-fdns_any.json.gz | pigz -dc | head -n 10 | jq . 714 | ``` 715 | 716 | ```bash 717 | cat 2018-04-21-1524297601-fdns_any.json.gz | pigz -dc | grep "\.example\.com" | jq .name 718 | ``` 719 | 720 | 721 | https://opendata.rapid7.com/about/ 722 | 723 | 724 | 725 | ## Hunting sub-domain in FDNS dataset 726 | 727 | ![fdns-enum](imgs/fdns_enum.png) 728 | 729 | 730 | 731 |
cheatsheet_full
732 | https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration 733 | 734 | 735 | 736 | ## ICANN.ORG subdomains 737 | 738 | Number of **unique, resolvable sub-domains** each enumeration technique found independently against icann.org 739 | 740 | 741 |
icann-subdomains
742 | 743 | 744 | 745 | 746 | ## TALK MATERIAL 747 | 748 | https://github.com/appsecco/practical-recon-levelup0x02 749 | 750 | 751 | 752 | ## Take away 753 | 754 | **A gitbook on sub-domain enumeration** 755 | 756 | https://appsecco.com/books/subdomain-enumeration/ 757 | 758 | 759 | 760 | ## References 761 | 762 | 763 | - https://www.certificate-transparency.org/ 764 | - https://www.cloudflare.com/dns/dnssec/how-dnssec-works/ 765 | - https://www.cloudflare.com/dns/dnssec/dnssec-complexities-and-considerations/ 766 | - http://info.menandmice.com/blog/bid/73645/Take-your-DNSSEC-with-a-grain-of-salt 767 | - https://github.com/rapid7/sonar/wiki/Forward-DNS 768 | 769 | 770 | 771 | 772 | 773 |
774 | 775 |
776 | 777 | 778 | 779 | ### Thanks 780 | 781 | [@0xbharath](http://twitter.com/0xbharath) 782 | -------------------------------------------------------------------------------- /practical_recon.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/appsecco/practical-recon-levelup0x02/bd61338fc222de0f3d93f4cc77dafec77fc33889/practical_recon.pdf --------------------------------------------------------------------------------