├── .gitignore ├── .gitmodules ├── LICENSE.md ├── README.md └── docs ├── Makefile ├── make.bat └── source ├── conf.py ├── configure_minion.rst ├── contribute_to_minion.rst ├── developing_plugins.rst ├── images ├── .DS_Store ├── admin-groups-editor.png ├── admin-groups.png ├── admin-plans.png ├── admin-plugins.png ├── admin-sites.png ├── admin-users.png ├── dashboard-groups.png ├── dashboard.png ├── login-oauth.png ├── login.png ├── scan-detail.png └── scan-issue.png ├── include ├── frontend.json ├── minion_repos.rst └── plugin_repos.rst ├── index.rst ├── inside_minion.rst ├── install_guide.rst ├── install_plugins.rst ├── persona.rst └── using_minion.rst /.gitignore: -------------------------------------------------------------------------------- 1 | .coverage 2 | .coveragerc 3 | celerybeat-schedule 4 | pip-log.txt 5 | *.swp 6 | *.elc 7 | *.pyc 8 | *~ 9 | build/* 10 | dist/* 11 | sdist/* 12 | docs/_static/* 13 | docs/_templates/* 14 | docs/build/* 15 | docs/_build/* 16 | *egg-info 17 | *.egg 18 | .DS_Store 19 | docs/.DS_Store 20 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "docs/source/theme"] 2 | path = docs/source/theme 3 | url = https://github.com/yeukhon/mozilla-sphinx-theme.git 4 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Mozilla Public License, version 2.0 2 | 3 | 1. Definitions 4 | 5 | 1.1. "Contributor" 6 | 7 | means each individual or legal entity that creates, contributes to the 8 | creation of, or owns Covered Software. 9 | 10 | 1.2. "Contributor Version" 11 | 12 | means the combination of the Contributions of others (if any) used by a 13 | Contributor and that particular Contributor's Contribution. 14 | 15 | 1.3. "Contribution" 16 | 17 | means Covered Software of a particular Contributor. 18 | 19 | 1.4. "Covered Software" 20 | 21 | means Source Code Form to which the initial Contributor has attached the 22 | notice in Exhibit A, the Executable Form of such Source Code Form, and 23 | Modifications of such Source Code Form, in each case including portions 24 | thereof. 25 | 26 | 1.5. "Incompatible With Secondary Licenses" 27 | means 28 | 29 | a. that the initial Contributor has attached the notice described in 30 | Exhibit B to the Covered Software; or 31 | 32 | b. that the Covered Software was made available under the terms of 33 | version 1.1 or earlier of the License, but not also under the terms of 34 | a Secondary License. 35 | 36 | 1.6. "Executable Form" 37 | 38 | means any form of the work other than Source Code Form. 39 | 40 | 1.7. "Larger Work" 41 | 42 | means a work that combines Covered Software with other material, in a 43 | separate file or files, that is not Covered Software. 44 | 45 | 1.8. "License" 46 | 47 | means this document. 48 | 49 | 1.9. "Licensable" 50 | 51 | means having the right to grant, to the maximum extent possible, whether 52 | at the time of the initial grant or subsequently, any and all of the 53 | rights conveyed by this License. 54 | 55 | 1.10. "Modifications" 56 | 57 | means any of the following: 58 | 59 | a. any file in Source Code Form that results from an addition to, 60 | deletion from, or modification of the contents of Covered Software; or 61 | 62 | b. any new file in Source Code Form that contains any Covered Software. 63 | 64 | 1.11. "Patent Claims" of a Contributor 65 | 66 | means any patent claim(s), including without limitation, method, 67 | process, and apparatus claims, in any patent Licensable by such 68 | Contributor that would be infringed, but for the grant of the License, 69 | by the making, using, selling, offering for sale, having made, import, 70 | or transfer of either its Contributions or its Contributor Version. 71 | 72 | 1.12. "Secondary License" 73 | 74 | means either the GNU General Public License, Version 2.0, the GNU Lesser 75 | General Public License, Version 2.1, the GNU Affero General Public 76 | License, Version 3.0, or any later versions of those licenses. 77 | 78 | 1.13. "Source Code Form" 79 | 80 | means the form of the work preferred for making modifications. 81 | 82 | 1.14. "You" (or "Your") 83 | 84 | means an individual or a legal entity exercising rights under this 85 | License. For legal entities, "You" includes any entity that controls, is 86 | controlled by, or is under common control with You. For purposes of this 87 | definition, "control" means (a) the power, direct or indirect, to cause 88 | the direction or management of such entity, whether by contract or 89 | otherwise, or (b) ownership of more than fifty percent (50%) of the 90 | outstanding shares or beneficial ownership of such entity. 91 | 92 | 93 | 2. License Grants and Conditions 94 | 95 | 2.1. Grants 96 | 97 | Each Contributor hereby grants You a world-wide, royalty-free, 98 | non-exclusive license: 99 | 100 | a. under intellectual property rights (other than patent or trademark) 101 | Licensable by such Contributor to use, reproduce, make available, 102 | modify, display, perform, distribute, and otherwise exploit its 103 | Contributions, either on an unmodified basis, with Modifications, or 104 | as part of a Larger Work; and 105 | 106 | b. under Patent Claims of such Contributor to make, use, sell, offer for 107 | sale, have made, import, and otherwise transfer either its 108 | Contributions or its Contributor Version. 109 | 110 | 2.2. Effective Date 111 | 112 | The licenses granted in Section 2.1 with respect to any Contribution 113 | become effective for each Contribution on the date the Contributor first 114 | distributes such Contribution. 115 | 116 | 2.3. Limitations on Grant Scope 117 | 118 | The licenses granted in this Section 2 are the only rights granted under 119 | this License. No additional rights or licenses will be implied from the 120 | distribution or licensing of Covered Software under this License. 121 | Notwithstanding Section 2.1(b) above, no patent license is granted by a 122 | Contributor: 123 | 124 | a. for any code that a Contributor has removed from Covered Software; or 125 | 126 | b. for infringements caused by: (i) Your and any other third party's 127 | modifications of Covered Software, or (ii) the combination of its 128 | Contributions with other software (except as part of its Contributor 129 | Version); or 130 | 131 | c. under Patent Claims infringed by Covered Software in the absence of 132 | its Contributions. 133 | 134 | This License does not grant any rights in the trademarks, service marks, 135 | or logos of any Contributor (except as may be necessary to comply with 136 | the notice requirements in Section 3.4). 137 | 138 | 2.4. Subsequent Licenses 139 | 140 | No Contributor makes additional grants as a result of Your choice to 141 | distribute the Covered Software under a subsequent version of this 142 | License (see Section 10.2) or under the terms of a Secondary License (if 143 | permitted under the terms of Section 3.3). 144 | 145 | 2.5. Representation 146 | 147 | Each Contributor represents that the Contributor believes its 148 | Contributions are its original creation(s) or it has sufficient rights to 149 | grant the rights to its Contributions conveyed by this License. 150 | 151 | 2.6. Fair Use 152 | 153 | This License is not intended to limit any rights You have under 154 | applicable copyright doctrines of fair use, fair dealing, or other 155 | equivalents. 156 | 157 | 2.7. Conditions 158 | 159 | Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in 160 | Section 2.1. 161 | 162 | 163 | 3. Responsibilities 164 | 165 | 3.1. Distribution of Source Form 166 | 167 | All distribution of Covered Software in Source Code Form, including any 168 | Modifications that You create or to which You contribute, must be under 169 | the terms of this License. You must inform recipients that the Source 170 | Code Form of the Covered Software is governed by the terms of this 171 | License, and how they can obtain a copy of this License. You may not 172 | attempt to alter or restrict the recipients' rights in the Source Code 173 | Form. 174 | 175 | 3.2. Distribution of Executable Form 176 | 177 | If You distribute Covered Software in Executable Form then: 178 | 179 | a. such Covered Software must also be made available in Source Code Form, 180 | as described in Section 3.1, and You must inform recipients of the 181 | Executable Form how they can obtain a copy of such Source Code Form by 182 | reasonable means in a timely manner, at a charge no more than the cost 183 | of distribution to the recipient; and 184 | 185 | b. You may distribute such Executable Form under the terms of this 186 | License, or sublicense it under different terms, provided that the 187 | license for the Executable Form does not attempt to limit or alter the 188 | recipients' rights in the Source Code Form under this License. 189 | 190 | 3.3. Distribution of a Larger Work 191 | 192 | You may create and distribute a Larger Work under terms of Your choice, 193 | provided that You also comply with the requirements of this License for 194 | the Covered Software. If the Larger Work is a combination of Covered 195 | Software with a work governed by one or more Secondary Licenses, and the 196 | Covered Software is not Incompatible With Secondary Licenses, this 197 | License permits You to additionally distribute such Covered Software 198 | under the terms of such Secondary License(s), so that the recipient of 199 | the Larger Work may, at their option, further distribute the Covered 200 | Software under the terms of either this License or such Secondary 201 | License(s). 202 | 203 | 3.4. Notices 204 | 205 | You may not remove or alter the substance of any license notices 206 | (including copyright notices, patent notices, disclaimers of warranty, or 207 | limitations of liability) contained within the Source Code Form of the 208 | Covered Software, except that You may alter any license notices to the 209 | extent required to remedy known factual inaccuracies. 210 | 211 | 3.5. Application of Additional Terms 212 | 213 | You may choose to offer, and to charge a fee for, warranty, support, 214 | indemnity or liability obligations to one or more recipients of Covered 215 | Software. However, You may do so only on Your own behalf, and not on 216 | behalf of any Contributor. You must make it absolutely clear that any 217 | such warranty, support, indemnity, or liability obligation is offered by 218 | You alone, and You hereby agree to indemnify every Contributor for any 219 | liability incurred by such Contributor as a result of warranty, support, 220 | indemnity or liability terms You offer. You may include additional 221 | disclaimers of warranty and limitations of liability specific to any 222 | jurisdiction. 223 | 224 | 4. Inability to Comply Due to Statute or Regulation 225 | 226 | If it is impossible for You to comply with any of the terms of this License 227 | with respect to some or all of the Covered Software due to statute, 228 | judicial order, or regulation then You must: (a) comply with the terms of 229 | this License to the maximum extent possible; and (b) describe the 230 | limitations and the code they affect. Such description must be placed in a 231 | text file included with all distributions of the Covered Software under 232 | this License. Except to the extent prohibited by statute or regulation, 233 | such description must be sufficiently detailed for a recipient of ordinary 234 | skill to be able to understand it. 235 | 236 | 5. Termination 237 | 238 | 5.1. The rights granted under this License will terminate automatically if You 239 | fail to comply with any of its terms. However, if You become compliant, 240 | then the rights granted under this License from a particular Contributor 241 | are reinstated (a) provisionally, unless and until such Contributor 242 | explicitly and finally terminates Your grants, and (b) on an ongoing 243 | basis, if such Contributor fails to notify You of the non-compliance by 244 | some reasonable means prior to 60 days after You have come back into 245 | compliance. Moreover, Your grants from a particular Contributor are 246 | reinstated on an ongoing basis if such Contributor notifies You of the 247 | non-compliance by some reasonable means, this is the first time You have 248 | received notice of non-compliance with this License from such 249 | Contributor, and You become compliant prior to 30 days after Your receipt 250 | of the notice. 251 | 252 | 5.2. If You initiate litigation against any entity by asserting a patent 253 | infringement claim (excluding declaratory judgment actions, 254 | counter-claims, and cross-claims) alleging that a Contributor Version 255 | directly or indirectly infringes any patent, then the rights granted to 256 | You by any and all Contributors for the Covered Software under Section 257 | 2.1 of this License shall terminate. 258 | 259 | 5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user 260 | license agreements (excluding distributors and resellers) which have been 261 | validly granted by You or Your distributors under this License prior to 262 | termination shall survive termination. 263 | 264 | 6. Disclaimer of Warranty 265 | 266 | Covered Software is provided under this License on an "as is" basis, 267 | without warranty of any kind, either expressed, implied, or statutory, 268 | including, without limitation, warranties that the Covered Software is free 269 | of defects, merchantable, fit for a particular purpose or non-infringing. 270 | The entire risk as to the quality and performance of the Covered Software 271 | is with You. Should any Covered Software prove defective in any respect, 272 | You (not any Contributor) assume the cost of any necessary servicing, 273 | repair, or correction. This disclaimer of warranty constitutes an essential 274 | part of this License. No use of any Covered Software is authorized under 275 | this License except under this disclaimer. 276 | 277 | 7. Limitation of Liability 278 | 279 | Under no circumstances and under no legal theory, whether tort (including 280 | negligence), contract, or otherwise, shall any Contributor, or anyone who 281 | distributes Covered Software as permitted above, be liable to You for any 282 | direct, indirect, special, incidental, or consequential damages of any 283 | character including, without limitation, damages for lost profits, loss of 284 | goodwill, work stoppage, computer failure or malfunction, or any and all 285 | other commercial damages or losses, even if such party shall have been 286 | informed of the possibility of such damages. This limitation of liability 287 | shall not apply to liability for death or personal injury resulting from 288 | such party's negligence to the extent applicable law prohibits such 289 | limitation. Some jurisdictions do not allow the exclusion or limitation of 290 | incidental or consequential damages, so this exclusion and limitation may 291 | not apply to You. 292 | 293 | 8. Litigation 294 | 295 | Any litigation relating to this License may be brought only in the courts 296 | of a jurisdiction where the defendant maintains its principal place of 297 | business and such litigation shall be governed by laws of that 298 | jurisdiction, without reference to its conflict-of-law provisions. Nothing 299 | in this Section shall prevent a party's ability to bring cross-claims or 300 | counter-claims. 301 | 302 | 9. Miscellaneous 303 | 304 | This License represents the complete agreement concerning the subject 305 | matter hereof. If any provision of this License is held to be 306 | unenforceable, such provision shall be reformed only to the extent 307 | necessary to make it enforceable. Any law or regulation which provides that 308 | the language of a contract shall be construed against the drafter shall not 309 | be used to construe this License against a Contributor. 310 | 311 | 312 | 10. Versions of the License 313 | 314 | 10.1. New Versions 315 | 316 | Mozilla Foundation is the license steward. Except as provided in Section 317 | 10.3, no one other than the license steward has the right to modify or 318 | publish new versions of this License. Each version will be given a 319 | distinguishing version number. 320 | 321 | 10.2. Effect of New Versions 322 | 323 | You may distribute the Covered Software under the terms of the version 324 | of the License under which You originally received the Covered Software, 325 | or under the terms of any subsequent version published by the license 326 | steward. 327 | 328 | 10.3. Modified Versions 329 | 330 | If you create software not governed by this License, and you want to 331 | create a new license for such software, you may create and use a 332 | modified version of this License if you rename the license and remove 333 | any references to the name of the license steward (except to note that 334 | such modified license differs from this License). 335 | 336 | 10.4. Distributing Source Code Form that is Incompatible With Secondary 337 | Licenses If You choose to distribute Source Code Form that is 338 | Incompatible With Secondary Licenses under the terms of this version of 339 | the License, the notice described in Exhibit B of this License must be 340 | attached. 341 | 342 | Exhibit A - Source Code Form License Notice 343 | 344 | This Source Code Form is subject to the 345 | terms of the Mozilla Public License, v. 346 | 2.0. If a copy of the MPL was not 347 | distributed with this file, You can 348 | obtain one at 349 | http://mozilla.org/MPL/2.0/. 350 | 351 | If it is not possible or desirable to put the notice in a particular file, 352 | then You may include the notice in a location (such as a LICENSE file in a 353 | relevant directory) where a recipient would be likely to look for such a 354 | notice. 355 | 356 | You may add additional accurate notices of copyright ownership. 357 | 358 | Exhibit B - "Incompatible With Secondary Licenses" Notice 359 | 360 | This Source Code Form is "Incompatible 361 | With Secondary Licenses", as defined by 362 | the Mozilla Public License, v. 2.0. 363 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | About Minion 2 | ============ 3 | 4 | Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers. 5 | To do so, it enables developers to scan their projects using a friendly interface. 6 | 7 | For complete user and developer documentation see http://minion.readthedocs.org/en/latest/ 8 | 9 | About this project 10 | ------------------ 11 | 12 | This repository is an 'umbrella' project that simply ties together the separate projects that Minion needs. 13 | 14 | The following two projects are the bare minimum needed to get Minion up and running: 15 | 16 | * https://github.com/mozilla/minion-backend 17 | * https://github.com/mozilla/minion-frontend 18 | 19 | The following project helps developers setup their environment to get hacking on Minion: 20 | * https://github.com/mozilla/minion-vm 21 | 22 | The following projects are optional plugins for minion that add more functionality or wrap existing tools: 23 | 24 | * https://github.com/mozilla/minion-zap-plugin 25 | * https://github.com/mozilla/minion-ssl-plugin 26 | * https://github.com/mozilla/minion-skipfish-plugin 27 | * https://github.com/mozilla/minion-nmap-plugin 28 | 29 | License 30 | ------- 31 | This software is licensed under the MPL License. For more 32 | information, read the file ``LICENSE.md``. 33 | 34 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Minion.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Minion.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/Minion" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Minion" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source 10 | set I18NSPHINXOPTS=%SPHINXOPTS% source 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. xml to make Docutils-native XML files 37 | echo. pseudoxml to make pseudoxml-XML files for display purposes 38 | echo. linkcheck to check all external links for integrity 39 | echo. doctest to run all doctests embedded in the documentation if enabled 40 | goto end 41 | ) 42 | 43 | if "%1" == "clean" ( 44 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 45 | del /q /s %BUILDDIR%\* 46 | goto end 47 | ) 48 | 49 | 50 | %SPHINXBUILD% 2> nul 51 | if errorlevel 9009 ( 52 | echo. 53 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 54 | echo.installed, then set the SPHINXBUILD environment variable to point 55 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 56 | echo.may add the Sphinx directory to PATH. 57 | echo. 58 | echo.If you don't have Sphinx installed, grab it from 59 | echo.http://sphinx-doc.org/ 60 | exit /b 1 61 | ) 62 | 63 | if "%1" == "html" ( 64 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 68 | goto end 69 | ) 70 | 71 | if "%1" == "dirhtml" ( 72 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 76 | goto end 77 | ) 78 | 79 | if "%1" == "singlehtml" ( 80 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 84 | goto end 85 | ) 86 | 87 | if "%1" == "pickle" ( 88 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can process the pickle files. 92 | goto end 93 | ) 94 | 95 | if "%1" == "json" ( 96 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 97 | if errorlevel 1 exit /b 1 98 | echo. 99 | echo.Build finished; now you can process the JSON files. 100 | goto end 101 | ) 102 | 103 | if "%1" == "htmlhelp" ( 104 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 105 | if errorlevel 1 exit /b 1 106 | echo. 107 | echo.Build finished; now you can run HTML Help Workshop with the ^ 108 | .hhp project file in %BUILDDIR%/htmlhelp. 109 | goto end 110 | ) 111 | 112 | if "%1" == "qthelp" ( 113 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 114 | if errorlevel 1 exit /b 1 115 | echo. 116 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 117 | .qhcp project file in %BUILDDIR%/qthelp, like this: 118 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\Minion.qhcp 119 | echo.To view the help file: 120 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\Minion.ghc 121 | goto end 122 | ) 123 | 124 | if "%1" == "devhelp" ( 125 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished. 129 | goto end 130 | ) 131 | 132 | if "%1" == "epub" ( 133 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 137 | goto end 138 | ) 139 | 140 | if "%1" == "latex" ( 141 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 145 | goto end 146 | ) 147 | 148 | if "%1" == "latexpdf" ( 149 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 150 | cd %BUILDDIR%/latex 151 | make all-pdf 152 | cd %BUILDDIR%/.. 153 | echo. 154 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 155 | goto end 156 | ) 157 | 158 | if "%1" == "latexpdfja" ( 159 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 160 | cd %BUILDDIR%/latex 161 | make all-pdf-ja 162 | cd %BUILDDIR%/.. 163 | echo. 164 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 165 | goto end 166 | ) 167 | 168 | if "%1" == "text" ( 169 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 170 | if errorlevel 1 exit /b 1 171 | echo. 172 | echo.Build finished. The text files are in %BUILDDIR%/text. 173 | goto end 174 | ) 175 | 176 | if "%1" == "man" ( 177 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 178 | if errorlevel 1 exit /b 1 179 | echo. 180 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 181 | goto end 182 | ) 183 | 184 | if "%1" == "texinfo" ( 185 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 186 | if errorlevel 1 exit /b 1 187 | echo. 188 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 189 | goto end 190 | ) 191 | 192 | if "%1" == "gettext" ( 193 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 194 | if errorlevel 1 exit /b 1 195 | echo. 196 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 197 | goto end 198 | ) 199 | 200 | if "%1" == "changes" ( 201 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 202 | if errorlevel 1 exit /b 1 203 | echo. 204 | echo.The overview file is in %BUILDDIR%/changes. 205 | goto end 206 | ) 207 | 208 | if "%1" == "linkcheck" ( 209 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 210 | if errorlevel 1 exit /b 1 211 | echo. 212 | echo.Link check complete; look for any errors in the above output ^ 213 | or in %BUILDDIR%/linkcheck/output.txt. 214 | goto end 215 | ) 216 | 217 | if "%1" == "doctest" ( 218 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 219 | if errorlevel 1 exit /b 1 220 | echo. 221 | echo.Testing of doctests in the sources finished, look at the ^ 222 | results in %BUILDDIR%/doctest/output.txt. 223 | goto end 224 | ) 225 | 226 | if "%1" == "xml" ( 227 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 228 | if errorlevel 1 exit /b 1 229 | echo. 230 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 231 | goto end 232 | ) 233 | 234 | if "%1" == "pseudoxml" ( 235 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 236 | if errorlevel 1 exit /b 1 237 | echo. 238 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 239 | goto end 240 | ) 241 | 242 | :end 243 | -------------------------------------------------------------------------------- /docs/source/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Minion documentation build configuration file, created by 4 | # sphinx-quickstart on Mon Aug 5 10:53:21 2013. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | #import mozilla_sphinx_theme 15 | import sys, os 16 | 17 | # If extensions (or modules to document with autodoc) are in another directory, 18 | # add these directories to sys.path here. If the directory is relative to the 19 | # documentation root, use os.path.abspath to make it absolute, like shown here. 20 | #sys.path.insert(0, os.path.abspath('.')) 21 | 22 | # -- General configuration ----------------------------------------------------- 23 | 24 | # If your documentation needs a minimal Sphinx version, state it here. 25 | #needs_sphinx = '1.0' 26 | 27 | # Add any Sphinx extension module names here, as strings. They can be extensions 28 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 29 | extensions = [] 30 | 31 | # Add any paths that contain templates here, relative to this directory. 32 | templates_path = ['_templates'] 33 | 34 | # The suffix of source filenames. 35 | source_suffix = '.rst' 36 | 37 | # The encoding of source files. 38 | #source_encoding = 'utf-8-sig' 39 | 40 | # The master toctree document. 41 | master_doc = 'index' 42 | 43 | # General information about the project. 44 | project = u'Minion' 45 | copyright = u'2013-2015, Mozilla' 46 | 47 | # The version info for the project you're documenting, acts as replacement for 48 | # |version| and |release|, also used in various other places throughout the 49 | # built documents. 50 | # 51 | # The short X.Y version. 52 | version = '0.4' 53 | # The full version, including alpha/beta/rc tags. 54 | release = '0.4' 55 | 56 | # The language for content autogenerated by Sphinx. Refer to documentation 57 | # for a list of supported languages. 58 | #language = None 59 | 60 | # There are two options for replacing |today|: either, you set today to some 61 | # non-false value, then it is used: 62 | #today = '' 63 | # Else, today_fmt is used as the format for a strftime call. 64 | #today_fmt = '%B %d, %Y' 65 | 66 | # List of patterns, relative to source directory, that match files and 67 | # directories to ignore when looking for source files. 68 | exclude_patterns = ['_build'] 69 | 70 | # The reST default role (used for this markup: `text`) to use for all documents. 71 | #default_role = None 72 | 73 | # If true, '()' will be appended to :func: etc. cross-reference text. 74 | #add_function_parentheses = True 75 | 76 | # If true, the current module name will be prepended to all description 77 | # unit titles (such as .. function::). 78 | #add_module_names = True 79 | 80 | # If true, sectionauthor and moduleauthor directives will be shown in the 81 | # output. They are ignored by default. 82 | #show_authors = False 83 | 84 | # The name of the Pygments (syntax highlighting) style to use. 85 | pygments_style = 'sphinx' 86 | 87 | # A list of ignored prefixes for module index sorting. 88 | #modindex_common_prefix = [] 89 | 90 | # If true, keep warnings as "system message" paragraphs in the built documents. 91 | #keep_warnings = False 92 | 93 | 94 | # -- Options for HTML output --------------------------------------------------- 95 | 96 | # The theme to use for HTML and HTML Help pages. See the documentation for 97 | # a list of builtin themes. 98 | html_theme = 'mozilla' 99 | 100 | # Theme options are theme-specific and customize the look and feel of a theme 101 | # further. For a list of options available for each theme, see the 102 | # documentation. 103 | #html_theme_options = {} 104 | 105 | # Add any paths that contain custom themes here, relative to this directory. 106 | html_theme_path = ["theme/mozilla_sphinx_theme"] 107 | 108 | # The name for this set of Sphinx documents. If None, it defaults to 109 | # " v documentation". 110 | #html_title = None 111 | 112 | # A shorter title for the navigation bar. Default is the same as html_title. 113 | #html_short_title = None 114 | 115 | # The name of an image file (relative to this directory) to place at the top 116 | # of the sidebar. 117 | #html_logo = None 118 | 119 | # The name of an image file (within the static path) to use as favicon of the 120 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 121 | # pixels large. 122 | #html_favicon = None 123 | 124 | # Add any paths that contain custom static files (such as style sheets) here, 125 | # relative to this directory. They are copied after the builtin static files, 126 | # so a file named "default.css" will overwrite the builtin "default.css". 127 | html_static_path = ['_static'] 128 | 129 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 130 | # using the given strftime format. 131 | #html_last_updated_fmt = '%b %d, %Y' 132 | 133 | # If true, SmartyPants will be used to convert quotes and dashes to 134 | # typographically correct entities. 135 | #html_use_smartypants = True 136 | 137 | # Custom sidebar templates, maps document names to template names. 138 | #html_sidebars = {} 139 | 140 | # Additional templates that should be rendered to pages, maps page names to 141 | # template names. 142 | #html_additional_pages = {} 143 | 144 | # If false, no module index is generated. 145 | #html_domain_indices = True 146 | 147 | # If false, no index is generated. 148 | #html_use_index = True 149 | 150 | # If true, the index is split into individual pages for each letter. 151 | #html_split_index = False 152 | 153 | # If true, links to the reST sources are added to the pages. 154 | #html_show_sourcelink = True 155 | 156 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 157 | #html_show_sphinx = True 158 | 159 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 160 | #html_show_copyright = True 161 | 162 | # If true, an OpenSearch description file will be output, and all pages will 163 | # contain a tag referring to it. The value of this option must be the 164 | # base URL from which the finished HTML is served. 165 | #html_use_opensearch = '' 166 | 167 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 168 | #html_file_suffix = None 169 | 170 | # Output file base name for HTML help builder. 171 | htmlhelp_basename = 'Miniondoc' 172 | 173 | 174 | # -- Options for LaTeX output -------------------------------------------------- 175 | 176 | latex_elements = { 177 | # The paper size ('letterpaper' or 'a4paper'). 178 | #'papersize': 'letterpaper', 179 | 180 | # The font size ('10pt', '11pt' or '12pt'). 181 | #'pointsize': '10pt', 182 | 183 | # Additional stuff for the LaTeX preamble. 184 | #'preamble': '', 185 | } 186 | 187 | # Grouping the document tree into LaTeX files. List of tuples 188 | # (source start file, target name, title, author, documentclass [howto/manual]). 189 | latex_documents = [ 190 | ('index', 'Minion.tex', u'Minion Documentation', 191 | u'Mozilla', 'manual'), 192 | ] 193 | 194 | # The name of an image file (relative to this directory) to place at the top of 195 | # the title page. 196 | #latex_logo = None 197 | 198 | # For "manual" documents, if this is true, then toplevel headings are parts, 199 | # not chapters. 200 | #latex_use_parts = False 201 | 202 | # If true, show page references after internal links. 203 | #latex_show_pagerefs = False 204 | 205 | # If true, show URL addresses after external links. 206 | #latex_show_urls = False 207 | 208 | # Documents to append as an appendix to all manuals. 209 | #latex_appendices = [] 210 | 211 | # If false, no module index is generated. 212 | #latex_domain_indices = True 213 | 214 | 215 | # -- Options for manual page output -------------------------------------------- 216 | 217 | # One entry per manual page. List of tuples 218 | # (source start file, name, description, authors, manual section). 219 | man_pages = [ 220 | ('index', 'minion', u'Minion Documentation', 221 | [u'Mozilla'], 1) 222 | ] 223 | 224 | # If true, show URL addresses after external links. 225 | #man_show_urls = False 226 | 227 | 228 | # -- Options for Texinfo output ------------------------------------------------ 229 | 230 | # Grouping the document tree into Texinfo files. List of tuples 231 | # (source start file, target name, title, author, 232 | # dir menu entry, description, category) 233 | texinfo_documents = [ 234 | ('index', 'Minion', u'Minion Documentation', 235 | u'Mozilla', 'Minion', 'An open source security testing platform.', 236 | 'Security'), 237 | ] 238 | 239 | # Documents to append as an appendix to all manuals. 240 | #texinfo_appendices = [] 241 | 242 | # If false, no module index is generated. 243 | #texinfo_domain_indices = True 244 | 245 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 246 | #texinfo_show_urls = 'footnote' 247 | 248 | # If true, do not generate a @detailmenu in the "Top" node's menu. 249 | #texinfo_no_detailmenu = False 250 | -------------------------------------------------------------------------------- /docs/source/configure_minion.rst: -------------------------------------------------------------------------------- 1 | Configuring Minion 2 | ################## 3 | 4 | This document explains how to configure the Minion frontend and backend. 5 | 6 | As a convention, Minion will look in ``/etc/minion/`` and ``~minion/.minion`` for its configuration files. 7 | 8 | .. _configure_minion_frontend_label: 9 | 10 | Configure Minion Frontend 11 | ========================= 12 | 13 | Here is the `default configuration `_ for the Minion frontend server: 14 | 15 | .. literalinclude:: include/frontend.json 16 | :language: javascript 17 | 18 | To configure the frontend, place your configuration in a file called ``frontend.json`` in either ``/etc/minion`` or ``/home/user/.minion``. 19 | 20 | - ``backend-api`` 21 | 22 | - ``uri``: URI of the Minion backend server 23 | 24 | - ``login`` 25 | 26 | - ``type``: the type of authentication to use; currently supported types are ``persona``, which requires no configuration, ``ldap``, and ``oauth`` 27 | 28 | - ``ldap``: the configuration for LDAP, if ``ldap`` is the chosen authentication method in ``login -> type`` 29 | 30 | - ``uri``: URI to ldap server 31 | 32 | - ``baseDN``: baseDN for users; not needed for Active Directory 33 | 34 | - ``emailAttribute``: typically ``mail`` in OpenLDAP or ``userPrincipalName`` in Active Directory 35 | 36 | - ``groupMembershipAttribute``: typically ``member`` in OpenLDAP or ``uniqueMember`` in Active Directory 37 | 38 | - ``usernameAttribute``: typically ``uid`` in OpenLDAP or ``samAccountName`` in AD 39 | 40 | - ``checkAuthorizedGroups``: if true, require group membership in addition to valid user id 41 | 42 | - ``authorizedGroups``: list of groups where users are authorized to use Minion (if ``checkAuthorizedGroups`` is true) 43 | 44 | - ``oauth``: the configuration for OAuth, if ``oauth` is the chosen authentication method is set in ``login -> type``; see :ref:`configure_minion_oauth_label` 45 | 46 | - ``***``: supported providers are Facebook, Firefox Accounts, GitHub, and Google 47 | 48 | - ``client_id``: client_id for the chosen provider 49 | 50 | - ``client_secret``: client_secret for the chosen provider 51 | 52 | .. _configure_minion_backend_label: 53 | 54 | Configure Minion Backend 55 | ======================== 56 | 57 | Here is the `default configuration `_ for the Minion backend server:: 58 | 59 | { 60 | 'api': { 61 | 'url': 'http://127.0.0.1:8383', 62 | }, 63 | 'celery': { 64 | 'broker': 'amqp://guest@127.0.0.1:5672//', 65 | 'backend': 'amqp' 66 | }, 67 | 'mongodb': { 68 | 'host': '127.0.0.1', 69 | 'port': 27017 70 | }, 71 | 'email': { 72 | 'host': '127.0.0.1', 73 | 'port': 25, 74 | 'max_time_allowed': 604800 # 7 * 24 * 60 * 60 (7 days) 75 | } 76 | } 77 | 78 | To configure the backend, place your configuration in a file called ``backend.json`` at either ``/etc/minion`` or 79 | ``/home/user/.minion``. 80 | 81 | - ``api`` 82 | 83 | - ``url``: the full authority (hostname and port) of the backend server. 84 | 85 | - ``celery`` 86 | 87 | - ``broker``: URI of the celery broker 88 | 89 | - ``backend``: protocol used to speak to backend 90 | 91 | - ``mongodb``: 92 | 93 | - ``host``: hostname of MongoDB server 94 | 95 | - ``port``: port of the MongoDB server 96 | 97 | - ``email`` 98 | 99 | - ``host``: hostname of mail server 100 | 101 | - ``port``: port of mail server 102 | 103 | - ``max_time_allowed``: determines the life time of an invitation; by default it will remain valid for seven days. 104 | 105 | 106 | 107 | .. _configure_minion_oauth_label: 108 | 109 | Configuring OAuth 110 | ================= 111 | 112 | Minion currently supports Facebook, Firefox Accounts (FxA), GitHub, and Google Accounts as OAuth providers. 113 | 114 | .. image:: images/login-oauth.png 115 | :scale: 50% 116 | :height: 468px 117 | :width: 614px 118 | :align: center 119 | 120 | To enable a provider, simply input the ``client_id`` and ``client_secret`` provided by them into ``frontend.json``. Once input, they should automatically appear as options on the login page. For example: 121 | 122 | .. code-block:: javascript 123 | 124 | "facebook": { 125 | "client_id": "1234567890101112", 126 | "client_secret": "1c414b10981bfe1aa134874ac4daf780" 127 | } 128 | 129 | When configuring the provider, each will have a unique callback URI corresponding to its provider name. The URI should look like: 130 | 131 | ``:///ws/login/oauth/`` 132 | 133 | For example: 134 | 135 | ``https://minion.mozilla.org/ws/login/oauth/facebook`` 136 | 137 | 138 | .. _whitelist_blacklist_hostname_label: 139 | 140 | Whitelisting and Blacklisting Hosts 141 | =================================== 142 | 143 | By default, `Minion will blacklist `_ the following IP addresses from being scanned: 144 | 145 | .. code-block:: javascript 146 | 147 | "blacklist": [ 148 | "10.0.0.0/8", 149 | "127.0.0.0/8", 150 | "172.16.0.0/12", 151 | "192.168.0.0/16", 152 | "169.254.0.0/16" 153 | ] 154 | 155 | You can check the latest list at: . 156 | 157 | The effect of this is that Minion will refuse to scan any target site whose hostname falls in one of the ranges. 158 | For example, when Minion resolve the hostname ``localhost`` to ``127.0.0.1``, Minion will abort the scan because 159 | it is blacklisted. 160 | 161 | To configure the blacklist and whitelist, you can copy ``etc/scan.json`` into either ``/etc/minion/`` or ``~minion/.minion/``. Note that the whitelist will override the blacklist, so in this example, IP addresses in 192.168.1.0/24 can be scanned, despite 192.168.0.0/16 being in the blacklist: 162 | 163 | .. code-block:: javascript 164 | 165 | { 166 | "whitelist": [ 167 | "192.168.1.0/24" 168 | ], 169 | "blacklist": [ 170 | "10.0.0.0/8", 171 | "127.0.0.0/8", 172 | "172.16.0.0/12", 173 | "192.168.0.0/16", 174 | "169.254.0.0/16" 175 | ] 176 | } 177 | 178 | Any host that does not fall within the blacklist can be scanned. 179 | 180 | IP address blacklisting and whitelist also supports hostnames and hostname wildcards. For example: 181 | 182 | .. code-block:: javascript 183 | 184 | "blacklist": [ 185 | "mozilla.com", 186 | "*.mozilla.org" 187 | ] 188 | 189 | 190 | In this configuration, we allowed scanning LAN network and localhost, but we removed the ability to scan mozilla.com and any subdomain of mozilla.org. Note that if we wanted to block mozilla.org and subdomains, we would need entries for ``mozilla.org`` and ``*.mozilla.org``: 191 | 192 | .. code-block:: javascript 193 | 194 | "blacklist": [ 195 | "mozilla.org", 196 | "*.mozilla.org" 197 | ] 198 | -------------------------------------------------------------------------------- /docs/source/contribute_to_minion.rst: -------------------------------------------------------------------------------- 1 | Contribute 2 | ########## 3 | 4 | Minion is built using open source projects such as Python, angular.js, Flask, Celery, and Twisted. In order to make Minion feature richer and secure, we invite the community to contribute in any way they can. If you have ideas to submit, or have issues to report, you encourage to contact us via: 5 | 6 | * IRC: #websectools channel on irc.mozilla.org 7 | * Email list: http://groups.google.com/group/mozilla-minion-dev 8 | * Any of our GitHub repos: 9 | 10 | .. include:: include/minion_repos.rst 11 | 12 | .. include:: include/plugin_repos.rst 13 | -------------------------------------------------------------------------------- /docs/source/developing_plugins.rst: -------------------------------------------------------------------------------- 1 | Developing Plugins 2 | ################## 3 | 4 | Minion plugins are Python classes that talks to a security aessement tool. This tool can be pure Python code inside the class doing HTTP header check (which is what the basic plugins do), or invoke an external executable binary. It can be as simple as a few lines of Perl or as complex as `OWASP ZAP `_. 5 | 6 | 7 | 8 | How Minion discovers new plugins 9 | ================================ 10 | 11 | If we look at minion-zap-plugin's source code structure closely, we notice the layout of the package looks minimally like this: 12 | 13 | .. code-block:: bash 14 | 15 | $ tree minion-zap-plugin 16 | 17 | minion-zap-plugin 18 | ├── minion 19 | │   ├── __init__.py 20 | │   └── plugins 21 | │   ├── __init__.py 22 | │   └── zap 23 | │   ├── __init__.py 24 | │   ├── reference.py 25 | │   └── zap_plugin.py 26 | └── setup.py 27 | 28 | Every plugin must fall under the ``minion.plugins`` package namespace. 29 | This is how the backend detects an existence of a plugin. The third level 30 | in minion-zap-plugin is a directory called **zap**. This is the namespace 31 | of the plugin itself. In addition, the plugin class must be a subclass of 32 | ``AbstractPlugin`` and the following class member attributes: 33 | 34 | * ``PLUGIN_NAME``: the name of the plugin displayed to the frontend user 35 | 36 | * ``PLUGIN_VERSION``: the release version of the plugin 37 | 38 | * ``PLUGIN_WEIGHT``: level of resource required to launch this plugin (light, medium, heavy) 39 | 40 | 41 | The registry code is found in `base.py `_. 42 | 43 | Whenever a plugin is installed, the backend server and all the celery workers must be restarted, unless the backend is run with the --reload option. 44 | 45 | 46 | Minion Plugin Classes 47 | ===================== 48 | 49 | Now that you know how Minion discovers a new plugin, we can examine plugin classes. 50 | 51 | The first thing a plugin author needs to understand is the basics of 52 | Twisted. Minion's Task Engine uses `Twisted `_ 53 | for running plugins. We recommend plugin authors go over 54 | `An Introduction to Asynchronous Programming and Twisted `_ before jumping into writing plugin. We will only cover relevant information 55 | to get you understand how a Minion plugin works. 56 | 57 | 58 | ``AbstractPlugin`` class 59 | ------------------------- 60 | 61 | The ``AbstractPlugin`` class implements a set of methods and attributes that 62 | a plugin should define. It also provides additional methods 63 | that need not be override by plugin authors. The ``AbstractPlugin`` **should be** 64 | the base class for any plugin you want to write. Minion ships with two 65 | classes of plugins (``BlockingPlugin`` and 66 | ``ExternalProcessPlugin``) inherit from ``AbstractPlugin`` and other plugins 67 | Minion developers have developed so far are inheriting from one of the two 68 | classes. We will get into them in the later section; let's focus on 69 | ``AbstractPlugin``. 70 | 71 | Remember how Minion discovers and registers new plugin? The registry 72 | code expects the plugin to be found in the ``minion.plugins`` namespace, 73 | and in addition the plugins must have three class methods defined: 74 | ``name``, ``version``, and ``weight``. 75 | 76 | The ``AbstractPlugin`` implements these class methods with a default value. 77 | For example, if you don't define ``name``, it will use the plugin's class name 78 | as the name of the plugin. But it is always a good idea to define them 79 | yourself. 80 | 81 | To actual do work, a plugin should implement ``do_configure``, ``do_start`` 82 | and ``do_stop`` methods since the ``AbstractPlugin`` leaves them blank. 83 | 84 | ``BlockingPlugin`` class 85 | ------------------------ 86 | 87 | This class inherits from ``AbstractPlugin`` directly and is defined in 88 | `base.py `_ 89 | like ``AbstractPlugin``. 90 | 91 | The ``BlockingPlugin`` implements ``do_configure``, ``do_start``, and ``do_stop`` 92 | methods. Most of the Twsited logics are defined in the ``do_start`` method. The 93 | class simply expects a method called ``do_run`` and passes this method to the 94 | ``deferToThread`` which will return an asynchronous result that will return 95 | an actual result some time in the future. The nice ``deferToThread`` API 96 | has a pair of callbacks: success and failure. We simply add these two 97 | callbacks to the defer result object and returns. 98 | 99 | The magic of Twsited is the event-driven model, also known as reactor model. 100 | The idea is that an event loop will react to an event, possibly executes the 101 | event, and then emits a callback when the event is finished. 102 | 103 | You are probably wondering what happen to ``do_configure``. This particular 104 | class of plugin does not expect any pre-processing configuration needed. As 105 | we study the source code, we will see that all basic plugins are instances of 106 | ``BlockingPlugin``. After all, they just need to connect to the target URL, 107 | examine a header or validate the robots.txt. 108 | 109 | 110 | ``ExternalProcessPlugin`` class 111 | ------------------------------- 112 | 113 | This is another implementation of ``AbstractPlugin`` class and this class 114 | is useful for executing a plugin that relies on external process such as 115 | ZAP or Skipfish. Instead of calling ``subprocess``, we use Twsited's 116 | ``reactor.spawnProcess`` to launch the external process. 117 | 118 | Keen readers notice ``do_configure`` is also not defined. This is intentional. 119 | Every plugin that inherits from the ``ExternalProcessPlugin`` will implement 120 | its own ``do_configure`` because different process is launched differently. 121 | 122 | Import plugin class in __init__.py 123 | ================================== 124 | 125 | When we revisit the structure of a plugin package, it looks like this: 126 | 127 | .. code-block:: bash 128 | 129 | minion-zap-plugin 130 | ├── minion 131 | │   ├── __init__.py 132 | │   └── plugins 133 | │   ├── __init__.py 134 | │   └── zap 135 | │   ├── __init__.py 136 | │   ├── reference.py 137 | │   └── zap_plugin.py 138 | └── setup.py 139 | 140 | The third ``__init__.py`` should import the main plugin class. In the case of the zap plugin, it looks like this: 141 | 142 | .. code-block:: python 143 | 144 | from zap_plugin import ZAPPlugin 145 | 146 | If the file is left blank, Minion's plugin discovery code will not be able to import ``ZAPPlugin``. 147 | 148 | Plugin Template Generator 149 | ========================= 150 | 151 | Because developing a plugin with the right structure can be tedious and error-prone, there is a script 152 | to generate Minion plugin. The script lives here: https://gist.github.com/yeukhon/8309083 153 | 154 | To use this script, download **generate-plugin.py** and **config.ini**. Then edit config.ini to fit your 155 | need (see sample-config.ini on the same page as an example). 156 | -------------------------------------------------------------------------------- /docs/source/images/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/.DS_Store -------------------------------------------------------------------------------- /docs/source/images/admin-groups-editor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-groups-editor.png -------------------------------------------------------------------------------- /docs/source/images/admin-groups.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-groups.png -------------------------------------------------------------------------------- /docs/source/images/admin-plans.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-plans.png -------------------------------------------------------------------------------- /docs/source/images/admin-plugins.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-plugins.png -------------------------------------------------------------------------------- /docs/source/images/admin-sites.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-sites.png -------------------------------------------------------------------------------- /docs/source/images/admin-users.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/admin-users.png -------------------------------------------------------------------------------- /docs/source/images/dashboard-groups.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/dashboard-groups.png -------------------------------------------------------------------------------- /docs/source/images/dashboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/dashboard.png -------------------------------------------------------------------------------- /docs/source/images/login-oauth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/login-oauth.png -------------------------------------------------------------------------------- /docs/source/images/login.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/login.png -------------------------------------------------------------------------------- /docs/source/images/scan-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/scan-detail.png -------------------------------------------------------------------------------- /docs/source/images/scan-issue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mozilla/minion/37a3fb30955414d5903296e66a29ed843499c53b/docs/source/images/scan-issue.png -------------------------------------------------------------------------------- /docs/source/include/frontend.json: -------------------------------------------------------------------------------- 1 | { 2 | "backend-api": { 3 | "url": "http://minion-backend:8383" 4 | }, 5 | 6 | "login": { 7 | "type": "persona", 8 | 9 | "ldap": { 10 | "uri": "ldaps://ldap.server/", 11 | "baseDN": "ou=test,dc=test_dc", 12 | 13 | "emailAttribute": "mail", 14 | "groupMembershipAttribute": "member", 15 | "usernameAttribute": "uid", 16 | 17 | "checkAuthorizedGroups": false, 18 | "authorizedGroups": [ 19 | "ou=groupTest1,ou=test,dc=test_dc", 20 | "ou=groupTest2,ou=test,dc=test_dc" 21 | ] 22 | }, 23 | 24 | "oauth": { 25 | "providers": { 26 | "facebook": { 27 | "client_id": "", 28 | "client_secret": "" 29 | }, 30 | "firefoxaccounts": { 31 | "client_id": "", 32 | "client_secret": "" 33 | }, 34 | "github": { 35 | "client_id": "", 36 | "client_secret": "" 37 | }, 38 | "google": { 39 | "client_id": "", 40 | "client_secret": "" 41 | } 42 | } 43 | } 44 | } 45 | } 46 | -------------------------------------------------------------------------------- /docs/source/include/minion_repos.rst: -------------------------------------------------------------------------------- 1 | - `Minion Frontend `_, a Python, angular.js, and Bootstrap-based website that provides a HTML interface to authenticate and authorize users, manage sites, initiate scans, and report issues 2 | 3 | - `Minion Backend `_, a Python, Flask, and Twisted-based backend that provides an API for the Minion Frontend, and acts as a middleman between the frontend and external security tools 4 | 5 | - `Minion VM `_, a repository of recipes to allow quick installations of Minion either via Vagrant or Docker -------------------------------------------------------------------------------- /docs/source/include/plugin_repos.rst: -------------------------------------------------------------------------------- 1 | - `Minion ZAP `_, which utilizes the `OWASP Zed Attack Proxy `_ 2 | 3 | - `Minion Nmap `_, utilizing the `Nmap network scanner `_ 4 | 5 | - `Minion Skipfish `_, utilizing the `Skipfish reconnaissance tool `_ 6 | 7 | - `Minion SSLyze `_, utilizing the `SSLyze TLS scanner `_ 8 | 9 | - `Minion SSL `_, which uses the `sslscan TLS scanner `_ -------------------------------------------------------------------------------- /docs/source/index.rst: -------------------------------------------------------------------------------- 1 | Mozilla Minion 2 | ############## 3 | 4 | Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers. To do so, it enables developers to scan with a wide variety of security tools, using a simple HTML-based interface. 5 | 6 | It consists of three umbrella projects: 7 | 8 | .. include:: include/minion_repos.rst 9 | 10 | Minion has limited scanning functionality built into itself. Instead, it relies on the large variety of pre-existing open source and commercial scanning tools. These plugins include: 11 | 12 | .. include:: include/plugin_repos.rst 13 | 14 | .. toctree:: 15 | :maxdepth: 3 16 | :hidden: 17 | 18 | install_guide 19 | configure_minion 20 | install_plugins 21 | using_minion 22 | inside_minion 23 | developing_plugins 24 | contribute_to_minion 25 | 26 | Developers 27 | ========== 28 | 29 | We are available in the #websectools channel on irc.mozilla.org: 30 | 31 | * April King (April) 32 | 33 | * Mark Goodwin (mgoodwin) 34 | 35 | * Simon Bennetts (psiinon) 36 | 37 | * Yvan Boily (yvan) 38 | -------------------------------------------------------------------------------- /docs/source/inside_minion.rst: -------------------------------------------------------------------------------- 1 | Inside Minion 2 | ############# 3 | 4 | In this document, we will look at the internals of Minion. Many of the points 5 | below have been discussed previously at: 6 | 7 | * `Introducing Minion `_ 8 | 9 | * `Writing Minion Plugins `_ 10 | 11 | While the blog posts may be outdated, they are useful to look at. 12 | 13 | Minion's mission 14 | ================ 15 | 16 | Minion is developed as an open source project by the Mozilla Security Assurance team. 17 | The goal is to enable developers and security testers to bring web application security 18 | into a continuous testing cycle. To do this, both developers and testers are given an 19 | easy to use dashboard that lists all the sites they need to scan, detail scan summary page 20 | and detail issue page to explain vulnerability. Instead of learning a new scan tool, 21 | users can write Minion plugins to call their favorite scan tools, and Minion will spawn 22 | the tool as a new process, scan the target website, and return the results back to the user. 23 | 24 | 25 | Architecture 26 | ============ 27 | 28 | At a high level there are three major components in Minion: **Plugins**, 29 | **Task Engine**, and **Front End**. 30 | 31 | In principle, the backend of Minion consists of a task engine and a set of plugins. 32 | The Front End exposes a user interface for both regular users and administrators 33 | to manage configurations and to see scan results. 34 | 35 | Technology stack 36 | ---------------- 37 | 38 | Before we divde into what each component does, let us look at the technology stack 39 | we are using in Minion: 40 | 41 | * We build the backend and frontend heavily in Python, using the 42 | `Flask `_ web framework for building 43 | an APIs server. 44 | 45 | * The backend's task engine uses `Twisted `_ 46 | for plugin execution. 47 | 48 | * We use `RabbitMQ `_ 49 | to queue and transfer messages between executing plugin and the backend's task engine. 50 | 51 | * The Front-end has a layer of wrapper APIs serving with Flask web framework. 52 | 53 | * We create the front end interface using `angular.js `_ 54 | and `Twitter Bootstrap `_ by making 55 | http requests to our front end's APIs (which in turns make requests to the backend's 56 | API server). 57 | 58 | * We use MongoDB as our data store. 59 | 60 | * By default, our version of Minion is using Persona for authentication. 61 | 62 | 63 | This stack corresponds to the diagram shown below. 64 | 65 | .. image:: https://wiki.mozilla.org/images/8/86/Minion-03-diagram-draft.png 66 | :align: center 67 | :scale: 80% 68 | 69 | Plugins 70 | ------- 71 | 72 | Minion plugins are light-weight wrappers that perform tasks such as configuring, 73 | starting, stopping a plan, and accept a set of callbacks to notify the caller 74 | that information is available. Minion ships with several primitive plugins 75 | known as **basic plugin**. This group of plugins checks to see whether 76 | the target URL is accessible, examines various HTTP headers and verifies 77 | whether robots.txt exists and correct. 78 | 79 | Two base classes for plugins are provided in the Minion backend to get developers started: 80 | 81 | * **BlockingPlugin** provides the basic functionality to support 82 | a plugin that performs a task, and reports it’s completion state at 83 | the end. This is suitable for creating straightforward plugins directly 84 | within Python. 85 | 86 | * **ExternalProcessPlugin** provides the functionality required 87 | to kick-off an external tool, and the class provides the basis for 88 | several other extensions, especially those that wrap existing security tools. 89 | 90 | Besides the primitive basic plugins, Mozilla developers have created several 91 | plugins: 92 | 93 | .. include:: include/plugin_repos.rst 94 | 95 | Task Engine 96 | ----------- 97 | 98 | Minion Backend and Minion Task Engine are synonym in this docuemtnation. 99 | The Task Engine provides the core functionality for managing users, 100 | groups, sites, scans, and results within the Minion platform. 101 | Acting as a central hub, the Task Engine maintains a register of 102 | available plugins, provides facilities for creating and modifying plans, 103 | and managing user access to Minion, including which sites they can scan. 104 | 105 | To execute a scan on a target URL, one or more plan must be defined for the target 106 | URL and a scan is initiated by picking a specific plan. In the nutshell, a 107 | plan is a JSON document that specifies some information about what the 108 | plan does, and a sequence of plugins to invoke (and such sequence is 109 | called a **workflow**). An example is shown below: 110 | 111 | .. code-block:: python 112 | 113 | { 114 | "name": "Fuzz and Scan", 115 | "description": "Run Skipfish to fuzz the application, and perform a ZAP scan.", 116 | "workflow": [ 117 | { 118 | "plugin_name": "minion.plugins.skipfish.skipfish.SkipfishPlugin", 119 | "description": "", 120 | "configuration": {} 121 | }, 122 | { 123 | "plugin_name": "minion.plugins.zap.zap_plugin.ZAPPlugin", 124 | "description": "Run the ZAP Spider and Scanner", 125 | "configuration": { 126 | "scan": true 127 | } 128 | } 129 | ] 130 | 131 | 132 | This plan will invoke Skipfish and ZAP. The configuration for Skipfish 133 | plugin is left empty to use the default options. Some plugins have required 134 | and optional configuration parameters for users to specify. 135 | 136 | Front End 137 | --------- 138 | 139 | As we stated in the technology stack section, the front end server is a Flask 140 | web application, while the user interface is created using AngularJS. In simple 141 | terms, users do not make direct API calls to the backend. In the nutshell, 142 | the task engine has very very little access control built-in. The front end 143 | is responsible for creating the ACL by wrapping API requests to the task 144 | engine in the front end's API. AngularJS makes calls to the front end's 145 | API endpoints. 146 | 147 | This looks confusing and unncessary for newcomer, but the main advantage 148 | is that we can re-engineer the front end in anyway we want with little 149 | to zero impact to the task engine. For example, someone swap out the entire 150 | front end and our Persona authentiation with their own front end implementation 151 | and authentication model (e.g. LDAP authentication). 152 | -------------------------------------------------------------------------------- /docs/source/install_guide.rst: -------------------------------------------------------------------------------- 1 | Installing Minion 2 | ################# 3 | 4 | Prerequisites 5 | ============= 6 | 7 | Minion is developed on Ubuntu 14.04. While it should work on other operating systems, your mileage may vary. The installation guide assumes that you will install Minion in ``/opt/minion``, and a virtualenv of ``/opt/minion/minion-env``, with Minion Frontend and Minion Backend installed on different systems. 8 | 9 | .. _install_backend_label: 10 | 11 | Install using Minion VM 12 | ======================= 13 | 14 | For developers or testers, we strongly suggest using Minion VM, which will install Minion automatically using either Vagrant or Docker. Of the two, Vagrant is the preferred method, as it helps facilitate rapid development. 15 | 16 | Downloading minion-vm 17 | --------------------- 18 | 19 | Make sure you have ``git`` installed, and then clone the minion-vm repository:: 20 | 21 | $ git clone https://github.com/mozilla/minion-vm.git 22 | 23 | Then, regardless of whether we're using Vagrant or Docker, it is necessary to edit ``backend.sh`` to change the initial administrator's email address and name:: 24 | 25 | MINION_ADMINISTRATOR_EMAIL="youremail@yourorganization.org" 26 | MINION_ADMINISTRATOR_NAME="Your Name" 27 | 28 | Installation via Vagrant 29 | ------------------------ 30 | 31 | First, we'll grab minion-backend, and minion-frontend from GitHub:: 32 | 33 | $ git clone https://github.com/mozilla/minion-backend 34 | $ git clone https://github.com/mozilla/minion-frontend 35 | 36 | Then, edit the ``BACKEND_SRC`` and ``FRONTEND_SRC`` variables in ``Vagrantfile`` to point to the location where you cloned them on your local system. We'll also create a directory to store the packages that Vagrant VMs will pull down; this will greatly speed up installation. This directory is the ``APT_CACHE_SRC`` variable:: 37 | 38 | BACKEND_SRC = "/home/myuser/minion/minion-backend/" 39 | FRONTEND_SRC = "/home/myuser/minion/minion-frontend/" 40 | APT_CACHE_SRC = "/home/myuser/minion/apt-cache/com.hashicorp.vagrant/apt-cache/" 41 | 42 | If you don't want Minion to use the private network IP addresses 192.168.50.49 and 192.168.50.50 for the backend and frontend respectively, edit them inside both ``Vagrantfile`` and ``vagrant-hosts.sh``:: 43 | 44 | BACKEND_IP = "192.168.50.49" 45 | FRONTEND_IP = "192.168.50.50" 46 | 47 | 192.168.50.49 minion-backend 48 | 192.168.50.50 minion-frontend 49 | 50 | Now all we need to do is start it up:: 51 | 52 | $ vagrant up 53 | 54 | 55 | That's it! The Minion frontend should now be accessible at http://192.168.50.50:8080, or whatever you set the IP address to. 56 | 57 | You can also ssh into your new Minion instances with ``vagrant ssh minion-frontend`` and ``vagrant ssh minion-backend``. 58 | 59 | Installation via Docker 60 | ----------------------- 61 | 62 | The Docker installation will automatically pull down minion-backend and minion-frontend from GitHub. :: 63 | 64 | $ docker build -t 'mozilla/minion-backend' -f Dockerfile-backend . 65 | $ docker build -t 'mozilla/minion-frontend' -f Dockerfile-frontend . 66 | $ docker run -d --name 'minion-backend' 'mozilla/minion-backend' 67 | $ docker run -d -p 8080:8080 --name 'minion-frontend' \ 68 | --link minion-backend:minion-backend 'mozilla/minion-frontend' 69 | 70 | The Minion frontend should now be accessible over HTTP at the IP address of the system running Docker, on port 8080. 71 | 72 | You can also get a shell on your new Minion instances with `docker exec -i -t minion-frontend /bin/bash` and 73 | `docker exec -i -t minion-backend /bin/bash`. 74 | 75 | Manual Installation 76 | =================== 77 | 78 | .. _install_backend_label: 79 | 80 | Install Minion Backend 81 | ---------------------- 82 | 83 | We will be installing the Minion Backend into ``/opt/minion/minion-backend`` with a virtualenv location of ``/opt/minion/minion-env``. 84 | 85 | First install the essentials:: 86 | 87 | # apt-get update 88 | # apt-get install -y build-essential curl git libcurl4-openssl-dev libffi-dev \ 89 | libssl-dev mongodb-server postfix python python-dev python-virtualenv \ 90 | rabbitmq-server stunnel supervisor 91 | 92 | Then, create and source your virtual environment. This will help keep Minion isolated from the rest of your system. We 93 | also need to upgrade setuptools from the version included with Ubuntu by default:: 94 | 95 | # mkdir -p /etc/minion /opt/minion 96 | # cd /opt/minion 97 | # virtualenv minion-env 98 | # source minion-env/bin/activate 99 | 100 | (minion-env)# easy_install --upgrade setuptools # required for Mock 101 | 102 | Next, setup your system with the following directories and the `minion` user account. We'll also create some convenience shell commands, to make working with Minion easier when running as the `minion` user:: 103 | 104 | # useradd -m minion 105 | # install -m 700 -o minion -g minion -d /run/minion -d /var/lib/minion -d /var/log/minion -d ~minion/.python-eggs 106 | 107 | # echo -e "\n# Automatically source minion-backend virtualenv" >> ~minion/.profile 108 | # echo -e "source /opt/minion/minion-env/bin/activate" >> ~minion/.profile 109 | 110 | # echo -e "\n# Minion convenience commands" >> ~minion/.bashrc 111 | # echo -e "alias miniond=\"supervisord -c /opt/minion/minion-backend/etc/supervisord.conf\"" >> ~minion/.bashrc 112 | # echo -e "alias minionctl=\"supervisorctl -c /opt/minion/minion-backend/etc/supervisord.conf\"" >> ~minion/.bashrc 113 | 114 | Now we can checkout Minion and install it:: 115 | 116 | # cd /opt/minion 117 | # git clone https://github.com/mozilla/minion-backend.git 118 | # source minion-env/bin/activate 119 | (minion-env)# cd minion-backend 120 | (minion-env)# python setup.py develop 121 | 122 | To make sure that Minion starts when the system reboots, we need to install the Minion init script. We can also disable 123 | the global `supervisord` installed with `apt-get install` above, if it wasn't being used before:: 124 | 125 | # cp /opt/minion/minion-backend/scripts/minion-init /etc/init.d/minion 126 | # chown root:root /etc/init.d/minion 127 | # chmod 755 /etc/init.d/minion 128 | # update-rc.d minion defaults 40 129 | # update-rc.d -f supervisor remove 130 | 131 | Next, we enable debug logging and automatic reloading of Minion or plugins upon code changes, by adding the ``--debug`` and ``--reload`` options:: 132 | 133 | # sed -i 's/runserver/--debug --reload runserver/' /opt/minion/minion-backend/etc/minion-backend.supervisor.conf 134 | 135 | .. note:: 136 | 137 | Auto-debugging and auto-reloading shouldn't be enabled on production systems, for security purposes. 138 | 139 | And that's it! Provided that everything installed successfully, we can start everything up:: 140 | 141 | # service mongodb start 142 | # service rabbitmq-server start 143 | # service minion start 144 | 145 | From this point on, you should be able to control the Minion processes either as root or as the newly-created minion user. Let's become the ``minion`` user, and see if everything is running properly:: 146 | 147 | # su - minion 148 | (minion-env)$ service minion status 149 | minion-backend RUNNING pid 18010, uptime 0:00:04 150 | minion-plugin-worker RUNNING pid 18004, uptime 0:00:04 151 | minion-scan-worker RUNNING pid 18009, uptime 0:00:04 152 | minion-scanschedule-worker RUNNING pid 18008, uptime 0:00:04 153 | minion-scanscheduler RUNNING pid 18007, uptime 0:00:04 154 | minion-state-worker RUNNING pid 18005, uptime 0:00:04 155 | 156 | Success! You can also use ``minionctl`` (an alias to ``supervisorctl``, using the Minion ``supervisord.conf`` configuration) to stop and start individual services, or check on status:: 157 | 158 | (minion-env)$ minionctl stop minion-backend 159 | minion-backend: stopped 160 | 161 | (minion-env)$ minionctl status minion-backend 162 | minion-backend STOPPED Sep 03 09:18 PM 163 | 164 | (minion-env)$ minionctl start minion-backend 165 | minion-backend: started 166 | 167 | (minion-env)$ minionctl status minion-backend 168 | minion-backend RUNNING pid 18795, uptime 0:00:07 169 | 170 | All that's left to do now is initialize the Minion database and create an administrator:: 171 | 172 | (minion-env)$ minion-db-init 'Your Name' 'youremail@mozilla.com' y 173 | success: added 'Your Name' (youremail@yourcompany.com) as administrator 174 | 175 | And we're done! All logs for Minion, including stdout, stderr, and debug logs, should appear in ``/var/log/minion``. 176 | 177 | .. note:: 178 | 179 | If you use virtualenv (recommended), then the Minion convenience scripts (such as `minion-db-init`) are only available if the shell is sourced into the virtualenv. This is done automatically for the Minion user. In other words, if you open a new terminal and then try ``minion-`` with auto-completion, the chance is you won't see anything. If you install Minion without virtualenv, these scripts will be available to the $PATH. 180 | 181 | .. _install_frontend_label: 182 | 183 | Install Minion Frontend 184 | ----------------------- 185 | 186 | First, install the essentials:: 187 | 188 | # apt-get update 189 | # apt-get install -y build-essential git libldap2-dev libsasl2-dev \ 190 | libssl-dev python python-dev python-virtualenv supervisor 191 | 192 | Then, create and source your virtual environment. This will help keep Minion isolated from the rest of your system. We 193 | also need to upgrade setuptools from the version included with Ubuntu by default:: 194 | 195 | # mkdir -p /etc/minion /opt/minion 196 | # cd /opt/minion 197 | # virtualenv minion-env 198 | # source minion-env/bin/activate 199 | 200 | (minion-env)# easy_install --upgrade setuptools # required for Mock 201 | 202 | Next, setup your system with the following directories and the `minion` user account. We'll also create some convenience shell commands, to make working with Minion easier when running as the `minion` user:: 203 | 204 | # useradd -m minion 205 | # install -m 700 -o minion -g minion -d /run/minion -d /var/lib/minion -d /var/log/minion -d ~minion/.python-eggs 206 | 207 | # echo -e "\n# Automatically source minion-frontend virtualenv" >> ~minion/.profile 208 | # echo -e "source /opt/minion/minion-env/bin/activate" >> ~minion/.profile 209 | 210 | # echo -e "\n# Minion convenience commands" >> ~minion/.bashrc 211 | # echo -e "alias miniond=\"supervisord -c /opt/minion/minion-frontend/etc/supervisord.conf\"" >> ~minion/.bashrc 212 | # echo -e "alias minionctl=\"supervisorctl -c /opt/minion/minion-frontend/etc/supervisord.conf\"" >> ~minion/.bashrc 213 | 214 | Now we can checkout Minion and install it:: 215 | 216 | # cd /opt/minion 217 | # git clone https://github.com/mozilla/minion-frontend.git 218 | # source minion-env/bin/activate 219 | (minion-env)# python setup.py develop 220 | 221 | To make sure that Minion starts when the system reboots, we need to install the Minion init script. We can also disable 222 | the global `supervisord` installed with `apt-get install` above, if it wasn't being used before:: 223 | 224 | # cp /opt/minion/minion-frontend/scripts/minion-init /etc/init.d/minion 225 | # chown root:root /etc/init.d/minion 226 | # chmod 755 /etc/init.d/minion 227 | # update-rc.d minion defaults 40 228 | # update-rc.d -f supervisor remove 229 | 230 | And that's it! Provided that everything installed successfully, we can start everything up:: 231 | 232 | # service minion start 233 | 234 | From this point on, you should be able to control the Minion processes either as root or as the newly-created minion user. Let's `su - minion`, and see if everything is running properly:: 235 | 236 | # su - minion 237 | (minion-env)minion@minion-frontend:~$ service minion status 238 | minion-frontend RUNNING pid 1506, uptime 1 day, 1:25:41 239 | 240 | Success! You can also use `minionctl` (an alias to `supervisorctl`, using the Minion `supervisord.conf` configuration) 241 | to stop and start individual services, or check on status:: 242 | 243 | (minion-env)minion@minion-frontend:~$ minionctl stop minion-frontend 244 | minion-frontend: stopped 245 | (minion-env)minion@minion-frontend:~$ minionctl status minion-frontend 246 | minion-frontend STOPPED Sep 09 07:15 PM 247 | (minion-env)minion@minion-frontend:~$ minionctl start minion-frontend 248 | minion-frontend: started 249 | (minion-env)minion@minion-frontend:~$ minionctl status minion-frontend 250 | minion-frontend RUNNING pid 8713, uptime 0:00:05 251 | -------------------------------------------------------------------------------- /docs/source/install_plugins.rst: -------------------------------------------------------------------------------- 1 | Installing Plugins 2 | ################## 3 | 4 | Plugins are essential to Minion scan. Every attack plan will specify one or more Minion plugin. A plugin is essentially a Python wrapper around a scan tool. The tool can be as complex as `OWASP ZAP `_ or as simple as checking HTTP header using Python (such as basic.plan that is shipped with Minion). 5 | 6 | To view a list of plugins available to Minion backend, logged in the frontend with an admin account, click on 7 | **Administration**, and then click on **Plugins**. 8 | 9 | .. image:: images/admin-plugins.png 10 | 11 | The Minion instance this screenshot is taken from has custom plugin `Nmap `_. Plugins that are under 12 | the namespace ``minion.plugins.basic.*`` come from the **basic** plugin that is shipped with Minion backend. 13 | 14 | In this document, users can learn how to install the Nmap plugin. Each plugin should have roughly similar instructions for installation. 15 | 16 | First, we need to install whatever external tool the plugin requires, in this case, nmap:: 17 | 18 | # apt-get update && apt-get -y install nmap 19 | 20 | Then, source the Minion virtualenv, download the plugin, and install it:: 21 | 22 | # source /opt/minion/minion-env/bin/activate 23 | (minion-env)# cd /opt/minion 24 | (minion-env)# git clone https://github.com/mozilla/minion-nmap-plugin.git 25 | (minion-env)# cd minion-nmap-plugin 26 | (minion-env)# python setup.py install 27 | 28 | Finally, restart Minion to pick up the new plugin:: 29 | 30 | # /etc/init.d/minion stop 31 | # /etc/init.d/minion start 32 | 33 | This should add the Minion plugin to the plugin administration page. Simply configure its plan, assign it to a site, and you're ready to begin scanning! -------------------------------------------------------------------------------- /docs/source/persona.rst: -------------------------------------------------------------------------------- 1 | Sign up for a Persona account 2 | ============================= 3 | 4 | Minion ships with Persona as the default of its two authentication methods. If you do not have a Persona account, follow the steps below: 5 | 6 | 1. Go to `Persona Login `_ 7 | 8 | 2. Click on **Sign in** 9 | 10 | 3. Enter an email address. 11 | 12 | 4. If this email address has not been registered as a Persona account, the login dialog will ask you to enter a password. For Gmail and Yahoo Mail users, you will be authenticated against the Gmail or Yahoo Mail servers. 13 | 14 | 15 | -------------------------------------------------------------------------------- /docs/source/using_minion.rst: -------------------------------------------------------------------------------- 1 | Using Minion 2 | ############ 3 | 4 | Minion's frontend is separate from the backend repository. Here we show how the frontend looks. 5 | 6 | Login 7 | ===== 8 | 9 | Minion's frontend is authenticated using Persona. You can replace Persona if you want. The backend is authentication 10 | agonistic; it is the job of the frontend web service to verify that given user has permission to access a resource. 11 | We explain this in :doc:`inside_minion`. 12 | 13 | .. image:: images/login.png 14 | 15 | If you are new to Persona, please check :doc:`persona` to find out how to make your email address into a Persona-capable 16 | email address. 17 | 18 | Dashboard 19 | ========= 20 | 21 | The screenshot belows is the first thing a Minion user will see after logging in. The current logged in user 22 | can only see the sites that he or she has access to; the user must be in a group that the site is also in. We 23 | will explain this later in the admin section. 24 | 25 | .. image:: images/dashboard.png 26 | 27 | The dashboard shows the progress of the newest scan (if any), the number of issues discovered by the newest scan, a link 28 | to view the detail of the newest scan and a button to trigger a scan. 29 | 30 | The **Scan History** tab shows all the scans performed on the site. The **All Issues** tab lists issues of the newest scan for 31 | each site the user has access to. 32 | 33 | If the user is in multiple groups, the dashboard also provide a drop-down menu on the upper mid-right corner to switch 34 | the group view, as shown in the screenshot below. In this case this current logged in user belongs to two groups. 35 | 36 | .. image:: images/dashboard-groups.png 37 | 38 | 39 | Scan Detail 40 | =========== 41 | 42 | The scan detail page lists all the issues with a short title. If the scan failed it will list the reason. 43 | 44 | .. figure:: images/scan-detail.png 45 | 46 | 47 | Issue Detail 48 | ============ 49 | 50 | Every issue has a detail page that describes the problem, possible mitigation and a list of references to look up. 51 | 52 | .. image:: images/scan-issue.png 53 | 54 | 55 | 56 | Admin: Users 57 | ============ 58 | 59 | This panel lists all the users in Minion. As with all other panels in the rest of the administration dashboard, 60 | there is a filter and an in-page search box to find users. 61 | 62 | .. image:: images/admin-users.png 63 | 64 | Admin: Sites 65 | ============ 66 | 67 | This panel lists all the sites that have been added to this Minion instance. Currently only admin users can 68 | add sites and site url format is also restricted. As of 0.4 release, Minion can only accept url that looks like 69 | ``http[s]://*.``. For example, while ``192.168.1.100``, ``foobar.com`` and ``foo.bar.com`` 70 | are valid, the frontend will not accept paths so that ``foobar.com/project/`` is not acceptable. 71 | 72 | There is a `longstanding ticket `_ to loosen up this restriction in the future release of Minion. 73 | 74 | .. note:: 75 | 76 | Also, since 0.4 release there is a hostname blacklist and whitelist mechanism added to Minion. Hostnames like 127.0.0.1 are blocked by Minion by default. To whitelist and/or blacklist what hostnames can be scanned, please refer to :ref:`whitelist_blacklist_hostname_label`. 77 | 78 | .. image:: images/admin-sites.png 79 | 80 | Every site can have one or more attack plan. An attack plan is a JSON configuration which specifies the type of 81 | plugin the attack will use. Every site belongs to zero or more groups and only the users in those groups can 82 | scan the site. As of 0.4 release, you must add the site first and then add the site and user to a group in the **Group** 83 | tab. 84 | 85 | Admin: Groups 86 | ============= 87 | 88 | As aforementioned in the previous section, once a site has been added, it should be added to a group by going to 89 | the **Groups** panel. Every group contains a set of users and a set of sites; only these users can access these sites. 90 | Bob's personal blog can belong to two groups named Group A and Group B. As long as Alice is in one of these two groups, 91 | she can too scan Bob's personal blog on Minion. 92 | 93 | .. image:: images/admin-groups.png 94 | 95 | By pressing the **Edit** button, the user is redirected to an editor page. 96 | 97 | .. image:: images/admin-groups-editor.png 98 | 99 | 100 | Admin: Plugins 101 | ============== 102 | 103 | A Minion plugin is basically a Python class that invoke a security scan tool. This scan tool can be as simple as 104 | a few lines of Python code directly in the class or invoking an external exectuable binary (which can be a simple bash 105 | script or as complex as OWASP ZAP). 106 | 107 | .. image:: images/admin-plugins.png 108 | 109 | Minion's backend actually ships with some simple Python plugins called **basic** plugin. In the screenshot above, 110 | they are under the namespace **minion.plugins.basic**. These plugins check HTTP header and robots.txt. Other plugins are 111 | not distributed with the backend. You must install them indivdually (although the backend repository has the sample 112 | configuration file for zap, skipfish and nmap). 113 | 114 | To learn more about how to install plugins, please check :doc:`install_plugins`. For plugin development, 115 | please refer to :doc:`developing_plugins`. 116 | 117 | 118 | Admin: Plans 119 | ============ 120 | 121 | Plans are basically JSON blob/document that specifies what plugins to use (and their configurations). You can 122 | have a plan that utilizes the **basic** plugin and ZAP plugin. 123 | 124 | 125 | .. image:: images/admin-plans.png 126 | 127 | 128 | Admin: Invites 129 | ============== 130 | 131 | If you want to invite another person to use your Minion, you can send them an invitation. You can add, resend, or cancel 132 | invitation. You can also check whether the invitation is accepted, declined or expired. 133 | 134 | When a user is invited, the account is created, but the account is suspended until the user 135 | has accepted the invitation and is authenticated by logging in using a Persona address. If your friend 136 | has two email addresses and you invite your friend by using the non-Persona email address, 137 | your friend can accept the invitation and log in using his or her Persona email address. Minion will take 138 | care of the transition (replacing non-Persona email address in the database which his or her Persona account). 139 | 140 | 141 | --------------------------------------------------------------------------------