├── .github └── pull_request_template.md ├── .gitignore ├── .pre-commit-config.yaml ├── .python-version ├── .readthedocs.yml.bk ├── CHANGELOG.md ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.md ├── RELEASE.md ├── RULES.md ├── docs ├── CHANGELOG.md ├── Makefile ├── _static │ └── custom.css ├── _templates │ └── layout.html ├── conf.py ├── core │ ├── api.md │ ├── cli.md │ ├── conf.md │ ├── customizing.md │ ├── provisioning.md │ └── quickstart.md ├── index.rst └── providers │ ├── aws-ec2.md │ ├── digitalocean.md │ ├── docker.md │ └── lxc.md ├── pyproject.toml ├── rambo ├── Vagrantfile ├── __init__.py ├── app.py ├── auth │ └── env_scripts │ │ ├── aws.env.sh.dist │ │ └── digitalocean.env.sh.dist ├── cli.py ├── options.py ├── saltstack │ ├── etc │ │ └── minion.d │ │ │ └── minion.conf │ └── srv │ │ ├── pillars │ │ └── .gitinclude │ │ └── salt │ │ ├── basebox │ │ └── init.sls │ │ └── top.sls ├── settings.json ├── settings.py ├── utils.py ├── vagrant │ ├── host_vms │ │ ├── auxiliary_disks │ │ │ └── .gitignore │ │ └── docker_host │ ├── modules.rb │ ├── provider_support │ │ └── docker │ │ │ └── dockerfiles │ │ │ ├── debian_vagrant │ │ │ └── ubuntu_vagrant │ └── vagrantfiles │ │ ├── digitalocean │ │ ├── docker │ │ ├── ec2 │ │ ├── lxc │ │ └── virtualbox └── vagrant_providers.py ├── requirements-dev.lock ├── requirements.lock └── setup.cfg /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | **(issue reference)** 2 | Fixes # 3 | 4 | **Does this deserve / include a changlog entry?** 5 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | syntax: glob 2 | 3 | ## patterns 4 | *.orig 5 | *.tgz 6 | *.swp 7 | *.swo 8 | *~ 9 | *.DS_Store* 10 | *.vagrant* 11 | *.pem 12 | *.pub 13 | *.log 14 | *.pyc 15 | *.#* 16 | *# 17 | *.egg-info 18 | *flymd* 19 | 20 | ## paths 21 | dist 22 | activate.sh 23 | env.sh 24 | digitalocean.env.sh 25 | aws.env.sh 26 | docs/_build 27 | srv/salt/ssh_keys 28 | .tmp 29 | **/auth/keys 30 | **/licenses 31 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/charliermarsh/ruff-pre-commit 3 | # Ruff version. 4 | rev: 'v0.4.3' 5 | hooks: 6 | - id: ruff 7 | args: [ --fix, --exit-non-zero-on-fix ] 8 | -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | 3.12.0 2 | -------------------------------------------------------------------------------- /.readthedocs.yml.bk: -------------------------------------------------------------------------------- 1 | # .readthedocs.yml 2 | # Read the Docs configuration file 3 | # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details 4 | 5 | # Required 6 | version: 2 7 | 8 | # Build documentation in the docs/ directory with Sphinx 9 | sphinx: 10 | builder: dirhtml 11 | configuration: docs/conf.py 12 | fail_on_warning: true 13 | 14 | # Build documentation with MkDocs 15 | #mkdocs: 16 | # configuration: mkdocs.yml 17 | 18 | # Optionally build your docs in additional formats such as PDF and ePub 19 | formats: all 20 | 21 | # Optionally set the version of Python and requirements required to build your docs 22 | python: 23 | version: 3.7 24 | install: 25 | - method: pip 26 | path: . 27 | extra_requirements: 28 | - docs 29 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## TBD 4 | 5 | FEATURES: 6 | 7 | - Add option to set path to sync into VM. 8 | - Add option to provision from script with path. 9 | - Add option to start provisioning with command. 10 | - Add option to set VM name. 11 | - Add option to set hostname. 12 | - Add option to set cpus in virtualbox. 13 | - Add option to set sync_type and defauting to Vagrant's default. 14 | - Add option to provision with Salt. 15 | - Add option to set salt-bootstrap args. 16 | - Add option to sync all listed dirs. 17 | - Add optional setting overrides with my_rambo.conf. 18 | - Add option to resize base VirtualBox drive size with vagrant plugin. 19 | - Add option to set ports that are forwarded. 20 | 21 | IMPROVEMENTS: 22 | 23 | - `rambo destroy` will now use vboxmanage to fully poweroff and delete VirtualBox VMs. 24 | - No longer using any custom sources.list. 25 | - Renamed sync_dir to project_dir. 26 | - Now you can pass fancy pathing like `..` and symlinks to the CLI. 27 | - More comprehensive logging. 28 | - Splitting expected saltstack dir into saltstack/etc and saltstack/srv to more easily work with the common pattern of having Salt code in /etc/salt and /srv. 29 | - Left legacy style salt provisioning, but added a flag to use it instead of the newer style. 30 | 31 | BUGFIXES: 32 | 33 | - Fix bug incorrectly setting cwd, leading to nested temp dirs. 34 | - Passes provider explicitly as cmd arg to Vagrant. 35 | - Fixes guest hostname generation when given underscores in the path, casting it to "95", it's ascii code. 36 | - Fixes guest hostname generation when too long, truncating the part preceding the hash so total length stays below 64 chars. 37 | - Fix bug when setting machine-type. 38 | - Fix ability to set cwd. 39 | 40 | ## 0.4.4 (March 9, 2018) 41 | 42 | BUGFIX: 43 | 44 | - Now custom fork of click_configfile is added as a submodule so it's always present, and included this in the MANIFEST. 45 | 46 | ## 0.4.0 (March 9, 2018) 47 | 48 | FEATURES: 49 | 50 | - Added Ubuntu Dockerfile. 51 | - Added machine-type option for various cloud providers. 52 | - Added ramsize and drivesize options. 53 | - Added ability to load options via rambo.conf. 54 | - Added `createproject` cmd to create project dir, including default 55 | saltstack code, auth dir, and mandatory rambo.conf. 56 | - Added `install-plugins` cmd for installing vagrant plugins. 57 | - Refactored shell invocation of Vagrant to ouput near real-time stdout 58 | and stderr, keeps ANSI formatting, stderr and exit status passthrough. 59 | - Better logging of shell invocation of Vagrant. 60 | - Added the ability to custom saltstack code dir that is automatically used. 61 | - Added the ability to custom Vagrantfile that is automatically used. 62 | - Added export-vagrant-conf cmd for dropping vagrant files for customization. 63 | - Added option for guest os. 64 | 65 | IMPROVEMENTS: 66 | 67 | - Rounded out OS whitelist: Debian 8/9, CentOS 7, Ubuntu 14.04/16.04 68 | - Fix Docker bugs 69 | - Added readthedocs. 70 | - Defining a project by the existence of a conf file. 71 | - Remove support for using env vars and api simultaneously. 72 | - If no saltstack dir is in a the project dir, no salt is run. 73 | - Saltstack files moved to terminal-labs/sample-states repo. 74 | The 'basic' branch is pulled and used. 75 | - Added in vagrantfile setting kind of syncing. 76 | - Added toggle in vagrantfile for grabbing canonical apt sources or not 77 | since some images come with different, unreliable sources. 78 | - Renamed `vagrant_resources` to `vagrant`, and `salt_resources` to `saltstack`. 79 | - Changed default guest os from Debian 8 to Ubuntu 16.04. 80 | 81 | ## 0.3.3 (November 28, 2017) 82 | 83 | FEATURES: 84 | 85 | - Added ubuntu 14.04 to hosts list. 86 | 87 | IMPROVEMENTS: 88 | 89 | - Add version cli option. 90 | 91 | ## 0.3.2 (November 27, 2017) 92 | 93 | FEATURES: 94 | 95 | - Added additional Salt states for Hadoop edgenode and worker. 96 | - Allowing setting custom tmpdir path. 97 | 98 | IMPROVEMENTS: 99 | 100 | - Using standard get/set_env_var_rb functions. 101 | - Change VM_SIZE to RAMBO_RAM and created RAMBO_DRIVESIZE. 102 | - Changed VM_Host to RAMBO_GUEST_OS. 103 | - Cleaned up some Vagrant code. 104 | 105 | BUG FIXES: 106 | 107 | - Stop setting a default apt source on CentOS. 108 | - Stop setting hostname on CentOS. Another ticket was made for that. 109 | - Passing ctx to ssh and destroy commands. 110 | - Changed name of base box according to the box name change on 111 | app.vagrantup.com for the default Debian box. 112 | 113 | ## 0.3.1 (November 8, 2017) 114 | 115 | FEATURES: 116 | 117 | - Now AWS makes use of VM_Size flag to produce t2.nano, t2.micro, and t2.small VMs. 118 | 119 | IMPROVEMENTS: 120 | 121 | - Updated docs for CLI, Python API, Environment Variables 122 | - Renamed tmp dir to rambo-tmp. 123 | 124 | BUG FIXES: 125 | 126 | - `rambo destroy` now finds and deletes metadata in tmp dir. 127 | - Fix Docker failing on editing non-existant bashrc. Now ensuring existence first. 128 | - Fixing vagrant up exit trigger when VM not named 'default'. 129 | - Fixed bug preventing provisioning without Salt. 130 | 131 | ## 0.3.0 (October 26, 2017) 132 | 133 | FEATURES: 134 | 135 | - Added Salt states to apply Anaconda licenses. 136 | - Adding a Python API. 137 | - Added Nano to base Salt provisioning. 138 | - Able to set Vagrant environment variables via the CLI 139 | - Refactored packaging for PyPI. 140 | - Added in support for Ubuntu 14.04 and Centos 7 guest OSs. 141 | - Added in 4GB and 8GB RAM for all supported OSs. 142 | - Added Salt states for setting up licensed Anaconda. 143 | - Made Rambo a pip installable package. 144 | - Created a Python based CLI for Rambo. 145 | - Added support for multiple users on DigitalOcean. 146 | - Added a Salt state for Hadoop Ambari. 147 | - Added basic network modifications for clustering. 148 | - Setting the hostname to the VM_NAME. 149 | 150 | IMPROVEMENTS: 151 | 152 | - Now downloading base vagrant boxes from vagrantup.com. 153 | - Now enforcing Vagrant >=1.9.7. 154 | - VM_NAME now contains host's hostname, and rambo's working dir, and a unique hex id. 155 | - Now deletes broken symlinks found that would otherwise break Rambo 156 | during the rsync process. 157 | 158 | BUG FIXES: 159 | 160 | - Fix ability to set repository branch and then execute highstate. 161 | 162 | ## 0.2.1 (August 9, 2017) 163 | 164 | FEATURES: 165 | 166 | - Now activating conda environment upon `vagrant ssh`. 167 | - Added Salt State for Anaconda. 168 | - Added Salt State for loading a database dump from a local store, and 169 | allowing using this or the artifacts state. 170 | 171 | IMPROVEMENTS: 172 | 173 | - Added default fingerprints for BitBucket and GitHub. 174 | - Renamed miniconda state to conda. 175 | - Added documentation for Docker provider. 176 | 177 | BUG FIXES: 178 | 179 | - Fixed misnamed reference to Miniconda state. 180 | - Now requiring artifacts grains before trying to load a database. 181 | - Deduping state IDs for adding fingerprints for git and hg. 182 | - Specifying fingerprint hash type since that's now required by a Salt update. 183 | - Deduping state IDs for installing pip requirements with conda and venv. 184 | - Removing unused salt state directory. 185 | - Bumped required vagrant version. 186 | 187 | ## 0.2.0 (August 3, 2017) 188 | 189 | FEATURES: 190 | 191 | - Added a Salt State for Miniconda. 192 | - Added Docker as a provider. 193 | 194 | IMPROVEMENTS: 195 | 196 | - Using Packer made base boxes for VirtualBox. 197 | - Now using paravirtualization with VirtualBox for increased speed. 198 | - Enhanced documentation and helper markdown files. 199 | - Renamed 'AWS' provider to 'EC2' to avoid future confusion. 200 | - Updated documentation. 201 | - Some code cleaning. 202 | 203 | BUG FIXES: 204 | 205 | - Changed the standard AWS EC2 size to t2.micron. 206 | 207 | ## 0.1.0 (May 22, 2017) 208 | 209 | FEATURES: 210 | 211 | - Initial commit. 212 | 213 | The changelog began with open sourcing Rambo at version 0.1.0. 214 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2015-2019, Terminal Labs 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | * Redistributions of source code must retain the above copyright 7 | notice, this list of conditions and the following disclaimer. 8 | * Redistributions in binary form must reproduce the above copyright 9 | notice, this list of conditions and the following disclaimer in the 10 | documentation and/or other materials provided with the distribution. 11 | * Neither the name of Terminal Labs nor the 12 | names of its contributors may be used to endorse or promote products 13 | derived from this software without specific prior written permission. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 16 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | DISCLAIMED. IN NO EVENT SHALL TERMINAL LABS BE LIABLE FOR ANY 19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | recursive-include rambo/auth/env_scripts * 2 | recursive-include rambo/saltstack * 3 | recursive-include rambo/vagrant * 4 | recursive-include rambo/click-configfile * 5 | recursive-include docs * 6 | include rambo/Vagrantfile 7 | include rambo/settings.json 8 | include LICENSE 9 | include README.md 10 | include CHANGELOG.md 11 | global-exclude *.py[cdo] __pycache__ *.so *.pyd .DS_Store -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | APPNAME=rambo 2 | PYTHONVERSION = 3.6.9 3 | 4 | help: 5 | @echo "usage: make [command]" 6 | 7 | download_python_environment_manager: 8 | @if test ! -d ".tmp/python-environment-manager-master";then \ 9 | sudo su -m $(SUDO_USER) -c "mkdir -p .tmp"; \ 10 | sudo su -m $(SUDO_USER) -c "cd .tmp; wget https://github.com/terminal-labs/python-environment-manager/archive/master.zip"; \ 11 | sudo su -m $(SUDO_USER) -c "cd .tmp; unzip master.zip"; \ 12 | fi 13 | 14 | vagrant-pyenv: download_python_environment_manager 15 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/pyenv/build.sh $(APPNAME) $(SUDO_USER) vagrant 16 | 17 | vagrant-conda: download_python_environment_manager 18 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/build.sh $(APPNAME) $(SUDO_USER) vagrant 19 | 20 | mac-pyenv: download_python_environment_manager 21 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/pyenv/build.sh $(APPNAME) $(SUDO_USER) mac 22 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/pyenv/emit_activation_script.sh $(APPNAME) $(SUDO_USER) mac 23 | 24 | mac-conda: download_python_environment_manager 25 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/build.sh $(APPNAME) $(SUDO_USER) mac 26 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/emit_activation_script.sh $(APPNAME) $(SUDO_USER) mac 27 | 28 | linux-pyenv: download_python_environment_manager 29 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/pyenv/build.sh $(APPNAME) $(SUDO_USER) linux 30 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/pyenv/emit_activation_script.sh $(APPNAME) $(SUDO_USER) linux 31 | 32 | linux-conda: download_python_environment_manager 33 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/build.sh $(APPNAME) $(SUDO_USER) linux 34 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/emit_activation_script.sh $(APPNAME) $(SUDO_USER) linux 35 | 36 | mac-docs: mac-conda 37 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/docs.sh $(APPNAME) $(SUDO_USER) mac 38 | 39 | linux-docs: linux-conda 40 | @sudo bash .tmp/python-environment-manager-master/makefile_resources/scripts_python/conda/docs.sh $(APPNAME) $(SUDO_USER) linux 41 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # README 2 | 3 | [![PyPI version](https://badge.fury.io/py/rambo-vagrant.svg)](https://pypi.org/project/rambo-vagrant/) 4 | Code style: black 5 | 6 | ## Quickstart 7 | To get started fast, see: [Quickstart](http://terminallabs-rambo.readthedocs.io/en/latest/core/quickstart/), or go to our [docs homepage](http://terminallabs-rambo.readthedocs.io/en/latest/) 8 | 9 | ## What's Rambo For? 10 | This project is for provisioning and configuring virtual machines (and containers) in a simple, predictable, and highly reproducible way. Just run one command and your VM is up, code is deployed, and your app is running, on any supported platform. 11 | 12 | At this time this repo allows you to create a Linux VM on multiple providers (AWS EC2, DigitalOcean, VirtualBox, LXC). Several Operating Systems are available on select providers. The base machine configuration is a Ubuntu 16.04 64bit OS with 1024MB RAM, and 30GB drive. 13 | 14 | One of the goals of this project is be able to run a simple command and have a new VM be created on your provider of choice. Once the VM is initialized SaltStack is used to deploy code to and provision your machine. The SaltStack machine configuration code (states) will run the same regardless of which provider is actually running the machine. You can easily cycle your VMs by destroying and rebuilding them. 15 | 16 | Another goal of this repo is to have the spawned VMs be maximally similar across providers. Usually, your configuration will not need to change at all and will simply run on all providers. 17 | 18 | By default Rambo offers a basic VM configuration with SaltStack, but you can customize this. See [Customizing Rambo](http://terminallabs-rambo.readthedocs.io/en/latest/core/customizing/) for that. 19 | 20 | ## Basic Usage 21 | Once [installed](http://terminallabs-rambo.readthedocs.io/en/latest/core/quickstart/#installation), you can run one of these commands to get your VM: 22 | 23 | for [VirtualBox](https://www.virtualbox.org/) run 24 | ``` 25 | $ rambo up 26 | $ rambo ssh 27 | ``` 28 | 29 | for [AWS EC2](https://aws.amazon.com/ec2/) run 30 | ``` 31 | $ rambo up -p ec2 32 | $ rambo ssh 33 | ``` 34 | 35 | for [DigitalOcean](https://www.digitalocean.com/) run 36 | ``` 37 | $ rambo up -p digitalocean 38 | $ rambo ssh 39 | ``` 40 | 41 | for [Docker](https://www.docker.com/) run 42 | ``` 43 | $ rambo up -p docker 44 | $ rambo ssh 45 | ``` 46 | 47 | for [LXC](https://linuxcontainers.org/) run 48 | ``` 49 | $ rambo up -p lxc 50 | $ rambo ssh 51 | ``` 52 | 53 | ## Contributing 54 | We heartily welcome any contirubtions to this project, whether in the form of commenting on or posting an issue, or development. If you would like to submit a pull request, you might first want to look at our development [guidelines](https://github.com/terminal-labs/rambo/blob/master/RULES.md) for this project. 55 | 56 | ## Special Thanks 57 | Thanks go out to the Vagrant community and HashiCorp. Vagrant is a great tool it has helped us a lot over the years. 58 | 59 | Rambo is supported by [Terminal Labs](https://terminallabs.com/). 60 | -------------------------------------------------------------------------------- /RELEASE.md: -------------------------------------------------------------------------------- 1 | # Releasing Rambo 2 | 3 | This documents how to release Rambo. Various steps in this document may 4 | require privileged access to private systems, so this document is only 5 | targetted at Rambo core members who have the ability to cut a release. 6 | 7 | 1. Update `__version__` in `rambo/__init__.py` to the version you want to release. 8 | 9 | 1. Update [CHANGELOG.md](https://github.com/terminal-labs/rambo/blob/master/CHANGELOG.md) to have a header with the release version and date. 10 | 11 | 1. Commit those changes and also tag the release with the version: 12 | 13 | $ git tag X.Y.Z 14 | $ git push --tags 15 | 16 | 1. Release this version on [GitHub](https://github.com/terminal-labs/rambo/releases). 17 | 18 | 1. Update `version.txt` to the next version and append `.dev` and add 19 | [a new blank](https://github.com/terminal-labs/rambo/blob/c955146f3b8e88bb24dddc3755b3b8751a970b1a/CHANGELOG.md) entry in the CHANGELOG, commit, and push. 20 | -------------------------------------------------------------------------------- /RULES.md: -------------------------------------------------------------------------------- 1 | # RULES TO DEV BY 2 | 3 | 1. Aim for stability. 4 | 5 | 1. Aim for consistency across providers (e.g. VirtualBox, AWS) and interfaces (e.g. CLI / API). 6 | 7 | 1. Aim for easy repurposing and ease of use. 8 | 9 | 1. Aim for good documentation. 10 | 11 | 1. More providers and provisioners are always welcome. 12 | 13 | 1. Write in Ruby / Vagrant what is necessary, but keep any front-loaded logic in Python. 14 | -------------------------------------------------------------------------------- /docs/CHANGELOG.md: -------------------------------------------------------------------------------- 1 | ../CHANGELOG.md -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = Rambo 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /docs/_static/custom.css: -------------------------------------------------------------------------------- 1 | /* Sidebar header (and topbar for mobile) */ 2 | .wy-side-nav-search, .wy-nav-top { 3 | background: #2F9A41; 4 | } 5 | /* links */ 6 | .wy-nav-content a { 7 | color: #2F9A41; 8 | } 9 | .wy-nav-content a:visited { 10 | color: #2980B9; 11 | } 12 | 13 | .rst-content blockquote { 14 | padding: 0 1em; 15 | color: #6a737d; 16 | border-left: .25em solid #dfe2e5; 17 | } 18 | -------------------------------------------------------------------------------- /docs/_templates/layout.html: -------------------------------------------------------------------------------- 1 | {% extends "!layout.html" %} 2 | {% block extrahead %} 3 | 4 | 5 | 6 | {% endblock %} 7 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Configuration file for the Sphinx documentation builder. 4 | # 5 | # This file does only contain a selection of the most common options. For a 6 | # full list see the documentation: 7 | # http://www.sphinx-doc.org/en/stable/config 8 | import pkg_resources 9 | from recommonmark.parser import CommonMarkParser 10 | 11 | # -- Path setup -------------------------------------------------------------- 12 | 13 | # If extensions (or modules to document with autodoc) are in another directory, 14 | # add these directories to sys.path here. If the directory is relative to the 15 | # documentation root, use os.path.abspath to make it absolute, like shown here. 16 | # 17 | # import os 18 | # import sys 19 | # sys.path.insert(0, os.path.abspath('.')) 20 | 21 | 22 | # -- Project information ----------------------------------------------------- 23 | 24 | project = "Rambo" 25 | copyright = "2018-2020, Terminal Labs" 26 | author = "Terminal Labs" 27 | 28 | # The short X.Y version 29 | version = pkg_resources.get_distribution("rambo-vagrant").version 30 | # The full version, including alpha/beta/rc tags 31 | release = version 32 | 33 | 34 | # -- General configuration --------------------------------------------------- 35 | 36 | # If your documentation needs a minimal Sphinx version, state it here. 37 | # 38 | # needs_sphinx = '1.0' 39 | 40 | # Add any Sphinx extension module names here, as strings. They can be 41 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 42 | # ones. 43 | extensions = [ 44 | "sphinx.ext.autodoc", 45 | "sphinx.ext.coverage", 46 | ] 47 | 48 | # Add any paths that contain templates here, relative to this directory. 49 | templates_path = ["_templates"] 50 | 51 | # The suffix(es) of source filenames. 52 | # You can specify multiple suffix as a list of string: 53 | # 54 | source_suffix = [".rst", ".md"] 55 | 56 | # The master toctree document. 57 | master_doc = "index" 58 | 59 | # The language for content autogenerated by Sphinx. Refer to documentation 60 | # for a list of supported languages. 61 | # 62 | # This is also used if you do content translation via gettext catalogs. 63 | # Usually you set "language" from the command line for these cases. 64 | language = None 65 | 66 | # List of patterns, relative to source directory, that match files and 67 | # directories to ignore when looking for source files. 68 | # This pattern also affects html_static_path and html_extra_path . 69 | exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "**flymd*"] 70 | 71 | # The name of the Pygments (syntax highlighting) style to use. 72 | pygments_style = "default" 73 | 74 | 75 | # -- Options for HTML output ------------------------------------------------- 76 | 77 | gettext_compact = False 78 | 79 | # The theme to use for HTML and HTML Help pages. See the documentation for 80 | # a list of builtin themes. 81 | # 82 | html_theme = "sphinx_rtd_theme" 83 | 84 | # Theme options are theme-specific and customize the look and feel of a theme 85 | # further. For a list of options available for each theme, see the 86 | # documentation. 87 | # 88 | html_theme_options = { 89 | "analytics_id": "UA-36783686-3", 90 | "collapse_navigation": False, 91 | } 92 | 93 | # Add any paths that contain custom static files (such as style sheets) here, 94 | # relative to this directory. They are copied after the builtin static files, 95 | # so a file named "default.css" will overwrite the builtin "default.css". 96 | html_static_path = ["_static"] 97 | 98 | # Custom sidebar templates, must be a dictionary that maps document names 99 | # to template names. 100 | # 101 | # The default sidebars (for documents that don't match any pattern) are 102 | # defined by theme itself. Builtin themes are using these templates by 103 | # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', 104 | # 'searchbox.html']``. 105 | # 106 | html_sidebars = {} 107 | 108 | # If true, links to the reST sources are added to the pages. 109 | html_show_sourcelink = False 110 | 111 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 112 | html_show_sphinx = False 113 | 114 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 115 | html_show_copyright = True 116 | 117 | # -- Options for HTMLHelp output --------------------------------------------- 118 | 119 | # Output file base name for HTML help builder. 120 | htmlhelp_basename = "Rambodoc" 121 | 122 | # -- Options for LaTeX output ------------------------------------------------ 123 | 124 | latex_elements = { 125 | # The paper size ('letterpaper' or 'a4paper'). 126 | # 127 | # 'papersize': 'letterpaper', 128 | # The font size ('10pt', '11pt' or '12pt'). 129 | # 130 | # 'pointsize': '10pt', 131 | # Additional stuff for the LaTeX preamble. 132 | # 133 | # 'preamble': '', 134 | # Latex figure (float) alignment 135 | # 136 | # 'figure_align': 'htbp', 137 | } 138 | 139 | # Grouping the document tree into LaTeX files. List of tuples 140 | # (source start file, target name, title, 141 | # author, documentclass [howto, manual, or own class]). 142 | latex_documents = [ 143 | (master_doc, "Rambo.tex", "Rambo Documentation", "Terminal Labs", "manual"), 144 | ] 145 | 146 | 147 | # -- Options for manual page output ------------------------------------------ 148 | 149 | # One entry per manual page. List of tuples 150 | # (source start file, name, description, authors, manual section). 151 | man_pages = [(master_doc, "rambo", "Rambo Documentation", [author], 1)] 152 | 153 | 154 | # -- Options for Texinfo output ---------------------------------------------- 155 | 156 | # Grouping the document tree into Texinfo files. List of tuples 157 | # (source start file, target name, title, author, 158 | # dir menu entry, description, category) 159 | texinfo_documents = [ 160 | ( 161 | master_doc, 162 | "Rambo", 163 | "Rambo Documentation", 164 | author, 165 | "Rambo", 166 | "One line description of project.", 167 | "Miscellaneous", 168 | ), 169 | ] 170 | 171 | source_parsers = { 172 | ".md": CommonMarkParser, 173 | } 174 | 175 | source_suffix = [".rst", ".md"] 176 | -------------------------------------------------------------------------------- /docs/core/api.md: -------------------------------------------------------------------------------- 1 | # Python API 2 | 3 | Rambo's CLI and Python API are compatible. In other words, what you can do in the CLI, you can do in the Python API. To accomplish this the CLI is largely dependant on the Python API. You can access the Python API by importing the various functions in app.py, such as with `from rambo.app import up` 4 | 5 | Through the Python API you can call `up` and `destroy` to create and destroy VMs. `ssh` is also available and presents an interactive shell to the VM, as if you ran `rambo ssh` with the CLI. 6 | 7 | CLI options are available to be set as either functions in app.py, or as parameters to those functions, depending on whether the CLI option was on `rambo` or a command (e.g. `up`). For instance, the following are equivalent: 8 | 9 | ```shell 10 | rambo --vagrant-cwd /sample/path up -p virtualbox 11 | ``` 12 | 13 | ```python 14 | from rambo.app import up 15 | up(vagrant_cwd,"/sample/path", provider="virtualbox") 16 | ``` 17 | -------------------------------------------------------------------------------- /docs/core/cli.md: -------------------------------------------------------------------------------- 1 | # CLI 2 | 3 | Rambo's CLI is the expected normal way people interact with Rambo. At it's core, Rambo is an interface to Vagrant. Rambo duplicates several commands from Vagrant, that are either commonly used, or Rambo needs to do some preemptive work for before passing the reigns to Vagrant. For most other Vagrant commands, you can call Vagrant through Rambo. Many commands have various options that have defaults that are used when the option is not specified, e.g. `rambo up` defaults using VirtualBox as the `provider`. 4 | 5 | This is a short list of Rambo's commands, followed by a more detailed explanation of each: 6 | 7 | ## Commands 8 | 9 | - [createproject](#createproject): Create a Rambo project dir with basic setup. 10 | - [destroy](#destroy): Destroy a VM / container and all its metadata. Default leaves logs. 11 | - [export-vagrant-conf](#export-vagrant-conf): Get Vagrant configuration. 12 | - [halt](#halt): Halt VM. 13 | - [install-plugins](#install-plugins): Install Vagrant plugins. 14 | - [scp](#scp): Transfer files with scp. 15 | - [ssh](#ssh): Connect with `vagrant ssh` 16 | - [up](#up): Start a VM / container with `vagrant up`. 17 | - [vagrant](#vagrant): Run a Vagrant command through Rambo. 18 | 19 | ### createproject 20 | 21 | Create project takes an arguement for the name to give to the project it creates. It will create a directory in the CWD for this project. Upon creation, this project directory will contain a `rambo.conf` file, an `auth` directory, and a `saltstack` directory. 22 | 23 | - `rambo.conf` is the config file that is required to be present in your project to run `rambo up`, and is described [later in conf.md](conf). 24 | - `auth` contains some sample scripts that will aid in setting up keys / tokens for the cloud providers. It is not required. How to use that is described in the cloud provider specific documentation. 25 | - `saltstack` is a basic set of SaltStack configuration code that Rambo offers. [It can be modified for custom configuration.](customizing) 26 | 27 | ### destroy 28 | 29 | Destroy a VM / container. This will tell vagrant to forcibly destroy a VM, and to also destroy its Rambo metadata (provider and random_tag), and Vagrant metadata (`.vagrant` dir). 30 | 31 | ### export-vagrant-conf 32 | 33 | Places the default `Vagrantfile` and its resources (`vagrant` dir, `settings.json`) in the CWD for [customizing](customizing). 34 | 35 | ### halt 36 | 37 | Tells Vagrant to 'halt' the VM. Useful to free the Host's resources without destroying the VM. 38 | 39 | ### install-plugins 40 | 41 | Install passed args as Vagrant plugins. `all` or no args installs all default Vagrant plugins from host platform specific list. 42 | 43 | ### scp 44 | 45 | Transfer files or directories with scp. Accepts two args in one of the 46 | following forms: 47 | 48 | 49 | 50 | : 51 | 52 | : 53 | 54 | [vm_name]: 55 | 56 | [vm_name]: 57 | 58 | For example: `rambo scp localfile.txt remotefile.txt` 59 | 60 | ### ssh 61 | 62 | Connect to the VM / container over SSH. With `-c` / `--command`, will executed an SSH command directly. 63 | 64 | ### up 65 | 66 | Start a VM or container. Will create one and begin provisioning it if it did not already exist. Accepts many options to set aspects of your VM. Precedence is CLI > Config > Env Var > defaults. 67 | 68 | ### vagrant 69 | 70 | Accepts any args and forwards them to Vagrant directly, allowing you to run any Vagrant command. Rambo has first-class duplicates or wrappers for the most common Vagrant commands, but for less common commands or commands that are not customized, they don't need to be duplicated, so we call them directly. 71 | 72 | ## rambo.conf 73 | 74 | The `rambo.conf` file is used to add options to various Rambo commands without having to pass them to the CLI. This is encouraged and has a few benefits. See the following quick example: 75 | 76 | ```ini 77 | [up] 78 | provider = digitalocean 79 | guest_os = centos-7 80 | ``` 81 | 82 | is equivalent to: 83 | 84 | ```bash 85 | rambo up --provider digitalocean --guest-os centos-7 86 | ``` 87 | 88 | An optional `my_rambo.conf` is also used, so you can have personallized and untracked configuration. 89 | 90 | For a more detailed description, see the separate [rambo.conf docs](conf). 91 | 92 | ## Environment Variables 93 | 94 | ### RAMBO_ env vars 95 | 96 | Like the config file, options can also be specified as environment variables. All CLI options that you can pass to Rambo are available to set by environment variables. They take the form of the CLI option, but are all upper-cased, use underscores instead of dashes, and are prefixed with `RAMBO_`. E.g. `RAMBO_GUEST_OS` is the environment variable for the CLI option `--guest-os`. 97 | 98 | ### VAGRANT_ env vars 99 | 100 | **This is strongly discouraged.** 101 | 102 | Rambo uses Vagrant, so Vagrant specific environment variables can be used. Rambo itself sets some of these after its CLI invocation, so these may be overridden by Rambo. We do not support using these env vars manually with Rambo. 103 | -------------------------------------------------------------------------------- /docs/core/conf.md: -------------------------------------------------------------------------------- 1 | # rambo.conf 2 | 3 | The `rambo.conf` file is used to add options to various Rambo commands without having to pass them to the CLI. This is encouraged and has a few benefits. See the following quick example: 4 | 5 | ```ini 6 | [up] 7 | provider = digitalocean 8 | guest_os = centos-7 9 | ``` 10 | 11 | is equivalent to: 12 | 13 | ```bash 14 | rambo up --provider digitalocean --guest-os centos-7 15 | ``` 16 | 17 | The `rambo.conf` file is required at the top level in your project directory. It is an INI config file that can specify options. Options passed to the CLI will take precedence over options set via this config file. If you're repeating the same CLI options, setting those options in this config might make your life a little easier. Further, if you intend on tracking your Rambo project in version control, it can be very handy to set some options in this config that match the purpose of your project. 18 | 19 | Options can be set in `rambo.conf`. For example, a useful `rambo.conf` could look like this: 20 | 21 | ```ini 22 | [up] 23 | provider = digitalocean 24 | guest_os = centos-7 25 | ``` 26 | 27 | which is equivalent to: 28 | 29 | ```bash 30 | rambo up --provider digitalocean --guest-os centos-7 31 | ``` 32 | 33 | Setting the config file to this allows you to type simply `rambo up` to run `up` with the `provider` and `guest-os` options set in the rambo.conf, and not specified in the CLI. 34 | 35 | ## Option Names 36 | 37 | The options in the conf file are the same as the full option names in the CLI, with preceeding dashes removed and other dashes replaced with underscores. As examples: 38 | 39 | - `vagrant_dotfile_path` in the conf, corresponds to `--vagrant-dotfile-path` in the CLI 40 | - `provider` in the conf, corresponds to `--provider` or `-p` in the CLI 41 | - `guest_os` in the conf, corresponds to `--guest-os` or `-o` in the CLI 42 | - `ram_size` in the conf, corresponds to `--ram-size` or `-r` in the CLI 43 | 44 | The full list is available with `rambo up --help`. 45 | 46 | ## my_rambo.conf 47 | 48 | Rambo will also load configuration from a `my_rambo.conf` file. This file is optional, and configuration found here takes precedence over the main `rambo.conf` file. 49 | 50 | The intention is that a `rambo.conf` file is tracked (e.g. with git), but so that a shared project can have its configuration easily altered by individual users, values may be overridden by an untracked `my_rambo.conf`. For example, a project may use `provider = ec2`, but individual contributors may want to develop locally in Docker or VirtualBox instead. 51 | 52 | ## Option Precedence 53 | 54 | The precedence for options is: 55 | 56 | CLI > Environment Variable > `my_rambo.conf` > `rambo.conf` > defaults 57 | 58 | When an option is set in more than one place, the CLI takes precedence. Defaults are overridable by everything. 59 | 60 | ### Example 1 61 | 62 | ```ini 63 | # rambo.conf 64 | 65 | [up] 66 | provider = digitalocean 67 | ``` 68 | 69 | ```bash 70 | rambo up -p virtualbox 71 | ``` 72 | 73 | yields the provider `virtualbox`. 74 | 75 | ### Example 2 76 | 77 | If instead, the config still read 78 | 79 | ```ini 80 | # rambo.conf 81 | 82 | [up] 83 | provider = digitalocean 84 | ``` 85 | 86 | ```bash 87 | rambo up 88 | ``` 89 | 90 | yields the provider `digitalocean`. 91 | 92 | ### Example 3 93 | 94 | ```bash 95 | RAMBO_PROVIDER=digitalocean rambo up -p ec2 96 | ``` 97 | 98 | yields the provider `ec2`. 99 | 100 | ### Example 4 101 | 102 | ```ini 103 | # rambo.conf 104 | 105 | [up] 106 | provider = ec2 107 | ``` 108 | 109 | ```bash 110 | RAMBO_PROVIDER=digitalocean rambo up 111 | ``` 112 | 113 | yields the provider `digitalocean`. 114 | 115 | ### Example 4 116 | 117 | ```ini 118 | # my_rambo.conf 119 | 120 | [up] 121 | provider = virtualbox 122 | ``` 123 | 124 | ```ini 125 | # rambo.conf 126 | 127 | [up] 128 | provider = ec2 129 | ``` 130 | 131 | yields the provider `virtualbox`. 132 | -------------------------------------------------------------------------------- /docs/core/customizing.md: -------------------------------------------------------------------------------- 1 | # Customizing Rambo 2 | 3 | Rambo aims to make it easy for you to switch providers and customize provisioning. Below is documentation about how to go about cusomizing your provisioning with Salt Stack, switching provisioners, adding providers, and customizing provider-specific code. 4 | 5 | Rambo is young, and we'd love to improve Rambo and make this all easier still. Please consider opening [a pull request](https://github.com/terminal-labs/rambo/compare) if you add another provisioner or provider, or make any customization that would be a good contribution. :) 6 | 7 | ## Custom Provisioning 8 | Rambo provides a basic default provisioning with Vagrant and SaltStack. To build out what you need for your project you will need your own customized provisioning. You can do this provisioning with any tool you like through Vagrant, such as with shell scripts, SaltStack, or any other provisioning tool. 9 | 10 | All Rambo code that is used to provision the VM is kept where Rambo is installed. This directory is copied into the VM at an early stage in the spawn process so that it can be invoked to provision the VM. 11 | 12 | ### SaltStack 13 | Rambo has [a few basic Salt States](https://github.com/terminal-labs/sample-states/tree/basic) available that are placed in your project dir by `rambo createproject`. These run unless removed, and work out of the box. The `saltstack` dir can also be modified however you like for any SaltStack provisioning. You can add your custom Salt States right into the Salt code and they should be automatically picked up and used. 14 | 15 | ### Other Provisioners 16 | If you want to add provisioning with any other tool, you will need to modify the Vagrantfiles to add that provisioning. To export the Vagrantfiles, run `rambo export-vagrant-conf` inside your project dir. This will drop the Vagrantfile and several of its dependencies. You can likely add custom provisioning straight to the main Vagrantfile without worrying about the other files. 17 | 18 | For example, if you'd like to provision with Ansible, you will need to add custom Vagrant code to make this work. There are many useful introductions to various provisioners on Vagrant's website, such as the page on [Ansible Provisioning](https://www.vagrantup.com/docs/provisioning/ansible.html). 19 | 20 | ## Custom Providers / Provider configuration 21 | 22 | First grab the Vagrantfiles with `rambo export-vagrant-conf`. 23 | 24 | The main Vagrantfile is extended by other provider-specific vagrantfiles located in `vagrant/vagrantfiles` such as `vagrant/vagrantfiles/virtualbox` for VirtualBox. If you need to customize how Rambo works with a provider manually, these are the files you'll need to modify. For instance, you may want to customize many aspects of your VirtualBox VM's networking. 25 | -------------------------------------------------------------------------------- /docs/core/provisioning.md: -------------------------------------------------------------------------------- 1 | # Basic Provisioning 2 | 3 | By default Rambo will do a small amount of basic provisioning. It will: 4 | 5 | - Set the hostname 6 | - Sync your project directory 7 | - Sync custom directories 8 | - Run a custom command 9 | 10 | ## Hostname 11 | 12 | The hostname can be set by specifying the `--hostname` option. 13 | 14 | ## Syncing 15 | 16 | ### Sync Types 17 | 18 | The default syncing method is whatever Vagrant uses as its default ([link](https://www.vagrantup.com/docs/synced-folders/basic_usage.html#type)). 19 | 20 | > Vagrant will automatically choose the best synced folder option for your environment. 21 | 22 | For basic usage, that means the default for Rambo, using the VirtualBox provider, is shared folders. 23 | 24 | This can be changed to any of the syncing methods that Vagrant supports; see [their docs](https://www.vagrantup.com/docs/synced-folders/) for details. Syncing can also be entirely disabled. Common options are: 25 | 26 | - `disabled` 27 | - `rsync` 28 | - `shared` 29 | 30 | These can be specified with `--sync-type`. 31 | 32 | `--sync-type disabled` turns off syncing entirely. 33 | 34 | ### Synced directories 35 | 36 | Rambo will sync your project directory and any custom directories. The following mappings are synced in the order listed. 37 | 38 | Host (source) VM (target) 39 | ============= =========== 40 | 41 | [ project dir] /vagrant 42 | custom dirs on host custom dirs on VM 43 | 44 | 45 | Custom dirs can be passed to `--sync-dirs` as a list of lists of the form 46 | 47 | ``` 48 | --sync-dirs "[['path on host', 'absolute path on VM'], ['second path on host', 'second absolute path on VM']]" 49 | ``` 50 | 51 | This list of lists must 52 | 53 | - be able to be evaluated by Python and Ruby as a list of lists of strings, 54 | - specify target paths with absolute paths 55 | 56 | Since this is rather cumbersome to pass in the CLI, remember that it can also be set in the [configuration file](../core/conf), like 57 | 58 | ``` 59 | [up] 60 | sync_dirs = [['path on host', 'absolute path on VM'], ['second path on host', 'second absolute path on VM']] 61 | ``` 62 | 63 | ## Command Provisioning 64 | 65 | Rambo is able to provision with a command. This command can be passed to `rambo up` in the cli, or set in the `rambo.conf`. For example: 66 | 67 | ```shell 68 | rambo up -c "hostname" 69 | ``` 70 | 71 | will provision and run the command `hostname`, displaying the hostname of the instance as part of the provisioning process. 72 | 73 | This command is intended to be the entry point to any user-controlled way of provisioning, and easily used with other synced code. For instance, this command could run a script, or invoke configuration management tools like Salt or Puppet. 74 | 75 | For example, this setup will run a command, that calls a custom script that installs Salt and runs a highstate. This example works as-is with the basic Salt setup that Rambo provides in a new project. 76 | 77 | ```ini 78 | # rambo.conf 79 | 80 | [up] 81 | provider = virtualbox 82 | box = ubuntu/bionic64 83 | sync_dirs = [["saltstack/etc", "/etc/salt"], ["saltstack/srv", "/srv"]] 84 | command = bash /vagrant/provision.sh 85 | ``` 86 | 87 | ```bash 88 | # provision.sh 89 | if [ ! -f "bootstrap.sh" ]; then 90 | echo "Updating system and installing curl" 91 | apt update 92 | apt install curl -y 93 | 94 | echo "Downloading Salt Bootstrap" 95 | curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com 96 | fi 97 | 98 | echo "Installing Salt with master and Python 3" 99 | bash bootstrap-salt.sh -M -x python3 100 | 101 | echo "Accepting the local minion's key" 102 | salt-key -A -y 103 | 104 | 105 | # Is Salt ready yet? Proceed once it is. 106 | salt \* test.ping --force-color 107 | while [ $? -ne 0 ] 108 | do 109 | echo "Waiting for Salt to be up. Testing again." 110 | salt \* test.ping --force-color 111 | done 112 | 113 | 114 | echo "Running highstate. Waiting..." 115 | salt \* state.highstate --force-color 116 | ``` 117 | 118 | That script can then be used with `rambo up`. 119 | 120 | Alternatively, the command can be passed in the CLI, like 121 | 122 | ```bash 123 | rambo up -c 'bash /vagrant/provision.sh' 124 | ``` 125 | 126 | Note that when this is done, keep in mind that double quotes may use shell expansion. So if 127 | 128 | ```bash 129 | rambo up -c "echo $PWD" 130 | ``` 131 | 132 | is used with bash, the working directory _of the host_ will be echoed. 133 | -------------------------------------------------------------------------------- /docs/core/quickstart.md: -------------------------------------------------------------------------------- 1 | # Quickstart 2 | 3 | ## Hardware Recommendations 4 | For running VMs locally in VirtualBox (default), we suggest a minimum of: 5 | 6 | * Reasonably fast cpu with 2 cores and virtualization hardware support (e.g. an Intel i7-3612QM 2.1GHz, 4 core chip with VT-x) 7 | * 8GB RAM 8 | * 16GB free drive space 9 | 10 | For running containers locally (e.g. LXC) or spawning cloud based VMs (e.g. AWS EC2) you can get away with comparatively slow computer and you don't need VT-x, you don't even need VirtualBox. In fact, these providers can be managed from just a [Raspberry Pi](https://www.raspberrypi.org/). 11 | 12 | ## Supported Host Operating Systems 13 | 14 | - [Ubuntu 16.04 or newer](https://www.ubuntu.com/download/desktop) 15 | 16 | - [OSX](https://www.apple.com/mac-mini/) 17 | 18 | We expect it's likely you can get Rambo to work on any Unix-like system, but your milleage may vary. So far we have made no effort to get this working with Windows. Contributions are very welcome. 19 | 20 | ## Installation 21 | 22 | 1. Install / Use Python 3.6+ and pip (for example with a Virtual Environment). 23 | 24 | 1. Download and install [VirtualBox](https://www.virtualbox.org/) 5.1 or newer. 25 | 26 | 1. Download and install [Vagrant](https://www.vagrantup.com/). 27 | 28 | 1. Install Rambo with pip, 29 | 30 | - [latest release](https://github.com/terminal-labs/rambo/releases) with pypi: `pip install rambo-vagrant`, or 31 | - [from source](https://github.com/terminal-labs/rambo): go into the repository and `pip install -e .` 32 | 33 | 1. Install plugins with `rambo install-plugins` 34 | 35 | 36 | Note: Vagrant and VirtualBox update frequently, and sometimes with breaking changes. Additionally there are may be provider specific dependencies. 37 | 38 | ## Create Project 39 | 40 | Now that Rambo is installed, you must initialize a project. This will create a directory that will be tied to your VM. Outside of this directory, Rambo won't be able to find the VM to control it. This also means that if you want to create or control multiple VMs with Rambo, you can, by simply creating more projects and running Rambo commands from the directories where they reside. Create and go to your project: 41 | 42 | ``` 43 | rambo createproject yourprojectname 44 | cd yourprojectname 45 | ``` 46 | 47 | In this project directory Rambo gave you a few things to help you get started, a `rambo.conf`, `auth` dir, and `saltstack` dir. These are basic configs to start you out. You don't need to modify them for basic use. 48 | 49 | ## Providers 50 | 51 | Rambo supports various providers, and aims to let you switch between them as easily as possible. Nevertheless, some providers do have particular considerations, such as setting up keys and payment for cloud services, or specific dependencies for the host OS. This is a list of Rambo's supported providers, with links to specific documentation pages for each. 52 | 53 | - [AWS EC2](../../providers/aws-ec2) 54 | - [DigitalOcean](../../providers/digitalocean) 55 | - [Docker](../../providers/docker) 56 | - [LXC](../../providers/lxc) 57 | - VirtualBox (see below) 58 | 59 | ### Default Provider - VirtualBox: 60 | 61 | If you never specify any provider, Rambo will use the VirtualBox as its default choice, and is simply 62 | 63 | `rambo up` 64 | 65 | ## Provisioning 66 | 67 | Rambo does very little provisioning on its own. It can set a hostname, set up some synced directories, and allow a command to be run. That command is your entry point to doing anything else. 68 | 69 | See [provisioning](provisioning) for some examples. 70 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | Welcome to Rambo's documentation! 2 | ================================= 3 | 4 | |Join the chat at https://gitter.im/terminal-labs/rambo| 5 | 6 | Currently viewing version v\ |version| 7 | 8 | Quickstart 9 | ---------- 10 | 11 | To get started fast, see `Quickstart `__. 12 | 13 | .. toctree:: 14 | :caption: Usage Documentation 15 | :maxdepth: 2 16 | :glob: 17 | 18 | core/quickstart 19 | core/* 20 | 21 | .. toctree:: 22 | :caption: Provider Specific Documentation 23 | :maxdepth: 2 24 | :glob: 25 | 26 | providers/* 27 | 28 | .. toctree:: 29 | :caption: Changelog: 30 | :maxdepth: 1 31 | :glob: 32 | 33 | CHANGELOG 34 | 35 | What's Rambo For? 36 | ----------------- 37 | 38 | This project is for provisioning and configuring virtual machines (and 39 | containers) in a simple, predictable, and highly reproducible way. Just 40 | run one command and your VM is up, code is deployed, and your app is 41 | running, on any supported platform. 42 | 43 | At this time this repo allows you to create a Linux VM on multiple 44 | providers (AWS EC2, DigitalOcean, VirtualBox, LXC). Several Operating 45 | Systems are available on select providers. The base machine 46 | configuration is a Ubuntu 16.04 64bit OS with 1024MB RAM, and 30GB 47 | drive. 48 | 49 | One of the goals of this project is be able to run a simple command and 50 | have a new VM be created on your provider of choice. Once the VM is 51 | initialized a provisioner is used to deploy code to and provision your 52 | machine. The provisioning configuration code will run the same regardless of which 53 | provider is actually running the machine. You can easily cycle your VMs by destroying 54 | and rebuilding them. 55 | 56 | Another goal of this repo is to have the spawned VMs be maximally 57 | similar across providers. Usually, your configuration will not need to 58 | change at all and will simply run on all providers. 59 | 60 | By default Rambo offers a basic VM configuration with [SaltStack](https://github.com/saltstack/salt/), but you can customize this. See `Customizing Rambo `__ 61 | for that. 62 | 63 | Basic Usage 64 | ----------- 65 | 66 | Once `installed `__, you can run one of 67 | these commands to get your VM: 68 | 69 | for `VirtualBox `__ run 70 | 71 | :: 72 | 73 | $ rambo up 74 | $ rambo ssh 75 | 76 | for `AWS EC2 `__ run 77 | 78 | :: 79 | 80 | $ rambo up -p ec2 81 | $ rambo ssh 82 | 83 | for `DigitalOcean `__ run 84 | 85 | :: 86 | 87 | $ rambo up -p digitalocean 88 | $ rambo ssh 89 | 90 | for `Docker `__ run 91 | 92 | :: 93 | 94 | $ rambo up -p docker 95 | $ rambo ssh 96 | 97 | for `LXC `__ run 98 | 99 | :: 100 | 101 | $ rambo up -p lxc 102 | $ rambo ssh 103 | 104 | 105 | Contributing 106 | ------------ 107 | 108 | We heartily welcome any contirubtions to this project, whether in the 109 | form of commenting on or posting an issue, or development. If you would 110 | like to submit a pull request, you might first want to look at our 111 | development `guidelines `__ for this project. 112 | 113 | Special Thanks 114 | -------------- 115 | 116 | Thanks go out to the Vagrant community and HashiCorp. Vagrant is a great 117 | tool it has helped us a lot over the years. 118 | 119 | Rambo is supported by `Terminal Labs`_. 120 | 121 | .. _`Terminal Labs`: https://terminallabs.com 122 | .. |Join the chat at https://gitter.im/terminal-labs/rambo| image:: https://badges.gitter.im/terminal-labs/rambo.svg 123 | :target: https://gitter.im/terminal-labs/rambo?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge 124 | -------------------------------------------------------------------------------- /docs/providers/aws-ec2.md: -------------------------------------------------------------------------------- 1 | # AWS EC2 2 | 3 | ## Create Account 4 | 5 | After you installed the dependencies on your host computer you now need to create an account at AWS. 6 | This repo will create real resources on AWS so you need to provide AWS with valid payment and remember you might rack up a bill if you run a whole bunch of machines. You have been warned. 7 | 8 | ## Create SSH Keys 9 | 10 | Next you need to create a SSH key pair for AWS. 11 | 12 | Run: 13 | ``` 14 | mkdir -p auth/keys 15 | cd auth/keys 16 | ssh-keygen -t rsa -N '' -f "aws.pem" 17 | ``` 18 | 19 | *If you want multiple users or computers to access the same AWS profile or team, you must have unique key names. For example, you will need to change the base name of your `.pem` and `.pem.pub` files to something else like `aws-myname`. 20 | 21 | **Create a new key and name the key the same as the base name of your SSH key. If AWS's key name and the one one your host don't match, you won't communicate to your VM.** 22 | 23 | Now go to AWS's "EC2 Dashboard", on the left hand side go to "Key Pairs" and click the "Import Key Pair" button. 24 | 25 | Here are instructions on how to setup SSH keys with aws: 26 | 27 | http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html 28 | 29 | You will need to copy the contents of `aws.pem.pub`. 30 | 31 | **NOTE: You need the aws.pem file and the aws.pem.pub file. The aws.pem file needs permissions set to 600 and The aws.pem.pub file needs permissions set to 644** 32 | 33 | Be careful not to commit this file to the repo. We setup this repo to ignore all files ending in `.pem`. But, you could theoretically still commit the pem file (by forcing a commit for example). 34 | Store this pem file in a safe place with restricted access. Anyone who has this file can log into your machines on AWS and run arbitrary commands on them. 35 | 36 | ## Create Security Group 37 | 38 | Now go to AWS's "EC2 Dashboard", on the left hand side go to "Security Group" and click the "Create Security Group" button. 39 | 40 | Name the new security group **salted_server**. 41 | 42 | Add these inbound rules to the security group 43 | ``` 44 | "All ICMP - IPv4", ICMP, 0 - 65535, anywhere 45 | 46 | "SSH", TCP, 22, anywhere 47 | 48 | "HTTP", TCP, 80, anywhere 49 | 50 | "HTTPS", TCP, 443, anywhere 51 | 52 | "Custom TCP Rule", TCP, 4505, anywhere 53 | 54 | "Custom TCP Rule", TCP, 4506, anywhere 55 | 56 | "Custom TCP Rule", TCP, 5000, anywhere 57 | 58 | "Custom TCP Rule", TCP, 8080, anywhere 59 | 60 | "Custom TCP Rule", TCP, 8888, anywhere 61 | ``` 62 | 63 | ## Create API Token 64 | 65 | Next you need to manually create an API access token on AWS. 66 | 67 | Go to the "IAM Dashboard", then go to "users", now click on the user who will be creating the AWS EC2 instances. Click on the "Security Credectials" tab, click the "create access key" button. 68 | 69 | You MUST get both the **Access key ID** and the **Secret access key**. 70 | 71 | **NOTE: AWS will only show you this key ONCE.** 72 | 73 | ## Create SSH Key 74 | 75 | Now you need to create or upload an ssh key. This key is not your user's general key in IAM, but the one in the EC2 -> Key Pairs section. 76 | 77 | ## Edit Script to Load Environment Variables 78 | 79 | Here is the contents of the aws.env.sh file. Edit it by replacing the placeholder tags with your keys and tokens. 80 | 81 | ``` 82 | #!/bin/bash 83 | 84 | # for aws 85 | # associated with your aws user 86 | export AWS_ACCESS_KEY_ID= 87 | export AWS_SECRET_ACCESS_KEY= 88 | # associated with ec2 specifically (not your user's general ssh key) 89 | export AWS_KEYPAIR_NAME="name" 90 | export AWS_SSH_PRIVKEY="auth/keys/name.pem" 91 | ``` 92 | 93 | Put your aws access key token in the line. 94 | `export AWS_ACCESS_KEY_ID=` 95 | 96 | Put your aws secret acces key token in the line: 97 | `export AWS_SECRET_ACCESS_KEY=` 98 | 99 | Put the **name** of your aws ssh private key in the line: 100 | `export AWS_KEYPAIR_NAME="name"` 101 | 102 | Put the **path** to your aws ssh private key in the line: 103 | `export AWS_SSH_PRIVKEY="auth/keys/name.pem"` 104 | 105 | After editing, your aws.env.sh file will look similar to this: 106 | 107 | ``` 108 | #!/bin/bash 109 | 110 | # for aws 111 | # associated with your aws user 112 | export AWS_ACCESS_KEY_ID="AKIAITT673DAF4YNV7MA" 113 | export AWS_SECRET_ACCESS_KEY="m25AyjXtiYB2cCWMv1vQeyZtWqiWg0nqxi2Wm2QX" 114 | # associated with ec2 specifically (not your user's general ssh key) 115 | export AWS_KEYPAIR_NAME="name" 116 | export AWS_SSH_PRIVKEY="auth/keys/name.pem" 117 | ``` 118 | 119 | Note: the public key must be in the same dir as the private key and the public key must share the same base name as the private key (just append ".pub" on the public key file's name). 120 | 121 | Now you need to source the aws.env.sh file. cd into the repo and run: 122 | 123 | `source aws.env.sh` 124 | 125 | ## Launching Your AWS EC2 Instance 126 | Finally, run: 127 | 128 | ``` 129 | rambo up -p ec2 130 | rambo ssh 131 | ``` 132 | -------------------------------------------------------------------------------- /docs/providers/digitalocean.md: -------------------------------------------------------------------------------- 1 | # DigitalOcean 2 | 3 | ## Create Account 4 | 5 | After you installed the dependencies on your host computer you now need to create an account at DigitalOcean. 6 | 7 | This repo will create real resources on DigitalOcean so you need to provide DigitalOcean with valid payment and remember you might rack up a bill if you run a whole bunch of machines. You have been warned. 8 | 9 | ## Create SSH Keys 10 | 11 | Next you need to create a SSH key pair for DigitalOcean. 12 | 13 | Run*: 14 | ``` 15 | mkdir -p auth/keys 16 | cd auth/keys 17 | ssh-keygen -t rsa -N '' -f "digitalocean.pem" 18 | ``` 19 | 20 | *If you want multiple users or computers to access the same DigitalOcean profile or team, you must have unique key names. For example, you will need to change the base name of your `.pem` and `.pem.pub` files to something else like `digitalocean-myname`. 21 | 22 | Now go to [https://cloud.digitalocean.com/settings/security](https://cloud.digitalocean.com/settings/security) 23 | 24 | **Create a new key and name the key the same as the base name of your SSH key. If DigitalOcean's key name and the one one your host don't match, you won't communicate to your VM.** 25 | 26 | Copy the contents of `digitalocean.pem.pub` into the new key field. 27 | 28 | Here are instructions on how to setup SSH keys with DigitalOcean: 29 | 30 | [https://www.digitalocean.com/community/tutorials/how-to-use-ssh-keys-with-digitalocean-droplets](https://www.digitalocean.com/community/tutorials/how-to-use-ssh-keys-with-digitalocean-droplets) 31 | 32 | You will need to copy the contents of `digitalocean.pem.pub`. 33 | 34 | **NOTE: You need the `digitalocean.pem` file and the `digitalocean.pem.pub` file. The `digitalocean.pem` file needs permissions set to 600 and The `digitalocean.pem.pub` file needs permissions set to 644** 35 | 36 | Be careful not to commit this file to the repo. We setup this repo to ignore all files ending in `.pem`. But, you could theoretically still commit the pem file (by forcing a commit for example). 37 | Store this pem file in a safe place with restricted access. Anyone who has this file can log into your machines on DigitalOcean and run arbitrary commands on them. 38 | 39 | ## Create API Token 40 | 41 | Next you need to manually create an API access token on digitalocean.com 42 | 43 | Go to: 44 | 45 | [https://cloud.digitalocean.com/settings/api/](https://cloud.digitalocean.com/settings/api/) 46 | 47 | **NOTE: DigitalOcean will only show you this key ONCE.** 48 | 49 | Store this token in a safe place with restricted access. Anyone who has this token can create, edit, or destroy resources on digital ocean, they could rack up a huge bill for you or shut down all your vms. 50 | 51 | ## Edit Script to Load Environment Variables 52 | 53 | Here is the contents of the `digitalocean.env.sh` file. Edit it by replacing the placeholder tags with your key and token. 54 | 55 | ``` 56 | #!/bin/bash 57 | 58 | # for digitalocean 59 | export DIGITALOCEAN_TOKEN= 60 | export DIGITALOCEAN_PRIVATE_KEY_PATH="auth/keys/digitalocean.pem" 61 | ``` 62 | 63 | Put your DigitalOcean API token in the line: 64 | `export DIGITALOCEAN_TOKEN=` 65 | 66 | Put the **path** to your DigitalOcean ssh private key in the line: 67 | `export DIGITALOCEAN_PRIVATE_KEY_PATH=` 68 | 69 | After editing, your `digitalocean.env.sh` file will look similar to this: 70 | 71 | ``` 72 | #!/bin/bash 73 | 74 | # for digitalocean 75 | export DIGITALOCEAN_TOKEN="0bf1d884e737417e2ea6f7a29c6035752bf8c31b366489c5366745dad62a8132" 76 | export DIGITALOCEAN_PRIVATE_KEY_PATH="auth/keys/digitalocean.pem" 77 | ``` 78 | 79 | 80 | Note: the public key must be in the same dir as the private key and the public key must share the same base name as the private key (just append ".pub" on the public key file's name) 81 | 82 | Now you need to source the `digitalocean.env.sh` file. cd into the repo and run: 83 | 84 | `source digitalocean.env.sh` 85 | 86 | ## Launching Your DigitalOcean Instance 87 | Finally, run: 88 | 89 | ``` 90 | rambo up -p digitalocean 91 | rambo ssh 92 | ``` 93 | -------------------------------------------------------------------------------- /docs/providers/docker.md: -------------------------------------------------------------------------------- 1 | # Docker 2 | 3 | At this time Docker is supported by using an intermediate host. The intermediate host is created with VirtualBox, and is the same OS as is used for the Docker container inside it. Setting both of those OSes to be the same avoids certain complexities and potential problems that would otherwise present. 4 | 5 | Basic usage of Docker: 6 | 7 | ``` 8 | rambo up -p docker 9 | rambo ssh 10 | ``` 11 | -------------------------------------------------------------------------------- /docs/providers/lxc.md: -------------------------------------------------------------------------------- 1 | # LXC 2 | 3 | **NOTE: At this time, this will only work on Ubuntu 16.04+ host OS** 4 | 5 | At this time LXC is supported natively on Ubuntu 16.04. For this native support, you need a few additional dependencies. They can all be installed with this: 6 | 7 | ``` 8 | sudo apt install -y build-essential linux-headers-$(uname -r) lxc lxc-templates cgroup-lite redir 9 | ``` 10 | 11 | After that, starting an LXC container with basic usage is: 12 | 13 | ``` 14 | rambo up -p lxc 15 | rambo ssh 16 | ``` 17 | 18 | **Note:** At this time using LXC as a provider will require root priveleges / sudo. 19 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [project] 2 | authors = [ 3 | { name = "Joseph Nix", email = "nixjdm@terminallabs.com" }, 4 | { name = "Terminal Labs", email = "solutions@terminallabs.com" } 5 | ] 6 | classifiers = [ 7 | "Programming Language :: Python :: 3", 8 | "Programming Language :: Python :: 3.7", 9 | "Programming Language :: Python :: 3.8", 10 | "Programming Language :: Python :: 3.9", 11 | "Programming Language :: Python :: 3.10", 12 | ] 13 | dependencies = [ 14 | "click", 15 | "setuptools" 16 | ] 17 | description = "A Provider Agnostic Provioning Framework" 18 | license = "BSD-3-Clause" 19 | name = "Rambo-vagrant" 20 | readme = "README.md" 21 | requires-python = ">= 3.7" 22 | url="https://github.com/terminal-labs/rambo" 23 | version = "0.4.5.dev0" 24 | 25 | [project.scripts] 26 | rambo = "rambo.cli:main" 27 | 28 | [build-system] 29 | requires = ["hatchling"] 30 | build-backend = "hatchling.build" 31 | 32 | [tool.rye] 33 | managed = true 34 | dev-dependencies = [ 35 | "ipdb", 36 | "pre-commit", 37 | "recommonmark", 38 | "sphinx_rtd_theme", 39 | ] 40 | 41 | [tool.hatch.metadata] 42 | allow-direct-references = true 43 | 44 | [tool.hatch.build.targets.wheel] 45 | packages = ["rambo"] 46 | -------------------------------------------------------------------------------- /rambo/Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | # 4 | # This file is meant to contain vars that are most likely to be changed 5 | # when configuring Rambo. Further logic is loaded with a Vagrantfile for a specific provider (e.g. ec2, lxc). 6 | 7 | require "json" 8 | 9 | # Change CWD for each VM, as set by Rambo, otherwise relative path resources break. 10 | if ENV.has_key?("VAGRANT_CWD") 11 | Dir.chdir ENV["VAGRANT_CWD"] 12 | end 13 | 14 | require_relative "vagrant/modules.rb" # for random_tag 15 | 16 | ## Default Settings 17 | SETTINGS = JSON.parse(File.read('settings.json')) 18 | PROJECT_NAME = SETTINGS["PROJECT_NAME"] 19 | RANDOMTAG = random_tag() 20 | FORWARD_SSH = true 21 | if get_env_var_rb('vm_name') 22 | VM_NAME = get_env_var_rb('vm_name') 23 | else 24 | VM_NAME = PROJECT_NAME + "-" + RANDOMTAG 25 | end 26 | unless get_env_var_rb('cwd') 27 | set_env_var_rb('cwd', Dir.pwd) 28 | end 29 | 30 | unless get_env_var_rb("ENV") 31 | puts "", "***CAUTION***", 32 | "Running Vagrant directly and without %s's official CLI." % PROJECT_NAME.capitalize, 33 | "This is not supported.", 34 | "***/CAUTION***", "" 35 | end 36 | 37 | Vagrant.require_version ">= 2.1.0" 38 | 39 | orphan_links = `find . -xtype l`.split(/\n+/) 40 | for link in orphan_links 41 | puts "Deleting broken symlink #{link}" 42 | File.delete(link) 43 | end 44 | 45 | #load the rest of the vagrant ruby code 46 | provider = if get_env_var_rb("PROVIDER") 47 | get_env_var_rb("PROVIDER") 48 | elsif read_provider_file() 49 | read_provider_file() 50 | else 51 | 'virtualbox' 52 | end 53 | 54 | if (SETTINGS["PROVIDERS"].include? provider) 55 | write_provider_file(provider) 56 | load File.expand_path("vagrant/vagrantfiles/" + provider) 57 | else # Bad arg - we don't have this provider. 58 | abort("ABORTED - Provider not in providers list. Did you have a typo?") 59 | end 60 | 61 | # Set defaults if values not set, otherwise 62 | unless get_env_var_rb('GUEST_OS') 63 | set_env_var_rb('GUEST_OS', SETTINGS['GUEST_OSES_DEFAULT']) 64 | end 65 | unless get_env_var_rb('RAMSIZE') 66 | set_env_var_rb('RAMSIZE', SETTINGS['RAMSIZE_DEFAULT']) 67 | end 68 | unless get_env_var_rb('DRIVESIZE') 69 | set_env_var_rb('DRIVESIZE', SETTINGS['DRIVESIZE_DEFAULT']) 70 | end 71 | 72 | 73 | 74 | # Provisioning 75 | Vagrant.configure("2") do |config| 76 | # Set hostname 77 | if get_env_var_rb('hostname') 78 | config.vm.hostname = get_env_var_rb('hostname') 79 | else 80 | config.vm.hostname = VM_NAME 81 | end 82 | 83 | # Experiments to fix WinRM 84 | # config.vm.boot_timeout = 60 85 | # config.vm.provision "shell", inline: <<-SHELL 86 | # netsh interface ip set address "Ethernet" static 192.168.56.11 255.255.255.0 87 | # SHELL 88 | 89 | # config.vm.provision "shell", path: "provision.ps1" 90 | 91 | # config.vm.provision "shell", inline: <<-SHELL 92 | # echo "Hello, World! - inline" 93 | # winrm quickconfig -q 94 | # winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="512"}' 95 | # winrm set winrm/config '@{MaxTimeoutms="1800000"}' 96 | # winrm set winrm/config/service '@{AllowUnencrypted="true"; MaxConcurrentOperationsPerUser="120"}' 97 | # winrm set winrm/config/service/auth '@{Basic="true"}' 98 | # winrm set winrm/config/client/auth '@{Basic="true"}' 99 | # netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow 100 | # SHELL 101 | 102 | # Syncing / Shared folders 103 | if get_env_var_rb('project_dir') 104 | sync_dir = get_env_var_rb('project_dir') 105 | else 106 | sync_dir = get_env_var_rb('cwd') 107 | end 108 | 109 | if get_env_var_rb('sync_type') != 'disabled' 110 | # Sync rambo project dir to /vagrant 111 | config.vm.synced_folder sync_dir, "/vagrant", type: get_env_var_rb('sync_type') 112 | 113 | # Sync custom dirs 114 | if get_env_var_rb('sync_dirs') 115 | sync_dirs = eval(get_env_var_rb('sync_dirs')) 116 | for sd in sync_dirs 117 | config.vm.synced_folder sd[0], sd[1], type: get_env_var_rb('sync_type') 118 | end 119 | end 120 | 121 | else # Disable all syncing. Breaks default method of provisioning. 122 | config.vm.synced_folder sync_dir, "/vagrant", disabled: true 123 | end 124 | 125 | # Custom Provisioning 126 | if get_env_var_rb('command') 127 | config.vm.provision "shell", 128 | inline: get_env_var_rb('command'), keep_color: true, privileged: false 129 | end 130 | end 131 | 132 | # clean up files on the host after the guest is destroyed 133 | Vagrant.configure("2") do |config| 134 | config.trigger.after :up do |trigger| 135 | trigger.info = "Vagrant done with up." 136 | end 137 | config.trigger.after :destroy do |trigger| 138 | trigger.info = "Vagrant done with destroy." 139 | end 140 | end 141 | -------------------------------------------------------------------------------- /rambo/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/terminal-labs/rambo/16ea57a6ce6d7e5e5777669624f006b0f53d401c/rambo/__init__.py -------------------------------------------------------------------------------- /rambo/app.py: -------------------------------------------------------------------------------- 1 | import ast 2 | import errno 3 | import os 4 | import platform 5 | import pty 6 | import shutil 7 | import sys 8 | from pathlib import Path 9 | from select import select 10 | from subprocess import Popen 11 | 12 | import click 13 | from distutils.dir_util import copy_tree 14 | from distutils.errors import DistutilsFileError 15 | 16 | import rambo.options as options 17 | import rambo.utils as utils 18 | import rambo.vagrant_providers as vagrant_providers 19 | from rambo.settings import PROJECT_LOCATION 20 | from rambo.settings import PROJECT_NAME 21 | from rambo.settings import SETTINGS 22 | from rambo.utils import abort 23 | from rambo.utils import get_env_var 24 | from rambo.utils import set_env_var 25 | 26 | VAGRANT_EXE = os.getenv("VAGRANT_EXE", "vagrant") 27 | 28 | 29 | def _invoke_vagrant(cmd=None): 30 | """Pass a command to vagrant. This outputs in near real-time, 31 | logs both stderr and stdout in a combined file, and detects stderr for 32 | our own error handling. 33 | 34 | Returns returncode (exitcode) of the command. 35 | 36 | Args: 37 | cmd (str): The cmd string that is appended to `vagrant ...`, 38 | passed to the shell and executed. 39 | """ 40 | masters, slaves = zip(pty.openpty(), pty.openpty()) 41 | cmd = " ".join([VAGRANT_EXE, cmd]).split() 42 | 43 | with Popen(cmd, stdin=slaves[0], stdout=slaves[0], stderr=slaves[1]) as p: 44 | for fd in slaves: 45 | os.close(fd) # no input 46 | readable = { 47 | masters[0]: sys.stdout.buffer, # store buffers seperately 48 | masters[1]: sys.stderr.buffer, 49 | } 50 | while readable: 51 | for fd in select(readable, [], [])[0]: 52 | try: 53 | data = os.read(fd, 1024) # read available 54 | except OSError as e: 55 | if e.errno != errno.EIO: 56 | raise # XXX cleanup 57 | del readable[fd] # EIO means EOF on some systems 58 | else: 59 | if not data: # EOF 60 | del readable[fd] 61 | else: 62 | if fd == masters[0]: # We caught stdout 63 | utils.echo(data.rstrip()) 64 | utils.write_to_log(data) 65 | else: # We caught stderr 66 | utils.echo(data.rstrip(), err=True) 67 | utils.write_to_log(data, "stderr") 68 | readable[fd].flush() 69 | for fd in masters: 70 | os.close(fd) 71 | return p.returncode 72 | 73 | 74 | def _set_init_vars(cwd=None, tmpdir=None): 75 | """Set env vars that are used throughout Ruby and Python. If Ruby weren't used 76 | this would be managed very differently, as these are effectively project scoped 77 | global variables. 78 | 79 | These calls are split into separate functions because on occaision we need to call 80 | them more carefully than with this function, as in createproject. 81 | """ 82 | _set_env() 83 | _set_cwd(cwd) 84 | _set_tmpdir(tmpdir) 85 | 86 | 87 | def _set_env(): 88 | set_env_var("ENV", PROJECT_LOCATION) # installed location of this code 89 | 90 | 91 | # Defs used by main cli cmd 92 | def _set_cwd(cwd=None): 93 | """Set cwd environment variable and change the actual cwd to it. 94 | 95 | Args: 96 | cwd (path): Location of project (conf file, provisioning scripts, etc.) 97 | """ 98 | # effective CWD (likely real CWD, but may be changed by user. 99 | if cwd: # cli / api 100 | set_env_var("cwd", cwd) 101 | elif get_env_var("cwd"): 102 | pass # Already set - keep it. 103 | else: 104 | found_project = utils.find_conf(os.getcwd()) 105 | if found_project: 106 | set_env_var("cwd", found_project) 107 | else: 108 | set_env_var("cwd", os.getcwd()) 109 | 110 | os.chdir(get_env_var("cwd")) 111 | return get_env_var("cwd") 112 | 113 | 114 | def _set_tmpdir(tmpdir=None): 115 | """Set tmpdir and log_path locations. This should defaut to be inside the cwd. 116 | 117 | Args: 118 | tmpdir (path): Location of project's tmp dir. 119 | """ 120 | # loc of tmpdir 121 | if tmpdir: # cli / api 122 | set_env_var("TMPDIR", os.path.join(tmpdir, ".%s-tmp" % PROJECT_NAME)) 123 | elif get_env_var("TMPDIR"): # Previously set env var 124 | set_env_var( 125 | "TMPDIR", os.path.join(get_env_var("TMPDIR"), ".%s-tmp" % PROJECT_NAME) 126 | ) 127 | else: # Not set, set to default loc 128 | set_env_var( 129 | "TMPDIR", os.path.join(os.getcwd(), ".%s-tmp" % PROJECT_NAME) 130 | ) # default (cwd) 131 | 132 | set_env_var("LOG_PATH", os.path.join(get_env_var("TMPDIR"), "logs")) 133 | 134 | 135 | def _set_vagrant_vars(vagrant_cwd=None, vagrant_dotfile_path=None): 136 | """Set the environment varialbes prefixed with `VAGRANT_` that vagrant 137 | expects, and that we use, to modify some use paths. 138 | 139 | Agrs: 140 | vagrant_cwd (path): Location of `Vagrantfile`. Used if invoked with API only. 141 | vagrant_dotfile_path (path): Location of `.vagrant` metadata directory. Used if invoked with API only. 142 | """ 143 | # loc of Vagrantfile 144 | if vagrant_cwd: # cli / api 145 | os.environ["VAGRANT_CWD"] = vagrant_cwd 146 | elif "VAGRANT_CWD" not in os.environ: # Not set in env var 147 | # if custom Vagrantfile exists in the default location. 148 | if os.path.isfile(os.path.join(os.getcwd(), "Vagrantfile")): 149 | os.environ["VAGRANT_CWD"] = os.getcwd() 150 | else: # use default (installed) path 151 | os.environ["VAGRANT_CWD"] = PROJECT_LOCATION 152 | # loc of .vagrant dir 153 | if vagrant_dotfile_path: # cli / api 154 | os.environ["VAGRANT_DOTFILE_PATH"] = vagrant_dotfile_path 155 | elif "VAGRANT_DOTFILE_PATH" not in os.environ: # Not set in env var 156 | os.environ["VAGRANT_DOTFILE_PATH"] = os.path.normpath( 157 | os.path.join(os.getcwd(), ".vagrant") 158 | ) # default (cwd) 159 | 160 | 161 | # Defs for cli subcommands 162 | def createproject(project_name, cwd, tmpdir, config_only=None, ctx=None): 163 | """Create project with basic configuration files. 164 | 165 | Agrs: 166 | project_name (path): Place to create a new project. Must be non-existing dir. 167 | config_only (bool): Determins if we should only place a conf file in the new project. 168 | """ 169 | # initialize paths 170 | _set_env() 171 | cwd = _set_cwd(cwd) 172 | path = os.path.join(cwd, project_name) 173 | _set_tmpdir(path) 174 | 175 | # create new project dir 176 | try: 177 | os.makedirs(path) # Make parent dirs if needed. 178 | except FileExistsError: 179 | utils.abort("Directory already exists.") 180 | utils.echo( 181 | 'Created %s project "%s" in %s.' 182 | % (PROJECT_NAME.capitalize(), project_name, path) 183 | ) 184 | 185 | # Fill project dir with basic configs. 186 | install_config(ctx, output_path=path) 187 | install_gitignore(ctx, output_path=path) 188 | if not config_only: 189 | export("saltstack", path) 190 | install_auth(ctx, output_path=path) 191 | 192 | 193 | def destroy(ctx=None, **params): 194 | """Destroy a VM / container and all its metadata. Default leaves logs. 195 | All str args can also be set as an environment variable; arg takes precedence. 196 | 197 | Agrs: 198 | ctx (object): Click Context object. 199 | vagrant_cwd (path): Location of `Vagrantfile`. Used if invoked with API only. 200 | vagrant_dotfile_path (path): Location of `.vagrant` metadata directory. Used if invoked with API only. 201 | """ 202 | # TODO add finding and deleting of all VMs registered to this installation. 203 | # TODO (optional) add finding and deleting of all VMs across all installations. 204 | # TODO add an --all flag to delete the whole .rambo-tmp dir. Default leaves logs. 205 | 206 | if not ctx: # Using API. Else handled by cli. 207 | _set_init_vars(params.get("cwd"), params.get("tmpdir")) 208 | _set_vagrant_vars(params.get("vagrant_cwd"), params.get("vagrant_dotfile_path")) 209 | 210 | destroy_cmd = vagrant_general_command("destroy --force") 211 | 212 | # If there's any error code from Vagrant, don't delete the metadata. 213 | if not destroy_cmd: # I.e. we succeeded - ret code == 0 214 | utils.file_delete(os.path.join(get_env_var("TMPDIR"), "provider")) 215 | utils.file_delete(os.path.join(get_env_var("TMPDIR"), "random_tag")) 216 | utils.dir_delete(os.environ.get("VAGRANT_DOTFILE_PATH")) 217 | utils.echo("Temporary files removed") 218 | 219 | if params.get("vm_name"): # Additionally remove the box if we can. 220 | utils.echo(f"Now removing base VirtualBox data for VM {params['vm_name']}.") 221 | os.system(f"vboxmanage controlvm {params['vm_name']} poweroff") 222 | os.system(f"vboxmanage unregistervm {params['vm_name']} --delete") 223 | 224 | utils.echo("Destroy complete.") 225 | else: 226 | utils.echo("We received an error. Destroy may not be complete.") 227 | 228 | 229 | def export(resource=None, export_path=None, force=None): 230 | """Drop default code in the CWD / user defined space. Operate on saltstack 231 | and vagrant resources. 232 | 233 | Agrs: 234 | resource (str): Resource to export: saltstack or vagrant. 235 | export_path (path): Dir to export resources to. 236 | force (bool): Determins if we should overwrite and merge conflicting files in the target path. 237 | """ 238 | if export_path: 239 | output_dir = os.path.normpath(export_path) 240 | else: 241 | output_dir = os.getcwd() 242 | 243 | if resource in ("vagrant", "saltstack"): 244 | srcs = [os.path.normpath(os.path.join(PROJECT_LOCATION, resource))] 245 | dsts = [os.path.join(output_dir, resource)] 246 | 247 | if resource == "vagrant": 248 | srcs.append(os.path.normpath(os.path.join(PROJECT_LOCATION, "settings.json"))) 249 | srcs.append(os.path.normpath(os.path.join(PROJECT_LOCATION, "Vagrantfile"))) 250 | dsts.append(os.path.join(output_dir, "settings.json")) 251 | dsts.append(os.path.join(output_dir, "Vagrantfile")) 252 | 253 | if not force: 254 | try: 255 | for path in dsts: 256 | if os.path.exists(path): 257 | click.confirm( 258 | "One or more destination files or directories in " 259 | "'%s' already exists. Attempt to merge and " 260 | "overwrite?" % dsts, 261 | abort=True, 262 | ) 263 | break # We only need general confirmation of an overwrite once. 264 | except UnboundLocalError: # dsts referenced before assignement 265 | utils.abort("The resource '%s' is not a valid option." % resource) 266 | 267 | for src, dst in zip(srcs, dsts): 268 | try: 269 | copy_tree(src, dst) # Merge copy tree with overwrites. 270 | except DistutilsFileError: # It's a file, not a dir. 271 | try: 272 | shutil.copy(src, dst) # Copy file with overwrites. 273 | except FileNotFoundError: 274 | os.makedirs( 275 | os.path.dirname(dst), exist_ok=True 276 | ) # Make parent dirs if needed. # Py 3.2+ 277 | shutil.copy(src, dst) # Copy file with overwrites. 278 | 279 | utils.echo("Done exporting %s code." % resource) 280 | 281 | 282 | def halt(ctx=None, *args, **params): 283 | if not ctx: # Using API. Else handled by cli. 284 | _set_init_vars(params.get("cwd"), params.get("tmpdir")) 285 | _set_vagrant_vars(params.get("vagrant_cwd"), params.get("vagrant_dotfile_path")) 286 | else: 287 | args = ctx.args + list(args) 288 | 289 | vagrant_general_command("{} {}".format("halt", " ".join(args))) 290 | 291 | 292 | def install_auth(ctx=None, output_path=None, **kwargs): 293 | """Install auth directory. 294 | 295 | Agrs: 296 | ctx (object): Click Context object. 297 | output_path (path): Path to place auth dir. 298 | """ 299 | if not ctx: # Using API. Else handled by cli. 300 | _set_init_vars(kwargs.get("cwd"), kwargs.get("tmpdir")) 301 | 302 | if not output_path: 303 | output_path = get_env_var("cwd") 304 | license_dir = os.path.join(output_path, "auth/licenses") 305 | try: 306 | os.makedirs(license_dir) 307 | except FileExistsError: 308 | pass # Dir already created. Moving on. 309 | utils.echo( 310 | "Any (license) files you put in %s will be synced into your VM." % license_dir 311 | ) 312 | 313 | for filename in os.listdir(os.path.join(get_env_var("env"), "auth/env_scripts")): 314 | dst_dir = os.path.join(output_path, "auth/keys") 315 | dst = os.path.join(dst_dir, os.path.splitext(filename)[0]) 316 | if not os.path.isfile(dst): 317 | os.makedirs(dst_dir, exist_ok=True) # Make parent dirs if needed. # Py 3.2+ 318 | shutil.copy( 319 | os.path.join(get_env_var("env"), "auth/env_scripts", filename), dst 320 | ) 321 | utils.echo("Added template key loading scripts %s to auth/keys." % filename) 322 | else: 323 | utils.echo("File %s exists. Leaving it." % dst) 324 | 325 | # TODO: Have Rambo optionally store the same keys that may be in auth/keys in metadata, 326 | # added from the cli/api. Automatically check if keys in metatdata and not keys 327 | # in env vars, and set them. This is an avenue for expanding the cli/api's use 328 | # and not needing the auth key scripts. 329 | # load_provider_keys() 330 | 331 | 332 | def install_config(ctx=None, output_path=None, **kwargs): 333 | """Install config file. 334 | 335 | Agrs: 336 | ctx (object): Click Context object. 337 | output_path (path): Path to place conf file. 338 | """ 339 | if not ctx: # Using API. Else handled by cli. 340 | _set_init_vars(kwargs.get("cwd"), kwargs.get("tmpdir")) 341 | 342 | if not output_path: 343 | output_path = get_env_var("cwd") 344 | path = os.path.join(output_path, "%s.conf" % PROJECT_NAME) 345 | 346 | if os.path.exists(path): 347 | utils.abort("%s.conf already esists." % PROJECT_NAME) 348 | else: 349 | with open(path, "w") as f: 350 | f.write( 351 | """\ 352 | [up] 353 | provider = virtualbox 354 | box = ubuntu/bionic64 355 | sync_dirs = [["saltstack/etc", "/etc/salt"], ["saltstack/srv", "/srv"]] 356 | """ 357 | ) 358 | utils.echo("Created config at %s" % path) 359 | 360 | 361 | def install_gitignore(ctx=None, output_path=None, **kwargs): 362 | """Install config file. 363 | 364 | Agrs: 365 | ctx (object): Click Context object. 366 | output_path (path): Path to place conf file. 367 | """ 368 | if not ctx: # Using API. Else handled by cli. 369 | _set_init_vars(kwargs.get("cwd"), kwargs.get("tmpdir")) 370 | 371 | if not output_path: 372 | output_path = get_env_var("cwd") 373 | path = os.path.join(output_path, ".gitignore") 374 | 375 | if os.path.exists(path): 376 | pass 377 | else: 378 | with open(path, "w") as f: 379 | f.write( 380 | """\ 381 | .rambo-tmp/ 382 | .vagrant/ 383 | my_rambo.conf 384 | auth/ 385 | """ 386 | ) 387 | utils.echo("Created .gitignore") 388 | 389 | 390 | def install_plugins(force=None, plugins=("all",)): 391 | """Install all of the vagrant plugins needed for all plugins 392 | 393 | Agrs: 394 | force (bool): Forces bypassing of reinstallation prompt. 395 | plugins (tuple): Names of vagrant plugins to install. 396 | """ 397 | host_system = platform.system() 398 | for plugin in plugins: 399 | if plugin == "all": 400 | utils.echo("Installing all default plugins.") 401 | for plugin in SETTINGS["PLUGINS"][host_system]: 402 | _invoke_vagrant("plugin install %s" % plugin) 403 | elif plugin in SETTINGS["PLUGINS"][host_system]: 404 | _invoke_vagrant("plugin install %s" % plugin) 405 | else: 406 | if not force: 407 | click.confirm( 408 | 'The plugin "%s" is not in our list of plugins. Attempt ' 409 | "to install anyway?" % plugin, 410 | abort=True, 411 | ) 412 | vagrant_general_command("plugin install %s" % plugin) 413 | 414 | 415 | def scp(ctx=None, locations=None, **params): 416 | """Transfer file or dir with scp. This makes use of the vagrant-scp plugin, 417 | which allows for simplified args. 418 | """ 419 | if not ctx: # Using API. Else handled by cli. 420 | _set_init_vars(params.get("cwd"), params.get("tmpdir")) 421 | _set_vagrant_vars(params.get("vagrant_cwd"), params.get("vagrant_dotfile_path")) 422 | 423 | if len(locations) != 2: 424 | utils.abort( 425 | "There needs to be exactly two arguments for scp, a 'from' location " 426 | "and a 'to' location.\nYou gave: %s." % " ".join(locations) 427 | ) 428 | 429 | copy_from = locations[0] 430 | copy_to = locations[1] 431 | 432 | if ":" in copy_from: # copy_from is remote, fix copy_to which is local 433 | copy_to = os.path.abspath(copy_to) 434 | else: # if no ':' in copy_from, copy_to must be remote, fix copy_from which is local 435 | copy_from = os.path.abspath(copy_from) 436 | 437 | locations = [copy_from, copy_to] 438 | 439 | vagrant_general_command("{} {}".format("scp", " ".join(locations))) 440 | 441 | 442 | def ssh(ctx=None, command=None, ssh_args=None, **params): 443 | """Connect to an running VM / container over ssh. 444 | All str args can also be set as an environment variable; arg takes precedence. 445 | 446 | Agrs: 447 | ctx (object): Click Context object. 448 | command (str): Pass-through command to run with `vagrant ssh --command`. 449 | vagrant_cwd (path): Location of `Vagrantfile`. Used if invoked with API only. 450 | vagrant_dotfile_path (path): Location of `.vagrant` metadata directory. Used if invoked with API only. 451 | """ 452 | if not ctx: # Using API. Else handled by cli. 453 | _set_init_vars(params.get("cwd"), params.get("tmpdir")) 454 | _set_vagrant_vars(params.get("vagrant_cwd"), params.get("vagrant_dotfile_path")) 455 | 456 | # Add pass-through 'command' option. 457 | cmd = f"{VAGRANT_EXE} ssh" 458 | if command: 459 | cmd = " ".join([cmd, "--command", command]) 460 | 461 | if ssh_args: 462 | if isinstance(ssh_args, tuple): 463 | ssh_args = " ".join(ssh_args) 464 | else: 465 | ssh_args = ast.literal_eval(ssh_args) 466 | 467 | cmd = f"{cmd} -- {ssh_args}" 468 | 469 | # do not use _invoke_vagrant, that will give a persistent ssh session regardless. 470 | os.system(cmd) 471 | 472 | 473 | def up(ctx=None, up_args=None, **params): 474 | """Start a VM / container with `vagrant up`. 475 | All str args can also be set as an environment variable; arg takes precedence. 476 | 477 | Agrs: 478 | ctx (object): Click Context object. Used to detect if CLI is used. 479 | params (dict): Dict of all args passed to `up`. 480 | 481 | In params, this looks for: 482 | provider (str): Provider to use. 483 | box (str): Vagrant box to use. 484 | gui (bool): vagrant gui flag. Defaults to False. 485 | res (str): Screen resolution like `1920x1080`. 486 | cpus (int): Number of CPUs to give VirtualBox VM. 487 | guest_os (str): Guest OS to use. 488 | ram_size (int): RAM in MB to use. 489 | drive_size (int): Drive size in GB to use. 490 | machine_type (str): Machine type to use for cloud providers. 491 | sync_dirs (path): Paths to sync into VM. 492 | sync_type (str): Type of syncing to use. 493 | ports (str): Ports to forward. 494 | ip_address (str): IP address to assign. 495 | provision (bool): vagrant provisioning flag. 496 | command (str): Command used at beginning of provisioning. 497 | destroy_on_error (bool): vagrant destroy-on-error flag. 498 | vagrant_cwd (path): Location of `Vagrantfile`. Used if invoked with API only. 499 | vagrant_dotfile_path (path): Location of `.vagrant` metadata directory. Used if invoked with API only. 500 | vm_name (str): Name of the VM or container. 501 | """ 502 | # TODO: Add registering of VM for all of this installation to see 503 | if not ctx: # Using API. Else handled by cli. 504 | _set_init_vars(params.get("cwd"), params.get("tmpdir")) 505 | _set_vagrant_vars(params.get("vagrant_cwd"), params.get("vagrant_dotfile_path")) 506 | 507 | # Option Handling - These might modify the params dict and/or set env vars. 508 | params["guest_os"] = options.guest_os_option(params.get("guest_os")) 509 | params["box"] = options.box_option(params.get("box")) 510 | params["gui"] = options.gui_option(params.get("gui")) 511 | params["cpus"] = options.cpus_option(params.get("cpus")) 512 | params["hostname"] = options.hostname_option(params.get("hostname")) 513 | params["machine_type"] = options.machine_type_option( 514 | params.get("machine_type"), params.get("provider") 515 | ) 516 | params["project_dir"] = options.project_dir_option(params.get("project_dir")) 517 | params["provider"] = options.provider_option(params.get("provider")) 518 | params["command"] = options.command_option(params.get("command")) 519 | params["ram_size"], params["drive_size"] = options.size_option( 520 | params.get("ram_size"), params.get("drive_size") 521 | ) # both ram and drive size 522 | params["res"] = options.res_option(params.get("res")) 523 | params["sync_dirs"] = options.sync_dirs_option(params.get("sync_dirs")) 524 | params["sync_type"] = options.sync_type_option(params.get("sync_type")) 525 | params["ports"] = options.ports_option(params.get("ports")) 526 | params["ip_address"] = options.ip_address_option(params.get("ip_address")) 527 | params["vm_name"] = options.vm_name_option(params.get("vm_name")) 528 | 529 | cmd = "up" 530 | 531 | if up_args: 532 | if isinstance(up_args, tuple): 533 | up_args = " ".join(up_args) 534 | else: 535 | up_args = ast.literal_eval(up_args) 536 | cmd = f"{cmd} {up_args}" 537 | 538 | # Provider specific handling. 539 | # Must come after all else, because logic may be done on params above. 540 | if params["provider"] == "digitalocean": 541 | vagrant_providers.digitalocean() 542 | elif params["provider"] == "docker": 543 | vagrant_providers.docker() 544 | elif params["provider"] == "ec2": 545 | vagrant_providers.ec2(**params) 546 | else: 547 | cmd += " --provider={}".format(params["provider"]) 548 | 549 | # Add straight pass-through flags. Keep test for True/False explicit as only those values should work 550 | if params.get("provision") is True: 551 | cmd = "{} {}".format(cmd, "--provision") 552 | elif params.get("provision") is False: 553 | cmd = "{} {}".format(cmd, "--no-provision") 554 | 555 | if params.get("destroy_on_error") is True: 556 | cmd = "{} {}".format(cmd, "--destroy-on-error") 557 | elif params.get("destroy_on_error") is False: 558 | cmd = "{} {}".format(cmd, "--no-destroy-on-error") 559 | 560 | exit_code = vagrant_general_command(cmd) 561 | 562 | if exit_code: 563 | with open(Path(get_env_var("LOG_PATH")) / "stderr.log") as fp: 564 | stderr = fp.readlines() 565 | for idx, line in enumerate(reversed(stderr)): 566 | if "Unknown configuration section 'disksize'" in line: 567 | abort( 568 | "You probably don't have plugins installed.\nRun:\n" 569 | "\trambo install-plugins" 570 | ) 571 | elif idx > 5: 572 | # Only look through the recent stderr. 573 | break 574 | 575 | 576 | def vagrant_general_command(cmd): 577 | """Invoke vagrant with custom command. 578 | 579 | Args: 580 | cmd (str): String to append to command `vagrant ...` 581 | """ 582 | # Modify cmd in private function to keep enforcement of being a vagrant cmd there. 583 | return _invoke_vagrant(cmd) 584 | -------------------------------------------------------------------------------- /rambo/auth/env_scripts/aws.env.sh.dist: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # for aws 4 | # associated with your aws user 5 | export AWS_ACCESS_KEY_ID= 6 | export AWS_SECRET_ACCESS_KEY= 7 | # associated with ec2 specifically (not your user's general ssh key) 8 | export AWS_KEYPAIR_NAME="name" 9 | export AWS_SSH_PRIVKEY="auth/keys/name.pem" 10 | -------------------------------------------------------------------------------- /rambo/auth/env_scripts/digitalocean.env.sh.dist: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # for digitalocean 4 | export DIGITALOCEAN_TOKEN= 5 | export DIGITALOCEAN_PRIVATE_KEY_PATH="auth/keys/digitalocean.pem" 6 | -------------------------------------------------------------------------------- /rambo/cli.py: -------------------------------------------------------------------------------- 1 | import configparser 2 | import os 3 | import sys 4 | 5 | import click 6 | import pkg_resources 7 | 8 | import rambo.app as app 9 | import rambo.utils as utils 10 | from rambo.settings import CONF_FILES 11 | from rambo.settings import PROJECT_NAME 12 | from rambo.settings import SETTINGS 13 | 14 | 15 | version = pkg_resources.get_distribution("rambo-vagrant").version 16 | 17 | 18 | # Only used for the `vagrant` subcommand 19 | VAGRANT_CMD_CONTEXT_SETTINGS = { 20 | "ignore_unknown_options": True, 21 | "allow_extra_args": True, 22 | } 23 | 24 | # Used for all commands and subcommands 25 | CONTEXT_SETTINGS = { 26 | "help_option_names": ["-h", "--help"], 27 | "auto_envvar_prefix": PROJECT_NAME.upper(), 28 | } 29 | 30 | 31 | # Support for injecting config vars into click context 32 | # https://stackoverflow.com/questions/46358797/python-click-supply-arguments-and-options-from-a-configuration-file 33 | def load_config_for_command(ctx, section): 34 | parser = configparser.ConfigParser() 35 | parser.read(CONF_FILES) 36 | 37 | if section == "cli": 38 | section = "base" 39 | if section in parser: 40 | section = parser[section] 41 | for param, value in ctx.params.items(): 42 | if not value: 43 | if param in section: 44 | if param == "gui": 45 | ctx.params[param] = section.getboolean(param) 46 | else: 47 | ctx.params[param] = section[param] 48 | 49 | # native_val = parser[section][param] 50 | # try: 51 | # native_val = ast.literal_eval(native_val) 52 | # except ValueError: 53 | # pass 54 | # # ctx.params[param] = native_val 55 | # # else: 56 | # # breakpoint() 57 | # # ctx.params[param] = type(ctx.params[param])(native_val) 58 | 59 | # ctx_param = ctx.params[param] 60 | # if ctx_param is not None: 61 | # ctx.params[param] = type(ctx.params[param])(native_val) 62 | # else: 63 | # ctx.params[param] = native_val 64 | return ctx 65 | 66 | 67 | class GroupWithConfig(click.Group): 68 | def invoke(self, ctx): 69 | ctx = load_config_for_command(ctx, self.name) 70 | super().invoke(ctx) 71 | 72 | 73 | class CommandWithConfig(click.Command): 74 | def invoke(self, ctx): 75 | ctx = load_config_for_command(ctx, self.name) 76 | return super().invoke(ctx) 77 | 78 | 79 | # Main command / CLI entry point 80 | @click.group(context_settings=CONTEXT_SETTINGS, cls=GroupWithConfig) 81 | @click.option( 82 | "--vagrant-cwd", 83 | type=click.Path(resolve_path=True), 84 | help="Path entry point to the Vagrantfile. Defaults to " 85 | "the Vagrantfile provided by %s in the installed path." % PROJECT_NAME.capitalize(), 86 | ) 87 | @click.option( 88 | "--vagrant-dotfile-path", 89 | type=click.Path(resolve_path=True), 90 | help="Path location of the .vagrant directory for the " 91 | "virtual machine. Defaults to the current working directory.", 92 | ) 93 | @click.option( 94 | "--cwd", 95 | type=click.Path(resolve_path=True), 96 | help="The CWD of for this command. Defaults to " 97 | "actual CWD, but may be set for customization. Used to look " 98 | "for optional resources such as custom SaltStack code.", 99 | ) 100 | @click.option( 101 | "--tmpdir", 102 | type=click.Path(resolve_path=True), 103 | help="Path location of the .rambo-tmp directory for the virtual " 104 | "machine. Defaults to the current working directory", 105 | ) 106 | @click.version_option(prog_name=PROJECT_NAME.capitalize(), version=version) 107 | @click.pass_context 108 | def cli(ctx, cwd, tmpdir, vagrant_cwd, vagrant_dotfile_path): 109 | """The main cli entry point. Params can be passed as usual with 110 | click (CLI or env var) and also with an INI config file. 111 | Precedence is CLI > Config > Env Var > defaults. 112 | """ 113 | if ctx.invoked_subcommand not in ["createproject"]: 114 | # These need to be very early because they may change the cwd of this Python or of Vagrant 115 | app._set_init_vars(cwd, tmpdir) 116 | app._set_vagrant_vars(vagrant_cwd, vagrant_dotfile_path) 117 | 118 | utils.write_to_log("\nNEW CMD") 119 | utils.write_to_log(" ".join(sys.argv)) 120 | 121 | utils.write_to_log("\nNEW CMD", "stderr") 122 | utils.write_to_log(" ".join(sys.argv), "stderr") 123 | 124 | 125 | # Subcommands 126 | @cli.command("createproject") 127 | @click.argument("project_name") 128 | @click.option( 129 | "-c", 130 | "--config-only", 131 | is_flag=True, 132 | help="Only create project dir with config file.", 133 | ) 134 | @click.pass_context 135 | def createproject_cmd(ctx, project_name, config_only): 136 | """Create project takes an arguement for the name to give to the project 137 | it creates. It will create a directory in the CWD for this project. Upon 138 | creation, this project directory will contain a rambo.conf file, an auth 139 | directory, and a saltstack directory. 140 | 141 | - rambo.conf is the config file that is required to be present in your 142 | project to run rambo up, and is described later in this document. 143 | - auth contains some sample scripts that will aid in setting up keys / tokens 144 | for the cloud providers. It is not required. How to use that is described 145 | in the cloud provider specific documentation. 146 | - saltstack is a basic set of SaltStack configuration code that Rambo offers. 147 | It can be modified for custom configuration. 148 | """ 149 | app.createproject( 150 | project_name, 151 | ctx.parent.params["cwd"], 152 | ctx.parent.params["tmpdir"], 153 | config_only, 154 | ctx, 155 | ) 156 | 157 | 158 | @cli.command( 159 | "destroy", 160 | context_settings=CONTEXT_SETTINGS, 161 | short_help="Destroy VM and metadata.", 162 | ) 163 | @click.option( 164 | "-v", "--vm_name", type=str, help="The name of the VirtualMachine / Container." 165 | ) 166 | @click.pass_context 167 | def destroy_cmd(ctx, vm_name, **params): 168 | """Destroy a VM / container. This will tell vagrant to forcibly destroy 169 | a VM, and to also destroy its Rambo metadata (provider and random_tag), 170 | and Vagrant metadata (.vagrant dir). 171 | """ 172 | app.destroy(ctx, **ctx.params) 173 | 174 | 175 | @cli.command("export-vagrant-conf", short_help="Get Vagrant configuration") 176 | @click.option( 177 | "-f", "--force", is_flag=True, help="Accept attempts to overwrite and merge." 178 | ) 179 | @click.option( 180 | "-O", 181 | "--output-path", 182 | type=click.Path(resolve_path=True), 183 | help="The optional output path.", 184 | ) 185 | def export_vagrant_conf(output_path, force): 186 | """Places the default Vagrantfile and its resources (vagrant dir, 187 | settings.json) in the CWD for customizing. 188 | """ 189 | app.export("vagrant", output_path, force) 190 | 191 | 192 | @cli.command( 193 | "halt", context_settings=VAGRANT_CMD_CONTEXT_SETTINGS, short_help="Halt VM." 194 | ) 195 | @click.pass_context 196 | def halt_cmd(ctx): 197 | """Tells Vagrant to 'halt' the VM. Useful to free the Host's 198 | resources without destroying the VM. 199 | """ 200 | app.halt(ctx) 201 | 202 | 203 | @cli.command( 204 | "install-plugins", 205 | context_settings=CONTEXT_SETTINGS, 206 | short_help="Install Vagrant plugins", 207 | ) 208 | @click.option( 209 | "-f", "--force", is_flag=True, help="Install plugins without confirmation." 210 | ) 211 | @click.argument("plugins", nargs=-1, type=str) 212 | def install_plugins(force, plugins): 213 | """Install passed args as Vagrant plugins. `all` or no args installs 214 | all default Vagrant plugins from host platform specific list. 215 | """ 216 | # If auth and plugins are both not specified, run both. 217 | if not plugins: # No args means all default plugins. 218 | plugins = ("all",) 219 | app.install_plugins(force, plugins) 220 | 221 | 222 | @cli.command( 223 | "scp", 224 | context_settings=VAGRANT_CMD_CONTEXT_SETTINGS, 225 | short_help="Transfer files with scp.", 226 | ) 227 | @click.pass_context 228 | def scp_cmd(ctx): 229 | """Transfer files or directories with scp. Accepts two args in one of the 230 | following forms: 231 | 232 | 233 | 234 | : 235 | 236 | : 237 | 238 | [vm_name]: 239 | 240 | [vm_name]: 241 | 242 | For example: `rambo scp localfile.txt remotefile.txt` 243 | """ 244 | app.scp(ctx, ctx.args) 245 | 246 | 247 | @cli.command("ssh", short_help="Connect with ssh.", cls=CommandWithConfig) 248 | @click.option("-c", "--command", type=str, help="Execute an SSH command directly.") 249 | @click.argument("ssh_args", nargs=-1, type=str) 250 | @click.pass_context 251 | def ssh_cmd(ctx, command, ssh_args): 252 | """Connect to an running VM / container over ssh. With `-c` / `--command`, 253 | execute an SSH command directly. 254 | 255 | Supply a final `-- [args]` to pass additional arguments directly to ssh. 256 | """ 257 | if not ssh_args: 258 | ssh_args = ctx.params.get("ssh_args") 259 | 260 | app.ssh(ctx, command, ssh_args) 261 | 262 | 263 | @cli.command( 264 | "up", 265 | context_settings=CONTEXT_SETTINGS, 266 | short_help="Create or start VM.", 267 | cls=CommandWithConfig, 268 | ) 269 | @click.option( 270 | "-p", 271 | "--provider", 272 | type=str, 273 | help="Provider for the virtual machine. " 274 | "These providers are supported: %s. Default %s." 275 | % (SETTINGS["PROVIDERS"], SETTINGS["PROVIDERS_DEFAULT"]), 276 | ) 277 | @click.option("--gui", is_flag=True) 278 | @click.option( 279 | "--res", 280 | type=str, 281 | help="Screen resolution if using a gui, of the format '1920x1080'. " 282 | "Default determined by magic. Ignored if no gui." 283 | ) 284 | @click.option( 285 | "-o", 286 | "--guest-os", 287 | type=str, 288 | help="Operating System of the guest, inside the virtual machine. " 289 | "These guest OSs are supported: %s. Default %s." 290 | % (list(SETTINGS["GUEST_OSES"].keys()), SETTINGS["GUEST_OSES_DEFAULT"]), 291 | ) 292 | @click.option("-b", "--box", type=str, help="Vagrant Box to use.") 293 | @click.option("--hostname", type=str, help="Hostname to set.") 294 | @click.option( 295 | "-r", 296 | "--ram-size", 297 | type=int, 298 | help="Amount of RAM of the virtual machine in MB. " 299 | "These RAM sizes are supported: %s. Default %s." 300 | % (list(SETTINGS["SIZES"].keys()), SETTINGS["RAMSIZE_DEFAULT"]), 301 | ) 302 | @click.option("--cpus", type=int, help="Number of CPUs in a virtualbox VM.") 303 | @click.option( 304 | "-d", 305 | "--drive-size", 306 | type=int, 307 | help="The drive size of the virtual machine in GB. " 308 | "These drive sizes are supported: %s. Default %s." 309 | % (list(SETTINGS["SIZES"].values()), SETTINGS["DRIVESIZE_DEFAULT"]), 310 | ) 311 | @click.option( 312 | "-m", 313 | "--machine-type", 314 | type=str, 315 | help="Machine type for cloud providers.\n" 316 | "E.g. m5.medium for ec2, or s-8vcpu-32gb for digitalocean.\n", 317 | ) 318 | @click.option( 319 | "--sync-dirs", 320 | type=str, 321 | help=( 322 | "Paths to sync into VM, passed as a Python list of lists of the form " 323 | """"[['guest_dir', 'host_dir'], ['guest_dir2', 'host_dir2']]".""" 324 | ), 325 | ) 326 | @click.option( 327 | "--ec2-security-groups", 328 | type=str, 329 | help=( 330 | "A list of security groups to add to the EC2 VM, passed as a Python list of the form " 331 | """"['salted_server', 'security_group_2]".""" 332 | ), 333 | ) 334 | @click.option("--sync-type", type=str, help="Sync type") 335 | @click.option( 336 | "--ports", 337 | type=str, 338 | help=( 339 | "Additional ports to sync into VM, passed as a Python list of lists of the form " 340 | """[['guest_port', 'host_port'], ['guest_port2', 'host_port2']]".""" 341 | ), 342 | ) 343 | @click.option( 344 | "--ip_address", 345 | type=str, 346 | help=("IP address to assign the VM"), 347 | ) 348 | @click.option( 349 | "--project-dir", 350 | type=click.Path(resolve_path=True), 351 | help="List of path to sync into VM", 352 | ) 353 | @click.option( 354 | "--provision/--no-provision", default=None, help="Enable or disable provisioning" 355 | ) 356 | @click.option("-c", "--command", type=str, help="Command to start provisioning with") 357 | @click.option( 358 | "--destroy-on-error/--no-destroy-on-error", 359 | default=None, 360 | help="Destroy machine if any fatal error happens (default to true)", 361 | ) 362 | @click.option("--vm_name", type=str, help=( 363 | "The name of the VirtualMachine / Container. " 364 | "This is also the hostname if that wasn't explicitly set." 365 | ) 366 | ) 367 | @click.argument("up_args", nargs=-1, type=str) 368 | @click.pass_context 369 | def up_cmd( 370 | ctx, 371 | provider, 372 | box, 373 | gui, 374 | res, 375 | hostname, 376 | guest_os, 377 | ram_size, 378 | cpus, 379 | drive_size, 380 | machine_type, 381 | sync_dirs, 382 | ec2_security_groups, 383 | sync_type, 384 | ports, 385 | ip_address, 386 | project_dir, 387 | provision, 388 | command, 389 | destroy_on_error, 390 | vm_name, 391 | up_args, 392 | ): 393 | """Start a VM or container. Will create one and begin provisioning it if 394 | it did not already exist. Accepts many options to set aspects of your VM. 395 | Precedence is CLI > Config > Env Var > defaults. 396 | """ 397 | if not os.path.isfile(f"{PROJECT_NAME}.conf"): 398 | utils.abort( 399 | f"Config file {PROJECT_NAME}.conf must be present in working directory.\n" 400 | "A config file is automatically created when you run \n" 401 | "createproject. You can also make a config file manually." 402 | ) 403 | 404 | app.up(ctx, **ctx.params) 405 | 406 | 407 | @cli.command( 408 | "vagrant", 409 | context_settings=VAGRANT_CMD_CONTEXT_SETTINGS, 410 | short_help="Run a vagrant command through rambo.", 411 | ) 412 | @click.pass_context 413 | def vagrant_cmd(ctx): 414 | """Accepts any args and forwards them to Vagrant directly, allowing you to 415 | run any vagrant command. Rambo has first-class duplicates or wrappers for 416 | the most common Vagrant commands, but for less common commands or commands 417 | that are not customized, they don't need to be duplicated, so we call them 418 | directly. 419 | """ 420 | app.vagrant_general_command(" ".join(ctx.args)) 421 | 422 | 423 | main = cli 424 | -------------------------------------------------------------------------------- /rambo/options.py: -------------------------------------------------------------------------------- 1 | import ast 2 | import os 3 | 4 | import rambo.utils as utils 5 | from rambo.settings import SETTINGS 6 | from rambo.utils import set_env_var 7 | 8 | 9 | def box_option(box=None): 10 | """Set box 11 | 12 | Args: 13 | box (str): Vagrant box to use. 14 | 15 | Return box (str) 16 | """ 17 | if box: 18 | set_env_var("box", box) 19 | return box 20 | 21 | 22 | def gui_option(gui=None): 23 | """Set GUI flag 24 | 25 | Args: 26 | gui (bool): GUI flag, defaults to False. 27 | 28 | Return gui (bool) 29 | """ 30 | if not gui: 31 | set_env_var("gui", False) 32 | else: 33 | set_env_var("gui", True) 34 | return gui 35 | 36 | 37 | def cpus_option(cpus=None): 38 | """Set cpus 39 | 40 | Args: 41 | cpus (int): CPUs for VirtualBox VM 42 | 43 | Return cpus (int) 44 | """ 45 | if cpus and 1 <= int(cpus) <= 32: 46 | set_env_var("cpus", cpus) 47 | return cpus 48 | elif cpus: 49 | utils.warn("CPUs must be an int in [1, 32]. Falling back to 1.") 50 | 51 | set_env_var("cpus", 1) 52 | return 1 53 | 54 | 55 | def guest_os_option(guest_os=None): 56 | """Validate guest_os. If not supplied, set to default. Set as env var. 57 | 58 | Args: 59 | guest_os (str): Guest OS to use. 60 | 61 | Return guest_os (str) 62 | """ 63 | if not guest_os: 64 | guest_os = SETTINGS["GUEST_OSES_DEFAULT"] 65 | set_env_var("guest_os", guest_os) 66 | 67 | if guest_os not in SETTINGS["GUEST_OSES"]: 68 | msg = ( 69 | 'Guest OS "%s" is not in the guest OSes whitelist.\n' 70 | "Did you have a typo? We'll try anyway.\n" 71 | "Here is as list of avalible guest OSes:\n\n" % guest_os 72 | ) 73 | for supported_os in SETTINGS["GUEST_OSES"]: 74 | msg = msg + "%s\n" % supported_os 75 | utils.warn(msg) 76 | return guest_os 77 | 78 | 79 | def hostname_option(hostname=None): 80 | """Validate hostname 81 | 82 | Args: 83 | hostname (str): Hostname to set in VM / container 84 | 85 | Return hostname (str) 86 | 87 | """ 88 | if hostname and len(hostname) > 64: 89 | utils.warn( 90 | "Hostnames of many OSes are limited to 64 characters." 91 | f"The current hostname {hostname} is {len(hostname)}." 92 | ) 93 | 94 | if hostname: 95 | set_env_var("hostname", hostname) 96 | 97 | return hostname 98 | 99 | 100 | def machine_type_option(machine_type=None, provider=None): 101 | """Validate machine_type. If not supplied, set to default. Set as env var. 102 | 103 | Args: 104 | machine_type (str): Machine type to use for cloud providers. 105 | provider (str): Provider to use. 106 | 107 | Return machine_type (str) 108 | """ 109 | if machine_type: 110 | if provider in ("docker", "lxc", "virtualbox"): 111 | msg = ( 112 | "You have selected a machine-type, but are not using\n" 113 | "a cloud provider. You selected %s with %s.\n" 114 | % (machine_type, provider) 115 | ) 116 | utils.abort(msg) 117 | set_env_var("machinetype", machine_type) 118 | return machine_type 119 | 120 | 121 | def ports_option(ports=None): 122 | """Validate ports. If not supplied, set to default. Set as env var. 123 | 124 | ports must be list of lists of form: 125 | `[['guest_port', 'host_port'], ['guest_port2', 'host_port2']]` 126 | 127 | Args: 128 | ports: Paths to sync into VM, supplied as list of lists. 129 | 130 | Return ports (list) 131 | """ 132 | if not ports: 133 | return None 134 | 135 | try: 136 | ports = ast.literal_eval(ports) 137 | except SyntaxError: 138 | utils.abort("ports cannot be evaluated as valid Python.") 139 | if not isinstance(ports, list): 140 | utils.abort( 141 | f"`ports` was not evaluated as a Python list, but as '{type(ports)}'." 142 | ) 143 | for port_pair in ports: 144 | if not isinstance(port_pair, list): 145 | utils.abort( 146 | f"`ports` element {port_pair} was not evaluated as a Python list, but as " 147 | f"'{type(port_pair)}'." 148 | ) 149 | if len(port_pair) != 2: 150 | utils.abort(f"Not the right number of ports to forward in {port_pair}.") 151 | for port in port_pair: 152 | if not isinstance(port_pair, int) and not 0 < port < 65535: 153 | utils.abort(f"{port} in `ports` is not an int in a valid port range.") 154 | 155 | set_env_var("ports", ports) 156 | return ports 157 | 158 | 159 | def ip_address_option(ip_address="192.168.56.10"): 160 | """Set IP address.""" 161 | set_env_var("ip", ip_address) 162 | return ip_address 163 | 164 | 165 | def project_dir_option(project_dir=None): 166 | """Validate project_dir. If not supplied, set to default. Set as env var. 167 | 168 | Args: 169 | project_dir: Path to sync into VM. 170 | 171 | Return project_dir (path) 172 | """ 173 | if not project_dir: 174 | project_dir = os.getcwd() 175 | 176 | set_env_var("project_dir", project_dir) 177 | 178 | return project_dir 179 | 180 | 181 | def provider_option(provider=None): 182 | """Validate provider. If not supplied, set to default. Set as env var. 183 | 184 | Args: 185 | provider (str): Provider to use. 186 | 187 | Return provider (str) 188 | """ 189 | if not provider: 190 | provider = SETTINGS["PROVIDERS_DEFAULT"] 191 | set_env_var("provider", provider) 192 | 193 | if provider not in SETTINGS["PROVIDERS"]: 194 | msg = ( 195 | 'Provider "%s" is not in the provider list.\n' 196 | "Did you have a typo? Here is as list of avalible providers:\n\n" % provider 197 | ) 198 | for supported_provider in SETTINGS["PROVIDERS"]: 199 | msg = msg + "%s\n" % supported_provider 200 | utils.abort(msg) 201 | return provider 202 | 203 | 204 | def command_option(command=None): 205 | """Load command into env var. 206 | 207 | Args: 208 | command (str): Command to run at the begginning of provisioning. 209 | 210 | Return command (str) 211 | """ 212 | if command: 213 | set_env_var("command", command) 214 | 215 | return command 216 | 217 | 218 | def res_option(res=None): 219 | """Validate screen resolution. If not supplied, don't set it. 220 | 221 | res must be a string that is two ints joined by an `x`, like: 222 | `1920x1080` 223 | 224 | Args: 225 | res: screen resolution like `1920x1080` 226 | 227 | Return resolution (str) 228 | """ 229 | if not res: 230 | return 231 | 232 | try: 233 | dimensions = res.split('x') 234 | dimensions = (int(dimensions[0]), int(dimensions[1]),) 235 | except (IndexError, TypeError): 236 | utils.abort( 237 | f"`res` is not two ints joined by an `x`, but is {res}." 238 | ) 239 | 240 | res = f"{dimensions[0]},{dimensions[1]}" 241 | set_env_var("resolution", res) 242 | 243 | return res 244 | 245 | 246 | def size_option(ram_size=None, drive_size=None): 247 | """Validate ram and drive sizes. Pair them if possible. If not 248 | supplied, set to default. Set as env var. Reset in params as strings. 249 | 250 | Args: 251 | ram_size (int): RAM in MB to use. 252 | drive_size (int): Drive size in GB to use. 253 | 254 | Return (ram_size, drive_size) (tuple, where values are (str, str)) 255 | """ 256 | # Cast to strings if they exist so they can stored as env vars. 257 | if ram_size: 258 | ram_size = str(ram_size) 259 | if drive_size: 260 | drive_size = str(drive_size) 261 | 262 | if ram_size and not drive_size: 263 | try: 264 | drive_size = SETTINGS["SIZES"][ram_size] 265 | except KeyError: # Doesn't match, but we'll let them try it. 266 | drive_size = SETTINGS["DRIVESIZE_DEFAULT"] 267 | elif drive_size and not ram_size: 268 | try: 269 | ram_size = SETTINGS["SIZES"][ 270 | list(SETTINGS["SIZES"].values()).index(drive_size) 271 | ] 272 | except ValueError: # Doesn't match, but we'll let them try it. 273 | ram_size = SETTINGS["RAMSIZE_DEFAULT"] 274 | elif not ram_size and not drive_size: 275 | ram_size = SETTINGS["RAMSIZE_DEFAULT"] 276 | drive_size = SETTINGS["DRIVESIZE_DEFAULT"] 277 | # else both exist, just try using them 278 | 279 | set_env_var("ramsize", ram_size) 280 | set_env_var("drivesize", drive_size) 281 | 282 | # ram_size 283 | if ram_size not in SETTINGS["SIZES"]: 284 | msg = ( 285 | 'RAM Size "%s" is not in the RAM sizes list.\n' 286 | "Did you have a typo? We'll try anyway.\n" 287 | "Here is as list of avalible RAM sizes:\n\n" % ram_size 288 | ) 289 | for supported_ram_size in SETTINGS["SIZES"]: 290 | msg = msg + "%s\n" % supported_ram_size 291 | utils.warn(msg) 292 | 293 | # drive_size 294 | if drive_size not in SETTINGS["SIZES"].values(): 295 | msg = ( 296 | 'DRIVE Size "%s" is not in the DRIVE sizes list.\n' 297 | "Did you have a typo? We'll try anyway.\n" 298 | "Here is as list of avalible DRIVE sizes:\n\n" % drive_size 299 | ) 300 | for supported_drive_size in SETTINGS["SIZES"].values(): 301 | msg = msg + "%s\n" % supported_drive_size 302 | utils.warn(msg) 303 | return (ram_size, drive_size) 304 | 305 | 306 | def sync_dirs_option(sync_dirs=None): 307 | """Validate sync_dirs. If not supplied, set to default. Set as env var. 308 | 309 | sync_dirs must be list of lists of form: 310 | `"[['guest_dir', 'host_dir'], ['guest_dir2', 'host_dir2']]"` 311 | 312 | Args: 313 | sync_dirs: Paths to sync into VM, supplied as list of lists. 314 | 315 | Return sync_dirs (list) 316 | """ 317 | if not sync_dirs: 318 | return None 319 | 320 | try: 321 | sync_dirs = ast.literal_eval(sync_dirs) 322 | except SyntaxError: 323 | utils.abort("sync_dirs cannot be evaluated as valid Python.") 324 | if not isinstance(sync_dirs, list): 325 | utils.abort( 326 | f"sync_dirs was not evaluated as a Python list, but as '{type(sync_dirs)}'" 327 | ) 328 | for sd in sync_dirs: 329 | if not isinstance(sd, list): 330 | utils.abort( 331 | f"sync_dirs element {sd} was not evaluated as a Python list, but as " 332 | f"'{type(sd)}'" 333 | ) 334 | 335 | # Normalize source dirs. Target Dirs must be absolute / handled by Vagrant. 336 | sync_dirs = [ 337 | [os.path.realpath(os.path.expanduser(lst[0])), lst[1]] for lst in sync_dirs 338 | ] 339 | set_env_var("sync_dirs", sync_dirs) 340 | return sync_dirs 341 | 342 | 343 | def sync_type_option(sync_type=None): 344 | """Validate and set sync_type. 345 | 346 | Args: 347 | sync_type: Type of syncing to use. 348 | 349 | Return sync_type (str) 350 | """ 351 | if sync_type in SETTINGS["SYNC_TYPES"]: 352 | set_env_var("sync_type", sync_type) 353 | elif sync_type: 354 | utils.warn( 355 | f"Sync type {sync_type} not in approved list. Using Vagrant's default." 356 | f"Supported alternate sync types are {SETTINGS['SYNC_TYPES']}." 357 | ) 358 | sync_type = None 359 | 360 | return sync_type 361 | 362 | 363 | def vm_name_option(vm_name=None): 364 | """Set vm_name 365 | 366 | Args: 367 | vm_name (str): Vm_Name to set in VM / container 368 | 369 | Return vm_name (str) 370 | 371 | """ 372 | if vm_name: 373 | set_env_var("vm_name", vm_name) 374 | return vm_name 375 | -------------------------------------------------------------------------------- /rambo/saltstack/etc/minion.d/minion.conf: -------------------------------------------------------------------------------- 1 | file_client: local 2 | -------------------------------------------------------------------------------- /rambo/saltstack/srv/pillars/.gitinclude: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/terminal-labs/rambo/16ea57a6ce6d7e5e5777669624f006b0f53d401c/rambo/saltstack/srv/pillars/.gitinclude -------------------------------------------------------------------------------- /rambo/saltstack/srv/salt/basebox/init.sls: -------------------------------------------------------------------------------- 1 | {% set os, os_family = salt['grains.item']('os', 'os_family') %} 2 | 3 | setup_basebox: 4 | pkg.installed: 5 | - pkgs: 6 | - rsync 7 | - p7zip 8 | - zip 9 | - unzip 10 | - wget 11 | - curl 12 | - nano 13 | - emacs 14 | {% if os_family == 'Debian'%} 15 | - build-essential 16 | - libreadline6-dev 17 | - libbz2-dev 18 | - libssl-dev 19 | - libsqlite3-dev 20 | - libncursesw5-dev 21 | - libffi-dev 22 | - libdb-dev 23 | - libexpat1-dev 24 | - zlib1g-dev 25 | - liblzma-dev 26 | - libgdbm-dev 27 | - libffi-dev 28 | - libmpdec-dev 29 | - libfreetype6-dev 30 | - libpq-dev 31 | {% elif os == 'RedHat' %} 32 | - epel-release 33 | {% endif %} 34 | -------------------------------------------------------------------------------- /rambo/saltstack/srv/salt/top.sls: -------------------------------------------------------------------------------- 1 | base: 2 | '*': 3 | - basebox 4 | -------------------------------------------------------------------------------- /rambo/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "PLUGINS": { 3 | "Linux": [ 4 | "vagrant-aws", 5 | "vagrant-digitalocean", 6 | "vagrant-disksize", 7 | "vagrant-lxc", 8 | "vagrant-scp" 9 | ], 10 | "Darwin": [ 11 | "vagrant-aws", 12 | "vagrant-digitalocean", 13 | "vagrant-disksize", 14 | "vagrant-scp" 15 | ] 16 | }, 17 | "PROJECT_NAME": "rambo", 18 | "PROVIDERS": [ 19 | "digitalocean", 20 | "docker", 21 | "ec2", 22 | "lxc", 23 | "virtualbox" 24 | ], 25 | "PROVIDERS_DEFAULT": "virtualbox", 26 | "GUEST_OSES": { 27 | "debian-8": { 28 | "do": "debian-8-x64", 29 | "ec2": "ami-a5d621e1" 30 | }, 31 | "debian-9": { 32 | "do": "debian-9-x64", 33 | "ec2": "ami-8c828aec" 34 | }, 35 | "centos-7": { 36 | "do": "centos-7-x64", 37 | "ec2": "ami-65e0e305" 38 | }, 39 | "ubuntu-1404": { 40 | "do": "ubuntu-14-04-x64", 41 | "ec2": "ami-48030c28" 42 | }, 43 | "ubuntu-1604": { 44 | "do": "ubuntu-16-04-x64", 45 | "docker": "ubuntu/xenial64", 46 | "ec2": "ami-07585467" 47 | }, 48 | "ubuntu-1804": { 49 | "do": "ubuntu-18-04-x64", 50 | "docker": "ubuntu/bionic64", 51 | "ec2": "ami-03ba3948f6c37a4b0" 52 | } 53 | }, 54 | "GUEST_OSES_DEFAULT": "ubuntu-1604", 55 | "SIZES": { 56 | "512": "20", 57 | "1024": "30", 58 | "2048": "40", 59 | "4096": "60", 60 | "8192": "80" 61 | }, 62 | "SYNC_TYPES": [ 63 | "disabled", 64 | "rsync", 65 | "nfs", 66 | "smb" 67 | ], 68 | "RAMSIZE_DEFAULT": "1024", 69 | "DRIVESIZE_DEFAULT": "30" 70 | } 71 | -------------------------------------------------------------------------------- /rambo/settings.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | # GLOBALS 5 | # Create env var indicating where this code lives. This will be used latter by 6 | # Vagrant as a check that the python cli is being used, as well as being a useful var. 7 | PROJECT_LOCATION = os.path.dirname(os.path.realpath(__file__)) 8 | with open(os.path.join(PROJECT_LOCATION, "settings.json"), "r") as f: 9 | SETTINGS = json.load(f) 10 | PROVIDERS = SETTINGS["PROVIDERS"] 11 | PROJECT_NAME = SETTINGS["PROJECT_NAME"] 12 | CONF_FILES = [f"{PROJECT_NAME}.conf", f"my_{PROJECT_NAME}.conf"] 13 | -------------------------------------------------------------------------------- /rambo/utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | from pathlib import Path 4 | from shutil import copyfile 5 | from shutil import move 6 | from shutil import rmtree 7 | 8 | import click 9 | 10 | from rambo.settings import CONF_FILES 11 | from rambo.settings import PROJECT_NAME 12 | 13 | 14 | def set_env_var(name, value): 15 | """Set an environment variable in all caps that is prefixed with the name of the project""" 16 | os.environ[PROJECT_NAME.upper() + "_" + name.upper()] = str(value) 17 | 18 | 19 | def get_env_var(name): 20 | """Get an environment variable in all caps that is prefixed with the name of the project""" 21 | return os.environ.get(PROJECT_NAME.upper() + "_" + name.upper()) 22 | 23 | 24 | def abort(msg, log=True): 25 | msg = click.style("".join(["ABORTED - ", msg]), fg="red", bold=True) 26 | if log: 27 | write_to_log(msg, "stderr") 28 | sys.exit(msg) 29 | 30 | 31 | def echo(msg, err=None): 32 | if err: 33 | write_to_log(msg, "stderr") 34 | click.echo(msg, err=err) 35 | else: 36 | write_to_log(msg) 37 | click.echo(msg) 38 | 39 | 40 | def warn(msg): 41 | msg = click.style("".join(["WARNING - ", msg]), fg="yellow") 42 | echo(msg) 43 | 44 | 45 | def find_conf(path): 46 | if any(cf in os.listdir(path) for cf in CONF_FILES): 47 | return path 48 | for parent in Path(path).parents: 49 | if any(cf in os.listdir(parent) for cf in CONF_FILES): 50 | return parent 51 | 52 | 53 | def write_to_log(data=None, file_name=None): 54 | """Write data to log files. Will append data to a single combined log. 55 | Additionally write data to a log with a custom name (such as stderr) 56 | for any custom logs. 57 | 58 | Args: 59 | data (str or bytes): Data to write to log file. 60 | file_name (str): Used to create (or append to) an additional 61 | log file with a custom name. Custom name always gets 62 | `.log` added to the end. 63 | """ 64 | try: 65 | data = data.decode("utf-8") 66 | except AttributeError: 67 | pass # already a string 68 | 69 | # strip possible eol chars and add back exactly one 70 | data = "".join([data.rstrip(), "\n"]) 71 | 72 | dir_create(get_env_var("LOG_PATH")) 73 | fd_path = os.path.join(get_env_var("LOG_PATH"), "history.log") 74 | fd = open(fd_path, "a+") 75 | fd.write(data) 76 | fd.close() 77 | if file_name: 78 | fd_custom_path = os.path.join( 79 | get_env_var("LOG_PATH"), "".join([file_name, ".log"]) 80 | ) 81 | fd_custom = open(fd_custom_path, "a+") 82 | fd_custom.write(data) 83 | fd_custom.close() 84 | 85 | 86 | def dir_exists(path): 87 | return os.path.isdir(path) 88 | 89 | 90 | def dir_create(path): 91 | if not os.path.exists(path): 92 | os.makedirs(path) 93 | 94 | 95 | def dir_delete(path): 96 | try: 97 | rmtree(path) 98 | except FileNotFoundError: 99 | pass 100 | 101 | 102 | def file_delete(path): 103 | try: 104 | os.remove(path) 105 | except FileNotFoundError: 106 | pass 107 | 108 | 109 | def file_copy(src, dst): 110 | copyfile(src, dst) 111 | 112 | 113 | def file_rename(src, dst): 114 | os.rename(src, dst) 115 | 116 | 117 | def file_move(src, dst): 118 | move(src, dst) 119 | 120 | 121 | def get_user_home(): 122 | return str(Path.home()) 123 | -------------------------------------------------------------------------------- /rambo/vagrant/host_vms/auxiliary_disks/.gitignore: -------------------------------------------------------------------------------- 1 | # Ignore everything in this directory 2 | * 3 | # Except this file 4 | !.gitignore 5 | -------------------------------------------------------------------------------- /rambo/vagrant/host_vms/docker_host: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | btrfs_volume = 'vagrant/host_vms/auxiliary_disks/docker_host_btrfs_volume.vdi' 5 | 6 | Vagrant.configure("2") do |config| 7 | config.vm.provider :virtualbox do |provider, override| 8 | provider.name = "docker_host" 9 | override.vm.box = get_env_var_rb('docker_box') ? get_env_var_rb('docker_box') : 10 | SETTINGS['GUEST_OSES'][SETTINGS['GUEST_OSES_DEFAULT']]['docker'] 11 | provider.customize ['modifyvm', :id, '--nictype1', 'virtio'] 12 | provider.customize ['modifyvm', :id, '--nictype2', 'virtio'] 13 | provider.memory = get_env_var_rb('ramsize').to_i 14 | provider.customize ['modifyvm', :id, '--nictype1', 'virtio'] 15 | unless File.exist?(btrfs_volume) 16 | provider.customize ['createhd', '--filename', btrfs_volume, '--size', get_env_var_rb('drivesize').to_i * 1024] 17 | end 18 | provider.customize ['storageattach', :id, '--storagectl', 'SCSI', '--port', 2, '--device', 0, '--type', 'hdd', '--medium', btrfs_volume] 19 | end 20 | 21 | config.vm.define "docker_host" 22 | 23 | # Configure Docker in intermediate Ubuntu host 24 | $config_docker_host = <<-EOS 25 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - 26 | add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" 27 | apt-get update 28 | apt-get install -y docker-ce 29 | usermod -a -G docker ubuntu 30 | usermod -a -G docker vagrant 31 | ps aux | grep 'sshd:' | awk '{print $2}' | xargs kill 32 | EOS 33 | 34 | config.vm.provision "shell", 35 | inline: $config_docker_host, 36 | keep_color: true 37 | end 38 | -------------------------------------------------------------------------------- /rambo/vagrant/modules.rb: -------------------------------------------------------------------------------- 1 | require 'securerandom' 2 | 3 | def random_tag 4 | host = `hostname`.strip # Get the hostname from the shell and removing trailing \n 5 | tmp_dir = get_env_var_rb('TMPDIR') || File.join(Dir.pwd, '.' + PROJECT_NAME + '-tmp') 6 | Dir.mkdir(tmp_dir) unless Dir.exist?(tmp_dir) 7 | random_tag_path = File.join(tmp_dir, 'random_tag') 8 | if File.file?(random_tag_path) 9 | tag = File.read(random_tag_path) 10 | else 11 | # 95 is unlikely to be used. It is the ascii code for an underscore 12 | replacements = {'_' => '95'} 13 | guest_hostname = host + '-' + File.basename(File.dirname(tmp_dir)) 14 | guest_hostname = guest_hostname.gsub(Regexp.union(replacements.keys), replacements) 15 | guest_hostname = truncate(guest_hostname, 43) # 64 - 15 for the next line - 6 for "rambo-" 16 | tag = guest_hostname + '-' + SecureRandom.hex(6) 17 | File.write(random_tag_path, tag) 18 | end 19 | return tag 20 | end 21 | 22 | def read_provider_file 23 | tmp_dir = get_env_var_rb('TMPDIR') || File.join(Dir.pwd, '.' + PROJECT_NAME + '-tmp') 24 | provider_path = File.join(tmp_dir, 'provider') 25 | if File.file?(provider_path) 26 | provider='' 27 | File.open(provider_path, 'r') do |f| 28 | provider = f.read() 29 | end 30 | return provider 31 | else 32 | return false 33 | end 34 | end 35 | 36 | def write_provider_file(provider) 37 | tmp_dir = get_env_var_rb('TMPDIR') || File.join(Dir.pwd, '.' + PROJECT_NAME + '-tmp') 38 | provider_path = File.join(tmp_dir, 'provider') 39 | File.write(provider_path, provider) 40 | end 41 | 42 | def set_env_var_rb(name, value) 43 | # Set an environment variable in all caps that is prefixed with the name of the project 44 | ENV[PROJECT_NAME.upcase + "_" + name.upcase] = value 45 | end 46 | 47 | def get_env_var_rb(name) 48 | # Get an environment variable in all caps that is prefixed with the name of the project 49 | return ENV[PROJECT_NAME.upcase + "_" + name.upcase] 50 | end 51 | 52 | def truncate(string, max) 53 | string.length > max ? "#{string[0...max]}" : string 54 | end 55 | -------------------------------------------------------------------------------- /rambo/vagrant/provider_support/docker/dockerfiles/debian_vagrant: -------------------------------------------------------------------------------- 1 | FROM nishidayuya/docker-vagrant-debian 2 | RUN apt-get update 3 | RUN apt-get -y upgrade 4 | RUN apt-get -y install ca-certificates 5 | RUN apt update 6 | RUN apt -y upgrade 7 | RUN apt -y install locales 8 | RUN apt -y install locales-all 9 | RUN apt -y install openssl 10 | RUN apt -y install wget 11 | RUN apt -y install python 12 | CMD ["/usr/sbin/sshd", "-D", "-e"] 13 | EXPOSE 22 14 | -------------------------------------------------------------------------------- /rambo/vagrant/provider_support/docker/dockerfiles/ubuntu_vagrant: -------------------------------------------------------------------------------- 1 | FROM nishidayuya/docker-vagrant-ubuntu:xenial 2 | RUN apt-get update 3 | RUN apt-get -y upgrade 4 | RUN apt-get -y install ca-certificates 5 | RUN apt update 6 | RUN apt -y upgrade 7 | RUN apt -y install locales 8 | RUN apt -y install locales-all 9 | RUN apt -y install openssl 10 | RUN apt -y install wget 11 | RUN apt -y install python 12 | CMD ["/usr/sbin/sshd", "-D", "-e"] 13 | EXPOSE 22 14 | -------------------------------------------------------------------------------- /rambo/vagrant/vagrantfiles/digitalocean: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | Vagrant.configure('2') do |config| 5 | config.vm.provider :digital_ocean do |provider, config| 6 | if not ENV['DIGITALOCEAN_PRIVATE_KEY_PATH'] 7 | puts "You need to set a private key path for DigitalOcean." 8 | end 9 | # set unique vm name 10 | config.vm.hostname = VM_NAME 11 | provider.token = ENV['DIGITALOCEAN_TOKEN'] 12 | provider.ssh_key_name = File.basename(ENV['DIGITALOCEAN_PRIVATE_KEY_PATH'], '.*') 13 | provider.image = get_env_var_rb('do_image') ? get_env_var_rb('do_image') : 14 | SETTINGS['GUEST_OSES'][SETTINGS['GUEST_OSES_DEFAULT']]['do'] 15 | provider.region = 'nyc1' 16 | config.vm.allowed_synced_folder_types = :rsync 17 | 18 | if get_env_var_rb('machinetype') 19 | provider.size = get_env_var_rb('machinetype') 20 | elsif get_env_var_rb('ramsize') == '512' 21 | provider.size = '512mb' 22 | elsif get_env_var_rb('ramsize') == '1024' 23 | provider.size = '1gb' 24 | elsif get_env_var_rb('ramsize') == '2048' 25 | provider.size = '2gb' 26 | elsif get_env_var_rb('ramsize') == '4096' 27 | provider.size = '4gb' 28 | elsif get_env_var_rb('ramsize') == '8192' 29 | provider.size = '8gb' 30 | end 31 | end 32 | 33 | config.ssh.username = 'vagrant' 34 | config.ssh.private_key_path = ENV['DIGITALOCEAN_PRIVATE_KEY_PATH'] 35 | config.ssh.forward_agent = FORWARD_SSH 36 | config.vm.box = 'digital_ocean' 37 | config.vm.box_url = 'https://github.com/devopsgroup-io/vagrant-digitalocean/raw/master/box/digital_ocean.box' 38 | end 39 | -------------------------------------------------------------------------------- /rambo/vagrant/vagrantfiles/docker: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | ENV['VAGRANT_DEFAULT_PROVIDER'] = 'docker' 5 | 6 | Vagrant.configure("2") do |config| 7 | 8 | config.vm.define "my-little-container" do |m| 9 | 10 | m.vm.provider :docker do |d| 11 | d.name = 'ubuntu' 12 | d.build_dir = "." 13 | d.remains_running = true 14 | d.force_host_vm = true 15 | d.has_ssh = true 16 | d.dockerfile = "vagrant/provider_support/docker/dockerfiles/ubuntu_vagrant" 17 | d.vagrant_machine = "docker_host" 18 | d.vagrant_vagrantfile = "vagrant/host_vms/docker_host" 19 | end 20 | end 21 | config.vm.hostname = VM_NAME 22 | end 23 | -------------------------------------------------------------------------------- /rambo/vagrant/vagrantfiles/ec2: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | # Recent vagrant broke this plugin without this class 5 | # https://github.com/mitchellh/vagrant-aws/issues/566#issuecomment-580812210 6 | class Hash 7 | def slice(*keep_keys) 8 | h = {} 9 | keep_keys.each { |key| h[key] = fetch(key) if has_key?(key) } 10 | h 11 | end unless Hash.method_defined?(:slice) 12 | def except(*less_keys) 13 | slice(*keys - less_keys) 14 | end unless Hash.method_defined?(:except) 15 | end 16 | 17 | 18 | Vagrant.configure('2') do |config| 19 | config.vm.provider :aws do |provider| 20 | # set unique vm name 21 | provider.tags = { 22 | 'Name' => VM_NAME 23 | } 24 | 25 | provider.access_key_id = ENV['AWS_ACCESS_KEY_ID'] 26 | provider.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY'] 27 | provider.keypair_name = ENV['AWS_KEYPAIR_NAME'] 28 | if get_env_var_rb('ec2_security_groups') 29 | provider.security_groups = eval(get_env_var_rb('ec2_security_groups')) 30 | else 31 | provider.security_groups = 'salted_server' 32 | end 33 | provider.availability_zone = 'us-west-1c' 34 | provider.region = 'us-west-1' 35 | provider.ami = get_env_var_rb('ami') ? get_env_var_rb('ami') : 36 | SETTINGS['GUEST_OSES'][SETTINGS['GUEST_OSES_DEFAULT']]['ec2'] 37 | 38 | if get_env_var_rb('machinetype') 39 | provider.instance_type = get_env_var_rb('machinetype') 40 | elsif get_env_var_rb('ramsize') == '512' 41 | provider.instance_type = 't2.nano' 42 | elsif get_env_var_rb('ramsize') == '1024' 43 | provider.instance_type = 't2.micro' 44 | elsif get_env_var_rb('ramsize') == '2048' 45 | provider.instance_type = 't2.small' 46 | elsif get_env_var_rb('ramsize') == '4096' 47 | provider.instance_type = 't2.medium' 48 | elsif get_env_var_rb('ramsize') == '8192' 49 | provider.instance_type = 't2.large' 50 | end 51 | 52 | if (get_env_var_rb('guest_os') != 'debian-9') && (get_env_var_rb('guest_os') != 'debian-8') 53 | provider.block_device_mapping = [{ 54 | 'DeviceName' => '/dev/sda1', 55 | 'Ebs.VolumeSize' => get_env_var_rb('drivesize'), 56 | 'Ebs.VolumeType' => 'gp2', 57 | 'Ebs.DeleteOnTermination' => 'true', 58 | }] 59 | end 60 | 61 | provider.user_data = '#!/bin/bash 62 | oldusers=( admin root ubuntu centos fedora ec2-user ) 63 | for olduser in "${oldusers[@]}"; do 64 | if grep -q $olduser "/etc/sudoers.d/90-cloud-init-users"; then 65 | user=vagrant 66 | usermod -l $user $olduser 67 | groupmod -n $user $olduser 68 | usermod -d /home/$user -m $user 69 | sed -i "s/$olduser/vagrant/g" /etc/sudoers.d/90-cloud-init-users 70 | fi 71 | done 72 | ' 73 | end 74 | config.ssh.username = 'vagrant' 75 | config.ssh.private_key_path = ENV['AWS_SSH_PRIVKEY'] 76 | config.ssh.forward_agent = true 77 | 78 | config.vm.box = 'dummy' 79 | config.vm.box_url = 'https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box' 80 | end 81 | -------------------------------------------------------------------------------- /rambo/vagrant/vagrantfiles/lxc: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | Vagrant.configure("2") do |config| 5 | config.vm.provider :lxc do |provider| 6 | provider.backingstore = "loop" 7 | provider.customize "cgroup.memory.limit_in_bytes", get_env_var_rb("RAMSIZE") + 'M' 8 | provider.backingstore_option "--fssize", get_env_var_rb("DRIVESIZE") + 'G' 9 | end 10 | config.ssh.username = "vagrant" 11 | config.ssh.forward_agent = FORWARD_SSH 12 | config.vm.hostname = VM_NAME 13 | config.vm.network :forwarded_port, 14 | :guest => 5000, 15 | :host => 5000, 16 | auto_correct: true 17 | config.vm.network :forwarded_port, 18 | :guest => 80, 19 | :host => 8080, 20 | auto_correct: true 21 | box = 'terminal-labs/tl-' + get_env_var_rb("GUEST_OS") + '-64bit-lxc' 22 | config.vm.box = box 23 | config.vm.box_url = box 24 | end 25 | -------------------------------------------------------------------------------- /rambo/vagrant/vagrantfiles/virtualbox: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | Vagrant.configure("2") do |config| 5 | config.vm.provider :virtualbox do |provider| 6 | if get_env_var_rb("BOX") 7 | box = get_env_var_rb("BOX") 8 | config.disksize.size = get_env_var_rb("DRIVESIZE") + "GB" 9 | else 10 | box = "terminal-labs/tl-" + get_env_var_rb("GUEST_OS") + "-64bit-" + get_env_var_rb("DRIVESIZE") + 'gb' 11 | end 12 | config.vm.box = box 13 | config.vm.box_url = box 14 | config.vm.network :private_network, 15 | ip: get_env_var_rb("IP") 16 | 17 | provider.name = VM_NAME # unique 18 | provider.gui = get_env_var_rb("GUI") || false 19 | provider.memory = get_env_var_rb("RAMSIZE") 20 | provider.cpus = get_env_var_rb("CPUS") 21 | 22 | # OS specific settings 23 | if config.vm.box.include? "windows" 24 | # Configuration specific to Windows 25 | if provider.gui 26 | provider.customize ['setextradata', :id, 'GUI/LastGuestSizeHint', get_env_var_rb("RESOLUTION")] 27 | puts "res:" 28 | puts get_env_var_rb("RESOLUTION") 29 | end 30 | provider.customize ["modifyvm", :id, "--nic-type1", "82540EM"] 31 | provider.customize ["modifyvm", :id, "--nic-type2", "82540EM"] 32 | config.vm.communicator = "winrm" 33 | config.winrm.host = get_env_var_rb("IP") 34 | config.winrm.username = "vagrant" 35 | config.winrm.password = "vagrant" 36 | else 37 | # Configuration specific to Unix/Linux 38 | provider.customize ['modifyvm', :id, '--nictype1', 'virtio'] 39 | provider.customize ['modifyvm', :id, '--nictype2', 'virtio'] 40 | config.vm.communicator = "ssh" 41 | config.ssh.username = "vagrant" 42 | config.ssh.forward_agent = FORWARD_SSH 43 | end 44 | end 45 | end 46 | -------------------------------------------------------------------------------- /rambo/vagrant_providers.py: -------------------------------------------------------------------------------- 1 | from rambo.settings import SETTINGS 2 | from rambo.utils import abort 3 | from rambo.utils import get_env_var 4 | from rambo.utils import set_env_var 5 | 6 | 7 | def digitalocean(): 8 | """DigitalOcean specific preparation for Vagrant. Setting and validating env vars. 9 | 10 | Set env vars: do_image 11 | 12 | Sanitize against non-whitelist ramsize and drivesize. 13 | """ 14 | if get_env_var("guest_os"): # only set during `up` 15 | set_env_var("do_image", SETTINGS["GUEST_OSES"][get_env_var("guest_os")]["do"]) 16 | if get_env_var("ramsize") not in SETTINGS["SIZES"]: 17 | abort( 18 | "Sorry, we really need a RAM size from our whitelist for " 19 | "digitalocean. \nThe only way around that is if you specify " 20 | "a machine-type like s-8vcpu-32gb." 21 | ) 22 | if get_env_var("drivesize") not in SETTINGS["SIZES"].values(): 23 | abort( 24 | "Sorry, we really need a drive size from our whitelist for " 25 | "digitalocean. \nThe only way around that is if you specify " 26 | "a machine-type like s-8vcpu-32gb." 27 | ) 28 | 29 | 30 | def docker(): 31 | """Docker specific preparation for Vagrant. Setting and validating env vars. 32 | 33 | Set env vars: docker_box 34 | """ 35 | if get_env_var("guest_os"): # only set during `up` 36 | set_env_var( 37 | "docker_box", SETTINGS["GUEST_OSES"][get_env_var("guest_os")]["docker"] 38 | ) 39 | 40 | 41 | def ec2(**params): 42 | """EC2 specific preparation for Vagrant. Setting and validating env vars. 43 | 44 | Set env vars: ami 45 | 46 | Sanitize against non-whitelist ramsize. 47 | """ 48 | if get_env_var("guest_os"): # only set during `up` 49 | set_env_var("ami", SETTINGS["GUEST_OSES"][get_env_var("guest_os")]["ec2"]) 50 | if get_env_var("ramsize") not in SETTINGS["SIZES"]: 51 | abort( 52 | "Sorry, we really need a RAM size from our whitelist for " 53 | "digitalocean. \nThe only way around that is if you specify " 54 | "a machine-type like m3.medium." 55 | ) 56 | 57 | if params.get("ec2_security_groups"): 58 | set_env_var("ec2_security_groups", params["ec2_security_groups"]) 59 | -------------------------------------------------------------------------------- /requirements-dev.lock: -------------------------------------------------------------------------------- 1 | # generated by rye 2 | # use `rye lock` or `rye sync` to update this lockfile 3 | # 4 | # last locked with the following flags: 5 | # pre: false 6 | # features: [] 7 | # all-features: false 8 | # with-sources: false 9 | 10 | -e file:. 11 | alabaster==0.7.16 12 | # via sphinx 13 | asttokens==2.4.1 14 | # via stack-data 15 | babel==2.14.0 16 | # via sphinx 17 | certifi==2024.2.2 18 | # via requests 19 | cfgv==3.4.0 20 | # via pre-commit 21 | charset-normalizer==3.3.2 22 | # via requests 23 | click==8.1.7 24 | # via rambo-vagrant 25 | commonmark==0.9.1 26 | # via recommonmark 27 | decorator==5.1.1 28 | # via ipdb 29 | # via ipython 30 | distlib==0.3.8 31 | # via virtualenv 32 | docutils==0.20.1 33 | # via recommonmark 34 | # via sphinx 35 | # via sphinx-rtd-theme 36 | executing==2.0.1 37 | # via stack-data 38 | filelock==3.14.0 39 | # via virtualenv 40 | identify==2.5.36 41 | # via pre-commit 42 | idna==3.7 43 | # via requests 44 | imagesize==1.4.1 45 | # via sphinx 46 | ipdb==0.13.13 47 | ipython==8.24.0 48 | # via ipdb 49 | jedi==0.19.1 50 | # via ipython 51 | jinja2==3.1.3 52 | # via sphinx 53 | markupsafe==2.1.5 54 | # via jinja2 55 | matplotlib-inline==0.1.7 56 | # via ipython 57 | nodeenv==1.8.0 58 | # via pre-commit 59 | packaging==24.0 60 | # via sphinx 61 | parso==0.8.4 62 | # via jedi 63 | pexpect==4.9.0 64 | # via ipython 65 | platformdirs==4.2.1 66 | # via virtualenv 67 | pre-commit==3.7.0 68 | prompt-toolkit==3.0.43 69 | # via ipython 70 | ptyprocess==0.7.0 71 | # via pexpect 72 | pure-eval==0.2.2 73 | # via stack-data 74 | pygments==2.18.0 75 | # via ipython 76 | # via sphinx 77 | pyyaml==6.0.1 78 | # via pre-commit 79 | recommonmark==0.7.1 80 | requests==2.31.0 81 | # via sphinx 82 | setuptools==69.5.1 83 | # via nodeenv 84 | # via rambo-vagrant 85 | six==1.16.0 86 | # via asttokens 87 | snowballstemmer==2.2.0 88 | # via sphinx 89 | sphinx==7.3.7 90 | # via recommonmark 91 | # via sphinx-rtd-theme 92 | # via sphinxcontrib-jquery 93 | sphinx-rtd-theme==2.0.0 94 | sphinxcontrib-applehelp==1.0.8 95 | # via sphinx 96 | sphinxcontrib-devhelp==1.0.6 97 | # via sphinx 98 | sphinxcontrib-htmlhelp==2.0.5 99 | # via sphinx 100 | sphinxcontrib-jquery==4.1 101 | # via sphinx-rtd-theme 102 | sphinxcontrib-jsmath==1.0.1 103 | # via sphinx 104 | sphinxcontrib-qthelp==1.0.7 105 | # via sphinx 106 | sphinxcontrib-serializinghtml==1.1.10 107 | # via sphinx 108 | stack-data==0.6.3 109 | # via ipython 110 | traitlets==5.14.3 111 | # via ipython 112 | # via matplotlib-inline 113 | urllib3==2.2.1 114 | # via requests 115 | virtualenv==20.26.1 116 | # via pre-commit 117 | wcwidth==0.2.13 118 | # via prompt-toolkit 119 | -------------------------------------------------------------------------------- /requirements.lock: -------------------------------------------------------------------------------- 1 | # generated by rye 2 | # use `rye lock` or `rye sync` to update this lockfile 3 | # 4 | # last locked with the following flags: 5 | # pre: false 6 | # features: [] 7 | # all-features: false 8 | # with-sources: false 9 | 10 | -e file:. 11 | click==8.1.7 12 | # via rambo-vagrant 13 | setuptools==69.5.1 14 | # via rambo-vagrant 15 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [flake8] 2 | # Staple ignores. Exclude common dirs and ignore rules black handles. 3 | exclude=wsgi.py,env/*,*/migrations/*,venv/*,.env/*,.venv/*,local_settings.py,doc/*,*/node_modules/* 4 | ignore=E203,F403,E128,E126,E123,E121,E265,E501,N802,N803,N806,C901,D100,D102,D102,D10,W503,E731,E402 5 | --------------------------------------------------------------------------------