├── .gitignore ├── LICENSE ├── README.md └── terraform ├── ToDo ├── create-urbit-comet.exp ├── provision-urbit.sh ├── urbit-pier.tf └── vars.tf /.gitignore: -------------------------------------------------------------------------------- 1 | .terraform/ 2 | terraform/ip_address.txt 3 | terraform/*.tfstate* 4 | terraform/*.tfvars 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Daryl Richter 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # urbit-devops 2 | 3 | ## The easiest, fastest way to get an urbit up and running in the AWS cloud. 4 | 5 | Initially, this is only available for Amazon AWS, but I’ll be looking to expand it to other cloud services in the future. The first method uses [Hashicorp’s Terraform](https://www.terraform.io) and is basically restricted to just getting an urbit up and running, but I am also working on another branch to have an even more fully-featured set of scripts using [ansible](https://www.ansible.com). 6 | 7 | 8 | ### Step 1: Create an [Amazon AWS](https://aws.amazon.com) Account. 9 | 10 | If you haven’t done this already you are in luck because Amazon will let you run your urbit on an EC2 micro instance for a whole year for free. 11 | 12 | Now, log into your account and go to the Identity and Access Management (IAM) Dashboard. 13 | 14 | Amazon will now want you to make your account more secure by performing 5 steps. 15 | 16 | 17 | ![Once you have completed all 5 steps it will look like this](http://www.ngzax.com/images/terraform/amazon-security-status.png) 18 | 19 | 20 | These are all good ideas. I am not going to cover #1, 2, or 5 but Amazon’s docs are pretty decent. We need to make the group and non-root user for our scripts anyway so let’s dive in. 21 | 22 | 23 | ### Step 2: Create a group for defining permissions for your automation script user 24 | 25 | 26 | - Click on the “Groups” link on the left sidebar and then click the “Create New Group” button: 27 | 28 | ![](http://www.ngzax.comhttp://www.ngzax.com/images/terraform/create-group-0.png) 29 | 30 | - Give your new groups a name, I chose “Deployers”: 31 | 32 | ![](http://www.ngzax.com/images/terraform/create-group-1.png) 33 | 34 | - Attach the Policy “PowerUser” to the group. This will give it the appropriate permissions for our scripts to be able to create and manipulate our urbit EC2 instances 35 | 36 | ![](http://www.ngzax.com/images/terraform/create-group-2.png) 37 | 38 | - Review and Accept 39 | 40 | ![](http://www.ngzax.com/images/terraform/create-group-3.png) 41 | 42 | 43 | ### Step 3: Create a dedicated user to run your devops scripts 44 | 45 | We want to create a separate user account to perform these operations. It is not considered wise to use your root Amazon account to do this. 46 | 47 | 48 | - Click on the “Users” link on the left-hand side and then click the button to add a User. 49 | 50 | 51 | ![](http://www.ngzax.com/images/terraform/add-user-0.png) 52 | 53 | - You will then be able to give the user a name, in this case I chose “ansible,” but anything will do. Make sure to check the box for “programmatic access” as this user should never be able to log into the AWS console. 54 | 55 | ![](http://www.ngzax.com/images/terraform/add-user-1.png) 56 | 57 | - Add your user into the “Deployers” group that we created in **Step 2**. 58 | 59 | ![](http://www.ngzax.com/images/terraform/add-user-2.png) 60 | 61 | - Review and confirm. 62 | 63 | ![](http://www.ngzax.com/images/terraform/add-user-3.png) 64 | 65 | - Amazon will now give you the opportunity to download the credentials (Access Key and Secret Key) for you user. You will need this. Please download and store it in a safe place. 66 | 67 | ![](http://www.ngzax.com/images/terraform/add-user-4.png) 68 | 69 | ### Step 4: Create a key pair and download the private key for your desired EC2 zone. 70 | ![](http://www.ngzax.com/images/terraform/create-keypair-0.png) 71 | 72 | 73 | As you can see, I named it “{user}-key-pair-{region}”. This is arbitrary but will help you identify it later. Key pairs are unique within regions so it helps to have that in the name as well. When you create it it will automatically download the private key portion to your computer with the name “{keypair name}.pem.txt”. 74 | 75 | You need to take this file, remove the .txt extension and save it in whatever directory you store your private keys. On most unix systems this is ~/.ssh/. You also need to change the permission on it so that only the owner can access it. On unix this is accomplished by running, for example: 76 | 77 | 78 | chmod 400 ~/.ssh/ansible-key-pair-ohio.pem 79 | 80 | 81 | ### Step 5: Set up your Amazon credentials 82 | 83 | If you don’t have it already, create a .aws directory in your home directory. Create a file named “credentials” in it and add your AWS access key and secret accees key to it: 84 | 85 | 86 | [default] 87 | aws_access_key_id={YOUR_ACCESS_KEY} 88 | aws_secret_access_key={YOUR_SECRET_KEY} 89 | 90 | If you have multiple AWS account you can add multiple named sections to this file for each, I will show you how to pick which one later one. 91 | 92 | 93 | ### Step 6: Clone the [urbit-devops git repository](https://github.com/ngzax/urbit-devops) 94 | 95 | 96 | $ git clone git@github.com:ngzax/urbit-devops.git 97 | $ cd urbit-devops/terraform 98 | 99 | ### Step 7: Create a terraform.tfvars file in the terraform directory 100 | 101 | This is the file that contains the specific information about your Amazon 102 | account and credentials and where they exist on your local computer. At minimum, 103 | the file needs to contain 3 lines that should look something like this: 104 | 105 | $ cat terraform.tfvars 106 | PATH_TO_PRIVATE_KEY = "~/.ssh/ansible-key-pair-ohio.pem" 107 | PRIVATE_KEY_NAME = "ansible-key-pair-ohio" 108 | SHARED_CREDENTIALS_FILE = "~/.aws/credentials" 109 | 110 | The path to private key is the one that you created and downloaded in step 4. 111 | The name is just its name and the shared credentials is what you created in step 112 | 5. 113 | 114 | If you have more than one Amazon account profile in your ~/.aws/credentials file 115 | you can select the one you want to use by adding it to terraform.tfvars: 116 | 117 | PROFILE = "ansible" 118 | 119 | Another important variable that you probably want to override is the 120 | ALLOW_SSH_FROM_IPS. This variable controls what ip addresses (or ranges) are 121 | allowed to access your urbit server via ssh. You will want to determine your 122 | local ip address and make sure to put that in there instead of the default which 123 | allows all addresses. 124 | 125 | [Click here for Amazon's documentation about security groups.](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html) 126 | 127 | ### Step 8: Install and Run Terraform 128 | 129 | Go to https://www.terraform.io/downloads.html and download the terraform binary. 130 | Put it somewhere in your path. 131 | 132 | From the urbit-devops/terraform directory you can run “terraform plan” to see 133 | what terraform is going to execute on your behalf. Once you are ready, to 134 | actually deploy your urbit, use “terraform apply” 135 | 136 | 137 | $ terraform apply 138 | aws_security_group.urbit-sg: Creating... 139 | description: "" => "Urbit Default Security Group" 140 | egress.#: "" => "" 141 | ingress.#: "" => "4" 142 | . 143 | . 144 | . 145 | aws_security_group.urbit-sg: Creation complete (ID: sg-eda66985) 146 | aws_instance.urbit-pier: Creating... 147 | ami: "" => "ami-8a7859ef" 148 | associate_public_ip_address: "" => "" 149 | . 150 | . 151 | . 152 | aws_instance.urbit-pier: Still creating... (3m10s elapsed) 153 | aws_instance.urbit-pier: Still creating... (3m20s elapsed) 154 | 155 | 156 | 157 | ### Step 9: Check out your new urbit comet 158 | 159 | The terraform process created a file in the terraform directory called “ip_address.txt”. This is the public ip_address of your new urbit comet. 160 | 161 | 162 | $ more ip_address.txt 163 | 52.15.141.16 164 | 165 | Put this address in your web browser and you should see the homepage of your new comet: 166 | 167 | ![](http://www.ngzax.com/images/terraform/urbit-new-comet.png) 168 | 169 | You can click on the Log In link and it will ask you for a code. To get the code you’ll need to connect to your new comet via ssh using the -i option and passing it the path to your key pair as shown: 170 | 171 | 172 | $ ssh -i ~/.ssh/ngzax-key-pair-ohio.pem ec2-user@52.15.141.16 173 | Last login: Tue Aug 22 14:40:15 2017 from 158.81.208.251 174 | 175 | __| __|_ ) 176 | _| ( / Amazon Linux AMI 177 | ___|\___|___| 178 | 179 | https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/ 180 | 181 | Your comet is running in a “screen” session. You can reattach to it and type “+code” in the dojo to get your code: 182 | 183 | 184 | $ screen -r 185 | ~ 186 | urbit 0.4.5 187 | urbit: home is new-comet 188 | loom: mapped 2048MB 189 | protected loom 190 | live: loaded: MB/71.188.480 191 | boot: installed 239 jets 192 | arvo: time: ~2017.8.22..14.47.04..c937 193 | rest: checkpoint to event 2.106 194 | rest: old 0vb.s9sdj, new 0v1u.3slgt 195 | loaded passcode from new-comet/.urb/code.~maplux-tanleg 196 | 197 | ---------------- playback complete---------------- 198 | ames: on localhost, UDP 53033. 199 | http: live (insecure, public) on 8080 200 | http: live ("secure", public) on 8443 201 | http: live (insecure, loopback) on 12321 202 | ames: czar zod.urbit.org: ip .104.197.214.171 203 | ~sipmun_borpex:dojo> +code 204 | 205 | To detach from your screen session use Ctrl-a and then “d” Your comet will continue to run until to reattach to the session and type Ctrl-d to stop urbit and then “exit”. 206 | 207 | You can disconnect your ssh session by typing “exit”. 208 | 209 | Enjoy! 210 | 211 | ---------- 212 | 213 | ## Contributing 214 | 215 | Contributions and corrections are welcome. Please submit a PR. 216 | 217 | ---------- 218 | 219 | ## FAQ 220 | 221 | ### Does my server have any security? 222 | 223 | The terraform script creates a security group for you to make your urbit more secure. If it didn’t, your urbit would be wide open to all connections inbound and outbound from the entire internet and thus vulnerable to attach. (The amazon default) 224 | 225 | 226 | - The group by default is named “urbit-sg-ohio” you can change the state suffix if you decide to run your urbit in a different AWS availability zone by setting the STATE var in your terraform.tfvars file. 227 | - You can see the security group in the EC2 console 228 | 229 | 230 | ### Do I have to use screen? 231 | 232 | No. The terraform script has also install tmux if you prefer to use that to run your urbit in the background. 233 | 234 | 235 | ### What if I have a planet? 236 | 237 | You ssh to your urbit server as described above, stop your comet with Ctrl-d and exit screen. 238 | 239 | 240 | $ screen # or tmux 241 | $ cd ~/piers/ 242 | $ urbit -w {PLANET_NAME} -t {PLANET_TICKET} 243 | 244 | 245 | ### How do I remove my urbit server? 246 | 247 | From urbit-devops/terraform directory: 248 | 249 | 250 | terraform destroy 251 | 252 | ---------- 253 | 254 | ## Contributing 255 | 256 | I would appreciate contributions to urbit-devops. 257 | 258 | Everyone involved in the ** urbit-devops ** project needs to understand and 259 | respect our Code of Conduct, which has been shamelessly and appropriately 260 | borrowed in its entirety from [urbit][2]: 261 | 262 | > "don't be rude." 263 | 264 | If you can code and abide by the simple Code of Conduct, fork the repo and submit a PR. I will review it as soon as I can. 265 | 266 | There is a list of requested features and bugs [in the issue tracker][1]. 267 | 268 | 269 | ## License 270 | 271 | "urbit-devops" is copyright (c) 2017 by Daryl Richter (ngzax). 272 | 273 | urbit-devops source code is released under the MIT License. 274 | 275 | Check the [LICENSE](LICENSE) file for more information. 276 | 277 | 278 | [1]: https://github.com/ngzax/urbit-devops/issues 279 | [2]: https://github.com/urbit/urbit 280 | -------------------------------------------------------------------------------- /terraform/ToDo: -------------------------------------------------------------------------------- 1 | Security: Change the operator group from Admin to the group with minimal privelages. 2 | Docs: Keypairs are no longer unique to regions. 3 | Consider rewriting firewall in provision-urbit.sh with UFW which is what the docs use. 4 | 5 | Docs: Rewrite the docs for planents instead of comets. 6 | -------------------------------------------------------------------------------- /terraform/create-urbit-comet.exp: -------------------------------------------------------------------------------- 1 | #!/usr/bin/bash 2 | 3 | 4 | ##!/usr/bin/expect -f 5 | 6 | #set timeout -1 7 | 8 | #spawn screen 9 | 10 | #send "cd piers && urbit -c new-comet\n" 11 | #expect { 12 | # "urbit 0.4.5" { 13 | # send "\001" 14 | # send "d" 15 | # } 16 | #} 17 | 18 | #Screen would hang after launching the comet, and my attempt at using tmux failed as well. I assumed that the user will access login code with HD wallet. 19 | 20 | cd piers && ./urbit -c new-comet 21 | -------------------------------------------------------------------------------- /terraform/provision-urbit.sh: -------------------------------------------------------------------------------- 1 | # Make sure all our base packages are up-to-date 2 | sudo apt update 3 | 4 | # Install all urbit dependencies 5 | sudo apt install -y screen automake autoconf libtool cmake 6 | 7 | # Download build, and install the urbit source package 8 | cd ~ 9 | mkdir source 10 | cd source 11 | wget https://bootstrap.urbit.org/urbit-v0.10.8-linux64.tgz 12 | 13 | tar xfvz urbit-v0.10.8-linux64.tgz 14 | cd urbit-v0.10.8-linux64 15 | mv urbit* ../../piers 16 | 17 | # Add 2GB of swap so that we have enough memory to start up 18 | sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=2048 19 | sudo /sbin/mkswap /var/swap.1 20 | sudo chmod 600 /var/swap.1 21 | sudo /sbin/swapon /var/swap.1 22 | 23 | #ToDo: The following line does not seem to write to fstab. Also, the above does not persist after rebooting; test and fix this. 24 | #sudo echo "swap /var/swap.1 swap defaults 0 0" >> /etc/fstab 25 | 26 | # Set up forwarding from port 80 (http) to 8080 and 443 (https) to 8443 27 | sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination :8080 28 | sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j DNAT --to-destination :8443 29 | 30 | # Other packages that are nice to have on the server 31 | sudo apt install -y expect git tmux 32 | -------------------------------------------------------------------------------- /terraform/urbit-pier.tf: -------------------------------------------------------------------------------- 1 | provider "aws" { 2 | shared_credentials_file = "${var.SHARED_CREDENTIALS_FILE}" 3 | profile = "${var.PROFILE}" 4 | region = "${var.AWS_REGION}" 5 | } 6 | 7 | resource "aws_security_group" "urbit-sg" { 8 | name = "urbit-sg-${var.STATE}" 9 | description = "Urbit Default Security Group" 10 | 11 | ingress { 12 | from_port = 22 13 | to_port = 22 14 | protocol = "tcp" 15 | cidr_blocks = "${var.ALLOW_SSH_FROM_IPS}" 16 | } 17 | 18 | ingress { 19 | from_port = 80 20 | to_port = 80 21 | protocol = "tcp" 22 | cidr_blocks = ["0.0.0.0/0"] 23 | } 24 | 25 | ingress { 26 | from_port = 443 27 | to_port = 443 28 | protocol = "tcp" 29 | cidr_blocks = ["0.0.0.0/0"] 30 | } 31 | 32 | ingress { 33 | from_port = 0 34 | to_port = 65535 35 | protocol = "udp" 36 | cidr_blocks = ["0.0.0.0/0"] 37 | } 38 | 39 | egress { 40 | from_port = 0 41 | to_port = 0 42 | protocol = "-1" 43 | cidr_blocks = ["0.0.0.0/0"] 44 | } 45 | } 46 | 47 | resource "aws_instance" "urbit-pier" { 48 | ami = "${var.AWS_AMI_ID}" 49 | instance_type = "${var.AWS_INSTANCE_TYPE}" 50 | key_name = "${var.PRIVATE_KEY_NAME}" 51 | security_groups = ["${aws_security_group.urbit-sg.name}"] 52 | 53 | #Added = to upgrade from <=0.11 to Terraform 0.13 54 | tags = { 55 | purpose = "urbit" 56 | } 57 | #Added "host = self.public_ip" to upgrade from <=0.11 to Terraform 0.13 58 | provisioner "file" { 59 | connection { 60 | type = "ssh" 61 | host = self.public_ip 62 | user = "${var.AWS_USERNAME}" 63 | private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}" 64 | } 65 | source = "provision-urbit.sh" 66 | destination = "provision-urbit.sh" 67 | } 68 | 69 | provisioner "file" { 70 | connection { 71 | type = "ssh" 72 | host = self.public_ip 73 | user = "${var.AWS_USERNAME}" 74 | private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}" 75 | } 76 | source = "create-urbit-comet.exp" 77 | destination = "create-urbit-comet.exp" 78 | } 79 | 80 | provisioner "remote-exec" { 81 | connection { 82 | type = "ssh" 83 | host = self.public_ip 84 | user = "${var.AWS_USERNAME}" 85 | private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}" 86 | } 87 | inline = [ 88 | "cd ~", 89 | "mkdir piers", 90 | "chmod +x provision-urbit.sh", 91 | "chmod +x create-urbit-comet.exp", 92 | "./provision-urbit.sh", 93 | "./create-urbit-comet.exp" 94 | ] 95 | } 96 | } 97 | 98 | resource "aws_eip" "ip" { 99 | instance = "${aws_instance.urbit-pier.id}" 100 | 101 | provisioner "local-exec" { 102 | command = "rm -f ip_address.txt" 103 | } 104 | 105 | provisioner "local-exec" { 106 | command = "echo ${aws_eip.ip.public_ip} > ip_address.txt" 107 | } 108 | } 109 | -------------------------------------------------------------------------------- /terraform/vars.tf: -------------------------------------------------------------------------------- 1 | # --- 2 | # These variable MUST be set in your terraform.tfvars file 3 | # --- 4 | variable "PATH_TO_PRIVATE_KEY" { } 5 | variable "PRIVATE_KEY_NAME" { } 6 | variable "SHARED_CREDENTIALS_FILE" { } 7 | 8 | # --- 9 | # These variable can be optionally overridden in your terraform.tfvars file 10 | # --- 11 | variable "ALLOW_SSH_FROM_IPS" { 12 | type = "list" 13 | default = ["0.0.0.0/0", "0.0.0.0/0"] 14 | } 15 | 16 | variable "ALLOW_SSH_FROM_IP_2" { 17 | default = "0.0.0.0/0" 18 | } 19 | 20 | variable "AWS_AMI_ID" { 21 | default = "ami-06e54d05255faf8f6" 22 | } 23 | 24 | variable "AWS_INSTANCE_TYPE" { 25 | default = "t2.micro" 26 | } 27 | 28 | variable "AWS_REGION" { 29 | default = "us-west-2" 30 | } 31 | 32 | variable "AWS_USERNAME" { 33 | default = "ubuntu" 34 | } 35 | 36 | variable "PROFILE" { 37 | default = "default" 38 | } 39 | 40 | variable "STATE" { 41 | default = "oregon" 42 | } 43 | 44 | variable "URBIT_SHIP_NAME" { 45 | default = "new-urbit-pier" 46 | } 47 | --------------------------------------------------------------------------------