├── .gitignore ├── Makefile ├── README.md ├── aws-bosh.tf ├── provision.sh ├── terraform.tfvars.example └── variables.tf /.gitignore: -------------------------------------------------------------------------------- 1 | *.tfvars 2 | *.tfplan 3 | *.tfstate 4 | *.tfstate.backup 5 | crash.log 6 | .terraform/ 7 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all plan apply destroy 2 | 3 | all: plan apply 4 | 5 | plan: 6 | terraform get -update 7 | terraform plan -module-depth=-1 -var-file terraform.tfvars -out terraform.tfplan 8 | 9 | apply: 10 | terraform apply -var-file terraform.tfvars 11 | 12 | destroy: 13 | terraform plan -destroy -var-file terraform.tfvars -out terraform.tfplan 14 | terraform apply terraform.tfplan 15 | 16 | clean: 17 | rm -f terraform.tfplan 18 | rm -f terraform.tfstate 19 | rm -fR .terraform/ 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | terraform-aws-bosh 2 | ================== 3 | 4 | This project will create an AWS VPC with subnets/route tables, a bastion VM (aka jumpbox/inception server), NAT (for outbound traffic), and a Micro BOSH. 5 | 6 | Architecture 7 | ------------ 8 | 9 | This terraform project will deploy the following networking and instances (pretty diagram from https://ide.visualops.io): 10 | 11 | ![](http://cl.ly/image/1u1F462W2W0p/terraform-aws-bosh_architecture.png) 12 | 13 | We rely on one other terraform repository: 14 | 15 | - [terraform-aws-vpc](https://github.com/cloudfoundry-community/terraform-aws-vpc) repo creates the base VPC infrastructure, including a bastion subnet, the`microbosh` subnet, a NAT server, various route tables, and the VPC itself 16 | 17 | This repository then creates a bastion VM and uses it to bootstrap a Micro BOSH (using the [bosh-bootstrap](https://github.com/cloudfoundry-community/bosh-bootstrap) project) into one of the private subnets. 18 | 19 | To access the BOSH, first SSH into the bastion VM. 20 | 21 | Deploy 22 | ------ 23 | 24 | ### Prerequisites 25 | 26 | The one step that isn't automated is the creation of SSH keys. We are waiting for that feature to be [added to terraform](https://github.com/hashicorp/terraform/issues/28). An AWS SSH Key need to be created in desired region prior to running the following commands. Note the name of the key and the path to the pem/private key file for use further down. 27 | 28 | You **must** being using at least terraform version 0.3.6. Follow the `make dev` [build instructions](https://github.com/hashicorp/terraform/#developing-terraform) to ensure plugins are built too. 29 | 30 | ``` 31 | $ terraform -v 32 | Terraform v0.3.6.dev 33 | ``` 34 | 35 | Optionally for using the `Unattended Install` instruction, install git. 36 | 37 | ### Setup 38 | 39 | ```bash 40 | git clone https://github.com/cloudfoundry-community/terraform-aws-bosh 41 | cd terraform-aws-bosh 42 | cp terraform.tfvars.example terraform.tfvars 43 | ``` 44 | 45 | Next, edit `terraform.tfvars` using your text editor and fill out the variables with your own values (AWS credentials, AWS region, etc). 46 | 47 | ### Deploy 48 | 49 | ```bash 50 | make plan 51 | make apply 52 | ``` 53 | 54 | The final output might look like: 55 | 56 | ``` 57 | Outputs: 58 | 59 | aws_internet_gateway_id = igw-6439f501 60 | aws_route_table_private_id = rtb-51e94a34 61 | aws_route_table_public_id = rtb-49e94a2c 62 | aws_subnet_bastion = subnet-1419a771 63 | aws_subnet_bastion_availability_zone = us-west-2a 64 | aws_vpc_id = vpc-b72581d2 65 | bastion_ip = 54.1.2.3 66 | ``` 67 | 68 | After Initial Install 69 | --------------------- 70 | 71 | At the end of the output of the terraform run, there will be a section called `Outputs` that will have at least `bastion_ip` and an IP address. If not, or if you cleared the terminal without noting it, you can log into the AWS console and look for an instance called `bastion`, with the `bastion` security group. Use the public IP associated with that instance, and ssh in as the ubuntu user, using the ssh key listed as `aws_key_path` in your configuration (if you used the Unattended Install). 72 | 73 | ``` 74 | ssh -i ~/.ssh/example.pem ubuntu@54.1.2.3 75 | ``` 76 | 77 | You can also access the "Outputs" from above using `terraform output`: 78 | 79 | ``` 80 | ssh ubuntu@$(terraform output bastion_ip) 81 | ``` 82 | 83 | Once inside you can access your BOSH. 84 | 85 | ``` 86 | $ bosh target 87 | Current target is https://10.10.1.4:25555 (vpc-b72581d2-keypair) 88 | 89 | $ bosh status 90 | Config 91 | /home/ubuntu/.bosh_config 92 | 93 | Director 94 | Name vpc-b72581d2-keypair 95 | URL https://10.10.1.4:25555 96 | ... 97 | ``` 98 | 99 | ### Cleanup / Tear down 100 | 101 | Terraform does not yet quite cleanup after itself. 102 | 103 | First, using the AWS Console you must manually delete all Instances (VMs). 104 | 105 | Second, run `make destroy`. 106 | 107 | Finally, run `make clean`. 108 | -------------------------------------------------------------------------------- /aws-bosh.tf: -------------------------------------------------------------------------------- 1 | provider "aws" { 2 | access_key = "${var.aws_access_key}" 3 | secret_key = "${var.aws_secret_key}" 4 | region = "${var.aws_region}" 5 | } 6 | 7 | module "vpc" { 8 | source = "github.com/cloudfoundry-community/terraform-aws-vpc" 9 | network = "${var.network}" 10 | aws_key_name = "${var.aws_key_name}" 11 | aws_access_key = "${var.aws_access_key}" 12 | aws_secret_key = "${var.aws_secret_key}" 13 | aws_region = "${var.aws_region}" 14 | aws_key_path = "${var.aws_key_path}" 15 | } 16 | 17 | output "aws_vpc_id" { 18 | value = "${module.vpc.aws_vpc_id}" 19 | } 20 | 21 | output "aws_internet_gateway_id" { 22 | value = "${module.vpc.aws_internet_gateway_id}" 23 | } 24 | 25 | output "aws_route_table_public_id" { 26 | value = "${module.vpc.aws_route_table_public_id}" 27 | } 28 | 29 | output "aws_route_table_private_id" { 30 | value = "${module.vpc.aws_route_table_private_id}" 31 | } 32 | 33 | output "aws_subnet_bastion" { 34 | value = "${module.vpc.bastion_subnet}" 35 | } 36 | 37 | output "aws_subnet_bastion_availability_zone" { 38 | value = "${module.vpc.aws_subnet_bastion_availability_zone}" 39 | } 40 | 41 | output "aws_key_path" { 42 | value = "${var.aws_key_path}" 43 | } 44 | 45 | resource "aws_instance" "bastion" { 46 | ami = "${lookup(var.aws_ubuntu_ami, var.aws_region)}" 47 | instance_type = "m3.medium" 48 | key_name = "${var.aws_key_name}" 49 | associate_public_ip_address = true 50 | security_groups = ["${module.vpc.aws_security_group_bastion_id}"] 51 | subnet_id = "${module.vpc.bastion_subnet}" 52 | 53 | ebs_block_device { 54 | device_name = "xvdc" 55 | volume_size = "40" 56 | } 57 | 58 | tags { 59 | Name = "bastion" 60 | } 61 | 62 | connection { 63 | user = "ubuntu" 64 | key_file = "${var.aws_key_path}" 65 | } 66 | 67 | provisioner "file" { 68 | source = "${path.module}/provision.sh" 69 | destination = "/home/ubuntu/provision.sh" 70 | } 71 | 72 | provisioner "remote-exec" { 73 | inline = [ 74 | "chmod +x /home/ubuntu/provision.sh", 75 | "/home/ubuntu/provision.sh ${var.aws_access_key} ${var.aws_secret_key} ${var.aws_region} ${module.vpc.aws_vpc_id} ${module.vpc.aws_subnet_microbosh_id} ${var.network} ${aws_instance.bastion.availability_zone} ${aws_instance.bastion.id} ${var.bosh_type}", 76 | ] 77 | } 78 | 79 | } 80 | 81 | output "bastion_ip" { 82 | value = "${aws_instance.bastion.public_ip}" 83 | } 84 | -------------------------------------------------------------------------------- /provision.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Variables passed in from terraform, see aws-vpc.tf, the "remote-exec" provisioner 4 | AWS_KEY_ID=${1} 5 | AWS_ACCESS_KEY=${2} 6 | REGION=${3} 7 | VPC=${4} 8 | BOSH_SUBNET=${5} 9 | IPMASK=${6} 10 | BASTION_AZ=${7} 11 | BASTION_ID=${8} 12 | BOSH_TYPE=${9} 13 | 14 | function log() { 15 | echo "--> $1" 16 | } 17 | 18 | # Prepare the jumpbox to be able to install ruby and git-based bosh and cf repos 19 | cd $HOME 20 | 21 | log "Installing dependencies" 22 | sudo apt-get update --fix-missing 23 | sudo apt-get install -y git unzip 24 | 25 | log "Installing RVM and Ruby 2.3.0" 26 | gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 27 | curl -sSL https://get.rvm.io | bash -s stable --ruby=2.3.0 28 | source /home/ubuntu/.rvm/scripts/rvm 29 | # Fix for RVM root .gem dir 30 | sudo chown ubuntu:ubuntu ~/.gem -R 31 | gem install bundler --no-rdoc --no-ri 32 | 33 | log "Generate the key that will be used to ssh between the inception server and the# microbosh machine" 34 | ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa 35 | 36 | log "Installing spiff" 37 | wget -qq https://github.com/cloudfoundry-incubator/spiff/releases/download/v1.0.7/spiff_linux_amd64.zip 38 | unzip -q -o spiff_linux_amd64.zip 39 | sudo mv spiff /usr/bin/ 40 | rm spiff_linux_amd64.zip 41 | 42 | # We use fog below, and bosh-bootstrap uses it as well 43 | log "Configuring Fog" 44 | cat < ~/.fog 45 | :default: 46 | :aws_access_key_id: $AWS_KEY_ID 47 | :aws_secret_access_key: $AWS_ACCESS_KEY 48 | :region: $REGION 49 | EOF 50 | 51 | # This volume is created using terraform in aws-bosh.tf 52 | log "Creating volume for workspace" 53 | sudo /sbin/mkfs.ext4 /dev/xvdc 54 | sudo /sbin/e2label /dev/xvdc workspace 55 | echo 'LABEL=workspace /home/ubuntu/workspace ext4 defaults,discard 0 0' | sudo tee -a /etc/fstab 56 | mkdir -p /home/ubuntu/workspace 57 | sudo mount -a 58 | sudo chown -R ubuntu:ubuntu /home/ubuntu/workspace 59 | 60 | # As long as we have a large volume to work with, we'll move /tmp over there 61 | # You can always use a bigger /tmp 62 | log "Move /tmp to workspace" 63 | sudo rsync -avq /tmp/ /home/ubuntu/workspace/tmp/ 64 | sudo rm -fR /tmp 65 | sudo ln -s /home/ubuntu/workspace/tmp /tmp 66 | 67 | # Install BOSH CLI, bosh-bootstrap, spiff and other helpful plugins/tools 68 | log "Installing BOSH CLI and bosh-bootstrap" 69 | gem install httpclient --version=2.7.1 --no-rdoc --no-ri 70 | gem install builder --version=3.1.4 --no-rdoc --no-ri 71 | gem install aws-sdk-v1 --version=1.60.2 --no-rdoc --no-ri 72 | gem install bosh_cli --no-ri --no-rdoc 73 | 74 | log "Cloning bosh-bootstrap" 75 | pushd workspace 76 | git clone https://github.com/cloudfoundry-community/bosh-bootstrap.git 77 | cd bosh-bootstrap 78 | gem install bundler 79 | bundle 80 | gem build bosh-bootstrap.gemspec 81 | gem install --local bosh-bootstrap*.gem 82 | popd 83 | 84 | # bosh-bootstrap handles provisioning the microbosh machine and installing bosh 85 | # on it. This is very nice of bosh-bootstrap. Everyone make sure to thank bosh-bootstrap 86 | log "Prepare deployment for bosh-bootstrap" 87 | mkdir -p {bin,workspace/deployments,workspace/tools,workspace/deployments/bosh-bootstrap} 88 | pushd workspace/deployments 89 | cat < settings.yml 90 | --- 91 | bosh: 92 | name: bosh-${VPC} 93 | provider: 94 | name: aws 95 | credentials: 96 | provider: AWS 97 | aws_access_key_id: ${AWS_KEY_ID} 98 | aws_secret_access_key: ${AWS_ACCESS_KEY} 99 | region: ${REGION} 100 | address: 101 | vpc_id: ${VPC} 102 | subnet_id: ${BOSH_SUBNET} 103 | ip: ${IPMASK}.1.4 104 | EOF 105 | 106 | if [[ "${BOSH_TYPE}" = "ruby" ]]; then 107 | 108 | log "Boostrap deploy" 109 | bosh bootstrap deploy 110 | 111 | # We've hardcoded the IP of the microbosh machine, because convenience 112 | log "Target the director" 113 | bosh -n target https://${IPMASK}.1.4:25555 114 | log "Login as admin" 115 | bosh login admin admin 116 | fi 117 | popd 118 | -------------------------------------------------------------------------------- /terraform.tfvars.example: -------------------------------------------------------------------------------- 1 | aws_access_key = "XXXXXXXXXXXXXX" 2 | aws_secret_key = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" 3 | aws_key_path = "~/.ssh/bosh.pem" 4 | aws_key_name = "bosh" 5 | aws_region = "us-east-1" 6 | network = "10.10" 7 | bosh_type = "ruby" # or "none"; soon "golang" 8 | -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | variable "aws_access_key" {} 2 | variable "aws_secret_key" {} 3 | variable "aws_key_path" {} 4 | variable "aws_key_name" {} 5 | variable "aws_region" { 6 | default = "us-west-2" 7 | } 8 | variable "network" { 9 | default = "10.10" 10 | } 11 | variable "cf_admin_pass" { 12 | default = "c1oudc0wc1oudc0w" 13 | } 14 | variable "bosh_type" { 15 | default = "ruby" 16 | } 17 | 18 | variable "aws_centos_ami" { 19 | default = { 20 | us-east-1 = "ami-00a11e68" 21 | us-west-1 = "ami-ba3c3bff" 22 | us-west-2 = "ami-3425be04" 23 | ap-northeast-1 = "ami-9392dc92" 24 | ap-southeast-1 = "ami-dcbeed8e" 25 | ap-southeast-2 = "ami-89e88db3" 26 | eu-west-1 = "ami-af6faad8" 27 | sa-east-1 = "ami-73ee416e" 28 | } 29 | } 30 | 31 | variable "aws_ubuntu_ami" { 32 | default = { 33 | us-east-1 = "ami-98aa1cf0" 34 | us-west-1 = "ami-736e6536" 35 | us-west-2 = "ami-37501207" 36 | ap-northeast-1 = "ami-df4b60de" 37 | ap-southeast-1 = "ami-2ce7c07e" 38 | ap-southeast-2 = "ami-1f117325" 39 | eu-west-1 = "ami-f6b11181" 40 | sa-east-1 = "ami-71d2676c" 41 | } 42 | } 43 | --------------------------------------------------------------------------------