├── .gitignore ├── Makefile ├── README.md ├── Vagrantfile ├── admin.openrc ├── boot-cirros.py ├── demo.openrc ├── devstack.yml ├── files └── keyringrc.cfg ├── requirements.txt ├── templates └── local.conf.j2 └── topology.png /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | vagrant_ansible_inventory_default 3 | devstack.retry 4 | venv 5 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: run 2 | 3 | run: 4 | vagrant destroy --force 5 | vagrant up 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Neutron-enabled DevStack in a Vagrant VM with Ansible 2 | 3 | This repository contains a Vagrantfile and an accompanying Ansible playbook 4 | that sets up a VirtualBox virtual machine that installs [DevStack][4]. 5 | 6 | You'll also be able to ssh directly from your laptop into the VMs without 7 | needing to the ssh into the Vagrant box first. 8 | 9 | Ansible generates a `local.conf` file that defaults to: 10 | 11 | * Use Neutron for networking 12 | * Disable security groups 13 | * No Application Catalog 14 | 15 | You can enable Swift, Heat, Application Catalog and security groups 16 | by editing the devstack.yml file. 17 | 18 | This project was inspired by Brian Waldon's [vagrant_devstack][1] repository. 19 | 20 | ## Memory usage 21 | 22 | By default, the VM uses 6GB of RAM and 2 cpus. If you want to change this, edit the 23 | following lines in Vagrantfile: 24 | 25 | vb.customize ["modifyvm", :id, "--memory", 6144] 26 | vb.customize ["modifyvm", :id, "--cpus", 2] 27 | 28 | ## Prereqs 29 | 30 | 31 | Install the following applications on your local machine first: 32 | 33 | * [VirtualBox][5] 34 | * [Vagrant][2] 35 | * [Ansible][3] 36 | 37 | If you want to try out the OpenStack command-line tools once DevStack is 38 | running, you'll also need to install the following Python packages: 39 | 40 | * python-novaclient 41 | * python-neutronclient 42 | * python-openstackclient 43 | 44 | The easiest way to install Ansible and the Python packages are with pip: 45 | 46 | sudo pip install -r requirements.txt 47 | 48 | ## Boot the virtual machine and install DevStack 49 | 50 | Grab this repo and do a `vagrant up`, like so: 51 | 52 | git clone https://github.com/lorin/devstack-vm 53 | cd devstack-vm 54 | vagrant up 55 | 56 | The `vagrant up` command will: 57 | 58 | 1. Download an Ubuntu 14.04 (trusty) vagrant box if it hasn't previously been downloaded to your machine. 59 | 2. Boot the virtual machine (VM). 60 | 3. Clone the DevStack git repository inside of the VM. 61 | 4. Run DevStack inside of the VM. 62 | 5. Add eth2 to the br-ex bridge inside of the VM to enable floating IP access from the host machine. 63 | 64 | It will take at least ten minutes for this to run, and possibly much longer depending on your internet connection and whether it needs to download the Ubuntu vagrant box. 65 | 66 | 67 | 68 | ## Troubleshooting 69 | 70 | ### Fails to connect 71 | 72 | You may ocassionally see the following error message: 73 | 74 | ``` 75 | [default] Waiting for VM to boot. This can take a few minutes. 76 | [default] Failed to connect to VM! 77 | Failed to connect to VM via SSH. Please verify the VM successfully booted 78 | by looking at the VirtualBox GUI. 79 | ``` 80 | 81 | If you see this, retry by doing: 82 | 83 | vagrant destroy --force && vagrant up 84 | 85 | 86 | ## Logging in the virtual machine 87 | 88 | The VM is accessible at 192.168.27.100 89 | 90 | You can type `vagrant ssh` to start an ssh session. 91 | 92 | Note that you do not need to be logged in to the VM to run commands against the OpenStack endpoint. 93 | 94 | 95 | ## Loading OpenStack credentials 96 | 97 | From your local machine, to run as the demo user: 98 | 99 | source demo.openrc 100 | 101 | To run as the admin user: 102 | 103 | source admin.openrc 104 | 105 | ## Horizon 106 | 107 | * URL: http://192.168.27.100 108 | * Username: admin or demo 109 | * Password: password 110 | 111 | 112 | ## Initial networking configuration 113 | 114 | ![Network topology](topology.png) 115 | 116 | 117 | DevStack configures an internal network ("private") and an external network ("public"), with a router ("router1") connecting the two together. The router is configured to use its interface on the "public" network as the gateway. 118 | 119 | ``` 120 | $ openstack network list 121 | +--------------------------------------+---------+------------------------------------------------------------------------+ 122 | | ID | Name | Subnets | 123 | +--------------------------------------+---------+------------------------------------------------------------------------+ 124 | | 3d910901-12a0-4997-8335-948c66e1ab46 | public | 1c458c90-3bd3-45b1-a9bf-6ed8cd56e128, | 125 | | | | 94f2f87c-c8a4-48e5-a27c-752e7be14988 | 126 | | c83dc6a9-615e-4a42-b462-b5d9871a923f | private | 6e58ab8b-bc1a-4ae8-9233-f2d69a5c1821, | 127 | | | | 830a36ce-4bb4-4266-8411-5d4447e8e2e3 | 128 | +--------------------------------------+---------+------------------------------------------------------------------------+ 129 | 130 | $ neutron router-list 131 | +--------------------------------------+---------+------------------------------------------------------------------------+ 132 | | id | name | external_gateway_info | 133 | +--------------------------------------+---------+------------------------------------------------------------------------+ 134 | | c182627f-2c78-4f0e-aa14-f740aa7a02d3 | router1 | {"network_id": "3d910901-12a0-4997-8335-948c66e1ab46", | 135 | | | | "external_fixed_ips": [{"ip_address": "172.24.4.2", "subnet_id": | 136 | | | | "1c458c90-3bd3-45b1-a9bf-6ed8cd56e128"}, {"ip_address": "2001:db8::1", | 137 | | | | "subnet_id": "94f2f87c-c8a4-48e5-a27c-752e7be14988"}], "enable_snat": | 138 | | | | true} | 139 | +--------------------------------------+---------+------------------------------------------------------------------------+ 140 | ``` 141 | 142 | 143 | ## Add ssh and ping to the default security group 144 | 145 | openstack security group rule create default --proto tcp --dst-port 22 146 | openstack security group rule create default --proto icmp 147 | 148 | 149 | ## Launch a cirros instance and attach a floating IP. 150 | 151 | Source the credentials of the "demo" user and boot an instance. 152 | 153 | source demo.openrc 154 | nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey 155 | nova boot --flavor m1.tiny --image cirros-0.3.4-x86_64-uec --key-name mykey cirros 156 | 157 | Once the instance has booted, get its ID. 158 | 159 | $ nova list 160 | 161 | +--------------------------------------+--------+--------+------------+-------------+------------------------------------------------------+ 162 | | ID | Name | Status | Task State | Power State | Networks | 163 | +--------------------------------------+--------+--------+------------+-------------+------------------------------------------------------+ 164 | | 62cf0635-aa9e-4223-bbcd-3808966959c1 | cirros | ACTIVE | - | Running | private=fdbc:59ac:894:0:f816:3eff:fefe:221, 10.0.0.3 | 165 | +--------------------------------------+--------+--------+------------+-------------+------------------------------------------------------+ 166 | 167 | Use the instance ID to get its neutron port : 168 | 169 | $ neutron port-list -c id --device_id b24fc4ad-2d66-4f28-928b-f1cf78075d33 170 | 171 | +--------------------------------------+ 172 | | id | 173 | +--------------------------------------+ 174 | | 02491b08-919e-4582-9eb7-f8119c03b8f9 | 175 | +--------------------------------------+ 176 | 177 | 178 | Use the neutron port ID to create an attach a floating IP to the "public"" network: 179 | 180 | $ neutron floatingip-create public --port-id 02491b08-919e-4582-9eb7-f8119c03b8f9 181 | 182 | Created a new floatingip: 183 | +---------------------+--------------------------------------+ 184 | | Field | Value | 185 | +---------------------+--------------------------------------+ 186 | | fixed_ip_address | 10.0.0.3 | 187 | | floating_ip_address | 172.24.4.227 | 188 | | floating_network_id | 5770a693-cfc7-431d-ae29-76f36a2e63c0 | 189 | | id | 480524e1-a5b3-491f-a6ee-9356fc52f81d | 190 | | port_id | 02491b08-919e-4582-9eb7-f8119c03b8f9 | 191 | | router_id | 0deb0811-78b0-415c-9464-f05d278e9e3d | 192 | | tenant_id | 512e45b937a149d283718ffcfc36b8c7 | 193 | +---------------------+--------------------------------------+ 194 | 195 | Finally, access your instance: 196 | 197 | ssh cirros@172.24.4.227 198 | 199 | 200 | ## Python bindings example 201 | 202 | The included `boot-cirros.py` file illustrates how to execute all of the 203 | above commands using the Python bindings. 204 | 205 | ## Allow VMs to connect out to the Internet 206 | 207 | By default, VMs started by OpenStack will not be able to connect to the 208 | internet. For this to work, your host machine must be configured to do NAT 209 | (Network Address Translation) for the VMs. 210 | 211 | ### On Mac OS X 212 | 213 | #### Enable IP forwarding 214 | 215 | Turn on IP forwarding if it isn't on yet: 216 | 217 | sudo sysctl -w net.inet.ip.forwarding=1 218 | 219 | Note that you have to do this each time you reboot. 220 | 221 | #### Edit the pfctl config file to NAT the floating IP subnet 222 | 223 | Edit `/etc/pf.conf` as root, and add the following line after the "net-anchor" line: 224 | 225 | nat on en0 from 172.24.4.1/24 -> (en0) 226 | 227 | #### Load the file and enable PF 228 | 229 | sudo pfctl -f /etc/pf.conf 230 | sudo pfctl -e 231 | 232 | (From [Martin Nash's blog][6]. See info there on how to make the IP forwarding 233 | persist across reboots ). 234 | 235 | 236 | ### On Linux 237 | 238 | To enable NAT, issue the following commands in your host, as root: 239 | 240 | ``` 241 | echo 1 > /proc/sys/net/ipv4/ip_forward 242 | iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE 243 | ``` 244 | 245 | [1]: https://github.com/bcwaldon/vagrant_devstack 246 | [2]: http://vagrantup.com 247 | [3]: http://ansible.com 248 | [4]: http://devstack.org 249 | [5]: http://virtualbox.org 250 | [6]: http://blog.nasmart.me/internet-access-with-virtualbox-host-only-networks-on-os-x-mavericks/ 251 | 252 | ## Troubleshooting 253 | 254 | Logs are in `/opt/stack/logs` 255 | 256 | ### Instance immediately goes into error state 257 | 258 | Check the nova-conductor log and search for ERROR 259 | 260 | ``` 261 | vagrant ssh 262 | less -R /opt/stack/logs/n-cond.log 263 | ``` 264 | 265 | For example, if it's failing because there isn't enough free memory in the 266 | virtual machine, you'll see an error like this: 267 | 268 | ``` 269 | 2016-08-01 05:42:50.237 ERROR nova.scheduler.utils [req-581add06-ba33-4b5d-9a1b-af7c74f3ce86 demo demo] [instance: 70713d2f-96fa-4ee7-a73a-4e019b78b1f9] Error from last host: vagrant-ubuntu-trusty-64 (node vagrant-ubuntu-trusty-64): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1926, in _do_build_and_run_instance\n filter_properties)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 2116, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 70713d2f-96fa-4ee7-a73a-4e019b78b1f9 was re-scheduled: internal error: process exited while connecting to monitor: Cannot set up guest memory 'pc.ram': Cannot allocate memory\n\n"] 270 | ``` 271 | 272 | 273 | 274 | 275 | 276 | 277 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | 5 | Vagrant.configure("2") do |config| 6 | 7 | config.vm.box = "ubuntu/trusty64" 8 | config.ssh.forward_agent = true 9 | # eth1, this will be the endpoint 10 | config.vm.network :private_network, ip: "192.168.27.100" 11 | # eth2, this will be the OpenStack "public" network 12 | # ip and subnet mask should match floating_ip_range var in devstack.yml 13 | config.vm.network :private_network, ip: "172.24.4.225", :netmask => "255.255.255.0", :auto_config => false 14 | config.vm.provider :virtualbox do |vb| 15 | vb.customize ["modifyvm", :id, "--memory", 6144] 16 | vb.customize ["modifyvm", :id, "--cpus", 2] 17 | # eth2 must be in promiscuous mode for floating IPs to be accessible 18 | vb.customize ["modifyvm", :id, "--nicpromisc3", "allow-all"] 19 | end 20 | config.vm.provision :ansible do |ansible| 21 | ansible.host_key_checking = false 22 | ansible.playbook = "devstack.yml" 23 | ansible.verbose = "v" 24 | end 25 | config.vm.provision :shell, :inline => "cd devstack; sudo -u vagrant env HOME=/home/vagrant ./stack.sh" 26 | config.vm.provision :shell, :inline => "ovs-vsctl add-port br-ex eth2" 27 | config.vm.provision :shell, :inline => "virsh net-destroy default" 28 | 29 | end 30 | -------------------------------------------------------------------------------- /admin.openrc: -------------------------------------------------------------------------------- 1 | export OS_USERNAME="admin" 2 | export OS_PASSWORD="password" 3 | export OS_TENANT_NAME="demo" 4 | export OS_AUTH_URL="http://192.168.27.100:5000/v2.0" 5 | export OS_REGION_NAME="" 6 | -------------------------------------------------------------------------------- /boot-cirros.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python -u 2 | ''' 3 | 4 | This script does the following 5 | 6 | 1. Connect the router to the public network 7 | 2. Add a public key 8 | 3. Boot a cirros instance 9 | 4. Attach a floating IP 10 | 11 | 12 | ''' 13 | from __future__ import print_function 14 | 15 | import datetime 16 | import os.path 17 | import socket 18 | import sys 19 | import time 20 | 21 | from novaclient import client as novaclient 22 | from neutronclient.v2_0 import client as neutronclient 23 | 24 | 25 | auth_url = "http://192.168.27.100:35357/v2.0" 26 | username = "demo" 27 | password = "password" 28 | tenant_name = "demo" 29 | version = 2 30 | 31 | neutron = neutronclient.Client(auth_url=auth_url, 32 | username=username, 33 | password=password, 34 | tenant_name=tenant_name) 35 | 36 | nova = novaclient.Client(version=version, 37 | auth_url=auth_url, 38 | username=username, 39 | api_key=password, 40 | project_id=tenant_name) 41 | 42 | 43 | if not nova.keypairs.findall(name="mykey"): 44 | print("Creating keypair: mykey...") 45 | with open(os.path.expanduser('~/.ssh/id_rsa.pub')) as fpubkey: 46 | nova.keypairs.create(name="mykey", public_key=fpubkey.read()) 47 | print("done") 48 | 49 | print("Booting cirros instance...", end='') 50 | image = nova.images.find(name="cirros-0.3.4-x86_64-uec") 51 | flavor = nova.flavors.find(name="m1.tiny") 52 | instance = nova.servers.create(name="cirros", image=image, flavor=flavor, 53 | key_name="mykey") 54 | 55 | # Poll at 5 second intervals, until the status is no longer 'BUILD' 56 | status = instance.status 57 | while status == 'BUILD': 58 | time.sleep(5) 59 | # Retrieve the instance again so the status field updates 60 | instance = nova.servers.get(instance.id) 61 | status = instance.status 62 | print("done") 63 | 64 | 65 | print("Creating floating ip...", end='') 66 | # Get external network 67 | ext_net, = [x for x in neutron.list_networks()['networks'] 68 | if x['router:external']] 69 | 70 | # Get the port corresponding to the instance 71 | port, = [x for x in neutron.list_ports()['ports'] 72 | if x['device_id'] == instance.id] 73 | 74 | # Create the floating ip 75 | args = dict(floating_network_id=ext_net['id'], 76 | port_id=port['id']) 77 | ip_obj = neutron.create_floatingip(body={'floatingip': args}) 78 | print("done") 79 | 80 | ip = ip_obj['floatingip']['floating_ip_address'] 81 | print("IP:{}".format(ip)) 82 | 83 | print("Waiting for ssh to be ready on cirros instance...", end='') 84 | start = datetime.datetime.now() 85 | timeout = 120 86 | end = start + datetime.timedelta(seconds=timeout) 87 | port = 22 88 | connect_timeout = 5 89 | # From utilities/wait_for of ansible 90 | while datetime.datetime.now() < end: 91 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 92 | s.settimeout(connect_timeout) 93 | try: 94 | s.connect((ip, port)) 95 | s.shutdown(socket.SHUT_RDWR) 96 | s.close() 97 | print() 98 | break 99 | except: 100 | time.sleep(1) 101 | pass 102 | else: 103 | print("ssh server never came up!") 104 | sys.exit(1) 105 | -------------------------------------------------------------------------------- /demo.openrc: -------------------------------------------------------------------------------- 1 | export OS_USERNAME="demo" 2 | export OS_PASSWORD="password" 3 | export OS_TENANT_NAME="demo" 4 | export OS_AUTH_URL="http://192.168.27.100:5000/v2.0" 5 | export OS_REGION_NAME="" 6 | -------------------------------------------------------------------------------- /devstack.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - hosts: default 3 | vars: 4 | devstack_repo: git://github.com/openstack-dev/devstack.git 5 | public_ip: 192.168.27.100 6 | version: stable/mitaka 7 | floating_ip_range: 172.24.4.0/24 8 | heat: False 9 | neutron: True 10 | neutron_lbaasv2: False 11 | swift: False 12 | security_groups: False 13 | tempest: False 14 | keyring_config_dir: /home/vagrant/.local/share/python_keyring 15 | app_catalog: False 16 | 17 | tasks: 18 | - name: create keyring config directory 19 | file: > 20 | state=directory 21 | dest={{ keyring_config_dir }} 22 | 23 | - name: copy keyring config file to workaround https://bugs.launchpad.net/bugs/1242992 24 | copy: > 25 | src=files/keyringrc.cfg 26 | dest={{ keyring_config_dir }}/keyringrc.cfg 27 | 28 | - name: install git 29 | apt: "name=git update_cache=yes" 30 | become: True 31 | 32 | - name: checkout devstack 33 | git: repo={{ devstack_repo }} dest=/home/vagrant/devstack version={{ version }} accept_hostkey=yes 34 | 35 | - name: local.conf 36 | template: src=templates/local.conf.j2 dest=/home/vagrant/devstack/local.conf 37 | 38 | - name: source openrc in profile 39 | lineinfile: dest=/home/vagrant/.profile regexp=".*openrc" line='. /home/vagrant/devstack/openrc' 40 | 41 | - name: enable eth2 42 | command: ip link set dev eth2 up 43 | become: True 44 | 45 | -------------------------------------------------------------------------------- /files/keyringrc.cfg: -------------------------------------------------------------------------------- 1 | [backend] 2 | default-keyring=keyring.backends.file.PlaintextKeyring 3 | 4 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | ansible 2 | python-novaclient 3 | python-neutronclient 4 | python-openstackclient 5 | -------------------------------------------------------------------------------- /templates/local.conf.j2: -------------------------------------------------------------------------------- 1 | [[local|localrc]] 2 | # Default passwords 3 | ADMIN_PASSWORD=password 4 | MYSQL_PASSWORD=password 5 | RABBIT_PASSWORD=password 6 | SERVICE_PASSWORD=password 7 | SERVICE_TOKEN=password 8 | 9 | 10 | SCREEN_LOGDIR=/opt/stack/logs 11 | 12 | 13 | HOST_IP={{ public_ip }} 14 | 15 | FLOATING_RANGE={{ floating_ip_range }} 16 | 17 | {% if neutron %} 18 | # 19 | # Enable Neutron 20 | # 21 | # https://wiki.openstack.org/wiki/NeutronDevstack 22 | disable_service n-net 23 | enable_service q-svc 24 | enable_service q-agt 25 | enable_service q-dhcp 26 | enable_service q-l3 27 | enable_service q-meta 28 | enable_service neutron 29 | 30 | {% if neutron_lbaasv2 %} 31 | # 32 | # Enable Neutron LBaas v2 33 | # 34 | # https://wiki.openstack.org/wiki/Neutron/LBaaS 35 | enable_plugin neutron-lbaas https://git.openstack.org/openstack/neutron-lbaas {{ version }} 36 | enable_plugin barbican https://git.openstack.org/openstack/barbican {{ version }} 37 | enable_plugin octavia https://git.openstack.org/openstack/octavia {{ version }} 38 | ENABLED_SERVICES+=,q-lbaasv2 39 | ENABLED_SERVICES+=,octavia,o-cw,o-hk,o-hm,o-api 40 | # Enable LBaaS v2 dashboard 41 | enable_plugin neutron-lbaas-dashboard https://git.openstack.org/openstack/neutron-lbaas-dashboard {{ version }} 42 | {% endif %} 43 | 44 | {% endif %} 45 | 46 | {% if swift %} 47 | # Enable Swift 48 | enable_service s-proxy 49 | enable_service s-object 50 | enable_service s-container 51 | enable_service s-account 52 | {% endif %} 53 | 54 | {% if heat %} 55 | enable_service heat 56 | enable_service h-api 57 | enable_service h-api-cfn 58 | enable_service h-api-cw 59 | enable_service h-eng 60 | {% endif %} 61 | 62 | {% if not security_groups %} 63 | # Disable security groups entirely 64 | Q_USE_SECGROUP=False 65 | LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver 66 | {% endif %} 67 | 68 | {% if not tempest %} 69 | disable_service tempest 70 | {% endif %} 71 | 72 | {% if app_catalog %} 73 | enable_plugin app-catalog-ui https://git.openstack.org/openstack/app-catalog-ui 74 | {% endif %} 75 | -------------------------------------------------------------------------------- /topology.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lorin/devstack-vm/994ac6486a0f300cc5d1b88ab0ceac9e7ba03c8f/topology.png --------------------------------------------------------------------------------