├── .gitignore ├── LICENSE ├── README.md ├── dockenstack-tempest └── Dockerfile ├── dockenstack ├── Dockerfile ├── devstack.sudo ├── localrc ├── localrc.d │ ├── 050-services │ └── 099-tempest └── scripts │ ├── apt-cache-devstack │ ├── devstack-init │ ├── openstack-git-checkout │ ├── start │ ├── tempest │ └── wrapdocker ├── docker-compose.yml └── fig.yml /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2013 Paul Czarkowski 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Openstack on Docker 2 | 3 | Dockenstack builds an image for running OpenStack's devstack development and testing environment inside of a Docker container. This image currently supports running the docker and libvirt-lxc virtualization drivers for Nova. KVM/Qemu support is being tested. 4 | 5 | Using dockenstack, developers may quickly iterate changes in a container and locally invoke functional tests without needing to first submit their changes for code-review. 6 | 7 | The quick iteration cycle of dockenstack versus other local environments (such as devstack-vagrant) is accomplished by precaching and preinstalling most or all network resources and OS packages. This speeds up running the container and, when running many, eliminates the problems that might result from offline or rate-limited apt and pip services. 8 | 9 | Users may expect dockenstack to take 2-4 minutes on a fast machine from "docker run" through having an operational OpenStack installation. 10 | 11 | # Build & Run 12 | 13 | ## Quickstart: Using Docker Compose 14 | 15 | ``` 16 | $ git clone https://github.com/ewindisch/dockenstack.git 17 | $ cd dockenstack 18 | $ docker-compose up 19 | ``` 20 | 21 | This will automatically build a Dockenstack image and run OpenStack. 22 | 23 | The first run will take a long time due to the length process of 24 | building the Docker image (~60m). Subsequent runs of this image will be 25 | quicker (~5m). Even faster, of course, is restarting a container. 26 | 27 | ## Alternative Install: Building Manually 28 | 29 | The following is the process undertaken by Docker Compose. 30 | Building the image may take approximately 60 minutes. 31 | 32 | ``` 33 | git clone https://github.com/ewindisch/dockenstack.git 34 | cd dockenstack 35 | docker build -t ewindisch/dockenstack dockenstack 36 | docker build -t ewindisch/dockenstack-tempest dockenstack-tempest 37 | docker run --privileged -t -i ewindisch/dockenstack 38 | ``` 39 | 40 | # Using OpenStack 41 | 42 | If you've started dockenstack interactively without extra arguments, you'll end up with a shell and can run these steps immediately. 43 | 44 | ``` 45 | source /devstack/openrc 46 | nova boot --image busybox --flavor 1 test 47 | nova list 48 | docker ps 49 | ``` 50 | 51 | A future version of this README will explain how to use the OpenStack installation from outside of the dockenstack container. 52 | 53 | # Running Tempest 54 | 55 | Launch the container as such: 56 | 57 | ``` 58 | docker run --privileged -t -i ewindisch/dockenstack-tempest 59 | ``` 60 | 61 | Running Tempest in Dockenstack may take approximately 30 minutes. 62 | 63 | Arguments to run-tempest may be passed, the arguments are the same as run_tempest.sh (see Tempest documentation / source) 64 | 65 | # Configuration 66 | 67 | Dockenstack should understand all of the devstack environment variables 68 | passed as enviroment variables to 'docker run'. If using Docker Compose, 69 | these environment variables may be added to the fig.yml file. 70 | 71 | # Notes 72 | 73 | * Requires Docker 1.5 or later. 74 | * AUFS / Volumes - Using AUFS and nested-docker, one may need to mount /var/lib/docker as a volume or a bind-mount. (pass '-v /var/lib/docker' to 'docker run') 75 | * Libvirt guests may need kernel modules loaded. Libvirt/Qemu support is neither tested nor complete. 76 | 77 | # Authors 78 | 79 | * Eric Windisch 80 | * Paul Czarkowski 81 | 82 | # License 83 | 84 | Apache2 - see `LICENSE` 85 | -------------------------------------------------------------------------------- /dockenstack-tempest/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ewindisch/dockenstack 2 | MAINTAINER Eric Windisch "ewindisch@docker.com" 3 | 4 | ENTRYPOINT ["/opt/dockenstack/bin/tempest"] 5 | -------------------------------------------------------------------------------- /dockenstack/Dockerfile: -------------------------------------------------------------------------------- 1 | # Eric Windisch '2014 2 | # Forked from code by original author: Paul Czarkowski 3 | 4 | FROM ubuntu:trusty 5 | MAINTAINER Eric Windisch "ewindisch@docker.com" 6 | 7 | EXPOSE 80 5000 8773 8774 8776 9292 8 | # Set DEBIAN_FRONTEND to avoid warning like "debconf: (TERM is not set, so the dialog frontend is not usable.)" 9 | ENV DEBIAN_FRONTEND="noninteractive" 10 | # Install Docker 11 | RUN apt-get update; apt-get install -qqy apt-transport-https; apt-key adv --keyserver "hkp://keyserver.ubuntu.com:80" --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9; \ 12 | echo 'deb http://get.docker.io/ubuntu docker main' > /etc/apt/sources.list.d/docker.list; \ 13 | apt-get update; \ 14 | apt-get install -qqy lxc-docker 15 | 16 | # Install utilities 17 | RUN apt-get -qqy install git socat curl sudo vim wget net-tools 18 | 19 | # Install apparmor 20 | RUN apt-get -qqy install apparmor 21 | 22 | # Extra requirements for pip-requirements 23 | RUN apt-get install -qqy libffi-dev libkrb5-dev libev-dev libvirt-dev libsqlite3-dev libxml2-dev libxslt-dev \ 24 | libpq-dev libssl-dev libyaml-dev 25 | 26 | 27 | # Configure and install MySQL 28 | RUN echo 'mysql-server mysql-server/root_password password devstack' | debconf-set-selections; \ 29 | echo 'mysql-server mysql-server/root_password_again password devstack' | debconf-set-selections; \ 30 | apt-get -qqy install mysql-server 31 | 32 | # Install RabbitMQ 33 | RUN apt-get -qqy install rabbitmq-server 34 | 35 | # Copy in docker images 36 | RUN docker daemon -b none -s vfs & sleep 1; docker pull cirros 37 | 38 | # Setup devstack user 39 | RUN mkdir -p /opt; \ 40 | useradd -m -s /bin/bash -d /opt/stack devstack && \ 41 | usermod -a -G docker devstack 42 | ADD devstack.sudo /etc/sudoers.d/devstack 43 | RUN chown root:root /etc/sudoers.d/devstack 44 | 45 | # Local scripts 46 | ADD scripts /opt/dockenstack/bin 47 | RUN chmod 755 /opt/dockenstack/bin/* 48 | 49 | # Install devstack scripts 50 | RUN git clone https://github.com/openstack-dev/devstack /devstack 51 | 52 | # Install prereq packages. 53 | RUN /devstack/tools/install_prereqs.sh 54 | 55 | # Pre-download all "NOPRIME" packages 56 | RUN /bin/bash /opt/dockenstack/bin/apt-cache-devstack 57 | 58 | # Install/configure dbus for libvirt (only needed if using libvirt drivers, not docker) 59 | # creates symlink to /usr/bin due to bug in saucy's init script for dbus. 60 | RUN apt-get install -q -y dbus; \ 61 | ln -s /bin/dbus-daemon /usr/bin/dbus-daemon 62 | 63 | # Mask python-six because the apt package can conflict with the pypi version. 64 | RUN apt-get remove -q -y python-six 65 | 66 | # python-pip just became collatoral damage. reinstall it. 67 | RUN apt-get install -q -y python-pip 68 | 69 | # Install six from pip before getting from global-requirements (due to bugs...) 70 | RUN pip install six 71 | 72 | # Install all pip requirements 73 | RUN pip install -r https://raw.github.com/openstack/requirements/master/global-requirements.txt 74 | RUN pip install -r https://raw.github.com/openstack/tempest/master/requirements.txt 75 | 76 | # Pre-checkout git repos 77 | RUN su devstack -c '/bin/bash /opt/dockenstack/bin/openstack-git-checkout' 78 | 79 | RUN git clone https://github.com/stackforge/nova-docker /opt/stack/nova-docker 80 | WORKDIR /opt/stack/nova-docker 81 | RUN cp contrib/devstack/extras.d/70-docker.sh /devstack/extras.d/; \ 82 | cp contrib/devstack/lib/nova_plugins/hypervisor-docker /devstack/lib/nova_plugins/; 83 | 84 | WORKDIR /devstack 85 | ADD localrc /devstack/localrc 86 | ADD localrc.d /devstack/localrc.d 87 | 88 | WORKDIR / 89 | # Fix ownership of all files 90 | RUN chown -R devstack /devstack 91 | 92 | CMD ["/opt/dockenstack/bin/start"] 93 | -------------------------------------------------------------------------------- /dockenstack/devstack.sudo: -------------------------------------------------------------------------------- 1 | # devstack sudoers rule 2 | devstack ALL=(ALL) NOPASSWD: ALL 3 | -------------------------------------------------------------------------------- /dockenstack/localrc: -------------------------------------------------------------------------------- 1 | export PATH="$PATH:/sbin" 2 | 3 | DATABASE_PASSWORD=devstack 4 | RABBIT_PASSWORD=devstack 5 | SERVICE_TOKEN=devstack 6 | SERVICE_PASSWORD=devstack 7 | ADMIN_PASSWORD=devstack 8 | 9 | LOGFILE=/opt/stack/logs/stack.sh.log 10 | VERBOSE=True 11 | LOG_COLOR=True 12 | SCREEN_LOGDIR=/opt/stack/logs 13 | 14 | # Workaround bug in devstack that 15 | # prefers 'lo' over better options such as eth0. 16 | # 17 | # There also happens to be a bug with the docker driver 18 | # and the use of 127.0.0.1: 19 | # http://lists.openstack.org/pipermail/openstack-dev/2014-January/024383.html 20 | HOST_IP_IFACE=eth0 21 | 22 | IMAGE_URLS= 23 | DEFAULT_IMAGE_NAME=cirros 24 | 25 | VIRT_DRIVER=docker 26 | 27 | # Include ".d" directory 28 | for script in /devstack/localrc.d/*; do 29 | . $script 30 | done 31 | -------------------------------------------------------------------------------- /dockenstack/localrc.d/050-services: -------------------------------------------------------------------------------- 1 | disable_service horizon 2 | disable_service n-novnc 3 | disable_service n-xvnc 4 | disable_service n-spice 5 | 6 | disable_service cinder 7 | disable_service c-sch 8 | disable_service c-api 9 | disable_service c-vol 10 | disable_service c-bak 11 | -------------------------------------------------------------------------------- /dockenstack/localrc.d/099-tempest: -------------------------------------------------------------------------------- 1 | # Extracted from devstack-gate 2 | ACTIVE_TIMEOUT=90 3 | BOOT_TIMEOUT=90 4 | ASSOCIATE_TIMEOUT=60 5 | TERMINATE_TIMEOUT=60 6 | SWIFT_HASH=1234123412341234 7 | ROOTSLEEP=0 8 | #ERROR_ON_CLONE=True 9 | #SERVICE_HOST=127.0.0.1 10 | # Screen console logs will capture service logs. 11 | SYSLOG=False 12 | VERBOSE=True 13 | FIXED_RANGE=10.1.0.0/24 14 | FIXED_NETWORK_SIZE=256 15 | SWIFT_REPLICAS=1 16 | LOG_COLOR=False 17 | #PIP_USE_MIRRORS=False 18 | #USE_GET_PIP=1 19 | # Don't reset the requirements.txt files after g-r updates 20 | UNDO_REQUIREMENTS=False 21 | CINDER_PERIODIC_INTERVAL=10 22 | export OS_NO_CACHE=True 23 | 24 | RECLONE=yes 25 | -------------------------------------------------------------------------------- /dockenstack/scripts/apt-cache-devstack: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | pushd /devstack 3 | apt-get install -qqyd $(grep -h NOPRIME files/apts/* | sed '/dist/d; s/\s*#.*//;') 4 | popd 5 | -------------------------------------------------------------------------------- /dockenstack/scripts/devstack-init: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | 3 | mysqld_safe & 4 | 5 | service rabbitmq-server start 6 | 7 | /opt/dockenstack/bin/wrapdocker 8 | 9 | # Matches the fixed range in the devstack localrc 10 | ifconfig eth0:1 inet 10.1.0.1/24 11 | 12 | if [ -n "$STABLE_RELEASE" ]; then 13 | # Sets the branch for each project, if tagging a stable release. 14 | # This regex sanitizes our input... 15 | STABLE_RELEASE=${STABLE_RELEASE//[^a-zA-Z0-9_]/} 16 | sed -n "/_BRANCH=/{ s/=.*/=stable\/${STABLE_RELEASE}/g; p; }" /devstack/stackrc >> /devstack/localrc.d/001-stable-branch 17 | 18 | cd /devstack 19 | git checkout stable/${STABLE_RELEASE} 20 | fi 21 | 22 | ## Git repos 23 | #cat <>/devstack/localrc.d/005-nova-git 24 | ## Base GIT Repo URL 25 | ## Another option is http://review.openstack.org/p 26 | #GIT_BASE=${GIT_BASE:-git://git.openstack.org} 27 | ## compute service 28 | #NOVA_REPO=${NOVA_REPO:-${GIT_BASE}/openstack/nova.git} 29 | #NOVA_BRANCH=${NOVA_BRANCH:-master} 30 | #EOF 31 | 32 | function docker_has_image { 33 | test $(docker images -q $1 | wc -l) -gt 0 34 | } 35 | 36 | # Some of these tags are to support differing versions of 37 | # devstack... 38 | # Icehouse-3 39 | if ! ( docker_has_image cirros ); then 40 | docker pull cirros 41 | fi 42 | 43 | su devstack -c '/devstack/stack.sh' 44 | -------------------------------------------------------------------------------- /dockenstack/scripts/openstack-git-checkout: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Embarrassingly bad shell scripting below (because one 'eval' isn't enough!) 3 | cd /devstack 4 | . functions 5 | . stackrc 6 | 7 | # Workaround https://bugs.launchpad.net/tripleo/+bug/1285954 8 | BM_POSEUR_REPO=https://github.com/tripleo/bm_poseur 9 | BM_POSEUR_DIR=/opt/stack/bm_poseur 10 | 11 | cat <(echo ". functions; . stackrc") <(for project in $(declare | sed -n '/_REPO=/{ s/^\(.*\)_REPO=.*/\1/; p; }'); do eval "echo git_clone \$${project}_REPO $(git grep -h ${project}_DIR= | sed 's/.*=//') \$${project}_BRANCH \|\| :"; done | grep git.openstack.org) | bash 12 | -------------------------------------------------------------------------------- /dockenstack/scripts/start: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | /opt/dockenstack/bin/devstack-init 3 | bash 4 | -------------------------------------------------------------------------------- /dockenstack/scripts/tempest: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | /opt/dockenstack/bin/devstack-init && 3 | su devstack -c "/opt/stack/tempest/run_tempest.sh -N $@" 4 | -------------------------------------------------------------------------------- /dockenstack/scripts/wrapdocker: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # First, make sure that cgroups are mounted correctly. 4 | CGROUP=/sys/fs/cgroup 5 | 6 | [ -d $CGROUP ] || 7 | mkdir $CGROUP 8 | 9 | mountpoint -q $CGROUP || 10 | mount -n -t tmpfs -o uid=0,gid=0,mode=0755 cgroup $CGROUP || { 11 | echo "Could not make a tmpfs mount. Did you use -privileged?" 12 | exit 1 13 | } 14 | 15 | if [ -d /sys/kernel/security ] && ! mountpoint -q /sys/kernel/security 16 | then 17 | mount -t securityfs none /sys/kernel/security || { 18 | echo "Could not mount /sys/kernel/security." 19 | echo "AppArmor detection and -privileged mode might break." 20 | } 21 | fi 22 | 23 | # Mount the cgroup hierarchies exactly as they are in the parent system. 24 | for SUBSYS in $(cut -d: -f2 /proc/1/cgroup) 25 | do 26 | [ -d $CGROUP/$SUBSYS ] || mkdir $CGROUP/$SUBSYS 27 | mountpoint -q $CGROUP/$SUBSYS || 28 | mount -n -t cgroup -o $SUBSYS cgroup $CGROUP/$SUBSYS 29 | 30 | # The two following sections address a bug which manifests itself 31 | # by a cryptic "lxc-start: no ns_cgroup option specified" when 32 | # trying to start containers withina container. 33 | # The bug seems to appear when the cgroup hierarchies are not 34 | # mounted on the exact same directories in the host, and in the 35 | # container. 36 | 37 | # Named, control-less cgroups are mounted with "-o name=foo" 38 | # (and appear as such under /proc//cgroup) but are usually 39 | # mounted on a directory named "foo" (without the "name=" prefix). 40 | # Systemd and OpenRC (and possibly others) both create such a 41 | # cgroup. To avoid the aforementioned bug, we symlink "foo" to 42 | # "name=foo". This shouldn't have any adverse effect. 43 | echo $SUBSYS | grep -q ^name= && { 44 | NAME=$(echo $SUBSYS | sed s/^name=//) 45 | ln -s $SUBSYS $CGROUP/$NAME 46 | } 47 | 48 | # Likewise, on at least one system, it has been reported that 49 | # systemd would mount the CPU and CPU accounting controllers 50 | # (respectively "cpu" and "cpuacct") with "-o cpuacct,cpu" 51 | # but on a directory called "cpu,cpuacct" (note the inversion 52 | # in the order of the groups). This tries to work around it. 53 | [ $SUBSYS = cpuacct,cpu ] && ln -s $SUBSYS $CGROUP/cpu,cpuacct 54 | done 55 | 56 | # Note: as I write those lines, the LXC userland tools cannot setup 57 | # a "sub-container" properly if the "devices" cgroup is not in its 58 | # own hierarchy. Let's detect this and issue a warning. 59 | grep -q :devices: /proc/1/cgroup || 60 | echo "WARNING: the 'devices' cgroup should be in its own hierarchy." 61 | grep -qw devices /proc/1/cgroup || 62 | echo "WARNING: it looks like the 'devices' cgroup is not mounted." 63 | 64 | # Now, close extraneous file descriptors. 65 | pushd /proc/self/fd >/dev/null 66 | for FD in * 67 | do 68 | case "$FD" in 69 | # Keep stdin/stdout/stderr 70 | [012]) 71 | ;; 72 | # Nuke everything else 73 | *) 74 | eval exec "$FD>&-" 75 | ;; 76 | esac 77 | done 78 | popd >/dev/null 79 | 80 | # If we were given a PORT environment variable, start as a simple daemon; 81 | # otherwise, spawn a shell as well 82 | if [ "$PORT" ] 83 | then 84 | exec docker -d -H 0.0.0.0:$PORT 2>&1 >/dev/null 85 | else 86 | docker -d 2>&1 >/dev/null & 87 | while ! /bin/echo -e 'GET /v1.3/version HTTP/1.0\n\n' | socat - unix-connect:/var/run/docker.sock | grep -q '200 OK'; do 88 | sleep 1 89 | done 90 | 91 | fi 92 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | dockenstack: 2 | build: dockenstack 3 | privileged: true 4 | -------------------------------------------------------------------------------- /fig.yml: -------------------------------------------------------------------------------- 1 | docker-compose.yml --------------------------------------------------------------------------------