├── LICENSE ├── README.md ├── examples ├── guestbook │ ├── README.md │ ├── frontend-controller.json │ ├── frontend-service.json │ ├── redis-master-service.json │ ├── redis-master.json │ ├── redis-slave-controller.json │ └── redis-slave-service.json └── nginx │ ├── README.md │ ├── nginx-controller.json │ └── nginx-service.json └── kubernetes.yaml /LICENSE: -------------------------------------------------------------------------------- 1 | # Copyright 2014 Zuercher Hochschule fuer Angewandte Wissenschaften 2 | # All Rights Reserved. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); you may 5 | # not use this file except in compliance with the License. You may obtain 6 | # a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | # License for the specific language governing permissions and limitations 14 | # under the License. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | icclab/kubernetes-on-openstack-demo 2 | ========= 3 | 4 | This repo guides you through the process of setting up your own Kubernetes Cluster on your existing OpenStack Cloud using its Orchestration Service Heat. This Kubernetes Cluster should only be used as a Proof of Concept. 5 | 6 | Big thanks @ kelseyhightower and brendanburns for their tutorials/examples! 7 | 8 | ###Technology involved: 9 | 10 | * Kubernetes: https://github.com/GoogleCloudPlatform/kubernetes 11 | * CoreOS: https://coreos.com/ 12 | * etcd: https://github.com/coreos/etcd 13 | * fleet: https://github.com/coreos/fleet 14 | * flannel: https://github.com/coreos/flannel 15 | * kube-register: https://github.com/kelseyhightower/kube-register 16 | 17 | ###Architecture 18 | 19 | The provisioned Cluster consists of 5 VMs. The first one, discovery, is a dedicated etcd host. This allows easy etcd discovery thanks to a static IP-Address. 20 | 21 | A Kubernetes Master host is setup with the Kubernetes components apiserver, scheduler, kube-register, controller-manager as well as proxy. This machine also gets a floating IP assined and acts as a access point to your Kubernetes cluster. 22 | 23 | Three Kubernetes Minion hosts are setup with the Kubernetes components kubelet and proxy. 24 | 25 | ##Start the Cluster 26 | 27 | This Template was written for and tested on OpenStack Icehouse. 28 | 29 | ###Upload the CoreOS Image 30 | 31 | Import the CoreOS Image from the alpha channel following these Instructions: 32 | 33 | https://coreos.com/docs/running-coreos/platforms/openstack/#import-the-image 34 | 35 | ###Clone the Git Repo: 36 | ``` 37 | $ git clone https://github.com/icclab/kubernetes-on-openstack-demo.git 38 | ``` 39 | ### Parameters 40 | To start the stack, you need a few parameters. These can be retrieved directly from OpenStack: 41 | 42 | 43 | ``` 44 | image_id: glance image-list (choose the CoreOS image you uploaded) 45 | key-pair: nova keypair-list 46 | public_net_id: neutron net-list (choose the external network) 47 | ``` 48 | ###Start the template 49 | Start the stack either using Horizon or the heat-cli: 50 | ``` 51 | $ heat stack-create -f \ 52 | -P image_id= \ 53 | -P key-pair= \ 54 | -P public_net_id= \ 55 | kubernetes 56 | ``` 57 | You can check the progress using: 58 | ``` 59 | $ heat stack-show kubernetes 60 | ``` 61 | ###Get Floating IP 62 | After a few seconds the stack should be up and running. Retrieve its floating IP via Horizon or the heat-cli: 63 | 64 | ``` 65 | $ heat output-show kubernetes floating_ip 66 | ``` 67 | 68 | ###(Optional) Get Kubecfg binary and setup ssh tunnel 69 | If you would like to control your kubernetes cluster from your local machine, get the kubecfg binary for your OS: 70 | ####Linux 71 | ``` 72 | $ wget http://storage.googleapis.com/kubernetes/kubecfg -O /usr/local/bin/kubecfg 73 | ``` 74 | ####Darwin 75 | ``` 76 | $ wget http://storage.googleapis.com/kubernetes/darwin/kubecfg -O /usr/local/bin/kubecfg 77 | ``` 78 | Make the downloaded binary executable 79 | ``` 80 | $ chmod +x /usr/local/bin/kubecfg 81 | ``` 82 | 83 | Now setup a SSH tunnel to your kubernetes cluster 84 | ``` 85 | $ ssh -f -nNT -L 8080:127.0.0.1:8080 core@${floating_ip} 86 | ``` 87 | ##Test it 88 | To verify that the Cluster is up, list all minions: 89 | ``` 90 | $ kubecfg list minions 91 | ``` 92 | It should show you 3 minions: 93 | ``` 94 | Minion identifier 95 | ---------- 96 | 192.168.12.20 97 | 192.168.12.22 98 | 192.168.12.23 99 | ``` 100 | If it works, follow one of the examples provided. 101 | The guestbook example was taken from the kubernetes repo: 102 | https://github.com/GoogleCloudPlatform/kubernetes 103 | The frontend was slightly modified to display the hostname. 104 | 105 | Version 106 | ---- 107 | 108 | 1.0 109 | 110 | 111 | 112 | License 113 | ---- 114 | ``` 115 | # Copyright 2014 Zuercher Hochschule fuer Angewandte Wissenschaften 116 | # All Rights Reserved. 117 | # 118 | # Licensed under the Apache License, Version 2.0 (the "License"); you may 119 | # not use this file except in compliance with the License. You may obtain 120 | # a copy of the License at 121 | # 122 | # http://www.apache.org/licenses/LICENSE-2.0 123 | # 124 | # Unless required by applicable law or agreed to in writing, software 125 | # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 126 | # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 127 | # License for the specific language governing permissions and limitations 128 | # under the License. 129 | ``` 130 | -------------------------------------------------------------------------------- /examples/guestbook/README.md: -------------------------------------------------------------------------------- 1 | examples/guestbook 2 | ========= 3 | 4 | The Guestbook example from the kubernetes repo: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/guestbook/README.md 5 | 6 | ##HowTo: 7 | ###Step 0 - setup Kubernetes cluster 8 | see README.md 9 | ###Step 1 - create redis-master pod 10 | ``` 11 | $ kubecfg -c examples/guestbook/redis-master.json create pods 12 | ``` 13 | ###Step 2 - create redis-master service 14 | ``` 15 | $ kubecfg -c examples/guestbook/redis-master-service.json create services 16 | ``` 17 | ###Step 3 - create redis-slave replication controller 18 | ``` 19 | $ kubecfg -c examples/guestbook/redis-slave-controller.json create replicationControllers 20 | ``` 21 | ###Step 4 - create redis-slave service 22 | ``` 23 | $ kubecfg -c examples/guestbook/redis-slave-service.json create services 24 | ``` 25 | ###Step 5 - create frontend replication controller 26 | ``` 27 | $ kubecfg -c examples/guestbook/frontend-controller.json create replicationControllers 28 | ``` 29 | ###Step 6 - create frontend service 30 | ``` 31 | $ kubecfg -c examples/guestbook/frontend-service.json create services 32 | ``` 33 | ###Step 7 - verify 34 | ``` 35 | $ for kind in pods replicationControllers services; do kubecfg list ${kind}; done 36 | ``` 37 | Do note that the first time a pod gets scheduled to a minion, the docker images has to be pulled from the Docker Hub. This may take some time. 38 | 39 | The Output should look similar to this: 40 | ``` 41 | ID Image(s) Host Labels Status 42 | ---------- ---------- ---------- ---------- ---------- 43 | 3b8cb8cd-5852-11e4-b2d0-fa163edd4b96 michaelerne/php-redis 192.168.12.20/ name=frontend,replicationController=frontendController Waiting 44 | 3b8cdd81-5852-11e4-b2d0-fa163edd4b96 michaelerne/php-redis 192.168.12.23/ name=frontend,replicationController=frontendController Running 45 | redis-master-2 dockerfile/redis 192.168.12.22/ name=redis-master Waiting 46 | 354dd321-5852-11e4-b2d0-fa163edd4b96 michaelerne/redis-slave 192.168.12.22/ name=redisslave,replicationController=redisSlaveController Waiting 47 | 354e040c-5852-11e4-b2d0-fa163edd4b96 michaelerne/redis-slave 192.168.12.20/ name=redisslave,replicationController=redisSlaveController Waiting 48 | 3b8c9754-5852-11e4-b2d0-fa163edd4b96 michaelerne/php-redis 192.168.12.22/ name=frontend,replicationController=frontendController Waiting 49 | 50 | ID Image(s) Selector Replicas 51 | ---------- ---------- ---------- ---------- 52 | redisSlaveController michaelerne/redis-slave name=redisslave 2 53 | frontendController michaelerne/php-redis name=frontend 3 54 | 55 | ID Labels Selector Port 56 | ---------- ---------- ---------- ---------- 57 | redismaster name=redis-master name=redis-master 10000 58 | redisslave name=redisslave name=redisslave 10001 59 | frontend name=frontend 80 60 | ``` 61 | ### Remove 62 | ``` 63 | $ kubecfg delete services/frontend 64 | $ kubecfg delete services/redisslave 65 | $ kubecfg delete services/redismaster 66 | $ kubecfg stop frontendController 67 | $ kubecfg rm frontendController 68 | $ kubecfg stop redisSlaveController 69 | $ kubecfg rm redisSlaveController 70 | $ kubecfg delete pods/redis-master-2 71 | ``` 72 | 73 | Version 74 | ---- 75 | 76 | 1.0 77 | 78 | 79 | 80 | License 81 | ---- 82 | 83 | see LICENSE 84 | 85 | -------------------------------------------------------------------------------- /examples/guestbook/frontend-controller.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "frontendController", 3 | "kind": "ReplicationController", 4 | "apiVersion": "v1beta1", 5 | "desiredState": { 6 | "replicas": 3, 7 | "replicaSelector": {"name": "frontend"}, 8 | "podTemplate": { 9 | "desiredState": { 10 | "manifest": { 11 | "version": "v1beta1", 12 | "id": "frontendController", 13 | "containers": [{ 14 | "name": "php-redis", 15 | "image": "michaelerne/php-redis", 16 | "cpu": 100, 17 | "ports": [{"containerPort": 80, "hostPort": 8000}] 18 | }] 19 | } 20 | }, 21 | "labels": {"name": "frontend"} 22 | }}, 23 | "labels": {"name": "frontend"} 24 | } 25 | -------------------------------------------------------------------------------- /examples/guestbook/frontend-service.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "frontend", 3 | "kind": "Service", 4 | "apiVersion": "v1beta1", 5 | "port": 80, 6 | "containerPort": 80, 7 | "selector": { 8 | "name": "frontend" 9 | } 10 | } 11 | -------------------------------------------------------------------------------- /examples/guestbook/redis-master-service.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "redismaster", 3 | "kind": "Service", 4 | "apiVersion": "v1beta1", 5 | "port": 10000, 6 | "containerPort": 6379, 7 | "selector": { 8 | "name": "redis-master" 9 | }, 10 | "labels": { 11 | "name": "redis-master" 12 | } 13 | } 14 | -------------------------------------------------------------------------------- /examples/guestbook/redis-master.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "redis-master-2", 3 | "kind": "Pod", 4 | "apiVersion": "v1beta1", 5 | "desiredState": { 6 | "manifest": { 7 | "version": "v1beta1", 8 | "id": "redis-master-2", 9 | "containers": [{ 10 | "name": "master", 11 | "image": "dockerfile/redis", 12 | "cpu": 100, 13 | "ports": [{ 14 | "containerPort": 6379, 15 | "hostPort": 6379 16 | }] 17 | }] 18 | } 19 | }, 20 | "labels": { 21 | "name": "redis-master" 22 | } 23 | } 24 | 25 | -------------------------------------------------------------------------------- /examples/guestbook/redis-slave-controller.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "redisSlaveController", 3 | "kind": "ReplicationController", 4 | "apiVersion": "v1beta1", 5 | "desiredState": { 6 | "replicas": 2, 7 | "replicaSelector": {"name": "redisslave"}, 8 | "podTemplate": { 9 | "desiredState": { 10 | "manifest": { 11 | "version": "v1beta1", 12 | "id": "redisSlaveController", 13 | "containers": [{ 14 | "name": "slave", 15 | "image": "michaelerne/redis-slave", 16 | "cpu": 200, 17 | "ports": [{"containerPort": 6379, "hostPort": 6380}] 18 | }] 19 | } 20 | }, 21 | "labels": {"name": "redisslave"} 22 | }}, 23 | "labels": {"name": "redisslave"} 24 | } 25 | -------------------------------------------------------------------------------- /examples/guestbook/redis-slave-service.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "redisslave", 3 | "kind": "Service", 4 | "apiVersion": "v1beta1", 5 | "port": 10001, 6 | "containerPort": 6379, 7 | "labels": { 8 | "name": "redisslave" 9 | }, 10 | "selector": { 11 | "name": "redisslave" 12 | } 13 | } 14 | -------------------------------------------------------------------------------- /examples/nginx/README.md: -------------------------------------------------------------------------------- 1 | examples/nginx 2 | ========= 3 | 4 | A simple example of a replication controller and a service to cerate a load balanced nginx. 5 | 6 | ##HowTo: 7 | ###Step 0 - setup Kubernetes cluster 8 | see README.md 9 | ###Step 1 - create replication controller 10 | ``` 11 | $ kubecfg -c examples/nginx/nginx-controller.json create replicationControllers 12 | ``` 13 | ###Step 2 - create service 14 | ``` 15 | $ kubecfg -c examples/nginx/nginx-service.json create services 16 | ``` 17 | ###Step 3 - verify 18 | ``` 19 | $ for kind in pods replicationControllers services; do kubecfg list ${kind}; done 20 | ``` 21 | Do note that the first time a pod gets scheduled to a minion, the docker images has to be pulled from the Docker Hub. This may take some time. 22 | 23 | The Output should look similar to this: 24 | ``` 25 | ID Image(s) Host Labels Status 26 | ---------- ---------- ---------- ---------- ---------- 27 | 680c2cca-5609-11e4-9743-fa163ee0b804 dockerfile/nginx 192.168.12.23/ name=nginx,replicationController=nginxController Running 28 | 680cc301-5609-11e4-9743-fa163ee0b804 dockerfile/nginx 192.168.12.22/ name=nginx,replicationController=nginxController Running 29 | 680e10a0-5609-11e4-9743-fa163ee0b804 dockerfile/nginx 192.168.12.20/ name=nginx,replicationController=nginxController Running 30 | 31 | ID Image(s) Selector Replicas 32 | ---------- ---------- ---------- ---------- 33 | nginxController dockerfile/nginx name=nginx 3 34 | 35 | ID Labels Selector Port 36 | ---------- ---------- ---------- ---------- 37 | nginx name=nginx 80 38 | 39 | ``` 40 | ###Step 4 - play with it 41 | 42 | Now nginx should be served on each host on port ```80``` and port ```8000```. 43 | 44 | Using port ```80```, the kubernetes proxy will route the request to any pod with the label ```nginx```. 45 | Using port ```8000```, you directly connect to the pod running on this machine. 46 | 47 | ###remove it 48 | 49 | ``` 50 | $ kubecfg stop nginxController 51 | $ kubecfg rm nginxController 52 | $ kubecfg delete services/nginx 53 | ``` 54 | 55 | Version 56 | ---- 57 | 58 | 1.0 59 | 60 | 61 | 62 | License 63 | ---- 64 | 65 | see LICENSE 66 | 67 | 68 | 69 | -------------------------------------------------------------------------------- /examples/nginx/nginx-controller.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "nginxController", 3 | "kind": "ReplicationController", 4 | "apiVersion": "v1beta1", 5 | "desiredState": { 6 | "replicas": 3, 7 | "replicaSelector": {"name": "nginx"}, 8 | "podTemplate": { 9 | "desiredState": { 10 | "manifest": { 11 | "version": "v1beta1", 12 | "id": "nginxController", 13 | "containers": [{ 14 | "name": "nginx", 15 | "image": "dockerfile/nginx", 16 | "ports": [{"containerPort": 80, "hostPort": 8000}] 17 | }] 18 | } 19 | }, 20 | "labels": {"name": "nginx"} 21 | }}, 22 | "labels": {"name": "nginx"} 23 | } 24 | -------------------------------------------------------------------------------- /examples/nginx/nginx-service.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "nginx", 3 | "kind": "Service", 4 | "apiVersion": "v1beta1", 5 | "port": 80, 6 | "containerPort": 80, 7 | "selector": { 8 | "name": "nginx" 9 | } 10 | } 11 | -------------------------------------------------------------------------------- /kubernetes.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2014 Zuercher Hochschule fuer Angewandte Wissenschaften 2 | # All Rights Reserved. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); you may 5 | # not use this file except in compliance with the License. You may obtain 6 | # a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | # License for the specific language governing permissions and limitations 14 | # under the License. 15 | 16 | heat_template_version: 2013-05-23 17 | description: > 18 | A simple Heat template to deploy CoreOS into an existing cluster. 19 | parameters: 20 | public_net_id: 21 | type: string 22 | label: Public Network ID 23 | description: ID of the public network to use 24 | image_id: 25 | type: string 26 | label: Glance Image ID 27 | description: ID of existing Glance image to use 28 | key-pair: 29 | type: string 30 | label: SSH Key Pair 31 | description: Name of the SSH Key pair to use 32 | 33 | resources: 34 | security_group: 35 | type: OS::Neutron::SecurityGroup 36 | properties: 37 | description: Kubernetes Security Group 38 | name: kube-secrurity-group 39 | rules: 40 | - remote_ip_prefix: 0.0.0.0/0 41 | protocol: tcp 42 | port_range_min: 0 43 | port_range_max: 65535 44 | - remote_ip_prefix: 0.0.0.0/0 45 | protocol: udp 46 | port_range_min: 0 47 | port_range_max: 65535 48 | - remote_ip_prefix: 0.0.0.0/0 49 | protocol: icmp 50 | private_net: 51 | type: OS::Neutron::Net 52 | properties: 53 | admin_state_up: true 54 | name: kube-net 55 | private_subnet: 56 | type: OS::Neutron::Subnet 57 | properties: 58 | name: kube-subnet 59 | cidr: 192.168.12.0/24 60 | gateway_ip: 192.168.12.1 61 | allocation_pools: 62 | - start: 192.168.12.20 63 | end: 192.168.12.99 64 | dns_nameservers: [8.8.8.8, 8.8.4.4] 65 | enable_dhcp: true 66 | network_id: { get_resource: private_net } 67 | router: 68 | type: OS::Neutron::Router 69 | properties: 70 | name: kube-router 71 | admin_state_up: true 72 | router_gw: 73 | type: OS::Neutron::RouterGateway 74 | properties: 75 | network_id: { get_param: public_net_id } 76 | router_id: { get_resource: router } 77 | router_interface: 78 | type: OS::Neutron::RouterInterface 79 | properties: 80 | router_id: { get_resource: router } 81 | subnet_id: { get_resource: private_subnet } 82 | 83 | discovery_port: 84 | type: OS::Neutron::Port 85 | properties: 86 | admin_state_up: true 87 | network_id: { get_resource: private_net } 88 | security_groups: 89 | - { get_resource: security_group } 90 | fixed_ips: 91 | - ip_address: "192.168.12.10" 92 | discovery: 93 | type: OS::Nova::Server 94 | properties: 95 | name: discovery 96 | image: { get_param: image_id } 97 | flavor: m1.small 98 | networks: 99 | - port: { get_resource: discovery_port } 100 | key_name: { get_param: key-pair } 101 | user_data_format: RAW 102 | user_data: | 103 | #cloud-config 104 | coreos: 105 | fleet: 106 | etcd_servers: http://127.0.0.1:4001 107 | metadata: role=etcd 108 | etcd: 109 | name: etcd 110 | addr: $private_ipv4:4001 111 | bind-addr: 0.0.0.0 112 | peer-addr: $private_ipv4:7001 113 | cluster-active-size: 1 114 | snapshot: true 115 | units: 116 | - name: etcd.service 117 | command: start 118 | 119 | 120 | master_port: 121 | type: OS::Neutron::Port 122 | properties: 123 | admin_state_up: true 124 | network_id: { get_resource: private_net } 125 | security_groups: 126 | - { get_resource: security_group } 127 | master_floatingip: 128 | type: OS::Neutron::FloatingIP 129 | properties: 130 | floating_network_id: { get_param: public_net_id } 131 | port_id: { get_resource: master_port } 132 | master: 133 | type: OS::Nova::Server 134 | properties: 135 | name: master 136 | image: { get_param: image_id } 137 | flavor: m1.small 138 | networks: 139 | - port: { get_resource: master_port } 140 | key_name: { get_param: key-pair } 141 | user_data_format: RAW 142 | user_data: 143 | str_replace: 144 | template: | 145 | #cloud-config 146 | coreos: 147 | fleet: 148 | public-ip: $private_ipv4 149 | metadata: role=master 150 | etcd: 151 | discovery: http://$etcd_discovery_ip$:4001/v2/keys/cluster 152 | addr: $private_ipv4:4001 153 | peer-addr: $private_ipv4:7001 154 | units: 155 | - name: etcd.service 156 | command: start 157 | - name: fleet.service 158 | command: start 159 | 160 | - name: flannel.service 161 | command: start 162 | content: | 163 | [Unit] 164 | Requires=etcd.service 165 | After=etcd.service 166 | 167 | [Service] 168 | ExecStartPre=-/usr/bin/mkdir -p /opt/bin 169 | ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/flannel/flanneld 170 | ExecStartPre=/usr/bin/chmod +x /opt/bin/flanneld 171 | ExecStartPre=/usr/bin/etcdctl -C http://$etcd_discovery_ip$:4001 set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 172 | ExecStart=/opt/bin/flanneld \ 173 | -etcd-endpoint http://$etcd_discovery_ip$:4001 \ 174 | -iface=$private_ipv4 \ 175 | -logtostderr=true 176 | 177 | [Install] 178 | WantedBy=multi-user.target 179 | 180 | - name: docker.service 181 | command: restart 182 | content: | 183 | [Unit] 184 | Description=Docker Application Container Engine 185 | Documentation=http://docs.docker.io 186 | Requires=flannel.service 187 | After=flannel.service 188 | 189 | [Service] 190 | EnvironmentFile=/run/flannel/subnet.env 191 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 192 | ExecStartPre=-/usr/sbin/brctl delbr docker0 193 | ExecStartPre=/bin/mount --make-rprivate / 194 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} 195 | Restart=on-failure 196 | RestartSec=5 197 | 198 | [Install] 199 | WantedBy=multi-user.target 200 | 201 | - name: download-kubernetes.service 202 | command: start 203 | content: | 204 | [Unit] 205 | After=network-online.target 206 | Before=apiserver.service 207 | Before=controller-manager.service 208 | Description=Download Kubernetes Binaries 209 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 210 | Requires=network-online.target 211 | 212 | [Service] 213 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/apiserver 214 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/controller-manager 215 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/kubecfg 216 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/scheduler 217 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/kube-register 218 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/proxy 219 | ExecStart=/usr/bin/chmod +x /opt/bin/apiserver 220 | ExecStart=/usr/bin/chmod +x /opt/bin/controller-manager 221 | ExecStart=/usr/bin/chmod +x /opt/bin/kubecfg 222 | ExecStart=/usr/bin/chmod +x /opt/bin/scheduler 223 | ExecStart=/usr/bin/chmod +x /opt/bin/kube-register 224 | ExecStart=/usr/bin/chmod +x /opt/bin/proxy 225 | RemainAfterExit=yes 226 | Type=oneshot 227 | 228 | - name: apiserver.service 229 | command: start 230 | content: | 231 | [Unit] 232 | After=etcd.service 233 | After=download-kubernetes.service 234 | ConditionFileIsExecutable=/opt/bin/apiserver 235 | Description=Kubernetes API Server 236 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 237 | Wants=etcd.service 238 | Wants=download-kubernetes.service 239 | 240 | [Service] 241 | ExecStart=/opt/bin/apiserver \ 242 | --address=0.0.0.0 \ 243 | --port=8080 \ 244 | --etcd_servers=http://$etcd_discovery_ip$:4001 \ 245 | --machines=192.168.12.14,192.168.12.15,192.168.12.16 \ 246 | --logtostderr=true 247 | Restart=always 248 | RestartSec=10 249 | 250 | [Install] 251 | WantedBy=multi-user.target 252 | 253 | - name: scheduler.service 254 | command: start 255 | content: | 256 | [Unit] 257 | After=apiserver.service 258 | ConditionFileIsExecutable=/opt/bin/scheduler 259 | Description=Kubernetes Scheduler 260 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 261 | Wants=apiserver.service 262 | 263 | [Service] 264 | ExecStart=/opt/bin/scheduler \ 265 | --logtostderr=true \ 266 | --master=127.0.0.1:8080 267 | Restart=always 268 | RestartSec=10 269 | 270 | [Install] 271 | WantedBy=multi-user.target 272 | 273 | - name: controller-manager.service 274 | command: start 275 | content: | 276 | [Unit] 277 | After=etcd.service 278 | After=download-kubernetes.service 279 | ConditionFileIsExecutable=/opt/bin/controller-manager 280 | Description=Kubernetes Controller Manager 281 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 282 | Wants=etcd.service 283 | Wants=download-kubernetes.service 284 | 285 | [Service] 286 | ExecStart=/opt/bin/controller-manager \ 287 | --master=127.0.0.1:8080 \ 288 | --logtostderr=true 289 | Restart=always 290 | RestartSec=10 291 | 292 | [Install] 293 | WantedBy=multi-user.target 294 | 295 | - name: kube-register.service 296 | command: start 297 | content: | 298 | [Unit] 299 | After=etcd.service 300 | After=download-kubernetes.service 301 | ConditionFileIsExecutable=/opt/bin/kube-register 302 | Description=Kubernetes Registration Service 303 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 304 | Wants=etcd.service 305 | Wants=download-kubernetes.service 306 | 307 | [Service] 308 | ExecStart=/opt/bin/kube-register \ 309 | --metadata=role=kubernetes \ 310 | --fleet-endpoint=unix:///var/run/fleet.sock \ 311 | --api-endpoint=http://127.0.0.1:8080 312 | Restart=always 313 | RestartSec=10 314 | 315 | [Install] 316 | WantedBy=multi-user.target 317 | - name: proxy.service 318 | command: start 319 | content: | 320 | [Unit] 321 | After=etcd.service 322 | After=download-kubernetes.service 323 | ConditionFileIsExecutable=/opt/bin/proxy 324 | Description=Kubernetes Proxy 325 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 326 | Wants=etcd.service 327 | Wants=download-kubernetes.service 328 | 329 | [Service] 330 | ExecStart=/opt/bin/proxy \ 331 | --etcd_servers=http://$etcd_discovery_ip$:4001 --logtostderr=true 332 | Restart=always 333 | RestartSec=10 334 | 335 | [Install] 336 | WantedBy=multi-user.target 337 | params: 338 | $etcd_discovery_ip$: 339 | get_attr: [ discovery, first_address ] 340 | minion-1_port: 341 | type: OS::Neutron::Port 342 | properties: 343 | admin_state_up: true 344 | network_id: { get_resource: private_net } 345 | security_groups: 346 | - { get_resource: security_group } 347 | fixed_ips: 348 | - ip_address: "192.168.12.14" 349 | minion-1: 350 | type: OS::Nova::Server 351 | properties: 352 | name: minion-1 353 | image: { get_param: image_id } 354 | flavor: m1.medium 355 | networks: 356 | - port: { get_resource: minion-1_port } 357 | key_name: { get_param: key-pair } 358 | user_data_format: RAW 359 | user_data: 360 | str_replace: 361 | template: | 362 | #cloud-config 363 | coreos: 364 | etcd: 365 | discovery: http://$etcd_discovery_ip$:4001/v2/keys/cluster 366 | addr: $private_ipv4:4001 367 | peer-addr: $private_ipv4:7001 368 | fleet: 369 | public-ip: $public_ipv4 370 | metadata: role=kubernetes 371 | units: 372 | - name: etcd.service 373 | command: start 374 | - name: fleet.service 375 | command: start 376 | - name: flannel.service 377 | command: start 378 | content: | 379 | [Unit] 380 | Requires=etcd.service 381 | After=etcd.service 382 | 383 | [Service] 384 | ExecStartPre=-/usr/bin/mkdir -p /opt/bin 385 | ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/flannel/flanneld 386 | ExecStartPre=/usr/bin/chmod +x /opt/bin/flanneld 387 | ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 388 | ExecStart=/opt/bin/flanneld \ 389 | -etcd-endpoint http://$etcd_discovery_ip$:4001 \ 390 | -iface=$private_ipv4 \ 391 | -logtostderr=true 392 | 393 | - name: docker.service 394 | command: restart 395 | content: | 396 | [Unit] 397 | Description=Docker Application Container Engine 398 | Documentation=http://docs.docker.io 399 | Requires=flannel.service 400 | After=flannel.service 401 | 402 | [Service] 403 | EnvironmentFile=/run/flannel/subnet.env 404 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 405 | ExecStartPre=-/usr/sbin/brctl delbr docker0 406 | ExecStartPre=/bin/mount --make-rprivate / 407 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} 408 | Restart=on-failure 409 | RestartSec=5 410 | 411 | [Install] 412 | WantedBy=multi-user.target 413 | 414 | - name: download-kubernetes.service 415 | command: start 416 | content: | 417 | [Unit] 418 | After=network-online.target 419 | Before=kubelet.service 420 | Before=proxy.service 421 | Description=Download Kubernetes Binaries 422 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 423 | Requires=network-online.target 424 | 425 | [Service] 426 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/kubelet 427 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/proxy 428 | ExecStart=/usr/bin/chmod +x /opt/bin/kubelet 429 | ExecStart=/usr/bin/chmod +x /opt/bin/proxy 430 | RemainAfterExit=yes 431 | Type=oneshot 432 | 433 | - name: kubelet.service 434 | command: start 435 | content: | 436 | [Unit] 437 | After=etcd.service 438 | After=download-kubernetes.service 439 | ConditionFileIsExecutable=/opt/bin/kubelet 440 | Description=Kubernetes Kubelet 441 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 442 | Wants=etcd.service 443 | Wants=download-kubernetes.service 444 | 445 | [Service] 446 | ExecStart=/opt/bin/kubelet \ 447 | --address=0.0.0.0 \ 448 | --port=10250 \ 449 | --hostname_override=$private_ipv4 \ 450 | --etcd_servers=http://$etcd_discovery_ip$:4001 \ 451 | --logtostderr=true 452 | Restart=always 453 | RestartSec=10 454 | 455 | [Install] 456 | WantedBy=multi-user.target 457 | 458 | - name: proxy.service 459 | command: start 460 | content: | 461 | [Unit] 462 | After=etcd.service 463 | After=download-kubernetes.service 464 | ConditionFileIsExecutable=/opt/bin/proxy 465 | Description=Kubernetes Proxy 466 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 467 | Wants=etcd.service 468 | Wants=download-kubernetes.service 469 | 470 | [Service] 471 | ExecStart=/opt/bin/proxy \ 472 | --etcd_servers=http://$etcd_discovery_ip$:4001 --logtostderr=true 473 | Restart=always 474 | RestartSec=10 475 | 476 | [Install] 477 | WantedBy=multi-user.target 478 | 479 | params: 480 | $etcd_discovery_ip$: 481 | get_attr: [ discovery, first_address ] 482 | minion-2_port: 483 | type: OS::Neutron::Port 484 | properties: 485 | admin_state_up: true 486 | network_id: { get_resource: private_net } 487 | security_groups: 488 | - { get_resource: security_group } 489 | fixed_ips: 490 | - ip_address: "192.168.12.15" 491 | minion-2: 492 | type: OS::Nova::Server 493 | properties: 494 | name: minion-2 495 | image: { get_param: image_id } 496 | flavor: m1.medium 497 | networks: 498 | - port: { get_resource: minion-2_port } 499 | key_name: { get_param: key-pair } 500 | user_data_format: RAW 501 | user_data: 502 | str_replace: 503 | template: | 504 | #cloud-config 505 | coreos: 506 | etcd: 507 | discovery: http://$etcd_discovery_ip$:4001/v2/keys/cluster 508 | addr: $private_ipv4:4001 509 | peer-addr: $private_ipv4:7001 510 | fleet: 511 | public-ip: $public_ipv4 512 | metadata: role=kubernetes 513 | units: 514 | - name: etcd.service 515 | command: start 516 | - name: fleet.service 517 | command: start 518 | - name: flannel.service 519 | command: start 520 | content: | 521 | [Unit] 522 | Requires=etcd.service 523 | After=etcd.service 524 | 525 | [Service] 526 | ExecStartPre=-/usr/bin/mkdir -p /opt/bin 527 | ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/flannel/flanneld 528 | ExecStartPre=/usr/bin/chmod +x /opt/bin/flanneld 529 | ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 530 | ExecStart=/opt/bin/flanneld \ 531 | -etcd-endpoint http://$etcd_discovery_ip$:4001 \ 532 | -iface=$private_ipv4 \ 533 | -logtostderr=true 534 | 535 | - name: docker.service 536 | command: restart 537 | content: | 538 | [Unit] 539 | Description=Docker Application Container Engine 540 | Documentation=http://docs.docker.io 541 | Requires=flannel.service 542 | After=flannel.service 543 | 544 | [Service] 545 | EnvironmentFile=/run/flannel/subnet.env 546 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 547 | ExecStartPre=-/usr/sbin/brctl delbr docker0 548 | ExecStartPre=/bin/mount --make-rprivate / 549 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} 550 | Restart=on-failure 551 | RestartSec=5 552 | 553 | [Install] 554 | WantedBy=multi-user.target 555 | 556 | - name: download-kubernetes.service 557 | command: start 558 | content: | 559 | [Unit] 560 | After=network-online.target 561 | Before=kubelet.service 562 | Before=proxy.service 563 | Description=Download Kubernetes Binaries 564 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 565 | Requires=network-online.target 566 | 567 | [Service] 568 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/kubelet 569 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/proxy 570 | ExecStart=/usr/bin/chmod +x /opt/bin/kubelet 571 | ExecStart=/usr/bin/chmod +x /opt/bin/proxy 572 | RemainAfterExit=yes 573 | Type=oneshot 574 | 575 | - name: kubelet.service 576 | command: start 577 | content: | 578 | [Unit] 579 | After=etcd.service 580 | After=download-kubernetes.service 581 | ConditionFileIsExecutable=/opt/bin/kubelet 582 | Description=Kubernetes Kubelet 583 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 584 | Wants=etcd.service 585 | Wants=download-kubernetes.service 586 | 587 | [Service] 588 | ExecStart=/opt/bin/kubelet \ 589 | --address=0.0.0.0 \ 590 | --port=10250 \ 591 | --hostname_override=$private_ipv4 \ 592 | --etcd_servers=http://$etcd_discovery_ip$:4001 \ 593 | --logtostderr=true 594 | Restart=always 595 | RestartSec=10 596 | 597 | [Install] 598 | WantedBy=multi-user.target 599 | 600 | - name: proxy.service 601 | command: start 602 | content: | 603 | [Unit] 604 | After=etcd.service 605 | After=download-kubernetes.service 606 | ConditionFileIsExecutable=/opt/bin/proxy 607 | Description=Kubernetes Proxy 608 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 609 | Wants=etcd.service 610 | Wants=download-kubernetes.service 611 | 612 | [Service] 613 | ExecStart=/opt/bin/proxy \ 614 | --etcd_servers=http://$etcd_discovery_ip$:4001 --logtostderr=true 615 | Restart=always 616 | RestartSec=10 617 | 618 | [Install] 619 | WantedBy=multi-user.target 620 | 621 | params: 622 | $etcd_discovery_ip$: 623 | get_attr: [ discovery, first_address ] 624 | minion-3_port: 625 | type: OS::Neutron::Port 626 | properties: 627 | admin_state_up: true 628 | network_id: { get_resource: private_net } 629 | security_groups: 630 | - { get_resource: security_group } 631 | fixed_ips: 632 | - ip_address: "192.168.12.16" 633 | minion-3: 634 | type: OS::Nova::Server 635 | properties: 636 | name: minion-3 637 | image: { get_param: image_id } 638 | flavor: m1.medium 639 | networks: 640 | - port: { get_resource: minion-3_port } 641 | key_name: { get_param: key-pair } 642 | user_data_format: RAW 643 | user_data: 644 | str_replace: 645 | template: | 646 | #cloud-config 647 | coreos: 648 | etcd: 649 | discovery: http://$etcd_discovery_ip$:4001/v2/keys/cluster 650 | addr: $private_ipv4:4001 651 | peer-addr: $private_ipv4:7001 652 | fleet: 653 | public-ip: $public_ipv4 654 | metadata: role=kubernetes 655 | units: 656 | - name: etcd.service 657 | command: start 658 | - name: fleet.service 659 | command: start 660 | - name: flannel.service 661 | command: start 662 | content: | 663 | [Unit] 664 | Requires=etcd.service 665 | After=etcd.service 666 | 667 | [Service] 668 | ExecStartPre=-/usr/bin/mkdir -p /opt/bin 669 | ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/flannel/flanneld 670 | ExecStartPre=/usr/bin/chmod +x /opt/bin/flanneld 671 | ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 672 | ExecStart=/opt/bin/flanneld \ 673 | -etcd-endpoint http://$etcd_discovery_ip$:4001 \ 674 | -iface=$private_ipv4 \ 675 | -logtostderr=true 676 | 677 | - name: docker.service 678 | command: restart 679 | content: | 680 | [Unit] 681 | Description=Docker Application Container Engine 682 | Documentation=http://docs.docker.io 683 | Requires=flannel.service 684 | After=flannel.service 685 | 686 | [Service] 687 | EnvironmentFile=/run/flannel/subnet.env 688 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 689 | ExecStartPre=-/usr/sbin/brctl delbr docker0 690 | ExecStartPre=/bin/mount --make-rprivate / 691 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} 692 | Restart=on-failure 693 | RestartSec=5 694 | 695 | [Install] 696 | WantedBy=multi-user.target 697 | 698 | - name: download-kubernetes.service 699 | command: start 700 | content: | 701 | [Unit] 702 | After=network-online.target 703 | Before=kubelet.service 704 | Before=proxy.service 705 | Description=Download Kubernetes Binaries 706 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 707 | Requires=network-online.target 708 | 709 | [Service] 710 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/kubelet 711 | ExecStart=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/kubernetes/proxy 712 | ExecStart=/usr/bin/chmod +x /opt/bin/kubelet 713 | ExecStart=/usr/bin/chmod +x /opt/bin/proxy 714 | RemainAfterExit=yes 715 | Type=oneshot 716 | 717 | - name: kubelet.service 718 | command: start 719 | content: | 720 | [Unit] 721 | After=etcd.service 722 | After=download-kubernetes.service 723 | ConditionFileIsExecutable=/opt/bin/kubelet 724 | Description=Kubernetes Kubelet 725 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 726 | Wants=etcd.service 727 | Wants=download-kubernetes.service 728 | 729 | [Service] 730 | ExecStart=/opt/bin/kubelet \ 731 | --address=0.0.0.0 \ 732 | --port=10250 \ 733 | --hostname_override=$private_ipv4 \ 734 | --etcd_servers=http://$etcd_discovery_ip$:4001 \ 735 | --logtostderr=true 736 | Restart=always 737 | RestartSec=10 738 | 739 | [Install] 740 | WantedBy=multi-user.target 741 | 742 | - name: proxy.service 743 | command: start 744 | content: | 745 | [Unit] 746 | After=etcd.service 747 | After=download-kubernetes.service 748 | ConditionFileIsExecutable=/opt/bin/proxy 749 | Description=Kubernetes Proxy 750 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 751 | Wants=etcd.service 752 | Wants=download-kubernetes.service 753 | 754 | [Service] 755 | ExecStart=/opt/bin/proxy \ 756 | --etcd_servers=http://$etcd_discovery_ip$:4001 --logtostderr=true 757 | Restart=always 758 | RestartSec=10 759 | 760 | [Install] 761 | WantedBy=multi-user.target 762 | 763 | params: 764 | $etcd_discovery_ip$: 765 | get_attr: [ discovery, first_address ] 766 | outputs: 767 | floating_ip: 768 | description: Public IP Address of Kubernetes master 769 | value: { get_attr: [master_floatingip, floating_ip_address] } 770 | key-pair: 771 | description: SSH key-pair name for Kubernetes master 772 | value: { get_param: key-pair } 773 | --------------------------------------------------------------------------------