├── LICENSE ├── README.md ├── dropletsettings.png ├── master ├── apiserver.service ├── controller-manager.service ├── docker.service ├── kubelet.service ├── proxy.service └── rudder.service ├── minion1 ├── docker.service ├── kubelet.service ├── proxy.service └── rudder.service ├── minion2 ├── docker.service ├── kubelet.service ├── proxy.service └── rudder.service ├── selectimage.png └── settings.png /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Brian Ketelsen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | coreos-kubernetes-digitalocean 2 | ============================== 3 | 4 | [These notes are now 25 days old -- a lifetime in a fast moving project like Kubernetes. Check the Kubernetes Github repo for more up-to-date instructions before attempting to follow this.] 5 | 6 | Configuration for running Kubernetes/CoreOS/Rudder on Digital Ocean 7 | 8 | ## Use this guide to get a Kubernetes cluster running on CoreOS linux on your Digital Ocean account. 9 | 10 | This guide is far from perfect, and can be probably be improved/automated quite a bit. Please send pull requests for corrections, clarifications and omissions. 11 | 12 | ## Step 1 13 | You're going to need a Digital Ocean account. If you don't have one, click [here](https://www.digitalocean.com/?refcode=9dd266a276e6) and use my referral link to get you (and me!) some free credit on Digital Ocean. I host all the Gopher Academy and GopherCon sites on Digital Ocean, so your referral credits will be put to good use. 14 | 15 | ## Step 2 16 | 17 | Follow [this guide](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-coreos-cluster-on-digitalocean) to install a three node CoreOS Cluster on Digital Ocean. I named my servers "master", "minion1" and "minion2" instead of the recommended "coreos-1", "coreos-2", and "coreos-3" so that I would remember which machines had which responsibilities. 18 | 19 | ## Step 3 20 | 21 | Go to the Digital Ocean control panel and write down the public and private IP addresses of all three machines. 22 | 23 | Yours might look like this: 24 | 25 | | Machine Name | Public IP | Private IP | 26 | | ------------- |:-------------:| -----:| 27 | | master | 104.131.x.1 | 10.132.x.1 | 28 | | minion1 | 104.131.x.2 | 10.132.x.1 | 29 | | minion2 | 104.131.x.3 | 10.132.x.1 | 30 | 31 | ## Step 4 32 | 33 | Install [Flannel](https://github.com/coreos/flannel) so that each pod in the Kubernetes cluster can have it's own IP address. 34 | 35 | On one of the CoreOS machines, checkout the [Rudder source](https://github.com/coreos/rudder.git) and follow the [instructions](https://github.com/coreos/rudder#building-rudder) to use their docker container to build Rudder. 36 | 37 | When you're done you should have Rudder installed at /opt/bin/rudder 38 | 39 | ## Step 5 40 | 41 | Configure Rudder on each machine. 42 | 43 | In /etc/systemd/system on each CoreOS machine, create a service file for rudder. Use [this one](https://raw.githubusercontent.com/bketelsen/coreos-kubernetes-digitalocean/master/master/rudder.service) as a template. Replace the line IP address my template with the correct PRIVATE IP address for that machine. Remember you can get that by typing `ifconfig`. My private IP addresses were on `eth1`. 44 | 45 | Repeat this process for all three machines, ensuring that you use each machine's private ip address in the rudder.service file. 46 | 47 | Add the service to systemctl: 48 | `sudo systemctl enable /etc/systemd/system/rudder.service` 49 | 50 | Reload systemctl: 51 | `sudo systemctl daemon-reload` 52 | 53 | Start Rudder: 54 | `sudo systemctl start rudder` 55 | 56 | ## Step 6 57 | 58 | Follow this same pattern to add `docker`, `kubelet`, and `proxy` services to all three machines. Remember to use YOUR private IP address in the kubelet.service file. Add each service, reload systemctl and start each service. 59 | 60 | ## Step 7 61 | 62 | On the master, add `apiserver` and `controller-manager`. In the `apiserver.service` file, list the private IP addresses of all three CoreOS machines on [Line 15](https://github.com/bketelsen/coreos-kubernetes-digitalocean/blob/master/master/apiserver.service#L15u) 63 | 64 | *** Important - you need to add a scheduler service now, too. I'll try to add a service unit for this shortly. *** 65 | 66 | ## Step 8 67 | 68 | Download `kubecfg` pre-built binaries by following the instructions at the bottom of [Kelsey Hightower's Guide](https://github.com/kelseyhightower/kubernetes-coreos) I put mine in /opt/bin 69 | 70 | With any luck you'll now have a fully operational Kubernetes cluster running on Digital Ocean. To test it type `kubecfg list minions`. You should see all three private ip addresses of your cluster listed in the result. 71 | 72 | ### Thanks/Credit 73 | 74 | I couldn't have gotten the cluster running without lots of debugging help from Kelsey Hightower. I also drew a lot of inspriation from [this post](https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fqiita.com%2Fyungsang%2Fitems%2F530ae3d3277d2fba3343&edit-text=&act=url) 75 | 76 | 77 | -------------------------------------------------------------------------------- /dropletsettings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bketelsen/coreos-kubernetes-digitalocean/3e9437ecf254173a31e62c51b05bfb4ac1975398/dropletsettings.png -------------------------------------------------------------------------------- /master/apiserver.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | After=etcd.service 3 | ConditionFileIsExecutable=/opt/bin/apiserver 4 | Description=Kubernetes API Server 5 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 6 | Wants=etcd.service 7 | 8 | [Service] 9 | ExecStart=/opt/bin/apiserver \ 10 | --address=0.0.0.0 \ 11 | --port=8080 \ 12 | --etcd_servers=http://127.0.0.1:4001 \ 13 | --machines=10.132.245.174,10.132.245.177,10.132.245.179 \ 14 | --logtostderr=true 15 | Restart=always 16 | RestartSec=10 17 | [Install] 18 | WantedBy=multi-user.target 19 | -------------------------------------------------------------------------------- /master/controller-manager.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/controller-manager 3 | Description=Kubernetes Controller Manager 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/controller-manager \ 8 | --master=127.0.0.1:8080 \ 9 | --logtostderr=true 10 | Restart=on-failure 11 | RestartSec=1 12 | 13 | [Install] 14 | WantedBy=multi-user.target 15 | -------------------------------------------------------------------------------- /master/docker.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker Application Container Engine 3 | Documentation=http://docs.docker.io 4 | Requires=rudder.service 5 | After=rudder.service 6 | 7 | [Service] 8 | EnvironmentFile=/run/rudder/subnet.env 9 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 10 | ExecStartPre=-/usr/sbin/brctl delbr docker0 11 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${RUDDER_SUBNET} --mtu=${RUDDER_MTU} 12 | Restart=on-failure 13 | RestartSec=5 14 | 15 | [Install] 16 | WantedBy=multi-user.target 17 | -------------------------------------------------------------------------------- /master/kubelet.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/kubelet 3 | Description=Kubernetes Kubelet 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/kubelet \ 8 | --address=10.132.245.174 \ 9 | --port=10250 \ 10 | --hostname_override=10.132.245.174 \ 11 | --etcd_servers=http://127.0.0.1:4001 \ 12 | --logtostderr=true 13 | Restart=on-failure 14 | RestartSec=1 15 | 16 | [Install] 17 | WantedBy=multi-user.target 18 | -------------------------------------------------------------------------------- /master/proxy.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/proxy 3 | Description=Kubernetes Proxy 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/proxy --etcd_servers=http://127.0.0.1:4001 --logtostderr=true 8 | Restart=on-failure 9 | RestartSec=1 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | -------------------------------------------------------------------------------- /master/rudder.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Requires=etcd.service 3 | After=etcd.service 4 | 5 | [Service] 6 | ExecStartPre=-/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 7 | ExecStart=/opt/bin/rudder -iface=10.132.245.174 8 | 9 | [Install] 10 | WantedBy=multi-user.target 11 | -------------------------------------------------------------------------------- /minion1/docker.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker Application Container Engine 3 | Documentation=http://docs.docker.io 4 | Requires=rudder.service 5 | After=rudder.service 6 | 7 | [Service] 8 | EnvironmentFile=/run/rudder/subnet.env 9 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 10 | ExecStartPre=-/usr/sbin/brctl delbr docker0 11 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${RUDDER_SUBNET} --mtu=${RUDDER_MTU} 12 | Restart=on-failure 13 | RestartSec=5 14 | 15 | [Install] 16 | WantedBy=multi-user.target 17 | -------------------------------------------------------------------------------- /minion1/kubelet.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/kubelet 3 | Description=Kubernetes Kubelet 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/kubelet \ 8 | --address=10.132.245.179 \ 9 | --port=10250 \ 10 | --hostname_override=10.132.245.179 \ 11 | --etcd_servers=http://127.0.0.1:4001 \ 12 | --logtostderr=true 13 | Restart=on-failure 14 | RestartSec=1 15 | 16 | [Install] 17 | WantedBy=multi-user.target 18 | -------------------------------------------------------------------------------- /minion1/proxy.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/proxy 3 | Description=Kubernetes Proxy 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/proxy --etcd_servers=http://127.0.0.1:4001 --logtostderr=true 8 | Restart=on-failure 9 | RestartSec=1 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | 14 | -------------------------------------------------------------------------------- /minion1/rudder.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Requires=etcd.service 3 | After=etcd.service 4 | 5 | [Service] 6 | ExecStartPre=-/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 7 | ExecStart=/opt/bin/rudder -iface=10.132.245.179 8 | 9 | [Install] 10 | WantedBy=multi-user.target 11 | -------------------------------------------------------------------------------- /minion2/docker.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker Application Container Engine 3 | Documentation=http://docs.docker.io 4 | Requires=rudder.service 5 | After=rudder.service 6 | 7 | [Service] 8 | EnvironmentFile=/run/rudder/subnet.env 9 | ExecStartPre=-/usr/bin/ip link set dev docker0 down 10 | ExecStartPre=-/usr/sbin/brctl delbr docker0 11 | ExecStart=/usr/bin/docker -d -s=btrfs -H fd:// --bip=${RUDDER_SUBNET} --mtu=${RUDDER_MTU} 12 | Restart=on-failure 13 | RestartSec=5 14 | 15 | [Install] 16 | WantedBy=multi-user.target 17 | -------------------------------------------------------------------------------- /minion2/kubelet.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/kubelet 3 | Description=Kubernetes Kubelet 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/kubelet \ 8 | --address=10.132.245.177 \ 9 | --port=10250 \ 10 | --hostname_override=10.132.245.177 \ 11 | --etcd_servers=http://127.0.0.1:4001 \ 12 | --logtostderr=true 13 | Restart=on-failure 14 | RestartSec=1 15 | 16 | [Install] 17 | WantedBy=multi-user.target 18 | -------------------------------------------------------------------------------- /minion2/proxy.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFileIsExecutable=/opt/bin/proxy 3 | Description=Kubernetes Proxy 4 | Documentation=https://github.com/GoogleCloudPlatform/kubernetes 5 | 6 | [Service] 7 | ExecStart=/opt/bin/proxy --etcd_servers=http://127.0.0.1:4001 --logtostderr=true 8 | Restart=on-failure 9 | RestartSec=1 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | 14 | -------------------------------------------------------------------------------- /minion2/rudder.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Requires=etcd.service 3 | After=etcd.service 4 | 5 | [Service] 6 | ExecStartPre=-/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16"}' 7 | ExecStart=/opt/bin/rudder -iface=10.132.245.177 8 | 9 | [Install] 10 | WantedBy=multi-user.target 11 | -------------------------------------------------------------------------------- /selectimage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bketelsen/coreos-kubernetes-digitalocean/3e9437ecf254173a31e62c51b05bfb4ac1975398/selectimage.png -------------------------------------------------------------------------------- /settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bketelsen/coreos-kubernetes-digitalocean/3e9437ecf254173a31e62c51b05bfb4ac1975398/settings.png --------------------------------------------------------------------------------