├── LICENSE ├── cluster-petset-portworx ├── couchbase-petset.yml ├── petset.yaml ├── portworx.sh ├── pv-claim.yaml ├── pv.yaml └── readme.adoc ├── cluster-petset ├── Dockerfile ├── configure-node.sh ├── couchbase-petset.yml ├── couchbase-service.yml └── readme.adoc ├── cluster-statefulset ├── Dockerfile ├── configure-node.sh ├── couchbase-service.yml ├── couchbase-statefulset.yml └── readme.adoc ├── cluster ├── Dockerfile ├── client-service.yml ├── cluster-master-openshift.yml ├── cluster-master.yml ├── cluster-worker.yml ├── configure-node.sh └── readme.adoc ├── couchbase-pod.yml ├── couchbase-rc.yml ├── couchbase-service.yml ├── pv ├── couchbase-pv.yml ├── couchbase-pvc.yml ├── couchbase-rc.yml └── readme.adoc └── readme.adoc /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /cluster-petset-portworx/couchbase-petset.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: couchbase 5 | labels: 6 | app: couchbase 7 | spec: 8 | ports: 9 | - port: 8091 10 | name: couchbase 11 | # *.couchbase.default.svc.cluster.local 12 | clusterIP: None 13 | selector: 14 | app: couchbase 15 | --- 16 | apiVersion: v1 17 | kind: Service 18 | metadata: 19 | name: couchbase-ui 20 | labels: 21 | app: couchbase-ui 22 | spec: 23 | ports: 24 | - port: 8091 25 | name: couchbase 26 | selector: 27 | app: couchbase 28 | # sessionAffinity: ClientIP 29 | type: LoadBalancer 30 | --- 31 | apiVersion: apps/v1alpha1 32 | kind: PetSet 33 | metadata: 34 | name: couchbase 35 | spec: 36 | serviceName: "couchbase" 37 | replicas: 2 38 | template: 39 | metadata: 40 | labels: 41 | app: couchbase 42 | annotations: 43 | pod.alpha.kubernetes.io/initialized: "true" 44 | spec: 45 | terminationGracePeriodSeconds: 0 46 | containers: 47 | - name: couchbase 48 | image: saturnism/couchbase:k8s-petset 49 | ports: 50 | - containerPort: 8091 51 | volumeMounts: 52 | - name: couchbase-claim 53 | mountPath: /opt/couchbase/var 54 | env: 55 | - name: COUCHBASE_MASTER 56 | value: "couchbase-0.couchbase.default.svc.cluster.local" 57 | - name: AUTO_REBALANCE 58 | value: "false" 59 | volumeClaimTemplates: 60 | - metadata: 61 | name: couchbase-claim 62 | spec: 63 | accessModes: [ "ReadWriteOnce" ] 64 | resources: 65 | requests: 66 | storage: 1Gi 67 | -------------------------------------------------------------------------------- /cluster-petset-portworx/petset.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: nginx 5 | labels: 6 | app: nginx 7 | spec: 8 | ports: 9 | - port: 80 10 | name: petset-pwx 11 | # *.nginx.default.svc.cluster.local 12 | clusterIP: None 13 | selector: 14 | app: nginx 15 | --- 16 | apiVersion: apps/v1alpha1 17 | kind: PetSet 18 | metadata: 19 | name: petset-pwx 20 | spec: 21 | serviceName: "nginx" 22 | replicas: 23 | template: 24 | metadata: 25 | labels: 26 | app: nginx 27 | annotations: 28 | pod.alpha.kubernetes.io/initialized: "true" 29 | spec: 30 | terminationGracePeriodSeconds: 0 31 | containers: 32 | - name: nginx 33 | image: gcr.io/google_containers/nginx-slim:0.8 34 | ports: 35 | - containerPort: 80 36 | name: petset 37 | volumeMounts: 38 | - name: 39 | mountPath: /data 40 | volumeClaimTemplates: 41 | - metadata: 42 | name: 43 | spec: 44 | accessModes: [ "ReadWriteOnce" ] 45 | resources: 46 | requests: 47 | storage: 48 | -------------------------------------------------------------------------------- /cluster-petset-portworx/portworx.sh: -------------------------------------------------------------------------------- 1 | ssh -i ~/.ssh/kube_aws_rsa -o StrictHostKeyChecking=no ubuntu@$1 "sudo sed -i 's/MountFlags/#MountFlags/g' /lib/systemd/system/docker.service; \ 2 | sudo systemctl daemon-reload; \ 3 | sudo systemctl restart docker; \ 4 | sudo docker run --restart=always --name px -d --net=host \ 5 | --privileged=true \ 6 | -v /run/docker/plugins:/run/docker/plugins \ 7 | -v /var/lib/osd:/var/lib/osd:shared \ 8 | -v /dev:/dev \ 9 | -v /etc/pwx:/etc/pwx \ 10 | -v /opt/pwx/bin:/export_bin \ 11 | -v /usr/libexec/kubernetes/kubelet-plugins/volume/exec/px~flexvolume:/export_flexvolume:shared \ 12 | -v /var/run/docker.sock:/var/run/docker.sock \ 13 | -v /var/cores:/var/cores \ 14 | -v /var/lib/kubelet:/var/lib/kubelet:shared \ 15 | --ipc=host \ 16 | portworx/px-enterprise -k etcd:http://etcd-us-east-1b.portworx.com:4001 -c $2 -a -f" 17 | -------------------------------------------------------------------------------- /cluster-petset-portworx/pv-claim.yaml: -------------------------------------------------------------------------------- 1 | kind: PersistentVolumeClaim 2 | apiVersion: v1 3 | metadata: 4 | name: pv-claim 5 | spec: 6 | accessModes: 7 | - ReadWriteOnce 8 | resources: 9 | requests: 10 | storage: 11 | -------------------------------------------------------------------------------- /cluster-petset-portworx/pv.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolume 3 | metadata: 4 | name: pv-pwx1 5 | spec: 6 | capacity: 7 | storage: 1Gi 8 | accessModes: 9 | - ReadWriteOnce 10 | persistentVolumeReclaimPolicy: Recycle 11 | flexVolume: 12 | driver: "px/flexvolume" 13 | fsType: "ext4" 14 | options: 15 | volumeID: "couchbase1" 16 | size: "1Gi" 17 | osdDriver: "pxd" 18 | --- 19 | apiVersion: v1 20 | kind: PersistentVolume 21 | metadata: 22 | name: pv-pwx2 23 | spec: 24 | capacity: 25 | storage: 1Gi 26 | accessModes: 27 | - ReadWriteOnce 28 | persistentVolumeReclaimPolicy: Recycle 29 | flexVolume: 30 | driver: "px/flexvolume" 31 | fsType: "ext4" 32 | options: 33 | volumeID: "couchbase2" 34 | size: "1Gi" 35 | osdDriver: "pxd" 36 | -------------------------------------------------------------------------------- /cluster-petset-portworx/readme.adoc: -------------------------------------------------------------------------------- 1 | = Couchbase Cluster with PetSet and Portworx 2 | 3 | == Why https://portworx.com/[Portworx]? 4 | 5 | . *Container granular volumes* - Portworx can take multiple EBS volumes per host and aggregate the capacity and derive container granular virtual (soft) volumes per container. 6 | . *Cross Availability Zone HA* - Portworx will protect the data, at block level, across multiple compute instances across availability zones. As replication controllers restart pods on different nodes, the data will still be highly available on those nodes. 7 | . *Support for enterprise data operations* - Portworx implements container granular snapshots, class of service, tiering on top of the available physical volumes. 8 | . *Ease of deployment and provisioning* - Portworx itself is deployed as a container and integrated with the orchestration tools. DevOps can programmatically provision container granular storage with any property such as size, class of service, encryption key etc. 9 | 10 | == Portworx on Kubernetes 11 | 12 | . Start Kubernetes cluster: 13 | + 14 | ``` 15 | export KUBERNETES_PROVIDER=aws 16 | KUBE_OS_DISTRIBUTION=wily NODE_SIZE=m3.medium MASTER_SIZE=m3.medium ./cluster/kube-up.sh 17 | ``` 18 | + 19 | .. Default distribution for Kubernetes cluster uses Debian operating system. There were issues starting Portworx container on it (TODO: Add bug link). 20 | .. Default `NODE_SIZE` is `t2.medium`. This is not sufficient for running Couchbase and PWX container. 21 | . Make sure AWS CLI is configured correctly for `us-west-2` region. Use the command `aws configure`, in case you need to. 22 | . Create 4 EBS volumes one for each node: 23 | + 24 | ``` 25 | aws ec2 create-volume --region us-west-2 --availability-zone us-west-2a --volume-type io1 --iops 1000 --size 128 26 | ``` 27 | + 28 | . Get the volume ids 29 | + 30 | ``` 31 | aws ec2 describe-volumes --filters "Name=size, Values=128" --query 'Volumes[*].{ID:VolumeId}' 32 | ``` 33 | + 34 | This will show the result: 35 | + 36 | ``` 37 | [ 38 | { 39 | "ID": "vol-ce734146" 40 | }, 41 | { 42 | "ID": "vol-cf734147" 43 | }, 44 | { 45 | "ID": "vol-df734157" 46 | }, 47 | { 48 | "ID": "vol-ec734164" 49 | } 50 | ] 51 | ``` 52 | + 53 | . Get the instance ids for all minions 54 | + 55 | ``` 56 | aws ec2 describe-instances --filters "Name=tag:Role, Values=kubernetes-minion" --query 'Reservations[*].Instances[*].InstanceId' 57 | ``` 58 | + 59 | This will show output as: 60 | + 61 | ``` 62 | [ 63 | [ 64 | "i-b57ee620", 65 | "i-b47ee621", 66 | "i-b37ee626", 67 | "i-b27ee627" 68 | ] 69 | ] 70 | ``` 71 | + 72 | . Attach a volume to each instance: 73 | + 74 | ``` 75 | aws ec2 attach-volume --volume-id --instance-id --device /dev/sdf 76 | ``` 77 | + 78 | For example: 79 | + 80 | ``` 81 | aws ec2 attach-volume --volume-id vol-df734157 --instance-id i-b27ee627 --device /dev/sdk 82 | aws ec2 attach-volume --volume-id vol-ce734146 --instance-id i-b37ee626 --device /dev/sdk 83 | aws ec2 attach-volume --volume-id vol-ec734164 --instance-id i-b47ee621 --device /dev/sdk 84 | aws ec2 attach-volume --volume-id vol-cf734147 --instance-id i-b57ee620 --device /dev/sdk 85 | ``` 86 | + 87 | . Kubernetes 1.4.5 uses Docker 1.11.2. Since Portworx itself runs as a container, it requires `MountFlags=shared` flag (or `MountFlags=slave` to be not set) in order for it to export volumes to other containers. More details at https://github.com/docker/docker/pull/17034. 88 | + 89 | This works OOTB on Docker 1.12. For a seamless experience, will have to wait for Kubernetes to pick up that release. 90 | + 91 | .. Get public IP addresses of all minions: 92 | + 93 | ``` 94 | aws ec2 describe-instances --filters "Name=tag:Role, Values=kubernetes-minion" --query 'Reservations[*].Instances[*].PublicDnsName' 95 | ``` 96 | + 97 | .. For each public IP address: 98 | + 99 | ``` 100 | portworx.sh 101 | ``` 102 | + 103 | .. Check status of the provisioned storage: 104 | + 105 | ``` 106 | ssh -i ~/.ssh/kube_aws_rsa -o StrictHostKeyChecking=no ubuntu@ "sudo /opt/pwx/bin/pxctl status" 107 | Status: PX is operational 108 | Node ID: aa8b5ce9-dc23-4b98-b5c4-491b9edb619e 109 | IP: 172.20.0.229 110 | Local Storage Pool: 1 pool 111 | Pool Cos Size Used Status Zone Region 112 | 0 COS_TYPE_LOW 128 GiB 2.0 GiB Online a us-west-2 113 | Local Storage Devices: 1 device 114 | Device Path Media Type Size Last-Scan 115 | 0:1 /dev/xvdk STORAGE_MEDIUM_SSD 128 GiB 07 Dec 16 00:30 UTC 116 | total - 128 GiB 117 | Cluster Summary 118 | Cluster ID: couchbase_portworx2 119 | Node IP: 172.20.0.228 - Capacity: 2.0 GiB/128 GiB Online 120 | Node IP: 172.20.0.231 - Capacity: 2.0 GiB/128 GiB Online 121 | Node IP: 172.20.0.230 - Capacity: 2.0 GiB/128 GiB Online 122 | Node IP: 172.20.0.229 - Capacity: 2.0 GiB/128 GiB Online (This node) 123 | Global Storage Pool 124 | Total Used : 8.1 GiB 125 | Total Capacity : 512 GiB 126 | ``` 127 | 128 | == Couchbase with Portworx on Kubernetes 129 | 130 | . Create 2 PWX volumes (on any Kubernetes worker host) - volumes are visible cluster-wide 131 | + 132 | ``` 133 | sudo /opt/pwx/bin/pxctl volume create couchbase1 134 | sudo /opt/pwx/bin/pxctl volume create couchbase2 135 | ``` 136 | + 137 | . Create 2 PV 138 | + 139 | ``` 140 | ./cluster/kubectl.sh create -f /Users/arungupta/workspaces/couchbase-kubernetes/cluster-petset-portworx/pv.yaml 141 | ``` 142 | + 143 | . Create Couchbase Petset 144 | + 145 | ``` 146 | ./cluster/kubectl.sh create -f /Users/arungupta/workspaces/couchbase-kubernetes/cluster-petset-portworx/couchbase-petset.yaml 147 | ``` 148 | + 149 | . Describe service using `kubectl.sh describe service/couchbase-ui`. Couchbase UI should be accessible at `LoadBalancer Ingress` value. 150 | 151 | == Misc 152 | 153 | . Optional verification 154 | .. Log in to minion: `ssh -i ~/.ssh/kube_aws_rsa admin@` 155 | .. Verify etcd: `curl -fs -X PUT "http://:2379/v2/keys/_test"` 156 | -------------------------------------------------------------------------------- /cluster-petset/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM couchbase 2 | 3 | COPY configure-node.sh /opt/couchbase 4 | 5 | CMD ["/opt/couchbase/configure-node.sh"] 6 | 7 | -------------------------------------------------------------------------------- /cluster-petset/configure-node.sh: -------------------------------------------------------------------------------- 1 | set -m 2 | 3 | /entrypoint.sh couchbase-server & 4 | 5 | sleep 15 6 | 7 | # Setup index and memory quota 8 | curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300 9 | 10 | # Setup services 11 | curl -v http://127.0.0.1:8091/node/controller/setupServices -d services=kv%2Cn1ql%2Cindex 12 | 13 | # Setup credentials 14 | curl -v http://127.0.0.1:8091/settings/web -d port=8091 -d username=Administrator -d password=password 15 | 16 | # Setup Memory Optimized Indexes 17 | curl -i -u Administrator:password -X POST http://127.0.0.1:8091/settings/indexes -d 'storageMode=memory_optimized' 18 | 19 | # Load travel-sample bucket 20 | curl -v -u Administrator:password -X POST http://127.0.0.1:8091/sampleBuckets/install -d '["travel-sample"]' 21 | 22 | if [[ "$HOSTNAME" == *-0 ]]; then 23 | TYPE="MASTER" 24 | else 25 | TYPE="WORKER" 26 | fi 27 | 28 | echo "Type: $TYPE" 29 | 30 | if [ "$TYPE" = "WORKER" ]; then 31 | sleep 15 32 | 33 | IP=`hostname -I` 34 | 35 | echo "Auto Rebalance: $AUTO_REBALANCE" 36 | if [ "$AUTO_REBALANCE" = "true" ]; then 37 | couchbase-cli rebalance --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 38 | else 39 | couchbase-cli server-add --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 40 | fi; 41 | fi; 42 | 43 | fg 1 44 | 45 | -------------------------------------------------------------------------------- /cluster-petset/couchbase-petset.yml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1alpha1 2 | kind: PetSet 3 | metadata: 4 | name: couchbase 5 | spec: 6 | serviceName: "couchbase" 7 | replicas: 2 8 | template: 9 | metadata: 10 | labels: 11 | app: couchbase 12 | annotations: 13 | pod.alpha.kubernetes.io/initialized: "true" 14 | spec: 15 | terminationGracePeriodSeconds: 0 16 | containers: 17 | - name: couchbase 18 | image: saturnism/couchbase:k8s-petset 19 | ports: 20 | - containerPort: 8091 21 | volumeMounts: 22 | - name: couchbase-data 23 | mountPath: /opt/couchbase/var 24 | env: 25 | - name: COUCHBASE_MASTER 26 | value: "couchbase-0.couchbase.default.svc.cluster.local" 27 | - name: AUTO_REBALANCE 28 | value: "false" 29 | volumeClaimTemplates: 30 | - metadata: 31 | name: couchbase-data 32 | annotations: 33 | volume.alpha.kubernetes.io/storage-class: anything 34 | spec: 35 | accessModes: [ "ReadWriteOnce" ] 36 | resources: 37 | requests: 38 | storage: 1Gi 39 | -------------------------------------------------------------------------------- /cluster-petset/couchbase-service.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: couchbase 5 | labels: 6 | app: couchbase 7 | spec: 8 | ports: 9 | - port: 8091 10 | name: couchbase 11 | # *.couchbase.default.svc.cluster.local 12 | clusterIP: None 13 | selector: 14 | app: couchbase 15 | --- 16 | apiVersion: v1 17 | kind: Service 18 | metadata: 19 | name: couchbase-ui 20 | labels: 21 | app: couchbase-ui 22 | spec: 23 | ports: 24 | - port: 8091 25 | name: couchbase 26 | selector: 27 | app: couchbase 28 | sessionAffinity: ClientIP 29 | type: LoadBalancer 30 | -------------------------------------------------------------------------------- /cluster-petset/readme.adoc: -------------------------------------------------------------------------------- 1 | = Couchbase Cluster on Kubernetes 2 | 3 | The files in this directory can serve as an example of how to deploy a Couchbase Server cluster on Kubernetes. Tested on GKE, Amazon as well as a self-deployed Kubernetes cluster. 4 | 5 | == Usage 6 | 7 | It should be noted at the beginning that this setup depends on KubeDNS to be enabled and functional to successfully deploy. 8 | 9 | . Create the Service 10 | + 11 | ``` 12 | kubectl apply -f couchbase-service.yml 13 | ``` 14 | + 15 | This file will provision a LoadBalancer for the Web Console of the master node as well. 16 | + 17 | 18 | . Create the PetSet 19 | + 20 | ``` 21 | kubectl apply -f couchbase-petset.yml 22 | ``` 23 | + 24 | + 25 | . Once the master is up and LB has been provisioned, connect to the WebUI by going to `$LOADBALANCER_IP:8091` in your browser 26 | . Create either auto-rebalancing nodes or just attach the nodes to the cluster with one of the two worker files: 27 | 28 | . Scale workers with the normal scaling commands for the PetSet 29 | + 30 | ``` 31 | kubectl patch petset couchbase -p '{"spec":{"replicas":2}}' 32 | ``` 33 | 34 | === Build your own image 35 | 36 | . Create the image with `docker build -t $DESIRED_IMG_NAME .` 37 | . Update all `.yml` files with your new image name if necessary. 38 | -------------------------------------------------------------------------------- /cluster-statefulset/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM couchbase 2 | 3 | COPY configure-node.sh /opt/couchbase 4 | 5 | CMD ["/opt/couchbase/configure-node.sh"] 6 | 7 | -------------------------------------------------------------------------------- /cluster-statefulset/configure-node.sh: -------------------------------------------------------------------------------- 1 | set -m 2 | 3 | /entrypoint.sh couchbase-server & 4 | 5 | sleep 15 6 | 7 | # Setup index and memory quota 8 | curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300 9 | 10 | # Setup services 11 | curl -v http://127.0.0.1:8091/node/controller/setupServices -d services=kv%2Cn1ql%2Cindex 12 | 13 | # Setup credentials 14 | curl -v http://127.0.0.1:8091/settings/web -d port=8091 -d username=Administrator -d password=password 15 | 16 | # Setup Memory Optimized Indexes 17 | curl -i -u Administrator:password -X POST http://127.0.0.1:8091/settings/indexes -d 'storageMode=memory_optimized' 18 | 19 | # Load travel-sample bucket 20 | curl -v -u Administrator:password -X POST http://127.0.0.1:8091/sampleBuckets/install -d '["travel-sample"]' 21 | 22 | if [[ "$HOSTNAME" == *-0 ]]; then 23 | TYPE="MASTER" 24 | else 25 | TYPE="WORKER" 26 | fi 27 | 28 | echo "Type: $TYPE" 29 | 30 | if [ "$TYPE" = "WORKER" ]; then 31 | sleep 15 32 | 33 | IP=`hostname -I` 34 | 35 | echo "Auto Rebalance: $AUTO_REBALANCE" 36 | if [ "$AUTO_REBALANCE" = "true" ]; then 37 | couchbase-cli rebalance --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 38 | else 39 | couchbase-cli server-add --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 40 | fi; 41 | fi; 42 | 43 | fg 1 44 | 45 | -------------------------------------------------------------------------------- /cluster-statefulset/couchbase-service.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: couchbase 5 | labels: 6 | app: couchbase 7 | spec: 8 | ports: 9 | - port: 8091 10 | name: couchbase 11 | # *.couchbase.default.svc.cluster.local 12 | clusterIP: None 13 | selector: 14 | app: couchbase 15 | --- 16 | apiVersion: v1 17 | kind: Service 18 | metadata: 19 | name: couchbase-ui 20 | labels: 21 | app: couchbase-ui 22 | spec: 23 | ports: 24 | - port: 8091 25 | name: couchbase 26 | selector: 27 | app: couchbase 28 | sessionAffinity: ClientIP 29 | type: LoadBalancer 30 | -------------------------------------------------------------------------------- /cluster-statefulset/couchbase-statefulset.yml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1beta1 2 | kind: StatefulSet 3 | metadata: 4 | name: couchbase 5 | spec: 6 | serviceName: "couchbase" 7 | replicas: 2 8 | template: 9 | metadata: 10 | labels: 11 | app: couchbase 12 | spec: 13 | terminationGracePeriodSeconds: 0 14 | containers: 15 | - name: couchbase 16 | image: saturnism/couchbase:k8s-petset 17 | ports: 18 | - containerPort: 8091 19 | volumeMounts: 20 | - name: couchbase-data 21 | mountPath: /opt/couchbase/var 22 | env: 23 | - name: COUCHBASE_MASTER 24 | value: "couchbase-0.couchbase.default.svc.cluster.local" 25 | - name: AUTO_REBALANCE 26 | value: "false" 27 | volumeClaimTemplates: 28 | - metadata: 29 | name: couchbase-data 30 | annotations: 31 | volume.alpha.kubernetes.io/storage-class: anything 32 | spec: 33 | accessModes: [ "ReadWriteOnce" ] 34 | resources: 35 | requests: 36 | storage: 1Gi 37 | -------------------------------------------------------------------------------- /cluster-statefulset/readme.adoc: -------------------------------------------------------------------------------- 1 | = Couchbase Cluster on Kubernetes 2 | 3 | The files in this directory can serve as an example of how to deploy a Couchbase Server cluster on Kubernetes. Tested on GKE, Amazon as well as a self-deployed Kubernetes cluster. 4 | 5 | == Usage 6 | 7 | It should be noted at the beginning that this setup depends on KubeDNS to be enabled and functional to successfully deploy. 8 | 9 | . Create the Service 10 | + 11 | ``` 12 | kubectl apply -f couchbase-service.yml 13 | ``` 14 | + 15 | This file will provision a LoadBalancer for the Web Console of the master node as well. 16 | + 17 | 18 | . Create the StatefulSet 19 | + 20 | ``` 21 | kubectl apply -f couchbase-statefulset.yml 22 | ``` 23 | + 24 | + 25 | . Once the master is up and LB has been provisioned, connect to the WebUI by going to `$LOADBALANCER_IP:8091` in your browser 26 | . Create either auto-rebalancing nodes or just attach the nodes to the cluster with one of the two worker files: 27 | 28 | . Scale workers with the normal scaling commands for the StatefulSet 29 | + 30 | ``` 31 | kubectl scale statefulset couchbase --replicas=3 32 | ``` 33 | 34 | === Build your own image 35 | 36 | . Create the image with `docker build -t $DESIRED_IMG_NAME .` 37 | . Update all `.yml` files with your new image name if necessary. 38 | -------------------------------------------------------------------------------- /cluster/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM couchbase 2 | 3 | COPY configure-node.sh /opt/couchbase 4 | 5 | CMD ["/opt/couchbase/configure-node.sh"] 6 | 7 | -------------------------------------------------------------------------------- /cluster/client-service.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: client-rc 5 | spec: 6 | replicas: 1 7 | selector: 8 | app: client-pod 9 | template: 10 | metadata: 11 | labels: 12 | app: client-pod 13 | spec: 14 | containers: 15 | - name: client 16 | image: arungupta/couchbase-javaee 17 | env: 18 | - name: COUCHBASE_URI 19 | value: couchbase-master-service 20 | ports: 21 | - containerPort: 8080 22 | --- 23 | apiVersion: v1 24 | kind: Service 25 | metadata: 26 | name: client-service 27 | labels: 28 | app: client-service 29 | spec: 30 | ports: 31 | - port: 8080 32 | selector: 33 | app: client-pod 34 | type: LoadBalancer 35 | -------------------------------------------------------------------------------- /cluster/cluster-master-openshift.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase-master-rc 5 | spec: 6 | replicas: 1 7 | selector: 8 | app: couchbase-master-pod 9 | template: 10 | metadata: 11 | labels: 12 | app: couchbase-master-pod 13 | spec: 14 | containers: 15 | - name: couchbase-master 16 | image: arungupta/couchbase:k8s 17 | env: 18 | - name: TYPE 19 | value: MASTER 20 | ports: 21 | - containerPort: 8091 22 | --- 23 | apiVersion: v1 24 | kind: Service 25 | metadata: 26 | name: couchbase-master-service 27 | labels: 28 | app: couchbase-master-service 29 | spec: 30 | ports: 31 | - port: 8091 32 | selector: 33 | app: couchbase-master-pod 34 | loadBalancerIP: 10.2.2.2 35 | type: LoadBalancer 36 | -------------------------------------------------------------------------------- /cluster/cluster-master.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase-master-rc 5 | spec: 6 | replicas: 1 7 | selector: 8 | app: couchbase-master-pod 9 | template: 10 | metadata: 11 | labels: 12 | app: couchbase-master-pod 13 | spec: 14 | containers: 15 | - name: couchbase-master 16 | image: arungupta/couchbase:k8s 17 | env: 18 | - name: TYPE 19 | value: MASTER 20 | ports: 21 | - containerPort: 8091 22 | --- 23 | apiVersion: v1 24 | kind: Service 25 | metadata: 26 | name: couchbase-master-service 27 | labels: 28 | app: couchbase-master-service 29 | spec: 30 | ports: 31 | - port: 8091 32 | selector: 33 | app: couchbase-master-pod 34 | type: LoadBalancer 35 | -------------------------------------------------------------------------------- /cluster/cluster-worker.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase-worker-rc 5 | spec: 6 | replicas: 1 7 | selector: 8 | app: couchbase-worker-pod 9 | template: 10 | metadata: 11 | labels: 12 | app: couchbase-worker-pod 13 | spec: 14 | containers: 15 | - name: couchbase-worker 16 | image: arungupta/couchbase:k8s 17 | env: 18 | - name: TYPE 19 | value: "WORKER" 20 | - name: COUCHBASE_MASTER 21 | value: "couchbase-master-service" 22 | - name: AUTO_REBALANCE 23 | value: "false" 24 | ports: 25 | - containerPort: 8091 26 | -------------------------------------------------------------------------------- /cluster/configure-node.sh: -------------------------------------------------------------------------------- 1 | set -m 2 | 3 | /entrypoint.sh couchbase-server & 4 | 5 | sleep 15 6 | 7 | # Setup index and memory quota 8 | curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300 9 | 10 | # Setup services 11 | curl -v http://127.0.0.1:8091/node/controller/setupServices -d services=kv%2Cn1ql%2Cindex 12 | 13 | # Setup credentials 14 | curl -v http://127.0.0.1:8091/settings/web -d port=8091 -d username=Administrator -d password=password 15 | 16 | # Setup Memory Optimized Indexes 17 | curl -i -u Administrator:password -X POST http://127.0.0.1:8091/settings/indexes -d 'storageMode=memory_optimized' 18 | 19 | # Load travel-sample bucket 20 | curl -v -u Administrator:password -X POST http://127.0.0.1:8091/sampleBuckets/install -d '["travel-sample"]' 21 | 22 | echo "Type: $TYPE" 23 | 24 | if [ "$TYPE" = "WORKER" ]; then 25 | sleep 15 26 | 27 | IP=`hostname -I` 28 | 29 | echo "Auto Rebalance: $AUTO_REBALANCE" 30 | if [ "$AUTO_REBALANCE" = "true" ]; then 31 | couchbase-cli rebalance --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 32 | else 33 | couchbase-cli server-add --cluster=$COUCHBASE_MASTER:8091 --user=Administrator --password=password --server-add=$IP --server-add-username=Administrator --server-add-password=password 34 | fi; 35 | fi; 36 | 37 | fg 1 38 | 39 | -------------------------------------------------------------------------------- /cluster/readme.adoc: -------------------------------------------------------------------------------- 1 | = Couchbase Cluster on Kubernetes 2 | 3 | The files in this directory can serve as an example of how to deploy a Couchbase Server cluster on Kubernetes. Tested on GKE, Amazon as well as a self-deployed Kubernetes cluster. 4 | 5 | It should be noted at the beginning that this setup depends on KubeDNS to be enabled and functional to successfully deploy. 6 | 7 | == Create "`master`" node 8 | 9 | ``` 10 | kubectl create -f cluster-master.yml 11 | ``` 12 | 13 | This will provision a LoadBalancer for the Web Console of the master node. 14 | 15 | Use `kubectl describe svc couchbase-master-service` to get the IP address of `LOAD_BALANCER`. Give ~3 mins for laod balancer to be ready. Connect to the Web Console by going to `$LOADBALANCER_IP:8091` in your browser. 16 | 17 | == Attach "`worker`" node 18 | 19 | Attach worker node to the cluster: 20 | 21 | ``` 22 | kubectl create -f cluster-worker.yml 23 | ``` 24 | 25 | Rebalance the cluster using the Web Console. 26 | 27 | Alternatively, attach the node and rebalance the cluster in one command: 28 | 29 | ``` 30 | kubectl create -e AUTO_REBALANCE=true -f cluster-worker.yml 31 | ``` 32 | 33 | == Run Application 34 | 35 | Run the application 36 | 37 | ``` 38 | kubectl create -f client-service.yml 39 | ``` 40 | 41 | This will provision a Load Balancer for the WildFly application server. 42 | 43 | Describe service using `kubectl describe svc/client-service`. Access the endpoint `curl http://$LOADBALANCER_IP:8080/couchbase-javaee/resources/airline` 44 | 45 | This will show the output: 46 | 47 | ``` 48 | curl http://a5b669ab9907711e68bf502342b4e804-1085842488.us-west-2.elb.amazonaws.com:8080/couchbase-javaee/resources/airline 49 | [{"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"TQ","callsign":"TXW","name":"Texas Wings","icao":"TXW","id":10123,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"A1","callsign":"atifly","name":"Atifly","icao":"A1F","id":10226,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":null,"callsign":null,"name":"Jc royal.britannica","icao":"JRB","id":10642,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"ZQ","callsign":"LOCAIR","name":"Locair","icao":"LOC","id":10748,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"KO","callsign":"ACE AIR","name":"Alaska Central Express","icao":"AER","id":109,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"UU","callsign":"REUNION","name":"Air Austral","icao":"REU","id":1191,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}] 50 | ``` 51 | 52 | == Scale Couchbase Cluster 53 | 54 | Scale Couchbase cluster using scaling commands for worker RC: 55 | 56 | ``` 57 | kubectl scale rc couchbase-worker-rebalance-rc --replicas=3 58 | ``` 59 | 60 | == Build your own image 61 | 62 | . Create the image with `docker build -t $DESIRED_IMG_NAME .` 63 | . Update `cluster-master.yml` and `cluster-worker.yml` with your new image name, if necessary. 64 | . `arungupta/couchbase-javaee` is built from https://github.com/arun-gupta/couchbase-javaee. Use the command `mvn clean package -Pdocker`. 65 | 66 | -------------------------------------------------------------------------------- /couchbase-pod.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Pod 3 | # labels attached to this Pod 4 | metadata: 5 | name: couchbase-pod 6 | spec: 7 | containers: 8 | - name: arungupta/couchbase 9 | # Docker image that will run in this Pod 10 | image: couchbase 11 | ports: 12 | - containerPort: 8091 13 | -------------------------------------------------------------------------------- /couchbase-rc.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase-rc 5 | spec: 6 | # Two replicas of the Pod to be created 7 | replicas: 2 8 | # Identifies the label key and value on the Pod that 9 | # this Replication Controller is responsible for managing 10 | selector: 11 | app: couchbase-rc-pod 12 | # 'cookie cutter' used for creating new pods when necessary 13 | template: 14 | metadata: 15 | labels: 16 | # label key and value on the pod. These must match the selector above. 17 | app: couchbase-rc-pod 18 | spec: 19 | containers: 20 | - name: couchbase 21 | image: arungupta/couchbase 22 | ports: 23 | - containerPort: 4369 24 | - containerPort: 8091 25 | - containerPort: 8092 26 | - containerPort: 8093 27 | - containerPort: 11209 28 | - containerPort: 11210 29 | - containerPort: 11211 30 | - containerPort: 11214 31 | - containerPort: 11215 32 | -------------------------------------------------------------------------------- /couchbase-service.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase-rc 5 | spec: 6 | replicas: 1 7 | selector: 8 | app: couchbase-pod 9 | template: 10 | metadata: 11 | labels: 12 | app: couchbase-pod 13 | spec: 14 | containers: 15 | - name: couchbase 16 | image: arungupta/couchbase:k8s 17 | ports: 18 | - containerPort: 8091 19 | --- 20 | apiVersion: v1 21 | kind: Service 22 | metadata: 23 | name: couchbase-service 24 | labels: 25 | app: couchbase-service 26 | spec: 27 | ports: 28 | - port: 8091 29 | selector: 30 | app: couchbase-pod 31 | type: LoadBalancer 32 | -------------------------------------------------------------------------------- /pv/couchbase-pv.yml: -------------------------------------------------------------------------------- 1 | kind: PersistentVolume 2 | apiVersion: v1 3 | metadata: 4 | name: couchbase-pv 5 | labels: 6 | type: amazonEBS 7 | spec: 8 | capacity: 9 | storage: 5Gi 10 | accessModes: 11 | - ReadWriteOnce 12 | awsElasticBlockStore: 13 | volumeID: vol-47f59cce 14 | fsType: ext4 15 | -------------------------------------------------------------------------------- /pv/couchbase-pvc.yml: -------------------------------------------------------------------------------- 1 | kind: PersistentVolumeClaim 2 | apiVersion: v1 3 | metadata: 4 | name: couchbase-pvc 5 | labels: 6 | type: amazonEBS 7 | spec: 8 | accessModes: 9 | - ReadWriteOnce 10 | resources: 11 | requests: 12 | storage: 5Gi 13 | -------------------------------------------------------------------------------- /pv/couchbase-rc.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ReplicationController 3 | metadata: 4 | name: couchbase 5 | spec: 6 | replicas: 1 7 | template: 8 | metadata: 9 | name: couchbase-pod 10 | labels: 11 | name: couchbase-pod 12 | context: couchbase-pv 13 | spec: 14 | containers: 15 | - name: couchbase-pod 16 | image: arungupta/couchbase 17 | volumeMounts: 18 | - mountPath: "/opt/couchbase/var" 19 | name: mypd 20 | ports: 21 | - containerPort: 8091 22 | - containerPort: 8092 23 | - containerPort: 8093 24 | - containerPort: 11210 25 | volumes: 26 | - name: mypd 27 | persistentVolumeClaim: 28 | claimName: couchbase-pvc 29 | -------------------------------------------------------------------------------- /pv/readme.adoc: -------------------------------------------------------------------------------- 1 | = How to use Kubernetes Persistent Volume bound to Amazon EBS with Couchbase 2 | 3 | == Create a Volume on EBS 4 | 5 | Create a 5GB SSD volume on EBS: `aws ec2 create-volume --region us-west-2 --availability-zone us-west-2a --size 5 --volume-type gp2`. Make sure the availability zone where the Kubernetes cluster and volume is created matches. 6 | 7 | Output is: 8 | 9 | ``` 10 | { 11 | "AvailabilityZone": "us-west-2a", 12 | "Encrypted": false, 13 | "VolumeType": "gp2", 14 | "VolumeId": "vol-47f59cce", 15 | "State": "creating", 16 | "Iops": 100, 17 | "SnapshotId": "", 18 | "CreateTime": "2016-07-29T21:57:43.343Z", 19 | "Size": 5 20 | } 21 | ``` 22 | 23 | Check if the volume is available: `aws --region us-west-2 ec2 describe-volumes --volume-id vol-47f59cce`. 24 | 25 | Output is: 26 | 27 | ``` 28 | { 29 | "Volumes": [ 30 | { 31 | "AvailabilityZone": "us-west-2a", 32 | "Attachments": [], 33 | "Encrypted": false, 34 | "VolumeType": "gp2", 35 | "VolumeId": "vol-47f59cce", 36 | "State": "available", 37 | "Iops": 100, 38 | "SnapshotId": "", 39 | "CreateTime": "2016-07-29T21:57:43.343Z", 40 | "Size": 5 41 | } 42 | ] 43 | } 44 | ``` 45 | 46 | Note the unique identifier for the volume in `VolumeId` attribute. 47 | 48 | == Start Kubernetes Cluster 49 | 50 | ``` 51 | export KUBERNETES_PROVIDER=aws 52 | KUBE_AWS_ZONE=us-west-2a NODE_SIZE=m3.large NUM_NODES=3 ./kubernetes/cluster/kube-up.sh 53 | ``` 54 | 55 | Log output: 56 | 57 | ``` 58 | ... Starting cluster in us-west-2a using provider aws 59 | ... calling verify-prereqs 60 | ... calling kube-up 61 | Starting cluster using os distro: jessie 62 | Uploading to Amazon S3 63 | +++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel 64 | upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.ISohbaGM/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script 65 | Uploaded server tars: 66 | SERVER_BINARY_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-server-linux-amd64.tar.gz 67 | SALT_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-salt.tar.gz 68 | BOOTSTRAP_SCRIPT_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script 69 | INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-master 2016-07-29T15:13:35Z AIPAJF3XKLNKOXOTQOCTkubernetes-master / 70 | ROLES arn:aws:iam::598307997273:role/kubernetes-master 2016-07-29T15:13:33Z / AROAI3Q2KFBD5PCKRXCRM kubernetes-master 71 | ASSUMEROLEPOLICYDOCUMENT 2012-10-17 72 | STATEMENT sts:AssumeRole Allow 73 | PRINCIPAL ec2.amazonaws.com 74 | INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-minion 2016-07-29T15:13:39Z AIPAIYSH5DJA4UPQIP4Bkubernetes-minion / 75 | ROLES arn:aws:iam::598307997273:role/kubernetes-minion 2016-07-29T15:13:37Z / AROAIQ57MPQYSHRPQCT2Q kubernetes-minion 76 | ASSUMEROLEPOLICYDOCUMENT 2012-10-17 77 | STATEMENT sts:AssumeRole Allow 78 | PRINCIPAL ec2.amazonaws.com 79 | Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8 80 | Creating vpc. 81 | Adding tag to vpc-fa3d6c9e: Name=kubernetes-vpc 82 | Adding tag to vpc-fa3d6c9e: KubernetesCluster=kubernetes 83 | Using VPC vpc-fa3d6c9e 84 | Adding tag to dopt-3aad625e: Name=kubernetes-dhcp-option-set 85 | Adding tag to dopt-3aad625e: KubernetesCluster=kubernetes 86 | Using DHCP option set dopt-3aad625e 87 | Creating subnet. 88 | Adding tag to subnet-e11f5985: KubernetesCluster=kubernetes 89 | Using subnet subnet-e11f5985 90 | Creating Internet Gateway. 91 | Using Internet Gateway igw-5c748f38 92 | Associating route table. 93 | Creating route table 94 | Adding tag to rtb-84fcf1e0: KubernetesCluster=kubernetes 95 | Associating route table rtb-84fcf1e0 to subnet subnet-e11f5985 96 | Adding route to route table rtb-84fcf1e0 97 | Using Route Table rtb-84fcf1e0 98 | Creating master security group. 99 | Creating security group kubernetes-master-kubernetes. 100 | Adding tag to sg-91590bf7: KubernetesCluster=kubernetes 101 | Creating minion security group. 102 | Creating security group kubernetes-minion-kubernetes. 103 | Adding tag to sg-9d590bfb: KubernetesCluster=kubernetes 104 | Using master security group: kubernetes-master-kubernetes sg-91590bf7 105 | Using minion security group: kubernetes-minion-kubernetes sg-9d590bfb 106 | Creating master disk: size 20GB, type gp2 107 | Adding tag to vol-def79e57: Name=kubernetes-master-pd 108 | Adding tag to vol-def79e57: KubernetesCluster=kubernetes 109 | Allocated Elastic IP for master: 52.40.216.69 110 | Adding tag to vol-def79e57: kubernetes.io/master-ip=52.40.216.69 111 | Generating certs for alternate-names: IP:52.40.216.69,IP:172.20.0.9,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master 112 | Starting Master 113 | Adding tag to i-5a7cebf5: Name=kubernetes-master 114 | Adding tag to i-5a7cebf5: Role=kubernetes-master 115 | Adding tag to i-5a7cebf5: KubernetesCluster=kubernetes 116 | Waiting for master to be ready 117 | Attempt 1 to check for master nodeWaiting for instance i-5a7cebf5 to be running (currently pending) 118 | Sleeping for 3 seconds... 119 | Waiting for instance i-5a7cebf5 to be running (currently pending) 120 | Sleeping for 3 seconds... 121 | Waiting for instance i-5a7cebf5 to be running (currently pending) 122 | Sleeping for 3 seconds... 123 | Waiting for instance i-5a7cebf5 to be running (currently pending) 124 | Sleeping for 3 seconds... 125 | [master running] 126 | Attaching IP 52.40.216.69 to instance i-5a7cebf5 127 | Attaching persistent data volume (vol-def79e57) to master 128 | 2016-07-29T22:00:36.909Z /dev/sdb i-5a7cebf5 attaching vol-def79e57 129 | cluster "aws_kubernetes" set. 130 | user "aws_kubernetes" set. 131 | context "aws_kubernetes" set. 132 | switched to context "aws_kubernetes". 133 | user "aws_kubernetes-basic-auth" set. 134 | Wrote config for aws_kubernetes to /Users/arungupta/.kube/config 135 | Creating minion configuration 136 | Creating autoscaling group 137 | 0 minions started; waiting 138 | 0 minions started; waiting 139 | 0 minions started; waiting 140 | 0 minions started; waiting 141 | 3 minions started; ready 142 | Waiting for cluster initialization. 143 | 144 | This will continually check to see if the API for kubernetes is reachable. 145 | This might loop forever if there was some uncaught error during start 146 | up. 147 | 148 | ..........................................................................................................................................................................................................Kubernetes cluster created. 149 | Sanity checking cluster... 150 | Attempt 1 to check Docker on node @ 52.42.0.65 ...not working yet 151 | Attempt 2 to check Docker on node @ 52.42.0.65 ...not working yet 152 | Attempt 3 to check Docker on node @ 52.42.0.65 ...working 153 | Attempt 1 to check Docker on node @ 52.36.195.201 ...working 154 | Attempt 1 to check Docker on node @ 52.43.35.173 ...working 155 | 156 | Kubernetes cluster is running. The master is running at: 157 | 158 | https://52.40.216.69 159 | 160 | The user name and password to use is located in /Users/arungupta/.kube/config. 161 | 162 | ... calling validate-cluster 163 | Found 3 node(s). 164 | NAME STATUS AGE 165 | ip-172-20-0-26.us-west-2.compute.internal Ready 1m 166 | ip-172-20-0-27.us-west-2.compute.internal Ready 1m 167 | ip-172-20-0-28.us-west-2.compute.internal Ready 1m 168 | Validate output: 169 | NAME STATUS MESSAGE ERROR 170 | controller-manager Healthy ok 171 | scheduler Healthy ok 172 | etcd-0 Healthy {"health": "true"} 173 | etcd-1 Healthy {"health": "true"} 174 | Cluster validation succeeded 175 | Done, listing cluster services: 176 | 177 | Kubernetes master is running at https://52.40.216.69 178 | Elasticsearch is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging 179 | Heapster is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/heapster 180 | Kibana is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/kibana-logging 181 | KubeDNS is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/kube-dns 182 | kubernetes-dashboard is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard 183 | Grafana is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana 184 | InfluxDB is running at https://52.40.216.69/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb 185 | 186 | To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. 187 | ``` 188 | 189 | == Create Kubernetes Artifacts 190 | 191 | === Create a Kubernetes Persistent Volume 192 | 193 | Create a Persistent Volume as `kubectl create -f couchbase-pv.yml`. Make sure to specify the volume id from the previous step in `couchbase-pv.yml`. 194 | 195 | ``` 196 | kind: PersistentVolume 197 | apiVersion: v1 198 | metadata: 199 | name: couchbase-pv 200 | labels: 201 | type: amazonEBS 202 | spec: 203 | capacity: 204 | storage: 5Gi 205 | accessModes: 206 | - ReadWriteOnce 207 | awsElasticBlockStore: 208 | volumeID: vol-47f59cce 209 | fsType: ext4 210 | ``` 211 | 212 | Output is: 213 | 214 | ``` 215 | persistentvolume "couchbase-pv" created 216 | ``` 217 | 218 | === Create Persistent Volume Claim 219 | 220 | Create a Persistent Volume Claim as `kubectl create -f couchbase-pvc.yml` 221 | 222 | ``` 223 | kind: PersistentVolumeClaim 224 | apiVersion: v1 225 | metadata: 226 | name: couchbase-pvc 227 | labels: 228 | type: amazonEBS 229 | spec: 230 | accessModes: 231 | - ReadWriteOnce 232 | resources: 233 | requests: 234 | storage: 5Gi 235 | ``` 236 | 237 | Output is: 238 | 239 | ``` 240 | persistentvolumeclaim "couchbase-pvc" created 241 | ``` 242 | 243 | === Create a Kubernetes Replication Controller 244 | 245 | Create a RC using `arungupta/couchbase` image and claiming the PVC as `kubectl create -f couchbase-rc.yml`: 246 | 247 | ``` 248 | apiVersion: v1 249 | kind: ReplicationController 250 | metadata: 251 | name: couchbase 252 | spec: 253 | replicas: 1 254 | template: 255 | metadata: 256 | name: couchbase-rc-pod 257 | labels: 258 | name: couchbase-rc-pod 259 | context: couchbase-pv 260 | spec: 261 | containers: 262 | - name: couchbase-rc-pod 263 | image: arungupta/couchbase 264 | volumeMounts: 265 | - mountPath: "/opt/couchbase/var" 266 | name: mypd 267 | ports: 268 | - containerPort: 8091 269 | - containerPort: 8092 270 | - containerPort: 8093 271 | - containerPort: 11210 272 | volumes: 273 | - name: mypd 274 | persistentVolumeClaim: 275 | claimName: couchbase-pvc 276 | ``` 277 | 278 | Output is: 279 | 280 | ``` 281 | replicationcontroller "couchbase-rc" created 282 | ``` 283 | 284 | Check for Pod: 285 | 286 | ``` 287 | ~/tools/kubernetes/kubernetes-1.3.3/kubernetes/cluster/kubectl.sh get -w po 288 | NAME READY STATUS RESTARTS AGE 289 | couchbase-jx3fn 0/1 ContainerCreating 0 3s 290 | NAME READY STATUS RESTARTS AGE 291 | couchbase-jx3fn 1/1 Running 0 20s 292 | 293 | ``` 294 | 295 | == Expose Pod on Kubernetes as Service 296 | 297 | ``` 298 | ~/tools/kubernetes/kubernetes-1.3.3/kubernetes/cluster/kubectl.sh expose rc couchbase --target-port=8091 --port=8091 --type=LoadBalancer 299 | ``` 300 | 301 | Output is: 302 | 303 | ``` 304 | service "couchbase" exposed 305 | 306 | ``` 307 | 308 | Describe the service: 309 | 310 | ``` 311 | ~/tools/kubernetes/kubernetes-1.3.3/kubernetes/cluster/kubectl.sh describe svc couchbase 312 | ``` 313 | 314 | Output is: 315 | 316 | ``` 317 | Name: couchbase 318 | Namespace: default 319 | Labels: context=couchbase-pv 320 | name=couchbase-pod 321 | Selector: context=couchbase-pv,name=couchbase-pod 322 | Type: LoadBalancer 323 | IP: 10.0.49.129 324 | LoadBalancer Ingress: a6179426155e211e6b664022b850255f-1850736155.us-west-2.elb.amazonaws.com 325 | Port: 8091/TCP 326 | NodePort: 31636/TCP 327 | Endpoints: 10.244.1.3:8091 328 | Session Affinity: None 329 | Events: 330 | FirstSeen LastSeen Count From SubobjectPath Type Reason Message 331 | --------- -------- ----- ---- ------------- -------- ------ ------- 332 | 31s 31s 1 {service-controller } Normal CreatingLoadBalancer Creating load balancer 333 | 29s 29s 1 {service-controller } Normal CreatedLoadBalancer Created load balancer 334 | ``` 335 | 336 | Wait for 3 minutes for Load Balancer to settle. 337 | 338 | == Show stateful container 339 | 340 | . Open up Couchbase Web Console at `:8091`, go to Data Buckets tab and create a new data bucket. 341 | . Get all pods: 342 | + 343 | ``` 344 | kubectl.sh get po 345 | NAME READY STATUS RESTARTS AGE 346 | couchbase-jx3fn 1/1 Running 0 7m 347 | ``` 348 | + 349 | . Delete the pod: 350 | + 351 | ``` 352 | kubectl.sh delete po couchbase-jx3fn 353 | pod "couchbase-jx3fn" deleted 354 | ``` 355 | + 356 | . Pod gets recreated: 357 | + 358 | ``` 359 | kubectl.sh get -w po 360 | NAME READY STATUS RESTARTS AGE 361 | couchbase-jx3fn 1/1 Terminating 0 8m 362 | couchbase-qq6wu 0/1 ContainerCreating 0 4s 363 | NAME READY STATUS RESTARTS AGE 364 | couchbase-qq6wu 1/1 Running 0 5s 365 | ``` 366 | + 367 | . Access Couchbase Web Console again to see that the bucket still exists. 368 | 369 | 370 | 371 | -------------------------------------------------------------------------------- /readme.adoc: -------------------------------------------------------------------------------- 1 | = Couchbase on Kubernetes 2 | 3 | == Start Kubernetes cluster 4 | 5 | ``` 6 | export KUBERNETES_PROVIDER=aws 7 | ./cluster/kube-up.sh 8 | ``` 9 | 10 | == Create Couchbase Service and Expose as LB 11 | 12 | Create a Service, Replication Controller with 1 replica, and expose all of this as a Load Balancer 13 | 14 | ``` 15 | ./cluster/kubectl.sh -f couchbase-service.yml 16 | ``` 17 | 18 | == Couchbase Cluster 19 | 20 | === Implicitly (preferred) 21 | 22 | link:cluster/readme.adoc[Couchbase Cluster using Kubernetes] 23 | 24 | === Explicitly 25 | 26 | . Create Couchbase Replication Controller with 2 replicas: 27 | + 28 | 29 | ``` 30 | ./cluster/kubectl.sh -f couchbase-rc.yml 31 | ``` 32 | + 33 | . Get pods in the RC: 34 | 35 | ./kubernetes/cluster/kubectl.sh get po 36 | 37 | . Get IP address of the two pods: 38 | 39 | ./kubernetes/cluster/kubectl.sh describe 40 | ./kubernetes/cluster/kubectl.sh describe 41 | 42 | . Create the cluster and rebalance: 43 | 44 | couchbase-cli rebalance --cluster=:8091 --user Administrator --password password --server-add= --server-add-username=Administrator --server-add-password=password 45 | 46 | == Persistent Volumes 47 | 48 | link:pv/readme.adoc[Kubernetes Persistent Volume bound to Amazon EBS with Couchbase] --------------------------------------------------------------------------------