├── .gitignore ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Gregivy 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Kubernetes with Meteor. Basics 2 | draft v 1 3 | 4 | ### Intro 5 | If you are using Meteor for real world projects sooner or later you will face that: 6 | 7 | - Your app doesn't handle well the increasing the load. Meteor and primarily Node.js doesn't support multi-core parallelism so there is little use of vertical scaling. For horizontal scaling you need to split your users between separate instances of the identical app. 8 | - Deploying new versions becomes more of a struggle. 9 | - Your server becomes complicated and it will be a difficult task to create a copy of it for a similar project or for a horizontal scaling. 10 | - Some parts of your monolithic project should be divided into smaller pieces (services) perhaps even rewritten using other technological stack. 11 | - You need to operate everything by hands. 12 | 13 | Kubernetes is a great tool in dealing with all of these things and more. 14 | 15 | This guide will give a basic knowledge of how to deploy a Meteor app with Kubernetes. 16 | 17 | ### Kubernetes. Brief info 18 | Kubernetes is the well know tool, eco-system and platform created by Google ([https://kubernetes.io/](https://kubernetes.io/)). 19 | Its open-source and you can run it anywhere you want or use existing cloud services 20 | like [Google Cloud](https://cloud.google.com/kubernetes-engine/). 21 | 22 | Kubernetes is used for deploying and orchestrating applications. In heart of it lies a containerized applications mainly [docker containers](https://en.wikipedia.org/wiki/Docker_(software). 23 | 24 | Containers killing feature is the ability to encapsulate your application and the required environment (OS, other applications, system settings, etc) into a relatively small isolated chunks called *images*. 25 | Think of it as an *operating-system-level virtualization*. 26 | 27 | Kubernetes operates the containers and allows you to use different stack of technologies 28 | for your project (Meteor, plain Node.js, Java, Python and everything else). 29 | 30 | Kubernetes consists of tools and a cluster. 31 | 32 | The cluster is where your apps are going to run. The cluster works on a number of physical or cloud machines bound with network. 33 | These machines are called _Nodes_ and _Mater Nodes_. Your apps and other customs run on _Nodes_ while _Master Nodes_ are 34 | the Kubernetes cores. 35 | 36 | The tools are used to manipulate the cluster and everything inside of it. 37 | Most of the time you will create YAML files (declarative _instructions_) and pass them (_apply_) 38 | to the cluster. 39 | 40 | Here are the main Kubernetes features mentioned on the official website: 41 | - **Service discovery and load balancing**. Means that you can balance the load (traffic, for example) across your apps, give routes to your apps, etc. 42 | - **Horizontal scaling**. You can create as many active instances of your app (or even use your own script for smart scaling) as you want and your computational power allows. 43 | - **Storage orchestration**. Mount any storage system your app needs (local storage, cloud storage, network storage). 44 | - **Automated rollouts and rollbacks**. Allows you to rollout new versions of your app without any downtime and rollback to previous versions in case of an error. 45 | - **Batch execution**. Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired. 46 | - **Automatic bin packing**. Optimal usage of all the computational resources. 47 | - **Self-healing**. If a container fails Kubernetes is smart enough to create a new one which replaces the failed. 48 | - **Secret and configuration management**. You can store environment variables, sensitive information, passwords or configurations for your app separately from it. Changes in secrets doesn't require rebuilding the image. 49 | 50 | There are a lot of information about Kubernetes on the Internet, if you are not familiar with it you should 51 | start with the basics [https://kubernetes.io/docs/tutorials/kubernetes-basics/](https://kubernetes.io/docs/tutorials/kubernetes-basics/) but **it is not mandatory**. 52 | If you are going to use cloud Kubernetes providers to deploy your Meteor project this guide is enough for 53 | you to start. 54 | 55 | ### Required Terminology 56 | - **Image**. Your app inside build Docker container. 57 | - **Pod**. Instance of an Image inside Kubernetes Cluster. 58 | - **Replica set**. Controls the number of the identical Pods. 59 | - **Secret**. Encrypted information inside the Kubernetes Cluster. 60 | - **Deployment**. Your app, its version, its settings, Docker Image, required Secrets, Replica policy and more. 61 | - **Service**. Groups Pods of your app. 62 | - **Load balancer**. Splits the incoming load between Services and their Pods. 63 | 64 | ### 1. Creating a Docker Image 65 | First of all you need to install Docker on the machine where you are going to build your Meteor project. 66 | You can read about installation here [https://docs.docker.com/install/](https://docs.docker.com/install/). 67 | 68 | You can create a Docker image from scratch but we are going to use `abernix/meteord:onbuild` image as a the base image. This image was specially created for Meteor, uses Ubuntu and builds your Meteor project into Node.js automatically. 69 | 70 | You can find more Docker images for Meteor at[ Abernix hub](https://hub.docker.com/r/abernix/meteord). 71 | 72 | Make a text file with name `Dockerfile` in the root of your project. 73 | 74 | The file contents should be: 75 | ```FROM abernix/meteord:onbuild``` 76 | 77 | The`Dockerfile` can contain other useful references and commands beside the base image. 78 | [Read more](https://docs.docker.com/engine/reference/builder/). 79 | 80 | For example, you can include `graphicsmagick` into your Docker image. 81 | The `Dockerfile` should look like this: 82 | ``` 83 | FROM abernix/meteord:onbuild 84 | RUN apt-get update && apt-get install graphicsmagick -y 85 | ``` 86 | 87 | ### 2. Building an Image, Docker Hub Registry 88 | After you build your Image you have to put it somewhere so Kubernetes Cluster can use the Image to create the Pods. For this purpose exists Docker Registry. You can run your local Docker Registry, run Docker Registry on a dedicated server or even inside the Kubernetes Cluster. 89 | 90 | In this guide we are going to use existing solution _Docker Hub_. 91 | 92 | Docker Hub is like _npm_ but for containers. 93 | It has _free_ pricing plan for individuals which allows you to have *unlimited* public repositories, *1* private repository and *1* parallel build (May, 2019). 94 | 95 | [Create an account on Docker Hub](https://hub.docker.com/). You will need **username**, **email** and **password**. 96 | 97 | Now you can authenticate on your building machine with this command: 98 | ```bash 99 | docker login --username your_username --password your_password 100 | ``` 101 | 102 | To build a Meteor project run the following command in the terminal from your project's root directory: 103 | ```bash 104 | docker build -t your_username/project_name:some_tag . 105 | ``` 106 | - `.` means that Dockerfile is in the same directory. 107 | - `-t` means that we want to tag the built Image to distinguish it among different projects and builds. 108 | You can use a projects build version in place of `some_tag` like `1.0.0`. 109 | 110 | After the build you should upload the Image to Docker Registry: 111 | ```bash 112 | docker push your_username/project_name 113 | ``` 114 | 115 | ### 3. Running a Kubernetes Cluster 116 | **WIP** 117 | [https://cloud.google.com/kubernetes-engine/docs/quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart). 118 | 119 | 1. Read _Before you begin_. 120 | 2. Choose a local shell `kubectl`. 121 | 3. Configuring default settings for gcloud. 122 | 4. Creating a GKE cluster. 123 | 124 | After that you can move on reading this guide. 125 | 126 | ### 4.0 What is a Secret 127 | As we can find in the official docs: 128 | > Kubernetes secret objects let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Putting this information in a secret is safer and more flexible than putting it verbatim in a Pod Lifecycle definition or in a container image . 129 | 130 | [Read more about Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/). 131 | 132 | ### 4.1 Creating a Secret for Docker Hub 133 | To create a Secret for pulling Images from Docker Registry run: 134 | ```bash 135 | kubectl create secret docker-registry regsec \ 136 | --docker-server=https://index.docker.io/v1/ \ 137 | --docker-username=your_username \ 138 | --docker-password=your_password \ 139 | --docker-email=your_email 140 | ``` 141 | 142 | where: 143 | - **regsec** is secret's name 144 | - **docker-server** is your Private Docker Registry. (https://index.docker.io/v1/ for DockerHub) 145 | - **docker-username** is your Docker username. 146 | - **docker-password** is your Docker password. 147 | - **docker-email** is your Docker email. 148 | 149 | [Read more](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). 150 | 151 | ### 4.2 Creating Secrets for ENVs 152 | Secrets are good for storing environment variables for further use inside the containers. 153 | We going to create secrets both for **METEOR_SETTINGS** and **MONGO_URL** environment variables. 154 | 155 | Let's assume you have a file with Meteor settings called _meteor_settings.json_. 156 | ```json 157 | { 158 | "public": {}, 159 | "someThingCustom": { 160 | "first": 0, 161 | "last": 10 162 | } 163 | } 164 | ``` 165 | To create a Secret from this file run: 166 | ```bash 167 | kubectl create secret generic meteor-settings --from-file=./meteor_settings.json 168 | ``` 169 | 170 | This will create a Secret with name `my-meteor-settings` which includes only one key `meteor_settings.json`. 171 | 172 | The same way you can create a _mongo_url.txt_ with the desired url and use the same command to create the Secret. 173 | 174 | Create a file _mongo_url.txt: 175 | ``` 176 | mongodb://login:password@ip:port/db 177 | ``` 178 | 179 | Replace **login**, **password**, **ip**, **port** and **db** with your data. 180 | 181 | Run: 182 | ```bash 183 | kubectl create secret generic mongo-url --from-file=./mongo_url.txt 184 | ``` 185 | 186 | You will see how to use created ENVs in deployment section of this guide. 187 | 188 | ### 4.3 Updating a Secret 189 | If you need to update a Secret the easiest way is to delete the Secret and create it again. 190 | 191 | To delete a Secret run: 192 | ```bash 193 | kubectl delete secret secret_name 194 | ``` 195 | 196 | If you want to update a Secret created from a file you can use this command: 197 | ```bash 198 | kubectl create secret generic secret_name \ 199 | --from-file=./file.name --dry-run -o yaml | 200 | kubectl apply -f - 201 | ``` 202 | 203 | Its the same command you use to create a Secret plus you add `--dry-run -o yaml | 204 | kubectl apply -f -` in the end. 205 | 206 | ### 5. Little about the instructions 207 | As it was mentioned everything you pass to Kubernetes is a YAML instruction (even if you write 208 | this instruction in JSON or use commands from terminal it is compiled to YAML). 209 | 210 | So if you need to apply the instruction to Kubernetes from a file or update a 211 | previously applied instruction you can use one simple command for both tasks: 212 | ```bash 213 | kubectl apply -f your_file 214 | ``` 215 | 216 | I suggest to create a special folder for yml files and don't mess them with your project sources. 217 | 218 | Read more about [YAML](https://en.wikipedia.org/wiki/YAML). 219 | Read more about [different API versions of YAML instructions](https://matthewpalmer.net/kubernetes-app-developer/articles/kubernetes-apiversion-definition-guide.html). 220 | 221 | ### 6.1 Deploying a Meteor app 222 | To deploy your app you should create a Deployment. 223 | 224 | Create the following YAML file (_deployment.yml_). Everything you need is commented: 225 | ```YAML 226 | apiVersion: extensions/v1beta1 227 | kind: Deployment 228 | metadata: 229 | labels: 230 | app: my-project # your project name 231 | name: my-project-deployment # your project deployment name 232 | spec: 233 | progressDeadlineSeconds: 600 234 | replicas: 1 # number of Pods your deployment creates 235 | revisionHistoryLimit: 10 # number of old ReplicaSets to retain to allow rollback 236 | selector: 237 | matchLabels: 238 | app: my-project # your project name 239 | strategy: 240 | rollingUpdate: 241 | maxSurge: 25% # maximum number of Pods that can be created over the desired number of Pods 242 | maxUnavailable: 25% # maximum number of Pods that can be unavailable during the update process 243 | type: RollingUpdate 244 | template: 245 | metadata: 246 | labels: 247 | app: my-project # your project name 248 | spec: 249 | containers: 250 | - env: # ENV variables for your project 251 | - name: ROOT_URL 252 | value: https://my.domain/ # string value 253 | - name: MONGO_URL 254 | valueFrom: # value from previosly created Secret 255 | secretKeyRef: 256 | key: mongo_url.txt 257 | name: mongo-url 258 | - name: METEOR_SETTINGS 259 | valueFrom: # value from previosly created Secret 260 | secretKeyRef: 261 | key: meteor_settings.json 262 | name: meteor-settings 263 | image: your_username/project_name:some_tag # image of your app 264 | imagePullPolicy: Always 265 | name: my-project # your project name 266 | ports: 267 | - containerPort: 80 # which port should be open on Pods 268 | name: http-server 269 | protocol: TCP 270 | resources: {} 271 | terminationMessagePath: /dev/termination-log 272 | terminationMessagePolicy: File 273 | volumeMounts: 274 | - mountPath: /userimages 275 | name: images 276 | - mountPath: /userdocuments 277 | name: documents 278 | dnsPolicy: ClusterFirst 279 | imagePullSecrets: 280 | - name: regsec # secret with Docker Hub credentials 281 | restartPolicy: Always 282 | schedulerName: default-scheduler 283 | securityContext: {} 284 | terminationGracePeriodSeconds: 30 285 | volumes: 286 | - emptyDir: {} 287 | name: images 288 | - emptyDir: {} 289 | name: documents 290 | status: {} 291 | ``` 292 | 293 | In this example we assume that the app uses folders `/userimages` and `/userdocuments` for temp files. 294 | 295 | For this we describe `volumes` type of `emptyDir` (you can check other types of `volumes` [here](https://kubernetes.io/docs/concepts/storage/volumes/#local)): 296 | ```YAML 297 | volumes: 298 | - emptyDir: {} 299 | name: images 300 | - emptyDir: {} 301 | name: documents 302 | ``` 303 | 304 | And mount them with corresponding dir names: 305 | ```YAML 306 | volumeMounts: 307 | - mountPath: /userimages 308 | name: images 309 | - mountPath: /userdocuments 310 | name: documents 311 | ``` 312 | 313 | To create a Deployment run: 314 | ```bash 315 | kubectl apply -f deployment.yml 316 | ``` 317 | 318 | [Read more](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/). 319 | 320 | ### 6.2 Updating the app 321 | 322 | To update your app in Kubernetes cluster simply build new image with new tag and push it to Docker Hub. 323 | 324 | Then update _deployment.yml_ with new image name and tag `image: your_username/project_name:some_tag`. 325 | 326 | Run to apply changes: 327 | ```bash 328 | kubectl apply -f deployment.yml 329 | ``` 330 | 331 | ### 7. Auto scaling 332 | Kubernetes allows you to auto scale your deployment based on CPU usage. 333 | 334 | For this create the following YAML file (_autoscaler.yml_): 335 | ```YAML 336 | apiVersion: autoscaling/v1 337 | kind: HorizontalPodAutoscaler 338 | metadata: 339 | name: my-project 340 | namespace: default 341 | spec: 342 | scaleTargetRef: 343 | apiVersion: extensions/v1beta1 344 | kind: Deployment 345 | name: my-project 346 | minReplicas: 1 347 | maxReplicas: 10 348 | targetCPUUtilizationPercentage: 50 349 | ``` 350 | 351 | where: 352 | - `minReplicas` is a minimum number of Pods 353 | - `maxReplicas` is a maximum number of Pods 354 | - `targetCPUUtilizationPercentage` is a CPU utilization limit in percents when a new Pod should be created. 355 | 356 | Don't forget to replace `my-project` with the name of deployed app. 357 | 358 | To apply this YAML instruction run: 359 | ```bash 360 | kubectl apply -f autoscaler.yml 361 | ``` 362 | 363 | [Read more](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/). 364 | 365 | ### 8. Creating a Service 366 | Your app is deployed and functioning but it is invisible to outside world. 367 | 368 | To make it visible first of all you should create a corresponding Service for your app. 369 | > Service is an abstraction which defines a logical set of Pods and a policy by which to access them. 370 | 371 | [Read more.](https://kubernetes.io/docs/concepts/services-networking/service/) 372 | 373 | Create the following YAML file (_service.yml_): 374 | ```YAML 375 | kind: Service 376 | apiVersion: v1 377 | metadata: 378 | name: my-project-service 379 | spec: 380 | ports: 381 | - port: 80 382 | targetPort: http-server 383 | selector: 384 | app: my-project 385 | ``` 386 | 387 | This service provides _80_ port on your pods as _http-server_. 388 | 389 | Don't forget to replace `my-project` with the name of deployed app. 390 | 391 | To apply this YAML instruction run: 392 | ```bash 393 | kubectl apply -f service.yml 394 | ``` 395 | 396 | ### 9. Creating an Ingress Controller. Load balancing 397 | Now you app Pods are targeted by a Service. 398 | Lets make the Service accessible from outside by creating a corresponding Ingress. 399 | 400 | > Ingress is an API object that manages external access to the services in a cluster, typically HTTP. 401 | Ingress can provide load balancing, SSL termination and name-based virtual hosting. 402 | 403 | [Read more.](https://kubernetes.io/docs/concepts/services-networking/ingress/) 404 | 405 | As it was mentioned Ingress also provides load balancing between Service's Pods and SSL/TLS (HTTPS). 406 | You can achieve this by creating Ingress Controller. 407 | There are a lot of different controllers, you can find more [here](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/). 408 | 409 | In this guide we use _nginx_ based controller which is officially supported by Kubernetes team. 410 | As you might guess this Ingress Controller will use _nginx_ for load balancing and SSL/TLS termination. 411 | 412 | We will use simplest router possible which sends every HTTP request for your domain to one Service. 413 | 414 | Create the following YAML file (_ingress.yml_): 415 | ```YAML 416 | apiVersion: extensions/v1beta1 417 | kind: Ingress 418 | metadata: 419 | name: my-project-ingress 420 | annotations: 421 | kubernetes.io/ingress.class: "nginx" 422 | nginx.org/websocket-services: "my-project" 423 | nginx.ingress.kubernetes.io/proxy-body-size: 10m 424 | nginx.ingress.kubernetes.io/affinity: "cookie" 425 | nginx.ingress.kubernetes.io/session-cookie-hash: "sha1" 426 | spec: 427 | rules: 428 | - host: my.domain 429 | http: 430 | paths: 431 | - path: / 432 | backend: 433 | serviceName: my-project-service 434 | servicePort: 80 435 | ``` 436 | 437 | where: 438 | - `nginx.ingress.kubernetes.io/proxy-body-size` is a maximum size of incoming requests, 439 | `10m` means 10 megabytes. For example, if your app allows users to upload their photos, 440 | a user won't be able to upload a file with size more than 10 Mb. 441 | 442 | Don't forget to replace `my.domain`, `my-project-service`, `nginx.org/websocket-services`, 443 | `my-project` with your data. 444 | 445 | Apply file with: 446 | ```bash 447 | kubectl apply -f ingress.yml 448 | ``` 449 | 450 | Now you have to bind your domain with the load balancer's IP. 451 | 452 | To check Ingress Controller IP use command: 453 | ```bash 454 | kubectl get svc -n ingress-nginx | grep ingress | grep LoadBalancer 455 | ``` 456 | 457 | The last step is to delegate DNS _A record_ for your domain and the obtained IP. 458 | 459 | ### 10.1 HTTPS with custom certificates 460 | If you have custom TLS certificate that you want to use for HTTPS you need to create an corresponding Secret 461 | with `kubectl`: 462 | ```bash 463 | kubectl create secret tls test-secret-tls --cert=server.crt --key=server.key 464 | ``` 465 | 466 | where: 467 | - **test-secret-tls** is the name of the Secret 468 | - **server.crt** is the certificate with public key 469 | - **server.key** is the private key 470 | 471 | Now you can modify your _ingress.yml_: 472 | ```YAML 473 | apiVersion: extensions/v1beta1 474 | kind: Ingress 475 | metadata: 476 | name: test-ingress 477 | annotations: 478 | kubernetes.io/ingress.class: "nginx" 479 | nginx.org/websocket-services: "my-project" 480 | nginx.ingress.kubernetes.io/proxy-body-size: 10m 481 | nginx.ingress.kubernetes.io/affinity: "cookie" 482 | nginx.ingress.kubernetes.io/session-cookie-hash: "sha1" 483 | ingress.kubernetes.io/ssl-redirect: "true" 484 | spec: 485 | tls: 486 | - hosts: 487 | - my.domain 488 | secretName: ingress-tls-secret 489 | rules: 490 | - host: lk.nobi.services 491 | http: 492 | paths: 493 | - path: / 494 | backend: 495 | serviceName: my-project-service 496 | servicePort: 80 497 | ``` 498 | 499 | As you can see we added `tls` to `spec` and described there the host we want to accociate with the Ingress and the Secret that holds certificate: 500 | ```YAML 501 | tls: 502 | - hosts: 503 | - my.domain 504 | secretName: ingress-tls-secret 505 | ``` 506 | 507 | We have also added `ingress.kubernetes.io/ssl-redirect: "true"` to annotations section which forces HTTPS usage over HTTP. 508 | 509 | ### 10.2 HTTPS with Cert-Manager (auto Let's Encrypt certificates) 510 | **WIP** 511 | [https://docs.cert-manager.io/en/latest/tutorials/acme/quick-start/index.html](https://docs.cert-manager.io/en/latest/tutorials/acme/quick-start/index.html) 512 | 513 | ### Best practices 514 | Usually we do automated testing of our application locally but its also a good idea to check that 515 | everything works as expected in real production before it is available to the endpoint user. 516 | The suggest solution is to create a test deployment with a separate domain (test.my.domain) and Ingress 517 | and deploy your new project version to test zone first. 518 | 519 | Here you can find a guide of using basic auth with your Nginx Ingress [https://imti.co/kubernetes-ingress-basic-auth/](https://imti.co/kubernetes-ingress-basic-auth/). 520 | 521 | ### Pipelines 522 | **WIP** 523 | 524 | ### Meteor and micro-services. Programmable scaling 525 | **WIP** 526 | 527 | --------------------------------------------------------------------------------