├── README.md └── examples └── k8s_elastic_kibana_apm ├── node └── node.yaml ├── elastic ├── kibana.yaml ├── elasticsearch.yaml └── apm-server.yaml └── README.md /README.md: -------------------------------------------------------------------------------- 1 | # DevOps Resources 2 | List of resources related to cloud development used in my tutorials. 3 | 4 | # Kubernetes 5 | ## Kubernetes + Elasticsearch + Kibana + APM Server + Node.js app descriptor 6 | 7 | [Examples](/examples/k8s_elastic_kibana_apm) 8 | 9 | ## Contributing 10 | Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. 11 | 12 | ## License 13 | [MIT](https://choosealicense.com/licenses/mit/) 14 | -------------------------------------------------------------------------------- /examples/k8s_elastic_kibana_apm/node/node.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: node-app-deployment 5 | labels: 6 | app: node-app 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: node-app 12 | template: 13 | metadata: 14 | labels: 15 | app: node-app 16 | spec: 17 | containers: 18 | - name: node-app 19 | image: node-app:latest 20 | ports: 21 | - containerPort: 3000 22 | name: webinterface 23 | --- 24 | apiVersion: v1 25 | kind: Service 26 | metadata: 27 | name: node-app 28 | labels: 29 | service: node-app 30 | spec: 31 | type: NodePort 32 | ports: 33 | - port: 3000 34 | name: webinterface 35 | selector: 36 | app: node-app 37 | -------------------------------------------------------------------------------- /examples/k8s_elastic_kibana_apm/elastic/kibana.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: kibana-deployment 5 | labels: 6 | app: kibana 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: kibana 12 | template: 13 | metadata: 14 | labels: 15 | app: kibana 16 | spec: 17 | containers: 18 | - name: kibana 19 | image: docker.elastic.co/kibana/kibana:6.6.1 20 | ports: 21 | - containerPort: 5601 22 | name: webinterface 23 | --- 24 | apiVersion: v1 25 | kind: Service 26 | metadata: 27 | name: kibana 28 | labels: 29 | service: kibana 30 | spec: 31 | type: NodePort 32 | ports: 33 | - port: 5601 34 | name: webinterface 35 | selector: 36 | app: kibana 37 | -------------------------------------------------------------------------------- /examples/k8s_elastic_kibana_apm/README.md: -------------------------------------------------------------------------------- 1 | # Setup Example of Elastic stack on Kubernetes 2 | 3 | This example include all the necessary configuration to run Elasticsearch + Kibana + Elastic APM server and a Node.js app on Kubernetes. 4 | It was tested on Minikube so some configurations might change to run in a Cloud provider but it's expected that these resources will cover 5 | the most common cases. 6 | 7 | ## How to run 8 | 9 | 1. Apply/Create the kubernetes configuration: 10 | ``` 11 | $ kubectl create -f elastic/apm-server.yaml 12 | ``` 13 | ``` 14 | $ kubectl create -f elastic/elasticsearch.yaml 15 | ``` 16 | ``` 17 | $ kubectl create -f elastic/kibana.yaml 18 | ``` 19 | 20 | 2. You will have to add the proper APM for your application, this example includes a yaml for a Node.js APP. 21 | It expects a docker image called `node-app` built on the VMs host [if you don't know how to build on the VM's host have a read here](https://medium.com/@brianbmathews/getting-started-with-minikube-docker-container-images-for-testing-kubernetes-locally-on-mac-e39adb60bd41) 22 | 23 | Your application have to be writing metrics using the [Node.js Agent](https://www.elastic.co/guide/en/apm/agent/nodejs/index.html), when configuring the agent make sure that the `serverUrl` is `http://apm-server:8200` which is the APM server url on the cluster. 24 | 25 | Then once the agent is configured in your app and you've built the Docker image for the app, apply the config: 26 | 27 | ``` 28 | $ kubectl create -f node/node.yaml 29 | ``` 30 | 31 | *Note* For non Node.js apps the logic is the same, just add the proper elastic agent, point the the APM server and deploy the app on the cluster. 32 | -------------------------------------------------------------------------------- /examples/k8s_elastic_kibana_apm/elastic/elasticsearch.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | # StatefulSets intestead of Deployments for persistent data 3 | # https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ 4 | kind: StatefulSet 5 | metadata: 6 | name: elasticsearch 7 | spec: 8 | serviceName: "elasticsearch" 9 | replicas: 1 10 | selector: 11 | matchLabels: 12 | app: elasticsearch 13 | template: 14 | metadata: 15 | labels: 16 | app: elasticsearch 17 | spec: 18 | containers: 19 | - name: elasticsearch 20 | image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1 21 | env: 22 | - name: discovery.type 23 | value: single-node 24 | ports: 25 | - containerPort: 9200 26 | name: client 27 | - containerPort: 9300 28 | name: nodes 29 | volumeMounts: 30 | - name: data 31 | mountPath: /usr/share/elasticsearch/data 32 | volumes: 33 | - name: data 34 | persistentVolumeClaim: 35 | claimName: elasticsearch 36 | --- 37 | apiVersion: v1 38 | kind: Service 39 | metadata: 40 | name: elasticsearch 41 | labels: 42 | service: elasticsearch 43 | spec: 44 | ports: 45 | - port: 9200 46 | name: client 47 | - port: 9300 48 | name: nodes 49 | selector: 50 | app: elasticsearch 51 | --- 52 | apiVersion: v1 53 | kind: PersistentVolumeClaim 54 | metadata: 55 | name: elasticsearch 56 | labels: 57 | app: elasticsearch 58 | stack: logging 59 | spec: 60 | # storageClassName: standard 61 | accessModes: 62 | - ReadWriteOnce 63 | resources: 64 | requests: 65 | storage: 1G 66 | -------------------------------------------------------------------------------- /examples/k8s_elastic_kibana_apm/elastic/apm-server.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ConfigMap 4 | metadata: 5 | name: apm-server-config 6 | labels: 7 | k8s-app: apm-server 8 | data: 9 | apm-server.yml: |- 10 | apm-server: 11 | host: "0.0.0.0:8200" 12 | frontend: 13 | enabled: false 14 | setup.template.settings: 15 | index: 16 | number_of_shards: 1 17 | codec: best_compression 18 | setup.dashboards.enabled: true 19 | setup.kibana: 20 | host: "kibana" 21 | output.elasticsearch: 22 | hosts: ["elasticsearch"] 23 | indices: 24 | - index: "apm-%{[beat.version]}-sourcemap" 25 | when.contains: 26 | processor.event: "sourcemap" 27 | - index: "apm-%{[beat.version]}-error-%{+yyyy.MM.dd}" 28 | when.contains: 29 | processor.event: "error" 30 | - index: "apm-%{[beat.version]}-transaction-%{+yyyy.MM.dd}" 31 | when.contains: 32 | processor.event: "transaction" 33 | - index: "apm-%{[beat.version]}-span-%{+yyyy.MM.dd}" 34 | when.contains: 35 | processor.event: "span" 36 | --- 37 | apiVersion: v1 38 | kind: Service 39 | metadata: 40 | name: apm-server 41 | labels: 42 | app: apm-server 43 | spec: 44 | ports: 45 | - port: 8200 46 | targetPort: 8200 47 | name: http 48 | nodePort: 31000 49 | selector: 50 | app: apm-server 51 | type: NodePort 52 | --- 53 | apiVersion: extensions/v1beta1 54 | kind: Deployment 55 | metadata: 56 | name: apm-server 57 | spec: 58 | # this replicas value is default 59 | # modify it according to your case 60 | replicas: 1 61 | template: 62 | metadata: 63 | labels: 64 | app: apm-server 65 | spec: 66 | containers: 67 | - name: apm-server 68 | image: docker.elastic.co/apm/apm-server:6.5.0 69 | ports: 70 | - containerPort: 8200 71 | name: apm-port 72 | volumeMounts: 73 | - name: apm-server-config 74 | mountPath: /usr/share/apm-server/apm-server.yml 75 | readOnly: true 76 | subPath: apm-server.yml 77 | volumes: 78 | - name: apm-server-config 79 | configMap: 80 | name: apm-server-config 81 | --------------------------------------------------------------------------------