├── LICENSE ├── README.md ├── amazon-cloudwatch ├── cloudwatch-agent-configmap.yaml.tmpl ├── cloudwatch-agent-daemonset.yaml ├── cloudwatch-agent-rbac.yaml ├── fluentd-configmap-cluster-info.yaml.tmpl ├── fluentd-configmap-fluentd-config.yaml ├── fluentd-daemonset.yaml └── fluentd-rbac.yaml ├── demo └── helm-release.yaml ├── kube-system ├── aws-load-balancer-controller.yaml.tmpl ├── cluster-autoscaler-deployment.yaml.tmpl └── cluster-autoscaler-rbac.yaml ├── kubernetes-dashboard ├── dashboard-metrics-scraper-deployment.yaml ├── dashboard-metrics-scraper-service.yaml ├── kubernetes-dashboard-configmap.yaml ├── kubernetes-dashboard-deployment.yaml ├── kubernetes-dashboard-rbac.yaml ├── kubernetes-dashboard-secrets.yaml └── kubernetes-dashboard-service.yaml ├── monitoring ├── metrics-server.yaml └── prometheus-operator.yaml └── namespaces ├── amazon-cloudwatch.yaml ├── demo.yaml ├── kubernetes-dashboard.yaml └── monitoring.yaml /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Weaveworks. All rights reserved. 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # EKS Quickstart App Dev 2 | 3 | This repo contains an initial set of cluster components to be installed and 4 | configured by [eksctl](https://eksctl.io) through GitOps. 5 | 6 | ## Components 7 | 8 | - [AWS load balancer controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/) -- to easily expose services to the World. 9 | - [Cluster autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler) -- to [automatically add/remove nodes](https://aws.amazon.com/premiumsupport/knowledge-center/eks-cluster-autoscaler-setup/) to/from your cluster based on its usage. 10 | - The autoscaler is configured as [recommended in the AWS autoscaler docs](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html#ca-deploy) 11 | - See also [the autoscaler docs for the AWS provider](https://github.com/kubernetes/autoscaler/blob/f18b65a80c6f83b35cac057c136af14871552d3c/cluster-autoscaler/cloudprovider/aws/README.md) 12 | - [Prometheus](https://prometheus.io/) (its [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/), its [operator](https://github.com/coreos/prometheus-operator), its [`node-exporter`](https://github.com/prometheus/node_exporter), [`kube-state-metrics`](https://github.com/kubernetes/kube-state-metrics), and [`metrics-server`](https://github.com/kubernetes-incubator/metrics-server)) -- for powerful metrics & alerts. 13 | - [Grafana](https://grafana.com) -- for a rich way to visualize metrics via dashboards you can create, explore, and share. 14 | - [Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) -- Kubernetes' standard dashboard. 15 | - [Fluentd](https://www.fluentd.org/) & Amazon's [CloudWatch agent](https://aws.amazon.com/cloudwatch/) -- for cluster & containers' [log collection, aggregation & analytics in CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs.html). 16 | - [podinfo](https://github.com/stefanprodan/podinfo) -- a toy demo application. 17 | 18 | ## Pre-requisites 19 | 20 | A running EKS cluster with [IAM policies](https://eksctl.io/usage/iam-policies/) for: 21 | 22 | - ALB ingress 23 | - auto-scaler 24 | - CloudWatch 25 | 26 | These policies can be added to a nodegroup by including the following `iam` options in your nodegroup config: 27 | 28 | ``` 29 | nodeGroups: 30 | - iam: 31 | withAddonPolicies: 32 | albIngress: true 33 | autoScaler: true 34 | cloudWatch: true 35 | ``` 36 | 37 | **N.B.**: policies are configured at the nodegroup level. 38 | Therefore, depending on your use case, you may want to: 39 | 40 | - add these policies to all nodegroups, 41 | - add [node selectors](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) to the ALB ingress, auto-scaler and CloudWatch pods, so that they are deployed on the nodes configured with these policies. 42 | 43 | ## How to access workloads 44 | 45 | For security reasons, this quickstart profile does not expose any workload publicly. However, should you want to access one of the workloads, various solutions are possible. 46 | 47 | ### Port-forwarding 48 | 49 | You could port-forward into a pod, so that you (and _only_ you) could access it locally. 50 | 51 | For example, for `demo/podinfo`: 52 | 53 | - run: 54 | ```console 55 | kubectl --namespace demo port-forward service/podinfo 9898:9898 56 | ``` 57 | - go to http://localhost:9898 58 | 59 | ### Ingress 60 | 61 | You could expose a service publicly, _at your own risks_, via ALB ingress. 62 | 63 | **N.B.**: the ALB ingress controller requires services: 64 | 65 | - to be of `NodePort` type, 66 | - to have the following annotations: 67 | ```yaml 68 | annotations: 69 | kubernetes.io/ingress.class: alb 70 | alb.ingress.kubernetes.io/scheme: internet-facing 71 | ``` 72 | 73 | #### `NodePort` services 74 | 75 | For any `NodePort` service: 76 | 77 | ```yaml 78 | apiVersion: extensions/v1beta1 79 | kind: Ingress 80 | metadata: 81 | name: ${name} 82 | namespace: ${namespace} 83 | annotations: 84 | kubernetes.io/ingress.class: alb 85 | alb.ingress.kubernetes.io/scheme: internet-facing 86 | labels: 87 | app: ${service-app-selector} 88 | spec: 89 | rules: 90 | - http: 91 | paths: 92 | - path: /* 93 | backend: 94 | serviceName: ${service-name} 95 | servicePort: 80 96 | ``` 97 | 98 | A few minutes after deploying the above `Ingress` object, you should be able to see the public URL for the service: 99 | ```console 100 | $ kubectl get ingress --namespace demo podinfo 101 | NAME HOSTS ADDRESS PORTS AGE 102 | podinfo * xxxxxxxx-${namespace}-${name}-xxxx-xxxxxxxxxx.${region}.elb.amazonaws.com 80 1s 103 | ``` 104 | 105 | #### `HelmRelease` objects 106 | 107 | For `HelmRelease` objects, you would have to configure `spec.values.service` and `spec.values.ingress`, e.g. for `demo/podinfo`: 108 | 109 | ```yaml 110 | apiVersion: helm.fluxcd.io/v1 111 | kind: HelmRelease 112 | metadata: 113 | name: podinfo 114 | namespace: demo 115 | spec: 116 | releaseName: podinfo 117 | chart: 118 | git: https://github.com/stefanprodan/podinfo 119 | ref: 3.0.0 120 | path: charts/podinfo 121 | values: 122 | service: 123 | enabled: true 124 | type: NodePort 125 | ingress: 126 | enabled: true 127 | annotations: 128 | kubernetes.io/ingress.class: alb 129 | alb.ingress.kubernetes.io/scheme: internet-facing 130 | path: /* 131 | ``` 132 | 133 | **N.B.**: the above `HelmRelease` 134 | 135 | - changes the type of `podinfo`'s service from its default value, `ClusterIP`, to `NodePort`, 136 | - adds the annotations required for the ALB ingress controller to expose the service, and 137 | - exposes all of `podinfo`'s URLs, so that all assets can be served over HTTP. 138 | 139 | A few minutes after deploying the above `HelmRelease` object, you should be able to see the following `Ingress` object, and the public URL for `podinfo`: 140 | 141 | ```console 142 | $ kubectl get ingress --namespace demo podinfo 143 | NAME HOSTS ADDRESS PORTS AGE 144 | podinfo * xxxxxxxx-demo-podinfo-xxxx-xxxxxxxxxx.${region}.elb.amazonaws.com 80 1s 145 | ``` 146 | 147 | ## Securing your endpoints 148 | For a production-grade deployment, it's recommended to secure your endpoints with SSL. See [Ingress annotations for SSL](https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/ingress/annotation/#ssl). 149 | 150 | Any sensitive service that needs to be exposed must have some form of authentication. To add authentication to Grafana for e.g., see [Grafana configuration](https://github.com/helm/charts/tree/master/stable/prometheus-operator#grafana). 151 | To add authentication to other components, please consult their documentation. 152 | 153 | ## Get in touch 154 | 155 | [Create an issue](https://github.com/weaveworks/eks-quickstart-app-dev/issues/new), or 156 | login to [Weave Community Slack (#eksctl)][slackchan] ([signup][slackjoin]). 157 | 158 | [slackjoin]: https://slack.weave.works/ 159 | [slackchan]: https://weave-community.slack.com/messages/eksctl/ 160 | 161 | Weaveworks follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting a Weaveworks project maintainer, or Alexis Richardson (alexis@weave.works). 162 | -------------------------------------------------------------------------------- /amazon-cloudwatch/cloudwatch-agent-configmap.yaml.tmpl: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | data: 4 | # The below configuration file is in JSON format. 5 | # Please ensure you keep it well-formed if you modify it. 6 | cwagentconfig.json: | 7 | { 8 | "logs": { 9 | "metrics_collected": { 10 | "kubernetes": { 11 | "cluster_name": "{{ .ClusterName }}", 12 | "metrics_collection_interval": 60 13 | } 14 | }, 15 | "force_flush_interval": 5 16 | } 17 | } 18 | kind: ConfigMap 19 | metadata: 20 | name: cwagentconfig 21 | namespace: amazon-cloudwatch 22 | -------------------------------------------------------------------------------- /amazon-cloudwatch/cloudwatch-agent-daemonset.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: DaemonSet 4 | metadata: 5 | name: cloudwatch-agent 6 | namespace: amazon-cloudwatch 7 | spec: 8 | selector: 9 | matchLabels: 10 | name: cloudwatch-agent 11 | template: 12 | metadata: 13 | labels: 14 | name: cloudwatch-agent 15 | spec: 16 | containers: 17 | - name: cloudwatch-agent 18 | image: amazon/cloudwatch-agent:latest 19 | imagePullPolicy: Always 20 | #ports: 21 | # - containerPort: 8125 22 | # hostPort: 8125 23 | # protocol: UDP 24 | resources: 25 | limits: 26 | cpu: 200m 27 | memory: 200Mi 28 | requests: 29 | cpu: 200m 30 | memory: 200Mi 31 | # Please don't change below envs 32 | env: 33 | - name: HOST_IP 34 | valueFrom: 35 | fieldRef: 36 | fieldPath: status.hostIP 37 | - name: HOST_NAME 38 | valueFrom: 39 | fieldRef: 40 | fieldPath: spec.nodeName 41 | - name: K8S_NAMESPACE 42 | valueFrom: 43 | fieldRef: 44 | fieldPath: metadata.namespace 45 | # Please don't change the mountPath 46 | volumeMounts: 47 | - name: cwagentconfig 48 | mountPath: /etc/cwagentconfig 49 | - name: rootfs 50 | mountPath: /rootfs 51 | readOnly: true 52 | - name: dockersock 53 | mountPath: /var/run/docker.sock 54 | readOnly: true 55 | - name: varlibdocker 56 | mountPath: /var/lib/docker 57 | readOnly: true 58 | - name: sys 59 | mountPath: /sys 60 | readOnly: true 61 | - name: devdisk 62 | mountPath: /dev/disk 63 | readOnly: true 64 | volumes: 65 | - name: cwagentconfig 66 | configMap: 67 | name: cwagentconfig 68 | - name: rootfs 69 | hostPath: 70 | path: / 71 | - name: dockersock 72 | hostPath: 73 | path: /var/run/docker.sock 74 | - name: varlibdocker 75 | hostPath: 76 | path: /var/lib/docker 77 | - name: sys 78 | hostPath: 79 | path: /sys 80 | - name: devdisk 81 | hostPath: 82 | path: /dev/disk/ 83 | terminationGracePeriodSeconds: 60 84 | serviceAccountName: cloudwatch-agent 85 | -------------------------------------------------------------------------------- /amazon-cloudwatch/cloudwatch-agent-rbac.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | name: cloudwatch-agent 6 | namespace: amazon-cloudwatch 7 | 8 | --- 9 | kind: ClusterRole 10 | apiVersion: rbac.authorization.k8s.io/v1 11 | metadata: 12 | name: cloudwatch-agent-role 13 | rules: 14 | - apiGroups: [""] 15 | resources: ["pods", "nodes", "endpoints"] 16 | verbs: ["list", "watch"] 17 | - apiGroups: ["apps"] 18 | resources: ["replicasets"] 19 | verbs: ["list", "watch"] 20 | - apiGroups: ["batch"] 21 | resources: ["jobs"] 22 | verbs: ["list", "watch"] 23 | - apiGroups: [""] 24 | resources: ["nodes/proxy"] 25 | verbs: ["get"] 26 | - apiGroups: [""] 27 | resources: ["nodes/stats", "configmaps", "events"] 28 | verbs: ["create"] 29 | - apiGroups: [""] 30 | resources: ["configmaps"] 31 | resourceNames: ["cwagent-clusterleader"] 32 | verbs: ["get","update"] 33 | 34 | --- 35 | kind: ClusterRoleBinding 36 | apiVersion: rbac.authorization.k8s.io/v1 37 | metadata: 38 | name: cloudwatch-agent-role-binding 39 | subjects: 40 | - kind: ServiceAccount 41 | name: cloudwatch-agent 42 | namespace: amazon-cloudwatch 43 | roleRef: 44 | kind: ClusterRole 45 | name: cloudwatch-agent-role 46 | apiGroup: rbac.authorization.k8s.io 47 | -------------------------------------------------------------------------------- /amazon-cloudwatch/fluentd-configmap-cluster-info.yaml.tmpl: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | data: 4 | cluster.name: {{ .ClusterName }} 5 | logs.region: {{ .Region }} 6 | # The below configuration file is in JSON format. 7 | # Please ensure you keep it well-formed if you modify it. 8 | cluster-info.json: | 9 | { 10 | "cluster": { 11 | "name": "{{ .ClusterName }}" 12 | }, 13 | "logs": { 14 | "region": "{{ .Region }}" 15 | } 16 | } 17 | kind: ConfigMap 18 | metadata: 19 | name: cluster-info 20 | namespace: amazon-cloudwatch 21 | -------------------------------------------------------------------------------- /amazon-cloudwatch/fluentd-configmap-fluentd-config.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ConfigMap 4 | metadata: 5 | name: fluentd-config 6 | namespace: amazon-cloudwatch 7 | labels: 8 | k8s-app: fluentd-cloudwatch 9 | data: 10 | fluent.conf: | 11 | @include containers.conf 12 | @include systemd.conf 13 | @include host.conf 14 | 15 | 16 | @type null 17 | 18 | containers.conf: | 19 | 20 | @type tail 21 | @id in_tail_container_logs 22 | @label @containers 23 | path /var/log/containers/*.log 24 | pos_file /var/log/fluentd-containers.log.pos 25 | tag * 26 | read_from_head true 27 | 28 | @type json 29 | time_format %Y-%m-%dT%H:%M:%S.%NZ 30 | 31 | 32 | 33 | 63 | systemd.conf: | 64 | 65 | @type systemd 66 | @id in_systemd_kubelet 67 | @label @systemd 68 | filters [{ "_SYSTEMD_UNIT": "kubelet.service" }] 69 | 70 | field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"} 71 | field_map_strict true 72 | 73 | path /var/log/journal 74 | pos_file /var/log/fluentd-journald-kubelet.pos 75 | read_from_head true 76 | tag kubelet.service 77 | 78 | 79 | 80 | @type systemd 81 | @id in_systemd_kubeproxy 82 | @label @systemd 83 | filters [{ "_SYSTEMD_UNIT": "kubeproxy.service" }] 84 | 85 | field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"} 86 | field_map_strict true 87 | 88 | path /var/log/journal 89 | pos_file /var/log/fluentd-journald-kubeproxy.pos 90 | read_from_head true 91 | tag kubeproxy.service 92 | 93 | 94 | 95 | @type systemd 96 | @id in_systemd_docker 97 | @label @systemd 98 | filters [{ "_SYSTEMD_UNIT": "docker.service" }] 99 | 100 | field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"} 101 | field_map_strict true 102 | 103 | path /var/log/journal 104 | pos_file /var/log/fluentd-journald-docker.pos 105 | read_from_head true 106 | tag docker.service 107 | 108 | 109 | 139 | host.conf: | 140 | 141 | @type tail 142 | @id in_tail_dmesg 143 | @label @hostlogs 144 | path /var/log/dmesg 145 | pos_file /var/log/dmesg.log.pos 146 | tag host.dmesg 147 | read_from_head true 148 | 149 | @type syslog 150 | 151 | 152 | 153 | 154 | @type tail 155 | @id in_tail_secure 156 | @label @hostlogs 157 | path /var/log/secure 158 | pos_file /var/log/secure.log.pos 159 | tag host.secure 160 | read_from_head true 161 | 162 | @type syslog 163 | 164 | 165 | 166 | 167 | @type tail 168 | @id in_tail_messages 169 | @label @hostlogs 170 | path /var/log/messages 171 | pos_file /var/log/messages.log.pos 172 | tag host.messages 173 | read_from_head true 174 | 175 | @type syslog 176 | 177 | 178 | 179 | 209 | -------------------------------------------------------------------------------- /amazon-cloudwatch/fluentd-daemonset.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: DaemonSet 4 | metadata: 5 | name: fluentd-cloudwatch 6 | namespace: amazon-cloudwatch 7 | labels: 8 | k8s-app: fluentd-cloudwatch 9 | spec: 10 | selector: 11 | matchLabels: 12 | k8s-app: fluentd-cloudwatch 13 | template: 14 | metadata: 15 | labels: 16 | k8s-app: fluentd-cloudwatch 17 | annotations: 18 | configHash: 8915de4cf9c3551a8dc74c0137a3e83569d28c71044b0359c2578d2e0461825 19 | spec: 20 | serviceAccountName: fluentd 21 | terminationGracePeriodSeconds: 30 22 | # Because the image's entrypoint requires to write on /fluentd/etc but we mount configmap there which is read-only, 23 | # this initContainers workaround or other is needed. 24 | # See https://github.com/fluent/fluentd-kubernetes-daemonset/issues/90 25 | initContainers: 26 | - name: copy-fluentd-config 27 | image: busybox 28 | command: ['sh', '-c', 'cp /config-volume/..data/* /fluentd/etc'] 29 | volumeMounts: 30 | - name: config-volume 31 | mountPath: /config-volume 32 | - name: fluentdconf 33 | mountPath: /fluentd/etc 34 | - name: update-log-driver 35 | image: busybox 36 | command: ['sh','-c',''] 37 | containers: 38 | - name: fluentd-cloudwatch 39 | image: fluent/fluentd-kubernetes-daemonset:v1.11.2-debian-cloudwatch-1.0 40 | env: 41 | - name: REGION 42 | valueFrom: 43 | configMapKeyRef: 44 | name: cluster-info 45 | key: logs.region 46 | - name: CLUSTER_NAME 47 | valueFrom: 48 | configMapKeyRef: 49 | name: cluster-info 50 | key: cluster.name 51 | resources: 52 | limits: 53 | memory: 200Mi 54 | requests: 55 | cpu: 100m 56 | memory: 200Mi 57 | volumeMounts: 58 | - name: config-volume 59 | mountPath: /config-volume 60 | - name: fluentdconf 61 | mountPath: /fluentd/etc 62 | - name: varlog 63 | mountPath: /var/log 64 | - name: varlibdockercontainers 65 | mountPath: /var/lib/docker/containers 66 | readOnly: true 67 | - name: runlogjournal 68 | mountPath: /run/log/journal 69 | readOnly: true 70 | - name: dmesg 71 | mountPath: /var/log/dmesg 72 | readOnly: true 73 | volumes: 74 | - name: config-volume 75 | configMap: 76 | name: fluentd-config 77 | - name: fluentdconf 78 | emptyDir: {} 79 | - name: varlog 80 | hostPath: 81 | path: /var/log 82 | - name: varlibdockercontainers 83 | hostPath: 84 | path: /var/lib/docker/containers 85 | - name: runlogjournal 86 | hostPath: 87 | path: /run/log/journal 88 | - name: dmesg 89 | hostPath: 90 | path: /var/log/dmesg 91 | -------------------------------------------------------------------------------- /amazon-cloudwatch/fluentd-rbac.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | name: fluentd 6 | namespace: amazon-cloudwatch 7 | 8 | --- 9 | apiVersion: rbac.authorization.k8s.io/v1beta1 10 | kind: ClusterRole 11 | metadata: 12 | name: fluentd-role 13 | rules: 14 | - apiGroups: [""] 15 | resources: 16 | - namespaces 17 | - pods 18 | - pods/logs 19 | verbs: ["get", "list", "watch"] 20 | 21 | --- 22 | apiVersion: rbac.authorization.k8s.io/v1beta1 23 | kind: ClusterRoleBinding 24 | metadata: 25 | name: fluentd-role-binding 26 | roleRef: 27 | apiGroup: rbac.authorization.k8s.io 28 | kind: ClusterRole 29 | name: fluentd-role 30 | subjects: 31 | - kind: ServiceAccount 32 | name: fluentd 33 | namespace: amazon-cloudwatch 34 | -------------------------------------------------------------------------------- /demo/helm-release.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: helm.fluxcd.io/v1 2 | kind: HelmRelease 3 | metadata: 4 | name: podinfo 5 | namespace: demo 6 | spec: 7 | releaseName: podinfo 8 | chart: 9 | git: https://github.com/stefanprodan/podinfo 10 | ref: v3.x 11 | path: charts/podinfo 12 | -------------------------------------------------------------------------------- /kube-system/aws-load-balancer-controller.yaml.tmpl: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: helm.fluxcd.io/v1 3 | kind: HelmRelease 4 | metadata: 5 | name: aws-load-balancer-controller 6 | namespace: kube-system 7 | spec: 8 | helmVersion: v3 9 | releaseName: aws-load-balancer-controller 10 | chart: 11 | repository: https://aws.github.io/eks-charts 12 | name: aws-load-balancer-controller 13 | version: 1.1.0 14 | values: 15 | clusterName: {{.ClusterName}} 16 | -------------------------------------------------------------------------------- /kube-system/cluster-autoscaler-deployment.yaml.tmpl: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: cluster-autoscaler 6 | namespace: kube-system 7 | labels: 8 | app: cluster-autoscaler 9 | annotations: 10 | cluster-autoscaler.kubernetes.io/safe-to-evict: "false" 11 | spec: 12 | replicas: 1 13 | selector: 14 | matchLabels: 15 | app: cluster-autoscaler 16 | template: 17 | metadata: 18 | labels: 19 | app: cluster-autoscaler 20 | annotations: 21 | prometheus.io/scrape: 'true' 22 | prometheus.io/port: '8085' 23 | spec: 24 | serviceAccountName: cluster-autoscaler 25 | containers: 26 | - image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.17.3 27 | name: cluster-autoscaler 28 | resources: 29 | limits: 30 | cpu: 100m 31 | memory: 300Mi 32 | requests: 33 | cpu: 100m 34 | memory: 300Mi 35 | command: 36 | - ./cluster-autoscaler 37 | - --v=4 38 | - --stderrthreshold=info 39 | - --cloud-provider=aws 40 | - --skip-nodes-with-local-storage=false 41 | - --expander=least-waste 42 | - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/{{ .ClusterName }} 43 | - --balance-similar-node-groups 44 | - --skip-nodes-with-system-pods=false 45 | volumeMounts: 46 | - name: ssl-certs 47 | mountPath: /etc/ssl/certs/ca-certificates.crt #/etc/ssl/certs/ca-bundle.crt for Amazon Linux Worker Nodes 48 | readOnly: true 49 | imagePullPolicy: "Always" 50 | volumes: 51 | - name: ssl-certs 52 | hostPath: 53 | path: "/etc/ssl/certs/ca-bundle.crt" 54 | -------------------------------------------------------------------------------- /kube-system/cluster-autoscaler-rbac.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | labels: 6 | k8s-addon: cluster-autoscaler.addons.k8s.io 7 | k8s-app: cluster-autoscaler 8 | name: cluster-autoscaler 9 | namespace: kube-system 10 | 11 | --- 12 | apiVersion: rbac.authorization.k8s.io/v1 13 | kind: ClusterRole 14 | metadata: 15 | name: cluster-autoscaler 16 | labels: 17 | k8s-addon: cluster-autoscaler.addons.k8s.io 18 | k8s-app: cluster-autoscaler 19 | rules: 20 | - apiGroups: [""] 21 | resources: ["events", "endpoints"] 22 | verbs: ["create", "patch"] 23 | - apiGroups: [""] 24 | resources: ["pods/eviction"] 25 | verbs: ["create"] 26 | - apiGroups: [""] 27 | resources: ["pods/status"] 28 | verbs: ["update"] 29 | - apiGroups: [""] 30 | resources: ["endpoints"] 31 | resourceNames: ["cluster-autoscaler"] 32 | verbs: ["get", "update"] 33 | - apiGroups: [""] 34 | resources: ["nodes"] 35 | verbs: ["watch", "list", "get", "update"] 36 | - apiGroups: [""] 37 | resources: 38 | - "pods" 39 | - "services" 40 | - "replicationcontrollers" 41 | - "persistentvolumeclaims" 42 | - "persistentvolumes" 43 | verbs: ["watch", "list", "get"] 44 | - apiGroups: ["extensions"] 45 | resources: ["replicasets", "daemonsets"] 46 | verbs: ["watch", "list", "get"] 47 | - apiGroups: ["policy"] 48 | resources: ["poddisruptionbudgets"] 49 | verbs: ["watch", "list"] 50 | - apiGroups: ["apps"] 51 | resources: ["statefulsets", "replicasets", "daemonsets"] 52 | verbs: ["watch", "list", "get"] 53 | - apiGroups: ["storage.k8s.io"] 54 | resources: ["storageclasses", "csinodes"] 55 | verbs: ["watch", "list", "get"] 56 | - apiGroups: ["batch", "extensions"] 57 | resources: ["jobs"] 58 | verbs: ["get", "list", "watch", "patch"] 59 | - apiGroups: ["coordination.k8s.io"] 60 | resources: ["leases"] 61 | verbs: ["create"] 62 | - apiGroups: ["coordination.k8s.io"] 63 | resourceNames: ["cluster-autoscaler"] 64 | resources: ["leases"] 65 | verbs: ["get", "update"] 66 | 67 | --- 68 | apiVersion: rbac.authorization.k8s.io/v1 69 | kind: Role 70 | metadata: 71 | name: cluster-autoscaler 72 | namespace: kube-system 73 | labels: 74 | k8s-addon: cluster-autoscaler.addons.k8s.io 75 | k8s-app: cluster-autoscaler 76 | rules: 77 | - apiGroups: [""] 78 | resources: ["configmaps"] 79 | verbs: ["create","list","watch"] 80 | - apiGroups: [""] 81 | resources: ["configmaps"] 82 | resourceNames: ["cluster-autoscaler-status", "cluster-autoscaler-priority-expander"] 83 | verbs: ["delete", "get", "update", "watch"] 84 | 85 | --- 86 | apiVersion: rbac.authorization.k8s.io/v1 87 | kind: ClusterRoleBinding 88 | metadata: 89 | name: cluster-autoscaler 90 | labels: 91 | k8s-addon: cluster-autoscaler.addons.k8s.io 92 | k8s-app: cluster-autoscaler 93 | roleRef: 94 | apiGroup: rbac.authorization.k8s.io 95 | kind: ClusterRole 96 | name: cluster-autoscaler 97 | subjects: 98 | - kind: ServiceAccount 99 | name: cluster-autoscaler 100 | namespace: kube-system 101 | 102 | --- 103 | apiVersion: rbac.authorization.k8s.io/v1 104 | kind: RoleBinding 105 | metadata: 106 | name: cluster-autoscaler 107 | namespace: kube-system 108 | labels: 109 | k8s-addon: cluster-autoscaler.addons.k8s.io 110 | k8s-app: cluster-autoscaler 111 | roleRef: 112 | apiGroup: rbac.authorization.k8s.io 113 | kind: Role 114 | name: cluster-autoscaler 115 | subjects: 116 | - kind: ServiceAccount 117 | name: cluster-autoscaler 118 | namespace: kube-system 119 | -------------------------------------------------------------------------------- /kubernetes-dashboard/dashboard-metrics-scraper-deployment.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: Deployment 17 | apiVersion: apps/v1 18 | metadata: 19 | labels: 20 | k8s-app: dashboard-metrics-scraper 21 | name: dashboard-metrics-scraper 22 | namespace: kubernetes-dashboard 23 | spec: 24 | replicas: 1 25 | revisionHistoryLimit: 10 26 | selector: 27 | matchLabels: 28 | k8s-app: dashboard-metrics-scraper 29 | template: 30 | metadata: 31 | labels: 32 | k8s-app: dashboard-metrics-scraper 33 | spec: 34 | containers: 35 | - name: dashboard-metrics-scraper 36 | image: kubernetesui/metrics-scraper:v1.0.4 37 | ports: 38 | - containerPort: 8000 39 | protocol: TCP 40 | livenessProbe: 41 | httpGet: 42 | scheme: HTTP 43 | path: / 44 | port: 8000 45 | initialDelaySeconds: 30 46 | timeoutSeconds: 30 47 | serviceAccountName: kubernetes-dashboard 48 | # Comment the following tolerations if Dashboard must not be deployed on master 49 | tolerations: 50 | - key: node-role.kubernetes.io/master 51 | effect: NoSchedule 52 | -------------------------------------------------------------------------------- /kubernetes-dashboard/dashboard-metrics-scraper-service.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: Service 17 | apiVersion: v1 18 | metadata: 19 | labels: 20 | k8s-app: dashboard-metrics-scraper 21 | name: dashboard-metrics-scraper 22 | namespace: kubernetes-dashboard 23 | spec: 24 | ports: 25 | - port: 8000 26 | targetPort: 8000 27 | selector: 28 | k8s-app: dashboard-metrics-scraper 29 | -------------------------------------------------------------------------------- /kubernetes-dashboard/kubernetes-dashboard-configmap.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: ConfigMap 17 | apiVersion: v1 18 | metadata: 19 | labels: 20 | k8s-app: kubernetes-dashboard 21 | name: kubernetes-dashboard-settings 22 | namespace: kubernetes-dashboard 23 | -------------------------------------------------------------------------------- /kubernetes-dashboard/kubernetes-dashboard-deployment.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: Deployment 17 | apiVersion: apps/v1 18 | metadata: 19 | labels: 20 | k8s-app: kubernetes-dashboard 21 | name: kubernetes-dashboard 22 | namespace: kubernetes-dashboard 23 | spec: 24 | replicas: 1 25 | revisionHistoryLimit: 10 26 | selector: 27 | matchLabels: 28 | k8s-app: kubernetes-dashboard 29 | template: 30 | metadata: 31 | labels: 32 | k8s-app: kubernetes-dashboard 33 | spec: 34 | containers: 35 | - name: kubernetes-dashboard 36 | image: kubernetesui/dashboard:v2.0.3 37 | imagePullPolicy: Always 38 | ports: 39 | - containerPort: 8443 40 | protocol: TCP 41 | args: 42 | - --auto-generate-certificates 43 | - --namespace=kubernetes-dashboard 44 | # Uncomment the following line to manually specify Kubernetes API server Host 45 | # If not specified, Dashboard will attempt to auto discover the API server and connect 46 | # to it. Uncomment only if the default does not work. 47 | # - --apiserver-host=http://my-address:port 48 | volumeMounts: 49 | - name: kubernetes-dashboard-certs 50 | mountPath: /certs 51 | # Create on-disk volume to store exec logs 52 | - mountPath: /tmp 53 | name: tmp-volume 54 | livenessProbe: 55 | httpGet: 56 | scheme: HTTPS 57 | path: / 58 | port: 8443 59 | initialDelaySeconds: 30 60 | timeoutSeconds: 30 61 | volumes: 62 | - name: kubernetes-dashboard-certs 63 | secret: 64 | secretName: kubernetes-dashboard-certs 65 | - name: tmp-volume 66 | emptyDir: {} 67 | serviceAccountName: kubernetes-dashboard 68 | # Comment the following tolerations if Dashboard must not be deployed on master 69 | tolerations: 70 | - key: node-role.kubernetes.io/master 71 | effect: NoSchedule 72 | -------------------------------------------------------------------------------- /kubernetes-dashboard/kubernetes-dashboard-rbac.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: ServiceAccount 17 | apiVersion: v1 18 | metadata: 19 | labels: 20 | k8s-app: kubernetes-dashboard 21 | name: kubernetes-dashboard 22 | namespace: kubernetes-dashboard 23 | 24 | --- 25 | kind: Role 26 | apiVersion: rbac.authorization.k8s.io/v1 27 | metadata: 28 | labels: 29 | k8s-app: kubernetes-dashboard 30 | name: kubernetes-dashboard 31 | namespace: kubernetes-dashboard 32 | rules: 33 | # Allow Dashboard to get, update and delete Dashboard exclusive secrets. 34 | - apiGroups: [""] 35 | resources: ["secrets"] 36 | resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] 37 | verbs: ["get", "update", "delete"] 38 | # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. 39 | - apiGroups: [""] 40 | resources: ["configmaps"] 41 | resourceNames: ["kubernetes-dashboard-settings"] 42 | verbs: ["get", "update"] 43 | # Allow Dashboard to get metrics. 44 | - apiGroups: [""] 45 | resources: ["services"] 46 | resourceNames: ["heapster", "dashboard-metrics-scraper"] 47 | verbs: ["proxy"] 48 | - apiGroups: [""] 49 | resources: ["services/proxy"] 50 | resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"] 51 | verbs: ["get"] 52 | 53 | --- 54 | kind: ClusterRole 55 | apiVersion: rbac.authorization.k8s.io/v1 56 | metadata: 57 | labels: 58 | k8s-app: kubernetes-dashboard 59 | name: kubernetes-dashboard 60 | rules: 61 | # Allow Metrics Scraper to get metrics from the Metrics server 62 | - apiGroups: ["metrics.k8s.io"] 63 | resources: ["pods", "nodes"] 64 | verbs: ["get", "list", "watch"] 65 | 66 | --- 67 | kind: RoleBinding 68 | apiVersion: rbac.authorization.k8s.io/v1 69 | metadata: 70 | labels: 71 | k8s-app: kubernetes-dashboard 72 | name: kubernetes-dashboard 73 | namespace: kubernetes-dashboard 74 | roleRef: 75 | apiGroup: rbac.authorization.k8s.io 76 | kind: Role 77 | name: kubernetes-dashboard 78 | subjects: 79 | - kind: ServiceAccount 80 | name: kubernetes-dashboard 81 | namespace: kubernetes-dashboard 82 | 83 | --- 84 | kind: ClusterRoleBinding 85 | apiVersion: rbac.authorization.k8s.io/v1 86 | metadata: 87 | name: kubernetes-dashboard 88 | namespace: kubernetes-dashboard 89 | roleRef: 90 | apiGroup: rbac.authorization.k8s.io 91 | kind: ClusterRole 92 | name: kubernetes-dashboard 93 | subjects: 94 | - kind: ServiceAccount 95 | name: kubernetes-dashboard 96 | namespace: kubernetes-dashboard 97 | -------------------------------------------------------------------------------- /kubernetes-dashboard/kubernetes-dashboard-secrets.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: Secret 17 | apiVersion: v1 18 | metadata: 19 | labels: 20 | k8s-app: kubernetes-dashboard 21 | name: kubernetes-dashboard-certs 22 | namespace: kubernetes-dashboard 23 | type: Opaque 24 | 25 | --- 26 | kind: Secret 27 | apiVersion: v1 28 | metadata: 29 | labels: 30 | k8s-app: kubernetes-dashboard 31 | name: kubernetes-dashboard-csrf 32 | namespace: kubernetes-dashboard 33 | type: Opaque 34 | data: 35 | csrf: "" 36 | 37 | --- 38 | kind: Secret 39 | apiVersion: v1 40 | metadata: 41 | labels: 42 | k8s-app: kubernetes-dashboard 43 | name: kubernetes-dashboard-key-holder 44 | namespace: kubernetes-dashboard 45 | type: Opaque 46 | -------------------------------------------------------------------------------- /kubernetes-dashboard/kubernetes-dashboard-service.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | kind: Service 17 | apiVersion: v1 18 | metadata: 19 | labels: 20 | k8s-app: kubernetes-dashboard 21 | name: kubernetes-dashboard 22 | namespace: kubernetes-dashboard 23 | spec: 24 | ports: 25 | - port: 443 26 | targetPort: 8443 27 | selector: 28 | k8s-app: kubernetes-dashboard 29 | -------------------------------------------------------------------------------- /monitoring/metrics-server.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: helm.fluxcd.io/v1 3 | kind: HelmRelease 4 | metadata: 5 | name: metrics-server 6 | namespace: monitoring 7 | spec: 8 | releaseName: metrics-server 9 | chart: 10 | repository: https://kubernetes-charts.storage.googleapis.com/ 11 | name: metrics-server 12 | version: 2.11.1 13 | values: 14 | args: 15 | - --kubelet-preferred-address-types=InternalIP 16 | -------------------------------------------------------------------------------- /monitoring/prometheus-operator.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: helm.fluxcd.io/v1 3 | kind: HelmRelease 4 | metadata: 5 | name: prometheus-operator 6 | namespace: monitoring 7 | spec: 8 | releaseName: prometheus-operator 9 | chart: 10 | repository: https://kubernetes-charts.storage.googleapis.com/ 11 | name: prometheus-operator 12 | version: 9.3.1 13 | values: 14 | grafana: 15 | enabled: true 16 | defaultDashboardsEnabled: true 17 | nodeExporter: 18 | serviceMonitor: 19 | relabelings: 20 | - sourceLabels: [__meta_kubernetes_pod_node_name] 21 | targetLabel: instance 22 | -------------------------------------------------------------------------------- /namespaces/amazon-cloudwatch.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | labels: 6 | name: amazon-cloudwatch 7 | name: amazon-cloudwatch 8 | -------------------------------------------------------------------------------- /namespaces/demo.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | labels: 6 | name: demo 7 | name: demo 8 | -------------------------------------------------------------------------------- /namespaces/kubernetes-dashboard.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | --- 16 | apiVersion: v1 17 | kind: Namespace 18 | metadata: 19 | name: kubernetes-dashboard 20 | -------------------------------------------------------------------------------- /namespaces/monitoring.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | labels: 6 | name: monitoring 7 | name: monitoring 8 | --------------------------------------------------------------------------------