├── .gitignore ├── .travis.yml ├── Makefile ├── README.md ├── applications ├── Chart.yaml ├── templates │ ├── ambassador │ │ ├── ambassador-mapping.yaml │ │ ├── ambassador.yaml │ │ ├── namespace.yaml │ │ └── project.yaml │ ├── argocd │ │ └── argocd.yaml │ ├── guestbook │ │ ├── guestbook.yaml │ │ ├── namespace.yaml │ │ └── project.yaml │ ├── kube-system │ │ ├── kubernetes-dashboard.yaml │ │ ├── metrics-server.yaml │ │ ├── project.yaml │ │ └── spotify-docker-gc.yaml │ └── observe │ │ ├── cerebro.yaml │ │ ├── curator.yaml │ │ ├── elasticsearch.yaml │ │ ├── fluent-bit.yaml │ │ ├── kibana.yaml │ │ ├── kube-ops-view.yaml │ │ ├── namespace.yaml │ │ ├── project.yaml │ │ └── prometheus-operator.yaml └── values.yaml ├── charts ├── ambassador-mapping │ ├── Chart.yaml │ ├── templates │ │ ├── ambassador-mapping.yaml │ │ ├── guestbook-mapping.yaml │ │ └── httpbin-mapping.yaml │ └── values.yaml └── seed │ ├── Chart.yaml │ ├── requirements.yaml │ ├── templates │ ├── applications.yaml │ ├── namespace.yaml │ └── project.yaml │ └── values.yaml ├── docs ├── Weaveworks_ContinuousDelivery_wp_2018.pdf ├── charts-TODO.txt ├── img │ ├── architecture.png │ ├── argocd-ui.png │ ├── bootstrap.png │ ├── charts.png │ └── gitops-k8s.drawio └── setup.md └── scripts ├── bootstrap.sh └── publish_seed.sh /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | .DS_Store 3 | 4 | *.lock 5 | *.tgz 6 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: bash 2 | 3 | before_install: 4 | - curl -L https://git.io/get_helm.sh | bash 5 | 6 | script: 7 | # verify all charts 8 | - find . -maxdepth 3 -type f -name 'Chart.yaml' -exec dirname {} \; | xargs helm lint 9 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .DEFAULT_GOAL := all 2 | 3 | HELM := $(shell command -v helm 2> /dev/null) 4 | 5 | .PHONY: requirements 6 | requirements: 7 | ifndef HELM 8 | $(error "helm" not found) 9 | endif 10 | 11 | # publish to https://edgelevel.github.io/gitops-k8s 12 | .PHONY: publish-seed 13 | publish-seed: requirements 14 | ./scripts/publish_seed.sh 15 | 16 | .PHONY: bootstrap 17 | bootstrap: requirements 18 | ./scripts/bootstrap.sh 19 | 20 | .PHONY: all 21 | all: requirements 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # gitops-k8s 2 | 3 | [![Build Status][travis-image]][travis-url] 4 | 5 | [travis-image]: https://travis-ci.org/edgelevel/gitops-k8s.svg?branch=master 6 | [travis-url]: https://travis-ci.org/edgelevel/gitops-k8s 7 | 8 | This document aims to provide an opinionated working solution leveraging Kubernetes and proven GitOps techniques to have a resilient, composable and scalable Kubernetes platform. 9 | 10 | Nothing outlined below is new or innovative, but it should be at least a good starting point to have a cluster up and running pretty quickly and give you a chance to remain focused and try out new ideas. 11 | 12 | Feedback and help are always welcome! 13 | 14 | * [Introduction](#introduction) 15 | * [Argo CD](#argo-cd) 16 | * [Applications](#applications) 17 | 18 | --- 19 | 20 | ## Introduction 21 | 22 | ### TL;DR 23 | 24 | * Kubernetes is a declarative system 25 | * Git can be used to describe infrastructure and applications 26 | * Git repository is the source of truth and represents a cluster 27 | * GitOps is a way to do Continuous Delivery and operate Kubernetes via Git pull requests 28 | * GitOps empowers developers to do operations 29 | * CI pipelines should only run builds, tests and publish images 30 | * In a pull-based approach, an operator deploys new images from inside of the cluster 31 | * You can only observe the actual state of the cluster and react when it diverges from the desired state 32 | 33 | ### Imperative vs Declarative 34 | 35 | In an *imperative* system, the user knows the desired state, determines the sequence of commands to transition the system to the desired state and supplies a representation of the commands to the system. 36 | 37 | By contrast, in a *declarative* system, the user knows the desired state, supplies a representation of the desired state to the system, then the system reads the current state and determines the sequence of commands to transition the system to the desired state. 38 | 39 | Declarative systems have the distinct advantage of being able to react to unintended state changes without further supervision. In the event of an unintended state change leading to a state drift, the system may autonomously determine and apply the set of mitigating actions leading to a state match. This process is called a **control loop**, a popular choice for the implementation of controllers. 40 | 41 | ### What is GitOps? 42 | 43 | GitOps is the art and science of using Git pull requests to manage infrastructure provisioning and software deployment. 44 | 45 | The concept of GitOps originated at Weaveworks, whose developers described how they use Git to create a **single source of truth**. Kubernetes is a **declarative** system and by using declarative tools, the entire set of configuration files can be version controlled in Git. 46 | 47 | More generally, GitOps is a way to do Continuous Delivery and operate Kubernetes via Git. 48 | 49 | ### Push vs Pull 50 | 51 | In a *push*-based pipeline, the CI system runs build and tests, followed by a deployment directly to Kubernetes. This is an *anti-pattern*. CI server is not an orchestration tool. You need something that continually attempts to make progress until there are no more diffs because CI fails when it encounters a difference and then you could end up being in a partial and unknown state. 52 | 53 | In a *pull*-based pipeline, a Kubernetes **operator** deploys new images from inside of the cluster. The operator notices when a new image has been pushed to the registry. Convergence of the cluster state is then triggered and the new image is pulled from the registry, the manifest is automatically updated and the new image is deployed to the cluster. 54 | 55 | A CI pipeline should be used to merge and integrate updates with master, while with GitOps you should rely on Kubernetes or the cluster to internally manage deployments based on those master updates. 56 | 57 | You could potentially have multiple cluster pointing to the same GitOps repository, but you won't have a centralized view of them, all the clusters will be independent. 58 | 59 | ### Observability 60 | 61 | Git provides a source of truth for the desired state of the system and *observability* provides a source of truth for the actual state of the running system. 62 | 63 | You cannot say what actual state is in the cluster. You can only observe it. This is why diffs are so important. 64 | 65 | A system is observable if developers can understand its current state from the outside. Observability is a property of systems like Availability and Scalability. Monitoring, Tracing and Logging are techniques for baseline observations. 66 | 67 | Observability is a source of truth for the actual running state of the system right now. You observe the running system in order to understand and control it. Observed state must be compared with the desired state in Git and usually you want to monitor and alert when the system diverge from the desired state. 68 | 69 | **Resources** 70 | 71 | * [Imperative vs Declarative](https://medium.com/@dominik.tornow/imperative-vs-declarative-8abc7dcae82e) 72 | * [GitOps - Operations by Pull Request](https://www.weave.works/blog/gitops-operations-by-pull-request) (Part 1) 73 | * [The GitOps Pipeline](https://www.weave.works/blog/the-gitops-pipeline) (Part 2) 74 | * [GitOps - Observability](https://www.weave.works/blog/gitops-part-3-observability) (Part 3) 75 | * [GitOps - Application Delivery Compliance and Secure CICD](https://www.weave.works/blog/gitops-compliance-and-secure-cicd) (Part 4) 76 | * [Making the Leap from Continuous Integration to Continuous Delivery](docs/Weaveworks_ContinuousDelivery_wp_2018.pdf) (Whitepaper) 77 | * [What is GitOps really?](https://www.weave.works/blog/what-is-gitops-really) 78 | * [Why is a PULL vs a PUSH pipeline important?](https://www.weave.works/blog/why-is-a-pull-vs-a-push-pipeline-important) 79 | * [Kubernetes anti-patterns: Let's do GitOps, not CIOps!](https://www.weave.works/blog/kubernetes-anti-patterns-let-s-do-gitops-not-ciops) 80 | * [GitOps: High velocity CICD for Kubernetes](https://www.weave.works/blog/gitops-high-velocity-cicd-for-kubernetes) 81 | * [GitOps - What you need to know](https://www.weave.works/technologies/gitops) 82 | * [GitOps for Kubernetes - A DevOps Iteration Focused on Declarative Infrastructure](https://youtu.be/wJleh-6DZJ0) 83 | * [Automating continuous delivery with Kubernetes, Google Cloud and Git](https://vimeo.com/255633066) 84 | * [Continuous Delivery the Hard Way](https://www.weave.works/blog/continuous-delivery-the-hard-way) 85 | 86 | ## Argo CD 87 | 88 | Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. It automates the deployment of the desired application states in the specified target environments. In this project Kubernetes manifests are specified as [helm](https://helm.sh/docs) charts. 89 | 90 | This guide will explain how to setup in few steps the whole infrastructure via GitOps with Argo CD. Note that it's not tightly coupled to any specific vendor and you should be able to easily run it on [DigitalOcean](https://www.digitalocean.com/docs/kubernetes), [EKS](https://aws.amazon.com/eks) or [GKE](https://cloud.google.com/kubernetes-engine) for example. 91 | 92 | 95 | 96 |

97 | architecture 98 |

99 | 100 | Most of the steps have been kept manual on purpose, but they should be automated in a production enviroment. 101 | 102 | ### Prerequisites 103 | 104 | * [Setup](docs/setup.md) required tools 105 | * Create a Kubernetes cluster locally or with your favourite provider 106 | * Download the cluster configs and test connection 107 | ```bash 108 | export KUBECONFIG=~/.kube/-kubeconfig.yaml 109 | kubectl get nodes 110 | ``` 111 | 112 | ### Bootstrap 113 | 114 | 1. *TODO Setup secrets (optional)* 115 | 2. Setup Argo CD and all the applications 116 | ```bash 117 | make bootstrap 118 | ``` 119 | 3. Access Argo CD 120 | ```bash 121 | # username: admin 122 | # password: (autogenerated) the pod name of the Argo CD API server 123 | kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2 124 | 125 | # port forward the service 126 | kubectl port-forward service/argocd-server -n argocd 8080:443 127 | 128 | # from the UI 129 | [open|xdg-open] https://localhost:8080 130 | # from the CLI 131 | argocd login localhost:8080 --username admin 132 | ``` 133 | * You might need to *Allow invalid certificates for resources loaded from localhost* on Chrome enabling the flag `chrome://flags/#allow-insecure-localhost` to access it 134 | 4. First time only sync all the `OutOfSync` applications 135 | * manually 136 | * *TODO with a cronjob (optional)* 137 | * verify [guestbook](https://github.com/argoproj/argocd-example-apps/tree/master/guestbook) example 138 | ```bash 139 | # port forward the service 140 | kubectl port-forward service/guestbook-ui -n guestbook 8081:80 141 | # open browser 142 | [open|xdg-open] http://localhost:8081 143 | ``` 144 | 145 | This is how it should looks like on the UI 146 | 147 | ![argocd-ui](docs/img/argocd-ui.png) 148 | 149 | **Resources** 150 | 151 | * [Introducing Argo CD](https://blog.argoproj.io/introducing-argo-cd-declarative-continuous-delivery-for-kubernetes-da2a73a780cd) 152 | * [Argo CD - Declarative Continuous Delivery for Kubernetes](https://argoproj.github.io/argo-cd) 153 | 154 | ## Applications 155 | 156 | Applications in this repository are defined in the parent [applications](applications/templates) chart and are logically split into folders which represent Kubernetes namespaces. 157 | 158 |

159 | charts 160 |

161 | 162 | **`ambassador`** namespace is dedicated for [Ambassador](https://www.getambassador.io), a lightweight Kubernetes-native microservices API gateway built on the Envoy Proxy which is mainly used for routing and supports canary deployments, traffic shadowing, rate limiting, authentication and more 163 | ```bash 164 | # retrieve EXTERNAL-IP 165 | kubectl get service ambassador -n ambassador 166 | [open|xdg-open] http:///ambassador 167 | [open|xdg-open] http:///httpbin/ 168 | [open|xdg-open] http:///guestbook 169 | 170 | # debug ambassador 171 | kubectl port-forward service/ambassador-admins 8877 -n ambassador 172 | [open|xdg-open] http://localhost:8877/ambassador/v0/diag 173 | ``` 174 | 175 | *Ambassador is [disabled](applications/values.yaml) by default because the recommended way is to use host-based routing which requires a domain* 176 | 177 | For a working example on DigitalOcean using [`external-dns`](https://github.com/helm/charts/tree/master/stable/external-dns) you can have a look at [niqdev/do-k8s](https://github.com/niqdev/do-k8s) 178 | 179 | *TODO Service mesh* 180 | 181 | * [Istio](https://istio.io) 182 | * [A Crash Course For Running Istio](https://medium.com/namely-labs/a-crash-course-for-running-istio-1c6125930715) 183 | 184 | **`observe`** namespace is dedicated for observability and in the specific Monitoring, Alerting and Logging 185 | 186 | * [`prometheus-operator`](https://github.com/helm/charts/tree/master/stable/prometheus-operator) provides monitoring and alerting managing Prometheus, Alertmanager and Grafana 187 | ```bash 188 | # prometheus 189 | kubectl port-forward service/prometheus-operator-prometheus 8001:9090 -n observe 190 | 191 | # alertmanager 192 | kubectl port-forward service/prometheus-operator-alertmanager 8002:9093 -n observe 193 | 194 | # grafana 195 | # username: admin 196 | # password: prom-operator 197 | kubectl port-forward service/prometheus-operator-grafana 8003:80 -n observe 198 | ``` 199 | 200 | * [`kube-ops-view`](https://github.com/helm/charts/tree/master/stable/kube-ops-view) provides a read-only system dashboard for multiple k8s clusters 201 | ```bash 202 | kubectl port-forward service/kube-ops-view -n observe 8004:80 203 | ``` 204 | 205 | *EFK stack for logging* 206 | 207 | * [`elasticsearch`](https://github.com/elastic/helm-charts/tree/master/elasticsearch) is a distributed, RESTful search and analytics engine and it's used for log storage 208 | ```bash 209 | kubectl port-forward service/elasticsearch-master 9200:9200 -n observe 210 | ``` 211 | 212 | * [`cerebro`](https://github.com/helm/charts/tree/master/stable/cerebro) is an Elasticsearch web admin tool 213 | ```bash 214 | kubectl port-forward service/cerebro 9000:80 -n observe 215 | ``` 216 | 217 | * [`kibana`](https://github.com/elastic/helm-charts/tree/master/kibana) visualize and query the log data stored in an Elasticsearch index 218 | ```bash 219 | kubectl port-forward service/kibana-kibana 9001:5601 -n observe 220 | ``` 221 | * [`fluentbit`](https://github.com/helm/charts/tree/master/stable/fluent-bit) is a fast and lightweight Log Processor and Forwarder 222 | 223 | * [`elasticsearch-curator`](https://github.com/helm/charts/tree/master/stable/elasticsearch-curator) or [`curator`](https://github.com/giantswarm/curator) helps to curate, or manage, Elasticsearch indices and snapshots 224 | 225 | **Resources** 226 | 227 | * [Prometheus](https://prometheus.io/docs/introduction/overview) 228 | * Prometheus Operator - [Getting Started Guide](https://coreos.com/operators/prometheus/docs/latest/user-guides/getting-started.html) 229 | * Grafana - [Dashboards](https://grafana.com/dashboards) 230 | * [Fluent Bit](https://docs.fluentbit.io/manual) 231 | * [Logging Best Practices for Kubernetes using Elasticsearch, Fluent Bit and Kibana](https://itnext.io/logging-best-practices-for-kubernetes-using-elasticsearch-fluent-bit-and-kibana-be9b7398dfee) 232 | * [Exporting Kubernetes Logs to Elasticsearch Using Fluent Bit](https://supergiant.io/blog/exporting-kubernetes-logs-to-elasticsearch-using-fluent-bit) 233 | * [Fluentd vs. Fluent Bit: Side by Side Comparison](https://logz.io/blog/fluentd-vs-fluent-bit) 234 | * [Logging & Monitoring of Kubernetes Applications: Requirements & Recommended Toolset](https://platform9.com/blog/logging-monitoring-of-kubernetes-applications-requirements-recommended-toolset) 235 | * [Loki](https://github.com/grafana/loki) 236 | 237 | **`kube-system`** namespace is reserved for Kubernete system applications 238 | 239 | * [`kubernetes-dashboard`](https://github.com/helm/charts/tree/master/stable/kubernetes-dashboard) is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them, as well as manage the cluster itself 240 | ```bash 241 | kubectl port-forward service/kubernetes-dashboard -n kube-system 8000:443 242 | ``` 243 | 244 | * [`metrics-server`](https://github.com/helm/charts/tree/master/stable/metrics-server) is an add-on which extends the metrics api group and enables the Kubernetes resource `HorizontalPodAutoscaler` 245 | ```bash 246 | kubectl top node 247 | kubectl top pod --all-namespaces 248 | ``` 249 | 250 | * [`spotify-docker-gc`](https://github.com/helm/charts/tree/master/stable/spotify-docker-gc) performs garbage collection in the Kubernetes cluster and the default configurations have the gc running once a day which: 251 | * removes containers that exited more than a hour ago 252 | * removes images that don't belong to any container 253 | * removes volumes that are not associated to any remaining container 254 | 255 | --- 256 | 257 | ### TODO (not in order) 258 | 259 | * [ ] argocd: example secrets for private charts 260 | * [ ] argocd: override default `admin.password` 261 | * [ ] argocd-bootstrap: open source and explain solution of how to sync automatically first time with cronjob 262 | * [ ] expose argocd over http i.e. `--insecure` flag 263 | * [ ] configure TLS/cert and authentication on ambassador for all services 264 | * [ ] centralize auth on ambassador/istio 265 | * [ ] [Jaeger](https://www.jaegertracing.io) tracing 266 | * [ ] [kube-monkey](https://github.com/asobti/kube-monkey) or [chaoskube](https://github.com/helm/charts/tree/master/stable/chaoskube) 267 | * [ ] explain how to switch cluster via DNS 268 | * [ ] Kafka from public chart + [JMX fix](https://github.com/helm/charts/pull/10799/files) 269 | * [ ] stateless vs stateful: disaster recovery stratecy e.g S3 backup/restore 270 | * [ ] example with multiple providers: DigitalOcean, EKS, GKE 271 | * [ ] add prometheus adapter for custom metrics that can be used by the [HorizontalPodAutoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale) 272 | * [ ] explain how to test a branch i.e. change target revision from the UI 273 | * [ ] TODO fix `alertmanager: error: unrecognized log format "", try --help` 274 | * [ ] add screenshots to readme for each app 275 | * [ ] explain how to add grafana dashboards with `ConfigMap` 276 | * [ ] add alerting example on Slack/PagerDuty 277 | * [ ] add example of prometheus `ServiceMonitor` + dashboard 278 | * [ ] explain how to init es index on kibana for logging + screenshot 279 | * [ ] add `kubefwd` to docs 280 | * [ ] argocd [issue](https://github.com/argoproj/argo-cd/issues/1786): Add support for secrets in Application parameters 281 | * [ ] argocd [issue](https://github.com/argoproj/argo-cd/issues/1145): Helm repository as first class Argo CD Application source 282 | -------------------------------------------------------------------------------- /applications/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | name: applications 3 | version: 0.1.0 4 | -------------------------------------------------------------------------------- /applications/templates/ambassador/ambassador-mapping.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.ambassador.enabled }} 2 | --- 3 | apiVersion: argoproj.io/v1alpha1 4 | kind: Application 5 | metadata: 6 | name: ambassador-mapping 7 | spec: 8 | project: ambassador 9 | source: 10 | repoURL: https://github.com/edgelevel/gitops-k8s.git 11 | path: charts/ambassador-mapping 12 | destination: 13 | server: https://kubernetes.default.svc 14 | namespace: ambassador 15 | {{- end }} 16 | -------------------------------------------------------------------------------- /applications/templates/ambassador/ambassador.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.ambassador.enabled }} 2 | --- 3 | apiVersion: argoproj.io/v1alpha1 4 | kind: Application 5 | metadata: 6 | name: ambassador 7 | spec: 8 | project: ambassador 9 | source: 10 | repoURL: https://github.com/helm/charts.git 11 | path: stable/ambassador 12 | helm: 13 | parameters: 14 | - name: replicaCount 15 | value: "1" 16 | destination: 17 | server: https://kubernetes.default.svc 18 | namespace: ambassador 19 | {{- end }} 20 | -------------------------------------------------------------------------------- /applications/templates/ambassador/namespace.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.ambassador.enabled }} 2 | --- 3 | apiVersion: v1 4 | kind: Namespace 5 | metadata: 6 | name: ambassador 7 | {{- end }} 8 | -------------------------------------------------------------------------------- /applications/templates/ambassador/project.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.ambassador.enabled }} 2 | --- 3 | apiVersion: argoproj.io/v1alpha1 4 | kind: AppProject 5 | metadata: 6 | name: ambassador 7 | spec: 8 | sourceRepos: 9 | - https://github.com/helm/charts.git 10 | - https://github.com/edgelevel/gitops-k8s.git 11 | destinations: 12 | - namespace: ambassador 13 | server: https://kubernetes.default.svc 14 | clusterResourceWhitelist: 15 | - group: rbac.authorization.k8s.io 16 | kind: ClusterRoleBinding 17 | - group: rbac.authorization.k8s.io 18 | kind: ClusterRole 19 | - group: apiextensions.k8s.io 20 | kind: CustomResourceDefinition 21 | {{- end }} 22 | -------------------------------------------------------------------------------- /applications/templates/argocd/argocd.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: argocd 6 | spec: 7 | project: argocd 8 | source: 9 | repoURL: https://github.com/edgelevel/helm-charts.git 10 | path: argocd 11 | destination: 12 | server: https://kubernetes.default.svc 13 | namespace: argocd 14 | -------------------------------------------------------------------------------- /applications/templates/guestbook/guestbook.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: guestbook 6 | spec: 7 | project: guestbook 8 | source: 9 | repoURL: https://github.com/argoproj/argocd-example-apps.git 10 | path: guestbook 11 | #targetRevision: HEAD 12 | destination: 13 | server: https://kubernetes.default.svc 14 | namespace: guestbook 15 | -------------------------------------------------------------------------------- /applications/templates/guestbook/namespace.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | name: guestbook 6 | -------------------------------------------------------------------------------- /applications/templates/guestbook/project.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: AppProject 4 | metadata: 5 | name: guestbook 6 | spec: 7 | sourceRepos: 8 | - https://github.com/argoproj/argocd-example-apps.git 9 | destinations: 10 | - namespace: guestbook 11 | server: https://kubernetes.default.svc 12 | -------------------------------------------------------------------------------- /applications/templates/kube-system/kubernetes-dashboard.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: kubernetes-dashboard 6 | spec: 7 | project: kube-system 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/kubernetes-dashboard 11 | helm: 12 | parameters: 13 | - name: enableSkipLogin 14 | value: "true" 15 | - name: enableInsecureLogin 16 | value: "true" 17 | - name: rbac.clusterAdminRole 18 | value: "true" 19 | destination: 20 | server: https://kubernetes.default.svc 21 | namespace: kube-system 22 | -------------------------------------------------------------------------------- /applications/templates/kube-system/metrics-server.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: metrics-server 6 | spec: 7 | project: kube-system 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/metrics-server 11 | helm: 12 | parameters: 13 | - name: serviceAccount.name 14 | value: metrics-server 15 | - name: resources.requests.memory 16 | value: 50Mi 17 | - name: resources.limits.memory 18 | value: 50Mi 19 | - name: args[0] 20 | value: "--kubelet-preferred-address-types=InternalIP" 21 | - name: args[1] 22 | value: "--logtostderr" 23 | - name: args[2] 24 | value: "--kubelet-insecure-tls" 25 | destination: 26 | server: https://kubernetes.default.svc 27 | namespace: kube-system 28 | -------------------------------------------------------------------------------- /applications/templates/kube-system/project.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: AppProject 4 | metadata: 5 | name: kube-system 6 | spec: 7 | clusterResourceWhitelist: 8 | - group: rbac.authorization.k8s.io 9 | kind: ClusterRoleBinding 10 | - group: rbac.authorization.k8s.io 11 | kind: ClusterRole 12 | - group: apiextensions.k8s.io 13 | kind: CustomResourceDefinition 14 | - group: apiregistration.k8s.io 15 | kind: APIService 16 | sourceRepos: 17 | - https://github.com/helm/charts.git 18 | destinations: 19 | - server: https://kubernetes.default.svc 20 | namespace: kube-system 21 | -------------------------------------------------------------------------------- /applications/templates/kube-system/spotify-docker-gc.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: spotify-docker-gc 6 | spec: 7 | project: kube-system 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/spotify-docker-gc 11 | destination: 12 | server: https://kubernetes.default.svc 13 | namespace: kube-system 14 | -------------------------------------------------------------------------------- /applications/templates/observe/cerebro.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: cerebro 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/cerebro 11 | helm: 12 | parameters: 13 | - name: config.hosts[0].host 14 | value: http://elasticsearch-master.observe:9200 15 | - name: config.hosts[0].name 16 | value: elasticsearch-observe 17 | destination: 18 | server: https://kubernetes.default.svc 19 | namespace: observe 20 | -------------------------------------------------------------------------------- /applications/templates/observe/curator.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: curator 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/elasticsearch-curator 11 | destination: 12 | server: https://kubernetes.default.svc 13 | namespace: observe 14 | -------------------------------------------------------------------------------- /applications/templates/observe/elasticsearch.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: elasticsearch 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/elastic/helm-charts.git 10 | path: elasticsearch 11 | helm: 12 | # see https://github.com/elastic/helm-charts/blob/master/elasticsearch/examples/minikube/values.yaml 13 | parameters: 14 | - name: antiAffinity 15 | value: "soft" 16 | - name: esJavaOpts 17 | value: "-Xmx128m -Xms128m" 18 | - name: resources.requests.cpu 19 | value: "100m" 20 | - name: resources.requests.memory 21 | value: "512M" 22 | - name: resources.limits.memory 23 | value: "512M" 24 | - name: volumeClaimTemplate.resources.requests.storage 25 | value: "1Gi" 26 | - name: volumeClaimTemplate.storageClassName 27 | value: "standard" 28 | destination: 29 | server: https://kubernetes.default.svc 30 | namespace: observe 31 | -------------------------------------------------------------------------------- /applications/templates/observe/fluent-bit.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: fluent-bit 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/fluent-bit 11 | helm: 12 | parameters: 13 | - name: rbac.create 14 | value: "false" 15 | - name: serviceAccount.create 16 | value: "false" 17 | - name: backend.type 18 | value: es 19 | - name: backend.es.host 20 | value: elasticsearch-master.observe 21 | - name: backend.es.port 22 | value: "9200" 23 | - name: parsers.enabled 24 | value: "true" 25 | - name: parsers.json[0].name 26 | value: "docker" 27 | - name: parsers.json[0].timeKeep 28 | value: "On" 29 | - name: parsers.json[0].timeKey 30 | value: "time" 31 | - name: parsers.json[0].timeFormat 32 | value: "%Y-%m-%dT%H:%M:%S.%L" 33 | - name: parsers.json[0].extraEntries 34 | value: "\nDecode_Field_As escaped_utf8 log do_next\nDecode_Field_As json log" 35 | destination: 36 | server: https://kubernetes.default.svc 37 | namespace: observe 38 | -------------------------------------------------------------------------------- /applications/templates/observe/kibana.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: kibana 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/elastic/helm-charts.git 10 | path: kibana 11 | destination: 12 | server: https://kubernetes.default.svc 13 | namespace: observe 14 | -------------------------------------------------------------------------------- /applications/templates/observe/kube-ops-view.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: kube-ops-view 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/kube-ops-view 11 | helm: 12 | parameters: 13 | - name: rbac.create 14 | value: "true" 15 | destination: 16 | server: https://kubernetes.default.svc 17 | namespace: observe 18 | -------------------------------------------------------------------------------- /applications/templates/observe/namespace.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | name: observe 6 | -------------------------------------------------------------------------------- /applications/templates/observe/project.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: AppProject 4 | metadata: 5 | name: observe 6 | spec: 7 | sourceRepos: 8 | - https://github.com/helm/charts.git 9 | - https://github.com/elastic/helm-charts.git 10 | destinations: 11 | - namespace: observe 12 | server: https://kubernetes.default.svc 13 | - namespace: kube-system 14 | server: https://kubernetes.default.svc 15 | clusterResourceWhitelist: 16 | - group: rbac.authorization.k8s.io 17 | kind: ClusterRoleBinding 18 | - group: rbac.authorization.k8s.io 19 | kind: ClusterRole 20 | - group: apiextensions.k8s.io 21 | kind: CustomResourceDefinition 22 | - group: policy 23 | kind: PodSecurityPolicy 24 | - group: extensions 25 | kind: PodSecurityPolicy 26 | -------------------------------------------------------------------------------- /applications/templates/observe/prometheus-operator.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: prometheus-operator 6 | spec: 7 | project: observe 8 | source: 9 | repoURL: https://github.com/helm/charts.git 10 | path: stable/prometheus-operator 11 | helm: 12 | parameters: 13 | - name: kubeEtcd.enabled 14 | value: "false" 15 | - name: kubeControllerManager.enabled 16 | value: "false" 17 | - name: kubeScheduler.enabled 18 | value: "false" 19 | destination: 20 | server: https://kubernetes.default.svc 21 | namespace: observe 22 | -------------------------------------------------------------------------------- /applications/values.yaml: -------------------------------------------------------------------------------- 1 | ambassador: 2 | enabled: false 3 | -------------------------------------------------------------------------------- /charts/ambassador-mapping/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | name: ambassador-mapping 3 | version: 0.1.0 4 | -------------------------------------------------------------------------------- /charts/ambassador-mapping/templates/ambassador-mapping.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: getambassador.io/v1 3 | kind: Mapping 4 | metadata: 5 | name: ambassador-mapping 6 | spec: 7 | prefix: /ambassador 8 | rewrite: /ambassador/v0/diag/ 9 | service: ambassador-admins.ambassador:8877 10 | -------------------------------------------------------------------------------- /charts/ambassador-mapping/templates/guestbook-mapping.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: getambassador.io/v1 3 | kind: Mapping 4 | metadata: 5 | name: guestbook-mapping 6 | spec: 7 | prefix: /guestbook 8 | # SERVICE_LABEL.NAMESPACE:PORT 9 | service: guestbook-ui.guestbook 10 | -------------------------------------------------------------------------------- /charts/ambassador-mapping/templates/httpbin-mapping.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: getambassador.io/v1 3 | kind: Mapping 4 | metadata: 5 | name: httpbin-mapping 6 | spec: 7 | prefix: /httpbin/ 8 | host_rewrite: httpbin.org 9 | service: httpbin.org:80 10 | -------------------------------------------------------------------------------- /charts/ambassador-mapping/values.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/charts/ambassador-mapping/values.yaml -------------------------------------------------------------------------------- /charts/seed/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | name: seed 3 | version: 0.6.0 4 | -------------------------------------------------------------------------------- /charts/seed/requirements.yaml: -------------------------------------------------------------------------------- 1 | dependencies: 2 | - name: argocd 3 | version: 1.1.1-0 4 | repository: https://edgelevel.github.io/helm-charts 5 | condition: argocd.enabled 6 | -------------------------------------------------------------------------------- /charts/seed/templates/applications.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: Application 4 | metadata: 5 | name: applications 6 | spec: 7 | project: argocd 8 | source: 9 | repoURL: {{ .Values.repository.url }} 10 | targetRevision: {{ .Values.repository.targetRevision }} 11 | path: applications 12 | destination: 13 | server: https://kubernetes.default.svc 14 | namespace: argocd 15 | # https://argoproj.github.io/argo-cd/user-guide/auto_sync 16 | syncPolicy: 17 | automated: {} 18 | -------------------------------------------------------------------------------- /charts/seed/templates/namespace.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | name: argocd 6 | -------------------------------------------------------------------------------- /charts/seed/templates/project.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: argoproj.io/v1alpha1 3 | kind: AppProject 4 | metadata: 5 | name: argocd 6 | spec: 7 | sourceRepos: 8 | - {{ .Values.repository.url }} 9 | - https://github.com/edgelevel/helm-charts.git 10 | {{- range .Values.argocd.sourceRepos }} 11 | - {{ . }} 12 | {{- end }} 13 | destinations: 14 | - namespace: argocd 15 | server: https://kubernetes.default.svc 16 | clusterResourceWhitelist: 17 | - group: '*' 18 | kind: '*' 19 | -------------------------------------------------------------------------------- /charts/seed/values.yaml: -------------------------------------------------------------------------------- 1 | argocd: 2 | enabled: true 3 | sourceRepos: 4 | # - https://github.com/niqdev/do-k8s.git 5 | 6 | repository: 7 | url: https://github.com/edgelevel/gitops-k8s.git 8 | # e.g. branch, SHA, tag 9 | targetRevision: HEAD 10 | -------------------------------------------------------------------------------- /docs/Weaveworks_ContinuousDelivery_wp_2018.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/docs/Weaveworks_ContinuousDelivery_wp_2018.pdf -------------------------------------------------------------------------------- /docs/charts-TODO.txt: -------------------------------------------------------------------------------- 1 | https://stackoverflow.com/questions/23145621/how-to-publish-pages-on-github 2 | 3 | # init gh-pages 4 | git checkout --orphan gh-pages 5 | git rm -rf . 6 | echo "edgelevel-gitops" > index.html 7 | git add index.html 8 | git commit -a -m "add index.html" 9 | git push origin -u gh-pages 10 | 11 | # reset gh-pages 12 | git push origin :gh-pages 13 | git branch -D gh-pages 14 | 15 | https://github.com/edgelevel/gitops-k8s/settings 16 | 17 | Section: GitHub Pages 18 | Source: "gh-pages branch" 19 | Tick: "Enforce HTTPS" 20 | 21 | https://edgelevel.github.io/gitops-k8s 22 | https://edgelevel.github.io/gitops-k8s/index.html 23 | 24 | --- 25 | 26 | https://stackoverflow.com/questions/13897717/push-commits-to-another-branch 27 | https://stackoverflow.com/questions/23145621/how-to-publish-pages-on-github 28 | https://help.github.com/en/articles/creating-project-pages-using-the-command-line 29 | 30 | --- 31 | 32 | TODO master branch 33 | 34 | ``` 35 | git checkout seed 36 | git pull --rebase 37 | git clone -b gh-pages git@github.com:edgelevel/gitops-k8s.git output 38 | 39 | # temporary for official argocd 40 | cd seed/ 41 | helm init --client-only 42 | helm repo add argo https://argoproj.github.io/argo-helm 43 | helm dependency update 44 | cd .. 45 | 46 | cd output/ 47 | helm lint ../seed/ 48 | # output in this folder 49 | helm package ../seed/ 50 | helm repo index . 51 | git add . 52 | git commit -m "release seed v0.1.0" 53 | git push origin gh-pages 54 | cd .. 55 | rm -fr output/ 56 | ``` 57 | 58 | # charts 59 | 60 | * [Using Helm to manage charts](https://helm.sh/docs/developing_charts/#using-helm-to-manage-charts) 61 | * [The chart repository guide](https://helm.sh/docs/developing_charts/#the-chart-repository-guide) 62 | * [Github Pages example](https://helm.sh/docs/developing_charts/#github-pages-example) 63 | * [How to make and share your own Helm package](https://medium.com/containerum/how-to-make-and-share-your-own-helm-package-50ae40f6c221) 64 | * [helm-github-pages](https://github.com/marcellodesales/helm-github-pages) 65 | 66 | 67 | 68 | Public Helm Charts repository 69 | 70 | 71 |

Public Helm Charts repository

72 |

Add this repository

73 | 
74 |       helm repo add edgelevel-gitops https://edgelevel.github.io/gitops-k8s
75 |
76 | 77 | 78 | -------------------------------------------------------------------------------- /docs/img/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/docs/img/architecture.png -------------------------------------------------------------------------------- /docs/img/argocd-ui.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/docs/img/argocd-ui.png -------------------------------------------------------------------------------- /docs/img/bootstrap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/docs/img/bootstrap.png -------------------------------------------------------------------------------- /docs/img/charts.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/edgelevel/gitops-k8s/e067e831961b3f3bf85273606b8fc3da6bc03bc2/docs/img/charts.png -------------------------------------------------------------------------------- /docs/setup.md: -------------------------------------------------------------------------------- 1 | ## Setup 2 | 3 | * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl) to run commands against Kubernetes clusters 4 | * [helm](https://helm.sh/docs/using_helm/#installing-helm) for templating purposes only 5 | * [argocd](https://argoproj.github.io/argo-cd/getting_started/#2-download-argo-cd-cli) CLI 6 | 7 | ### Ubuntu 8 | 9 | ```bash 10 | # kubectl 11 | sudo snap install kubectl --classic 12 | 13 | # helm 14 | sudo snap install helm --classic 15 | 16 | # argocd 17 | sudo mkdir -p /opt/argo 18 | ARGO_VERSION=1.0.0 && \ 19 | sudo curl -L -o /opt/argo/argocd-linux-amd64 \ 20 | https://github.com/argoproj/argo-cd/releases/download/v$ARGO_VERSION/argocd-linux-amd64 21 | sudo chmod +x /opt/argo/argocd-linux-amd64 22 | sudo ln -s /opt/argo/argocd-linux-amd64 /usr/local/bin/argocd 23 | ``` 24 | 25 | ### macOS 26 | 27 | ```bash 28 | # kubectl 29 | brew install kubernetes-cli 30 | 31 | # helm 32 | brew install kubernetes-helm 33 | 34 | # argocd 35 | brew tap argoproj/tap 36 | brew install argoproj/tap/argocd 37 | ``` 38 | 39 | ## Recommended tools 40 | 41 | * [kubectx](https://ahmet.im/blog/kubectx/index.html) - A tool to switch between Kubernetes contexts 42 | * [kube-ps1](https://github.com/jonmosco/kube-ps1) - Kubernetes prompt 43 | 44 | Add to `.bashrc` or `.bash_profile` 45 | ```bash 46 | # K8S PROMPT 47 | [[ -f /opt/kube-ps1/kube-ps1.sh ]] && source /opt/kube-ps1/kube-ps1.sh 48 | #export KUBE_PS1_SYMBOL_COLOR=green 49 | PS1='$(kube_ps1)'$PS1 50 | # default 51 | kubeoff 52 | 53 | # K8S CONFIGS 54 | function kube-config-local { 55 | export KUBECONFIG= 56 | } 57 | function kube-config- { 58 | export KUBECONFIG=~/.kube/-kubeconfig.yaml 59 | kubeon 60 | } 61 | # default 62 | kube-config-local 63 | ``` 64 | -------------------------------------------------------------------------------- /scripts/bootstrap.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # unofficial bash strict mode 4 | set -euo pipefail 5 | IFS=$'\n\t' 6 | 7 | # run from any directory (no symlink allowed) 8 | CURRENT_PATH=$(cd "$(dirname "${BASH_SOURCE[0]}")"; pwd -P) 9 | cd ${CURRENT_PATH} 10 | 11 | ############################## 12 | 13 | ROOT_PATH="${CURRENT_PATH}/.." 14 | SEED_PATH="${ROOT_PATH}/charts/seed/" 15 | 16 | cd ${SEED_PATH} 17 | 18 | # add helm repository 19 | helm repo add edgelevel-public https://edgelevel.github.io/helm-charts 20 | 21 | # download chart locally 22 | helm dependency update 23 | 24 | # apply chart 25 | helm template --values values.yaml . | kubectl apply -n argocd -f - 26 | -------------------------------------------------------------------------------- /scripts/publish_seed.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # unofficial bash strict mode 4 | set -euo pipefail 5 | IFS=$'\n\t' 6 | 7 | # run from any directory (no symlink allowed) 8 | CURRENT_PATH=$(cd "$(dirname "${BASH_SOURCE[0]}")"; pwd -P) 9 | cd ${CURRENT_PATH} 10 | 11 | ############################## 12 | 13 | ROOT_PATH="${CURRENT_PATH}/.." 14 | SEED_PATH="${ROOT_PATH}/charts/seed/" 15 | TMP_REPOSITORY="tmp-gh-pages" 16 | 17 | cd ${SEED_PATH} 18 | helm repo add edgelevel-public https://edgelevel.github.io/helm-charts 19 | helm dependency update 20 | helm template --values values.yaml . 21 | helm lint . 22 | 23 | git clone -b gh-pages git@github.com:edgelevel/gitops-k8s.git ${ROOT_PATH}/${TMP_REPOSITORY} 24 | 25 | cd ${ROOT_PATH}/${TMP_REPOSITORY} 26 | helm package ${SEED_PATH} 27 | helm repo index . 28 | git add . 29 | git commit -m "release seed chart" 30 | git push origin gh-pages 31 | cd ${ROOT_PATH} 32 | rm -fr ${ROOT_PATH}/${TMP_REPOSITORY} 33 | --------------------------------------------------------------------------------