├── README.md └── cluster-XXXX ├── bootstrap ├── base │ ├── argocd-ns.yaml │ └── kustomization.yaml └── overlays │ └── default │ └── kustomization.yaml ├── components ├── applicationsets │ ├── core-components-appset.yaml │ ├── kustomization.yaml │ └── tenants-appset.yaml └── argocdproj │ ├── kustomization.yaml │ └── test-project.yaml ├── core ├── gitops-controller │ └── kustomization.yaml └── sample-admin-workload │ ├── kustomization.yaml │ └── sample-admin-config.yaml └── tenants ├── bgd-blue ├── bgd-deployment.yaml └── kustomization.yaml └── myapp ├── kustomization.yaml ├── myapp-deployment.yaml ├── myapp-ns.yaml └── myapp-service.yaml /README.md: -------------------------------------------------------------------------------- 1 | # Kubernetes GitOps Repo 2 | 3 | This is an example on how I would structure a 1:1 (repo-to-single cluster) 4 | git repo for a Kubernetes cluster. This is based on the [OpenShift](https://github.com/christianh814/example-openshift-go-repo) 5 | repo I created. 6 | 7 | This example assumes (as I mentioned in the 1:1 part above) that it's a 8 | single repo for a single cluster. However, this can be modified (quite 9 | easily) for poly/mono repos or for multiple clusters. This is meant as 10 | a good starting point and not what your final repo will look like. 11 | 12 | This is based on Argo CD but the same principals can be applied to Flux. 13 | 14 | # Structure 15 | 16 | Below is an explanation on how this repo is laid out. You'll notice 17 | that I use [Kustomize](https://kustomize.io/) heavily. I do this since I 18 | follow the [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) 19 | principal when it comes to YAML files. 20 | 21 | ```shell 22 | cluster-XXXX/ # 1 23 | ├── bootstrap # 2 24 | │   ├── base 25 | │   │   ├── argocd-ns.yaml 26 | │   │   └── kustomization.yaml 27 | │   └── overlays 28 | │   └── default 29 | │   └── kustomization.yaml 30 | ├── components # 3 31 | │   ├── applicationsets 32 | │   │   ├── core-components-appset.yaml 33 | │   │   ├── kustomization.yaml 34 | │   │   └── tenants-appset.yaml 35 | │   └── argocdproj 36 | │   ├── kustomization.yaml 37 | │   └── test-project.yaml 38 | ├── core # 4 39 | │   ├── gitops-controller 40 | │   │   └── kustomization.yaml 41 | │   └── sample-admin-config 42 | │   ├── kustomization.yaml 43 | │   └── sample-admin-config.yaml 44 | └── tenants # 5 45 | ├── bgd-blue 46 | │   ├── bgd-deployment.yaml 47 | │   └── kustomization.yaml 48 | └── myapp 49 | ├── kustomization.yaml 50 | ├── myapp-deployment.yaml 51 | ├── myapp-ns.yaml 52 | └── myapp-service.yaml 53 | ``` 54 | 55 | |#|Directory Name|Description| 56 | |---|----------------|-----------------| 57 | | 1. |`cluster-XXXX`| This is the cluster name. This name should be unique to the specific cluster you're targeting. If you're using CAPI, this should be the name of your cluster, the output of `kubectl get cluster`| 58 | | 2. | `bootstrap` | This is where bootstrapping specifc configurations are stored. These are items that get the cluster/automation started. They are usually install manifests.

`base` is where are the "common" YAML would live and `overlays` are configurations specific to the cluster.

The `kustomization.yaml` file in `default` has `cluster-XXXX/components/applicationsets/` and `cluster-XXXX/components/argocdproj/` as a part of it's `bases` config.| 59 | | 3. | `components` | This is where specific components for the GitOps Controller lives (in this case Argo CD).

`applicationsets` is where all the ApplicationSets YAMLs live and `argocdproj` is where the ArgoAppProject YAMLs live.

Other things that can live here are RBAC, Git repo, and other Argo CD specific configurations (each in their repsective directories).| 60 | | 4. | `core` | This is where YAML for the core functionality of the cluster live. Here is where the Kubernetes administrator will put things that is necissary for the functionality of the cluster (like cluster configs or cluster workloads).

Under `gitops-controller` is where you are using Argo CD to manage itself. The `kustomization.yaml` file uses `cluster-XXXX/bootstrap/overlays/default` in it's `bases` configuration. This `core` directory gets deployed as an applicationset which can be found under `cluster-XXXX/components/applicationsets/core-components-appset.yaml`.

To add a new "core functionality" workoad, one needs to add a directory with some yaml in the `core` directory. See the `sample-admin-config` directory as an example.| 61 | | 5. | `tenants` | This is where the workloads for this cluster live.

Similar to `core`, the `tenants` directory gets loaded as part of an ApplicationSet that is under `cluster-XXXX/components/applicationsets/tenants-appset.yaml`.

This is where Devlopers/Release Engineers do the work. They just need to commit a directory with some YAML and the applicationset takes care of creating the workload.

**Note** that `bgd-blue/kustomization.yaml` file points to another Git repo. This is to show that you can host your YAML in one repo, or many repos.| 62 | 63 | # Testing 64 | 65 | To see this in action, first get yourself a cluster (using [kind](kind.sigs.k8s.io/) as an example) 66 | 67 | ```shell 68 | kind create cluster 69 | ``` 70 | 71 | Then, just apply this repo. 72 | 73 | ```shell 74 | until kubectl apply -k https://github.com/christianh814/example-kubernetes-go-repo/cluster-XXXX/bootstrap/overlays/default; do sleep 3; done 75 | ``` 76 | 77 | This should give you 4 applications 78 | 79 | ```shell 80 | $ kubectl get applications -n argocd 81 | NAME SYNC STATUS HEALTH STATUS 82 | bgd-blue Synced Healthy 83 | sample-admin-workload Synced Healthy 84 | myapp Synced Healthy 85 | gitops-controller Synced Healthy 86 | ``` 87 | 88 | Backed by 2 applicationsets 89 | 90 | ```shell 91 | $ kubectl get appsets -n argocd 92 | NAME AGE 93 | cluster 110s 94 | tenants 110s 95 | ``` 96 | 97 | To see the Argo CD UI, you'll first need the password 98 | 99 | ```shell 100 | kubectl get secret/argocd-initial-admin-secret -n argocd -o jsonpath='{.data.password}' | base64 -d ; echo 101 | ``` 102 | 103 | Then port-forward to see it in your browser (using `admin` as the username). 104 | 105 | ```shell 106 | kubectl -n argocd port-forward service/argocd-server 8080:443 107 | ``` 108 | 109 | # Enjoy 110 | 111 | Fork and Enjoy! 112 | 113 | > :warning: Feel free to fork this and play around with it, but remember if you'll have to change the applicationsets configuration and where the bases are pointing to if you're chanigng names of things 114 | -------------------------------------------------------------------------------- /cluster-XXXX/bootstrap/base/argocd-ns.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Namespace 3 | metadata: 4 | name: argocd 5 | spec: {} 6 | status: {} 7 | -------------------------------------------------------------------------------- /cluster-XXXX/bootstrap/base/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | resources: 5 | - argocd-ns.yaml 6 | - https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml 7 | #- https://raw.githubusercontent.com/argoproj-labs/applicationset/stable/manifests/install.yaml 8 | -------------------------------------------------------------------------------- /cluster-XXXX/bootstrap/overlays/default/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | namespace: argocd 4 | 5 | bases: 6 | - ../../base 7 | - ../../../components/applicationsets 8 | - ../../../components/argocdproj 9 | -------------------------------------------------------------------------------- /cluster-XXXX/components/applicationsets/core-components-appset.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: ApplicationSet 3 | metadata: 4 | name: cluster 5 | namespace: argocd 6 | spec: 7 | generators: 8 | - git: 9 | repoURL: https://github.com/christianh814/example-kubernetes-go-repo 10 | revision: main 11 | directories: 12 | - path: cluster-XXXX/core/* 13 | template: 14 | metadata: 15 | name: '{{path.basename}}' 16 | spec: 17 | project: default 18 | syncPolicy: 19 | automated: 20 | prune: true 21 | selfHeal: true 22 | retry: 23 | limit: 15 24 | backoff: 25 | duration: 15s 26 | factor: 2 27 | maxDuration: 5m 28 | source: 29 | repoURL: https://github.com/christianh814/example-kubernetes-go-repo 30 | targetRevision: main 31 | path: '{{path}}' 32 | destination: 33 | server: https://kubernetes.default.svc 34 | -------------------------------------------------------------------------------- /cluster-XXXX/components/applicationsets/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | resources: 5 | - core-components-appset.yaml 6 | - tenants-appset.yaml 7 | -------------------------------------------------------------------------------- /cluster-XXXX/components/applicationsets/tenants-appset.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: ApplicationSet 3 | metadata: 4 | name: tenants 5 | namespace: argocd 6 | spec: 7 | generators: 8 | - git: 9 | repoURL: https://github.com/christianh814/example-kubernetes-go-repo 10 | revision: main 11 | directories: 12 | - path: cluster-XXXX/tenants/* 13 | template: 14 | metadata: 15 | name: '{{path.basename}}' 16 | spec: 17 | project: default 18 | syncPolicy: 19 | automated: 20 | prune: true 21 | selfHeal: true 22 | retry: 23 | limit: 15 24 | backoff: 25 | duration: 15s 26 | factor: 2 27 | maxDuration: 5m 28 | source: 29 | repoURL: https://github.com/christianh814/example-kubernetes-go-repo 30 | targetRevision: main 31 | path: '{{path}}' 32 | destination: 33 | server: https://kubernetes.default.svc 34 | -------------------------------------------------------------------------------- /cluster-XXXX/components/argocdproj/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | resources: 5 | - test-project.yaml 6 | -------------------------------------------------------------------------------- /cluster-XXXX/components/argocdproj/test-project.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: AppProject 3 | metadata: 4 | name: test-project 5 | namespace: argocd 6 | spec: {} 7 | -------------------------------------------------------------------------------- /cluster-XXXX/core/gitops-controller/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | bases: 5 | - ../../bootstrap/overlays/default/ 6 | 7 | -------------------------------------------------------------------------------- /cluster-XXXX/core/sample-admin-workload/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | namespace: default 4 | 5 | resources: 6 | - sample-admin-config.yaml 7 | -------------------------------------------------------------------------------- /cluster-XXXX/core/sample-admin-workload/sample-admin-config.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ConfigMap 3 | metadata: 4 | name: cskb-config 5 | data: 6 | csbk.config: | 7 | cool=yes 8 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/bgd-blue/bgd-deployment.yaml: -------------------------------------------------------------------------------- 1 | - op: replace 2 | path: /spec/replicas 3 | value: 3 4 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/bgd-blue/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | bases: 5 | - https://gitlab.com/christianh814/test-applicationsets/list-generator/k8s/overlays/noing-blue?ref=main 6 | 7 | patchesJson6902: 8 | - target: 9 | version: v1 10 | group: apps 11 | kind: Deployment 12 | name: bgd 13 | path: bgd-deployment.yaml 14 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/myapp/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | 4 | resources: 5 | - myapp-ns.yaml 6 | - myapp-deployment.yaml 7 | - myapp-service.yaml 8 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/myapp/myapp-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | creationTimestamp: null 5 | labels: 6 | app: welcome-php 7 | name: welcome-php 8 | namespace: foobar 9 | spec: 10 | replicas: 1 11 | selector: 12 | matchLabels: 13 | app: welcome-php 14 | strategy: {} 15 | template: 16 | metadata: 17 | creationTimestamp: null 18 | labels: 19 | app: welcome-php 20 | spec: 21 | containers: 22 | - image: quay.io/redhatworkshops/welcome-php 23 | name: welcome-php 24 | resources: {} 25 | status: {} 26 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/myapp/myapp-ns.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Namespace 3 | metadata: 4 | name: foobar 5 | spec: {} 6 | status: {} 7 | -------------------------------------------------------------------------------- /cluster-XXXX/tenants/myapp/myapp-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | creationTimestamp: null 5 | labels: 6 | app: welcome-php 7 | name: welcome-php 8 | namespace: foobar 9 | spec: 10 | ports: 11 | - port: 8080 12 | protocol: TCP 13 | targetPort: 8080 14 | selector: 15 | app: welcome-php 16 | status: 17 | loadBalancer: {} 18 | --------------------------------------------------------------------------------