├── .tiltignore ├── .github ├── ISSUE_TEMPLATE │ ├── config.yml │ └── issue_template.yml ├── workflows │ ├── kind-versions.json │ ├── static-checks.yml │ ├── assets.yaml │ ├── load-kind-versions.yaml │ ├── release-please.yaml │ ├── unit-tests.yaml │ └── integration-tests.yaml ├── pull_request_template.md └── renovate.json ├── addons ├── knative │ ├── Chart.yaml │ ├── templates │ │ ├── eventing.yaml │ │ └── serving.yaml │ ├── .helmignore │ └── values.yaml ├── istio │ ├── ingressgateway │ │ ├── values.yaml │ │ ├── Chart.yaml │ │ └── templates │ │ │ └── gateway.yaml │ ├── istiod │ │ └── Chart.yaml │ └── base │ │ └── Chart.yaml ├── argocd │ ├── Chart.yaml │ ├── .helmignore │ └── values.yaml ├── kubernetes-replicator │ ├── Chart.yaml │ ├── .helmignore │ └── values.yaml ├── grafana │ ├── templates │ │ ├── grafana-secret.yaml │ │ └── virtual-service.yaml │ ├── Chart.yaml │ └── values.yaml ├── dex │ ├── Chart.yaml │ ├── .helmignore │ ├── templates │ │ └── virtual-service.yaml │ └── values.yaml ├── loki │ ├── Chart.yaml │ └── values.yaml ├── cert-manager │ ├── Chart.yaml │ ├── .helmignore │ └── values.yaml ├── kube-prometheus-stack │ ├── Chart.yaml │ ├── templates │ │ └── virtual-service.yaml │ └── values.yaml └── opentelemetry-collector │ ├── Chart.yaml │ └── values.yaml ├── app-of-apps ├── Chart.yaml ├── .helmignore ├── templates │ └── apps.yaml └── values.yaml ├── manifests ├── kind │ ├── values.yaml │ ├── .helmignore │ ├── templates │ │ └── cluster.yaml │ └── Chart.yaml ├── ci-service-account │ ├── ci-service-account.yaml │ ├── ci-token.yaml │ ├── ci-cluster-role-binding.yaml │ ├── ci-cluster-role.yaml │ └── README.md ├── bootstrap │ ├── values.yaml │ ├── .helmignore │ ├── overrides.local.yaml.tmpl │ ├── templates │ │ └── app-of-apps.yaml │ └── Chart.yaml ├── demo-app-config.yaml └── argocd-repo.yaml ├── scripts ├── create-ci-service-account.sh ├── tilt-up.sh ├── tilt-down.sh ├── kind-down.sh ├── tilt-validate.sh ├── clean-test.sh ├── kind-up.sh ├── generate-certs.sh ├── cloud.sh ├── run-test.sh ├── upload-assets.sh └── ci.sh ├── utils ├── metallb │ ├── templates │ │ ├── l2advertisement.yaml │ │ └── ipaddresspool.yaml │ ├── values.yaml │ └── Chart.yaml └── git-http-backend │ ├── chart │ ├── Chart.yaml │ ├── templates │ │ ├── serviceaccount.yaml │ │ ├── service.yaml │ │ ├── _helpers.tpl │ │ └── deployment.yaml │ ├── .helmignore │ └── values.yaml │ └── docker │ ├── nginx.conf │ └── Dockerfile ├── docs ├── img │ ├── inner-workings │ │ └── k8s-addons-internals.png │ └── quick-start │ │ └── k8s-addons-quick-start.png ├── KNOWN_ISSUES.md ├── LOG_AGGREGATION.md ├── SAMPLE_DEPLOY_APP.md ├── SAMPLE_PROMETHEUS_APP.md ├── DEX_GITHUB_INTEGRATION.md └── ADDONS.md ├── .gitignore ├── .tool-versions ├── examples ├── sample-deploy-app │ └── sample-deploy-app.yaml ├── sample-app │ ├── sample-app.yaml │ ├── sample_app_test.go │ └── go.mod └── sample-prometheus-app │ ├── sample_prometheus_app_test.go │ ├── go.mod │ └── sample-prometheus-app.yaml ├── LICENSE.md ├── tests ├── opentelemetry_collector_addon_test.go ├── argocd_addon_test.go ├── cert_manager_addon_test.go ├── dex_addon_test.go ├── go.mod ├── grafana_addon_test.go ├── kubernetes_replicator_test.go ├── knative_istio_addon_test.go ├── loki_addon_test.go ├── prometheus_addon_test.go └── helpers.go ├── .pre-commit-config.yaml ├── architectural-design-records ├── dependency-management-automation.md ├── gitops.md └── release-management-automation.md ├── .envrc ├── CHANGELOG.md ├── Makefile ├── Tiltfile ├── CONTRIBUTING.md └── README.md /.tiltignore: -------------------------------------------------------------------------------- 1 | **/charts 2 | **/tmpcharts 3 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/config.yml: -------------------------------------------------------------------------------- 1 | blank_issues_enabled: false 2 | -------------------------------------------------------------------------------- /addons/knative/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: knative 3 | version: 0.1.0 4 | -------------------------------------------------------------------------------- /app-of-apps/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: argocd-apps 3 | version: v0.1.0 # x-release-please-version 4 | -------------------------------------------------------------------------------- /manifests/kind/values.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | certSANs: 3 | - "127.0.0.1" 4 | - "localhost" 5 | - "*.tcp.ngrok.io" 6 | -------------------------------------------------------------------------------- /scripts/create-ci-service-account.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | kubectl apply -f manifests/ci-service-account 4 | -------------------------------------------------------------------------------- /manifests/ci-service-account/ci-service-account.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ServiceAccount 3 | metadata: 4 | name: ci-bot 5 | -------------------------------------------------------------------------------- /manifests/bootstrap/values.yaml: -------------------------------------------------------------------------------- 1 | revision: HEAD 2 | repoURL: "https://github.com/nearform/initium-platform.git" 3 | externalDomain: example.com 4 | -------------------------------------------------------------------------------- /utils/metallb/templates/l2advertisement.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: metallb.io/v1beta1 2 | kind: L2Advertisement 3 | metadata: 4 | name: initium-platform 5 | -------------------------------------------------------------------------------- /addons/istio/ingressgateway/values.yaml: -------------------------------------------------------------------------------- 1 | gateway: 2 | externalHost: "*.example.com" 3 | host: "*.kube.local" 4 | tlsCertName: wildcard.kube.local-tls 5 | -------------------------------------------------------------------------------- /docs/img/inner-workings/k8s-addons-internals.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nearform/initium-platform/HEAD/docs/img/inner-workings/k8s-addons-internals.png -------------------------------------------------------------------------------- /docs/img/quick-start/k8s-addons-quick-start.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nearform/initium-platform/HEAD/docs/img/quick-start/k8s-addons-quick-start.png -------------------------------------------------------------------------------- /scripts/tilt-up.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | if kind get kubeconfig --name ${INITIUM_REPO_NAME} > /dev/null 2>&1; then 6 | tilt up 7 | fi 8 | -------------------------------------------------------------------------------- /utils/metallb/values.yaml: -------------------------------------------------------------------------------- 1 | metallb-source: 2 | crds: 3 | validationFailurePolicy: Ignore 4 | 5 | cidrBlock: '10.0.0.0/24' # this value is overriden by set command in tiltfile 6 | -------------------------------------------------------------------------------- /scripts/tilt-down.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | if kind get kubeconfig --name ${INITIUM_REPO_NAME} > /dev/null 2>&1; then 6 | tilt down --delete-namespaces 7 | fi 8 | -------------------------------------------------------------------------------- /utils/metallb/templates/ipaddresspool.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: metallb.io/v1beta1 2 | kind: IPAddressPool 3 | metadata: 4 | name: initium-platform 5 | spec: 6 | addresses: 7 | - {{ .Values.cidrBlock }} 8 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | argocd/charts 2 | *Chart.lock 3 | *charts/ 4 | manifests/bootstrap/overrides.local.yaml 5 | .ssl 6 | .idea 7 | local.env 8 | kind.local.yaml 9 | .DS_Store 10 | .cache 11 | 12 | .ssl 13 | -------------------------------------------------------------------------------- /scripts/kind-down.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | if kind get kubeconfig --name ${INITIUM_REPO_NAME} > /dev/null 2>&1; then 6 | kind delete cluster --name ${INITIUM_REPO_NAME} 7 | fi 8 | -------------------------------------------------------------------------------- /scripts/tilt-validate.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | RET=$(tilt alpha tiltfile-result) 6 | if [ $? -ne 0 ]; then 7 | echo "$RET" 8 | else 9 | echo "Tiltfile validated successfully!" 10 | fi 11 | -------------------------------------------------------------------------------- /manifests/ci-service-account/ci-token.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: ci-bot-token 5 | annotations: 6 | kubernetes.io/service-account.name: "ci-bot" 7 | type: kubernetes.io/service-account-token 8 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: git-http-backend 3 | description: A Helm chart for a dead simple, insecure git-over-http server using nginx 4 | type: application 5 | version: 0.1.0 6 | appVersion: "latest" 7 | -------------------------------------------------------------------------------- /addons/argocd/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: argocd 3 | version: v0.1.0 # x-release-please-version 4 | dependencies: 5 | - alias: argocd-source 6 | name: argo-cd 7 | version: 5.16.14 8 | repository: https://argoproj.github.io/argo-helm 9 | -------------------------------------------------------------------------------- /.github/workflows/kind-versions.json: -------------------------------------------------------------------------------- 1 | [ 2 | "kindest/node:v1.26.6@sha256:5e5d789e90c1512c8c480844e0985bc3b4da4ba66179cc5b540fe5b785ca97b5", 3 | "kindest/node:v1.27.3@sha256:9dd3392d79af1b084671b05bcf65b21de476256ad1dcc853d9f3b10b4ac52dde" 4 | ] 5 | -------------------------------------------------------------------------------- /.tool-versions: -------------------------------------------------------------------------------- 1 | argocd 2.9.3 2 | awscli 2.14.5 3 | eksctl 0.151.0 4 | gcloud 445.0.0 5 | golang 1.21.4 6 | golangci-lint 1.55.2 7 | helm 3.13.2 8 | initium 0.5.0 9 | jq 1.6 10 | kind 0.17.0 11 | kubectl 1.28.4 12 | pre-commit 3.5.0 13 | tilt 0.33.3 14 | -------------------------------------------------------------------------------- /addons/kubernetes-replicator/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: kubernetes-replicator 3 | version: v0.1.0 # x-release-please-version 4 | dependencies: 5 | - alias: kubernetes-replicator-source 6 | name: kubernetes-replicator 7 | version: 2.9.2 8 | repository: https://helm.mittwald.de/ 9 | -------------------------------------------------------------------------------- /manifests/demo-app-config.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: app-config 5 | namespace: default 6 | annotations: 7 | replicator.v1.mittwald.de/replicate-to: "initium,initium.+" 8 | type: Opaque 9 | data: 10 | db_uname: YWRtaW4K 11 | db_pwd: YWRtaW4K 12 | -------------------------------------------------------------------------------- /addons/grafana/templates/grafana-secret.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: grafana-initial-admin-secret 5 | annotations: 6 | argocd.argoproj.io/sync-wave: "-1" 7 | data: 8 | admin-user: {{ b64enc "admin" }} 9 | admin-password: {{ randAlphaNum 20 | b64enc }} 10 | -------------------------------------------------------------------------------- /addons/knative/templates/eventing.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Namespace 3 | metadata: 4 | name: knative-eventing 5 | --- 6 | apiVersion: operator.knative.dev/v1beta1 7 | kind: KnativeEventing 8 | metadata: 9 | name: knative-eventing 10 | namespace: knative-eventing 11 | spec: 12 | version: "1.11" 13 | -------------------------------------------------------------------------------- /utils/metallb/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: metallb 3 | description: metallb wrapper chart 4 | type: application 5 | version: v0.4.0 # x-release-please-version 6 | dependencies: 7 | - name: metallb 8 | alias: metallb-source 9 | version: 0.13.12 10 | repository: https://metallb.github.io/metallb 11 | -------------------------------------------------------------------------------- /addons/dex/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: dex 3 | description: dex wrapper chart 4 | type: application 5 | version: v0.1.0 # x-release-please-version 6 | dependencies: 7 | - name: dex 8 | alias: dex-source 9 | version: 0.15.3 10 | repository: https://charts.dexidp.io 11 | condition: dex-source.enabled 12 | -------------------------------------------------------------------------------- /scripts/clean-test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | 5 | source .envrc 6 | 7 | if [ -d "$GOPATH" ]; then 8 | echo "Deleting ${GOPATH}..." 9 | sudo rm -rf $GOPATH 10 | fi 11 | 12 | if [ -d "$HOME/.kube/cache/oidc-login" ]; then 13 | echo "Deleting $HOME/.kube/cache/oidc-login..." 14 | sudo rm -rf $HOME/.kube/cache/oidc-login 15 | fi 16 | -------------------------------------------------------------------------------- /addons/loki/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: loki 3 | description: Grafana Loki Helm chart for Kubernetes 4 | type: application 5 | version: v0.0.1 # x-release-please-version 6 | dependencies: 7 | - name: loki 8 | alias: loki-source 9 | version: 5.39.0 10 | repository: https://grafana.github.io/helm-charts 11 | condition: loki-source.enabled 12 | -------------------------------------------------------------------------------- /addons/grafana/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: Grafana 3 | description: Grafana Helm chart for Kubernetes 4 | type: application 5 | version: v0.0.1 # x-release-please-version 6 | dependencies: 7 | - name: grafana 8 | alias: grafana-source 9 | version: 7.0.11 10 | repository: https://grafana.github.io/helm-charts 11 | condition: grafana-source.enabled 12 | -------------------------------------------------------------------------------- /addons/istio/istiod/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: istiod 3 | description: istiod wrapper chart 4 | type: application 5 | version: v0.1.0 # x-release-please-version 6 | dependencies: 7 | - name: istiod 8 | alias: istiod-source 9 | version: 1.20.0 10 | repository: https://istio-release.storage.googleapis.com/charts 11 | condition: istio-source.enabled 12 | -------------------------------------------------------------------------------- /addons/cert-manager/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: cert-manager 3 | description: cert-manager wrapper chart 4 | type: application 5 | version: v0.1.0 # x-release-please-version 6 | dependencies: 7 | - name: cert-manager 8 | alias: cert-manager-source 9 | version: v1.10.1 10 | repository: https://charts.jetstack.io 11 | condition: cert-manager-source.enabled 12 | -------------------------------------------------------------------------------- /addons/istio/base/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: istio-base 3 | description: istio-base wrapper chart 4 | type: application 5 | version: v0.1.0 # x-release-please-version 6 | dependencies: 7 | - name: base 8 | alias: istio-base-source 9 | version: 1.20.0 10 | repository: https://istio-release.storage.googleapis.com/charts 11 | condition: istio-source.enabled 12 | -------------------------------------------------------------------------------- /manifests/ci-service-account/ci-cluster-role-binding.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: rbac.authorization.k8s.io/v1 2 | kind: ClusterRoleBinding 3 | metadata: 4 | name: continuous-deployment 5 | roleRef: 6 | apiGroup: rbac.authorization.k8s.io 7 | kind: ClusterRole 8 | name: continuous-deployment 9 | subjects: 10 | - kind: ServiceAccount 11 | name: ci-bot 12 | namespace: default 13 | -------------------------------------------------------------------------------- /addons/istio/ingressgateway/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: istio-ingress 3 | description: istio-ingress wrapper chart 4 | type: application 5 | version: v0.0.1 # x-release-please-version 6 | dependencies: 7 | - name: gateway 8 | alias: istio-gateway-source 9 | version: 1.20.0 10 | repository: https://istio-release.storage.googleapis.com/charts 11 | condition: istio-source.enabled 12 | -------------------------------------------------------------------------------- /examples/sample-deploy-app/sample-deploy-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: serving.knative.dev/v1 2 | kind: Service 3 | metadata: 4 | name: "helloworld" 5 | spec: 6 | template: 7 | spec: 8 | containers: 9 | - image: "gcr.io/knative-samples/helloworld-go" 10 | env: 11 | - name: "TARGET" 12 | value: "world" 13 | ports: 14 | - containerPort: 80 15 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/templates/serviceaccount.yaml: -------------------------------------------------------------------------------- 1 | {{- if .Values.serviceAccount.create -}} 2 | apiVersion: v1 3 | kind: ServiceAccount 4 | metadata: 5 | name: {{ include "git-http-backend.serviceAccountName" . }} 6 | labels: 7 | {{- include "git-http-backend.labels" . | nindent 4 }} 8 | {{- with .Values.serviceAccount.annotations }} 9 | annotations: 10 | {{- toYaml . | nindent 4 }} 11 | {{- end }} 12 | {{- end }} 13 | -------------------------------------------------------------------------------- /addons/kube-prometheus-stack/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: kube-prometheus-stack 3 | description: kube-prometheus-stack wrapper chart 4 | type: application 5 | version: v0.1.0 # x-release-please-version 6 | dependencies: 7 | - name: kube-prometheus-stack 8 | alias: kube-prometheus-stack-source 9 | version: 54.2.2 10 | repository: https://prometheus-community.github.io/helm-charts 11 | condition: kube-prometheus-stack-source.enabled 12 | -------------------------------------------------------------------------------- /addons/dex/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /addons/opentelemetry-collector/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: opentelemetry-collector 3 | description: OpenTelemetry collector Helm chart for Kubernetes 4 | type: application 5 | version: v0.0.1 # x-release-please-version 6 | dependencies: 7 | - name: opentelemetry-collector 8 | alias: otlp-collector-source 9 | version: 0.74.1 10 | repository: https://open-telemetry.github.io/opentelemetry-helm-charts 11 | condition: otlp-collector-source.enabled 12 | -------------------------------------------------------------------------------- /app-of-apps/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /addons/argocd/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /addons/knative/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /manifests/kind/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /addons/cert-manager/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /manifests/bootstrap/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /addons/kubernetes-replicator/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *.orig 18 | *~ 19 | # Various IDEs 20 | .project 21 | .idea/ 22 | *.tmproj 23 | .vscode/ 24 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/templates/service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: {{ include "git-http-backend.fullname" . }} 5 | labels: 6 | {{- include "git-http-backend.labels" . | nindent 4 }} 7 | spec: 8 | type: {{ .Values.service.type }} 9 | ports: 10 | - port: {{ .Values.service.port }} 11 | targetPort: http 12 | protocol: TCP 13 | name: http 14 | selector: 15 | {{- include "git-http-backend.selectorLabels" . | nindent 4 }} 16 | -------------------------------------------------------------------------------- /addons/knative/templates/serving.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Namespace 3 | metadata: 4 | name: knative-serving 5 | --- 6 | apiVersion: operator.knative.dev/v1beta1 7 | kind: KnativeServing 8 | metadata: 9 | name: knative-serving 10 | namespace: knative-serving 11 | spec: 12 | version: "1.11" 13 | --- 14 | apiVersion: v1 15 | kind: ConfigMap 16 | metadata: 17 | name: config-domain 18 | namespace: knative-serving 19 | data: 20 | # Use this base domain for knative apps 21 | {{ .Values.externalDomain }}: "" 22 | -------------------------------------------------------------------------------- /manifests/bootstrap/overrides.local.yaml.tmpl: -------------------------------------------------------------------------------- 1 | # You can use this file to test values changes locally. 2 | # Just create a copy without the .tmpl extension and [re]run make 3 | # Tilt will see the new file and include it when loading the helm chart 4 | # Git ignores the file, so there is no risk of pushing it 5 | 6 | # This helm chart values will be merged with the values of 7 | # the app-of-apps chart defined int the root directory. 8 | # eg. You can disable the dex addon with: 9 | # apps: 10 | # dex: 11 | # excluded: true 12 | -------------------------------------------------------------------------------- /addons/knative/values.yaml: -------------------------------------------------------------------------------- 1 | requestLogTemplate: '{"httpRequest": {"requestMethod": "{{.Request.Method}}", "requestUrl": "{{js .Request.RequestURI}}", "requestSize": "{{.Request.ContentLength}}", "status": {{.Response.Code}}, "responseSize": "{{.Response.Size}}", "userAgent": "{{js .Request.UserAgent}}", "remoteIp": "{{js .Request.RemoteAddr}}", "serverIp": "{{.Revision.PodIP}}", "referer": "{{js .Request.Referer}}", "latency": "{{.Response.Latency}}s", "protocol": "{{.Request.Proto}}"}, "traceId": "{{index .Request.Header "X-B3-Traceid"}}"}' 2 | externalDomain: example.com 3 | -------------------------------------------------------------------------------- /manifests/argocd-repo.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: initium-platform 5 | namespace: argocd 6 | labels: 7 | argocd.argoproj.io/secret-type: repository 8 | stringData: 9 | type: git 10 | url: https://github.com/nearform/initium-platform.git 11 | # githubAppPrivateKey: ${GITHUB_APP_PRIVATE_KEY} 12 | # githubAppId: ${GITHUB_APP_ID} 13 | # githubAppInstallationId: ${GITHUB_APP_INSTALLATION_ID} 14 | # Use username and password if you don't have a GitHub App 15 | username: "${GH_USERNAME}" 16 | password: "${GH_PAT}" 17 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright 2022 NearForm Ltd. 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 4 | 5 | http://www.apache.org/licenses/LICENSE-2.0 6 | 7 | Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 8 | -------------------------------------------------------------------------------- /manifests/ci-service-account/ci-cluster-role.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: rbac.authorization.k8s.io/v1 2 | kind: ClusterRole 3 | metadata: 4 | name: continuous-deployment 5 | rules: 6 | - apiGroups: 7 | - '' 8 | - apps 9 | - networking.k8s.io 10 | resources: 11 | - namespaces 12 | - deployments 13 | - replicasets 14 | - ingresses 15 | - services 16 | - secrets 17 | verbs: 18 | - create 19 | - delete 20 | - deletecollection 21 | - get 22 | - list 23 | - patch 24 | - update 25 | - watch 26 | -------------------------------------------------------------------------------- /addons/dex/templates/virtual-service.yaml: -------------------------------------------------------------------------------- 1 | {{ if .Values.virtualService.enabled }} 2 | apiVersion: networking.istio.io/v1alpha3 3 | kind: VirtualService 4 | metadata: 5 | name: {{ .Values.virtualService.name }} 6 | namespace: {{ .Values.virtualService.namespace }} 7 | spec: 8 | gateways: 9 | - {{ .Values.virtualService.gateway }} 10 | hosts: 11 | - {{ .Values.virtualService.host }} 12 | http: 13 | - route: 14 | - destination: 15 | host: {{ .Values.virtualService.serviceName }} 16 | port: 17 | number: {{ .Values.virtualService.port }} 18 | {{ end }} 19 | -------------------------------------------------------------------------------- /addons/istio/ingressgateway/templates/gateway.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.istio.io/v1beta1 2 | kind: Gateway 3 | metadata: 4 | name: kube-gateway 5 | namespace: istio-ingress 6 | spec: 7 | selector: 8 | istio: ingressgateway 9 | servers: 10 | - hosts: 11 | - "*.{{ .Values.externalDomain }}" 12 | port: 13 | name: http 14 | number: 80 15 | protocol: HTTP 16 | - hosts: 17 | - {{ .Values.gateway.host | quote }} 18 | port: 19 | name: https 20 | number: 443 21 | protocol: HTTPS 22 | tls: 23 | credentialName: {{ .Values.gateway.tlsCertName }} 24 | mode: SIMPLE 25 | -------------------------------------------------------------------------------- /.github/workflows/static-checks.yml: -------------------------------------------------------------------------------- 1 | name: Run static checks 2 | 3 | on: 4 | pull_request: 5 | branches: 6 | - main 7 | 8 | jobs: 9 | format: 10 | name: Lint & format 11 | runs-on: ubuntu-latest 12 | steps: 13 | - uses: actions/checkout@v4 14 | 15 | - name: Static check 16 | run: | 17 | # Install asdf and expose to PATH 18 | git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.13.1 19 | . $HOME/.asdf/asdf.sh 20 | # Add asdf plugins and install tools in .tool-versions 21 | make asdf_install 22 | # Run checks 23 | make validate 24 | -------------------------------------------------------------------------------- /scripts/kind-up.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | KIND_PATH="manifests/kind" 4 | KIND_COMPUTED_PATH="kind.local.yaml" 5 | 6 | source .envrc 7 | 8 | if ! kind get kubeconfig --name ${INITIUM_REPO_NAME} > /dev/null 2>&1; then 9 | helm template ${KIND_PATH} --set k8s_version="${INITIUM_K8S_VERSION}" \ 10 | --set repo_host_path="${INITIUM_REPO_HOST_PATH}" \ 11 | --set repo_node_path="${INITIUM_REPO_NODE_PATH}" \ 12 | --set name="${INITIUM_REPO_NAME}" \ 13 | --set repo_name="${INITIUM_REPO_NAME}" > ${KIND_COMPUTED_PATH} 14 | 15 | kind create cluster --config ${KIND_COMPUTED_PATH} 16 | fi 17 | -------------------------------------------------------------------------------- /addons/grafana/templates/virtual-service.yaml: -------------------------------------------------------------------------------- 1 | {{- $name := .Values.externalDomain | required ".Values.externalDomain is required." -}} 2 | --- 3 | apiVersion: networking.istio.io/v1alpha3 4 | kind: VirtualService 5 | metadata: 6 | name: {{ .Values.virtualService.name }} 7 | namespace: {{ .Values.virtualService.namespace }} 8 | spec: 9 | gateways: 10 | - {{ .Values.virtualService.gateway }} 11 | hosts: 12 | - {{ .Values.virtualService.internalHost | quote }} 13 | - {{ .Values.virtualService.name}}.{{ .Values.externalDomain }} 14 | http: 15 | - route: 16 | - destination: 17 | host: {{ .Values.virtualService.serviceName }} 18 | port: 19 | number: {{ .Values.virtualService.port }} 20 | -------------------------------------------------------------------------------- /addons/kube-prometheus-stack/templates/virtual-service.yaml: -------------------------------------------------------------------------------- 1 | {{ if .Values.virtualService.enabled }} 2 | {{- $name := .Values.externalDomain | required ".Values.externalDomain is required." -}} 3 | apiVersion: networking.istio.io/v1alpha3 4 | kind: VirtualService 5 | metadata: 6 | name: {{ .Values.virtualService.name }} 7 | namespace: {{ .Values.virtualService.namespace }} 8 | spec: 9 | gateways: 10 | - {{ .Values.virtualService.gateway }} 11 | hosts: 12 | - "{{ .Values.virtualService.name }}.{{ .Values.externalDomain }}" 13 | http: 14 | - route: 15 | - destination: 16 | host: {{ .Values.virtualService.serviceName }} 17 | port: 18 | number: {{ .Values.virtualService.port }} 19 | {{ end }} 20 | -------------------------------------------------------------------------------- /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | ## What does this PR do? 2 | 3 | Please include a summary of the change, and please also include relevant motivation and context. List any dependencies that are required for this change. 4 | 5 | ## Related issues 6 | 7 | Please include a link to the issues related to this Pull Request. 8 | 9 | ## Checklist before merging 10 | 11 | - [ ] My code follows the style guidelines of this project. 12 | - [ ] I have performed a self-review of my own code. 13 | - [ ] I have checked the [contributing document](../CONTRIBUTING.MD). 14 | - [ ] I have checked the existing [Pull Requests](https://github.com/nearform/initium-platform/pulls) to see whether someone else has raised a similar idea or question. 15 | - [ ] I have added tests that prove my fix is effective or that my feature works. 16 | -------------------------------------------------------------------------------- /tests/opentelemetry_collector_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | 7 | _ "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func TestHelmOpentelemetryCollectorAddon(t *testing.T) { 11 | addonData := HelmAddonData{ 12 | namespaceName: "opentelemetry", 13 | releaseName: "", 14 | dependencyRepo: "", 15 | addonName: "opentelemetry-collector", 16 | addonAlias: "", 17 | chartPath: "", 18 | hasCustomValues: true, 19 | manageNamespace: true, 20 | } 21 | 22 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 23 | 24 | if err != nil { 25 | log.Fatal(err) 26 | } 27 | 28 | waitUntilDaemonSetAvailable(t, *helmOptions.KubectlOptions, "otlp-agent") 29 | 30 | // ---------------------------------- 31 | 32 | destroyHelmEnvironment(t, addonData, helmOptions) 33 | } 34 | -------------------------------------------------------------------------------- /.github/workflows/assets.yaml: -------------------------------------------------------------------------------- 1 | name: Upload assets 2 | 3 | on: 4 | release: 5 | types: 6 | - published 7 | 8 | jobs: 9 | upload-assets: 10 | runs-on: ubuntu-latest 11 | steps: 12 | - name: Install Helm 13 | uses: azure/setup-helm@v3 14 | 15 | - uses: actions/checkout@v4 16 | 17 | - name: Upload files to a GitHub release 18 | run: | 19 | helm template manifests/bootstrap --set revision=${{ github.ref_name }} > app-of-apps.yaml 20 | ./scripts/upload-assets.sh ${{ github.repository }} app-of-apps.yaml ${{ github.ref }} 21 | yq '.argocd-source' ./addons/argocd/values.yaml > argocd-helm-values.yaml 22 | ./scripts/upload-assets.sh ${{ github.repository }} argocd-helm-values.yaml ${{ github.ref }} 23 | env: 24 | GITHUB_TOKEN: ${{ github.token }} 25 | -------------------------------------------------------------------------------- /tests/argocd_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | 7 | _ "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func TestHelmArgoCdServerAddon(t *testing.T) { 11 | addonData := HelmAddonData{ 12 | namespaceName: "", 13 | releaseName: "", 14 | dependencyRepo: "", 15 | addonName: "argocd", 16 | addonAlias: "", 17 | chartPath: "", 18 | hasCustomValues: true, 19 | manageNamespace: true, 20 | } 21 | 22 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 23 | 24 | if err != nil { 25 | log.Fatal(err) 26 | } 27 | 28 | waitUntilHelmFormattedServicesAvailable(t, addonData, helmOptions, []string{ 29 | "redis", 30 | "repo-server", 31 | "server", 32 | }) 33 | 34 | // ---------------------------------- 35 | 36 | destroyHelmEnvironment(t, addonData, helmOptions) 37 | } 38 | -------------------------------------------------------------------------------- /.github/workflows/load-kind-versions.yaml: -------------------------------------------------------------------------------- 1 | name: Load Kind Versions 2 | # You can test this with 3 | # act workflow_call 4 | on: 5 | workflow_call: 6 | # Map the workflow outputs to job outputs 7 | outputs: 8 | matrix: 9 | description: "Matrix of kind versions to use for testing" 10 | value: ${{ jobs.load-kind-versions.outputs.matrix }} 11 | 12 | jobs: 13 | # https://docs.github.com/en/actions/learn-github-actions/expressions#example-returning-a-json-object 14 | load-kind-versions: 15 | runs-on: ubuntu-latest 16 | outputs: 17 | matrix: ${{ steps.set-matrix.outputs.matrix }} 18 | steps: 19 | - name: Checkout 20 | uses: actions/checkout@v4 21 | - id: set-matrix 22 | run: | 23 | JSON=$(cat .github/workflows/kind-versions.json | jq -c) 24 | echo "matrix=${JSON}" >> $GITHUB_OUTPUT 25 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v4.3.0 4 | hooks: 5 | - id: end-of-file-fixer 6 | - id: trailing-whitespace 7 | - id: check-added-large-files 8 | - repo: https://github.com/dnephin/pre-commit-golang 9 | rev: v0.5.0 10 | hooks: 11 | - id: go-fmt 12 | - repo: https://github.com/gruntwork-io/pre-commit 13 | rev: v0.1.17 # Get the latest from: https://github.com/gruntwork-io/pre-commit/releases 14 | hooks: 15 | - id: helmlint 16 | - repo: local 17 | hooks: 18 | - id: tiltfile-validate 19 | name: Validate Tiltfile 20 | entry: ./scripts/tilt-validate.sh 21 | language: script 22 | files: ^Tiltfile$ 23 | - repo: https://github.com/zricethezav/gitleaks 24 | rev: v8.15.2 25 | hooks: 26 | - id: gitleaks 27 | -------------------------------------------------------------------------------- /tests/cert_manager_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | 7 | _ "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func TestHelmCertManagerAddon(t *testing.T) { 11 | addonData := HelmAddonData{ 12 | namespaceName: "", 13 | releaseName: "", 14 | dependencyRepo: "", 15 | addonName: "cert-manager", 16 | addonAlias: "", 17 | chartPath: "", 18 | hasCustomValues: true, 19 | manageNamespace: true, 20 | } 21 | 22 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 23 | 24 | if err != nil { 25 | log.Fatal(err) 26 | } 27 | 28 | waitUntilHelmFormattedServicesAvailable(t, addonData, helmOptions, []string{ 29 | "cert-manager-source", 30 | "cert-manager-source-webhook", 31 | }) 32 | 33 | // ---------------------------------- 34 | 35 | destroyHelmEnvironment(t, addonData, helmOptions) 36 | } 37 | -------------------------------------------------------------------------------- /manifests/kind/templates/cluster.yaml: -------------------------------------------------------------------------------- 1 | kind: Cluster 2 | apiVersion: kind.x-k8s.io/v1alpha4 3 | name: {{ .Values.name }} 4 | kubeadmConfigPatches: 5 | - |- 6 | kind: ClusterConfiguration 7 | apiServer: 8 | {{- if .Values.certSANs }} 9 | certSANs: {{ .Values.certSANs | toYaml | nindent 6 }} 10 | {{- end }} 11 | extraArgs: 12 | oidc-issuer-url: https://dex.kube.local 13 | oidc-client-id: kubelogin 14 | oidc-ca-file: /etc/ca-certificates/dex/ca.pem 15 | oidc-username-claim: email 16 | oidc-groups-claim: groups 17 | nodes: 18 | - role: control-plane 19 | image: {{ .Values.k8s_version }} 20 | extraMounts: 21 | - hostPath: {{ .Values.repo_host_path }} 22 | containerPath: {{ .Values.repo_node_path }} 23 | readOnly: true 24 | - hostPath: {{ .Values.repo_host_path }}/{{ .Values.repo_name }}/.ssl/ca.pem 25 | containerPath: /etc/ca-certificates/dex/ca.pem 26 | readOnly: true 27 | -------------------------------------------------------------------------------- /addons/loki/values.yaml: -------------------------------------------------------------------------------- 1 | loki-source: 2 | fullnameOverride: loki 3 | loki: 4 | auth_enabled: false 5 | storage: 6 | type: filesystem 7 | commonConfig: 8 | replication_factor: 1 9 | query_range: 10 | parallelise_shardable_queries: false 11 | limits_config: 12 | split_queries_by_interval: 0 13 | query_scheduler: 14 | max_outstanding_requests_per_tenant: 10000 15 | monitoring: 16 | dashboards: 17 | enabled: false 18 | serviceMonitor: 19 | enabled: false 20 | selfMonitoring: 21 | enabled: false 22 | grafanaAgent: 23 | installOperator: false 24 | lokiCanary: 25 | enabled: false 26 | test: 27 | enabled: false 28 | singleBinary: 29 | replicas: 1 # https://grafana.com/docs/loki/next/installation/helm/install-monolithic/ 30 | # decrease log level from info to error to reduce log output 31 | extraArgs: [ "-log.level=error" ] 32 | -------------------------------------------------------------------------------- /manifests/bootstrap/templates/app-of-apps.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: Application 3 | metadata: 4 | name: initium-platform 5 | namespace: argocd 6 | finalizers: 7 | - resources-finalizer.argocd.argoproj.io 8 | spec: 9 | destination: 10 | server: https://kubernetes.default.svc 11 | namespace: argocd 12 | project: default 13 | source: 14 | path: app-of-apps 15 | repoURL: {{ .Values.repoURL }} 16 | targetRevision: {{ .Values.revision }} 17 | helm: 18 | parameters: 19 | - name: repoURL 20 | value: {{ .Values.repoURL }} 21 | - name: subChartsRevision 22 | value: {{ .Values.revision }} 23 | - name: externalDomain 24 | value: {{ .Values.externalDomain }} 25 | {{- if .Values.apps }} 26 | values: |- 27 | apps: 28 | {{ .Values.apps | toYaml | indent 10}} 29 | {{- end }} 30 | syncPolicy: 31 | automated: 32 | prune: true 33 | selfHeal: true 34 | -------------------------------------------------------------------------------- /utils/git-http-backend/docker/nginx.conf: -------------------------------------------------------------------------------- 1 | worker_processes 1; 2 | 3 | error_log /var/log/nginx/error.log; 4 | pid /run/nginx.pid; 5 | user root; 6 | 7 | events { 8 | worker_connections 1024; 9 | } 10 | 11 | http { 12 | server { 13 | listen *:80; 14 | 15 | root /www/empty/; 16 | index index.html; 17 | 18 | server_name $hostname; 19 | access_log /var/log/nginx/access.log; 20 | 21 | location ~ /git(/.*) { 22 | # Set chunks to unlimited, as the bodies can be huge 23 | client_max_body_size 0; 24 | 25 | fastcgi_param SCRIPT_FILENAME /usr/libexec/git-core/git-http-backend; 26 | include fastcgi_params; 27 | fastcgi_param GIT_HTTP_EXPORT_ALL ""; 28 | fastcgi_param GIT_PROJECT_ROOT /git; 29 | fastcgi_param PATH_INFO $1; 30 | 31 | # Forward REMOTE_USER as we want to know when we are authenticated 32 | fastcgi_param REMOTE_USER $remote_user; 33 | fastcgi_pass unix:/run/fcgi.sock; 34 | } 35 | } 36 | } 37 | -------------------------------------------------------------------------------- /utils/git-http-backend/docker/Dockerfile: -------------------------------------------------------------------------------- 1 | # small is beautiful 2 | FROM alpine:latest 3 | 4 | # The container listens on port 80, map as needed 5 | EXPOSE 80 6 | 7 | # This is where the repositories will be stored, and 8 | # should be mounted from the host (or a volume container) 9 | VOLUME ["/git"] 10 | 11 | # We need the following: 12 | # - git-daemon, because that gets us the git-http-backend CGI script 13 | # - fcgiwrap, because that is how nginx does CGI 14 | # - spawn-fcgi, to launch fcgiwrap and to create the unix socket 15 | # - nginx, because it is our frontend 16 | RUN apk add --update nginx && \ 17 | apk add --update git-daemon && \ 18 | apk add --update fcgiwrap && \ 19 | apk add --update spawn-fcgi && \ 20 | rm -rf /var/cache/apk/* 21 | 22 | COPY nginx.conf /etc/nginx/nginx.conf 23 | 24 | # launch fcgiwrap via spawn-fcgi; launch nginx in the foreground 25 | # so the container doesn't die on us; supposedly we should be 26 | # using supervisord or something like that instead, but this 27 | # will do 28 | CMD spawn-fcgi -s /run/fcgi.sock /usr/bin/fcgiwrap && \ 29 | nginx -g "daemon off;" 30 | -------------------------------------------------------------------------------- /.github/workflows/release-please.yaml: -------------------------------------------------------------------------------- 1 | name: release-please 2 | 3 | on: 4 | push: 5 | branches: 6 | - main 7 | 8 | jobs: 9 | release-please: 10 | runs-on: ubuntu-latest 11 | steps: 12 | - uses: tibdex/github-app-token@v1.9.0 13 | id: get_installation_token 14 | with: 15 | app_id: ${{ secrets.RELEASE_PLEASE_APP_ID }} 16 | installation_id: ${{ secrets.RELEASE_PLEASE_APP_INSTALLATION_ID }} 17 | private_key: ${{ secrets.RELEASE_PLEASE_APP_PRIVATE_KEY }} 18 | 19 | - uses: google-github-actions/release-please-action@v3 20 | with: 21 | release-type: simple 22 | package-name: initium-platform 23 | bump-minor-pre-major: true 24 | token: ${{ steps.get_installation_token.outputs.token }} 25 | extra-files: | 26 | app-of-apps/Chart.yaml 27 | addons/argocd/Chart.yaml 28 | addons/cert-manager/Chart.yaml 29 | addons/dex/Chart.yaml 30 | addons/istio/base/Chart.yaml 31 | addons/istio/istiod/Chart.yaml 32 | addons/kube-prometheus-stack/Chart.yaml 33 | -------------------------------------------------------------------------------- /addons/cert-manager/values.yaml: -------------------------------------------------------------------------------- 1 | cert-manager-source: 2 | replicaCount: 1 3 | installCRDs: true 4 | namespace: cert-manager 5 | 6 | global: 7 | podSecurityPolicy: 8 | enabled: false 9 | useAppArmor: false 10 | loglevel: 2 11 | 12 | serviceAccount: 13 | create: true 14 | name: "cert-manager" 15 | annotations: {} 16 | resources: 17 | requests: 18 | cpu: 10m 19 | memory: 32Mi 20 | limits: 21 | cpu: 50m 22 | memory: 160Mi 23 | 24 | prometheus: 25 | enabled: true 26 | # servicemonitor: 27 | # enabled: true 28 | 29 | webhook: 30 | replicaCount: 1 31 | resources: 32 | requests: 33 | cpu: 10m 34 | memory: 32Mi 35 | limits: 36 | cpu: 50m 37 | memory: 200Mi 38 | 39 | caininjector: 40 | replicaCount: 1 41 | resources: 42 | requests: 43 | cpu: 10m 44 | memory: 32Mi 45 | limits: 46 | cpu: 50m 47 | memory: 200Mi 48 | 49 | startupapicheck: 50 | resources: 51 | requests: 52 | cpu: 10m 53 | memory: 32Mi 54 | limits: 55 | cpu: 50m 56 | memory: 200Mi 57 | -------------------------------------------------------------------------------- /manifests/kind/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: kind 3 | description: A Helm chart for Kind 4 | 5 | # A chart can be either an 'application' or a 'library' chart. 6 | # 7 | # Application charts are a collection of templates that can be packaged into versioned archives 8 | # to be deployed. 9 | # 10 | # Library charts provide useful utilities or functions for the chart developer. They're included as 11 | # a dependency of application charts to inject those utilities and functions into the rendering 12 | # pipeline. Library charts do not define any templates and therefore cannot be deployed. 13 | type: application 14 | 15 | # This is the chart version. This version number should be incremented each time you make changes 16 | # to the chart and its templates, including the app version. 17 | # Versions are expected to follow Semantic Versioning (https://semver.org/) 18 | version: 0.1.0 19 | 20 | # This is the version number of the application being deployed. This version number should be 21 | # incremented each time you make changes to the application. Versions are not expected to 22 | # follow Semantic Versioning. They should reflect the version the application is using. 23 | # It is recommended to use it with quotes. 24 | appVersion: "1.16.0" 25 | -------------------------------------------------------------------------------- /manifests/bootstrap/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: bootstrap 3 | description: Bootstrap the app of apps 4 | 5 | # A chart can be either an 'application' or a 'library' chart. 6 | # 7 | # Application charts are a collection of templates that can be packaged into versioned archives 8 | # to be deployed. 9 | # 10 | # Library charts provide useful utilities or functions for the chart developer. They're included as 11 | # a dependency of application charts to inject those utilities and functions into the rendering 12 | # pipeline. Library charts do not define any templates and therefore cannot be deployed. 13 | type: application 14 | 15 | # This is the chart version. This version number should be incremented each time you make changes 16 | # to the chart and its templates, including the app version. 17 | # Versions are expected to follow Semantic Versioning (https://semver.org/) 18 | version: 0.1.0 19 | 20 | # This is the version number of the application being deployed. This version number should be 21 | # incremented each time you make changes to the application. Versions are not expected to 22 | # follow Semantic Versioning. They should reflect the version the application is using. 23 | # It is recommended to use it with quotes. 24 | appVersion: "1.16.0" 25 | -------------------------------------------------------------------------------- /examples/sample-app/sample-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: serving.knative.dev/v1 2 | kind: Service 3 | metadata: 4 | name: sample-app 5 | spec: 6 | template: 7 | metadata: 8 | annotations: 9 | # For testing purposes set request-per-second to 1 to force scaling 10 | autoscaling.knative.dev/target: "1" 11 | autoscaling.knative.dev/metric: "rps" 12 | labels: 13 | service: "ksample" 14 | spec: 15 | containers: 16 | - image: nginx:stable 17 | ports: 18 | - containerPort: 80 19 | volumeMounts: 20 | - name: config 21 | mountPath: "/etc/nginx" 22 | readOnly: true 23 | volumes: 24 | - name: config 25 | configMap: 26 | name: nginx-config 27 | items: 28 | - key: "nginx.conf" 29 | path: "nginx.conf" 30 | --- 31 | apiVersion: v1 32 | kind: ConfigMap 33 | metadata: 34 | name: nginx-config 35 | data: 36 | nginx.conf: | 37 | worker_processes 1; 38 | 39 | events { 40 | worker_connections 1024; 41 | } 42 | 43 | http { 44 | server { 45 | listen 80; 46 | 47 | location / { 48 | return 200 'Hello world from initium-platform!'; 49 | } 50 | } 51 | } 52 | -------------------------------------------------------------------------------- /.github/renovate.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://docs.renovatebot.com/renovate-schema.json", 3 | "automergeType": "branch", 4 | "baseBranches": ["main"], 5 | "branchConcurrentLimit": 0, 6 | "branchNameStrict": true, 7 | "dependencyDashboard": false, 8 | "enabledManagers": ["asdf", "helmv3", "github-actions"], 9 | "extends": ["config:base", ":disableDependencyDashboard", ":semanticCommitTypeAll(ci)"], 10 | "gitAuthor": "NearForm Renovate App Bot <115552475+nearform-renovate-app[bot]@users.noreply.github.com>", 11 | "packageRules": 12 | [ 13 | { 14 | "matchUpdateTypes": ["minor", "patch"], 15 | "automerge": true 16 | }, 17 | { 18 | "matchPaths": ["addons/istio/**"], 19 | "groupName": "Istio Helm Chart" 20 | } 21 | ], 22 | "customManagers": [ 23 | { 24 | "customType": "regex", 25 | "fileMatch": ["^.tool-versions$"], 26 | "matchStrings": ["initium (?.*?)\\n"], 27 | "depNameTemplate": "nearform/initium-cli", 28 | "datasourceTemplate": "github-releases" 29 | } 30 | ], 31 | "platform": "github", 32 | "platformAutomerge": true, 33 | "prConcurrentLimit": 0, 34 | "rebaseWhen": "auto", 35 | "repositories": ["nearform/initium-platform"], 36 | "username": "nearform-renovate-app[bot]" 37 | } 38 | -------------------------------------------------------------------------------- /addons/grafana/values.yaml: -------------------------------------------------------------------------------- 1 | grafana-source: 2 | fullnameOverride: grafana 3 | rbac: 4 | pspEnabled: false 5 | # adminUser: admin 6 | # adminPassword: strongpassword 7 | # Use an existing secret for the admin user. 8 | admin: 9 | ## Name of the secret. Can be templated. 10 | existingSecret: grafana-initial-admin-secret 11 | userKey: admin-user 12 | passwordKey: admin-password 13 | 14 | plugins: 15 | - grafana-piechart-panel 16 | 17 | # loki config 18 | datasources: 19 | datasources.yaml: 20 | apiVersion: 1 21 | datasources: 22 | - name: Loki 23 | type: loki 24 | url: http://loki.loki:3100 25 | access: proxy 26 | isDefault: false 27 | jsonData: 28 | maxLines: 1000 29 | 30 | # prometheus config 31 | sidecar: 32 | dashboards: 33 | enabled: true 34 | label: grafana_dashboard 35 | labelValue: "1" 36 | datasources: 37 | enabled: true 38 | defaultDatasourceEnabled: true 39 | isDefaultDatasource: true 40 | uid: prometheus 41 | label: grafana_datasource 42 | labelValue: "1" 43 | 44 | virtualService: 45 | name: grafana 46 | namespace: grafana 47 | gateway: istio-ingress/kube-gateway 48 | internalHost: grafana.kube.local 49 | port: 80 50 | serviceName: grafana 51 | -------------------------------------------------------------------------------- /scripts/generate-certs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | mkdir -p .ssl 6 | 7 | # check if certificate expired or is going to expiry in less than 2 days 8 | openssl x509 -in .ssl/ca.pem -noout -checkend 172800 > /dev/null 2>&1 9 | if [ $? -eq 0 ]; then 10 | echo "Existing certificates are valid" 11 | else 12 | echo "Creating new certificates" 13 | 14 | cat << EOF > .ssl/req.cnf 15 | [req] 16 | req_extensions = v3_req 17 | distinguished_name = req_distinguished_name 18 | 19 | [req_distinguished_name] 20 | 21 | [ v3_req ] 22 | basicConstraints = CA:FALSE 23 | keyUsage = nonRepudiation, digitalSignature, keyEncipherment 24 | subjectAltName = @alt_names 25 | 26 | [alt_names] 27 | DNS.1 = *.kube.local 28 | EOF 29 | 30 | openssl genrsa -out .ssl/ca-key.pem 2048 31 | openssl req -x509 -new -nodes -key .ssl/ca-key.pem -days "${INITIUM_CERT_VALIDITY_DAYS}" -out .ssl/ca.pem -subj "/CN=root-ca" 32 | 33 | openssl genrsa -out .ssl/key-ingress-gateway.pem 2048 34 | openssl req -new -key .ssl/key-ingress-gateway.pem -out .ssl/csr.pem -subj "/CN=kube-local" -config .ssl/req.cnf 35 | openssl x509 -req -in .ssl/csr.pem -CA .ssl/ca.pem -CAkey .ssl/ca-key.pem -CAcreateserial -out .ssl/cert-ingress-gateway.pem -days "${INITIUM_CERT_VALIDITY_DAYS}" -extensions v3_req -extfile .ssl/req.cnf 36 | fi 37 | -------------------------------------------------------------------------------- /scripts/cloud.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | # Generate report of current env vars 6 | echo "======================================================" 7 | printenv | grep "INITIUM_.*" 8 | echo "======================================================" 9 | 10 | # Install ArgoCD 11 | kubectl create namespace argocd 12 | helm dependency update ./addons/argocd 13 | helm install argocd ./addons/argocd/ -f ./addons/argocd/values.yaml --namespace=argocd 14 | # Ensure ArgoCD apps are all healthy and in sync 15 | echo ">> Waiting for argocd to be healty and in sync..." 16 | 17 | while true; do 18 | # Check the status of the ArgoCD deployment 19 | deployment_status=$(kubectl get deployment -n argocd argocd-server -o jsonpath='{.status.readyReplicas}') 20 | 21 | if [ "$deployment_status" != "1" ]; then 22 | echo "ArgoCD deployment is not ready." 23 | else 24 | # Check the status of the ArgoCD pod 25 | pod_status=$(kubectl get pod -n argocd -l app.kubernetes.io/name=argocd-server -o jsonpath='{.items[].status.phase}') 26 | 27 | if [ "$pod_status" != "Running" ]; then 28 | echo "ArgoCD pod is not running. Current pod status: $pod_status" 29 | else 30 | echo "ArgoCD is installed and ready to use." 31 | break; 32 | fi 33 | fi 34 | sleep 30 35 | done 36 | -------------------------------------------------------------------------------- /tests/dex_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | 7 | _ "github.com/stretchr/testify/assert" 8 | ) 9 | 10 | func TestHelmDexAddon(t *testing.T) { 11 | // add Istio CRDs to test Istio virtual service 12 | istioBaseAddonData := HelmAddonData{ 13 | namespaceName: "dex-test", 14 | releaseName: "", 15 | dependencyRepo: "", 16 | addonName: "istio-base", 17 | addonAlias: "", 18 | chartPath: "../addons/istio/base", 19 | manageNamespace: true, 20 | overrideValues: map[string]string{ 21 | "global.istioNamespace": "dex-test", 22 | }, 23 | } 24 | 25 | istioBaseHelmOptions, err := prepareHelmEnvironment(t, &istioBaseAddonData) 26 | 27 | if err != nil { 28 | log.Fatal(err) 29 | } 30 | 31 | addonData := HelmAddonData{ 32 | namespaceName: "", 33 | releaseName: "", 34 | dependencyRepo: "", 35 | addonName: "dex", 36 | addonAlias: "", 37 | chartPath: "", 38 | hasCustomValues: true, 39 | manageNamespace: true, 40 | } 41 | 42 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 43 | 44 | if err != nil { 45 | log.Fatal(err) 46 | } 47 | 48 | waitUntilServicesAvailable(t, *helmOptions.KubectlOptions, []string{"dex"}) 49 | 50 | // ---------------------------------- 51 | 52 | destroyHelmEnvironment(t, addonData, helmOptions) 53 | destroyHelmEnvironment(t, istioBaseAddonData, istioBaseHelmOptions) 54 | } 55 | -------------------------------------------------------------------------------- /docs/KNOWN_ISSUES.md: -------------------------------------------------------------------------------- 1 | # Known Issues 2 | 3 | ### Knative Deletion Race Condition 4 | 5 | Knative Operator inserts finalizers in `knativeserving.operator.knative.dev` and `knativeeventing.operator.knative.dev` 6 | resources. As all three ArgoCD applications (knative-serving, knative-eventing, knative-operator) are deleted at the 7 | same time, Knative Operator is removed before it can fulfill finalizer condition when `knativeserving.operator.knative.dev` 8 | and `knativeeventing.operator.knative.dev` resources are to be deleted. This results in knative-serving and knative-eventing 9 | apps stuck in deletion. 10 | 11 | Similar issue is present in the ArgoCD project: [Keycloak deletion deadlock](https://github.com/argoproj/argo-cd/issues/9296). 12 | 13 | **Resolution** 14 | 15 | Proposed solution is to add [post-delete hook](https://github.com/argoproj/argo-cd/issues/7575) to ArgoCD. Until this 16 | feature is available, manual solution is offered. 17 | 18 | After deletion action is executed, it is necessary to remove finalizers from `knativeserving.operator.knative.dev` and 19 | `knativeeventing.operator.knative.dev` resources. This will allow application removal process to finish. 20 | ```bash 21 | kubectl patch knativeeventing knative-eventing -n knative-eventing -p '{"metadata":{"finalizers":null}}' --type=merge 22 | kubectl patch knativeserving knative-serving -n knative-serving -p '{"metadata":{"finalizers":null}}' --type=merge 23 | ``` 24 | -------------------------------------------------------------------------------- /app-of-apps/templates/apps.yaml: -------------------------------------------------------------------------------- 1 | {{- range $name, $props := .Values.apps }} 2 | {{- with $props }} 3 | {{- $defaultPath := print "addons/" .name "/" -}} 4 | {{- if not $props.excluded }} 5 | apiVersion: argoproj.io/v1alpha1 6 | kind: Application 7 | metadata: 8 | name: {{ .name }} 9 | namespace: {{ $.Values.namespace }} 10 | finalizers: 11 | - resources-finalizer.argocd.argoproj.io 12 | annotations: 13 | argocd.argoproj.io/sync-wave: "{{ .syncWave | default 0 }}" 14 | spec: 15 | destination: 16 | server: {{ $.Values.spec.destination.server }} 17 | namespace: {{ .targetNamespace | default "default" | quote }} 18 | project: default 19 | source: 20 | path: {{ or .path $defaultPath }} 21 | repoURL: {{ $.Values.repoURL }} 22 | targetRevision: {{ coalesce .targetRevision $.Values.subChartsRevision "HEAD" | quote }} 23 | helm: 24 | parameters: 25 | - name: externalDomain 26 | value: {{ $.Values.externalDomain }} 27 | {{- if .helmValues }} 28 | values: |- 29 | {{- .helmValues | toYaml | nindent 8 }} 30 | {{- end}} 31 | syncPolicy: 32 | automated: 33 | prune: true 34 | selfHeal: true 35 | syncOptions: 36 | - CreateNamespace=true 37 | {{- if .serverSideApply }} 38 | - ServerSideApply=true 39 | {{- end }} 40 | {{- if .replace }} 41 | - Replace=true 42 | {{- end }} 43 | {{- if .ignoreDifferences }} 44 | ignoreDifferences: 45 | {{- toYaml .ignoreDifferences | nindent 4 }} 46 | {{- end }} 47 | --- 48 | {{- end }} 49 | {{- end }} 50 | {{- end }} 51 | -------------------------------------------------------------------------------- /addons/kube-prometheus-stack/values.yaml: -------------------------------------------------------------------------------- 1 | virtualService: 2 | enabled: false 3 | name: prometheus 4 | namespace: prometheus 5 | gateway: istio-ingress/kube-gateway 6 | port: 9090 7 | serviceName: kube-prometheus-stack-kube-prometheus 8 | 9 | kube-prometheus-stack-source: 10 | prometheus-node-exporter: 11 | hostRootFsMount: 12 | enabled: false 13 | # rbac: 14 | # pspEnabled: false 15 | grafana: 16 | # external grafana app is used by default 17 | enabled: false 18 | 19 | namespaceOverride: "grafana" 20 | # ForceDeployDatasources Create datasource configmap even if grafana deployment has been disabled 21 | forceDeployDatasources: true 22 | # ForceDeployDashboard Create dashboard configmap even if grafana deployment has been disabled 23 | forceDeployDashboards: true 24 | 25 | prometheus: 26 | prometheusSpec: 27 | enableRemoteWriteReceiver: true 28 | # Required to fix the out-of-sync issues on the this apps suite 29 | kubelet: 30 | serviceMonitor: 31 | cAdvisorRelabelings: 32 | - action: replace 33 | sourceLabels: [__metrics_path__] 34 | targetLabel: metrics_path 35 | probesRelabelings: 36 | - action: replace 37 | sourceLabels: [__metrics_path__] 38 | targetLabel: metrics_path 39 | resourceRelabelings: 40 | - action: replace 41 | sourceLabels: [__metrics_path__] 42 | targetLabel: metrics_path 43 | relabelings: 44 | - action: replace 45 | sourceLabels: [__metrics_path__] 46 | targetLabel: metrics_path 47 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/values.yaml: -------------------------------------------------------------------------------- 1 | image: 2 | repository: initium-platform/git-http-backend 3 | pullPolicy: IfNotPresent 4 | # Overrides the image tag whose default is the chart appVersion. 5 | tag: "" 6 | 7 | volumes: 8 | git_volume: 9 | # Current directory by default 10 | path: . 11 | 12 | imagePullSecrets: [] 13 | nameOverride: "" 14 | fullnameOverride: "" 15 | 16 | serviceAccount: 17 | # Specifies whether a service account should be created 18 | create: true 19 | # Annotations to add to the service account 20 | annotations: {} 21 | # The name of the service account to use. 22 | # If not set and create is true, a name is generated using the fullname template 23 | name: "" 24 | 25 | podAnnotations: {} 26 | 27 | podSecurityContext: {} 28 | # fsGroup: 2000 29 | 30 | securityContext: {} 31 | # capabilities: 32 | # drop: 33 | # - ALL 34 | # readOnlyRootFilesystem: true 35 | # runAsNonRoot: true 36 | # runAsUser: 1000 37 | 38 | service: 39 | type: ClusterIP 40 | port: 80 41 | 42 | resources: {} 43 | # We usually recommend not to specify default resources and to leave this as a conscious 44 | # choice for the user. This also increases chances charts run on environments with little 45 | # resources, such as Minikube. If you do want to specify resources, uncomment the following 46 | # lines, adjust them as necessary, and remove the curly braces after 'resources:'. 47 | # limits: 48 | # cpu: 100m 49 | # memory: 128Mi 50 | # requests: 51 | # cpu: 100m 52 | # memory: 128Mi 53 | 54 | nodeSelector: {} 55 | 56 | tolerations: [] 57 | 58 | affinity: {} 59 | -------------------------------------------------------------------------------- /manifests/ci-service-account/README.md: -------------------------------------------------------------------------------- 1 | # Service account for CI Bot 2 | 3 | There are various ways to integrate a CI system with kubernetes clusters. OpenID Connect is one option, cloud providers offer 4 | integration points with their identity and access management systems, etc. For this project however, we aim to be agnostic 5 | to these services and selected a solution that would be applicable to all by creating a k8s `ServiceAccount` and `ServiceAccountToken`. 6 | This is an optional configuration for a cluster and not the only valid solution to solve authentication and 7 | authorization patterns for a CI system. 8 | 9 | This cluster can deploy a `ServiceAccount` with the sole purpose of being utilized by the 10 | CI System to interact with the cluster. 11 | 12 | The `ServiceAccount` is associated with a `kubernetes.io/service-account-token`. 13 | Once this token is generated, the cluster administrator must export this token to the 14 | CI System as a secret. Once done, the pipeline for the CI system can use this token to authenticate 15 | to the cluster. 16 | 17 | In the case of GitHub, follow the [official guide](https://docs.github.com/en/actions/security-guides/encrypted-secrets#about-encrypted-secrets) 18 | for creating a secret. Depending on your specific needs the secret could be applicable to a single repo, 19 | an environment, or the entire GitHub organization. 20 | 21 | ### Deploy the Service Account 22 | 23 | There is a Makefile Goal that can be run against the cluster to create the aforementioned `ServiceAccount`, token, 24 | and permission sets. To run this, simply execute the following command: 25 | ```bash 26 | $ make create-ci-service-account 27 | ``` 28 | 29 | The permission sets should be modified for your use case. 30 | -------------------------------------------------------------------------------- /scripts/run-test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -e 4 | 5 | source .envrc 6 | 7 | PROJECT_TYPE="${1:-unit}" 8 | PROJECT_PATH="${2:-tests}" 9 | PROXY_HTTP_CONTAINER_NAME="istio-lb-proxy-80" 10 | 11 | if [ "$PROJECT_TYPE" == "integration" ]; then 12 | export INITIUM_LB_ENDPOINT="$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{(index .status.loadBalancer.ingress 0).ip}}'):80" 13 | 14 | case "$OSTYPE" in 15 | darwin*) 16 | INITIUM_LB_INT_HTTP_PORT=$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{range .spec.ports}}{{if (eq .port 80)}}{{.nodePort}}{{end}}{{end}}') 17 | 18 | # Run a reverse proxy that will forward traffic between host and the LB, you can reach the LB HTTP port via 127.0.0.1:80 19 | docker run \ 20 | -d \ 21 | --restart always \ 22 | --name $PROXY_HTTP_CONTAINER_NAME \ 23 | --publish 127.0.0.1:80:$INITIUM_LB_INT_HTTP_PORT \ 24 | --link ${INITIUM_REPO_NAME}-control-plane:target \ 25 | --network kind \ 26 | alpine/socat -dd tcp-listen:$INITIUM_LB_INT_HTTP_PORT,fork,reuseaddr tcp-connect:target:$INITIUM_LB_INT_HTTP_PORT || true 27 | 28 | export INITIUM_LB_ENDPOINT="127.0.0.1:80" 29 | ;; 30 | msys*) 31 | ;; 32 | linux*) 33 | ;; 34 | esac 35 | fi 36 | 37 | echo "Running test on ${PROJECT_PATH}" 38 | 39 | cd $PROJECT_PATH 40 | go mod init test || true 41 | go install . || true 42 | go test . -v -timeout=30m 43 | 44 | if [ "$PROJECT_TYPE" == "integration" ]; then 45 | case "$OSTYPE" in 46 | darwin*) 47 | docker stop $PROXY_HTTP_CONTAINER_NAME 48 | docker rm $PROXY_HTTP_CONTAINER_NAME 49 | ;; 50 | msys*) 51 | ;; 52 | linux*) 53 | ;; 54 | esac 55 | fi 56 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/issue_template.yml: -------------------------------------------------------------------------------- 1 | name: "Issue" 2 | description: Create a new issue for an enhancement, a bug or a chore. 3 | title: "[ISSUE] - " 4 | labels: ["issue"] 5 | body: 6 | - type: dropdown 7 | id: issue-type 8 | attributes: 9 | label: "Issue Type" 10 | description: What kind of an issue are you reporting? 11 | multiple: true 12 | options: 13 | - Enhancement 14 | - Bug 15 | - Chore 16 | validations: 17 | required: true 18 | - type: textarea 19 | id: description 20 | attributes: 21 | label: "Description" 22 | description: Please enter an short description of your issue 23 | placeholder: Short description of the bug/enhancement incident... 24 | validations: 25 | required: true 26 | - type: textarea 27 | id: reprod 28 | attributes: 29 | label: "Detailed steps" 30 | description: Please enter an explicit description of your issue 31 | value: | 32 | 1. Go to '...' 33 | 2. Run this command '....' 34 | 3. See error 35 | render: bash 36 | validations: 37 | required: true 38 | - type: textarea 39 | id: screenshot 40 | attributes: 41 | label: "Screenshots" 42 | description: If applicable, add screenshots to help explain your problem. 43 | value: | 44 | Add any relevant screenshots here 45 | render: bash 46 | validations: 47 | required: false 48 | - type: textarea 49 | id: logs 50 | attributes: 51 | label: "Logs" 52 | description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. 53 | render: bash 54 | validations: 55 | required: false 56 | -------------------------------------------------------------------------------- /addons/kubernetes-replicator/values.yaml: -------------------------------------------------------------------------------- 1 | kubernetes-replicator-source: 2 | image: 3 | repository: quay.io/mittwald/kubernetes-replicator 4 | #tag: stable # if no tag is given, the chart's appVersion is used 5 | pullPolicy: Always 6 | imagePullSecrets: [] 7 | nameOverride: "" 8 | fullnameOverride: "" 9 | grantClusterAdmin: false 10 | automountServiceAccountToken: true 11 | args: [] 12 | # - -resync-period=30m 13 | # - -allow-all=false 14 | 15 | ## Deployment strategy / DaemonSet updateStrategy 16 | ## 17 | updateStrategy: {} 18 | # type: RollingUpdate 19 | # rollingUpdate: 20 | # maxUnavailable: 1 21 | 22 | serviceAccount: 23 | create: true 24 | annotations: {} 25 | name: 26 | privileges: [] 27 | automountServiceAccountToken: true 28 | # - apiGroups: [""] 29 | # resources: ["configmaps"] 30 | podSecurityContext: {} 31 | # fsGroup: 2000 32 | 33 | securityContext: {} 34 | # capabilities: 35 | # drop: 36 | # - ALL 37 | # readOnlyRootFilesystem: true 38 | # runAsNonRoot: true 39 | # runAsUser: 1000 40 | 41 | priorityClassName: "" 42 | 43 | resources: {} 44 | # limits: 45 | # cpu: 100m 46 | # memory: 128Mi 47 | # requests: 48 | # cpu: 100m 49 | # memory: 128Mi 50 | 51 | nodeSelector: {} 52 | 53 | tolerations: [] 54 | 55 | affinity: {} 56 | 57 | podLabels: {} 58 | 59 | podAnnotations: {} 60 | 61 | livenessProbe: 62 | initialDelaySeconds: 60 63 | periodSeconds: 10 64 | timeoutSeconds: 1 65 | failureThreshold: 3 66 | successThreshold: 1 67 | 68 | readinessProbe: 69 | initialDelaySeconds: 60 70 | periodSeconds: 10 71 | timeoutSeconds: 1 72 | failureThreshold: 3 73 | successThreshold: 1 74 | -------------------------------------------------------------------------------- /scripts/upload-assets.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [ $# -ne 3 ] 4 | then 5 | echo "Usage: ./upload-assets.sh <GITHUB_REPOSITORY> <FILE_NAME> <GITHUB_REF>" 6 | exit 1 7 | fi 8 | 9 | ACTION_REPOSITORY=$1 10 | ACTION_FILE=$2 11 | ACTION_TAG=$(echo "$3" | sed -e s/refs\\/tags\\///g | sed -e s/refs\\/heads\\///g) 12 | RELEASE_RESPONSE=$(curl \ 13 | -H "Accept: application/vnd.github+json" \ 14 | -H "Authorization: Bearer $GITHUB_TOKEN" \ 15 | https://api.github.com/repos/"$ACTION_REPOSITORY"/releases/tags/"$ACTION_TAG") 16 | 17 | ASSET_LIST=$(curl \ 18 | -H "Accept: application/vnd.github+json" \ 19 | -H "Authorization: Bearer $GITHUB_TOKEN" \ 20 | https://api.github.com/repos/"$ACTION_REPOSITORY"/releases/$(echo "$RELEASE_RESPONSE" | jq .id)/assets) 21 | 22 | echo "Pre-existing asset list: $(echo "$ASSET_LIST" | jq)" 23 | i=0 24 | for f in $(echo "$ASSET_LIST" | jq -r '.[]? | .name'); do 25 | if [ "$f" = "$ACTION_FILE" ] ; then 26 | echo "This file already exists, we'll overwrite it" 27 | curl \ 28 | -X DELETE \ 29 | -H "Accept: application/vnd.github+json" \ 30 | -H "Authorization: Bearer $GITHUB_TOKEN" \ 31 | https://api.github.com/repos/"$ACTION_REPOSITORY"/releases/assets/$(echo "$ASSET_LIST" | jq -r --argjson i "$i" '.[$i].id') 32 | fi 33 | (( i+= 1 )) 34 | done 35 | 36 | curl \ 37 | -X POST \ 38 | -H "Accept: application/vnd.github+json" \ 39 | -H "Authorization: Bearer $GITHUB_TOKEN" \ 40 | -H "Content-Type: application/yaml" \ 41 | -H "Content-Length: $(wc -c "$ACTION_FILE" | awk '{print $1}')" \ 42 | --data-binary "@$ACTION_FILE" \ 43 | "$(echo "$RELEASE_RESPONSE" | jq .assets_url | sed -e s/api.github.com/uploads.github.com/g | sed -e s/\"//g)?name=$ACTION_FILE&label=$ACTION_FILE"; 44 | 45 | echo "File $ACTION_FILE uploaded to release $ACTION_TAG successfully" 46 | exit 0 47 | -------------------------------------------------------------------------------- /addons/opentelemetry-collector/values.yaml: -------------------------------------------------------------------------------- 1 | otlp-collector-source: 2 | fullnameOverride: otlp 3 | mode: daemonset 4 | presets: 5 | kubernetesAttributes: 6 | enabled: true 7 | logsCollection: 8 | enabled: true 9 | config: 10 | receivers: 11 | otlp: 12 | protocols: 13 | http: 14 | 15 | processors: 16 | batch: {} 17 | 18 | attributes: 19 | actions: 20 | - action: insert 21 | key: log_iostream 22 | from_attribute: log.iostream 23 | - action: insert 24 | key: loki.attribute.labels 25 | value: log_iostream 26 | 27 | resource: 28 | attributes: 29 | - action: insert 30 | key: namespace_name 31 | from_attribute: k8s.namespace.name 32 | - action: insert 33 | key: pod_name 34 | from_attribute: k8s.pod.name 35 | - action: insert 36 | key: container_name 37 | from_attribute: k8s.container.name 38 | - action: insert 39 | key: container_restart_count 40 | from_attribute: k8s.container.restart_count 41 | - action: insert 42 | key: loki.resource.labels 43 | value: namespace_name, pod_name, container_name, container_restart_count 44 | 45 | exporters: 46 | loki: 47 | endpoint: http://loki.loki:3100/loki/api/v1/push 48 | tls: 49 | insecure: true 50 | 51 | service: 52 | pipelines: 53 | logs: 54 | receivers: [otlp] 55 | processors: [attributes, resource, batch] 56 | exporters: [loki] 57 | 58 | # enable opentelemetry collector debug logging 59 | # telemetry: 60 | # logs: 61 | # level: "DEBUG" 62 | # development: true 63 | # encoding: "json" 64 | -------------------------------------------------------------------------------- /addons/argocd/values.yaml: -------------------------------------------------------------------------------- 1 | # https://github.com/argoproj/argo-helm/blob/main/charts/argo-cd/values.yaml 2 | argocd-source: 3 | applicationSet: 4 | enabled: false 5 | 6 | configs: 7 | params: 8 | server.insecure: false 9 | 10 | dex: 11 | enabled: false 12 | 13 | notifications: 14 | enabled: false 15 | 16 | server: 17 | ingress: 18 | enabled: true 19 | https: true 20 | hosts: 21 | - argocd.local 22 | tls: 23 | - hosts: 24 | - argocd.local 25 | secretName: argocd.dex-tls 26 | config: 27 | repositories: | 28 | - type: helm 29 | name: argo-cd 30 | url: https://argoproj.github.io/argo-helm 31 | # Required to support sycn-wave on ArgoCD Application kind 32 | resource.customizations.health.argoproj.io_Application: | 33 | hs = {} 34 | hs.status = "Progressing" 35 | hs.message = "" 36 | if obj.status ~= nil then 37 | if obj.status.health ~= nil then 38 | hs.status = obj.status.health.status 39 | if obj.status.health.message ~= nil then 40 | hs.message = obj.status.health.message 41 | end 42 | end 43 | end 44 | return hs 45 | # Required to fix the out-of-sync issues on the Istio apps suite 46 | resource.customizations.ignoreDifferences.admissionregistration.k8s.io_MutatingWebhookConfiguration: | 47 | jqPathExpressions: 48 | - '.webhooks[]?.clientConfig.caBundle' 49 | resource.customizations.ignoreDifferences.admissionregistration.k8s.io_ValidatingWebhookConfiguration: | 50 | jqPathExpressions: 51 | - '.webhooks[]?.clientConfig.caBundle' 52 | - '.webhooks[]?.failurePolicy' 53 | # Required to fix the out-of-sync issues on the Knative apps suite 54 | resource.compareoptions: | 55 | # disables status field diffing in specified resource types 56 | ignoreAggregatedRoles: true 57 | -------------------------------------------------------------------------------- /.github/workflows/unit-tests.yaml: -------------------------------------------------------------------------------- 1 | name: Run Unit Tests 2 | 3 | on: 4 | pull_request_review: 5 | types: 6 | - submitted 7 | 8 | jobs: 9 | load-matrix: 10 | if: github.event.review.state == 'approved' && !startsWith(github.event.pull_request.title, 'docs:') 11 | uses: ./.github/workflows/load-kind-versions.yaml 12 | 13 | unit-tests: 14 | if: github.event.review.state == 'approved' && !startsWith(github.event.pull_request.title, 'docs:') 15 | needs: load-matrix 16 | strategy: 17 | matrix: 18 | INITIUM_K8S_VERSION: ${{ fromJSON(needs.load-matrix.outputs.matrix) }} 19 | runs-on: ubuntu-latest 20 | timeout-minutes: 30 21 | steps: 22 | - name: Checkout 23 | uses: actions/checkout@v4 24 | 25 | - name: Unit test 26 | run: | 27 | # Install asdf and expose to PATH 28 | git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.2 29 | . $HOME/.asdf/asdf.sh 30 | # Add asdf plugins and install tools in .tool-versions 31 | make asdf_install 32 | # Create a local branch to allow the next step to work 33 | git checkout -b ci 34 | # Deploy cluster 35 | make ci 36 | # Run tests 37 | make unit-test 38 | env: 39 | GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }} 40 | INITIUM_DEPLOY_MINIMAL: true 41 | INITIUM_K8S_VERSION: ${{ matrix.INITIUM_K8S_VERSION }} 42 | 43 | # https://github.com/orgs/community/discussions/26822#discussioncomment-5122101 44 | unit-tests-result: 45 | if: github.event.review.state == 'approved' 46 | runs-on: ubuntu-latest 47 | name: Unit Tests 48 | needs: [unit-tests] 49 | steps: 50 | - run: exit 1 51 | # see https://stackoverflow.com/a/67532120/4907315 52 | if: >- 53 | ${{ 54 | contains(needs.*.result, 'failure') 55 | || contains(needs.*.result, 'cancelled') 56 | }} 57 | -------------------------------------------------------------------------------- /addons/dex/values.yaml: -------------------------------------------------------------------------------- 1 | dex-source: 2 | fullnameOverride: dex 3 | config: 4 | # The base path of dex and the external name of the OpenID Connect service. 5 | # This is the canonical URL that all clients MUST use to refer to dex. If a 6 | # path is provided, dex's HTTP service will listen at a non-root URL. 7 | issuer: http://127.0.0.1:5556 8 | 9 | # See https://dexidp.io/docs/storage/ for more options 10 | storage: 11 | type: memory 12 | 13 | # Enable at least one connector 14 | # See https://dexidp.io/docs/connectors/ for more options 15 | enablePasswordDB: true 16 | 17 | # Example config for Kubernetes Authentication Through Dex and GitHub 18 | # 19 | # # URL and port as seen from outside the cluster, through MetalLB and Istio ingress gateway 20 | # issuer: https://dex.kube.local 21 | # storage: 22 | # type: kubernetes 23 | # config: 24 | # inCluster: true 25 | # connectors: 26 | # - type: github 27 | # id: github 28 | # name: GitHub 29 | # config: 30 | # clientID: $GITHUB_CLIENT_ID 31 | # clientSecret: $GITHUB_CLIENT_SECRET 32 | # redirectURI: https://dex.kube.local/callback 33 | # staticClients: 34 | # - id: kubelogin 35 | # redirectURIs: 36 | # - 'http://localhost:8000' 37 | # name: 'Kubelogin' 38 | # secretEnv: KUBELOGIN_CLIENT_SECRET 39 | # envVars: 40 | # - name: GITHUB_CLIENT_ID 41 | # valueFrom: 42 | # secretKeyRef: 43 | # name: github-client 44 | # key: client-id 45 | # - name: GITHUB_CLIENT_SECRET 46 | # valueFrom: 47 | # secretKeyRef: 48 | # name: github-client 49 | # key: client-secret 50 | # - name: KUBELOGIN_CLIENT_SECRET 51 | # valueFrom: 52 | # secretKeyRef: 53 | # name: kubelogin-client 54 | # key: client-secret 55 | 56 | virtualService: 57 | enabled: false 58 | name: dex 59 | namespace: dex 60 | gateway: istio-ingress/kube-gateway 61 | host: dex.kube.local 62 | port: 5556 63 | serviceName: dex 64 | -------------------------------------------------------------------------------- /examples/sample-prometheus-app/sample_prometheus_app_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "crypto/tls" 5 | "fmt" 6 | "strings" 7 | "testing" 8 | "time" 9 | 10 | _ "github.com/stretchr/testify/assert" 11 | 12 | "github.com/gruntwork-io/terratest/modules/k8s" 13 | 14 | http_helper "github.com/gruntwork-io/terratest/modules/http-helper" 15 | ) 16 | 17 | func TestSamplePrometheusApp(t *testing.T) { 18 | // ============================================================= 19 | kubectlOptions := k8s.NewKubectlOptions("", "", "istio-ingress") 20 | 21 | k8s.WaitUntilServiceAvailable(t, kubectlOptions, "istio-ingressgateway", 10, 30*time.Second) 22 | 23 | // ============================================================= 24 | kubectlOptions = k8s.NewKubectlOptions("", "", "knative-serving") 25 | 26 | k8s.WaitUntilServiceAvailable(t, kubectlOptions, "controller", 10, 5*time.Second) 27 | 28 | // ============================================================= 29 | kubectlOptions = k8s.NewKubectlOptions("", "", "prometheus") 30 | 31 | k8s.KubectlApply(t, kubectlOptions, "sample-prometheus-app.yaml") 32 | 33 | // ============================================================= 34 | tunnel := k8s.NewTunnel(kubectlOptions, k8s.ResourceTypePod, "prometheus-kube-prometheus-stack-kube-prometheus-0", 0, 9090) 35 | 36 | defer tunnel.Close() 37 | 38 | tunnel.ForwardPort(t) 39 | 40 | // ============================================================= 41 | endpoint := fmt.Sprintf("http://%s%s", tunnel.Endpoint(), "/api/v1/targets?state=active") 42 | 43 | tlsConfig := tls.Config{} 44 | 45 | http_helper.HttpGetWithRetryWithCustomValidation( 46 | t, 47 | endpoint, 48 | &tlsConfig, 49 | 30, 50 | 3*time.Second, 51 | verifySamplePrometheusTarget, 52 | ) 53 | 54 | // ============================================================= 55 | k8s.KubectlDelete(t, kubectlOptions, "sample-prometheus-app.yaml") 56 | } 57 | 58 | func verifySamplePrometheusTarget(statusCode int, body string) bool { 59 | if statusCode != 200 { 60 | return false 61 | } 62 | return strings.Contains(body, "podMonitor/prometheus/sample-prometheus-app") 63 | } 64 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/templates/_helpers.tpl: -------------------------------------------------------------------------------- 1 | {{/* 2 | Expand the name of the chart. 3 | */}} 4 | {{- define "git-http-backend.name" -}} 5 | {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} 6 | {{- end }} 7 | 8 | {{/* 9 | Create a default fully qualified app name. 10 | We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). 11 | If release name contains chart name it will be used as a full name. 12 | */}} 13 | {{- define "git-http-backend.fullname" -}} 14 | {{- if .Values.fullnameOverride }} 15 | {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} 16 | {{- else }} 17 | {{- $name := default .Chart.Name .Values.nameOverride }} 18 | {{- if contains $name .Release.Name }} 19 | {{- .Release.Name | trunc 63 | trimSuffix "-" }} 20 | {{- else }} 21 | {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} 22 | {{- end }} 23 | {{- end }} 24 | {{- end }} 25 | 26 | {{/* 27 | Create chart name and version as used by the chart label. 28 | */}} 29 | {{- define "git-http-backend.chart" -}} 30 | {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} 31 | {{- end }} 32 | 33 | {{/* 34 | Common labels 35 | */}} 36 | {{- define "git-http-backend.labels" -}} 37 | helm.sh/chart: {{ include "git-http-backend.chart" . }} 38 | {{ include "git-http-backend.selectorLabels" . }} 39 | {{- if .Chart.AppVersion }} 40 | app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} 41 | {{- end }} 42 | app.kubernetes.io/managed-by: {{ .Release.Service }} 43 | {{- end }} 44 | 45 | {{/* 46 | Selector labels 47 | */}} 48 | {{- define "git-http-backend.selectorLabels" -}} 49 | app.kubernetes.io/name: {{ include "git-http-backend.name" . }} 50 | app.kubernetes.io/instance: {{ .Release.Name }} 51 | {{- end }} 52 | 53 | {{/* 54 | Create the name of the service account to use 55 | */}} 56 | {{- define "git-http-backend.serviceAccountName" -}} 57 | {{- if .Values.serviceAccount.create }} 58 | {{- default (include "git-http-backend.fullname" .) .Values.serviceAccount.name }} 59 | {{- else }} 60 | {{- default "default" .Values.serviceAccount.name }} 61 | {{- end }} 62 | {{- end }} 63 | -------------------------------------------------------------------------------- /.github/workflows/integration-tests.yaml: -------------------------------------------------------------------------------- 1 | name: Run Integration Tests 2 | 3 | on: 4 | pull_request_review: 5 | types: 6 | - submitted 7 | 8 | jobs: 9 | load-matrix: 10 | if: github.event.review.state == 'approved' && !startsWith(github.event.pull_request.title, 'docs:') 11 | uses: ./.github/workflows/load-kind-versions.yaml 12 | 13 | integration-tests: 14 | if: github.event.review.state == 'approved' && !startsWith(github.event.pull_request.title, 'docs:') 15 | needs: load-matrix 16 | strategy: 17 | matrix: 18 | INITIUM_K8S_VERSION: ${{ fromJSON(needs.load-matrix.outputs.matrix) }} 19 | runs-on: 20 | labels: ubuntu-latest-4-cores 21 | timeout-minutes: 15 22 | steps: 23 | - name: Checkout 24 | uses: actions/checkout@v4 25 | 26 | - name: Integration test 27 | run: | 28 | # Install asdf and expose to PATH 29 | git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.2 30 | . $HOME/.asdf/asdf.sh 31 | # Add asdf plugins and install tools in .tool-versions 32 | make asdf_install 33 | # Create a local branch to allow the next step to work 34 | git checkout -b ci 35 | # Deploy cluster 36 | make ci 37 | # Run tests 38 | make integration-test 39 | env: 40 | GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }} 41 | INITIUM_AOA_EXCLUDE_DEX: true 42 | INITIUM_AOA_EXCLUDE_PROMETHEUSSTACK: true 43 | INITIUM_AOA_EXCLUDE_OPENTELEMETRY: true 44 | INITIUM_K8S_VERSION: ${{ matrix.INITIUM_K8S_VERSION }} 45 | 46 | # https://github.com/orgs/community/discussions/26822#discussioncomment-5122101 47 | integration-tests-result: 48 | if: github.event.review.state == 'approved' 49 | runs-on: ubuntu-latest 50 | name: Integration Tests 51 | needs: [integration-tests] 52 | steps: 53 | - run: exit 1 54 | # see https://stackoverflow.com/a/67532120/4907315 55 | if: >- 56 | ${{ 57 | contains(needs.*.result, 'failure') 58 | || contains(needs.*.result, 'cancelled') 59 | }} 60 | -------------------------------------------------------------------------------- /architectural-design-records/dependency-management-automation.md: -------------------------------------------------------------------------------- 1 | ## Summary 2 | 3 | Use a Renovate bot to automatically update dependencies in our GitHub repository. 4 | 5 | ### Problem 6 | 7 | Keeping dependencies up to date can be a time-consuming task, especially for large projects with many dependencies. It can be easy to miss updates, and updating dependencies manually can introduce errors and inconsistencies. As the number of charts that the project supports will grow, 8 | keeping up-to-date with the latest releases manually is a time-consuming effort. 9 | 10 | ### Decision 11 | 12 | We will use a Renovate bot to automatically update dependencies in our GitHub repository. The bot will check for updates regularly and open pull requests for any updates that are found and merge automatically if the checks will pass and the change is backward compatible (patch or minor). The pull requests with major changes will then be reviewed and merged by the project maintainers. 13 | 14 | ### Status 15 | 16 | In-use 17 | 18 | ## Details 19 | 20 | ### Constraints 21 | 22 | * The bot should be configured to respect the semantic versioning of the packages. 23 | * Should not update packages that are not compatible with the current version. 24 | * Must be able to configure the schedule of updates. 25 | * Should be able to lock certain package versions. 26 | 27 | ### Argument 28 | 29 | Using a Renovate bot will save time and reduce errors by automating the process of updating dependencies. Additionally, the bot can be configured to follow best practices for dependency management, such as respecting semantic versioning and not updating packages that are not compatible with the current version. 30 | 31 | ### Implications 32 | 33 | * The team of maintainers will need to review and merge pull requests created by the bot regularly. 34 | * Dependencies will be kept up to date and the team can focus on other tasks. 35 | * The bot can break the current build if not configured properly 36 | 37 | ## Notes 38 | * The Renovate bot can be configured to only update certain types of dependencies (e.g. only direct dependencies). 39 | * Some add-ons will need additional renovate bot configuration update to group the versions update. 40 | 41 | ## Record history 42 | 11/03/2023 - Problem defined and decisions recorded 43 | -------------------------------------------------------------------------------- /architectural-design-records/gitops.md: -------------------------------------------------------------------------------- 1 | # GitOps Architectural Decision Record 2 | 3 | Contents: 4 | 5 | * [Summary](#summary) 6 | * [Problem](#problem) 7 | * [Decision](#decision) 8 | * [Status](#status) 9 | * [Details](#details) 10 | * [Constraints](#constraints) 11 | * [Argument](#argument) 12 | * [Implications](#implications) 13 | * [Notes](#notes) 14 | * [Record history](#Record history) 15 | 16 | 17 | ## Summary 18 | 19 | In initium-platform, one of the key functionalities is managing the addons. Ideally, we needed a GitOps solution for handling that. During the idealization of this repo, our engineers suggested ArgoCD as the technology for solving the problem, since it is the solution of choice for GitOps in Nearform. It also fits the GitOps principles, such as declarative definitions, versioning of the system (in our case, the addons), checking for changes automatically and drift consolidation. 20 | 21 | ### Problem 22 | 23 | Decide which tool is going to be used for GitOps, and handling the management of the addons for initium-platform. 24 | 25 | ### Decision 26 | 27 | ArgoCD was accepted as a great addition to initium-platform. 28 | 29 | ### Status 30 | 31 | ArgoCD is currently being used in initium-platform. 32 | 33 | ## Details 34 | 35 | ### Constraints 36 | 37 | * We need a technology that is declarative and supports specific revisions for each addon. 38 | * The technology should also look out for changes and solve drifts. 39 | * The technology should be part of Nearform's tech radar. 40 | * The technology must be well-known by the community and with active contributors. 41 | * The technology must be open-source. 42 | * The software should also be easy to use and configure. 43 | * A developer should be able to learn the addon in a few days, so initium-platform won't be an overhead to the implementation. 44 | 45 | ### Argument 46 | 47 | ArgoCD was deployed and tested on a few developers' dev environments. 48 | The process of installing and configuring ArgoCD, as well as running an addon was easy - less than a day of work. 49 | ArgoCD fits the constraints mentioned above. 50 | 51 | ### Implications 52 | 53 | ArgoCD is the most opinionated part of the solution and it is nested into further assumptions. 54 | 55 | ## Notes 56 | 57 | n/a 58 | 59 | ## Record history 60 | 11/11/2022 - Problem defined and decisions recorded 61 | -------------------------------------------------------------------------------- /utils/git-http-backend/chart/templates/deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: {{ include "git-http-backend.fullname" . }} 5 | labels: 6 | {{- include "git-http-backend.labels" . | nindent 4 }} 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | {{- include "git-http-backend.selectorLabels" . | nindent 6 }} 12 | template: 13 | metadata: 14 | {{- with .Values.podAnnotations }} 15 | annotations: 16 | {{- toYaml . | nindent 8 }} 17 | {{- end }} 18 | labels: 19 | {{- include "git-http-backend.selectorLabels" . | nindent 8 }} 20 | spec: 21 | {{- with .Values.imagePullSecrets }} 22 | imagePullSecrets: 23 | {{- toYaml . | nindent 8 }} 24 | {{- end }} 25 | serviceAccountName: {{ include "git-http-backend.serviceAccountName" . }} 26 | securityContext: 27 | {{- toYaml .Values.podSecurityContext | nindent 8 }} 28 | containers: 29 | - name: {{ .Chart.Name }} 30 | securityContext: 31 | {{- toYaml .Values.securityContext | nindent 12 }} 32 | image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" 33 | imagePullPolicy: {{ .Values.image.pullPolicy }} 34 | ports: 35 | - name: http 36 | containerPort: 80 37 | protocol: TCP 38 | volumeMounts: 39 | - mountPath: /git 40 | name: git-volume 41 | readinessProbe: 42 | tcpSocket: 43 | port: 80 44 | initialDelaySeconds: 5 45 | periodSeconds: 10 46 | livenessProbe: 47 | tcpSocket: 48 | port: 80 49 | initialDelaySeconds: 15 50 | periodSeconds: 20 51 | resources: 52 | {{- toYaml .Values.resources | nindent 12 }} 53 | volumes: 54 | - name: git-volume 55 | hostPath: 56 | path: {{ .Values.volumes.git_volume.path }} 57 | {{- with .Values.nodeSelector }} 58 | nodeSelector: 59 | {{- toYaml . | nindent 8 }} 60 | {{- end }} 61 | {{- with .Values.affinity }} 62 | affinity: 63 | {{- toYaml . | nindent 8 }} 64 | {{- end }} 65 | {{- with .Values.tolerations }} 66 | tolerations: 67 | {{- toYaml . | nindent 8 }} 68 | {{- end }} 69 | -------------------------------------------------------------------------------- /architectural-design-records/release-management-automation.md: -------------------------------------------------------------------------------- 1 | ## Summary 2 | 3 | Use a Release-please GitHub action to automatically release new versions of our software on GitHub. 4 | 5 | ### Problem 6 | 7 | Releasing new versions of software can be a time-consuming task, especially for teams that release software frequently. It can be easy to miss steps or make errors during the release process, and releasing software manually can take up valuable development time. 8 | 9 | ### Decision 10 | 11 | We will use a Release-please GitHub action to automatically release new versions of our software on GitHub. The action will be triggered by a merge to the main branch or specific commit tag, it will check all the CI checks passed, update the version of the package, create a release and push it to the appropriate git tag and branch. Additionally, there'll be an action that will upload the app-of-apps.yaml to the release assets. 12 | 13 | ### Status 14 | 15 | In-use 16 | 17 | ## Details 18 | 19 | ### Constraints 20 | 21 | * The action should be configured to respect the semantic versioning of the packages. 22 | * Should be able to define the release branch, where it should push the release. 23 | * Must be able to define the git tag format. 24 | * Should be able to validate the release notes. 25 | * Should be able to push the release notes to GitHub release notes. 26 | 27 | ### Argument 28 | 29 | Using a Release-please GitHub action will save time and reduce errors by automating the process of releasing new versions of software. Additionally, the action can be configured to follow best practices for releasing software, such as respecting semantic versioning, validating release notes, and pushing the release notes to GitHub releases. 30 | 31 | ### Implications 32 | 33 | * The team will need to make sure that the git tag and branch are correctly defined in the workflow configuration. 34 | * New versions will be released automatically, and the team can focus on other tasks. 35 | * The action might break the release process if not configured properly 36 | 37 | ## Notes 38 | 39 | * The Release-please GitHub action can be configured to use different release strategies, such as major, minor, and patch releases. 40 | * The action can also be configured to add "release" labels to pull requests, making it easy to identify releases that are waiting to be deployed. 41 | 42 | ## Record history 43 | 11/03/2023 - Problem defined and decisions recorded 44 | -------------------------------------------------------------------------------- /app-of-apps/values.yaml: -------------------------------------------------------------------------------- 1 | namespace: argocd 2 | spec: 3 | destination: 4 | server: https://kubernetes.default.svc 5 | subChartsRevision: HEAD 6 | repoURL: https://github.com/nearform/initium-platform.git 7 | externalDomain: example.com 8 | apps: 9 | # ArgoCD - https://github.com/argoproj/argo-cd/ 10 | argocd: 11 | name: argocd 12 | targetNamespace: argocd 13 | excluded: true 14 | 15 | # Cert-manager - https://cert-manager.io/ 16 | cert-manager: 17 | name: cert-manager 18 | targetNamespace: cert-manager 19 | excluded: false 20 | 21 | # Dex - https://dexidp.io/ 22 | dex: 23 | name: dex 24 | targetNamespace: dex 25 | 26 | # Prometheus + Operator - https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack 27 | # External grafana app needs to be deployed together with this stack 28 | kube-prometheus-stack: 29 | name: kube-prometheus-stack 30 | targetNamespace: prometheus 31 | serverSideApply: true 32 | 33 | # Istio - https://istio.io/ 34 | istio-base: 35 | name: istio-base 36 | path: addons/istio/base/ 37 | targetNamespace: istio-system 38 | syncWave: -20 39 | 40 | istiod: 41 | name: istiod 42 | path: addons/istio/istiod/ 43 | targetNamespace: istio-system 44 | syncWave: -20 45 | 46 | istio-ingressgateway: 47 | name: istio-ingressgateway 48 | path: addons/istio/ingressgateway/ 49 | targetNamespace: istio-ingress 50 | 51 | # Knative - https://knative.dev/ 52 | knative: 53 | name: knative 54 | targetNamespace: knative 55 | 56 | # OpenTelemetry - https://opentelemetry.io/ 57 | opentelemetry-collector: 58 | name: opentelemetry-collector 59 | targetNamespace: opentelemetry 60 | 61 | # Grafana Loki - https://grafana.com/oss/loki/ 62 | loki: 63 | name: loki 64 | targetNamespace: loki 65 | ignoreDifferences: 66 | - group: apps 67 | kind: StatefulSet 68 | jsonPointers: 69 | - /spec/persistentVolumeClaimRetentionPolicy 70 | 71 | # Grafana - https://grafana.com/oss/ 72 | grafana: 73 | name: grafana 74 | targetNamespace: grafana 75 | ignoreDifferences: 76 | - group: v1 77 | kind: Secret 78 | jsonPointers: 79 | - /data/admin-password 80 | 81 | # Kubernetes Replicator 82 | kubernetes-replicator-source: 83 | name: kubernetes-replicator 84 | targetNamespace: kubernetes-replicator 85 | -------------------------------------------------------------------------------- /scripts/ci.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source .envrc 4 | 5 | # Generate report of current env vars 6 | echo "======================================================" 7 | printenv | grep "INITIUM_.*" 8 | echo "======================================================" 9 | 10 | # Run Tilt CI 11 | tilt ci 12 | 13 | if [ "${INITIUM_DEPLOY_MINIMAL}" == "false" ]; then 14 | # Login on ArgoCD 15 | argocd login --core --name initium-platform 16 | kubectl config set-context --current --namespace=argocd 17 | 18 | # Ensure ArgoCD apps are all healthy and in sync 19 | echo ">> Waiting for initium-platform to be healty and in sync..." 20 | while [ true ] 21 | do 22 | ALL_HEALTHY=true 23 | IFS=$'\n' read -r -d '' -a apps_health < <(argocd app get initium-platform -o json | jq -r '.status.resources | .[]? | select(.kind | contains("Application")) | .health.status') 24 | 25 | if (( ${#apps_health[@]} == 0 )); then 26 | ALL_HEALTHY=false 27 | else 28 | for status in ${apps_health[@]} 29 | do 30 | if [ "$status" != "Healthy" ]; then 31 | ALL_HEALTHY=false 32 | fi 33 | done 34 | fi 35 | 36 | ALL_SYNCED=true 37 | IFS=$'\n' read -r -d '' -a apps_sync < <(argocd app get initium-platform -o json | jq -r '.status.resources | .[]? | select(.kind | contains("Application")) | .status') 38 | if (( ${#apps_sync[@]} == 0 )); then 39 | ALL_SYNCED=false 40 | else 41 | for status in ${apps_sync[@]} 42 | do 43 | if [ "$status" != "Synced" ]; then 44 | ALL_SYNCED=false 45 | fi 46 | done 47 | fi 48 | 49 | if [[ "$ALL_HEALTHY" == "true" && "$ALL_SYNCED" == "true" ]]; then 50 | break 51 | else 52 | argocd app get initium-platform 53 | 54 | # Print the 10 last lines of logs of apps not currently healthy 55 | IFS=$'\n' read -r -d '' -a apps < <(argocd app get initium-platform -o json | jq -r '.status.resources | .[]? | select(.kind | contains("Application")) | select(.health.status | contains("Progressing")) | .name') 56 | for app in ${apps[@]} 57 | do 58 | echo ">> Printing last 10 log lines for $app..." 59 | argocd app logs $app --tail 10 60 | done 61 | fi 62 | 63 | sleep 15 64 | done 65 | 66 | # List all the installed apps 67 | argocd app list 68 | 69 | # Logout on ArgoCD 70 | argocd logout initium-platform 71 | kubectl config set-context --current --namespace=default 72 | fi 73 | -------------------------------------------------------------------------------- /examples/sample-app/sample_app_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "strings" 7 | "testing" 8 | "time" 9 | 10 | "github.com/stretchr/testify/assert" 11 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 12 | 13 | "github.com/gruntwork-io/terratest/modules/k8s" 14 | 15 | http_helper "github.com/gruntwork-io/terratest/modules/http-helper" 16 | ) 17 | 18 | func TestExampleApp(t *testing.T) { 19 | // ============================================================= 20 | kubectlOptions := k8s.NewKubectlOptions("", "", "istio-ingress") 21 | 22 | k8s.WaitUntilServiceAvailable(t, kubectlOptions, "istio-ingressgateway", 10, 30*time.Second) 23 | 24 | // ============================================================= 25 | kubectlOptions = k8s.NewKubectlOptions("", "", "knative-serving") 26 | 27 | k8s.WaitUntilServiceAvailable(t, kubectlOptions, "controller", 10, 5*time.Second) 28 | 29 | // ============================================================= 30 | kubectlOptions = k8s.NewKubectlOptions("", "", "default") 31 | 32 | k8s.KubectlApply(t, kubectlOptions, "sample-app.yaml") 33 | 34 | // ============================================================= 35 | 36 | numRequests := 5 37 | 38 | for i := 0; i < numRequests; i++ { 39 | 40 | http_helper.HTTPDoWithRetry( 41 | t, 42 | "GET", 43 | fmt.Sprintf("http://%s", os.Getenv("INITIUM_LB_ENDPOINT")), 44 | []byte("Hello world from initium-platform!"), 45 | map[string]string{"Host": "sample-app.default.example.com"}, 46 | 200, 47 | 30, 48 | 3*time.Second, 49 | nil, 50 | ) 51 | } 52 | 53 | sampleAppLabel := "ksample" 54 | prefix := "sample-app" 55 | 56 | //Wait for the pods to be created 57 | time.Sleep(5 * time.Second) 58 | 59 | podOptions := metav1.ListOptions{ 60 | LabelSelector: fmt.Sprintf("service=%s", sampleAppLabel), 61 | } 62 | 63 | pods := k8s.ListPods(t, kubectlOptions, podOptions) 64 | 65 | assert.NotNil(t, pods) 66 | // Validate knative scale-out process was effective 67 | assert.Greater(t, len(pods), 1) 68 | 69 | // Wait for knative to scale-in 70 | time.Sleep(70 * time.Second) 71 | 72 | replicaSets, err := k8s.RunKubectlAndGetOutputE(t, kubectlOptions, "get", "rs") 73 | if err != nil { 74 | t.Fatal(err) 75 | } 76 | 77 | replicaSetNames := strings.Fields(replicaSets) 78 | 79 | var replicaSet string 80 | for _, rs := range replicaSetNames { 81 | if strings.HasPrefix(rs, prefix) { 82 | replicaSet = rs 83 | } 84 | } 85 | 86 | rs := k8s.GetReplicaSet(t, kubectlOptions, replicaSet) 87 | // Validate knative scale-in process was effective 88 | assert.Equal(t, int(rs.Status.Replicas), 0) 89 | 90 | // ============================================================= 91 | k8s.KubectlDelete(t, kubectlOptions, "sample-app.yaml") 92 | } 93 | -------------------------------------------------------------------------------- /examples/sample-app/go.mod: -------------------------------------------------------------------------------- 1 | module test 2 | 3 | go 1.19 4 | 5 | require ( 6 | github.com/gruntwork-io/terratest v0.41.3 7 | github.com/stretchr/testify v1.8.1 8 | ) 9 | 10 | require ( 11 | cloud.google.com/go v0.83.0 // indirect 12 | github.com/aws/aws-sdk-go v1.40.56 // indirect 13 | github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect 14 | github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect 15 | github.com/davecgh/go-spew v1.1.1 // indirect 16 | github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c // indirect 17 | github.com/go-errors/errors v1.0.2-0.20180813162953-d98b870cc4e0 // indirect 18 | github.com/go-logr/logr v0.2.0 // indirect 19 | github.com/go-sql-driver/mysql v1.4.1 // indirect 20 | github.com/gogo/protobuf v1.3.2 // indirect 21 | github.com/golang/protobuf v1.5.2 // indirect 22 | github.com/google/gofuzz v1.1.0 // indirect 23 | github.com/google/uuid v1.2.0 // indirect 24 | github.com/googleapis/gnostic v0.4.1 // indirect 25 | github.com/gruntwork-io/go-commons v0.8.0 // indirect 26 | github.com/hashicorp/errwrap v1.0.0 // indirect 27 | github.com/hashicorp/go-multierror v1.1.0 // indirect 28 | github.com/imdario/mergo v0.3.11 // indirect 29 | github.com/jmespath/go-jmespath v0.4.0 // indirect 30 | github.com/json-iterator/go v1.1.11 // indirect 31 | github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 // indirect 32 | github.com/mitchellh/go-homedir v1.1.0 // indirect 33 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 34 | github.com/modern-go/reflect2 v1.0.1 // indirect 35 | github.com/pmezard/go-difflib v1.0.0 // indirect 36 | github.com/pquerna/otp v1.2.0 // indirect 37 | github.com/russross/blackfriday/v2 v2.1.0 // indirect 38 | github.com/spf13/pflag v1.0.5 // indirect 39 | github.com/urfave/cli v1.22.2 // indirect 40 | golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a // indirect 41 | golang.org/x/net v0.0.0-20210614182718-04defd469f4e // indirect 42 | golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c // indirect 43 | golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f // indirect 44 | golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect 45 | golang.org/x/text v0.3.8 // indirect 46 | golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e // indirect 47 | google.golang.org/appengine v1.6.7 // indirect 48 | google.golang.org/protobuf v1.26.0 // indirect 49 | gopkg.in/inf.v0 v0.9.1 // indirect 50 | gopkg.in/yaml.v2 v2.4.0 // indirect 51 | gopkg.in/yaml.v3 v3.0.1 // indirect 52 | k8s.io/api v0.20.6 // indirect 53 | k8s.io/apimachinery v0.20.6 // indirect 54 | k8s.io/client-go v0.20.6 // indirect 55 | k8s.io/klog/v2 v2.4.0 // indirect 56 | k8s.io/utils v0.0.0-20201110183641-67b214c5f920 // indirect 57 | sigs.k8s.io/structured-merge-diff/v4 v4.0.3 // indirect 58 | sigs.k8s.io/yaml v1.2.0 // indirect 59 | ) 60 | -------------------------------------------------------------------------------- /examples/sample-prometheus-app/go.mod: -------------------------------------------------------------------------------- 1 | module test 2 | 3 | go 1.18 4 | 5 | require ( 6 | cloud.google.com/go v0.83.0 // indirect 7 | github.com/aws/aws-sdk-go v1.40.56 // indirect 8 | github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect 9 | github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect 10 | github.com/davecgh/go-spew v1.1.1 // indirect 11 | github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c // indirect 12 | github.com/go-errors/errors v1.0.2-0.20180813162953-d98b870cc4e0 // indirect 13 | github.com/go-logr/logr v0.2.0 // indirect 14 | github.com/go-sql-driver/mysql v1.4.1 // indirect 15 | github.com/gogo/protobuf v1.3.2 // indirect 16 | github.com/golang/protobuf v1.5.2 // indirect 17 | github.com/google/gofuzz v1.1.0 // indirect 18 | github.com/google/uuid v1.2.0 // indirect 19 | github.com/googleapis/gnostic v0.4.1 // indirect 20 | github.com/gruntwork-io/go-commons v0.8.0 // indirect 21 | github.com/gruntwork-io/terratest v0.41.7 // indirect 22 | github.com/hashicorp/errwrap v1.0.0 // indirect 23 | github.com/hashicorp/go-multierror v1.1.0 // indirect 24 | github.com/imdario/mergo v0.3.11 // indirect 25 | github.com/jmespath/go-jmespath v0.4.0 // indirect 26 | github.com/json-iterator/go v1.1.11 // indirect 27 | github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 // indirect 28 | github.com/mitchellh/go-homedir v1.1.0 // indirect 29 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 30 | github.com/modern-go/reflect2 v1.0.1 // indirect 31 | github.com/pmezard/go-difflib v1.0.0 // indirect 32 | github.com/pquerna/otp v1.2.0 // indirect 33 | github.com/russross/blackfriday/v2 v2.1.0 // indirect 34 | github.com/spf13/pflag v1.0.5 // indirect 35 | github.com/stretchr/testify v1.7.0 // indirect 36 | github.com/urfave/cli v1.22.2 // indirect 37 | golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a // indirect 38 | golang.org/x/net v0.0.0-20210614182718-04defd469f4e // indirect 39 | golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c // indirect 40 | golang.org/x/sys v0.0.0-20220517195934-5e4e11fc645e // indirect 41 | golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect 42 | golang.org/x/text v0.3.6 // indirect 43 | golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e // indirect 44 | google.golang.org/appengine v1.6.7 // indirect 45 | google.golang.org/protobuf v1.26.0 // indirect 46 | gopkg.in/inf.v0 v0.9.1 // indirect 47 | gopkg.in/yaml.v2 v2.4.0 // indirect 48 | gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect 49 | k8s.io/api v0.20.6 // indirect 50 | k8s.io/apimachinery v0.20.6 // indirect 51 | k8s.io/client-go v0.20.6 // indirect 52 | k8s.io/klog/v2 v2.4.0 // indirect 53 | k8s.io/utils v0.0.0-20201110183641-67b214c5f920 // indirect 54 | sigs.k8s.io/structured-merge-diff/v4 v4.0.3 // indirect 55 | sigs.k8s.io/yaml v1.2.0 // indirect 56 | ) 57 | -------------------------------------------------------------------------------- /.envrc: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Go environment 4 | # =================== 5 | 6 | CACHE_PATH=".cache/gopath" 7 | mkdir -p $CACHE_PATH 8 | export GOPATH=$(realpath $CACHE_PATH) 9 | 10 | # Local variables 11 | # =============== 12 | CURRENT_PATH=$(pwd) 13 | REPO_NAME=$(basename $CURRENT_PATH) 14 | REPO_HOST_PATH=$(realpath $CURRENT_PATH/..) 15 | REPO_NODE_PATH="/srv/git" 16 | REPO_URI="http://git-http-backend/git/${REPO_NAME}" 17 | REPO_BRANCH=$(git branch --show-current) 18 | 19 | OS_ARCH=$(uname -m) 20 | if [ "$OS_ARCH" == "arm64" ] || [ "$OS_ARCH" == "aarch64" ]; then 21 | K8S_VERSION="kindest/node:v1.27.3@sha256:de0b3dfe848ccf07e24f4278eaf93edb857b6231b39773f46b36a2b1a6543ae9" # arm64 image 22 | else 23 | K8S_VERSION="kindest/node:v1.27.3@sha256:9dd3392d79af1b084671b05bcf65b21de476256ad1dcc853d9f3b10b4ac52dde" # amd64 image 24 | fi 25 | 26 | CERT_VALIDITY_DAYS=180 27 | 28 | # Export as environment variables if not already defined 29 | # ====================================================== 30 | export INITIUM_REPO_NAME=${INITIUM_REPO_NAME:-${REPO_NAME}} 31 | export INITIUM_REPO_HOST_PATH=${INITIUM_REPO_HOST_PATH:-${REPO_HOST_PATH}} 32 | export INITIUM_REPO_NODE_PATH=${INITIUM_REPO_NODE_PATH:-${REPO_NODE_PATH}} 33 | export INITIUM_REPO_URI=${INITIUM_REPO_URI:-${REPO_URI}} 34 | export INITIUM_REPO_BRANCH=${INITIUM_REPO_BRANCH:-${REPO_BRANCH}} 35 | export INITIUM_K8S_VERSION=${INITIUM_K8S_VERSION:-${K8S_VERSION}} 36 | export INITIUM_CERT_VALIDITY_DAYS=${INITIUM_CERT_VALIDITY_DAYS:-${CERT_VALIDITY_DAYS}} 37 | 38 | # Required by MetalLB - https://kind.sigs.k8s.io/docs/user/loadbalancer/ 39 | # ====================================================================== 40 | if [ $(docker network ls -f "name=kind" -q) ]; then 41 | KIND_NET_CIDR=$(docker network inspect -f '{{ (index .IPAM.Config 0).Subnet }}' kind) 42 | METALLB_IP_START=$(echo ${KIND_NET_CIDR} | sed "s@0.0/16@255.200@") 43 | METALLB_IP_END=$(echo ${KIND_NET_CIDR} | sed "s@0.0/16@255.209@") 44 | export INITIUM_METALLB_CIDR="${METALLB_IP_START}-${METALLB_IP_END}" 45 | fi 46 | 47 | # Enable this flag if you want to just deploy the K8s cluster 48 | # with MetalLB in place, to allow Loadbalancer services to work 49 | # ============================================================= 50 | export INITIUM_DEPLOY_MINIMAL="${INITIUM_DEPLOY_MINIMAL:-false}" 51 | 52 | # Control which addons we might want to exclude from default deployment, 53 | # if INITIUM_DEPLOY_MINIMAL == false 54 | # ====================================================================== 55 | # Istio 56 | export INITIUM_AOA_EXCLUDE_ISTIO="${INITIUM_AOA_EXCLUDE_ISTIO:-false}" 57 | # Knative depends on Istio by default, if we exclude it we exclude Knative as well 58 | if [ "${INITIUM_AOA_EXCLUDE_ISTIO}" = "true" ]; then 59 | export INITIUM_AOA_EXCLUDE_KNATIVE="true" 60 | else 61 | export INITIUM_AOA_EXCLUDE_KNATIVE="${INITIUM_AOA_EXCLUDE_KNATIVE:-false}" 62 | fi 63 | # Prometheus + Grafana stack 64 | export INITIUM_AOA_EXCLUDE_PROMETHEUSSTACK="${INITIUM_AOA_EXCLUDE_PROMETHEUSSTACK:-false}" 65 | # Dex 66 | export INITIUM_AOA_EXCLUDE_DEX="${INITIUM_AOA_EXCLUDE_DEX:-false}" 67 | # OpenTelemetry 68 | export INITIUM_AOA_EXCLUDE_OPENTELEMETRY="${INITIUM_AOA_EXCLUDE_OPENTELEMETRY:-false}" 69 | -------------------------------------------------------------------------------- /docs/LOG_AGGREGATION.md: -------------------------------------------------------------------------------- 1 | # Log Aggregation 2 | 3 | ## Introduction 4 | The log aggregation layer is based on Loki and OpenTelemetry (OTLP) collector. Logs are fetched, processed and exported to Loki using the OTLP collector. 5 | Ingested logs can be further analyzed using Grafana. 6 | 7 | ## OpenTelemetry Collector 8 | The OpenTelemetry Collector offers a vendor-agnostic implementation which supports the collection of Kubernetes container logs and their export to the Loki backend. 9 | The collector is deployed as a daemon set (agent mode) which is the most suitable for collecting container logs. It has been deployed using Helm Chart 10 | rather than the Operator since the Helm chart support presets for `log collection` and `kubernetes attributes` while these features are still 11 | missing from the OpenTelemetryCollector CRD used by the Operator. 12 | 13 | Loki backend (exporter) is supported through [Contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/lokiexporter) where components that are not suitable for the core repository are placed. 14 | In the processor stage, the Loki exporter is used to convert OTLP resources and log attributes into Loki labels, which are indexed. 15 | In that way, labels like `name_space`, `pod_name`, `container_name`, and `log_iostream`, can be used for issuing fast log queries from Grafana. 16 | 17 | ## Loki 18 | Loki is a log aggregation system which is simple and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. 19 | It has been deployed in `single binary` mode with the logs stored in the file system. Parameters like `parallelise_shardable_queries=false`, 20 | `split_queries_by_interval=0` and `max_outstanding_requests_per_tenant=10000` are used to tune performance for mentioned deployment mode. 21 | 22 | ## Grafana 23 | Grafana is deployed independently of Loki to be used by both Loki and Kube Prometheus applications. 24 | It can be accessed on `https://grafana.kube.local` through MetalLB load balancer, but first resolving needs to be configured in /etc/hosts file: 25 | ```bash 26 | 172.18.255.200 grafana.kube.local 27 | ``` 28 | At this moment logging in to Grafana is done by using default credentials defined in [Grafana values](https://github.com/nearform/initium-platform/blob/main/addons/grafana/values.yaml#L6) 29 | In the future default password will be replaced with generated one. 30 | Grafana is preconfigured to use Loki data source and logs are available shortly upon stack deployment. Custom dashboards for Loki are not included with Grafana, rather idea is to use Grafana’s `explore` feature. 31 | 32 | Log filtering is based on LogQL, a Grafana Loki’s PromQL-inspired query language. It uses labels and operators for filtering. A few examples are shown below: 33 | ```bash 34 | {exporter="OTLP"} |= "error" # uses exporter=OTLP label to select all logs and further filter for errors 35 | ``` 36 | ```bash 37 | {container_name =~ "istio.+"} |= "timeout" # uses container_name label and regular expression to filter for all istio containers and timeout keyword 38 | ``` 39 | 40 | More details can be found in Grafana [Log queries documentation](https://grafana.com/docs/loki/latest/logql/log_queries/) 41 | -------------------------------------------------------------------------------- /tests/go.mod: -------------------------------------------------------------------------------- 1 | module github.com/nearform/initium-platform 2 | 3 | go 1.21 4 | 5 | require ( 6 | github.com/gruntwork-io/terratest v0.43.11 7 | github.com/stretchr/testify v1.8.4 8 | gopkg.in/yaml.v3 v3.0.1 9 | k8s.io/apimachinery v0.27.3 10 | ) 11 | 12 | require ( 13 | github.com/aws/aws-sdk-go v1.44.122 // indirect 14 | github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect 15 | github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect 16 | github.com/davecgh/go-spew v1.1.1 // indirect 17 | github.com/emicklei/go-restful/v3 v3.9.0 // indirect 18 | github.com/ghodss/yaml v1.0.0 // indirect 19 | github.com/go-errors/errors v1.0.2-0.20180813162953-d98b870cc4e0 // indirect 20 | github.com/go-logr/logr v1.2.3 // indirect 21 | github.com/go-openapi/jsonpointer v0.19.6 // indirect 22 | github.com/go-openapi/jsonreference v0.20.1 // indirect 23 | github.com/go-openapi/swag v0.22.3 // indirect 24 | github.com/go-sql-driver/mysql v1.4.1 // indirect 25 | github.com/gogo/protobuf v1.3.2 // indirect 26 | github.com/golang/protobuf v1.5.3 // indirect 27 | github.com/google/gnostic v0.5.7-v3refs // indirect 28 | github.com/google/go-cmp v0.5.9 // indirect 29 | github.com/google/gofuzz v1.1.0 // indirect 30 | github.com/google/uuid v1.3.0 // indirect 31 | github.com/gruntwork-io/go-commons v0.8.0 // indirect 32 | github.com/hashicorp/errwrap v1.0.0 // indirect 33 | github.com/hashicorp/go-multierror v1.1.0 // indirect 34 | github.com/imdario/mergo v0.3.11 // indirect 35 | github.com/jmespath/go-jmespath v0.4.0 // indirect 36 | github.com/josharian/intern v1.0.0 // indirect 37 | github.com/json-iterator/go v1.1.12 // indirect 38 | github.com/mailru/easyjson v0.7.7 // indirect 39 | github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 // indirect 40 | github.com/mitchellh/go-homedir v1.1.0 // indirect 41 | github.com/moby/spdystream v0.2.0 // indirect 42 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect 43 | github.com/modern-go/reflect2 v1.0.2 // indirect 44 | github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect 45 | github.com/pmezard/go-difflib v1.0.0 // indirect 46 | github.com/pquerna/otp v1.2.0 // indirect 47 | github.com/russross/blackfriday/v2 v2.1.0 // indirect 48 | github.com/spf13/pflag v1.0.5 // indirect 49 | github.com/urfave/cli v1.22.2 // indirect 50 | golang.org/x/crypto v0.1.0 // indirect 51 | golang.org/x/net v0.8.0 // indirect 52 | golang.org/x/oauth2 v0.1.0 // indirect 53 | golang.org/x/sys v0.6.0 // indirect 54 | golang.org/x/term v0.6.0 // indirect 55 | golang.org/x/text v0.8.0 // indirect 56 | golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 // indirect 57 | google.golang.org/appengine v1.6.7 // indirect 58 | google.golang.org/protobuf v1.31.0 // indirect 59 | gopkg.in/inf.v0 v0.9.1 // indirect 60 | gopkg.in/yaml.v2 v2.4.0 // indirect 61 | k8s.io/api v0.27.2 // indirect 62 | k8s.io/client-go v0.27.2 // indirect 63 | k8s.io/klog/v2 v2.90.1 // indirect 64 | k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f // indirect 65 | k8s.io/utils v0.0.0-20230209194617-a36077c30491 // indirect 66 | sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect 67 | sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect 68 | sigs.k8s.io/yaml v1.3.0 // indirect 69 | ) 70 | -------------------------------------------------------------------------------- /tests/grafana_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "net/http" 7 | "os" 8 | "testing" 9 | 10 | _ "github.com/stretchr/testify/assert" 11 | "gopkg.in/yaml.v3" 12 | 13 | "github.com/gruntwork-io/terratest/modules/k8s" 14 | ) 15 | 16 | type GrafanaValues struct { 17 | GrafanaSource struct { 18 | Service struct { 19 | TargetPort int `yaml:"targetPort"` 20 | } `yaml:"service"` 21 | } `yaml:"grafana-source"` 22 | } 23 | 24 | func TestHelmGrafanaAddon(t *testing.T) { 25 | // add Istio CRDs to test Istio virtual service 26 | istioBaseAddonData := HelmAddonData{ 27 | namespaceName: "grafana-test", 28 | releaseName: "", 29 | dependencyRepo: "", 30 | addonName: "istio-base", 31 | addonAlias: "", 32 | chartPath: "../addons/istio/base", 33 | manageNamespace: true, 34 | overrideValues: map[string]string{ 35 | "global.istioNamespace": "grafana-test", 36 | }, 37 | } 38 | 39 | istioBaseHelmOptions, err := prepareHelmEnvironment(t, &istioBaseAddonData) 40 | 41 | if err != nil { 42 | log.Fatal(err) 43 | } 44 | 45 | grafanaNamespaceAndAddonName := "grafana" 46 | 47 | addonData := HelmAddonData{ 48 | namespaceName: "", 49 | releaseName: "", 50 | dependencyRepo: "", 51 | addonName: grafanaNamespaceAndAddonName, 52 | addonAlias: "", 53 | chartPath: "", 54 | hasCustomValues: true, 55 | manageNamespace: true, 56 | } 57 | 58 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 59 | 60 | if err != nil { 61 | log.Fatal(err) 62 | } 63 | 64 | waitUntilServicesAvailable(t, *helmOptions.KubectlOptions, []string{grafanaNamespaceAndAddonName}) 65 | waitUntilDeploymentsAvailable(t, *helmOptions.KubectlOptions, []string{grafanaNamespaceAndAddonName}) 66 | 67 | valuesFile, err := os.ReadFile("../addons/grafana/values.yaml") 68 | 69 | if err != nil { 70 | log.Fatal(err) 71 | } 72 | 73 | var yamlData GrafanaValues 74 | 75 | err = yaml.Unmarshal(valuesFile, &yamlData) 76 | 77 | targetPort := 3000 78 | if err == nil && yamlData.GrafanaSource.Service.TargetPort != 0 { 79 | targetPort = yamlData.GrafanaSource.Service.TargetPort 80 | } 81 | 82 | grafanaKubectlOptions := k8s.NewKubectlOptions("", "", grafanaNamespaceAndAddonName) 83 | 84 | tunnel := k8s.NewTunnel(grafanaKubectlOptions, k8s.ResourceTypeService, grafanaNamespaceAndAddonName, 0, targetPort) 85 | defer tunnel.Close() 86 | 87 | tunnel.ForwardPort(t) 88 | 89 | healthCheck, err := http.NewRequest("GET", fmt.Sprintf("http://%s/healthz", tunnel.Endpoint()), nil) 90 | if err != nil { 91 | log.Fatalf("Error when building the request: %s", err) 92 | } 93 | 94 | resp, reqErr := http.DefaultClient.Do(healthCheck) 95 | if reqErr != nil { 96 | log.Fatalf("Error when making the request: %s", reqErr) 97 | } 98 | 99 | if resp.StatusCode != http.StatusOK { 100 | log.Fatalf("Grafana is not ready for metrics: status code %d", resp.StatusCode) 101 | } else { 102 | log.Print("Grafana is ready for metrics!") 103 | } 104 | // ---------------------------------- 105 | 106 | destroyHelmEnvironment(t, addonData, helmOptions) 107 | destroyHelmEnvironment(t, istioBaseAddonData, istioBaseHelmOptions) 108 | } 109 | -------------------------------------------------------------------------------- /tests/kubernetes_replicator_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | "time" 7 | 8 | "github.com/gruntwork-io/terratest/modules/k8s" 9 | "github.com/stretchr/testify/assert" 10 | corev1 "k8s.io/api/core/v1" 11 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 12 | "sigs.k8s.io/yaml" 13 | ) 14 | 15 | func TestSecretReplication(t *testing.T) { 16 | 17 | // Helm deploy the kubernetes-replicator chart 18 | addonData := HelmAddonData{ 19 | namespaceName: "kubernetes-replicator", 20 | releaseName: "", 21 | dependencyRepo: "", 22 | addonName: "kubernetes-replicator", 23 | addonAlias: "", 24 | chartPath: "", 25 | hasCustomValues: true, 26 | manageNamespace: true, 27 | } 28 | kubectlOptions := k8s.NewKubectlOptions("", "", addonData.namespaceName) 29 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 30 | 31 | if err != nil { 32 | log.Fatal(err) 33 | } 34 | 35 | // Target Namespace for secret replication 36 | k8s.CreateNamespace(t, kubectlOptions, "initium-test1") 37 | k8s.CreateNamespace(t, kubectlOptions, "initium-test2") 38 | k8s.CreateNamespace(t, kubectlOptions, "initium") 39 | 40 | replicateToNamespaces := []string{"initium", "initium-test2", "initium-test1"} 41 | secretName := "secret-replicate" 42 | 43 | // Create a Secret with `replicator.v1.mittwald.de/replicate-to` annotation in default namespace 44 | secretData := map[string]string{ 45 | "project": "initium", 46 | } 47 | byteSecretData := make(map[string][]byte) 48 | for k, v := range secretData { 49 | byteSecretData[k] = []byte(v) 50 | } 51 | 52 | // Define the Secret object with the replicator annotation 53 | secret := &corev1.Secret{ 54 | TypeMeta: metav1.TypeMeta{ 55 | Kind: "Secret", 56 | APIVersion: "v1", 57 | }, 58 | ObjectMeta: metav1.ObjectMeta{ 59 | Name: secretName, 60 | Namespace: addonData.namespaceName, 61 | Annotations: map[string]string{ 62 | "replicator.v1.mittwald.de/replicate-to": "initium,initium-.+,initium.+", 63 | }, 64 | }, 65 | Data: byteSecretData, 66 | } 67 | 68 | // Convert the Secret object to YAML 69 | secretYAML, err := yaml.Marshal(secret) 70 | if err != nil { 71 | t.Fatalf("Failed to convert secret to YAML: %v", err) 72 | } 73 | 74 | // Create the Secret 75 | k8s.KubectlApplyFromString(t, kubectlOptions, string(secretYAML)) 76 | 77 | // Wait for 20 seconds to allow the operator to replicate the secret 78 | time.Sleep(20 * time.Second) 79 | 80 | // Check each namespace for the replicated secret 81 | for _, namespace := range replicateToNamespaces { 82 | // Update k8s options to the target namespace 83 | targetOptions := k8s.NewKubectlOptions("", "", namespace) 84 | 85 | // Try to get the replicated secret from the target namespace 86 | replicatedSecret, err := k8s.GetSecretE(t, targetOptions, secretName) 87 | 88 | // Check if the replicated Secret is not nil and has the correct data 89 | assert.NoError(t, err) 90 | assert.NotNil(t, replicatedSecret) 91 | assert.Equal(t, byteSecretData, replicatedSecret.Data) 92 | } 93 | 94 | // Cleanup 95 | error := k8s.KubectlDeleteFromStringE(t, kubectlOptions, string(secretYAML)) 96 | assert.NoError(t, error) 97 | 98 | destroyHelmEnvironment(t, addonData, helmOptions) 99 | k8s.DeleteNamespace(t, kubectlOptions, "initium-test1") 100 | k8s.DeleteNamespace(t, kubectlOptions, "initium-test2") 101 | k8s.DeleteNamespace(t, kubectlOptions, "initium") 102 | } 103 | -------------------------------------------------------------------------------- /tests/knative_istio_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "log" 5 | "testing" 6 | "time" 7 | 8 | "github.com/gruntwork-io/terratest/modules/k8s" 9 | ) 10 | 11 | func TestKnativeAndHelmIstioAddons(t *testing.T) { 12 | istioBaseAddonData := HelmAddonData{ 13 | namespaceName: "istio-system", 14 | releaseName: "", 15 | dependencyRepo: "", 16 | addonName: "istio-base", 17 | addonAlias: "", 18 | chartPath: "../addons/istio/base", 19 | manageNamespace: true, 20 | } 21 | 22 | istioBaseHelmOptions, err := prepareHelmEnvironment(t, &istioBaseAddonData) 23 | 24 | if err != nil { 25 | log.Fatal(err) 26 | } 27 | 28 | // ---------------------------------- 29 | 30 | istioDiscoveryAddonData := HelmAddonData{ 31 | namespaceName: "istio-system", 32 | releaseName: "", 33 | dependencyRepo: "", 34 | addonName: "istiod", 35 | addonAlias: "", 36 | chartPath: "../addons/istio/istiod", 37 | } 38 | 39 | istioDiscoveryHelmOptions, err := prepareHelmEnvironment(t, &istioDiscoveryAddonData) 40 | 41 | if err != nil { 42 | log.Fatal(err) 43 | } 44 | 45 | waitUntilServicesAvailable(t, *istioDiscoveryHelmOptions.KubectlOptions, []string{"istiod"}) 46 | 47 | // ---------------------------------- 48 | 49 | istioIngressAddonData := HelmAddonData{ 50 | namespaceName: "istio-ingress", 51 | releaseName: "", 52 | dependencyRepo: "", 53 | addonName: "istio-ingressgateway", 54 | addonAlias: "", 55 | chartPath: "../addons/istio/ingressgateway", 56 | manageNamespace: true, 57 | } 58 | 59 | istioIngressHelmOptions, err := prepareHelmEnvironment(t, &istioIngressAddonData) 60 | 61 | if err != nil { 62 | log.Fatal(err) 63 | } 64 | 65 | waitUntilLoadBalancerAvailable(t, *istioIngressHelmOptions.KubectlOptions) 66 | 67 | // ---------------------------------- 68 | operatorResourcePath := "../addons/knative/templates/operator.yaml" 69 | 70 | operatorOptions := k8s.NewKubectlOptions("", "", "default") 71 | 72 | k8s.KubectlApply(t, operatorOptions, operatorResourcePath) 73 | 74 | waitUntilServicesAvailable(t, *operatorOptions, []string{"operator-webhook"}) 75 | 76 | // ---------------------------------- 77 | 78 | servingResourcePath := "../addons/knative/templates/serving.yaml" 79 | 80 | servingOptions := k8s.NewKubectlOptions("", "", "knative-serving") 81 | 82 | k8s.KubectlApply(t, servingOptions, servingResourcePath) 83 | 84 | waitUntilServicesAvailable(t, *servingOptions, []string{ 85 | "activator-service", 86 | "autoscaler", 87 | "controller", 88 | "webhook", 89 | }) 90 | 91 | // ---------------------------------- 92 | 93 | eventingResourcePath := "../addons/knative/templates/eventing.yaml" 94 | 95 | eventingOptions := k8s.NewKubectlOptions("", "", "knative-eventing") 96 | 97 | k8s.KubectlApply(t, eventingOptions, eventingResourcePath) 98 | 99 | waitUntilServicesAvailable(t, *eventingOptions, []string{ 100 | "broker-filter", 101 | "broker-ingress", 102 | "eventing-webhook", 103 | "imc-dispatcher", 104 | "inmemorychannel-webhook", 105 | }) 106 | 107 | // ---------------------------------- 108 | 109 | time.Sleep(90 * time.Second) 110 | 111 | k8s.KubectlDelete(t, eventingOptions, eventingResourcePath) 112 | k8s.KubectlDelete(t, servingOptions, servingResourcePath) 113 | k8s.KubectlDelete(t, operatorOptions, operatorResourcePath) 114 | 115 | destroyHelmEnvironment(t, istioIngressAddonData, istioIngressHelmOptions) 116 | destroyHelmEnvironment(t, istioDiscoveryAddonData, istioDiscoveryHelmOptions) 117 | destroyHelmEnvironment(t, istioBaseAddonData, istioBaseHelmOptions) 118 | } 119 | -------------------------------------------------------------------------------- /tests/loki_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "io/ioutil" 7 | "log" 8 | "net/http" 9 | "net/url" 10 | "os" 11 | "testing" 12 | "time" 13 | 14 | "github.com/gruntwork-io/terratest/modules/k8s" 15 | 16 | _ "github.com/stretchr/testify/assert" 17 | "gopkg.in/yaml.v3" 18 | ) 19 | 20 | type LokiValues struct { 21 | LokiSource struct { 22 | Loki struct { 23 | Server struct { 24 | HttpListenPort int `yaml:"http_listen_port"` 25 | } `yaml:"server"` 26 | } `yaml:"loki"` 27 | } `yaml:"loki-source"` 28 | } 29 | 30 | type LokiAPIResponse struct { 31 | Status string `json:"status"` 32 | Data struct { 33 | Result []struct { 34 | Value []interface{} `json:"value"` 35 | } `json:"result"` 36 | } `json:"data"` 37 | } 38 | 39 | func TestHelmLokiAddon(t *testing.T) { 40 | lokiNamespaceAndAddonName := "loki" 41 | addonData := HelmAddonData{ 42 | namespaceName: "", 43 | releaseName: "", 44 | dependencyRepo: "", 45 | addonName: lokiNamespaceAndAddonName, 46 | addonAlias: "", 47 | chartPath: "", 48 | hasCustomValues: true, 49 | manageNamespace: true, 50 | } 51 | 52 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 53 | 54 | if err != nil { 55 | log.Fatal(err) 56 | } 57 | 58 | waitUntilServicesAvailable(t, *helmOptions.KubectlOptions, []string{ 59 | lokiNamespaceAndAddonName, 60 | fmt.Sprintf("%s-headless", lokiNamespaceAndAddonName), 61 | fmt.Sprintf("%s-memberlist", lokiNamespaceAndAddonName), 62 | }) 63 | 64 | statefulSetAvailable := waitUntilStatefulSetsAvailable(t, *helmOptions.KubectlOptions, []string{lokiNamespaceAndAddonName}) 65 | if !statefulSetAvailable { 66 | log.Fatalf("Error waiting for StatefulSet %s to become ready.", lokiNamespaceAndAddonName) 67 | } 68 | 69 | // Wait for the applications to send logs to Loki 70 | time.Sleep(120 * time.Second) 71 | 72 | valuesFile, err := os.ReadFile("../addons/loki/values.yaml") 73 | 74 | if err != nil { 75 | log.Fatal(err) 76 | } 77 | 78 | var yamlData LokiValues 79 | 80 | err = yaml.Unmarshal(valuesFile, &yamlData) 81 | 82 | targetPort := 3100 83 | if err == nil && yamlData.LokiSource.Loki.Server.HttpListenPort != 0 { 84 | targetPort = yamlData.LokiSource.Loki.Server.HttpListenPort 85 | } 86 | 87 | tunnel := k8s.NewTunnel(helmOptions.KubectlOptions, k8s.ResourceTypeService, lokiNamespaceAndAddonName, 0, targetPort) 88 | defer tunnel.Close() 89 | 90 | tunnel.ForwardPort(t) 91 | 92 | healthCheck, err := http.NewRequest("GET", fmt.Sprintf("http://%s/loki/api/v1/query?query=%s", tunnel.Endpoint(), url.QueryEscape("sum by (msg) (rate({container_name=~\".+\"}[5m]))")), nil) 93 | if err != nil { 94 | log.Fatalf("Error when building the request: %s", err) 95 | } 96 | 97 | resp, reqErr := http.DefaultClient.Do(healthCheck) 98 | if reqErr != nil { 99 | log.Fatalf("Error when making the request: %s", reqErr) 100 | } 101 | 102 | respDataFromHttp, err := ioutil.ReadAll(resp.Body) 103 | if err != nil { 104 | log.Fatalf("Error converting response into JSON: %s", err) 105 | } 106 | 107 | var jsonResp LokiAPIResponse 108 | err = json.Unmarshal([]byte(respDataFromHttp), &jsonResp) 109 | if err != nil { 110 | log.Fatalf("Error when unmarshalling response: %s", err) 111 | } 112 | 113 | log.Print(jsonResp) 114 | if jsonResp.Status != "success" { 115 | log.Fatalf("Error when querying the Loki API. Response: %s.", jsonResp) 116 | } else if jsonResp.Status != "error" { 117 | log.Printf("Loki is ready. API response: %s!", jsonResp.Data.Result) 118 | } 119 | // ---------------------------------- 120 | 121 | destroyHelmEnvironment(t, addonData, helmOptions) 122 | } 123 | -------------------------------------------------------------------------------- /docs/SAMPLE_DEPLOY_APP.md: -------------------------------------------------------------------------------- 1 | # Deploying a Sample App on our Kubernetes cluster with Knative 2 | 3 | We will walk through the steps of deploying a sample application using Knative on Kubernetes cluster where different addons are installed and the cluster is ready for serving traffic. 4 | 5 | ## Prerequisites 6 | 7 | - Our type of Kubernetes cluster with all addons should be installed as it is described [here](https://github.com/nearform/initium-platform#readme) 8 | - The kubectl command-line tool for interacting with the Kubernetes cluster. 9 | - Already configured `kubectl` context for your CLI client. If the Kubernetes cluster is installed with the `make` commands from the description [here](https://github.com/nearform/initium-platform#readme), then the context setup question is already solved. 10 | - (Optional) Knative CLI for interacting with your Knative installation. 11 | 12 | ## Step 1: Deploy a sample app with Knative 13 | 14 | We have a very simple Golang application where the output is just a simple string. With only one single command that app will be deployed on the cluster. 15 | 16 | ```bash 17 | kubectl apply -f examples/sample-deploy-app/sample-deploy-app.yaml 18 | ``` 19 | 20 | The `sample-deploy-app.yaml` file defines the configuration for the Knative Service, then it specifies the Docker image for the app, an ENV variable and a container port. 21 | 22 | Wait for the Knative Service to be created and for the app to be deployed, the following command will help you with the first verification: 23 | 24 | ```bash 25 | kubectl get ksvc 26 | ``` 27 | 28 | You should see the sample app listed with a Ready status. 29 | 30 | ## Step 2: Test the App 31 | 32 | To test the app response, you can run a simple `curl` command against the deployed 'Hello World' service: 33 | 34 | Depending on the OS type that you are using, the `INITIUM_LB_ENDPOINT` variable is exported differently, for Linux users we have: 35 | 36 | ```bash 37 | export INITIUM_LB_ENDPOINT="$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{(index .status.loadBalancer.ingress 0).ip}}'):80" 38 | ``` 39 | 40 | For the MacOS users we have (accessing the Kind cluster is different for MacOS and Windos and it is described [here](https://kind.sigs.k8s.io/docs/user/loadbalancer/)): 41 | 42 | ```bash 43 | export PROXY_HTTP_CONTAINER_NAME="istio-lb-proxy-80" 44 | export INITIUM_LB_ENDPOINT="127.0.0.1:80" 45 | export INITIUM_LB_INT_HTTP_PORT=$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{range .spec.ports}}{{if (eq .port 80)}}{{.nodePort}}{{end}}{{end}}') 46 | export INITIUM_REPO_NAME="initium-platform" 47 | docker run \ 48 | -d \ 49 | --restart always \ 50 | --name $PROXY_HTTP_CONTAINER_NAME \ 51 | --publish 127.0.0.1:80:$INITIUM_LB_INT_HTTP_PORT \ 52 | --link ${INITIUM_REPO_NAME}-control-plane:target \ 53 | --network kind \ 54 | alpine/socat -dd tcp-listen:$INITIUM_LB_INT_HTTP_PORT,fork,reuseaddr tcp-connect:target:$INITIUM_LB_INT_HTTP_PORT 55 | ``` 56 | 57 | Then you need to export one extra variable and the `curl` command is ready to use: 58 | 59 | ```bash 60 | export SAMPLE_APP_URL=$(kubectl get ksvc -o json | jq -r '.items[] | select(.metadata.name == "helloworld") | .status.url' | sed 's#http://##') 61 | curl -H "Host: $SAMPLE_APP_URL" "http://$INITIUM_LB_ENDPOINT" 62 | ``` 63 | 64 | The responded string of 'Hello World' is enough for verification that the application is successfully deployed on the Kubernetes cluster. 65 | 66 | ## Step 3: Clean Up 67 | 68 | To delete the Knative Service and the running app, run the following command: 69 | 70 | ```bash 71 | kubectl delete -f examples/sample-deploy-app/sample-deploy-app.yaml 72 | ``` 73 | 74 | This will delete the Knative Service and all the resources associated with it, including the app pods. 75 | -------------------------------------------------------------------------------- /tests/prometheus_addon_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "io/ioutil" 7 | "log" 8 | "net/http" 9 | "os" 10 | "testing" 11 | 12 | "github.com/gruntwork-io/terratest/modules/k8s" 13 | _ "github.com/stretchr/testify/assert" 14 | "gopkg.in/yaml.v3" 15 | ) 16 | 17 | type PrometheusValues struct { 18 | KubePrometheusStackSource struct { 19 | Prometheus struct { 20 | Service struct { 21 | Port int `yaml:"port"` 22 | } `yaml:"service"` 23 | } `yaml:"prometheus"` 24 | } `yaml:"kube-prometheus-stack-source"` 25 | } 26 | 27 | type PrometheusAPIResponse struct { 28 | Status string `json:"status"` 29 | Data struct { 30 | Result []struct { 31 | Value []interface{} `json:"value"` 32 | } `json:"result"` 33 | } `json:"data"` 34 | } 35 | 36 | func TestHelmPrometheusServerAddon(t *testing.T) { 37 | addonData := HelmAddonData{ 38 | namespaceName: "monitoring", 39 | releaseName: "", 40 | dependencyRepo: "", 41 | addonName: "kube-prometheus-stack", 42 | addonAlias: "", 43 | chartPath: "", 44 | hasCustomValues: true, 45 | manageNamespace: true, 46 | } 47 | 48 | // add Grafana namespace as it is expected by Prometheus stack 49 | kubectlOptions := k8s.NewKubectlOptions("", "", addonData.namespaceName) 50 | k8s.CreateNamespace(t, kubectlOptions, "grafana") 51 | 52 | helmOptions, err := prepareHelmEnvironment(t, &addonData) 53 | 54 | if err != nil { 55 | log.Fatal(err) 56 | } 57 | 58 | waitUntilHelmFormattedServicesAvailable(t, addonData, helmOptions, []string{ 59 | "kube-state-metrics", 60 | "prometheus-node-exporter", 61 | }) 62 | 63 | prometheusServiceName := "kube-prometheus-stack-test-prometheus" 64 | waitUntilServicesAvailable(t, *helmOptions.KubectlOptions, []string{ 65 | "kube-prometheus-stack-test-alertmanager", 66 | "kube-prometheus-stack-test-operator", 67 | prometheusServiceName, 68 | }) 69 | 70 | prometheusStatefulSetName := fmt.Sprintf("prometheus-%s", prometheusServiceName) 71 | statefulSetAvailable := waitUntilStatefulSetsAvailable(t, *kubectlOptions, []string{prometheusStatefulSetName}) 72 | if !statefulSetAvailable { 73 | log.Fatalf("Error waiting for StatefulSet %s to become ready.", prometheusStatefulSetName) 74 | } 75 | 76 | valuesFile, err := os.ReadFile("../addons/kube-prometheus-stack/values.yaml") 77 | 78 | if err != nil { 79 | log.Fatal(err) 80 | } 81 | 82 | var yamlData PrometheusValues 83 | 84 | err = yaml.Unmarshal(valuesFile, &yamlData) 85 | 86 | targetPort := 9090 87 | if err == nil && yamlData.KubePrometheusStackSource.Prometheus.Service.Port != 0 { 88 | targetPort = yamlData.KubePrometheusStackSource.Prometheus.Service.Port 89 | } 90 | 91 | tunnel := k8s.NewTunnel(kubectlOptions, k8s.ResourceTypeService, prometheusServiceName, 0, targetPort) 92 | defer tunnel.Close() 93 | 94 | tunnel.ForwardPort(t) 95 | 96 | metricCheck, err := http.NewRequest("GET", fmt.Sprintf("http://%s/api/v1/query?query=prometheus_tsdb_head_series", tunnel.Endpoint()), nil) 97 | if err != nil { 98 | log.Fatalf("Error when building the request: %s", err) 99 | } 100 | 101 | resp, reqErr := http.DefaultClient.Do(metricCheck) 102 | if reqErr != nil { 103 | log.Fatalf("Error when making the request: %s", reqErr) 104 | } 105 | 106 | respDataFromHttp, err := ioutil.ReadAll(resp.Body) 107 | if err != nil { 108 | log.Fatalf("Error converting response into JSON: %s", err) 109 | } 110 | 111 | var jsonResp PrometheusAPIResponse 112 | err = json.Unmarshal([]byte(respDataFromHttp), &jsonResp) 113 | if err != nil { 114 | log.Fatalf("Error when unmarshalling response: %s", err) 115 | } 116 | 117 | if jsonResp.Status != "success" || jsonResp.Data.Result[0].Value[1].(string) == "0" { 118 | log.Fatalf("Error when querying the Prometheus API. status %s, result %s", jsonResp.Status, jsonResp.Data.Result) 119 | } else if jsonResp.Status != "error" { 120 | log.Printf("Prometheus has received %s metrics!", jsonResp.Data.Result[0].Value[1].(string)) 121 | } 122 | 123 | // ---------------------------------- 124 | 125 | destroyHelmEnvironment(t, addonData, helmOptions) 126 | k8s.DeleteNamespace(t, kubectlOptions, "grafana") 127 | } 128 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## [0.1.0](https://github.com/nearform/k8s-kurated-addons/compare/v0.0.1...v0.1.0) (2023-03-23) 4 | 5 | 6 | ### Features 7 | 8 | * Add Dex addon ([#38](https://github.com/nearform/k8s-kurated-addons/issues/38)) ([62966d4](https://github.com/nearform/k8s-kurated-addons/commit/62966d410f7d119e24ef6e012ffd1f85f4974126)) 9 | * Add Grafana Loki ([#50](https://github.com/nearform/k8s-kurated-addons/issues/50)) ([087b319](https://github.com/nearform/k8s-kurated-addons/commit/087b3195d9d24f23844716f25ecb572b7b501d90)) 10 | * Add OpenTelemetry addon ([#33](https://github.com/nearform/k8s-kurated-addons/issues/33)) ([99b2dea](https://github.com/nearform/k8s-kurated-addons/commit/99b2dea139cf377855e32feb82d4defd809b7988)) 11 | * argocd improvements ([#85](https://github.com/nearform/k8s-kurated-addons/issues/85)) ([b1c8cca](https://github.com/nearform/k8s-kurated-addons/commit/b1c8ccaa43da5bbc73a1f05cbbebee940ce2b77f)) 12 | * Configurable sync policy for ArgoCD ([#56](https://github.com/nearform/k8s-kurated-addons/issues/56)) ([f2730e8](https://github.com/nearform/k8s-kurated-addons/commit/f2730e8d2c1387374146d2f695e18eba3efd2b8f)) 13 | * create github k8s service account ([#77](https://github.com/nearform/k8s-kurated-addons/issues/77)) ([74ab411](https://github.com/nearform/k8s-kurated-addons/commit/74ab411d5872df076a8ea33fad15e75e20b61180)) 14 | * Inject helmValues from bootstrap app to addons ([#53](https://github.com/nearform/k8s-kurated-addons/issues/53)) ([9818fc7](https://github.com/nearform/k8s-kurated-addons/commit/9818fc7eb760b5c7e4a8d4ddcd5aa1a4107fe2bb)) 15 | * knative improve deployment ([#107](https://github.com/nearform/k8s-kurated-addons/issues/107)) ([2120882](https://github.com/nearform/k8s-kurated-addons/commit/212088215d1e7651bd8cf12a98921f05ecd1b848)) 16 | * Move app-of-apps manifest to helm-chart ([#31](https://github.com/nearform/k8s-kurated-addons/issues/31)) ([5a6db64](https://github.com/nearform/k8s-kurated-addons/commit/5a6db64a5bbf6d9d88086d6d4c3eddb0df278445)) 17 | * remove envsubst requirement ([#87](https://github.com/nearform/k8s-kurated-addons/issues/87)) ([8984f9c](https://github.com/nearform/k8s-kurated-addons/commit/8984f9c4b8e2fb6cbe7ada8feb3624ece9141dd1)) 18 | * remove kubernetes 1.23 from github actions matrix ([#61](https://github.com/nearform/k8s-kurated-addons/issues/61)) ([b52c5cd](https://github.com/nearform/k8s-kurated-addons/commit/b52c5cde1dfce35c1948a80133474def6c12fb54)) 19 | * Sample Grafana dashboard ([#8](https://github.com/nearform/k8s-kurated-addons/issues/8)) ([1bb35e2](https://github.com/nearform/k8s-kurated-addons/commit/1bb35e20744da739dc36b383688f65acd8c9176f)) 20 | 21 | 22 | ### Bug Fixes 23 | 24 | * change renovate commit prefix ([#60](https://github.com/nearform/k8s-kurated-addons/issues/60)) ([672e4d0](https://github.com/nearform/k8s-kurated-addons/commit/672e4d0a22294e05d6f2ae258bd2fdfc3e03b9bb)) 25 | * comment back dex default values ([#63](https://github.com/nearform/k8s-kurated-addons/issues/63)) ([b89546d](https://github.com/nearform/k8s-kurated-addons/commit/b89546decb0d602395d1ecb54ba82483601d01a7)) 26 | * Downgrade ArgoCD traffic to HTTP ([#30](https://github.com/nearform/k8s-kurated-addons/issues/30)) ([1a5b1c9](https://github.com/nearform/k8s-kurated-addons/commit/1a5b1c9ba88788354afc1f692699afbc77afb92a)) 27 | * inject values to addons from bootstrap app + revert syncPolicy templating ([#99](https://github.com/nearform/k8s-kurated-addons/issues/99)) ([c478692](https://github.com/nearform/k8s-kurated-addons/commit/c478692788fde4b72d4e785c6b1d5bca1dcddd53)) 28 | * remove the update changelog entry ([#86](https://github.com/nearform/k8s-kurated-addons/issues/86)) ([57da682](https://github.com/nearform/k8s-kurated-addons/commit/57da6822ff0eae24a52455bb431ed2e478536f81)) 29 | * replace readarray ([#92](https://github.com/nearform/k8s-kurated-addons/issues/92)) ([683bc44](https://github.com/nearform/k8s-kurated-addons/commit/683bc44582f07d4c3fc8456e3189fc0a1f46d1c7)) 30 | * updating default dex config ([#101](https://github.com/nearform/k8s-kurated-addons/issues/101)) ([26ae141](https://github.com/nearform/k8s-kurated-addons/commit/26ae1412f93a860d36a262a5d6f94feeb18f1581)) 31 | * use POSIX compatible comparison operator ([#89](https://github.com/nearform/k8s-kurated-addons/issues/89)) ([231c3cc](https://github.com/nearform/k8s-kurated-addons/commit/231c3ccc687f244903a46f681f10ba078ca2ffc0)) 32 | 33 | ## Changelog 34 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | ############################################################################### 2 | # ENVIRONMENT CONFIGURATION 3 | ############################################################################### 4 | MAKEFLAGS += --no-print-directory 5 | SHELL=/bin/bash 6 | 7 | .PHONY: default 8 | 9 | ############################################################################### 10 | # GOALS ( safe defaults ) 11 | ############################################################################### 12 | 13 | default: generate-certs kind-up tilt-up 14 | 15 | clean: tilt-down kind-down 16 | @./scripts/clean-test.sh 17 | 18 | ci: generate-certs kind-up 19 | @./scripts/ci.sh 20 | 21 | help: 22 | @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' 23 | 24 | argocd: 25 | @./scripts/cloud.sh 26 | 27 | ############################################################################### 28 | # GOALS ( bootstrap) 29 | ############################################################################### 30 | 31 | plugin_install: ## install asdf plugins 32 | @asdf plugin add argocd https://github.com/beardix/asdf-argocd.git || true 33 | @asdf plugin add awscli https://github.com/MetricMike/asdf-awscli.git || true 34 | @asdf plugin add eksctl https://github.com/elementalvoid/asdf-eksctl.git || true 35 | @asdf plugin add golang https://github.com/kennyp/asdf-golang.git || true 36 | @asdf plugin add golangci-lint https://github.com/hypnoglow/asdf-golangci-lint.git || true 37 | @asdf plugin add jq https://github.com/AZMCode/asdf-jq.git || true 38 | @asdf plugin add helm https://github.com/Antiarchitect/asdf-helm.git || true 39 | @asdf plugin add kind https://github.com/reegnz/asdf-kind.git || true 40 | @asdf plugin add kubectl https://github.com/asdf-community/asdf-kubectl.git || true 41 | @asdf plugin add pre-commit https://github.com/jonathanmorley/asdf-pre-commit.git || true 42 | @asdf plugin add tilt https://github.com/virtualstaticvoid/asdf-tilt.git || true 43 | @asdf plugin add gcloud https://github.com/jthegedus/asdf-gcloud.git || true 44 | @asdf plugin add initium https://github.com/nearform/asdf-initium.git || true 45 | 46 | 47 | plugin_uninstall: ## uninstall asdf plugins 48 | @asdf plugin remove argocd || true 49 | @asdf plugin remove awscli || true 50 | @asdf plugin remove eksctl || true 51 | @asdf plugin remove golang || true 52 | @asdf plugin remove golangci-lint || true 53 | @asdf plugin remove jq || true 54 | @asdf plugin remove helm || true 55 | @asdf plugin remove kind || true 56 | @asdf plugin remove kubectl || true 57 | @asdf plugin remove pre-commit || true 58 | @asdf plugin remove tilt || true 59 | @asdf plugin remove gcloud || true 60 | @asdf plugin remove initium || true 61 | 62 | asdf_install: plugin_install ## install all plugins and packages present in .tool-versions file 63 | @asdf install 64 | 65 | asdf_uninstall: plugin_uninstall ## uninstall all plugins and packages present in .tool-versions file 66 | 67 | pre-commit: ## set up the git hook scripts 68 | @pre-commit install 69 | 70 | bootstrap: asdf_install pre-commit ## setup all the needed plugins and install pre-commit hook 71 | 72 | ############################################################################### 73 | # GOALS 74 | ############################################################################### 75 | 76 | generate-certs: ## Generate TLS assets for to be used by cluster API server and istio ingress gateway 77 | @./scripts/generate-certs.sh 78 | 79 | kind-up: ## Create the K8s cluster 80 | @./scripts/kind-up.sh 81 | 82 | kind-down: ## Destroy the K8s cluster 83 | @./scripts/kind-down.sh 84 | 85 | tilt-up: ## Start Tilt and deploy all the apps 86 | @./scripts/tilt-up.sh 87 | 88 | tilt-down: ## Stop Tilt and destroy all the apps 89 | @./scripts/tilt-down.sh 90 | 91 | tilt-validate: ## Validate the Tiltfile 92 | @./scripts/tilt-validate.sh 93 | 94 | unit-test: ## Run unit tests 95 | @./scripts/run-test.sh unit 96 | 97 | integration-test: ## Run integration tests 98 | @./scripts/run-test.sh integration examples/sample-app 99 | 100 | validate: ## Run static checks 101 | @ASDF_DEFAULT_TOOL_VERSIONS_FILENAME=$(CURDIR)/.tool-versions pre-commit run --color=always --show-diff-on-failure --all-files 102 | 103 | create-ci-service-account: ## Create a k8s service account that would be used by CI systems 104 | @./scripts/create-ci-service-account.sh 105 | -------------------------------------------------------------------------------- /docs/SAMPLE_PROMETHEUS_APP.md: -------------------------------------------------------------------------------- 1 | # Deploying a Sample App with sample Prometheus metrics 2 | 3 | This application will be deployed with `knative` same as the other sample application where the whole process is described [here](https://github.com/nearform/initium-platform/blob/main/docs/SAMPLE_DEPLOY_APP.md). 4 | 5 | For exposing the application in Prometheus a `PodMonitor` CRD is used, [here](https://github.com/prometheus-operator/prometheus-operator#customresourcedefinitions) you can read more about the Prometheus CustomResourceDefinitions. 6 | 7 | ## Prerequisites 8 | 9 | - Our type of Kubernetes cluster with all addons should be installed as it is described [here](https://github.com/nearform/initium-platform#readme) 10 | - The kubectl command-line tool for interacting with the Kubernetes cluster. 11 | - Already configured `kubectl` context for your CLI client. If the Kubernetes cluster is installed with the `make` commands from the description [here](https://github.com/nearform/initium-platform#readme), then the context setup question is already solved. 12 | 13 | ## Step 1: Deploy a sample app with Knative 14 | 15 | This is a sample Golang application where the sample Prometheus metrics are exported using the official Prometheus [client](https://github.com/prometheus/client_golang). 16 | 17 | ```bash 18 | kubectl apply -f examples/sample-prometheus-app/sample-prometheus-app.yaml 19 | ``` 20 | 21 | ## Step 2: Setup verification 22 | 23 | First, we need to get the service endpoint, then let's make a few `curl` requests with the `curl` command mentioned below. 24 | 25 | Depending on the OS type that you are using, the `INITIUM_LB_ENDPOINT` variable is exported differently, for Linux users we have: 26 | 27 | ```bash 28 | export INITIUM_LB_ENDPOINT="$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{(index .status.loadBalancer.ingress 0).ip}}'):80" 29 | ``` 30 | 31 | For the MacOS users we have (accessing the Kind cluster is different for MacOS and Windos and it is described [here](https://kind.sigs.k8s.io/docs/user/loadbalancer/)): 32 | 33 | ```bash 34 | export PROXY_HTTP_CONTAINER_NAME="istio-lb-proxy-80" 35 | export INITIUM_LB_ENDPOINT="127.0.0.1:80" 36 | export INITIUM_LB_INT_HTTP_PORT=$(kubectl get service -n istio-ingress istio-ingressgateway -o go-template='{{range .spec.ports}}{{if (eq .port 80)}}{{.nodePort}}{{end}}{{end}}') 37 | export INITIUM_REPO_NAME="initium-platform" 38 | docker run \ 39 | -d \ 40 | --restart always \ 41 | --name $PROXY_HTTP_CONTAINER_NAME \ 42 | --publish 127.0.0.1:80:$INITIUM_LB_INT_HTTP_PORT \ 43 | --link ${INITIUM_REPO_NAME}-control-plane:target \ 44 | --network kind \ 45 | alpine/socat -dd tcp-listen:$INITIUM_LB_INT_HTTP_PORT,fork,reuseaddr tcp-connect:target:$INITIUM_LB_INT_HTTP_PORT 46 | ``` 47 | 48 | Then you need to export one extra variable and the `curl` command is ready to use: 49 | 50 | ```bash 51 | export SAMPLE_APP_URL=$(kubectl -n prometheus get ksvc -o json | jq -r '.items[] | select(.metadata.name == "sample-prometheus-app") | .status.url' | sed 's#http://##') 52 | curl -H "Host: $SAMPLE_APP_URL" "http://$INITIUM_LB_ENDPOINT" 53 | ``` 54 | 55 | Let's open the Prometheus targets for our internal Prometheus Stack. First, we need to expose the service using the `port-forward` feature: 56 | 57 | ```bash 58 | kubectl -n prometheus port-forward prometheus-kube-prometheus-stack-kube-prometheus-0 9090 59 | ``` 60 | 61 | Then just open http://127.0.0.1:9090/targets in your browser and a new target should be listed. 62 | 63 | ## Step 3: Grafana 64 | 65 | The manifest from where the Prometheus sample app is deployed contains a Grafana sample dashboard as well. For verifying if the metrics from the sample Prometheus app are correctly fetched in Grafana just expose the Grafana service locally and open the dashboard in browser. 66 | 67 | ```bash 68 | export GRAFANA_POD=$(kubectl -n prometheus get pods -l "app.kubernetes.io/name=grafana" -o json | jq -r '.items[].metadata.name') 69 | kubectl -n prometheus port-forward $GRAFANA_POD 3000 70 | ``` 71 | 72 | Get the Grafana's admin username and password: 73 | 74 | ```bash 75 | export GRAFANA_SECRET=$(kubectl -n prometheus get secrets -l "app.kubernetes.io/name=grafana" -o json | jq -r '.items[].metadata.name') 76 | kubectl -n prometheus get secrets kube-prometheus-stack-grafana -o jsonpath='{.data.admin-user}' | base64 -d 77 | kubectl -n prometheus get secrets kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d 78 | ``` 79 | 80 | Open http://127.0.0.1:3000 in brower and search for a dashboard named as `Sample Prometheus App`. 81 | 82 | ## Step 4: Clean Up 83 | 84 | To delete the deployed Sample Prometheus app just execute the following command: 85 | 86 | ```bash 87 | kubectl delete -f examples/sample-prometheus-app/sample-prometheus-app.yaml 88 | ``` 89 | -------------------------------------------------------------------------------- /Tiltfile: -------------------------------------------------------------------------------- 1 | # ===== Sanity check ===== 2 | for env in ['INITIUM_REPO_NAME', 'INITIUM_REPO_HOST_PATH', 'INITIUM_REPO_NODE_PATH', 'INITIUM_REPO_URI', 'INITIUM_REPO_BRANCH']: 3 | if os.getenv(env, '') == '': fail('Missing or empty {} env var. Did you run this project using the Makefile?'.format(env)) 4 | 5 | def parse_excluded_env_vars(prefix='INITIUM_AOA_EXCLUDE_'): 6 | mapping = { 7 | 'knative': [ 8 | 'knative-operator', 9 | 'knative-serving', 10 | 'knative-eventing' 11 | ], 12 | 'prometheusstack': [ 13 | 'kube-prometheus-stack' 14 | ] 15 | } 16 | 17 | excluded = [ 18 | var.removeprefix(prefix).lower().replace('_', '-') 19 | for var in os.environ.keys() 20 | if var.startswith(prefix) and os.getenv(var) == 'true' 21 | ] 22 | 23 | values = [] 24 | for e in excluded: 25 | values.extend(mapping.get(e, [e])) 26 | 27 | return ['apps.%s.excluded=true' % app for app in values ] 28 | 29 | # ===== Internal variables ===== 30 | ARGOCD_EXTERNAL_PORT = os.getenv('INITIUM_ARGOCD_EXTERNAL_PORT', 8080) 31 | ISTIO_HTTP_PORT = os.getenv('INITIUM_ISTIO_HTTP_PORT', 7080) 32 | ISTIO_HTTPS_PORT = os.getenv('INITIUM_ISTIO_HTTPS_PORT', 7443) 33 | 34 | # ===== Extensions ===== 35 | load('ext://namespace', 'namespace_yaml') 36 | 37 | # ===== Kubernetes deployment ===== 38 | 39 | def bootstrap_app_values(): 40 | VALUES_OVERRIDES='./manifests/bootstrap/overrides.local.yaml' 41 | valueFiles = [VALUES_OVERRIDES] if os.path.exists(VALUES_OVERRIDES) else [] 42 | values = ['repoURL=%s' % os.getenv('INITIUM_REPO_URI')] 43 | values += parse_excluded_env_vars() 44 | return valueFiles, values 45 | 46 | 47 | ## MetalLB 48 | local('helm dependency update ./utils/metallb') 49 | k8s_yaml(namespace_yaml('metallb-system'), allow_duplicates=False) 50 | k8s_yaml(helm( 51 | './utils/metallb', 52 | name='metallb', 53 | namespace='metallb-system', 54 | set=['cidrBlock="{}"'.format(os.getenv('INITIUM_METALLB_CIDR'))] 55 | )) 56 | 57 | if os.getenv('INITIUM_DEPLOY_MINIMAL', 'false') == 'false': 58 | ## Git HTTP Backend 59 | docker_build('initium-platform/git-http-backend', './utils/git-http-backend/docker') 60 | k8s_yaml(namespace_yaml('argocd'), allow_duplicates=False) 61 | k8s_yaml(helm( 62 | './utils/git-http-backend/chart', 63 | name='git-http-backend', 64 | namespace='argocd', 65 | set=['volumes.git_volume.path={}'.format(os.getenv('INITIUM_REPO_NODE_PATH'))] 66 | )) 67 | 68 | ## k8s secret with TLS cert for wildcard.kube.local domains 69 | k8s_yaml(namespace_yaml('istio-ingress'), allow_duplicates=False) 70 | k8s_resource( 71 | objects=['istio-ingress:namespace'], 72 | new_name='istio-ingress-namespace', 73 | ) 74 | local_resource( 75 | 'wildcard.kube.local-tls-secret', 76 | cmd='kubectl create secret tls -n istio-ingress wildcard.kube.local-tls --save-config \ 77 | --dry-run=client --cert=.ssl/cert-ingress-gateway.pem --key=.ssl/key-ingress-gateway.pem -o yaml | kubectl apply -f -', 78 | auto_init=True, 79 | resource_deps=['istio-ingress-namespace'] 80 | ) 81 | 82 | ## ArgoCD 83 | local('helm dependency update ./addons/argocd') 84 | k8s_yaml(helm( 85 | './addons/argocd', 86 | name='argocd', 87 | namespace='argocd', 88 | )) 89 | 90 | ## Deploy demo secret for replication 91 | k8s_yaml('manifests/demo-app-config.yaml') 92 | 93 | ## App-of-apps 94 | valueFiles, values = bootstrap_app_values() 95 | k8s_yaml(helm('./manifests/bootstrap', namespace="argocd", name="app-of-apps", values=valueFiles, set=values)) 96 | 97 | k8s_resource( 98 | objects=['initium-platform:Application:argocd'], 99 | new_name='initium-platform', 100 | resource_deps=['argocd-redis', 'argocd-server', 'argocd-repo-server', 'metallb-metallb-source-controller', 'metallb-metallb-source-speaker', 'git-http-backend'] 101 | ) 102 | 103 | # ===== Tilt local resources ===== 104 | 105 | local_resource( 106 | 'grafana-password', 107 | cmd='kubectl get secret -n grafana grafana-initial-admin-secret -o jsonpath="{.data.admin-password}" | base64 -d; echo', 108 | auto_init=False 109 | ) 110 | 111 | ## ArgoCD admin password 112 | local_resource( 113 | 'argocd-password', 114 | cmd='kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo', 115 | auto_init=False 116 | ) 117 | 118 | ## ArgoCD HTTPS port 119 | local_resource( 120 | 'argocd-portforward', 121 | serve_cmd='kubectl port-forward -n argocd svc/argocd-server {}:80'.format(ARGOCD_EXTERNAL_PORT), 122 | links=['http://localhost:{}'.format(ARGOCD_EXTERNAL_PORT)], 123 | readiness_probe=probe( 124 | initial_delay_secs = 20, 125 | timeout_secs = 5, 126 | period_secs = 10, 127 | success_threshold = 1, 128 | failure_threshold = 5, 129 | http_get=http_get_action(port=int(ARGOCD_EXTERNAL_PORT))), 130 | auto_init=False 131 | ) 132 | 133 | ## Istio Ingress HTTPS port 134 | local_resource( 135 | 'istio-ingress-portforward-https', 136 | serve_cmd='kubectl port-forward -n istio-ingress svc/istio-ingressgateway {}:443'.format(ISTIO_HTTPS_PORT), 137 | links=['https://localhost:{}'.format(ISTIO_HTTPS_PORT)], 138 | readiness_probe=probe( 139 | initial_delay_secs = 20, 140 | timeout_secs = 1, 141 | period_secs = 10, 142 | success_threshold = 1, 143 | failure_threshold = 5, 144 | http_get=http_get_action(port=int(ISTIO_HTTPS_PORT), scheme='https') 145 | ), 146 | auto_init=False 147 | ) 148 | 149 | ## Istio Ingress HTTP port 150 | local_resource( 151 | 'istio-ingress-portforward-http', 152 | serve_cmd='kubectl port-forward -n istio-ingress svc/istio-ingressgateway {}:80'.format(ISTIO_HTTP_PORT), 153 | links=['http://localhost:{}'.format(ISTIO_HTTP_PORT)], 154 | readiness_probe=probe( 155 | initial_delay_secs = 20, 156 | timeout_secs = 1, 157 | period_secs = 10, 158 | success_threshold = 1, 159 | failure_threshold = 5, 160 | http_get=http_get_action(port=int(ISTIO_HTTP_PORT), scheme='http') 161 | ), 162 | auto_init=False 163 | ) 164 | -------------------------------------------------------------------------------- /docs/DEX_GITHUB_INTEGRATION.md: -------------------------------------------------------------------------------- 1 | # Kubernetes Authentication Through Dex and GitHub 2 | 3 | ## Introduction 4 | The document contains a procedure to configure Kubernetes authentication using Dex, GitHub, and [Kubelogin plugin](https://github.com/int128/kubelogin). 5 | As the access to local `Kind` cluster is already possible with generated kubeconfig, the intention is to provide an 6 | authentication method that can be used in a general use case. 7 | 8 | In this example, we will use GitHub personal account. Users can also use their GitHub Enterprise account which is further 9 | explained in [GitHub Dex documentation](https://dexidp.io/docs/connectors/github/#github-enterprise) 10 | 11 | ## Prerequisites 12 | * It is assumed that local cluster with Dex addon is already deployed. The needed steps are explained in the [bootstrap section](https://github.com/nearform/initium-platform#bootstrap). 13 | During cluster bootstrap OIDC parameters are already configured. Parameters can be checked in the [kind manifest](https://github.com/nearform/initium-platform/blob/main/manifests/kind/templates/cluster.yaml). 14 | * The Kubelogin plugin is installed using [setup instructions](https://github.com/int128/kubelogin#setup). 15 | * For now, we will use default configuration for Dex addon and make changes as we progress. 16 | 17 | ## Local Environment Configuration 18 | 1. Retrieve the IP address allocated by MetalLB to the service of type `Loadbalancer`. This address will make Dex issuer reachable on the local network. 19 | ```bash 20 | kubectl get svc -n istio-ingress istio-ingressgateway \ 21 | -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 22 | ``` 23 | 2. Make dex domain resolves to the Loadbalancer IP address by configuring `/etc/hosts` file: 24 | ```bash 25 | 172.18.255.200 dex.kube.local 26 | ``` 27 | 3. Add self-signed CA created during cluster bootstrap to trusted certificate store of your machine. For example, if using Ubuntu: 28 | ```bash 29 | sudo apt-get install -y ca-certificates 30 | sudo cp .ssl/ca.pem /usr/local/share/ca-certificates/ca.crt 31 | sudo update-ca-certificates 32 | ``` 33 | 34 | **WSL Users Only**: 35 | As WSL networking will not allow Loadbalancer address to be reachable from the Windows host, additional steps are needed: 36 | 4. Start TCP forwarding between the WSL LAN address and the Loadbalancer address. Utility `socat` can be used for that purpose. 37 | ```bash 38 | sudo socat TCP4-LISTEN:443,fork,reuseaddr TCP4:172.18.255.200:443 & 39 | ``` 40 | 5. Instead of using the Loadbalancer address in WSL hosts file, configure the WSL LAN address in the Windows hosts file, for example: 41 | ```bash 42 | 172.23.41.241 dex.kube.local 43 | ``` 44 | 6. Make sure **not** to use `generateResolvConf=false` otherwise above setting will not be reflected in WSL 45 | 46 | ## GitHub Configuration 47 | 1. [Create](https://github.com/settings/applications/new ) a new OAuth app setting in GitHub 48 | * Application name: Dex 49 | * Homepage URL: any URL related to the app 50 | * Authorization callback URL: https://dex.kube.local/callback 51 | 52 | 2. Open the registered app and generate a new client secret. Registered application can be found under `Account Settings/Developer settings/Oauth Apps`. 53 | Save the GitHub client id and the GitHub client secret for later use. 54 | 55 | ## Configure Secrets 56 | 1. Generate Kubelogin client secret that will be used for communication between Kubelogin and Dex 57 | 2. Export prepared values as environment variables: 58 | ```bash 59 | export GITHUB_CLIENT_ID=[github-client-id] 60 | export GITHUB_CLIENT_SECRET=[github-client-secret] 61 | export KUBELOGIN_CLIENT_SECRET=[kubelogin-client-secret] 62 | ``` 63 | 3. Create `github-client` Kubernetes secret 64 | ```bash 65 | kubectl create secret generic github-client \ 66 | --namespace=dex \ 67 | --from-literal=client-id=${GITHUB_CLIENT_ID} \ 68 | --from-literal=client-secret=${GITHUB_CLIENT_SECRET} 69 | ``` 70 | 4. Create `kubelogin-client` Kubernetes secret 71 | ```bash 72 | kubectl create secret generic kubelogin-client \ 73 | --namespace=dex \ 74 | --from-literal=client-secret=${KUBELOGIN_CLIENT_SECRET} 75 | ``` 76 | 77 | ## Configure Dex 78 | 1. Uncomment the GitHub section in Dex [values.yaml](https://github.com/nearform/initium-platform/blob/main/addons/dex/values.yaml) file. Also, remove the default section at the start of the file. Resulting config should look like the below: 79 | ```yaml 80 | dex-source: 81 | nameOverride: dex 82 | config: 83 | issuer: https://dex.kube.local 84 | storage: 85 | type: kubernetes 86 | config: 87 | inCluster: true 88 | connectors: 89 | - type: github 90 | id: github 91 | name: GitHub 92 | config: 93 | clientID: $GITHUB_CLIENT_ID 94 | clientSecret: $GITHUB_CLIENT_SECRET 95 | redirectURI: https://dex.kube.local/callback 96 | 97 | staticClients: 98 | - id: kubelogin 99 | redirectURIs: 100 | - 'http://localhost:8000' 101 | name: 'Kubelogin' 102 | secretEnv: KUBELOGIN_CLIENT_SECRET 103 | envVars: 104 | - name: GITHUB_CLIENT_ID 105 | valueFrom: 106 | secretKeyRef: 107 | name: github-client 108 | key: client-id 109 | - name: GITHUB_CLIENT_SECRET 110 | valueFrom: 111 | secretKeyRef: 112 | name: github-client 113 | key: client-secret 114 | - name: KUBELOGIN_CLIENT_SECRET 115 | valueFrom: 116 | secretKeyRef: 117 | name: kubelogin-client 118 | key: client-secret 119 | 120 | virtualService: 121 | name: dex 122 | namespace: dex 123 | gateway: istio-ingress/kube-gateway 124 | host: dex.kube.local 125 | port: 5556 126 | serviceName: dex 127 | ``` 128 | 2. Commit changes and make sure ArgoCD deployed Dex with the new configuration. 129 | 130 | ## Accessing Kind Cluster 131 | 1. Create cluster role binding that binds GitHub user (its email address) to a read-only cluster role. At this stage changes to the cluster 132 | are still made using kubeconfig generated at cluster bootstrap. 133 | ```bash 134 | kubectl create clusterrolebinding oidc-cluster-viewer --clusterrole=view --user='[github_email_address]' 135 | ``` 136 | 2. Set up the Kubeconfig 137 | ```bash 138 | kubectl config set-credentials oidc \ 139 | --exec-api-version=client.authentication.k8s.io/v1beta1 \ 140 | --exec-command=kubectl \ 141 | --exec-arg=oidc-login \ 142 | --exec-arg=get-token \ 143 | --exec-arg=--oidc-issuer-url=https://dex.kube.local \ 144 | --exec-arg=--oidc-client-id=kubelogin \ 145 | --exec-arg=--oidc-extra-scope=email \ 146 | --exec-arg=--oidc-client-secret=[plain-text-kubelogin-client-secret] 147 | ``` 148 | 3. Verify cluster access by using `oidc` user. In this step `kubelogin` will open a browser and redirect user to GiHub login page. 149 | ```bash 150 | kubectl --user=oidc get pods -A 151 | ``` 152 | 4. Authenticate, authorize Dex application to use your GitHub account and grant using your email address. 153 | If everything is successful you should see pods running in the cluster. 154 | 5. Switch the default context to oidc user and try deleting a pod. 155 | ```bash 156 | kubectl config set-context --current --user=oidc 157 | kubectl delete pod argocd-application-controller-0 -n argocd 158 | ``` 159 | This operation should be forbidden as we now use a read-only role. 160 | -------------------------------------------------------------------------------- /tests/helpers.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "os" 7 | "path/filepath" 8 | "strings" 9 | "testing" 10 | "time" 11 | 12 | "github.com/stretchr/testify/require" 13 | "gopkg.in/yaml.v3" 14 | v1 "k8s.io/apimachinery/pkg/apis/meta/v1" 15 | 16 | "github.com/gruntwork-io/terratest/modules/helm" 17 | "github.com/gruntwork-io/terratest/modules/k8s" 18 | "github.com/gruntwork-io/terratest/modules/random" 19 | ) 20 | 21 | type HelmAddonData struct { 22 | namespaceName string 23 | releaseName string 24 | dependencyRepo string 25 | addonName string 26 | addonAlias string 27 | chartPath string 28 | hasCustomValues bool 29 | manageNamespace bool 30 | overrideValues map[string]string 31 | } 32 | 33 | func readYamlFile(filename string) (*map[string]interface{}, error) { 34 | 35 | var err error 36 | 37 | bufferedContent, err := os.ReadFile(filename) 38 | if err != nil { 39 | return nil, err 40 | } 41 | 42 | mapped := make(map[string]interface{}) 43 | 44 | err = yaml.Unmarshal(bufferedContent, &mapped) 45 | if err != nil { 46 | return nil, fmt.Errorf("values not found in file %q: %w", filename, err) 47 | } 48 | 49 | return &mapped, err 50 | } 51 | 52 | func getDependenciesFromYamlFile(data map[string]interface{}, dependencies []string) (map[string]interface{}, error) { 53 | 54 | dependenciesValues := make(map[string]interface{}) 55 | 56 | for _, v := range data["dependencies"].([]interface{}) { 57 | for _, depValue := range dependencies { 58 | if k, value := v.(map[string]interface{})[depValue]; value { 59 | dependenciesValues[depValue] = k 60 | } else { 61 | return nil, fmt.Errorf("key %s not found in dependencies field", depValue) 62 | } 63 | } 64 | } 65 | 66 | return dependenciesValues, nil 67 | 68 | } 69 | 70 | func prepareHelmEnvironment(t *testing.T, addonData *HelmAddonData) (helm.Options, error) { 71 | helmOptions := helm.Options{} 72 | 73 | if addonData.namespaceName == "" { 74 | addonData.namespaceName = addonData.addonName 75 | } 76 | if addonData.releaseName == "" { 77 | addonData.releaseName = fmt.Sprintf("%s-test-%v", addonData.addonName, strings.ToLower(random.UniqueId())) 78 | } 79 | if addonData.dependencyRepo == "" { 80 | addonData.dependencyRepo = fmt.Sprintf("terratest-%s-%v", addonData.addonName, strings.ToLower(random.UniqueId())) 81 | } 82 | if addonData.chartPath == "" { 83 | addonData.chartPath = fmt.Sprintf("../addons/%v", addonData.addonName) 84 | } 85 | 86 | helmChartPath, err := filepath.Abs(addonData.chartPath) 87 | 88 | if err != nil { 89 | t.Errorf("Error processing %s error = %s", addonData.chartPath, err) 90 | return helmOptions, err 91 | } 92 | 93 | yamlContent, err := readYamlFile(fmt.Sprintf("%s/Chart.yaml", helmChartPath)) 94 | 95 | if err != nil { 96 | t.Errorf("Error reading chart yaml file, error = %s", err) 97 | return helmOptions, err 98 | } 99 | 100 | dependencies, err := getDependenciesFromYamlFile(*yamlContent, []string{"alias", "repository"}) 101 | 102 | addonData.addonAlias = dependencies["alias"].(string) 103 | 104 | if err != nil { 105 | t.Errorf("Error reading chart yaml file, error = %s", err) 106 | return helmOptions, err 107 | } 108 | 109 | kubectlOptions := k8s.NewKubectlOptions("", "", addonData.namespaceName) 110 | 111 | if addonData.manageNamespace { 112 | k8s.CreateNamespace(t, kubectlOptions, addonData.namespaceName) 113 | } 114 | 115 | valuesFiles := []string{} 116 | if addonData.hasCustomValues { 117 | valuesFiles = []string{fmt.Sprintf("%s/values.yaml", helmChartPath)} 118 | } 119 | helmOptions = helm.Options{ 120 | KubectlOptions: kubectlOptions, 121 | ValuesFiles: valuesFiles, 122 | SetValues: addonData.overrideValues, 123 | } 124 | 125 | helm.AddRepo(t, &helmOptions, addonData.dependencyRepo, dependencies["repository"].(string)) 126 | 127 | _, err = helm.RunHelmCommandAndGetOutputE(t, &helmOptions, "dependency", "build", helmChartPath) 128 | 129 | if err != nil { 130 | t.Errorf("Chart dependencies instalation failed, error = %s", err) 131 | return helmOptions, err 132 | } 133 | 134 | helm.Install(t, &helmOptions, helmChartPath, addonData.releaseName) 135 | 136 | return helmOptions, err 137 | } 138 | 139 | func destroyHelmEnvironment(t *testing.T, addonData HelmAddonData, helmOptions helm.Options) { 140 | helm.Delete(t, &helmOptions, addonData.releaseName, true) 141 | 142 | helm.RemoveRepo(t, &helmOptions, addonData.dependencyRepo) 143 | 144 | if addonData.manageNamespace { 145 | k8s.DeleteNamespace(t, helmOptions.KubectlOptions, addonData.namespaceName) 146 | } 147 | } 148 | 149 | func waitUntilHelmFormattedServicesAvailable(t *testing.T, addonData HelmAddonData, helmOptions helm.Options, services []string) { 150 | for _, v := range services { 151 | k8s.WaitUntilServiceAvailable(t, helmOptions.KubectlOptions, fmt.Sprintf("%s-%s", addonData.releaseName, v), 10, 30*time.Second) 152 | } 153 | } 154 | 155 | func waitUntilServicesAvailable(t *testing.T, kubectlOptions k8s.KubectlOptions, services []string) { 156 | for _, v := range services { 157 | k8s.WaitUntilServiceAvailable(t, &kubectlOptions, v, 10, 30*time.Second) 158 | } 159 | } 160 | 161 | type StatefulSetJsonStruct struct { 162 | Status struct { 163 | ReadyReplicas int `json:"readyReplicas"` 164 | } `json:"status"` 165 | } 166 | 167 | func waitUntilStatefulSetsAvailable(t *testing.T, kubectlOptions k8s.KubectlOptions, statefulSets []string) (success bool) { 168 | tries := 10 169 | readySS := 0 170 | for _, v := range statefulSets { 171 | currentTries := 0 172 | ready := false 173 | for currentTries < tries && !ready { 174 | ssstatus, err := k8s.RunKubectlAndGetOutputE(t, &kubectlOptions, "get", "statefulset", v, "-o=json") 175 | if err == nil { 176 | var ssstatusJson StatefulSetJsonStruct 177 | err = json.Unmarshal([]byte(ssstatus), &ssstatusJson) 178 | if err == nil { 179 | if ssstatusJson.Status.ReadyReplicas > 0 { 180 | ready = true 181 | readySS++ 182 | } 183 | } 184 | } 185 | currentTries++ 186 | time.Sleep(30 * time.Second) 187 | } 188 | } 189 | return readySS == len(statefulSets) 190 | } 191 | 192 | func waitUntilDeploymentsAvailable(t *testing.T, kubectlOptions k8s.KubectlOptions, deployments []string) { 193 | for _, v := range deployments { 194 | k8s.WaitUntilDeploymentAvailable(t, &kubectlOptions, v, 10, 30*time.Second) 195 | } 196 | } 197 | 198 | func waitUntilLoadBalancerAvailable(t *testing.T, kubectlOptions k8s.KubectlOptions) { 199 | for _, v := range k8s.ListServices(t, &kubectlOptions, v1.ListOptions{}) { 200 | if v.Spec.Type == "LoadBalancer" { 201 | k8s.WaitUntilServiceAvailable(t, &kubectlOptions, v.Name, 10, 30*time.Second) 202 | } 203 | } 204 | } 205 | 206 | func waitUntilDaemonSetAvailable(t *testing.T, kubectlOptions k8s.KubectlOptions, daemonSetName string) { 207 | retries := 10 208 | sleep := time.Second * 1 209 | for i := 1; i < retries; i++ { 210 | podsReady := k8s.GetDaemonSet(t, &kubectlOptions, daemonSetName).Status.NumberReady 211 | if podsReady > 0 { 212 | break 213 | } 214 | time.Sleep(sleep) 215 | } 216 | pods := k8s.ListPods(t, &kubectlOptions, v1.ListOptions{}) 217 | require.Greater(t, len(pods), 0) 218 | pod := pods[0] 219 | 220 | k8s.WaitUntilPodAvailable(t, &kubectlOptions, pod.Name, 10, 30*time.Second) 221 | } 222 | -------------------------------------------------------------------------------- /examples/sample-prometheus-app/sample-prometheus-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: serving.knative.dev/v1 2 | kind: Service 3 | metadata: 4 | name: sample-prometheus-app 5 | namespace: prometheus 6 | labels: 7 | release: kube-prometheus-stack 8 | app.kubernetes.io/name: sample-prometheus-app 9 | spec: 10 | template: 11 | metadata: 12 | labels: 13 | app.kubernetes.io/name: sample-prometheus-app 14 | release: kube-prometheus-stack 15 | annotations: 16 | autoscaling.knative.dev/scale-to-zero-pod-retention-period: "5m" 17 | spec: 18 | containers: 19 | # TO DO this is just a sample image and it should be replaced 20 | - image: "quay.io/brancz/prometheus-example-app:v0.3.0" 21 | ports: 22 | - containerPort: 8080 23 | 24 | --- 25 | apiVersion: monitoring.coreos.com/v1 26 | kind: PodMonitor 27 | metadata: 28 | labels: 29 | app.kubernetes.io/name: sample-prometheus-app 30 | release: kube-prometheus-stack 31 | name: sample-prometheus-app 32 | namespace: prometheus 33 | spec: 34 | selector: 35 | matchLabels: 36 | app.kubernetes.io/name: sample-prometheus-app 37 | release: kube-prometheus-stack 38 | podMetricsEndpoints: 39 | - port: user-port 40 | 41 | --- 42 | apiVersion: v1 43 | kind: ConfigMap 44 | metadata: 45 | labels: 46 | grafana_dashboard: "1" 47 | name: sample-prometheus-app 48 | namespace: grafana 49 | data: 50 | k8s-pod-count.json: |- 51 | { 52 | "annotations": { 53 | "list": [ 54 | { 55 | "builtIn": 1, 56 | "datasource": { 57 | "type": "grafana", 58 | "uid": "-- Grafana --" 59 | }, 60 | "enable": true, 61 | "hide": true, 62 | "iconColor": "rgba(0, 211, 255, 1)", 63 | "name": "Annotations & Alerts", 64 | "target": { 65 | "limit": 100, 66 | "matchAny": false, 67 | "tags": [], 68 | "type": "dashboard" 69 | }, 70 | "type": "dashboard" 71 | } 72 | ] 73 | }, 74 | "editable": true, 75 | "fiscalYearStartMonth": 0, 76 | "graphTooltip": 0, 77 | "id": 27, 78 | "links": [], 79 | "liveNow": false, 80 | "panels": [ 81 | { 82 | "gridPos": { 83 | "h": 1, 84 | "w": 24, 85 | "x": 0, 86 | "y": 0 87 | }, 88 | "id": 8, 89 | "title": "Sample Prometheus App", 90 | "type": "row" 91 | }, 92 | { 93 | "datasource": { 94 | "type": "prometheus", 95 | "uid": "prometheus" 96 | }, 97 | "description": "", 98 | "fieldConfig": { 99 | "defaults": { 100 | "color": { 101 | "mode": "palette-classic" 102 | }, 103 | "custom": { 104 | "axisCenteredZero": false, 105 | "axisColorMode": "text", 106 | "axisLabel": "", 107 | "axisPlacement": "auto", 108 | "barAlignment": 0, 109 | "drawStyle": "line", 110 | "fillOpacity": 0, 111 | "gradientMode": "none", 112 | "hideFrom": { 113 | "legend": false, 114 | "tooltip": false, 115 | "viz": false 116 | }, 117 | "lineInterpolation": "linear", 118 | "lineWidth": 1, 119 | "pointSize": 5, 120 | "scaleDistribution": { 121 | "type": "linear" 122 | }, 123 | "showPoints": "auto", 124 | "spanNulls": false, 125 | "stacking": { 126 | "group": "A", 127 | "mode": "none" 128 | }, 129 | "thresholdsStyle": { 130 | "mode": "off" 131 | } 132 | }, 133 | "mappings": [], 134 | "thresholds": { 135 | "mode": "absolute", 136 | "steps": [ 137 | { 138 | "color": "green", 139 | "value": null 140 | }, 141 | { 142 | "color": "red", 143 | "value": 80 144 | } 145 | ] 146 | } 147 | }, 148 | "overrides": [] 149 | }, 150 | "gridPos": { 151 | "h": 8, 152 | "w": 12, 153 | "x": 0, 154 | "y": 1 155 | }, 156 | "id": 4, 157 | "options": { 158 | "legend": { 159 | "calcs": [], 160 | "displayMode": "list", 161 | "placement": "bottom", 162 | "showLegend": true 163 | }, 164 | "tooltip": { 165 | "mode": "single", 166 | "sort": "none" 167 | } 168 | }, 169 | "pluginVersion": "9.3.1", 170 | "targets": [ 171 | { 172 | "datasource": { 173 | "type": "prometheus", 174 | "uid": "prometheus" 175 | }, 176 | "editorMode": "code", 177 | "expr": "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{code=\"200\"}[$__rate_interval])) by (le))", 178 | "legendFormat": "__auto", 179 | "range": true, 180 | "refId": "A" 181 | }, 182 | { 183 | "datasource": { 184 | "type": "prometheus", 185 | "uid": "prometheus" 186 | }, 187 | "editorMode": "builder", 188 | "expr": "http_request_duration_seconds_sum{code=\"200\"}", 189 | "hide": false, 190 | "legendFormat": "__auto", 191 | "range": true, 192 | "refId": "B" 193 | }, 194 | { 195 | "datasource": { 196 | "type": "prometheus", 197 | "uid": "prometheus" 198 | }, 199 | "editorMode": "builder", 200 | "expr": "http_request_duration_seconds_count{code=\"200\"}", 201 | "hide": false, 202 | "legendFormat": "__auto", 203 | "range": true, 204 | "refId": "C" 205 | }, 206 | { 207 | "datasource": { 208 | "type": "prometheus", 209 | "uid": "prometheus" 210 | }, 211 | "editorMode": "builder", 212 | "expr": "http_requests_total{code=\"200\"}", 213 | "hide": false, 214 | "legendFormat": "__auto", 215 | "range": true, 216 | "refId": "D" 217 | } 218 | ], 219 | "title": "Sample Prometheus App", 220 | "type": "timeseries" 221 | } 222 | ], 223 | "refresh": "5s", 224 | "schemaVersion": 37, 225 | "style": "dark", 226 | "tags": [], 227 | "templating": { 228 | "list": [] 229 | }, 230 | "time": { 231 | "from": "now-6h", 232 | "to": "now" 233 | }, 234 | "timepicker": {}, 235 | "timezone": "", 236 | "title": "Sample Prometheus App", 237 | "uid": "1JFc6s24z", 238 | "version": 5, 239 | "weekStart": "" 240 | } 241 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Welcome to Initium platform 2 | 3 | Please take a second learn about [initium-platform code repo](https://github.com/nearform/initium-platform/blob/main/README.md). 4 | 5 | We greatly appreciate bug fixes, documentation improvements and new features, however when contributing a new major feature, it is a good idea to idea to first open an issue, to make sure the feature it fits with the goal of the project, so we don't waste your or our time. 6 | 7 | ## General mechanics 8 | 9 | ![Inner workings of make](docs/img/inner-workings/k8s-addons-internals.png) 10 | 11 | ## How To Contribute 12 | 13 | <a id="contributing-how-to"></a> 14 | 15 | If you'd like to contribute, start by searching through the 16 | [issues](https://github.com/nearform/initium-platform/issues) and [pull 17 | requests](https://github.com/nearform/initium-platform/pulls) to see whether someone else 18 | has raised a similar idea or question. 19 | 20 | If you don't see your idea listed, and you think it fits into the goals of this 21 | guide, do one of the following: 22 | 23 | * **If your contribution is minor,** such as a typo fix, open a pull request. 24 | * **If your contribution is major,** such as a new feature, start by opening an 25 | issue first. That way, other people can weigh in on the discussion before you 26 | do any work. 27 | 28 | ## Conventional Commits 29 | 30 | We use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) in this repo. We encourage contributors to name their PRs and commits accordingly. That is required for keeping the repo clean, and [release-please](https://github.com/googleapis/release-please) to do its job when automating release creation. 31 | 32 | Basically, the `fix` and `feat` words on commits will trigger new releases (with `fix` being patch versions and `feat` minor versions, unless there is an exclamation mark (`!`) after one of both, which will trigger a major version release). Other keywords won't trigger releases, but they are welcome in order to better readability of the changes made. Examples: `docs`, `chore`, `ci`, `test` and so on. 33 | 34 | ## Bug Reports 35 | 36 | A perfect bug report would have the following: 37 | 38 | 1. Summary of the issue you are experiencing. 39 | 2. A simple repeatable test case for us to run. Please try to run through it 2-3 times to ensure it is completely repeatable. 40 | 41 | We would like to avoid issues that require a follow up questions to identify the bug. These follow ups are difficult to do unless we have a repeatable test case. 42 | 43 | ## How to create a new release 44 | 45 | > **PLEASE NOTE:** In order to make sure this release pipeline works, every PR and commit MUST use the [Conventional Commits](https://www.conventionalcommits.org/) syntax. 46 | 47 | This project uses an automated release setup that is based on [release-please](https://github.com/googleapis/release-please). For a more detailed overview on how it works, feel free to read [their official introduction](https://github.com/googleapis/release-please#whats-a-release-pr). 48 | 49 | In short, what happens is that during the normal lifecycle of contributions, as you keep merging PRs, a new release PR will be made (or updated) which will list the changes that will be included in such release ( for eg. see https://github.com/nearform/initium-platform/pull/11 ). 50 | 51 | Once you merge the PR, a new release will be made and the [`CHANGELOG.md`](https://github.com/nearform/initium-platform/blob/main/CHANGELOG.md) will be updated as well. As soon as the release is created, [this workflow](https://github.com/nearform/initium-platform/blob/main/.github/workflows/assets.yaml) will be run on top of it, and as a result it will upload on such release the final assets that we deliver for the end-user. 52 | 53 | These assets can finally be found on the release Assets section ( for eg. see https://github.com/nearform/initium-platform/releases/tag/v0.0.1 ). 54 | 55 | ## Running the local stack 56 | 57 | Since this project is mostly based on the concept of [ArgoCD app of apps](https://argo-cd.readthedocs.io/en/stable/operator-manual/cluster-bootstrapping/#app-of-apps-pattern) it will require a Git repository to host the components required to be deployed by ArgoCD on the cluster. 58 | 59 | Usually you would see your ArgoCD Applications pointing to a Github repository or any other Git repository that is available to your network. On our own side we bypass this by providing your local repository ( the one you use once you clone this repository in your own workstation ) as a "remote repository" to the eyes of ArgoCD when you deploy the stack locally. 60 | 61 | How this is done is simple: 62 | - We use a project named [git-http-backend](https://github.com/ynohat/git-http-backend) where you basically mirror your local folder as an HTTP Git server 63 | - We mount the **parent folder** of where you clone this repository [in kind](./manifests/kind/templates/cluster.yaml#L6-9) so later on the charts can use the `containerPath` as mount volume 64 | - We build a docker image of this project locally to ensure it doesn't change in time and no malicious code comes with it, see [utils/git-http-backend/docker](utils/git-http-backend/docker) 65 | - We create a custom Helm chart to deploy the built container, see [utils/git-http-backend/chart](utils/git-http-backend/chart) 66 | - We [build and deploy the image as a chart in the cluster via Tilt](./Tiltfile#L12-18) and we pass to the container image the [kind mounted path](./manifests/kind/templates/cluster.yaml#L8) of where this repository lives, so you can later access it via the [INITIUM_REPO_URI defined in the .envrc](./.envrc#L8) ( eg. `http://git-http-backend/git/initium-platform` ) 67 | 68 | **REMEMBER:** As ArgoCD will use the latest commit to checkout changes, you **will have** to commit your changes locally otherwise ArgoCD won't be able to see them. Pushing is **NOT REQUIRED**. 69 | 70 | This change will allow you to test the entire solution without requiring a special access to the central Git server repository. All you will need is just your workstation and an internet connection. 71 | 72 | ## Renovate Bot 73 | 74 | This project makes use of the [Renovate Bot](https://docs.renovatebot.com/) to automatically bump chart versions. 75 | 76 | ### Bump your addons using a single PR 77 | 78 | There are some edge cases to be considered when your addon is divided in multiple charts ( for example, see the [istio addon](https://github.com/nearform/initium-platform/tree/main/addons/istio) ). In such cases, Renovate Bot will by default open a PR for each chart detected to be bumped, although this would make the PR difficult to be merged, as each part of the Istio stack has to be bumped as a whole to work correctly. 79 | 80 | In order to fix this behavior, you will need to add into the [packageRules](https://github.com/nearform/initium-platform/blob/main/.github/renovate.json5#L11) section an object that will instruct the Renovate bot to consider the addon as whole. Following the istio example, this is the configuration to be used: 81 | ```js 82 | { 83 | matchPaths: ["addons/istio/**"], 84 | groupName: "Istio Helm Chart" 85 | } 86 | ``` 87 | 88 | You can find out more about this syntax on the relative documentation page at this address: https://docs.renovatebot.com/configuration-options/#packagerules 89 | 90 | ## FAQ 91 | 92 | Q: Why does this repo create a Tilt local_resource for Istio ingress since there is already a MetalLB instance setting an Load Balancer IP address for it? 93 | 94 | A: As per [the Kind docs](https://kind.sigs.k8s.io/docs/user/loadbalancer/): 95 | 96 | > On MacOS and Windows, docker does not expose the docker network to the host. Because of this limitation, containers (including kind nodes) are only reachable from the host via port-forwards, however other containers/pods can reach other things running in docker including loadbalancers. 97 | 98 | The Tilt local_resource for port-forwarding the Istio ingress ports is a workaround for that limitation. 99 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # initium-platform 2 | 3 | This repository is a tentative approach to offer a set of add-ons that will provide a day-0 platform ready to be used in a matter of hours instead of weeks. 4 | 5 | To accomplish that, it uses the ArgoCD [app of apps concept](https://argo-cd.readthedocs.io/en/stable/operator-manual/cluster-bootstrapping/#app-of-apps-pattern). 6 | 7 | Therefore, ArgoCD is the main requirement to run this project on your cluster. 8 | 9 | > **PLEASE NOTE:** This project is currently WIP and NOT stable. Use at your own risk! 10 | 11 | ## Install on your own cluster 12 | 13 | 14 | > **HINT** This project uses `tilt` to bring up environment, but there are 2 executables named `tilt`. If you already have it on your system, make sure that you are using [tilt.dev](https://tilt.dev/) instead of [Ruby tilt](https://github.com/rtomayko/tilt) 15 | 16 | 17 | ![Quick Start](docs/img/quick-start/k8s-addons-quick-start.png) 18 | 19 | If you dont have `argocd` on your cluster, the following command will install it with required configuration. Make sure that you are using the correct Kubernetes context before run. 20 | 21 | ```bash 22 | $ make argocd 23 | ``` 24 | 25 | If you already have `argocd` (if installed with helm, the name of the chart should be argocd) deployed in your cluster to make deployment successful and all addons synced it is required to verify if following configuration is part of your ArgoCD configuration: [argocd/values.yaml](https://github.com/nearform/initium-platform/blob/main/addons/argocd/values.yaml#L23). 26 | You can check it by describing argo-cd config map: 27 | 28 | ``` 29 | kubectl describe cm argocd-cm -n argocd 30 | ``` 31 | 32 | Don't apply changes directly to the Config Map, but apply them the in your installation scripts/helm chart values. 33 | 34 | Next, you can download the provided `app-of-apps.yaml` in our [latest stable release](https://github.com/nearform/initium-platform/releases/latest) and then apply that manifest using this command: 35 | 36 | ```bash 37 | kubectl apply -f app-of-apps.yaml 38 | ``` 39 | 40 | Full list of the addons that the manifest is installing and additional information for each of them can be found in the [docs/ADDONS.md](docs/ADDONS.md) file. 41 | 42 | ### Pass your own values to the addons 43 | 44 | You can pass your own values or override default values to the addons by modifying the `app-of-apps.yaml` manifest, you can get more details here [docs/ADDONS.md](docs/ADDONS.md#override-values) 45 | 46 | ### Supported Setups 47 | 48 | Below there is a matrix with the cloud providers & Kubernetes versions our setup has been tested with. 49 | 50 | | Cloud Provider | Kubernetes Version | 51 | |------------------|:------------------:| 52 | | AWS | 1.27 | 53 | | GCP | 1.27 | 54 | | Azure | 1.27 | 55 | 56 | 57 | ## Run locally 58 | 59 | ### Pre-requisites 60 | 61 | You need Docker installed (or similar solutions) to run this project. 62 | 63 | Here you can find a list of possible candidates: 64 | 65 | - [Docker](https://docs.docker.com/engine/install/) ( cross-platform, paid solution ) 66 | - [Rancher Desktop](https://rancherdesktop.io/) ( cross-platform, FOSS ) 67 | - [lima](https://github.com/lima-vm/lima) + [nerdctl](https://github.com/containerd/nerdctl) ( macOS only ) 68 | 69 | Remember that to run this solution you also need at least: 70 | 71 | - 4 CPU cores 72 | - 8 GB RAM - Maybe is necessary to increase Docker limits to use more RAM or Swap to run all components with 8GB RAM. 73 | - 16 GB Disk space 74 | 75 | Those numbers are not written in stone so your mileage may vary depending on which components you choose to install. 76 | 77 | > **HINT:** To run everything on Windows machine is recommended to use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) and install Docker inside this subsystem. 78 | 79 | ### Bootstrap 80 | 81 | > **HINT:** If you want to remove the automatically installed dependencies via asdf once you're done with this project, you can run `make asdf_uninstall`. 82 | 83 | To continue with the deployment, you need a set of tools installed first. You can either install them manually (see [.tool-versions](.tool-versions) ), or you can install them automatically by following these steps: 84 | 85 | 1. Install [`asdf-vm`](https://asdf-vm.com/) 86 | 2. Run the following command: `make asdf_install` 87 | 88 | ### Deploy 89 | 90 | > **HINT:** If you want to know which commands are available you can always run `make help` on this project. 91 | 92 | #### CLI 93 | 94 | ![Inner workings of make](docs/img/inner-workings/k8s-addons-internals.png) 95 | Make sure you've followed the [bootstrap steps](#bootstrap), then: 96 | 97 | ```bash 98 | $ make ci 99 | ``` 100 | 101 | Once completed, you're ready to interact with the cluster using `kubectl` as usual. 102 | 103 | The current Kubernetes version that will run locally in the project is defined in the `.envrc` file, using the variable `K8S_VERSION`. If you want to run a different local k8s version, please change this value. 104 | 105 | Github runners are using a matrix strategy to run unit and integration tests on three different k8s versions. 106 | Take a look at the `.github/workflows/integration.yaml` or `.github/workflows/preview.yaml` for an example. 107 | 108 | #### GUI 109 | 110 | Make sure you've followed the [bootstrap steps](#bootstrap), then: 111 | 112 | 1. Deploy the local environment: 113 | 114 | ```bash 115 | # Deploy the cluster and run tilt visually 116 | $ make 117 | ``` 118 | 119 | You can access the Kind K8s cluster using any kubernetes client like kubectl or Lens.<br> 120 | [Accessing Argocd UI](https://argo-cd.readthedocs.io/en/stable/getting_started/#3-access-the-argo-cd-api-server) 121 | 122 | 2. (Optional) Run the two following resources in tilt to portforward argocd and get the default admin password 123 | 124 | ``` 125 | - argocd-portforward 126 | - argocd-password 127 | ``` 128 | 129 | > **PLEASE NOTE:** The port forwarding sometimes seems to drop, so re-run the tilt resource to get the connection up and running again. 130 | 131 | 3. (Optional) Test app-of-apps values changes using the override feature of the bootstrap app, following the instructions in the `./manifests/bootstrap/overrides.local.yaml.tmpl` file. 132 | 133 | #### Cleanup 134 | 135 | > **IMPORTANT:** Make sure to run this command while tilt is NOT running. 136 | 137 | In order to cleanup the solution from your own local environment you can run: 138 | 139 | ```bash 140 | $ make clean 141 | ``` 142 | 143 | ### Tests 144 | 145 | #### Unit tests 146 | 147 | All the tests are inside the `tests` folder. This folder contains tests written in golang to check if the charts 148 | configurations are valid. To create the tests, we use [terratest](https://terratest.gruntwork.io/). 149 | Terratest provides a variety of [helper functions](https://github.com/gruntwork-io/terratest) created by Gruntwork, 150 | To make the infrastructure test setup easier. 151 | 152 | Before running tests make sure you've followed the [bootstrap steps](#bootstrap). 153 | 154 | Once ready, you can use this simple set of commands: 155 | 156 | ```bash 157 | # Create the cluster 158 | $ INITIUM_DEPLOY_MINIMAL=true make ci 159 | # Run the unit tests 160 | $ make unit-test 161 | ``` 162 | 163 | #### Integration tests 164 | 165 | As unit tests, we also use [terratest](https://terratest.gruntwork.io/). The main difference is that each test is encapsulated by its goal. 166 | See the [examples/sample-app](examples/sample-app) as an example. 167 | 168 | Before running tests make sure you've followed the [bootstrap steps](#bootstrap). 169 | 170 | Once ready, you can use this simple set of commands: 171 | 172 | ```bash 173 | # Create the cluster and all the required dependencies 174 | $ make ci 175 | # Run the unit tests 176 | $ make integration-test 177 | ``` 178 | 179 | ## Contribute 180 | 181 | If you desire to contribute to this project, please make sure you have read the [CONTRIBUTING.md](CONTRIBUTING.md) file first. 182 | 183 | Once you feel ready for it, remember to [fork this repository](https://github.com/nearform/initium-platform/fork). 184 | 185 | Clone your fork into your own local workstation, then finally run the following command to install the project dependencies using [asdf-vm](https://asdf-vm.com/) and [precommit](https://pre-commit.com/) hooks: 186 | 187 | ```bash 188 | cd path/to/initium-platform 189 | make bootstrap 190 | ``` 191 | 192 | You're now ready to contribute! 193 | 194 | ## CI Process 195 | 196 | After deploying the cluster, you may wish to integrate it with a CI tool of choice. 197 | For more information on this subject, consult [CI Service Account Readme](manifests/ci-service-account/README.md) 198 | -------------------------------------------------------------------------------- /docs/ADDONS.md: -------------------------------------------------------------------------------- 1 | # initium-platform addon list 2 | 3 | This document is a list of addons, what they are, how to use them and their purpose in our repository. This is going to be updated as the repository grows. 4 | 5 | It is important to emphasize that none of the following addons are strictly **required**. That's why most of them can be disabled by adding the `excluded: true` to the app-of-apps `values.yaml` file. 6 | 7 | ## Summary 8 | - ArgoCD 9 | - cert-manager 10 | - Dex 11 | - Istio 12 | - Knative 13 | - kube-prometheus-stack 14 | - Additional Notes 15 | 16 | ### ArgoCD 17 | 18 | ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes. 19 | 20 | ArgoCD follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. Kubernetes manifests can be specified in several ways: 21 | 22 | - `kustomize` applications 23 | - `helm` charts 24 | - `jsonnet` files 25 | - Plain directory of YAML/json manifests 26 | - Any custom config management tool configured as a config management plugin 27 | 28 | ArgoCD automates the deployment of the desired application states in the specified target environments. Application deployments can track updates to branches, tags, or pinned to a specific version of manifests at a Git commit. See tracking strategies for additional details about the different tracking strategies available. 29 | 30 | We use ArgoCD in our repository for managing all the addons that will be installed on the Kubernetes clusters. It is possible to run all the addons on the same `initium-platform` revision, or pass down a specific revision to each addon, using the `app-of-apps/values.yaml` file `targetRevision` field. 31 | 32 | More information at [ArgoCD Docs](https://argo-cd.readthedocs.io/en/stable/). 33 | 34 | ### cert-manager 35 | 36 | *cert-manager is disabled by default* 37 | 38 | `cert-manager` is a cloud-native certificate management solution designed to work on Kubernetes. It integrates with AWS Certificate Manager, GCP Certificate Manager, CloudFlare, Let's Encrypt, as well as local issuers and other providers to create SSL/TLS certificates. It is a member of CNCF since 2020. 39 | 40 | `cert-manager` main responsibilities are to issue certificates and ensure they are valid and up to date, as well as attempt to renew them at a configured time before expiry. 41 | 42 | We use `cert-manager` in this repository for managing all the SSL/TLS certificates a Kubernetes cluster might need. It is listed on our addon dictionary because most clusters need working SSL/TLS certificates for their services that are exposed to the internet. 43 | 44 | The way `cert-manager` is set up in this repository, getting it to work once installed is just a matter of setting up a ClusterIssuer custom resource that will integrate with the desired provider (Let's Encrypt, for example), and configure secrets and desired domains. 45 | 46 | More info at [cert-manager Docs](https://cert-manager.io/docs/). 47 | 48 | ### dex 49 | 50 | Dex is a Federated OpenID Connect Provider, and a Sandbox project at CNCF. 51 | 52 | Dex acts as a portal to other identity providers through “connectors.” This lets Dex defer authentication to LDAP servers, SAML providers, or established identity providers like GitHub, Google, and Active Directory. Clients write their authentication logic once to talk to Dex, then Dex handles the protocols for a given backend. 53 | 54 | Once the user has dex up and running, the next step is to write applications that use dex to drive authentication. Apps that interact with dex generally fall into one of two categories: 55 | 56 | - Apps that request OpenID Connect ID tokens to authenticate users. 57 | - Used for authenticating an end user. 58 | - Must be web based. 59 | - Standard OAuth2 clients. Users show up at a website, and the application wants to authenticate those end users by pulling claims out of the ID token. 60 | - Apps that consume ID tokens from other apps. 61 | - Needs to verify that a client is acting on behalf of a user. 62 | - These consume ID tokens as credentials. 63 | - This lets another service handle OAuth2 flows, then use the ID token retrieved from dex to act on the end user’s behalf with the app. 64 | - An example of an app that falls into this category is the Kubernetes API server . 65 | 66 | More information at [Dex Docs](https://dexidp.io/docs/getting-started/). 67 | 68 | ### Istio 69 | 70 | Istio is an open source service mesh, which is a dedicated infrastructure layer that you can add to your applications. It allows you to transparently add capabilities like observability, traffic management, and security, without adding them to your own code. 71 | 72 | Istio provides: 73 | - Secure service-to-service communication in a cluster with TLS encryption, strong identity-based authentication and authorization 74 | - Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic 75 | - Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection 76 | - A pluggable policy layer and configuration API supporting access controls, rate limits and quotas 77 | - Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress 78 | 79 | More information at [Istio Docs](https://istio.io/latest/). 80 | 81 | ### Knative 82 | 83 | Knative is a platform-agnostic solution for running serverless deployments. It has two main components called `Serving` and `Eventing`, which empower teams working with Kubernetes. They work together to automate and manage tasks and applications. 84 | 85 | ##### Serving 86 | Knative Serving defines a set of objects as Kubernetes Custom Resource Definitions (CRDs). These resources are used to define and control how your serverless workload behaves on the cluster. 87 | 88 | Common use cases for Knative serving are: 89 | 90 | - Rapid deployment of serverless containers. 91 | - Autoscaling, including scaling pods down to zero. 92 | - Support for multiple networking layers, such as Contour, Kourier, and Istio, for integration into existing environments. 93 | 94 | The primary Knative Serving resources are: 95 | - Services, which automatically manage the whole lifecycle of your workload. They control the creation of other objects to ensure that your app has a route, a configuration, and a new revision for each update of the service. 96 | 97 | - Routes, which map a network endpoint to one or more revisions. 98 | 99 | - Configurations, which maintain the desired state for your deployment. It provides a clean separation between code and configuration and follows the Twelve-Factor App methodology. Modifying a configuration creates a new revision. 100 | 101 | - Revisions, which is a point-in-time snapshot of the code and configuration for each modification made to the workload. Revisions are immutable objects and can be retained for as long as useful. Knative Serving Revisions can be automatically scaled up and down according to incoming traffic. 102 | 103 | ##### Eventing 104 | Knative Eventing is a collection of APIs that enable you to use an event-driven architecture with your applications. You can use these APIs to create components that route events from event producers to event consumers, known as sinks, that receive events. Sinks can also be configured to respond to HTTP requests by sending a response event. 105 | 106 | Knative Eventing uses standard HTTP POST requests to send and receive events between event producers and sinks. These events conform to the CloudEvents specifications, which enables creating, parsing, sending, and receiving events in any programming language. 107 | 108 | Common use cases of Knative Eventing are: 109 | 110 | - Publishing an event without creating a consumer. 111 | - You can send events to a broker as an HTTP POST, and use binding to decouple the destination configuration from your application that produces events. 112 | 113 | - Consuming an event without creating a publisher. 114 | - You can use a trigger to consume events from a broker based on event attributes. The application receives events as an HTTP POST. 115 | 116 | More information at [Knative Docs](https://knative.dev/docs/). 117 | 118 | ### kube-prometheus-stack 119 | 120 | > **IMPORTANT:** This addon requires >= ArgoCD 2.5.x 121 | 122 | `kube-prometheus-stack` is a collection of Kubernetes manifests, Grafana dashboards, and Prometheus rules combined with documentation and scripts to provide easy to operate end-to-end Kubernetes cluster monitoring with Prometheus using the Prometheus Operator. 123 | 124 | We use `kube-prometheus-stack` as the main observability stack deployed on the Kubernetes cluster. It can also be tweaked with values like Grafana login credentials, and Prometheus rules, as well as ingress configurations. 125 | 126 | More information at [kube-prometheus-stack Docs](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack). 127 | 128 | #### ArgoCD < 2.5.x 129 | 130 | This addon requires server deployment, which is unavailable until ArgoCD 2.5 ( see https://github.com/argoproj/argo-cd/issues/820 ). Unfortunately, there's no way to deploy it using earlier versions. To disable this addon, you can use the snippet below: 131 | 132 | ```yaml 133 | apps: 134 | kube-prometheus-stack: 135 | excluded: true 136 | ``` 137 | 138 | ### OpenTelemetry 139 | OpenTelemetry is used to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. OpenTelemetry is generally available across several languages and is suitable for use. Depending on the project requirements, the OpenTelemetry addon can be enabled and disabled via an ENV variable. 140 | 141 | ##### Collector 142 | The OpenTelemetry Collector offers a vendor-agnostic implementation of how to receive, process and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors. 143 | 144 | ##### Operator 145 | The OpenTelemetry Operator is an implementation of a Kubernetes Operator, it manages collectors and auto-instrumentation of the workload using OpenTelemetry instrumentation libraries. 146 | 147 | ### Additional Notes 148 | 149 | We are constantly evaluating new addons that might become standards in the industry. That's not high priority, though, since our main goal is to keep this repository straight to the point and minimize overhead on the users' clusters. 150 | 151 | If you want to contribute with the repo, see [CONTRIBUTING.md](CONTRIBUTING.md). 152 | 153 | # Override values 154 | 155 | You can override values on the addons modifying the `app-of-apps.yaml` manifest. 156 | Just define an `helmValues` key on the addons you want to customize eg: 157 | 158 | ```yaml 159 | helm: 160 | values: | 161 | repoURL: https://github.com/nearform/initium-platform.git 162 | subChartsRevision: v0.0.1 163 | apps: 164 | dex: 165 | hemlValues: 166 | dex-source: 167 | fullnameOverride: dexy 168 | ``` 169 | 170 | We are using `dex-source` since that is the [alias](/addons/dex/Chart.yaml#L8) that we used for the dependency chart. 171 | Each addon has its own alias for the dependency chart, you can find it in the specific addon `Chart.yaml` file in the [/addons](/addons) folder. 172 | --------------------------------------------------------------------------------