├── src ├── python_based_operator │ ├── __init__.py │ ├── provisioners │ │ ├── __init__.py │ │ └── v1alpha1.py │ ├── charts │ │ └── prometheus │ │ │ ├── templates │ │ │ ├── configmap.yaml │ │ │ ├── lb-service.yaml │ │ │ ├── headless-service.yaml │ │ │ ├── statefulset.yaml │ │ │ └── _helpers.tpl │ │ │ ├── .helmignore │ │ │ ├── Chart.yaml │ │ │ └── values.yaml │ ├── logs.py │ └── operator.py ├── MANIFEST.in ├── requirements-dev.in ├── requirements.txt ├── setup.py └── requirements-dev.txt ├── .python-version ├── .flake8 ├── examples └── simple.yaml ├── .gitignore ├── templates ├── README.txt ├── ns.yml ├── dev_kubeconfig.yml ├── operator.yml ├── rbac.yml └── crd.yml ├── LICENSE ├── Dockerfile ├── scripts └── render ├── Makefile └── README.md /src/python_based_operator/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /src/python_based_operator/provisioners/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | python-based-operator-3.8.3 2 | 3.8.3 3 | -------------------------------------------------------------------------------- /src/MANIFEST.in: -------------------------------------------------------------------------------- 1 | recursive-include python_based_operator/charts * 2 | -------------------------------------------------------------------------------- /.flake8: -------------------------------------------------------------------------------- 1 | # https://flake8.pycqa.org/en/2.6.0/config.html 2 | [flake8] 3 | max-line-length = 88 4 | -------------------------------------------------------------------------------- /examples/simple.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: relaxdiego.com/v1alpha1 2 | kind: PrometheusCluster 3 | metadata: 4 | name: simple 5 | spec: {} 6 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Python files 2 | .venv 3 | *.egg-info 4 | __pycache__ 5 | 6 | # MacOS files 7 | .DS_Store 8 | 9 | # Common backup files 10 | *.swp 11 | *.bak 12 | *.tmp 13 | *~ 14 | 15 | # Various IDEs 16 | .project 17 | .idea/ 18 | *.tmproj 19 | .vscode/ 20 | 21 | # Make files 22 | .last-* 23 | .tmp/ 24 | -------------------------------------------------------------------------------- /src/requirements-dev.in: -------------------------------------------------------------------------------- 1 | # Using workaround from https://github.com/jazzband/pip-tools/issues/204#issuecomment-550051424 2 | -e file:.#egg=python-based-operator 3 | 4 | # Use dependency list in requirements.txt to constrain the dependencies that 5 | # gets resolved when compiling this file. 6 | -c requirements.txt 7 | 8 | pytest>=5.4.3,<5.5.0 9 | -------------------------------------------------------------------------------- /templates/README.txt: -------------------------------------------------------------------------------- 1 | You are free to convert this into a helm chart if you wish. In fact, that's 2 | what I did in the first iteration of this template but later decided to 3 | simplify it for those that are not familiar with Helm. 4 | Old style: https://github.com/relaxdiego/python-based-operator/tree/cd0fd5076226cb5aebb548d0160e791ea3046fdb/charts 5 | -------------------------------------------------------------------------------- /templates/ns.yml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Namespace 4 | metadata: 5 | name: python-based-operator 6 | labels: 7 | name: python-based-operator 8 | --- 9 | apiVersion: v1 10 | kind: ServiceAccount 11 | metadata: 12 | name: {{SERVICEACCOUNT_NAME}} 13 | namespace: python-based-operator 14 | labels: 15 | relaxdiego.com: python-based-operator 16 | -------------------------------------------------------------------------------- /templates/dev_kubeconfig.yml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Config 3 | clusters: 4 | - name: cluster 5 | cluster: 6 | certificate-authority-data: {{ca_crt}} 7 | server: {{server_addr}} 8 | contexts: 9 | - name: context 10 | context: 11 | cluster: cluster 12 | namespace: {{namespace}} 13 | user: user 14 | users: 15 | - name: user 16 | user: 17 | token: {{token}} 18 | current-context: context 19 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/templates/configmap.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ConfigMap 3 | metadata: 4 | name: {{ .Release.Name }}-prometheus-cluster-config 5 | labels: 6 | {{- include "prometheus.labels" . | nindent 4 }} 7 | data: 8 | prometheus.yml: | 9 | {{- /* 10 | We re-indent the literal block since strip those when parsing the values file(s) 11 | */ -}} 12 | {{ indent 4 .Values.prometheus.config }} 13 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/templates/lb-service.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Real service that load balances between pods 3 | apiVersion: v1 4 | kind: Service 5 | metadata: 6 | name: {{ .Release.Name }}-prometheus-cluster 7 | labels: 8 | {{- include "prometheus.labels" . | nindent 4 }} 9 | spec: 10 | selector: 11 | {{- include "prometheus.selectorLabels" . | nindent 4 }} 12 | ports: 13 | - protocol: TCP 14 | port: 9090 15 | targetPort: 9090 16 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/.helmignore: -------------------------------------------------------------------------------- 1 | # Patterns to ignore when building packages. 2 | # This supports shell glob matching, relative path matching, and 3 | # negation (prefixed with !). Only one pattern per line. 4 | .DS_Store 5 | # Common VCS dirs 6 | .git/ 7 | .gitignore 8 | .bzr/ 9 | .bzrignore 10 | .hg/ 11 | .hgignore 12 | .svn/ 13 | # Common backup files 14 | *.swp 15 | *.bak 16 | *.tmp 17 | *~ 18 | # Various IDEs 19 | .project 20 | .idea/ 21 | *.tmproj 22 | .vscode/ 23 | -------------------------------------------------------------------------------- /templates/operator.yml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: Deployment 4 | metadata: 5 | name: python-based-operator 6 | namespace: python-based-operator 7 | labels: 8 | relaxdiego.com: python-based-operator 9 | spec: 10 | replicas: 1 11 | selector: 12 | matchLabels: 13 | app.kubernetes.io/name: python-based-operator 14 | template: 15 | metadata: 16 | namespace: python-based-operator 17 | labels: 18 | app.kubernetes.io/name: python-based-operator 19 | spec: 20 | serviceAccountName: {{SERVICEACCOUNT_NAME}} 21 | containers: 22 | - name: python-based-operator 23 | image: "{{IMAGE_TAG}}" 24 | imagePullPolicy: Always 25 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/templates/headless-service.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Headless service that creates dns records for the backend pods allowing 3 | # an authorized entity to query .{{ .Release.Name }}.prometheus-cluster-pod-addresses 4 | # from within the namespace. If accessing from outside the namespace, append 5 | # 'svc..cluster.local' 6 | # Reference: https://kubernetes.io/docs/concepts/services-networking/service/#headless-services 7 | apiVersion: v1 8 | kind: Service 9 | metadata: 10 | name: {{ .Release.Name }}-prometheus-cluster-pod-addresses 11 | labels: 12 | {{- include "prometheus.labels" . | nindent 4 }} 13 | spec: 14 | selector: 15 | {{- include "prometheus.selectorLabels" . | nindent 4 }} 16 | clusterIP: None 17 | -------------------------------------------------------------------------------- /src/python_based_operator/logs.py: -------------------------------------------------------------------------------- 1 | import logging.config 2 | import textwrap 3 | import yaml 4 | 5 | 6 | def configure(verbosity=1): 7 | config_dict = yaml.safe_load(textwrap.dedent(""" 8 | version: 1 9 | disable_existing_loggers: False 10 | 11 | formatters: 12 | simple: 13 | format: '%(asctime)s %(name)-25s %(levelname)-7s %(message)s' 14 | 15 | handlers: 16 | stdout: 17 | class: logging.StreamHandler 18 | formatter: simple 19 | 20 | root: 21 | level: INFO 22 | handlers: 23 | - stdout""")) 24 | 25 | if verbosity > 0: 26 | config_dict['root']['level'] = 'DEBUG' 27 | else: 28 | config_dict['root']['level'] = 'INFO' 29 | 30 | logging.config.dictConfig(config_dict) 31 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/Chart.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v2 2 | name: prometheus 3 | description: A Helm chart for Prometheus 4 | 5 | # A chart can be either an 'application' or a 'library' chart. 6 | # 7 | # Application charts are a collection of templates that can be packaged into versioned archives 8 | # to be deployed. 9 | # 10 | # Library charts provide useful utilities or functions for the chart developer. They're included as 11 | # a dependency of application charts to inject those utilities and functions into the rendering 12 | # pipeline. Library charts do not define any templates and therefore cannot be deployed. 13 | type: application 14 | 15 | # This is the chart version. This version number should be incremented each time you make changes 16 | # to the chart and its templates, including the app version. 17 | version: 0.1.0 18 | 19 | # This is the version number of the application being deployed. This version number should be 20 | # incremented each time you make changes to the application. 21 | appVersion: v2.19.2 22 | -------------------------------------------------------------------------------- /src/requirements.txt: -------------------------------------------------------------------------------- 1 | # 2 | # This file is autogenerated by pip-compile 3 | # To update, run: 4 | # 5 | # make dependencies 6 | # 7 | cachetools==4.1.0 # via google-auth 8 | certifi==2020.6.20 # via kubernetes, requests 9 | chardet==3.0.4 # via requests 10 | google-auth==1.18.0 # via kubernetes 11 | idna==2.9 # via requests 12 | kubernetes==11.0.0 # via python-based-operator (setup.py) 13 | oauthlib==3.1.0 # via requests-oauthlib 14 | pyasn1-modules==0.2.8 # via google-auth 15 | pyasn1==0.4.8 # via pyasn1-modules, rsa 16 | python-dateutil==2.8.1 # via kubernetes 17 | pyyaml==5.4 # via kubernetes 18 | requests-oauthlib==1.3.0 # via kubernetes 19 | requests==2.24.0 # via kubernetes, requests-oauthlib 20 | rsa==4.7 # via google-auth 21 | six==1.15.0 # via google-auth, kubernetes, python-dateutil, websocket-client 22 | urllib3==1.26.5 # via kubernetes, requests 23 | websocket-client==0.57.0 # via kubernetes 24 | 25 | # The following packages are considered to be unsafe in a requirements file: 26 | # setuptools 27 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT LICENSE 2 | 3 | Copyright (c) 2020 Mark S. Maglana 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/setup.py: -------------------------------------------------------------------------------- 1 | from sys import version_info 2 | 3 | from setuptools import find_packages, setup 4 | 5 | minimum_python_version = (3, 8, 3) 6 | 7 | if version_info[:3] < minimum_python_version: 8 | raise RuntimeError( 9 | 'Unsupported python version {}. Please use {} or newer'.format( 10 | '.'.join(map(str, version_info[:3])), 11 | '.'.join(map(str, minimum_python_version)), 12 | ) 13 | ) 14 | 15 | 16 | _NAME = 'python-based-operator' 17 | setup( 18 | name=_NAME, 19 | version='0.2.0', 20 | packages=find_packages(), 21 | author='Mark S. Maglana', 22 | author_email='mmaglana@gmail.com', 23 | include_package_data=True, 24 | install_requires=[ 25 | 'kubernetes>=11.0.0,<11.1.0', 26 | ], 27 | entry_points={ 28 | 'console_scripts': [ 29 | f"{_NAME} = {_NAME.replace('-', '_')}.operator:main", 30 | ] 31 | }, 32 | # https://pypi.org/classifiers/ 33 | classifiers=[ 34 | "Programming Language :: Python :: 3.8", 35 | "License :: OSI Approved :: Apache Software License", 36 | "Operating System :: POSIX :: Linux", 37 | ] 38 | ) 39 | -------------------------------------------------------------------------------- /templates/rbac.yml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | kind: ClusterRole 4 | metadata: 5 | name: python-based-operator-clusterrole 6 | labels: 7 | relaxdiego.com: python-based-operator 8 | rules: 9 | - apiGroups: [""] 10 | resources: ["configmaps", "services"] 11 | verbs: ["get", "create", "delete"] 12 | 13 | # This operator uses secrets to manage app release information 14 | - apiGroups: [""] 15 | resources: ["secrets"] 16 | verbs: ["list", "get", "create", "update", "patch", "delete"] 17 | 18 | - apiGroups: ["apps"] 19 | resources: ["statefulsets"] 20 | verbs: ["get", "create", "patch", "delete"] 21 | 22 | - apiGroups: ["relaxdiego.com"] 23 | resources: ["prometheusclusters"] 24 | verbs: ["watch"] 25 | --- 26 | apiVersion: rbac.authorization.k8s.io/v1 27 | kind: ClusterRoleBinding 28 | metadata: 29 | name: python-based-operator-clusterrolebinding 30 | labels: 31 | relaxdiego.com: python-based-operator 32 | roleRef: 33 | apiGroup: rbac.authorization.k8s.io 34 | kind: ClusterRole 35 | name: python-based-operator-clusterrole 36 | subjects: 37 | - kind: ServiceAccount 38 | name: {{SERVICEACCOUNT_NAME}} 39 | namespace: python-based-operator 40 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/templates/statefulset.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: apps/v1 3 | kind: StatefulSet 4 | metadata: 5 | name: simple-prometheus 6 | spec: 7 | replicas: {{ .Values.prometheus.replicas }} 8 | selector: 9 | matchLabels: 10 | {{- include "prometheus.selectorLabels" . | nindent 6 }} 11 | serviceName: simple-prometheus-cluster-pod-addresses 12 | template: 13 | metadata: 14 | labels: 15 | {{- include "prometheus.selectorLabels" . | nindent 8 }} 16 | spec: 17 | containers: 18 | - image: {{ include "prometheus.imageTag" . }} 19 | name: prometheus 20 | ports: 21 | - containerPort: 9090 22 | name: web 23 | volumeMounts: 24 | - mountPath: /prometheus 25 | name: data 26 | - mountPath: /etc/prometheus 27 | name: prometheus-config 28 | terminationGracePeriodSeconds: 10 29 | volumes: 30 | - configMap: 31 | items: 32 | - key: prometheus.yml 33 | path: prometheus.yml 34 | name: {{ .Release.Name }}-prometheus-cluster-config 35 | name: prometheus-config 36 | volumeClaimTemplates: 37 | - metadata: 38 | name: data 39 | spec: 40 | accessModes: 41 | - ReadWriteOnce 42 | resources: 43 | requests: 44 | storage: 1Gi 45 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # 2 | # TESTER STAGE 3 | # 4 | 5 | FROM python:3.8.3-alpine3.12 as tester 6 | 7 | WORKDIR /operator-src 8 | 9 | ADD ./src/ . 10 | ADD .flake8 . 11 | 12 | RUN apk update 13 | RUN pip install --upgrade pip 14 | RUN pip install -r requirements-dev.txt 15 | 16 | # TODO: Run unit tests here 17 | 18 | # TODO: Submit test artifacts somewhere 19 | 20 | # 21 | # BUILDER STAGE 22 | # 23 | 24 | FROM python:3.8.3-alpine3.12 as builder 25 | 26 | WORKDIR /operator-src 27 | 28 | COPY --from=tester /operator-src/ . 29 | 30 | RUN apk update 31 | # Reference: https://pythonspeed.com/articles/activate-virtualenv-dockerfile/ 32 | ENV VIRTUAL_ENV=/operator 33 | RUN python3 -m venv $VIRTUAL_ENV 34 | ENV PATH="$VIRTUAL_ENV/bin:$PATH" 35 | RUN pip install --upgrade pip 36 | # We are using requirements.txt to constrain the dependency versions to the 37 | # ones that we have tested with so as the lessen the build and deployment 38 | # variables and make our deployments more deterministic. 39 | RUN pip install -c requirements.txt . 40 | 41 | # Install Helm 3 42 | RUN apk add wget 43 | RUN wget https://get.helm.sh/helm-v3.2.4-linux-amd64.tar.gz -O /tmp/helm.tar.gz 2>&1 44 | RUN mkdir -p /tmp/helm 45 | RUN tar -xvf /tmp/helm.tar.gz -C /tmp/helm 46 | RUN cp /tmp/helm/linux-amd64/helm $VIRTUAL_ENV/bin 47 | 48 | # 49 | # FINAL STAGE 50 | # 51 | 52 | FROM python:3.8.3-alpine3.12 53 | 54 | # Copy the virtual environment only since it has all that we need and 55 | # none of the cruft. 56 | WORKDIR /operator 57 | 58 | COPY --from=builder /operator/ . 59 | 60 | ENV VIRTUAL_ENV=/operator 61 | ENV PATH="$VIRTUAL_ENV/bin:$PATH" 62 | 63 | # This is based on the package name declared in src/setup.py 64 | ENTRYPOINT ["python-based-operator"] 65 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/values.yaml: -------------------------------------------------------------------------------- 1 | # Default values for prometheus-operator. 2 | # This is a YAML-formatted file. 3 | # Declare variables to be passed into your templates. 4 | 5 | replicaCount: 1 6 | 7 | image: 8 | repository: prom/prometheus 9 | pullPolicy: ifNotPresent 10 | 11 | prometheus: 12 | imageName: 'prom/prometheus' 13 | replicas: 1 14 | config: "" 15 | 16 | imagePullSecrets: [] 17 | nameOverride: "" 18 | fullnameOverride: "" 19 | 20 | serviceAccount: 21 | # Specifies whether a service account should be created 22 | create: true 23 | # The name of the service account to use. 24 | # If not set and create is true, a name is generated using the fullname template 25 | name: 26 | 27 | podSecurityContext: {} 28 | # fsGroup: 2000 29 | 30 | securityContext: {} 31 | # capabilities: 32 | # drop: 33 | # - ALL 34 | # readOnlyRootFilesystem: true 35 | # runAsNonRoot: true 36 | # runAsUser: 1000 37 | 38 | service: 39 | type: ClusterIP 40 | port: 80 41 | 42 | ingress: 43 | enabled: false 44 | annotations: {} 45 | # kubernetes.io/ingress.class: nginx 46 | # kubernetes.io/tls-acme: "true" 47 | hosts: 48 | - host: chart-example.local 49 | paths: [] 50 | tls: [] 51 | # - secretName: chart-example-tls 52 | # hosts: 53 | # - chart-example.local 54 | 55 | resources: {} 56 | # We usually recommend not to specify default resources and to leave this as a conscious 57 | # choice for the user. This also increases chances charts run on environments with little 58 | # resources, such as Minikube. If you do want to specify resources, uncomment the following 59 | # lines, adjust them as necessary, and remove the curly braces after 'resources:'. 60 | # limits: 61 | # cpu: 100m 62 | # memory: 128Mi 63 | # requests: 64 | # cpu: 100m 65 | # memory: 128Mi 66 | 67 | nodeSelector: {} 68 | 69 | tolerations: [] 70 | 71 | affinity: {} 72 | -------------------------------------------------------------------------------- /src/requirements-dev.txt: -------------------------------------------------------------------------------- 1 | # 2 | # This file is autogenerated by pip-compile 3 | # To update, run: 4 | # 5 | # make dependencies 6 | # 7 | -e file:.#egg=python-based-operator # via -r requirements-dev.in 8 | # via -r requirements-dev.in 9 | attrs==19.3.0 10 | # via pytest 11 | cachetools==4.1.0 12 | # via 13 | # -c requirements.txt 14 | # google-auth 15 | certifi==2020.6.20 16 | # via 17 | # -c requirements.txt 18 | # kubernetes 19 | # requests 20 | chardet==3.0.4 21 | # via 22 | # -c requirements.txt 23 | # requests 24 | google-auth==1.18.0 25 | # via 26 | # -c requirements.txt 27 | # kubernetes 28 | idna==2.9 29 | # via 30 | # -c requirements.txt 31 | # requests 32 | kubernetes==11.0.0 33 | # via 34 | # -c requirements.txt 35 | # sanitized-package 36 | more-itertools==8.4.0 37 | # via pytest 38 | oauthlib==3.1.0 39 | # via 40 | # -c requirements.txt 41 | # requests-oauthlib 42 | packaging==20.4 43 | # via pytest 44 | pluggy==0.13.1 45 | # via pytest 46 | py==1.10.0 47 | # via pytest 48 | pyasn1-modules==0.2.8 49 | # via 50 | # -c requirements.txt 51 | # google-auth 52 | pyasn1==0.4.8 53 | # via 54 | # -c requirements.txt 55 | # pyasn1-modules 56 | # rsa 57 | pyparsing==2.4.7 58 | # via packaging 59 | pytest==5.4.3 60 | # via -r requirements-dev.in 61 | python-dateutil==2.8.1 62 | # via 63 | # -c requirements.txt 64 | # kubernetes 65 | pyyaml==5.3.1 66 | # via 67 | # -c requirements.txt 68 | # kubernetes 69 | requests-oauthlib==1.3.0 70 | # via 71 | # -c requirements.txt 72 | # kubernetes 73 | requests==2.24.0 74 | # via 75 | # -c requirements.txt 76 | # kubernetes 77 | # requests-oauthlib 78 | rsa==4.6 79 | # via 80 | # -c requirements.txt 81 | # google-auth 82 | six==1.15.0 83 | # via 84 | # -c requirements.txt 85 | # google-auth 86 | # kubernetes 87 | # packaging 88 | # python-dateutil 89 | # websocket-client 90 | urllib3==1.25.9 91 | # via 92 | # -c requirements.txt 93 | # kubernetes 94 | # requests 95 | wcwidth==0.2.5 96 | # via pytest 97 | websocket-client==0.57.0 98 | # via 99 | # -c requirements.txt 100 | # kubernetes 101 | 102 | # The following packages are considered to be unsafe in a requirements file: 103 | # setuptools 104 | -------------------------------------------------------------------------------- /src/python_based_operator/charts/prometheus/templates/_helpers.tpl: -------------------------------------------------------------------------------- 1 | {{/* vim: set filetype=mustache: */}} 2 | {{/* 3 | Expand the name of the chart. 4 | */}} 5 | {{- define "prometheus.name" -}} 6 | {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} 7 | {{- end -}} 8 | 9 | {{ default false .Values.dev }} 10 | 11 | {{/* 12 | Create a default fully qualified app name. 13 | We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). 14 | If release name contains chart name it will be used as a full name. 15 | */}} 16 | {{- define "prometheus.fullname" -}} 17 | {{- if .Values.fullnameOverride -}} 18 | {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} 19 | {{- else -}} 20 | {{- $name := default .Chart.Name .Values.nameOverride -}} 21 | {{- if contains $name .Release.Name -}} 22 | {{- .Release.Name | trunc 63 | trimSuffix "-" -}} 23 | {{- else -}} 24 | {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} 25 | {{- end -}} 26 | {{- end -}} 27 | {{- end -}} 28 | 29 | {{- define "prometheus.imageTag" -}} 30 | {{- printf "%s:%s" .Values.prometheus.imageName .Chart.AppVersion -}} 31 | {{- end -}} 32 | 33 | {{/* 34 | Create chart name and version as used by the chart label. 35 | */}} 36 | {{- define "prometheus.chart" -}} 37 | {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} 38 | {{- end -}} 39 | 40 | {{/* 41 | Common labels 42 | Reference: https://helm.sh/docs/chart_best_practices/labels/ 43 | */}} 44 | {{- define "prometheus.labels" -}} 45 | helm.sh/chart: {{ include "prometheus.chart" . }} 46 | {{ include "prometheus.selectorLabels" . }} 47 | {{- if .Chart.AppVersion }} 48 | app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} 49 | {{- end }} 50 | app.kubernetes.io/managed-by: {{ .Release.Service }} 51 | {{- end -}} 52 | 53 | {{/* 54 | Selector labels 55 | Reference: https://helm.sh/docs/chart_best_practices/labels/ 56 | Be careful when modifying this. See https://helm.sh/docs/chart_best_practices/pods/#podtemplates-should-declare-selectors 57 | */}} 58 | {{- define "prometheus.selectorLabels" -}} 59 | app.kubernetes.io/name: {{ include "prometheus.name" . }} 60 | app.kubernetes.io/instance: {{ .Release.Name }} 61 | {{- end -}} 62 | 63 | {{/* 64 | Create the name of the service account to use 65 | */}} 66 | {{- define "prometheus.serviceAccountName" -}} 67 | {{- if .Values.serviceAccount.create -}} 68 | {{ default (include "prometheus.fullname" .) .Values.serviceAccount.name }} 69 | {{- else -}} 70 | {{ default "default" .Values.serviceAccount.name }} 71 | {{- end -}} 72 | {{- end -}} 73 | -------------------------------------------------------------------------------- /templates/crd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # References 3 | # 1. Guide: 4 | # https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/ 5 | # 6 | # 2. API Reference: 7 | # https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#customresourcedefinition-v1-apiextensions-k8s-io 8 | apiVersion: apiextensions.k8s.io/v1 9 | kind: CustomResourceDefinition 10 | metadata: 11 | # name must match the spec fields below, and be in the form: . 12 | name: prometheusclusters.relaxdiego.com 13 | labels: 14 | relaxdiego.com: python-based-operator 15 | spec: 16 | # group name to use for REST API: /apis// 17 | group: relaxdiego.com 18 | # either Namespaced or Cluster 19 | scope: Namespaced 20 | names: 21 | # plural name to be used in the URL: /apis/// 22 | plural: prometheusclusters 23 | # singular name to be used as an alias on the CLI and for display 24 | singular: prometheuscluster 25 | # kind is normally the CamelCased singular type. Your resource manifests use this. 26 | kind: PrometheusCluster 27 | # shortNames allow shorter string to match your resource on the CLI 28 | shortNames: 29 | - proms 30 | - promcluster 31 | - promclusters 32 | - prometheuses 33 | - prometheis 34 | # list of versions supported by this CustomResourceDefinition 35 | versions: 36 | - name: v1alpha1 37 | # Each version can be enabled/disabled by Served flag. 38 | served: true 39 | # Indicates whether this version should be used when persisting custom 40 | # resources to storage. One and only one version must be marked as 41 | # the storage version. 42 | storage: true 43 | schema: 44 | openAPIV3Schema: 45 | type: object 46 | required: 47 | - spec 48 | properties: 49 | # This is just a JSON Schema spec. Not too comfy with JSON Schemas? 50 | # Here's a starter: https://json-schema.org/learn/getting-started-step-by-step.html 51 | spec: 52 | type: object 53 | properties: 54 | config: 55 | type: string 56 | default: | 57 | # Reference: https://prometheus.io/docs/prometheus/latest/configuration/configuration/ 58 | scrape_configs: 59 | - job_name: 'prometheus' 60 | static_configs: 61 | - targets: ['localhost:9090'] 62 | description: Raw prometheus configuration file contents. See the official 63 | Prometheus documentation at https://prometheus.io/docs/prometheus/latest/configuration/configuration 64 | replicas: 65 | type: integer 66 | default: 1 67 | minimum: 1 68 | description: The desired number of running pods 69 | -------------------------------------------------------------------------------- /scripts/render: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # Copyright (c) Mark Maglana 4 | # Git repo at https://github.com/relaxdiego/renderest 5 | # Inspired by https://github.com/johanhaleby/bash-templater 6 | # and https://github.com/lavoiesl/bash-templater 7 | # 8 | # Permission is hereby granted, free of charge, to any person obtaining a copy 9 | # of this software and associated documentation files (the "Software"), to deal 10 | # in the Software without restriction, including without limitation the rights 11 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | # copies of the Software, and to permit persons to whom the Software is 13 | # furnished to do so, subject to the following conditions: 14 | # 15 | # The above copyright notice and this permission notice shall be included in all 16 | # copies or substantial portions of the Software. 17 | # 18 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | # SOFTWARE. 25 | set -eu 26 | 27 | no_empty=false 28 | error=false 29 | 30 | case $1 in 31 | -e|--no-empty) 32 | no_empty=true 33 | shift 34 | ;; 35 | esac 36 | 37 | # Get all variable names used in the template 38 | varnames=$(grep -oE '\{\{([A-Za-z0-9_]+)\}\}' $1 | sed -rn 's/.*\{\{([A-Za-z0-9_]+)\}\}.*/\1/p' | sort | uniq) 39 | 40 | # Initialize the string of expressions to be used with sed 41 | expressions="" 42 | 43 | for varname in $varnames; do 44 | # Check if the variable named $varname is defined 45 | if [ -z ${!varname+x} ]; then 46 | if [[ $error = false ]]; then 47 | echo "Error found in file $1:" >&2 48 | fi 49 | echo "ERROR: $varname is not defined" >&2 50 | error=true 51 | elif [ $no_empty = true ] && [ -z ${!varname} ]; then 52 | if [[ $error = false ]]; then 53 | echo "Error found in file $1:" >&2 54 | fi 55 | echo "ERROR: $varname is empty" >&2 56 | error=true 57 | else 58 | # Get the value of the variable named $varname and escape 59 | # all forward slashes so that it doesn't break sed 60 | value="$(echo "${!varname}" | sed 's/\//\\\//g')" 61 | expressions+="-e 's/\{\{$varname\}\}/${value}/g' " 62 | fi 63 | done 64 | 65 | if [[ $error = true ]]; then 66 | exit 1 67 | elif [[ -n $expressions ]]; then 68 | cat $1 | eval sed -r "$expressions" 69 | exit 0 70 | else 71 | cat $1 72 | exit 0 73 | fi 74 | -------------------------------------------------------------------------------- /src/python_based_operator/operator.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | from pathlib import Path 4 | import sys 5 | 6 | from kubernetes import ( 7 | client, 8 | config, 9 | watch, 10 | ) 11 | 12 | from python_based_operator import ( 13 | logs, 14 | ) 15 | from python_based_operator.provisioners import ( 16 | v1alpha1, 17 | ) 18 | 19 | logs.configure() 20 | log = logging.getLogger(__name__) 21 | 22 | 23 | def main(): 24 | load_kube_credentials() 25 | api_version = os.environ.get('PROMETHEUS_CLUSTER_CRD_VERSION_TO_WATCH', 'v1alpha1') 26 | watch_prometheusclusters(api_version=api_version) 27 | 28 | 29 | def load_kube_credentials(): 30 | log.debug("Looking for credentials...") 31 | user_kubeconfig = Path(os.path.expanduser("~")).joinpath('.kube', 'config') 32 | dev_kubeconfig = Path(__file__).joinpath('..', '..', '..', 33 | '.tmp', 'serviceaccount', 34 | 'dev_kubeconfig.yml').resolve() 35 | 36 | if dev_kubeconfig.exists(): 37 | log.debug("Loading from dev kube config") 38 | config.load_kube_config(config_file=str(dev_kubeconfig)) 39 | elif user_kubeconfig.exists(): 40 | log.debug("Loading user kube config") 41 | config.load_kube_config() 42 | else: 43 | log.debug("Loading in-cluster kube config") 44 | try: 45 | config.load_incluster_config() 46 | except config.ConfigException: 47 | log.error("Unable to load in-cluster config file. Exiting.") 48 | sys.exit(1) 49 | 50 | 51 | def watch_prometheusclusters(api_version): 52 | log = logging.getLogger(__name__) 53 | log.debug("Loading CustomObjectsApi client") 54 | # https://github.com/kubernetes-client/python/blob/v11.0.0/kubernetes/client/api/custom_objects_api.py 55 | coa_client = client.CustomObjectsApi() 56 | 57 | api_group = 'relaxdiego.com' 58 | crd_name = 'prometheusclusters' 59 | 60 | log.info(f"Watching {crd_name}.{api_group}/{api_version} events") 61 | # References: 62 | # 1. Watchable methods: 63 | # https://raw.githubusercontent.com/kubernetes-client/python/v11.0.0/kubernetes/client/api/core_v1_api.py 64 | # https://github.com/kubernetes-client/python/blob/master/kubernetes/client/api/apiextensions_v1_api.py 65 | # 66 | # 2. The Watch#stream() method 67 | # https://github.com/kubernetes-client/python-base/blob/d30f1e6fd4e2725aae04fa2f4982a4cfec7c682b/watch/watch.py#L107-L157 68 | for event in watch.Watch().stream(coa_client.list_cluster_custom_object, 69 | group=api_group, 70 | plural=crd_name, 71 | version=api_version): 72 | custom_obj = event['raw_object'] 73 | log.debug(f"Received: {custom_obj}") 74 | event_type = event['type'] 75 | 76 | if api_version == 'v1alpha1': 77 | provisioner = v1alpha1 78 | 79 | pco = provisioner.PrometheuClusterObject(**custom_obj) 80 | log.info(f"{event_type} {pco}") 81 | 82 | if event_type == "ADDED": 83 | provisioner.install(pco) 84 | elif event_type == "DELETED": 85 | provisioner.uninstall(pco) 86 | elif event_type == "MODIFIED": 87 | provisioner.upgrade(pco) 88 | else: 89 | log.info(f"Unhandled event type '{event_type}'") 90 | 91 | 92 | if __name__ == "__main__": 93 | main() 94 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: dependencies image deploy deploy-common deploy-dev reset uninstall 2 | .DEFAULT_GOAL := image 3 | 4 | tag ?= "" 5 | 6 | export IMAGE_TAG = ${tag} 7 | export SERVICEACCOUNT_NAME = "python-based-operator-serviceaccount" 8 | 9 | dependencies: .last-pip-tools-install src/requirements-dev.txt .last-pip-sync 10 | 11 | image: .last-docker-build 12 | @echo -n 13 | 14 | deploy: deploy-common .last-docker-build .last-docker-push 15 | @scripts/render templates/operator.yml > .tmp/yml/operator.yml || exit 1 16 | @kubectl apply -f .tmp/yml/ 17 | 18 | deploy-common: 19 | @rm -rf .tmp/yml && mkdir -p .tmp/yml 20 | @scripts/render templates/crd.yml > .tmp/yml/crd.yml || exit 1 21 | @scripts/render templates/ns.yml > .tmp/yml/ns.yml || exit 1 22 | @scripts/render templates/rbac.yml > .tmp/yml/rbac.yml || exit 1 23 | 24 | deploy-dev: deploy-common .last-pip-sync 25 | @kubectl apply -f .tmp/yml/ 26 | @rm -rf .tmp/serviceaccount && mkdir -p .tmp/serviceaccount 27 | @kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " " \ 28 | | xargs printf "export server_addr=%s\n" > .tmp/serviceaccount/generate_dev_kubeconfig.sh 29 | @kubectl get serviceaccount python-based-operator-serviceaccount -n python-based-operator -o go-template='{{ (index .secrets 0).name }}' \ 30 | | xargs -n1 kubectl get secret -n python-based-operator -o go-template='{{ (index .data "token") }}' \ 31 | | base64 --decode \ 32 | | xargs printf "export token=%s\n" >> .tmp/serviceaccount/generate_dev_kubeconfig.sh 33 | @kubectl get serviceaccount python-based-operator-serviceaccount -n python-based-operator -o go-template='{{ (index .secrets 0).name }}' \ 34 | | xargs -n1 kubectl get secret -n python-based-operator -o go-template='{{ (index .data "namespace") }}' \ 35 | | base64 --decode \ 36 | | xargs printf "export namespace=%s\n" >> .tmp/serviceaccount/generate_dev_kubeconfig.sh 37 | @kubectl get serviceaccount python-based-operator-serviceaccount -n python-based-operator -o go-template='{{ (index .secrets 0).name }}' \ 38 | | xargs -n1 kubectl get secret -n python-based-operator -o go-template='{{ (index .data "ca.crt") }}' \ 39 | | xargs -n1 printf "export ca_crt=%s\n" >> .tmp/serviceaccount/generate_dev_kubeconfig.sh 40 | @echo "scripts/render templates/dev_kubeconfig.yml" >> .tmp/serviceaccount/generate_dev_kubeconfig.sh 41 | @chmod +x .tmp/serviceaccount/generate_dev_kubeconfig.sh 42 | @.tmp/serviceaccount/generate_dev_kubeconfig.sh > .tmp/serviceaccount/dev_kubeconfig.yml 43 | @python-based-operator 44 | 45 | reset: uninstall 46 | @rm -v -f .last-* 47 | 48 | uninstall: 49 | @kubectl delete -f .tmp/yml/ || true 50 | 51 | .last-docker-build: Dockerfile LICENSE src/MANIFEST.in src/**/* src/requirements.txt src/requirements-dev.txt 52 | docker build -t ${tag} . 2>&1 | tee .last-docker-build 53 | @(grep -E "(Error response from daemon|returned a non-zero code)" .last-docker-build 1>/dev/null && rm -f .last-docker-build && echo "Error building image" && exit 1) || exit 0 54 | 55 | .last-docker-push: .last-docker-build 56 | @(test -n ${tag} && echo "Using image: ${tag}") || \ 57 | (echo "The tag argument is missing. See README for guidance" && exit 1) 58 | @test -f .last-docker-build || (echo "Last container image build was unsuccessful. Exiting." && exit 1) 59 | docker push ${tag} | tee .last-docker-push 60 | 61 | .last-pip-sync: .last-pip-tools-install src/requirements-dev.txt src/requirements.txt 62 | cd src && pip-sync requirements-dev.txt requirements.txt | tee ../.last-pip-sync 63 | (pyenv -v && pyenv rehash) || true 64 | 65 | .last-pip-tools-install: 66 | @(pip-compile --version 1>/dev/null 2>&1 || pip --disable-pip-version-check install "pip-tools>=5.3.0,<5.4" || echo "pip-tools install error") | tee .last-pip-tools-install 67 | @(grep "pip-tools install error" .last-pip-tools-install 1>/dev/null 2>&1 && rm -f .last-pip-tools-install && exit 1) || true 68 | (pyenv -v && pyenv rehash) || true 69 | 70 | src/requirements-dev.txt: .last-pip-tools-install src/requirements-dev.in src/requirements.txt 71 | cd src && CUSTOM_COMPILE_COMMAND="make dependencies" pip-compile requirements-dev.in 72 | 73 | src/requirements.txt: .last-pip-tools-install src/setup.py 74 | cd src && CUSTOM_COMPILE_COMMAND="make dependencies" pip-compile 75 | -------------------------------------------------------------------------------- /src/python_based_operator/provisioners/v1alpha1.py: -------------------------------------------------------------------------------- 1 | from dataclasses import dataclass, field, InitVar 2 | import logging 3 | from pathlib import Path 4 | import shutil 5 | import subprocess 6 | import tempfile 7 | import yaml 8 | 9 | from kubernetes import ( 10 | client, 11 | ) 12 | from kubernetes.client import ( 13 | models 14 | ) 15 | 16 | log = logging.getLogger(__name__) 17 | 18 | 19 | PROMETHEUS_ADVERTISED_PORT = 9090 20 | 21 | 22 | # MODELS 23 | 24 | @dataclass 25 | class MetadataField: 26 | creationTimestamp: str 27 | generation: int 28 | managedFields: list 29 | name: str 30 | namespace: str 31 | resourceVersion: str 32 | selfLink: str 33 | uid: str 34 | annotations: dict 35 | 36 | @dataclass 37 | class CustomResourceObject: 38 | apiVersion: str 39 | kind: str 40 | metadata: MetadataField 41 | 42 | def __post_init__(self): 43 | self.metadata = MetadataField(**self.metadata) 44 | 45 | 46 | @dataclass 47 | class PrometheusClusterObjectSpecField: 48 | replicas: int 49 | config: str 50 | 51 | 52 | @dataclass() 53 | class PrometheuClusterObject(CustomResourceObject): 54 | spec: PrometheusClusterObjectSpecField 55 | 56 | def __post_init__(self): 57 | super().__post_init__() 58 | self.spec = PrometheusClusterObjectSpecField(**self.spec) 59 | 60 | def __str__(self): 61 | return f"{self.kind} {self.apiVersion} " \ 62 | f"ns={self.metadata.namespace} name={self.metadata.name}" 63 | 64 | 65 | # BUSINESS LOGIC 66 | 67 | def install(pco: PrometheuClusterObject): 68 | cluster_name = pco.metadata.name 69 | 70 | with tempfile.TemporaryDirectory(prefix="prometheus-operator-") as tmpdir: 71 | values_yaml = Path(tmpdir).joinpath("values.yaml") 72 | values_yaml.write_text(yaml.dump({ 73 | 'prometheus': { 74 | 'replicas': pco.spec.replicas, 75 | 'config': pco.spec.config 76 | } 77 | })) 78 | 79 | success = \ 80 | _helm([ 81 | "install", 82 | "--atomic", 83 | "--wait", 84 | "--timeout=3m0s", 85 | "--values", str(values_yaml.resolve()), 86 | f"--namespace={pco.metadata.namespace}", 87 | cluster_name, 88 | Path(__file__).joinpath('..', '..', 'charts', 'prometheus').resolve(), 89 | ]) 90 | 91 | if success: 92 | log.info(f"Succesfully installed Prometheus Cluster '{cluster_name}' " 93 | f"in namespace '{pco.metadata.namespace}'") 94 | else: 95 | log.error(f"Failed to install Prometheus cluster '{cluster_name}' " 96 | f"in namespace '{pco.metadata.namespace}'") 97 | 98 | def uninstall(pco: PrometheuClusterObject): 99 | cluster_name = pco.metadata.name 100 | 101 | success = _helm([ 102 | "uninstall", 103 | f"--namespace={pco.metadata.namespace}", 104 | cluster_name, 105 | ]) 106 | 107 | if success: 108 | log.info(f"Succesfully uninstalled Prometheus Cluster '{cluster_name}' " 109 | f"in namespace '{pco.metadata.namespace}'") 110 | else: 111 | log.error(f"Failed to uninstall Prometheus cluster '{cluster_name}' " 112 | f"in namespace '{pco.metadata.namespace}'") 113 | 114 | def upgrade(pco: PrometheuClusterObject): 115 | cluster_name = pco.metadata.name 116 | 117 | with tempfile.TemporaryDirectory(prefix="prometheus-operator-") as tmpdir: 118 | values_yaml = Path(tmpdir).joinpath("values.yaml") 119 | values_yaml.write_text(yaml.dump({ 120 | 'prometheus': { 121 | 'replicas': pco.spec.replicas, 122 | 'config': pco.spec.config 123 | } 124 | })) 125 | 126 | success = \ 127 | _helm([ 128 | "upgrade", 129 | "--atomic", 130 | "--wait", 131 | "--timeout=3m0s", 132 | "--values", str(values_yaml.resolve()), 133 | f"--namespace={pco.metadata.namespace}", 134 | cluster_name, 135 | Path(__file__).joinpath('..', '..', 'charts', 'prometheus').resolve(), 136 | ]) 137 | 138 | if success: 139 | log.info(f"Succesfully upgraded Prometheus Cluster '{cluster_name}' " 140 | f"in namespace '{pco.metadata.namespace}'") 141 | else: 142 | log.error(f"Failed to upgrade Prometheus cluster '{cluster_name}' " 143 | f"in namespace '{pco.metadata.namespace}'") 144 | 145 | # HELPERS 146 | 147 | def _helm(args_list): 148 | cmd = [shutil.which('helm')] + [str(arg) for arg in args_list] 149 | log.debug(f"Running: {' '.join(cmd)}") 150 | output = subprocess.run(cmd, capture_output=True) 151 | log.debug(output) 152 | return output.returncode == 0 153 | 154 | # Add a representer to format literal block strings properly when dumping 155 | # Reference: https://stackoverflow.com/a/50519774/402145 156 | def _selective_representer(dumper, data): 157 | return dumper.represent_scalar(u"tag:yaml.org,2002:str", data, 158 | style="|" if "\n" in data else None) 159 | 160 | yaml.add_representer(str, _selective_representer) 161 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # A Python-Based k8s Operator 2 | 3 | This project demonstrates how you can use plain Python to create a 4 | fully-functional k8s operator. To avoid re-inventing the wheel, Helm 3 is 5 | used internally by the operator to maintain the releases of the 6 | application it manages. However, if you want to customize this project 7 | to fit your needs and if your needs don't include Helm 3, you may safely 8 | remove that requirement from the code. 9 | 10 | This operator can manage multiple instances of Prometheus. You can modify 11 | it to manage other types of applications if you wish. Prometheus was just 12 | chosen in this case because most engineers in the DevOps space are already 13 | familiar with it. 14 | 15 | You instantiate Prometheus in a namespace by creating a PrometheusCluster 16 | custom resource in said namespace. A simple instance with defaults can be 17 | created via the following custom resource: 18 | 19 | ```yaml 20 | apiVersion: relaxdiego.com/v1alpha1 21 | kind: PrometheusCluster 22 | metadata: 23 | name: simple-prometheus-instance 24 | spec: {} 25 | ``` 26 | 27 | ## Demo 28 | 29 | Watch it [on YouTube](https://youtu.be/lsuW9XGWosQ). 30 | 31 | 32 | ## Prior Art 33 | 34 | Inspired by [this Medium article](https://link.medium.com/rC0Nqcrgw7) 35 | 36 | 37 | ## Dependencies 38 | 39 | 1. Kubernetes 1.18 or higher 40 | 2. [Docker CE](https://docs.docker.com/engine/install/) 17.06 or higher (For building the operator image) 41 | 3. GNU Make 42 | 43 | 44 | ## Optionally Use Microk8s 45 | 46 | Use [microk8s](https://microk8s.io/) for testing this operator. It will 47 | make your life so much easier. Go on, I'll wait! 48 | 49 | Once you have microk8s installed, run the following: 50 | 51 | ``` 52 | microk8s.enable dns rbac ingress registry storage 53 | mkdir -p ~/.kube 54 | microk8s.config > ~/.kube/config 55 | kubectl cluster-info 56 | ``` 57 | 58 | ## Try It Out! 59 | 60 | #### Build and Deploy the Operator 61 | 62 | The following will build the image and deploy it in the `python-based-operator` 63 | namespace. 64 | 65 | ``` 66 | make image deploy tag=localhost:32000/python-based-operator 67 | ``` 68 | 69 | NOTE: The address `localhost:32000` is the address of the microk8s registry 70 | addon that we enabled in the previous step. If you're not using microk8s, 71 | just replace that address with either another registry address that you 72 | have access to, or your Docker Hub username. 73 | 74 | 75 | #### Create Your First Prometheus Cluster 76 | 77 | There are sample PrometheusCluster files under the `examples/` directory. After 78 | deploying the operator, create a sample Prometheus cluster via kubectl: 79 | 80 | ``` 81 | kubectl create ns simple-prometheus-cluster 82 | kubectl config set-context --current --namespace=simple-prometheus-cluster 83 | kubectl apply -f examples/simple.yaml 84 | ``` 85 | 86 | #### Scale Up Your Prometheus Cluster 87 | 88 | ``` 89 | kubectl edit -f examples/simple.yaml 90 | ``` 91 | 92 | Go to the `replicas:` field and change its value. Quit, save, then see your 93 | number of prometheus pods scale accordingly. 94 | 95 | 96 | #### Delete the Prometheus Cluster While Retaining its Data 97 | 98 | Just run: 99 | 100 | ``` 101 | kubectl delete -f examples/simple.yaml 102 | ``` 103 | 104 | The volumes assocated with the pods will be retained and will be re-attached to 105 | the correct pod later on if you want to revive them. 106 | 107 | 108 | #### Delete the Operator and Everything in the Example Namespace 109 | 110 | ``` 111 | kubectl delete -f examples/simple.yaml 112 | make uninstall 113 | kubectl delete ns simple-prometheus-cluster 114 | ``` 115 | 116 | 117 | ## Development Guide 118 | 119 | 120 | #### Dependencies 121 | 122 | 1. Python 3.8.x or higher 123 | 124 | TIP: Make your life easier by using [pyenv](https://github.com/pyenv/pyenv-installer) 125 | 126 | 127 | #### Prepare Your Virtualenv (venv style) 128 | 129 | ``` 130 | python3 -m venv --prompt prometheus-operator .venv 131 | source .venv/bin/activate 132 | ``` 133 | 134 | #### Prepare Your Virtualenv (pyenv-virtual style) 135 | 136 | ``` 137 | pyenv virtualenv 3.8.3 prometheus-operator-3.8.3 138 | ``` 139 | 140 | More on [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv): 141 | 142 | 143 | #### Install Development Dependencies 144 | 145 | ``` 146 | make dependencies 147 | ``` 148 | 149 | 150 | #### Add a Development Dependency 151 | 152 | ``` 153 | echo 'foo' >> src/requirements-dev.in 154 | make dependencies 155 | ``` 156 | 157 | The `src/requirements-dev.txt` file should now be updated and the `foo` 158 | package installed in your local machine. Make sure to commit both files 159 | to the repo to let your teammates know of the new dependency. 160 | 161 | ``` 162 | git add src/requirements-dev.* 163 | git commit -m "Add foo to src/requirements-dev.txt" 164 | git push origin 165 | ``` 166 | 167 | 168 | #### Add a Runtime Dependency 169 | 170 | Add it to the `install_requires` argument of the `setup()` call in 171 | `src/setup.py`. For example: 172 | 173 | ``` 174 | setup( 175 | name=_NAME, 176 | version='0.1.0', 177 | 178 | ... 179 | 180 | install_requires=[ 181 | 'kubernetes>=11.0.0,<11.1.0', 182 | 'bar>=1.0.0,<2.0.0' 183 | ], 184 | 185 | ... 186 | 187 | ) 188 | ``` 189 | 190 | After having added the `bar` dependency above, run the following: 191 | 192 | ``` 193 | make dependencies 194 | ``` 195 | 196 | The `src/requirements.txt` file should now be updated and the bar package 197 | installed in your local machine. Make sure to commit both files to the repo 198 | to let your teammates know of the new dependency. 199 | 200 | ``` 201 | git add src/setup.py src/requirements.txt 202 | git commit -m "Add bar as a runtime dependency" 203 | git push origin 204 | ``` 205 | 206 | #### Run the Operator in Development Mode (Experimental) 207 | 208 | This mode speeds up your development workflow by skipping the image creation 209 | process, opting instead for to deploying it directly on your local machine. 210 | To achieve a near-production runtime environment, we will create all the 211 | resources in the k8s cluster except for the Deployment resource. Furthermore 212 | this mode auto-generates a kubeconfig file from the Service Account we create 213 | in the cluster so that the operator will still be constrained by the RBAC rules 214 | that we specify under `templates/rbac.yml`. 215 | 216 | In order for Dev mode to work properly, we have to ensure that all runtime 217 | dependencies are installed locally. This includes [Helm 3](https://helm.sh/docs/intro/install/). 218 | Make sure to install that before proceeding. You do not need to manually install 219 | the requirements in `src/requirements.txt` since that will be done for you 220 | automatically. 221 | 222 | When all requirements are satisfied, go ahead and run: 223 | 224 | ``` 225 | make deploy-dev 226 | ``` 227 | 228 | You should see something like the following in the terminal: 229 | 230 | ``` 231 | python_based_operator.operator DEBUG Looking for credentials... 232 | python_based_operator.operator DEBUG Loading from dev kube config 233 | python_based_operator.operator DEBUG Loading CustomObjectsApi client 234 | python_based_operator.operator INFO Watching prometheusclusters.relaxdiego.com/v1alpha1 events 235 | ``` 236 | 237 | If you need to make changes to the code, just press `Ctrl-C`, edit the code, 238 | then run `make deploy-dev` again. 239 | 240 | If you need something more streamlined than this, [Okteto](https://okteto.com/blog/how-to-develop-python-apps-in-kubernetes/) 241 | might be something of interest to you. 242 | 243 | #### Force Re-Install Depedencies and Uninstall the Operator 244 | 245 | Run the following 246 | 247 | ``` 248 | make reset 249 | make dependencies 250 | ``` 251 | 252 | If you want something more thorough (and potentially destructive) than that, 253 | delete your virtual environment. Then start from the beginning of the 254 | Development Guide section. 255 | --------------------------------------------------------------------------------