├── .gitignore ├── README.md ├── azure ├── aks-iaac │ ├── README.md │ ├── main.tf │ ├── makefile │ ├── output.tf │ ├── providers.tf │ ├── terraform.tfvars │ └── variables.tf └── readme.md ├── docker ├── readme.md └── ubuntu-docker ├── gitlab ├── ci │ └── readme.md ├── preview-environment │ └── readme.md └── readme.md ├── kubernetes ├── argocd │ ├── argocd-hooks-example │ │ ├── argocd │ │ │ └── product-service-app.yaml │ │ ├── config │ │ │ ├── base │ │ │ │ ├── deployment.yaml │ │ │ │ ├── ingress.yaml │ │ │ │ ├── kustomization.yaml │ │ │ │ ├── migration.yaml │ │ │ │ └── service.yaml │ │ │ └── overlay │ │ │ │ └── dev │ │ │ │ ├── .env │ │ │ │ └── kustomization.yaml │ │ ├── productService │ │ │ ├── .gitignore │ │ │ ├── Controllers │ │ │ │ └── ProductController.cs │ │ │ ├── Dockerfile │ │ │ ├── Domain │ │ │ │ └── Entities │ │ │ │ │ ├── Product.cs │ │ │ │ │ ├── ProductFiles.cs │ │ │ │ │ └── ProductImages.cs │ │ │ ├── Infrastructure │ │ │ │ └── DbContext.cs │ │ │ ├── Migrations │ │ │ │ ├── 20230715114506_Initial.Designer.cs │ │ │ │ ├── 20230715114506_Initial.cs │ │ │ │ ├── 20230715132031_Product_Images.Designer.cs │ │ │ │ ├── 20230715132031_Product_Images.cs │ │ │ │ ├── 20230715175739_Product_files.Designer.cs │ │ │ │ ├── 20230715175739_Product_files.cs │ │ │ │ └── ProductContextModelSnapshot.cs │ │ │ ├── Program.cs │ │ │ ├── Properties │ │ │ │ └── launchSettings.json │ │ │ ├── appsettings.Development.json │ │ │ ├── appsettings.json │ │ │ ├── migration │ │ │ ├── productService.csproj │ │ │ └── traffic.sh │ │ └── readme.md │ ├── ingress.yaml │ ├── readme.md │ └── traefik-ingressroute.yaml ├── cert-manager │ ├── issuer.yaml │ └── readme.md ├── containous-delivery │ ├── blue-grean │ │ └── native │ │ │ ├── blue-deployment.yaml │ │ │ ├── blue-ingress.yaml │ │ │ ├── blue-service.yaml │ │ │ ├── green-deployment.yaml │ │ │ ├── green-ingress.yaml │ │ │ └── green-service.yaml │ └── progrssive-canary │ │ ├── architecture.png │ │ ├── argorollout-cli.png │ │ ├── argorollout-ui.png │ │ ├── canary-demo-argo-app.yaml │ │ ├── canary-demo-web │ │ ├── Dockerfile │ │ ├── go.mod │ │ ├── main.go │ │ └── traffic.sh │ │ ├── config │ │ ├── base │ │ │ ├── analysis-template.yaml │ │ │ ├── ingress.yaml │ │ │ ├── kustomization.yaml │ │ │ ├── rollout.yaml │ │ │ ├── service-canary.yaml │ │ │ └── service.yaml │ │ └── overlays │ │ │ └── dev │ │ │ └── kustomization.yaml │ │ └── readme.md ├── nginx │ ├── delete.sh │ ├── install.sh │ └── readme.md ├── observability │ ├── README.md │ ├── metrics │ │ ├── README.md │ │ └── prometheus │ │ │ ├── architecture-min.png │ │ │ ├── conf.yaml │ │ │ ├── default.values.yaml │ │ │ ├── readme.md │ │ │ └── values.yaml │ └── tracing │ │ ├── README.md │ │ └── jaeger-distributed-tracing │ │ └── readme.md ├── postgres │ ├── README.md │ └── patroni │ │ └── readme.md ├── readme.md ├── traefik │ └── readme.md └── trouble-shooting │ ├── README.md │ └── delete-terminating-stuck-resources │ ├── force-delete.py │ └── readme.md └── roadmap.md /.gitignore: -------------------------------------------------------------------------------- 1 | .vscode 2 | .terraform* 3 | *.tfstate* 4 | kubeconfig 5 | bin 6 | obj 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## DevOps/GitOps Lab Repository 2 | Welcome to the DevOps/GitOps Lab repository! This repository contains all the code and configurations discussed in our comprehensive [GitOps series on YouTube](https://www.youtube.com/playlist?list=PLTRDUPO2OmInz2Fo41zwnoR1IArx70Hig), and much more. 3 | 4 | ## Overview 5 | This repository serves as a hands-on lab for learning and practicing various DevOps and GitOps concepts. It includes practical examples, tutorials, and demonstrations to help you gain a deeper understanding of modern software delivery practices. 6 | 7 | ## What's Inside? 8 | - [Azure practice](azure) 9 | - [GitLab CI stuff](gitlab) 10 | - [Docker](docker) 11 | - [Gitops with Kubernetes](kubernetes) 12 | -------------------------------------------------------------------------------- /azure/aks-iaac/README.md: -------------------------------------------------------------------------------- 1 | # Create AKS cluster on Azure 2 | 3 | ### Changing variables values go to terraform.tfvars 4 | ### Install cluster 5 | ``` 6 | make init 7 | make apply 8 | 9 | ``` 10 | 11 | ### Destroy cluster 12 | ``` 13 | make destroy 14 | ``` 15 | -------------------------------------------------------------------------------- /azure/aks-iaac/main.tf: -------------------------------------------------------------------------------- 1 | resource "azurerm_resource_group" "aks-rg" { 2 | name = var.resource_group_name_prefix 3 | location = var.resource_group_location 4 | } 5 | resource "azurerm_kubernetes_cluster" "aks" { 6 | name = var.resource_name 7 | location = var.resource_group_location 8 | resource_group_name = azurerm_resource_group.aks-rg.name 9 | dns_prefix = var.dns_prefix 10 | default_node_pool { 11 | name = var.node_name_perfix 12 | vm_size = var.vm_size 13 | enable_auto_scaling = true 14 | min_count = var.min_node_count 15 | os_disk_size_gb = var.os_disk_size_gb 16 | max_count = var.max_node_count 17 | zones = [] 18 | type = var.default_pool_type 19 | } 20 | 21 | identity { 22 | type = "SystemAssigned" 23 | } 24 | network_profile { 25 | network_plugin = var.network_plugin 26 | load_balancer_sku = var.load_balancer_sku 27 | network_policy = var.network_policy 28 | } 29 | 30 | tags = { 31 | envrionment = "hands-on-lab" 32 | } 33 | 34 | } 35 | -------------------------------------------------------------------------------- /azure/aks-iaac/makefile: -------------------------------------------------------------------------------- 1 | init: 2 | terraform init 3 | 4 | apply: 5 | terraform apply 6 | 7 | destroy: 8 | terraform destroy 9 | -------------------------------------------------------------------------------- /azure/aks-iaac/output.tf: -------------------------------------------------------------------------------- 1 | resource "local_file" "kubeconfig" { 2 | depends_on = [azurerm_kubernetes_cluster.aks] 3 | filename = "kubeconfig" 4 | content = azurerm_kubernetes_cluster.aks.kube_config_raw 5 | } -------------------------------------------------------------------------------- /azure/aks-iaac/providers.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_providers { 3 | azurerm = { 4 | source = "hashicorp/azurerm" 5 | version = "=3.0.0" 6 | } 7 | } 8 | } 9 | 10 | provider "azurerm" { 11 | features {} 12 | } -------------------------------------------------------------------------------- /azure/aks-iaac/terraform.tfvars: -------------------------------------------------------------------------------- 1 | resource_group_location = "eastus" 2 | resource_group_name_prefix = "aks-rg" 3 | resource_name = "hands-on-lab-aks" 4 | dns_prefix = "hands-on-lab-dns-aks" 5 | node_name_perfix = "myaksnode" 6 | max_node_count = 5 7 | min_node_count = 3 8 | vm_size = "Standard_B2s" 9 | network_plugin = "azure" 10 | network_policy = "calico" 11 | load_balancer_sku = "standard" 12 | os_disk_size_gb = "50" 13 | default_pool_type = "VirtualMachineScaleSets" 14 | -------------------------------------------------------------------------------- /azure/aks-iaac/variables.tf: -------------------------------------------------------------------------------- 1 | variable "resource_group_location" { 2 | type = string 3 | default = "eastus" 4 | } 5 | 6 | variable "resource_group_name_prefix" { 7 | type = string 8 | default = "IOTHub-rg" 9 | } 10 | 11 | variable "resource_name" { 12 | type = string 13 | default = "my-aks" 14 | } 15 | 16 | variable "dns_prefix" { 17 | type = string 18 | default = "my-aks" 19 | } 20 | variable "node_name_perfix" { 21 | type = string 22 | default = "aks_node" 23 | } 24 | variable "max_node_count" { 25 | type = number 26 | default = 3 27 | } 28 | 29 | variable "min_node_count" { 30 | type = number 31 | default = 1 32 | } 33 | 34 | variable "vm_size" { 35 | type = string 36 | default = "Standard_B2s" 37 | } 38 | 39 | variable "os_type" { 40 | type = string 41 | default = "Linux" 42 | } 43 | 44 | variable "os_disk_size_gb" { 45 | type = string 46 | } 47 | variable "default_pool_type" { 48 | type = string 49 | } 50 | variable "network_plugin" { 51 | type = string 52 | default = "Azure CNI" 53 | } 54 | 55 | variable "network_policy" { 56 | type = string 57 | default = "Calico" 58 | } 59 | 60 | variable "load_balancer_sku" { 61 | type = string 62 | default = "Standard" 63 | } 64 | 65 | 66 | -------------------------------------------------------------------------------- /azure/readme.md: -------------------------------------------------------------------------------- 1 | ## Azure cloud services 2 | 3 | ## [Create AKS Service](aks-iaac/README.md) 4 | 5 | -------------------------------------------------------------------------------- /docker/readme.md: -------------------------------------------------------------------------------- 1 | ## Build a docker to be used as Devops linux machine 2 | ``` 3 | docker build -t ubuntu-devops -f ubuntu-docker . 4 | ``` -------------------------------------------------------------------------------- /docker/ubuntu-docker: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt update && apt install -y apt-transport-https ca-certificates curl gnupg python3 3 | 4 | # Add google cloud GPG key and Kubernetes repository 5 | RUN mkdir -p /etc/apt/keyrings/ && \ 6 | curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor > /etc/apt/keyrings/google-cloud.gpg && \ 7 | echo "deb [signed-by=/etc/apt/keyrings/google-cloud.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list 8 | 9 | RUN apt update && apt install -y kubectl 10 | -------------------------------------------------------------------------------- /gitlab/ci/readme.md: -------------------------------------------------------------------------------- 1 | # GitLab Complete CI including static code analysis and semantic versioning 2 | ## Install GitLab Runner 3 | - GitLab Runner machine to handle the CI stages with docker installed 4 | ``` 5 | Linux install 6 | # Download the binary for your system 7 | sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64 8 | 9 | # Give it permission to execute 10 | sudo chmod +x /usr/local/bin/gitlab-runner 11 | 12 | # Create a GitLab Runner user 13 | sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash 14 | #Register 15 | sudo gitlab-runner register -n \ 16 | --url https://gitlab.com/ \ 17 | --registration-token {YOUR_TOKEN} \ 18 | --executor docker \ 19 | --description "My Docker Runner" \ 20 | --docker-image "docker:20.10.16" \ 21 | --docker-privileged \ 22 | --docker-volumes "/certs/client" \ 23 | --tag-list "devops" 24 | 25 | ``` 26 | ## Create access token for the following 27 | ### - Sonarqube 28 | ### - Semantic release 29 | ### - GitLab K8s config repository (pull - push - create branch) 30 | ### - GitLab API to create MR(aka PR) 31 | 32 | ### Semantic release create .releaserc.json on your app repository (semantic release configuration) 33 | ``` 34 | { 35 | "plugins": [ 36 | "@semantic-release/commit-analyzer", 37 | "@semantic-release/release-notes-generator", 38 | [ 39 | "@semantic-release/changelog", 40 | { 41 | "changelogFile": "CHANGELOG.md" 42 | } 43 | ], 44 | [ 45 | "@semantic-release/git", 46 | { 47 | "assets": [ 48 | "CHANGELOG.md" 49 | ] 50 | } 51 | ] 52 | ], 53 | "branches": [ 54 | "main", 55 | "+([0-9])?(.{+([0-9]),x}).x", 56 | { 57 | "name": "beta", 58 | "prerelease": true 59 | } 60 | ] 61 | } 62 | ``` 63 | #### Add commit-msg in your .git/hooks 64 | ``` 65 | #!/usr/bin/env bash 66 | 67 | if [ -z "$1" ]; then 68 | echo "Missing argument (commit message). Did you try to run this manually?" 69 | exit 1 70 | fi 71 | 72 | commit_message_file=$1 73 | commit_message=$(cat $commit_message_file) 74 | semantic_release_pattern='^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\([^)]+\))?!?: .+' 75 | 76 | # ignore merge requests 77 | if echo "$commitTitle" | grep -qE "Merge branch"; then 78 | echo "Commit hook: ignoring branch merge" 79 | exit 0 80 | fi 81 | 82 | # check semantic versioning scheme 83 | if [[ ! $commit_message =~ $semantic_release_pattern ]]; then 84 | echo "Your commit title did not follow semantic versioning: $commitTitle" 85 | echo "Please see https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#commit-message-format" 86 | exit 1 87 | fi 88 | 89 | ``` 90 | ## SonarQube 91 | - You need Sonar instance either cloud based or custom install (developer edition) 92 | In GitLab > Setting > CI/CD > Add the following variables 93 | - SONAR_HOST_URL (your sonar instance URL) 94 | - SONAR_TOKEN (your sonar project token) 95 | #### Add sonar-project.properties for your code repository based on your programming languge example for our golang project 96 | 97 | ``` 98 | sonar.projectKey={SONAR_PROJECT_KEY} 99 | sonar.organization{SONAR_ORG} 100 | sonar.qualitygate.wait=true 101 | sonar.go.coverage.reportPaths=coverage-report.out 102 | sonar.exclusions=coverage-report.html 103 | sonar.test.inclusions=*_test.go 104 | ``` 105 | ### CI/CD 106 | #### Create .gitlab-ci.yml that include gitops process and steps 107 | ### Make merge request (PR) setting to pipeline must succeed 108 | 109 | ### Adding badges 110 | Your gitlab project > setting > General >Badges 111 | ### Pipeline badge 112 | ``` 113 | https://gitlab.com/{USER_NAME}/{REPO_NAME}/badges/main/pipeline.svg 114 | 115 | ``` 116 | ### Code converage 117 | ``` 118 | https://gitlab.com/{USER_NAME}/{REPO_NAME}/badges/main/coverage.svg 119 | ``` 120 | ### Sonar Quality gate status 121 | ``` 122 | https://sonarcloud.io/api/project_badges/measure?project={YOUR_SONAR_PROJECT_NAME}&metric=alert_status 123 | 124 | ``` 125 | ## GitLab CLI create the following file in your app repository 126 | ``` 127 | image: docker:20.10.16 128 | variables: 129 | DOCKER_TLS_CERTDIR: "/certs" 130 | services: 131 | - docker:20.10.16-dind 132 | stages: 133 | - build 134 | - test 135 | - sonarqube 136 | - docker-build 137 | - release 138 | - deploy 139 | 140 | build_job: 141 | stage: build 142 | image: golang:alpine 143 | tags: 144 | - devops 145 | script: 146 | - go get . 147 | - go build 148 | only: 149 | - tags 150 | - main 151 | - develop 152 | - merge_requests 153 | test_job: 154 | stage: test 155 | image: golang:alpine 156 | tags: 157 | - devops 158 | script: 159 | - go get . 160 | - go test ./... -coverprofile=coverage-report.out 161 | - go tool cover -html=coverage-report.out -o coverage-report.html 162 | - go tool cover -func=coverage-report.out 163 | artifacts: 164 | paths: 165 | - coverage-report.html 166 | - coverage-report.out 167 | expire_in: 1 hour 168 | coverage: "/\\(statements\\)\\s+\\d+.?\\d+%/" 169 | only: 170 | - tags 171 | - main 172 | - develop 173 | - merge_requests 174 | sonarqube_job: 175 | stage: sonarqube 176 | tags: 177 | - devops 178 | image: 179 | name: sonarsource/sonar-scanner-cli:latest 180 | entrypoint: [""] 181 | variables: 182 | SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" 183 | GIT_DEPTH: "0" 184 | cache: 185 | key: "${CI_JOB_NAME}" 186 | paths: 187 | - .sonar/cache 188 | script: 189 | - sonar-scanner 190 | allow_failure: true 191 | only: 192 | - merge_requests 193 | - main 194 | docker_job: 195 | stage: docker-build 196 | tags: 197 | - devops 198 | script: 199 | - | 200 | if [[ -z "${CI_COMMIT_TAG}" ]]; then 201 | export VERSION_TAG="${CI_COMMIT_SHORT_SHA}" 202 | else 203 | export VERSION_TAG="${CI_COMMIT_TAG//v}" 204 | fi 205 | - docker build -t $DOKCER_IMAGE_NAME:$VERSION_TAG . 206 | - echo $DOCKER_HUB_PASSWORD | docker login -u $DOCKER_HUB_USERNAME --password-stdin 207 | - docker push $DOKCER_IMAGE_NAME:$VERSION_TAG 208 | 209 | only: 210 | - tags 211 | - develop 212 | release_job: 213 | stage: release 214 | image: node 215 | tags: 216 | - devops 217 | script: 218 | - npm -g install @semantic-release/git semantic-release @semantic-release/changelog && semantic-release 219 | only: 220 | - main 221 | - /^(([0–9]+)\.)?([0–9]+)\.x/ 222 | 223 | update_k8s_config_job: 224 | stage: deploy 225 | image: ubuntu 226 | tags: 227 | - devops 228 | before_script: 229 | - apt update 230 | - apt install git -y && apt install curl -y 231 | script: 232 | - | 233 | if [[ -z "${CI_COMMIT_TAG}" ]]; then 234 | export VERSION_TAG="${CI_COMMIT_SHORT_SHA}" 235 | else 236 | export VERSION_TAG="${CI_COMMIT_TAG//v}" 237 | fi 238 | - echo ${VERSION_TAG} 239 | - git clone https://gitlab-ci-token:${GITLAB_TOKEN}@$CONFIG_REPOSITORY_URL 240 | - cd gitops-config 241 | - git config --global user.email $GIT_CONFIG_EMAIL 242 | - git config --global user.name $GIT_CONFIG_NAME 243 | - git checkout -b release 244 | - cd overlay/prod && sed -i "s/^ *newTag:.*/ newTag:${VERSION_TAG}/" kustomization.yaml 245 | - git add . && git commit -am "Add new build version ${VERSION_TAG}" 246 | - git push --set-upstream origin release 247 | - | 248 | curl --request POST --header "PRIVATE-TOKEN: $MR_CREATOR_TOKEN" \ 249 | --form "source_branch=release" \ 250 | --form "target_branch=main" \ 251 | --form "title=[CI update] New release ${VERSION_TAG}" \ 252 | --form "description=New release to be reviewed and promoted ${VERSION_TAG}" \ 253 | "https://gitlab.com/api/v4/projects/$CONFIG_PROJECT_ID/merge_requests" 254 | only: 255 | - tags 256 | 257 | ``` 258 | ### [Complete CI pipeline for Arabic speaker](https://www.youtube.com/watch?v=Sv9lZ2cO9E8&t=1375s) -------------------------------------------------------------------------------- /gitlab/preview-environment/readme.md: -------------------------------------------------------------------------------- 1 | ### Creating Environments for Every Pull Request using GitLab, Kustomize, and Kubernetes 2 | In this guide, we'll walk through the process of setting up automated environment creation for each pull request using GitLab, Kustomize, and Kubernetes. If you are not familiar with Kustomize, you can check out our [GitOps series on YouTube](https://www.youtube.com/playlist?list=PLTRDUPO2OmInz2Fo41zwnoR1IArx70Hig) 3 | 4 | ## Requirement 5 | - An existing Kubernetes cluster. 6 | - Kubeconfig file for cluster access. 7 | 8 | ## Create Job triggered only merge_request 9 | - prepare stage environment 10 | using alpine image as a runner and install curl,jq,kubectl , kustomize 11 | - create variable in gitlab project CI setting with type file , name KUBECONFIG and the content is your kubeconfig file content to access K8s cluster 12 | ``` 13 | merge_request_deploy_job: 14 | image: alpine:latest 15 | tags: 16 | - k8s 17 | stage: deploy 18 | before_script: 19 | - apk add --update --no-cache curl 20 | - apk --no-cache add jq 21 | - apk add bash 22 | - curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" 23 | - chmod +x ./kubectl && mv ./kubectl /usr/local/bin/kubectl 24 | - mkdir -p $HOME/.kube 25 | - echo "$KUBECONFIG" > $HOME/.kube/config 26 | - | 27 | curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash && \ 28 | mv kustomize /usr/local/bin/kustomize 29 | ``` 30 | #### script do the following 31 | - First, edit your DNS in the ingress to create an ingress for every pull request, using the format ${CI_MERGE_REQUEST_IID}.example.com, in the Kustomize files 32 | ``` 33 | - sed -i "s/items-service.${YOUR_DOMAIN}/pr-${CI_MERGE_REQUEST_IID}.${YOUR_DOMAIN}/" config/base/route.yaml 34 | 35 | ``` 36 | - In the overlay preview environment, create a namespace for the pull request. We utilize a dry run and apply strategy to prevent imperative actions and potential errors if the namespace already exists 37 | ``` 38 | kubectl create ns items-service-pr-$CI_MERGE_REQUEST_IID --dry-run=client -o yaml | kubectl apply -f - 39 | ``` 40 | - Create a Docker registry secret if it is needed to be used as a pull secret 41 | ``` 42 | kubectl create secret docker-registry docker-secret \ 43 | --docker-server=$CI_REGISTRY \ 44 | --docker-username=$CI_REGISTRY_USER \ 45 | --docker-password=$CI_REGISTRY_PASSWORD --dry-run=client -o yaml | kubectl apply -f - \ 46 | -n items-service-pr-$CI_MERGE_REQUEST_IID 47 | ``` 48 | - Utilize the kustomize edit command to dynamically set the image, incorporating the pull request prefix. 49 | ``` 50 | kustomize edit set image ${YOUR_DOCKER_USER_NAME}/items-service=$CI_REGISTRY_USER/$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA && \ 51 | kustomize build . | kubectl apply -f - -n items-service-pr-$CI_MERGE_REQUEST_IID 52 | ``` 53 | - Restrict the job to run only on merge requests. 54 | ``` 55 | only: 56 | - merge_requests 57 | ``` 58 | ### Now how we can cleanup the environment 59 | We need to clean up and remove the namespace when the pull request is merged 60 | #### Note: There is no direct way to accomplish this through pipeline jobs triggered by push changes. However, we can create a cleanup job in the base branch against which the pull request is created 61 | #### requirements 62 | - gitlab access token to read apis only 63 | - create a cleanup job and stage 64 | - use alpine image or what ever you want 65 | ``` 66 | cleanup: 67 | stage: cleanup 68 | image: alpine:latest 69 | ``` 70 | - prepare the runner environment we need curl,jq,kubectl 71 | ``` 72 | before_script: 73 | - apk add --update --no-cache curl 74 | - apk --no-cache add jq 75 | - apk add bash 76 | - curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" 77 | - chmod +x ./kubectl && mv ./kubectl /usr/local/bin/kubectl 78 | - mkdir -p $HOME/.kube 79 | - echo "$KUBECONFIG" > $HOME/.kube/config 80 | ``` 81 | - Subsequently, we need to call the GitLab API to retrieve a list of merged pull requests. By using jq, we extract only the 'iid' and prefix it with the namespace prefix, 'items-service-pr-'. This prepared list is then utilized to run the 'kubectl delete namespace' command against the respective namespaces 82 | ``` 83 | script: 84 | - kubectl delete ns $(curl -s --header "PRIVATE-TOKEN:$PRIVATE_TOKEN" "$GITLAB_API_URL/projects/$PROJECT_ID/merge_requests?state=merged" | jq '.[].iid | "items-service-pr-" + tostring' | tr -d '"') --ignore-not-found=true 85 | ``` 86 | - Restrict the job to run only on base branch 87 | ``` 88 | only: 89 | - main 90 | ``` -------------------------------------------------------------------------------- /gitlab/readme.md: -------------------------------------------------------------------------------- 1 | ## GitLab resources 2 | 3 | - [Complete CI](ci) 4 | - [Automate pull request environment creation](preview-environment) 5 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/argocd/product-service-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: Application 3 | metadata: 4 | name: product-service 5 | namespace: argocd 6 | spec: 7 | project: default 8 | source: 9 | repoURL: https://github.com/ragoob/gitops-dotnet-web-app.git 10 | targetRevision: HEAD 11 | path: productService/config/overlay/dev 12 | destination: 13 | server: https://kubernetes.default.svc 14 | namespace: services-development 15 | syncPolicy: 16 | automated: 17 | selfHeal: true -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/base/deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: product-service 5 | spec: 6 | selector: 7 | matchLabels: 8 | app: product-service 9 | template: 10 | metadata: 11 | annotations: 12 | linkerd.io/inject: enabled 13 | labels: 14 | app: product-service 15 | spec: 16 | containers: 17 | - name: product-service 18 | image: regoo707/product-service 19 | envFrom: 20 | - configMapRef: 21 | name: product-service-cm 22 | livenessProbe: 23 | httpGet: 24 | path: /health 25 | port: 80 26 | initialDelaySeconds: 3 27 | periodSeconds: 3 28 | readinessProbe: 29 | httpGet: 30 | path: /health 31 | port: 80 32 | initialDelaySeconds: 3 33 | periodSeconds: 3 34 | resources: 35 | limits: 36 | memory: "128Mi" 37 | cpu: "500m" 38 | ports: 39 | - containerPort: 80 -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/base/ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: traefik.containo.us/v1alpha1 2 | kind: IngressRoute 3 | metadata: 4 | name: product-service 5 | spec: 6 | entryPoints: 7 | - websecure 8 | routes: 9 | - kind: Rule 10 | match: Host(`dotnetapi.ragab.blog`) 11 | priority: 10 12 | services: 13 | - name: product-service 14 | port: 80 15 | tls: 16 | secretName: dotnet-tls-secret 17 | --- 18 | apiVersion: cert-manager.io/v1 19 | kind: Certificate 20 | metadata: 21 | name: dotnet-cert 22 | spec: 23 | secretName: dotnet-tls-secret 24 | issuerRef: 25 | name: le-prod 26 | kind: ClusterIssuer 27 | dnsNames: 28 | - dotnetapi.ragab.blog -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/base/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | resources: 4 | - deployment.yaml 5 | - service.yaml 6 | - ingress.yaml 7 | - migration.yaml 8 | 9 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/base/migration.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: batch/v1 2 | kind: Job 3 | metadata: 4 | generateName: schema-migrate- 5 | name: schema-migration 6 | annotations: 7 | argocd.argoproj.io/hook: PreSync 8 | spec: 9 | ttlSecondsAfterFinished: 100 10 | template: 11 | spec: 12 | containers: 13 | - name: ef-migration 14 | image: mcr.microsoft.com/dotnet/sdk:6.0 15 | command: ["/bin/bash","-c", "apt install git -y && git clone $GIT_REPO && cd $SERVICE_PATH && dotnet tool install --global dotnet-ef && export PATH=\"$PATH:/root/.dotnet/tools\" && dotnet ef database update"] 16 | env: 17 | - name: GIT_REPO 18 | value: https://github.com/ragoob/gitops-dotnet-web-app.git 19 | - name: ConnectionStrings__DefaultConnection 20 | value: Host=postgres.postgres.svc.cluster.local;Port=5432;Database=product_db;Username=postgres;Password=postgres 21 | - name: SERVICE_PATH 22 | value: gitops-dotnet-web-app/productService 23 | restartPolicy: Never 24 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/base/service.yaml: -------------------------------------------------------------------------------- 1 | kind: Service 2 | apiVersion: v1 3 | metadata: 4 | name: product-service 5 | annotations: 6 | linkerd.io/inject: enabled 7 | spec: 8 | selector: 9 | app: product-service 10 | type: ClusterIP 11 | ports: 12 | - name: http 13 | port: 80 14 | targetPort: 80 -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/overlay/dev/.env: -------------------------------------------------------------------------------- 1 | ConnectionStrings__DefaultConnection=Host=postgres.postgres.svc.cluster.local;Port=5432;Database=product_db;Username=postgres;Password=postgres 2 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/config/overlay/dev/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | resources: 4 | - ../../base 5 | images: 6 | - name: regoo707/product-service 7 | newTag: 2f4815d 8 | configMapGenerator: 9 | - name: product-service-cm 10 | envs: 11 | - .env -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/.gitignore: -------------------------------------------------------------------------------- 1 | # .NET Core 2 | *.dll 3 | *.exe 4 | *.pdb 5 | *.dll.config 6 | *.user.secrets 7 | *.rptproj.user 8 | /.vs 9 | /.build/ 10 | /build/ 11 | /bin 12 | /obj 13 | # Visual Studio Code 14 | .vscode/ 15 | 16 | # JetBrains Rider 17 | .idea/ 18 | 19 | # macOS 20 | .DS_Store 21 | 22 | # NuGet 23 | *.nuget.props 24 | *.nuget.targets 25 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Controllers/ProductController.cs: -------------------------------------------------------------------------------- 1 | using Microsoft.AspNetCore.Mvc; 2 | using Microsoft.EntityFrameworkCore; 3 | using productService.Doamin.Entities; 4 | using productService.Infrastructure; 5 | 6 | namespace productService.Controllers; 7 | 8 | [ApiController] 9 | [Route("[controller]")] 10 | public class ProductController : ControllerBase 11 | { 12 | 13 | private readonly ILogger _logger; 14 | private readonly ProductContext _context; 15 | 16 | public ProductController(ILogger logger, ProductContext context) 17 | { 18 | _logger = logger; 19 | _context = context; 20 | } 21 | 22 | [HttpPost] 23 | public async Task CreateAsync(Product model) 24 | { 25 | await _context.Products.AddAsync(model); 26 | await _context.SaveChangesAsync(); 27 | return NoContent(); 28 | } 29 | 30 | [HttpGet] 31 | public async Task> GetAsync(){ 32 | return await _context.Products.ToListAsync(); 33 | } 34 | 35 | } 36 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Dockerfile: -------------------------------------------------------------------------------- 1 | # Use the official .NET 6 SDK image as the base image 2 | FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build 3 | 4 | # Set the working directory in the container 5 | WORKDIR /app 6 | 7 | # Copy the .NET 6 project file and restore dependencies 8 | COPY *.csproj ./ 9 | RUN dotnet restore 10 | 11 | # Copy the entire application source code 12 | COPY . ./ 13 | 14 | # Build the application 15 | RUN dotnet publish -c Release -o out 16 | 17 | # Use the official .NET 6 runtime image as the base image 18 | FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS runtime 19 | 20 | # Set the working directory in the container 21 | WORKDIR /app 22 | 23 | # Copy the built application from the build image 24 | COPY --from=build /app/out ./ 25 | 26 | # Set the entry point for the container 27 | ENTRYPOINT ["dotnet", "productService.dll"] 28 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Domain/Entities/Product.cs: -------------------------------------------------------------------------------- 1 | namespace productService.Doamin.Entities { 2 | public class Product { 3 | public Product() 4 | { 5 | this.Images = new HashSet(); 6 | } 7 | public Guid Id { get; set; } 8 | 9 | public string Name { get; set; } = string.Empty; 10 | 11 | public decimal Price { get; set; } 12 | 13 | public virtual ICollection Images { get;set;} 14 | 15 | public virtual ICollection Files { get;set;} 16 | } 17 | } -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Domain/Entities/ProductFiles.cs: -------------------------------------------------------------------------------- 1 | namespace productService.Doamin.Entities { 2 | public class ProductFile { 3 | public Guid Id { get; set; } 4 | 5 | public string FilePath { get; set; } = string.Empty; 6 | 7 | public Product Product {get;set;} 8 | 9 | public Guid ProductId {get;set;} 10 | 11 | } 12 | } -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Domain/Entities/ProductImages.cs: -------------------------------------------------------------------------------- 1 | namespace productService.Doamin.Entities { 2 | public class ProductImages { 3 | public Guid Id { get; set; } 4 | 5 | public string ImageUrl { get; set; } = string.Empty; 6 | 7 | public Guid ProductId { get; set; } 8 | 9 | public virtual Product Product { get; set; } 10 | } 11 | } -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Infrastructure/DbContext.cs: -------------------------------------------------------------------------------- 1 | using Microsoft.EntityFrameworkCore; 2 | using productService.Doamin.Entities; 3 | 4 | namespace productService.Infrastructure { 5 | public class ProductContext : DbContext { 6 | 7 | public DbSet Products {get;set;} 8 | public ProductContext(DbContextOptions opts): base(opts) 9 | { 10 | 11 | } 12 | 13 | protected override void OnModelCreating(ModelBuilder modelBuilder) 14 | { 15 | base.OnModelCreating(modelBuilder); 16 | 17 | var product = modelBuilder.Entity(); 18 | product.ToTable("product"); 19 | product.HasKey(p=> p.Id); 20 | 21 | var productImages = modelBuilder.Entity(); 22 | productImages.ToTable("product_images"); 23 | productImages.HasKey(pm=> pm.Id); 24 | productImages.HasOne(pm=> pm.Product) 25 | .WithMany(p=> p.Images) 26 | .HasForeignKey(pm=> pm.ProductId); 27 | 28 | var ProductFiles = modelBuilder.Entity(); 29 | ProductFiles.ToTable("product_files"); 30 | ProductFiles.HasKey(pf=> pf.Id); 31 | ProductFiles.HasOne(pf=> pf.Product) 32 | .WithMany(p=> p.Files) 33 | .HasForeignKey(pm=> pm.ProductId); 34 | 35 | } 36 | } 37 | } -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715114506_Initial.Designer.cs: -------------------------------------------------------------------------------- 1 | // 2 | using System; 3 | using Microsoft.EntityFrameworkCore; 4 | using Microsoft.EntityFrameworkCore.Infrastructure; 5 | using Microsoft.EntityFrameworkCore.Migrations; 6 | using Microsoft.EntityFrameworkCore.Storage.ValueConversion; 7 | using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata; 8 | using productService.Infrastructure; 9 | 10 | #nullable disable 11 | 12 | namespace productService.Migrations 13 | { 14 | [DbContext(typeof(ProductContext))] 15 | [Migration("20230715114506_Initial")] 16 | partial class Initial 17 | { 18 | /// 19 | protected override void BuildTargetModel(ModelBuilder modelBuilder) 20 | { 21 | #pragma warning disable 612, 618 22 | modelBuilder 23 | .HasAnnotation("ProductVersion", "7.0.9") 24 | .HasAnnotation("Relational:MaxIdentifierLength", 63); 25 | 26 | NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder); 27 | 28 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 29 | { 30 | b.Property("Id") 31 | .ValueGeneratedOnAdd() 32 | .HasColumnType("uuid") 33 | .HasColumnName("id"); 34 | 35 | b.Property("Name") 36 | .IsRequired() 37 | .HasColumnType("text") 38 | .HasColumnName("name"); 39 | 40 | b.Property("Price") 41 | .HasColumnType("numeric") 42 | .HasColumnName("price"); 43 | 44 | b.HasKey("Id") 45 | .HasName("pk_product"); 46 | 47 | b.ToTable("product", (string)null); 48 | }); 49 | #pragma warning restore 612, 618 50 | } 51 | } 52 | } 53 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715114506_Initial.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.EntityFrameworkCore.Migrations; 3 | 4 | #nullable disable 5 | 6 | namespace productService.Migrations 7 | { 8 | /// 9 | public partial class Initial : Migration 10 | { 11 | /// 12 | protected override void Up(MigrationBuilder migrationBuilder) 13 | { 14 | migrationBuilder.CreateTable( 15 | name: "product", 16 | columns: table => new 17 | { 18 | id = table.Column(type: "uuid", nullable: false), 19 | name = table.Column(type: "text", nullable: false), 20 | price = table.Column(type: "numeric", nullable: false) 21 | }, 22 | constraints: table => 23 | { 24 | table.PrimaryKey("pk_product", x => x.id); 25 | }); 26 | } 27 | 28 | /// 29 | protected override void Down(MigrationBuilder migrationBuilder) 30 | { 31 | migrationBuilder.DropTable( 32 | name: "product"); 33 | } 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715132031_Product_Images.Designer.cs: -------------------------------------------------------------------------------- 1 | // 2 | using System; 3 | using Microsoft.EntityFrameworkCore; 4 | using Microsoft.EntityFrameworkCore.Infrastructure; 5 | using Microsoft.EntityFrameworkCore.Migrations; 6 | using Microsoft.EntityFrameworkCore.Storage.ValueConversion; 7 | using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata; 8 | using productService.Infrastructure; 9 | 10 | #nullable disable 11 | 12 | namespace productService.Migrations 13 | { 14 | [DbContext(typeof(ProductContext))] 15 | [Migration("20230715132031_Product_Images")] 16 | partial class Product_Images 17 | { 18 | /// 19 | protected override void BuildTargetModel(ModelBuilder modelBuilder) 20 | { 21 | #pragma warning disable 612, 618 22 | modelBuilder 23 | .HasAnnotation("ProductVersion", "7.0.9") 24 | .HasAnnotation("Relational:MaxIdentifierLength", 63); 25 | 26 | NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder); 27 | 28 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 29 | { 30 | b.Property("Id") 31 | .ValueGeneratedOnAdd() 32 | .HasColumnType("uuid") 33 | .HasColumnName("id"); 34 | 35 | b.Property("Name") 36 | .IsRequired() 37 | .HasColumnType("text") 38 | .HasColumnName("name"); 39 | 40 | b.Property("Price") 41 | .HasColumnType("numeric") 42 | .HasColumnName("price"); 43 | 44 | b.HasKey("Id") 45 | .HasName("pk_product"); 46 | 47 | b.ToTable("product", (string)null); 48 | }); 49 | 50 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 51 | { 52 | b.Property("Id") 53 | .ValueGeneratedOnAdd() 54 | .HasColumnType("uuid") 55 | .HasColumnName("id"); 56 | 57 | b.Property("ImageUrl") 58 | .IsRequired() 59 | .HasColumnType("text") 60 | .HasColumnName("image_url"); 61 | 62 | b.Property("ProductId") 63 | .HasColumnType("uuid") 64 | .HasColumnName("product_id"); 65 | 66 | b.HasKey("Id") 67 | .HasName("pk_product_images"); 68 | 69 | b.HasIndex("ProductId") 70 | .HasDatabaseName("ix_product_images_product_id"); 71 | 72 | b.ToTable("product_images", (string)null); 73 | }); 74 | 75 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 76 | { 77 | b.HasOne("productService.Doamin.Entities.Product", "Product") 78 | .WithMany("Images") 79 | .HasForeignKey("ProductId") 80 | .OnDelete(DeleteBehavior.Cascade) 81 | .IsRequired() 82 | .HasConstraintName("fk_product_images_products_product_id"); 83 | 84 | b.Navigation("Product"); 85 | }); 86 | 87 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 88 | { 89 | b.Navigation("Images"); 90 | }); 91 | #pragma warning restore 612, 618 92 | } 93 | } 94 | } 95 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715132031_Product_Images.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.EntityFrameworkCore.Migrations; 3 | 4 | #nullable disable 5 | 6 | namespace productService.Migrations 7 | { 8 | /// 9 | public partial class Product_Images : Migration 10 | { 11 | /// 12 | protected override void Up(MigrationBuilder migrationBuilder) 13 | { 14 | migrationBuilder.CreateTable( 15 | name: "product_images", 16 | columns: table => new 17 | { 18 | id = table.Column(type: "uuid", nullable: false), 19 | image_url = table.Column(type: "text", nullable: false), 20 | product_id = table.Column(type: "uuid", nullable: false) 21 | }, 22 | constraints: table => 23 | { 24 | table.PrimaryKey("pk_product_images", x => x.id); 25 | table.ForeignKey( 26 | name: "fk_product_images_products_product_id", 27 | column: x => x.product_id, 28 | principalTable: "product", 29 | principalColumn: "id", 30 | onDelete: ReferentialAction.Cascade); 31 | }); 32 | 33 | migrationBuilder.CreateIndex( 34 | name: "ix_product_images_product_id", 35 | table: "product_images", 36 | column: "product_id"); 37 | } 38 | 39 | /// 40 | protected override void Down(MigrationBuilder migrationBuilder) 41 | { 42 | migrationBuilder.DropTable( 43 | name: "product_images"); 44 | } 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715175739_Product_files.Designer.cs: -------------------------------------------------------------------------------- 1 | // 2 | using System; 3 | using Microsoft.EntityFrameworkCore; 4 | using Microsoft.EntityFrameworkCore.Infrastructure; 5 | using Microsoft.EntityFrameworkCore.Migrations; 6 | using Microsoft.EntityFrameworkCore.Storage.ValueConversion; 7 | using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata; 8 | using productService.Infrastructure; 9 | 10 | #nullable disable 11 | 12 | namespace productService.Migrations 13 | { 14 | [DbContext(typeof(ProductContext))] 15 | [Migration("20230715175739_Product_files")] 16 | partial class Product_files 17 | { 18 | /// 19 | protected override void BuildTargetModel(ModelBuilder modelBuilder) 20 | { 21 | #pragma warning disable 612, 618 22 | modelBuilder 23 | .HasAnnotation("ProductVersion", "7.0.9") 24 | .HasAnnotation("Relational:MaxIdentifierLength", 63); 25 | 26 | NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder); 27 | 28 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 29 | { 30 | b.Property("Id") 31 | .ValueGeneratedOnAdd() 32 | .HasColumnType("uuid") 33 | .HasColumnName("id"); 34 | 35 | b.Property("Name") 36 | .IsRequired() 37 | .HasColumnType("text") 38 | .HasColumnName("name"); 39 | 40 | b.Property("Price") 41 | .HasColumnType("numeric") 42 | .HasColumnName("price"); 43 | 44 | b.HasKey("Id") 45 | .HasName("pk_product"); 46 | 47 | b.ToTable("product", (string)null); 48 | }); 49 | 50 | modelBuilder.Entity("productService.Doamin.Entities.ProductFile", b => 51 | { 52 | b.Property("Id") 53 | .ValueGeneratedOnAdd() 54 | .HasColumnType("uuid") 55 | .HasColumnName("id"); 56 | 57 | b.Property("FilePath") 58 | .IsRequired() 59 | .HasColumnType("text") 60 | .HasColumnName("file_path"); 61 | 62 | b.Property("ProductId") 63 | .HasColumnType("uuid") 64 | .HasColumnName("product_id"); 65 | 66 | b.HasKey("Id") 67 | .HasName("pk_product_files"); 68 | 69 | b.HasIndex("ProductId") 70 | .HasDatabaseName("ix_product_files_product_id"); 71 | 72 | b.ToTable("product_files", (string)null); 73 | }); 74 | 75 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 76 | { 77 | b.Property("Id") 78 | .ValueGeneratedOnAdd() 79 | .HasColumnType("uuid") 80 | .HasColumnName("id"); 81 | 82 | b.Property("ImageUrl") 83 | .IsRequired() 84 | .HasColumnType("text") 85 | .HasColumnName("image_url"); 86 | 87 | b.Property("ProductId") 88 | .HasColumnType("uuid") 89 | .HasColumnName("product_id"); 90 | 91 | b.HasKey("Id") 92 | .HasName("pk_product_images"); 93 | 94 | b.HasIndex("ProductId") 95 | .HasDatabaseName("ix_product_images_product_id"); 96 | 97 | b.ToTable("product_images", (string)null); 98 | }); 99 | 100 | modelBuilder.Entity("productService.Doamin.Entities.ProductFile", b => 101 | { 102 | b.HasOne("productService.Doamin.Entities.Product", "Product") 103 | .WithMany("Files") 104 | .HasForeignKey("ProductId") 105 | .OnDelete(DeleteBehavior.Cascade) 106 | .IsRequired() 107 | .HasConstraintName("fk_product_files_product_product_id"); 108 | 109 | b.Navigation("Product"); 110 | }); 111 | 112 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 113 | { 114 | b.HasOne("productService.Doamin.Entities.Product", "Product") 115 | .WithMany("Images") 116 | .HasForeignKey("ProductId") 117 | .OnDelete(DeleteBehavior.Cascade) 118 | .IsRequired() 119 | .HasConstraintName("fk_product_images_products_product_id"); 120 | 121 | b.Navigation("Product"); 122 | }); 123 | 124 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 125 | { 126 | b.Navigation("Files"); 127 | 128 | b.Navigation("Images"); 129 | }); 130 | #pragma warning restore 612, 618 131 | } 132 | } 133 | } 134 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/20230715175739_Product_files.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.EntityFrameworkCore.Migrations; 3 | 4 | #nullable disable 5 | 6 | namespace productService.Migrations 7 | { 8 | /// 9 | public partial class Product_files : Migration 10 | { 11 | /// 12 | protected override void Up(MigrationBuilder migrationBuilder) 13 | { 14 | migrationBuilder.CreateTable( 15 | name: "product_files", 16 | columns: table => new 17 | { 18 | id = table.Column(type: "uuid", nullable: false), 19 | file_path = table.Column(type: "text", nullable: false), 20 | product_id = table.Column(type: "uuid", nullable: false) 21 | }, 22 | constraints: table => 23 | { 24 | table.PrimaryKey("pk_product_files", x => x.id); 25 | table.ForeignKey( 26 | name: "fk_product_files_product_product_id", 27 | column: x => x.product_id, 28 | principalTable: "product", 29 | principalColumn: "id", 30 | onDelete: ReferentialAction.Cascade); 31 | }); 32 | 33 | migrationBuilder.CreateIndex( 34 | name: "ix_product_files_product_id", 35 | table: "product_files", 36 | column: "product_id"); 37 | } 38 | 39 | /// 40 | protected override void Down(MigrationBuilder migrationBuilder) 41 | { 42 | migrationBuilder.DropTable( 43 | name: "product_files"); 44 | } 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Migrations/ProductContextModelSnapshot.cs: -------------------------------------------------------------------------------- 1 | // 2 | using System; 3 | using Microsoft.EntityFrameworkCore; 4 | using Microsoft.EntityFrameworkCore.Infrastructure; 5 | using Microsoft.EntityFrameworkCore.Storage.ValueConversion; 6 | using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata; 7 | using productService.Infrastructure; 8 | 9 | #nullable disable 10 | 11 | namespace productService.Migrations 12 | { 13 | [DbContext(typeof(ProductContext))] 14 | partial class ProductContextModelSnapshot : ModelSnapshot 15 | { 16 | protected override void BuildModel(ModelBuilder modelBuilder) 17 | { 18 | #pragma warning disable 612, 618 19 | modelBuilder 20 | .HasAnnotation("ProductVersion", "7.0.9") 21 | .HasAnnotation("Relational:MaxIdentifierLength", 63); 22 | 23 | NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder); 24 | 25 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 26 | { 27 | b.Property("Id") 28 | .ValueGeneratedOnAdd() 29 | .HasColumnType("uuid") 30 | .HasColumnName("id"); 31 | 32 | b.Property("Name") 33 | .IsRequired() 34 | .HasColumnType("text") 35 | .HasColumnName("name"); 36 | 37 | b.Property("Price") 38 | .HasColumnType("numeric") 39 | .HasColumnName("price"); 40 | 41 | b.HasKey("Id") 42 | .HasName("pk_product"); 43 | 44 | b.ToTable("product", (string)null); 45 | }); 46 | 47 | modelBuilder.Entity("productService.Doamin.Entities.ProductFile", b => 48 | { 49 | b.Property("Id") 50 | .ValueGeneratedOnAdd() 51 | .HasColumnType("uuid") 52 | .HasColumnName("id"); 53 | 54 | b.Property("FilePath") 55 | .IsRequired() 56 | .HasColumnType("text") 57 | .HasColumnName("file_path"); 58 | 59 | b.Property("ProductId") 60 | .HasColumnType("uuid") 61 | .HasColumnName("product_id"); 62 | 63 | b.HasKey("Id") 64 | .HasName("pk_product_files"); 65 | 66 | b.HasIndex("ProductId") 67 | .HasDatabaseName("ix_product_files_product_id"); 68 | 69 | b.ToTable("product_files", (string)null); 70 | }); 71 | 72 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 73 | { 74 | b.Property("Id") 75 | .ValueGeneratedOnAdd() 76 | .HasColumnType("uuid") 77 | .HasColumnName("id"); 78 | 79 | b.Property("ImageUrl") 80 | .IsRequired() 81 | .HasColumnType("text") 82 | .HasColumnName("image_url"); 83 | 84 | b.Property("ProductId") 85 | .HasColumnType("uuid") 86 | .HasColumnName("product_id"); 87 | 88 | b.HasKey("Id") 89 | .HasName("pk_product_images"); 90 | 91 | b.HasIndex("ProductId") 92 | .HasDatabaseName("ix_product_images_product_id"); 93 | 94 | b.ToTable("product_images", (string)null); 95 | }); 96 | 97 | modelBuilder.Entity("productService.Doamin.Entities.ProductFile", b => 98 | { 99 | b.HasOne("productService.Doamin.Entities.Product", "Product") 100 | .WithMany("Files") 101 | .HasForeignKey("ProductId") 102 | .OnDelete(DeleteBehavior.Cascade) 103 | .IsRequired() 104 | .HasConstraintName("fk_product_files_product_product_id"); 105 | 106 | b.Navigation("Product"); 107 | }); 108 | 109 | modelBuilder.Entity("productService.Doamin.Entities.ProductImages", b => 110 | { 111 | b.HasOne("productService.Doamin.Entities.Product", "Product") 112 | .WithMany("Images") 113 | .HasForeignKey("ProductId") 114 | .OnDelete(DeleteBehavior.Cascade) 115 | .IsRequired() 116 | .HasConstraintName("fk_product_images_products_product_id"); 117 | 118 | b.Navigation("Product"); 119 | }); 120 | 121 | modelBuilder.Entity("productService.Doamin.Entities.Product", b => 122 | { 123 | b.Navigation("Files"); 124 | 125 | b.Navigation("Images"); 126 | }); 127 | #pragma warning restore 612, 618 128 | } 129 | } 130 | } 131 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Program.cs: -------------------------------------------------------------------------------- 1 | using Microsoft.EntityFrameworkCore; 2 | using productService.Infrastructure; 3 | 4 | var builder = WebApplication.CreateBuilder(args); 5 | 6 | // Add services to the container. 7 | 8 | builder.Services.AddControllers(); 9 | // Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle 10 | builder.Services.AddEndpointsApiExplorer(); 11 | builder.Services.AddSwaggerGen(); 12 | builder.Services.AddDbContext(opts=> { 13 | opts.UseNpgsql(builder.Configuration.GetConnectionString("DefaultConnection")); 14 | opts.UseSnakeCaseNamingConvention(); 15 | 16 | }); 17 | builder.Services.AddHealthChecks(); 18 | var app = builder.Build(); 19 | 20 | // Configure the HTTP request pipeline. 21 | if (app.Environment.IsDevelopment()) 22 | { 23 | app.UseSwagger(); 24 | app.UseSwaggerUI(); 25 | } 26 | 27 | app.UseHttpsRedirection(); 28 | 29 | app.UseAuthorization(); 30 | 31 | app.MapControllers(); 32 | app.UseHealthChecks("/health"); 33 | app.Run(); 34 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/Properties/launchSettings.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://json.schemastore.org/launchsettings.json", 3 | "iisSettings": { 4 | "windowsAuthentication": false, 5 | "anonymousAuthentication": true, 6 | "iisExpress": { 7 | "applicationUrl": "http://localhost:54773", 8 | "sslPort": 44387 9 | } 10 | }, 11 | "profiles": { 12 | "productService": { 13 | "commandName": "Project", 14 | "dotnetRunMessages": true, 15 | "launchBrowser": true, 16 | "launchUrl": "swagger", 17 | "applicationUrl": "https://localhost:7154;http://localhost:5041", 18 | "environmentVariables": { 19 | "ASPNETCORE_ENVIRONMENT": "Development" 20 | } 21 | }, 22 | "IIS Express": { 23 | "commandName": "IISExpress", 24 | "launchBrowser": true, 25 | "launchUrl": "swagger", 26 | "environmentVariables": { 27 | "ASPNETCORE_ENVIRONMENT": "Development" 28 | } 29 | } 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/appsettings.Development.json: -------------------------------------------------------------------------------- 1 | { 2 | "Logging": { 3 | "LogLevel": { 4 | "Default": "Information", 5 | "Microsoft.AspNetCore": "Warning" 6 | } 7 | }, 8 | "ConnectionStrings": { 9 | "DefaultConnection" : "Host=localhost;Port=5432;Database=product_db;Username=postgres;Password=postgres;" 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/appsettings.json: -------------------------------------------------------------------------------- 1 | { 2 | "Logging": { 3 | "LogLevel": { 4 | "Default": "Information", 5 | "Microsoft.AspNetCore": "Warning" 6 | } 7 | }, 8 | "AllowedHosts": "*" 9 | } 10 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/migration: -------------------------------------------------------------------------------- 1 | FROM mcr.microsoft.com/dotnet/sdk:6.0 2 | RUN dotnet tool install --global dotnet-ef 3 | 4 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/productService.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | net6.0 5 | enable 6 | enable 7 | 8 | 9 | 10 | 11 | 12 | 13 | runtime; build; native; contentfiles; analyzers; buildtransitive 14 | all 15 | 16 | 17 | 18 | 19 | 20 | 21 | -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/productService/traffic.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | for ((i=1; i<=1000; i++)) 4 | do 5 | curl -k -X GET https://dotnetapi.ragab.blog/product 6 | done -------------------------------------------------------------------------------- /kubernetes/argocd/argocd-hooks-example/readme.md: -------------------------------------------------------------------------------- 1 | ## Use ARGOCD hook to run .net ef core db migration job before sync new application version 2 | ## Notes for real and on production case do not store stuff like Connection string in git instead use sealed secrets or vault 3 | In your config in base of kustomize add the following job 4 | ``` 5 | apiVersion: batch/v1 6 | kind: Job 7 | metadata: 8 | generateName: schema-migrate- 9 | name: schema-migration 10 | annotations: 11 | argocd.argoproj.io/hook: PreSync 12 | spec: 13 | ttlSecondsAfterFinished: 100 14 | template: 15 | spec: 16 | containers: 17 | - name: ef-migration 18 | image: mcr.microsoft.com/dotnet/sdk:6.0 19 | command: ["/bin/bash","-c", "apt install git -y && git clone $GIT_REPO && cd $SERVICE_PATH && dotnet tool install --global dotnet-ef && export PATH=\"$PATH:/root/.dotnet/tools\" && dotnet ef database update"] 20 | env: 21 | - name: GIT_REPO 22 | value: {{Your ef core migration file repository aka your code repository}} 23 | - name: ConnectionStrings__DefaultConnection 24 | value: Host={{Your connection string recommended to use secerts}} 25 | - name: SERVICE_PATH 26 | value: gitops-dotnet-web-app/productService 27 | restartPolicy: Never 28 | 29 | ``` 30 | ``` 31 | kubectl apply -f argocd/product-service-app.yaml . -n argocd 32 | 33 | ``` 34 | -------------------------------------------------------------------------------- /kubernetes/argocd/ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.k8s.io/v1 2 | kind: Ingress 3 | metadata: 4 | name: argocd-server 5 | namespace: argocd 6 | annotations: 7 | nginx.ingress.kubernetes.io/ssl-passthrough: "true" 8 | nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" 9 | nginx.ingress.kubernetes.io/force-ssl-redirect: "true" 10 | spec: 11 | ingressClassName: "nginx" 12 | rules: 13 | - host: argocd.ragab.biz 14 | http: 15 | paths: 16 | - path: / 17 | pathType: Prefix 18 | backend: 19 | service: 20 | name: argocd-server 21 | port: 22 | name: https -------------------------------------------------------------------------------- /kubernetes/argocd/readme.md: -------------------------------------------------------------------------------- 1 | ## Install argocd with UI 2 | ``` 3 | kubectl create namespace argocd 4 | kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml 5 | ``` 6 | ## Install argocd without UI/sso etc 7 | ``` 8 | kubectl create namespace argocd 9 | kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml 10 | 11 | ``` 12 | ## Port forwaring 13 | ``` 14 | kubectl port-forward svc/argocd-server -n argocd 8080:443 15 | 16 | ``` 17 | ## Expose Argocd LB 18 | ``` 19 | kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}' 20 | 21 | ``` 22 | cl;e 23 | ## Get default password 24 | ``` 25 | kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d 26 | 27 | 28 | ``` 29 | ## Expose via http ingress with nginx with SSL from lets encrypt 30 | ``` 31 | apiVersion: networking.k8s.io/v1 32 | kind: Ingress 33 | metadata: 34 | name: argocd-server-ingress 35 | namespace: argocd 36 | annotations: 37 | cert-manager.io/cluster-issuer: le-prod 38 | kubernetes.io/tls-acme: "true" 39 | nginx.ingress.kubernetes.io/ssl-passthrough: "true" 40 | # If you encounter a redirect loop or are getting a 307 response code 41 | # then you need to force the nginx ingress to connect to the backend using HTTPS. 42 | # 43 | nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" 44 | spec: 45 | ingressClassName: "nginx" 46 | rules: 47 | - host: argocd.example.com 48 | http: 49 | paths: 50 | - path: / 51 | pathType: Prefix 52 | backend: 53 | service: 54 | name: argocd-server 55 | port: 56 | name: https 57 | tls: 58 | - hosts: 59 | - argocd.example.com 60 | secretName: argocd-secret 61 | ``` 62 | ## Expose via http ingress with traefik with SSL from lets encrypt 63 | ``` 64 | apiVersion: traefik.containo.us/v1alpha1 65 | kind: IngressRoute 66 | metadata: 67 | name: argocd-server 68 | namespace: argocd 69 | spec: 70 | entryPoints: 71 | - websecure 72 | routes: 73 | - kind: Rule 74 | match: Host(`argocd.ragab.blog`) 75 | priority: 10 76 | services: 77 | - name: argocd-server 78 | port: 80 79 | - kind: Rule 80 | match: Host(`argocd.ragab.blog`) && Headers(`Content-Type`, `application/grpc`) 81 | priority: 11 82 | services: 83 | - name: argocd-server 84 | port: 80 85 | scheme: h2c 86 | tls: 87 | secretName: argocd-tls-secret 88 | --- 89 | apiVersion: cert-manager.io/v1 90 | kind: Certificate 91 | metadata: 92 | name: argocd-cert 93 | namespace: argocd 94 | spec: 95 | secretName: argocd-tls-secret 96 | issuerRef: 97 | name: le-prod 98 | kind: ClusterIssuer 99 | dnsNames: 100 | - argocd.ragab.blog 101 | ``` 102 | 103 | ## How to add Repository via CLI 104 | ``` 105 | argocd repo add 106 | # If repo is private 107 | argocd repo add --username --password 108 | # or via ssh 109 | argocd repo add --ssh-private-key-path 110 | 111 | ``` 112 | ## Create argo Project 113 | ``` 114 | apiVersion: argoproj.io/v1alpha1 115 | kind: Application 116 | metadata: 117 | name: {{YOUR PROJECT NAME}} 118 | namespace: argocd 119 | spec: 120 | project: default 121 | source: 122 | repoURL: {{Your Repo URL}} 123 | targetRevision: HEAD 124 | path: productService/config/overlay/dev 125 | destination: 126 | server: {{Your cluster 'default: https://kubernetes.default.svc'}} 127 | namespace: {{Your namespace}} 128 | syncPolicy: 129 | automated: 130 | selfHeal: true 131 | ``` 132 | ## [Run db migration job before every application sync ](argocd-hooks-example) 133 | 134 | -------------------------------------------------------------------------------- /kubernetes/argocd/traefik-ingressroute.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: traefik.containo.us/v1alpha1 2 | kind: IngressRoute 3 | metadata: 4 | name: argocd-server 5 | namespace: argocd 6 | spec: 7 | entryPoints: 8 | - websecure 9 | routes: 10 | - kind: Rule 11 | match: Host(`argocd.ragab.blog`) 12 | priority: 10 13 | services: 14 | - name: argocd-server 15 | port: 80 16 | - kind: Rule 17 | match: Host(`argocd.ragab.blog`) && Headers(`Content-Type`, `application/grpc`) 18 | priority: 11 19 | services: 20 | - name: argocd-server 21 | port: 80 22 | scheme: h2c 23 | tls: 24 | secretName: argocd-tls-secret 25 | --- 26 | apiVersion: cert-manager.io/v1 27 | kind: Certificate 28 | metadata: 29 | name: argocd-cert 30 | namespace: argocd 31 | spec: 32 | secretName: argocd-tls-secret 33 | issuerRef: 34 | name: le-prod 35 | kind: ClusterIssuer 36 | dnsNames: 37 | - argocd.ragab.blog -------------------------------------------------------------------------------- /kubernetes/cert-manager/issuer.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: cert-manager.io/v1 2 | kind: ClusterIssuer 3 | metadata: 4 | name: le-prod 5 | spec: 6 | acme: 7 | server: https://acme-v02.api.letsencrypt.org/directory 8 | email: mragab6010@gmail.com 9 | privateKeySecretRef: 10 | name: le-prod 11 | solvers: 12 | - http01: 13 | ingress: 14 | class: nginx -------------------------------------------------------------------------------- /kubernetes/cert-manager/readme.md: -------------------------------------------------------------------------------- 1 | ## Install cert manager 2 | ``` 3 | helm repo add jetstack https://charts.jetstack.io 4 | helm repo update 5 | helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true 6 | 7 | ``` 8 | 9 | ## Create cluster issuer for lets encrypt 10 | ``` 11 | apiVersion: cert-manager.io/v1 12 | kind: ClusterIssuer 13 | metadata: 14 | name: le-prod 15 | spec: 16 | acme: 17 | server: https://acme-v02.api.letsencrypt.org/directory 18 | email: {{Your email}} 19 | privateKeySecretRef: 20 | name: le-prod 21 | solvers: 22 | - http01: 23 | ingress: 24 | class: {{Your ingress technology such as nginx or traefik}} 25 | ``` -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/blue-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: blue 5 | namespace: blue-green-app 6 | spec: 7 | replicas: 2 8 | selector: 9 | matchLabels: 10 | app: blue 11 | template: 12 | metadata: 13 | labels: 14 | app: blue 15 | spec: 16 | containers: 17 | - name: blue-app 18 | image: regoo707/catalog-service:1.2.1 19 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/blue-ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.k8s.io/v1 2 | kind: Ingress 3 | metadata: 4 | name: demo-ingress 5 | labels: 6 | name: demo-ingress 7 | annotations: 8 | cert-manager.io/cluster-issuer: "le-prod" 9 | spec: 10 | ingressClassName: nginx 11 | tls: 12 | - hosts: 13 | - demo-service.ragab.blog 14 | secretName: demo-service-tls 15 | rules: 16 | - host: demo-service.ragab.blog 17 | http: 18 | paths: 19 | - pathType: Prefix 20 | path: "/" 21 | backend: 22 | service: 23 | name: blue-service 24 | port: 25 | number: 80 26 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/blue-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: blue-service 5 | labels: 6 | app: blue 7 | spec: 8 | ports: 9 | - protocol: TCP 10 | port: 80 11 | targetPort: 80 12 | selector: 13 | app: blue 14 | type: ClusterIP 15 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/green-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: green 5 | namespace: blue-green-app 6 | spec: 7 | replicas: 0 8 | selector: 9 | matchLabels: 10 | app: green 11 | template: 12 | metadata: 13 | labels: 14 | app: green 15 | spec: 16 | containers: 17 | - name: green-app 18 | image: regoo707/catalog-service:1.1.5 19 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/green-ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.k8s.io/v1 2 | kind: Ingress 3 | metadata: 4 | name: demo-ingress 5 | labels: 6 | name: demo-ingress 7 | annotations: 8 | cert-manager.io/cluster-issuer: "le-prod" 9 | spec: 10 | ingressClassName: nginx 11 | tls: 12 | - hosts: 13 | - demo-service.ragab.blog 14 | secretName: demo-service-tls 15 | rules: 16 | - host: demo-service.ragab.blog 17 | http: 18 | paths: 19 | - pathType: Prefix 20 | path: "/" 21 | backend: 22 | service: 23 | name: green-service 24 | port: 25 | number: 80 26 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/blue-grean/native/green-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: green-service 5 | labels: 6 | app: green 7 | spec: 8 | ports: 9 | - protocol: TCP 10 | port: 80 11 | targetPort: 80 12 | selector: 13 | app: green 14 | type: ClusterIP 15 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mrgb7/devops-lab/5bffd8452c5e339f3b46159d105d626a1dc3dea9/kubernetes/containous-delivery/progrssive-canary/architecture.png -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/argorollout-cli.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mrgb7/devops-lab/5bffd8452c5e339f3b46159d105d626a1dc3dea9/kubernetes/containous-delivery/progrssive-canary/argorollout-cli.png -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/argorollout-ui.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mrgb7/devops-lab/5bffd8452c5e339f3b46159d105d626a1dc3dea9/kubernetes/containous-delivery/progrssive-canary/argorollout-ui.png -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/canary-demo-argo-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: Application 3 | metadata: 4 | name: canary-demo-web 5 | spec: 6 | destination: 7 | name: '' 8 | namespace: devops-lab-demos 9 | server: 'https://kubernetes.default.svc' 10 | source: 11 | path: kubernetes/containous-delivery/progrssive-canary/config/overlays/dev 12 | repoURL: 'https://github.com/ragoob/devops-lab.git' 13 | targetRevision: HEAD 14 | sources: [] 15 | project: devops-lab-demos 16 | syncPolicy: 17 | automated: 18 | prune: true 19 | selfHeal: true 20 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/canary-demo-web/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang:1.21-bullseye 2 | WORKDIR /usr/src/app 3 | COPY go.mod ./ 4 | RUN go mod download && go mod verify 5 | COPY . . 6 | RUN go build -v -o /usr/local/bin/app ./... 7 | 8 | CMD ["app"] -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/canary-demo-web/go.mod: -------------------------------------------------------------------------------- 1 | module canary-demo-web 2 | 3 | go 1.21.3 4 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/canary-demo-web/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "net/http" 7 | "strconv" 8 | ) 9 | 10 | func main() { 11 | 12 | http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { 13 | if val, err := strconv.ParseBool(r.URL.Query().Get("error")); err == nil && val == true { 14 | http.Error(w, "An error occurred", 500) 15 | return 16 | } 17 | htmlContent := ` 18 | 19 | 20 | 21 | Blue deployment 22 | 23 | 24 |

Simple Go web app!

25 |

Welcome to blue version

26 | 27 | 28 | 29 | ` 30 | fmt.Fprintf(w, htmlContent) 31 | }) 32 | log.Fatal(http.ListenAndServe(":3000", nil)) 33 | } 34 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/canary-demo-web/traffic.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | while true; do 4 | # Make a cURL request to the URL 5 | curl -sS "http://canary-demo-web.ragab.biz?error=true" 6 | 7 | # Wait for 300 milliseconds before the next iteration 8 | sleep 0.3 9 | done 10 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/analysis-template.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: AnalysisTemplate 3 | metadata: 4 | name: success-rate 5 | spec: 6 | args: 7 | - name: service-name 8 | metrics: 9 | - name: success-rate 10 | interval: 1m 11 | successCondition: result[0] >= 0.95 12 | failureLimit: 3 13 | provider: 14 | prometheus: 15 | address: http://prometheus-server.monitoring.svc.cluster.local 16 | timeout: 40 17 | query: | 18 | sum(rate(nginx_ingress_controller_requests{controller_pod=~".*",controller_class=~".*",namespace=~".*",ingress=~"{{args.service-name}}",status!~"[4-5].*"}[2m])) by (ingress) 19 | / 20 | sum(rate(nginx_ingress_controller_requests{controller_pod=~".*",controller_class=~".*",namespace=~".*",ingress=~"{{args.service-name}}"}[2m])) by (ingress) 21 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.k8s.io/v1 2 | kind: Ingress 3 | metadata: 4 | name: canary-demo-web 5 | annotations: 6 | nginx.ingress.kubernetes.io/ssl-passthrough: "true" 7 | nginx.ingress.kubernetes.io/service-upstream: "true" 8 | spec: 9 | ingressClassName: "nginx" 10 | tls: 11 | - hosts: 12 | - canary-demo-web.ragab.home 13 | secretName: ragab-home-tls 14 | rules: 15 | - host: canary-demo-web.ragab.home 16 | http: 17 | paths: 18 | - path: / 19 | pathType: Prefix 20 | backend: 21 | service: 22 | name: canary-demo-web-stable 23 | port: 24 | name: http 25 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | resources: 4 | - analysis-template.yaml 5 | - rollout.yaml 6 | - service.yaml 7 | - service-canary.yaml 8 | - ingress.yaml 9 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/rollout.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: argoproj.io/v1alpha1 2 | kind: Rollout 3 | metadata: 4 | name: canary-demo-web 5 | spec: 6 | replicas: 3 7 | selector: 8 | matchLabels: 9 | app: canary-demo-web 10 | template: 11 | metadata: 12 | labels: 13 | app: canary-demo-web 14 | spec: 15 | containers: 16 | - name: canary-demo-web 17 | image: regoo707/canary-demo-web:1.0.0 18 | ports: 19 | - containerPort: 3000 20 | env: 21 | - name: NODE_NAME 22 | valueFrom: 23 | fieldRef: 24 | fieldPath: spec.nodeName 25 | - name: POD_NAME 26 | valueFrom: 27 | fieldRef: 28 | fieldPath: metadata.name 29 | - name: POD_NAMESPACE 30 | valueFrom: 31 | fieldRef: 32 | fieldPath: metadata.namespace 33 | - name: POD_IP 34 | valueFrom: 35 | fieldRef: 36 | fieldPath: status.podIP 37 | - name: POD_SERVICE_ACCOUNT 38 | valueFrom: 39 | fieldRef: 40 | fieldPath: spec.serviceAccountName 41 | 42 | minReadySeconds: 30 43 | revisionHistoryLimit: 3 44 | strategy: 45 | canary: 46 | canaryService: canary-demo-web-canary # canary backend service use to split traffic 47 | stableService: canary-demo-web-stable # stable backend service 48 | trafficRouting: 49 | nginx: 50 | stableIngress: canary-demo-web 51 | analysis: 52 | templates: 53 | - templateName: success-rate # analysis template object name 54 | startingStep: 2 55 | args: 56 | - name: service-name 57 | value: canary-demo-web 58 | steps: 59 | - setWeight: 20 60 | - pause: {duration: 2m} 61 | - setWeight: 40 62 | - pause: {duration: 3m} 63 | - setWeight: 60 64 | - pause: {duration: 4m} 65 | - setWeight: 80 66 | - pause: {duration: 5m} 67 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/service-canary.yaml: -------------------------------------------------------------------------------- 1 | kind: Service 2 | apiVersion: v1 3 | metadata: 4 | name: canary-demo-web-canary 5 | spec: 6 | selector: 7 | app: canary-demo-web 8 | type: ClusterIP 9 | ports: 10 | - name: http 11 | port: 80 12 | targetPort: 3000 -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/base/service.yaml: -------------------------------------------------------------------------------- 1 | kind: Service 2 | apiVersion: v1 3 | metadata: 4 | name: canary-demo-web-stable 5 | spec: 6 | selector: 7 | app: canary-demo-web 8 | type: ClusterIP 9 | ports: 10 | - name: http 11 | port: 80 12 | targetPort: 3000 -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/config/overlays/dev/kustomization.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: kustomize.config.k8s.io/v1beta1 2 | kind: Kustomization 3 | resources: 4 | - ../../base 5 | images: 6 | - name: regoo707/canary-demo-web 7 | newTag: 1.0.1 8 | -------------------------------------------------------------------------------- /kubernetes/containous-delivery/progrssive-canary/readme.md: -------------------------------------------------------------------------------- 1 | # Progessive Canary deployment using template analysis 2 | 3 | ![Architecture](architecture.png) 4 | 5 | ## Requirments tools 6 | - Kubernetes cluster 7 | ``` 8 | # using Minikube 9 | curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 10 | sudo install minikube-linux-amd64 /usr/local/bin/minikube 11 | minikube start --cpus 8 --memory 8192 12 | # Install Loadbalancer using metallb 13 | minikube addons enable metallb 14 | minikube addons configure metallb 15 | -- Enter Load Balancer Start IP: $(minikube ip) 16 | -- Enter Load Balancer End IP: $(minikube ip) 17 | # Install nginx ingress 18 | minikube addons enable ingress 19 | kubectl patch svc ingress-nginx-controller -n ingress-nginx -p '{"spec": {"type": "LoadBalancer"}}' 20 | 21 | # Now you can use output of minikube ip from your browser you will redirect to nginx default page 22 | 23 | ``` 24 | - Install service-mesh solution for this demo we use (Linkerd) for easy collect metric about services 25 | ``` 26 | # Install linkrd ctl 27 | curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh 28 | export PATH=$HOME/.linkerd2/bin:$PATH 29 | # check if your k8s cluster compatible with linkerd or not 30 | linkerd check --pre 31 | 32 | # Install linkerd CRDS 33 | 34 | linkerd install --crds | kubectl apply -f - 35 | 36 | #Install linkerd core compontents 37 | 38 | linkerd install | kubectl apply -f - 39 | 40 | # check Linkerd installation 41 | 42 | linkerd check 43 | 44 | # Install an on-cluster metric stack and dashboard such as prometheus 45 | 46 | linkerd viz install | kubectl apply -f - 47 | 48 | # check Linkerd 49 | 50 | linkerd check 51 | 52 | ``` 53 | - Install ArgoRollout 54 | 55 | ``` 56 | kubectl create namespace argo-rollouts 57 | kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml 58 | 59 | # install argo-rollouts kubectl plugin 60 | curl -LO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-amd64 61 | chmod +x ./kubectl-argo-rollouts-linux-amd64 62 | 63 | sudo mv ./kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts 64 | # Allow argo-rollout controller to scrape prometheus in viz 65 | linkerd viz allow-scrapes --namespace argo-rollouts | kubectl apply -f - 66 | 67 | ``` 68 | - Install ArgoCD 69 | 70 | ``` 71 | kubectl create namespace argocd 72 | kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml 73 | 74 | # If you want to create L7 ingress to access argocd 75 | 76 | apiVersion: networking.k8s.io/v1 77 | kind: Ingress 78 | metadata: 79 | name: argocd-server 80 | namespace: argocd 81 | annotations: 82 | nginx.ingress.kubernetes.io/ssl-passthrough: "true" 83 | nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" 84 | nginx.ingress.kubernetes.io/force-ssl-redirect: "true" 85 | spec: 86 | ingressClassName: "nginx" 87 | rules: 88 | - host: argocd.ragab.biz 89 | http: 90 | paths: 91 | - path: / 92 | pathType: Prefix 93 | backend: 94 | service: 95 | name: argocd-server 96 | port: 97 | name: https 98 | 99 | # to access the local ingress edit /etc/hosts and add the dns record with the ip of minikube 100 | 101 | # We can also use LoadBalancer IP with argocd PORT 102 | kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}' 103 | 104 | ``` 105 | - Install ArgoCD CLI 106 | 107 | ``` 108 | curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64 109 | sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd 110 | rm argocd-linux-amd64 111 | 112 | ``` 113 | 114 | ## Deploy ArgoCD project 115 | 116 | ``` 117 | # if you have L7 ingress you can use it or port-forward argocd server service 118 | kubectl port-forward svc/argocd-server 8080:8080 -n argocd 119 | 120 | export CONFIG_REPO=https://github.com/ragoob/devops-lab.git 121 | 122 | export ARGOCD_SERVER=argocd.ragab.biz 123 | 124 | # To get admin password 125 | kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 --decode 126 | 127 | argocd login $ARGOCD_SERVER 128 | argocd repo add $CONFIG_REPO 129 | 130 | # Create application namespace 131 | 132 | kubectl create namespace devops-lab-demos 133 | 134 | # Inject devops-lab-demos namespace with linkerd proxy sidecar 135 | kubectl annotate namespace devops-lab-demos linkerd.io/inject=enabled 136 | 137 | # create argocd project 138 | 139 | 140 | argocd proj create devops-lab-demos -d https://kubernetes.default.svc,devops-lab-demos -s $CONFIG_REPO 141 | # Create argocd app to sync k8s objects 142 | kubectl apply -f canary-demo-argo-app.yaml -n argocd 143 | # Note : You can create app using argocd UI 144 | ``` 145 | # Explain K8s manifest 146 | - Using Kustomize 147 | we have the following objects 148 | - argorollout which is replacement of traditional k8s deployment object but allow extra deployment 149 | stratgies such as blue-green, canary , progressive 150 | ``` 151 | apiVersion: argoproj.io/v1alpha1 152 | kind: Rollout 153 | metadata: 154 | name: canary-demo-web 155 | spec: 156 | replicas: 3 157 | selector: 158 | matchLabels: 159 | app: canary-demo-web 160 | template: 161 | metadata: 162 | labels: 163 | app: canary-demo-web 164 | spec: 165 | containers: 166 | - name: canary-demo-web 167 | image: regoo707/canary-demo-web:1.0.0 168 | ports: 169 | - containerPort: 3000 170 | minReadySeconds: 30 171 | revisionHistoryLimit: 3 172 | strategy: 173 | canary: 174 | canaryService: canary-demo-web-canary # canary backend service use to split traffic 175 | stableService: canary-demo-web-stable # stable backend service 176 | trafficRouting: 177 | nginx: 178 | stableIngress: canary-demo-web 179 | analysis: 180 | templates: 181 | - templateName: success-rate # analysis template object name 182 | startingStep: 2 183 | args: 184 | - name: service-name 185 | value: canary-demo-web 186 | steps: 187 | - setWeight: 20 188 | - pause: {duration: 2m} 189 | - setWeight: 40 190 | - pause: {duration: 3m} 191 | - setWeight: 60 192 | - pause: {duration: 4m} 193 | - setWeight: 80 194 | - pause: {duration: 5m} 195 | ``` 196 | - Two service objects one for stable ingress and the other one for canary ingress 197 | that argorollout controller will create it on the fly to control the traffic 198 | ``` 199 | kind: Service 200 | apiVersion: v1 201 | metadata: 202 | name: canary-demo-web-stable 203 | spec: 204 | selector: 205 | app: canary-demo-web 206 | type: ClusterIP 207 | ports: 208 | - name: http 209 | port: 80 210 | targetPort: 3000 211 | --- 212 | kind: Service 213 | apiVersion: v1 214 | metadata: 215 | name: canary-demo-web-canary 216 | spec: 217 | selector: 218 | app: canary-demo-web 219 | type: ClusterIP 220 | ports: 221 | - name: http 222 | port: 80 223 | targetPort: 3000 224 | ``` 225 | - Ingress for stable service (The canary one will create and mange by argorollout controller) 226 | ``` 227 | apiVersion: networking.k8s.io/v1 228 | kind: Ingress 229 | metadata: 230 | name: canary-demo-web 231 | namespace: devops-lab-demos 232 | annotations: 233 | nginx.ingress.kubernetes.io/service-upstream: "true" 234 | spec: 235 | ingressClassName: "nginx" 236 | rules: 237 | - host: canary-demo-web.ragab.biz 238 | http: 239 | paths: 240 | - path: / 241 | pathType: Prefix 242 | backend: 243 | service: 244 | name: canary-demo-web-stable 245 | port: 246 | name: http 247 | ``` 248 | - Analysis template used as source of health of deployment we use prometheus http sucess response / total response 249 | ``` 250 | apiVersion: argoproj.io/v1alpha1 251 | kind: AnalysisTemplate 252 | metadata: 253 | name: success-rate 254 | spec: 255 | args: 256 | - name: service-name 257 | metrics: 258 | - name: success-rate 259 | interval: 1m 260 | successCondition: result[0] >= 0.95 261 | failureLimit: 3 262 | provider: 263 | prometheus: 264 | address: http://prometheus.linkerd-viz.svc.cluster.local:9090 # prometheus instance from linkerd-viz 265 | timeout: 40 266 | query: | 267 | sum(irate(response_total{direction = "inbound", app="{{args.service-name}}",status_code !~"5.*"}[1m])) 268 | / 269 | sum(irate(response_total{direction = "inbound",app="{{args.service-name}}"}[1m])) 270 | 271 | ``` 272 | - Overlays for kustomize environments 273 | 274 | # Explain 275 | - When deploying a new version (e.g., 1.0.1), ArgoRollout dynamically generates a canary ingress with two distinct annotations for nginx: 276 | 277 | ``` 278 | nginx.ingress.kubernetes.io/canary: "true" 279 | nginx.ingress.kubernetes.io/canary-weight: "0" 280 | # https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary 281 | ``` 282 | 283 | - This facilitates traffic division between the stable version 1.0.0 and the new version 1.0.1, leveraging a Prometheus query result: 284 | 285 | ``` 286 | sum(irate(response_total{direction = "inbound", app="{{args.service-name}}", status_code !~ "5.*"}[1m])) 287 | / 288 | sum(irate(response_total{direction = "inbound", app="{{args.service-name}}"}[1m])) 289 | 290 | ``` 291 | 292 | 293 | - Under successful conditions, the nginx.ingress.kubernetes.io/canary-weight will increment based on the step template provided in the analysis. This involves incremental increases at 20%, 40%, and 60% until version 1.0.0 is entirely replaced by 1.0.1. Simultaneously, the routing of traffic to 1.0.0 will cease, and all replica pods running 1.0.0 will be stopped. 294 | 295 | - However, if the query yields insufficient results, a rollback to version 1.0.0 will commence, and the progression of the canary deployment will be halted." 296 | 297 | 298 | - To check progress of canary deployment 299 | ``` 300 | kubectl argo rollouts get rollout canary-demo-web -n devops-lab-demos -w 301 | 302 | ``` 303 | You should see 304 | ![canary progressive](argorollout-cli.png) 305 | 306 | You can also use ArgoRollouts dashboard 307 | ``` 308 | kubectl argo rollouts dashboard 309 | 310 | ``` 311 | Then open http://localhost:3100/rollouts 312 | You should see 313 | ![canary progressive UI](argorollout-ui.png) 314 | 315 | ## Note : Do not forget to generate some traffic to avoid progression failure due to insufficient metrics 316 | -------------------------------------------------------------------------------- /kubernetes/nginx/delete.sh: -------------------------------------------------------------------------------- 1 | cd kubernetes-ingress/deployments 2 | kubectl delete -f rbac/rbac.yaml 3 | kubectl delete -f rbac/ap-rbac.yaml 4 | kubectl delete -f rbac/apdos-rbac.yaml 5 | kubectl delete -f ../examples/shared-examples/default-server-secret/default-server-secret.yaml 6 | kubectl delete -f common/nginx-config.yaml 7 | kubectl delete -f common/ingress-class.yaml 8 | kubectl delete -f common/crds/k8s.nginx.org_virtualservers.yaml 9 | kubectl delete -f common/crds/k8s.nginx.org_virtualserverroutes.yaml 10 | kubectl delete -f common/crds/k8s.nginx.org_transportservers.yaml 11 | kubectl delete -f common/crds/k8s.nginx.org_policies.yaml 12 | kubectl delete -f common/crds/k8s.nginx.org_globalconfigurations.yaml 13 | kubectl delete -f common/crds/appprotect.f5.com_aplogconfs.yaml 14 | kubectl delete -f common/crds/appprotect.f5.com_appolicies.yaml 15 | kubectl delete -f common/crds/appprotect.f5.com_apusersigs.yaml 16 | # DDOS protection 17 | kubectl delete -f common/crds/appprotectdos.f5.com_apdoslogconfs.yaml 18 | kubectl delete -f common/crds/appprotectdos.f5.com_apdospolicy.yaml 19 | kubectl delete -f common/crds/appprotectdos.f5.com_dosprotectedresources.yaml 20 | kubectl delete -f deployment/appprotect-dos-arb.yaml 21 | kubectl delete -f service/appprotect-dos-arb-svc.yaml 22 | kubectl delete -f deployment/nginx-ingress.yaml 23 | kubectl delete -f daemon-set/nginx-ingress.yaml 24 | kubectl delete -f service/loadbalancer.yaml 25 | kubectl delete -f common/ns-and-sa.yaml 26 | -------------------------------------------------------------------------------- /kubernetes/nginx/install.sh: -------------------------------------------------------------------------------- 1 | git clone https://github.com/nginxinc/kubernetes-ingress.git --branch v3.2.0 2 | cd kubernetes-ingress/deployments 3 | kubectl apply -f common/ns-and-sa.yaml 4 | kubectl apply -f rbac/rbac.yaml 5 | kubectl apply -f rbac/ap-rbac.yaml 6 | kubectl apply -f rbac/apdos-rbac.yaml 7 | kubectl apply -f ../examples/shared-examples/default-server-secret/default-server-secret.yaml 8 | kubectl apply -f common/nginx-config.yaml 9 | kubectl apply -f common/ingress-class.yaml 10 | kubectl apply -f common/crds/k8s.nginx.org_virtualservers.yaml 11 | kubectl apply -f common/crds/k8s.nginx.org_virtualserverroutes.yaml 12 | kubectl apply -f common/crds/k8s.nginx.org_transportservers.yaml 13 | kubectl apply -f common/crds/k8s.nginx.org_policies.yaml 14 | kubectl apply -f common/crds/k8s.nginx.org_globalconfigurations.yaml 15 | kubectl apply -f common/crds/appprotect.f5.com_aplogconfs.yaml 16 | kubectl apply -f common/crds/appprotect.f5.com_appolicies.yaml 17 | kubectl apply -f common/crds/appprotect.f5.com_apusersigs.yaml 18 | # DDOS protection 19 | kubectl apply -f common/crds/appprotectdos.f5.com_apdoslogconfs.yaml 20 | kubectl apply -f common/crds/appprotectdos.f5.com_apdospolicy.yaml 21 | kubectl apply -f common/crds/appprotectdos.f5.com_dosprotectedresources.yaml 22 | kubectl apply -f deployment/appprotect-dos-arb.yaml 23 | kubectl apply -f service/appprotect-dos-arb-svc.yaml 24 | kubectl apply -f deployment/nginx-ingress.yaml 25 | kubectl apply -f daemon-set/nginx-ingress.yaml 26 | kubectl apply -f service/loadbalancer.yaml 27 | -------------------------------------------------------------------------------- /kubernetes/nginx/readme.md: -------------------------------------------------------------------------------- 1 | ## Install nginx ingress controller native 2 | ``` 3 | git clone https://github.com/nginxinc/kubernetes-ingress.git --branch v3.2.0 4 | cd kubernetes-ingress/deployments 5 | kubectl apply -f common/ns-and-sa.yaml 6 | kubectl apply -f rbac/rbac.yaml 7 | kubectl apply -f rbac/ap-rbac.yaml 8 | kubectl apply -f rbac/apdos-rbac.yaml 9 | kubectl apply -f ../examples/shared-examples/default-server-secret/default-server-secret.yaml 10 | kubectl apply -f common/nginx-config.yaml 11 | kubectl apply -f common/ingress-class.yaml 12 | kubectl apply -f common/crds/k8s.nginx.org_virtualservers.yaml 13 | kubectl apply -f common/crds/k8s.nginx.org_virtualserverroutes.yaml 14 | kubectl apply -f common/crds/k8s.nginx.org_transportservers.yaml 15 | kubectl apply -f common/crds/k8s.nginx.org_policies.yaml 16 | kubectl apply -f common/crds/k8s.nginx.org_globalconfigurations.yaml 17 | kubectl apply -f common/crds/appprotect.f5.com_aplogconfs.yaml 18 | kubectl apply -f common/crds/appprotect.f5.com_appolicies.yaml 19 | kubectl apply -f common/crds/appprotect.f5.com_apusersigs.yaml 20 | # DDOS protection 21 | kubectl apply -f common/crds/appprotectdos.f5.com_apdoslogconfs.yaml 22 | kubectl apply -f common/crds/appprotectdos.f5.com_apdospolicy.yaml 23 | kubectl apply -f common/crds/appprotectdos.f5.com_dosprotectedresources.yaml 24 | kubectl apply -f deployment/appprotect-dos-arb.yaml 25 | kubectl apply -f service/appprotect-dos-arb-svc.yaml 26 | 27 | kubectl apply -f deployment/nginx-ingress.yaml 28 | kubectl apply -f daemon-set/nginx-ingress.yaml 29 | kubectl apply -f service/loadbalancer.yaml 30 | 31 | ``` 32 | ## Install nginx ingress controller using helm 33 | ``` 34 | #Using Helm 35 | helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx 36 | helm repo update 37 | helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace --set controller.replicaCount=3 38 | 39 | ``` -------------------------------------------------------------------------------- /kubernetes/observability/README.md: -------------------------------------------------------------------------------- 1 | - [Metrics](metrics) 2 | - [Distributed tracing](tracing) -------------------------------------------------------------------------------- /kubernetes/observability/metrics/README.md: -------------------------------------------------------------------------------- 1 | - [prometheus and grafana](prometheus) -------------------------------------------------------------------------------- /kubernetes/observability/metrics/prometheus/architecture-min.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mrgb7/devops-lab/5bffd8452c5e339f3b46159d105d626a1dc3dea9/kubernetes/observability/metrics/prometheus/architecture-min.png -------------------------------------------------------------------------------- /kubernetes/observability/metrics/prometheus/conf.yaml: -------------------------------------------------------------------------------- 1 | cat: can't open 'C:/Program Files/Git/etc/prometheus/config_out/prometheus.env.yaml': No such file or directory 2 | -------------------------------------------------------------------------------- /kubernetes/observability/metrics/prometheus/readme.md: -------------------------------------------------------------------------------- 1 | ## How it works 2 | ![Alt Text](architecture-min.png) 3 | 4 | ### Add helm repo 5 | ``` 6 | helm repo add stable https://charts.helm.sh/stable 7 | 8 | ``` 9 | ### Add the Prometheus community helm chart in Kubernetes 10 | 11 | ``` 12 | helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 13 | 14 | ``` 15 | ``` 16 | helm search repo prometheus-community 17 | helm install [RELEASE_NAME] prometheus-community/kube-prometheus-stack 18 | 19 | ``` 20 | 21 | ## Install prometheus with grafana custom values and with postgres exporter 22 | ``` 23 | 24 | helm install prometheus-stack prometheus-community/kube-prometheus-stack -f values.yaml 25 | # Install with postgres exporter 26 | helm upgrade --install prometheus-stack prometheus-community/kube-prometheus-stack -f values.yaml 27 | 28 | ``` -------------------------------------------------------------------------------- /kubernetes/observability/metrics/prometheus/values.yaml: -------------------------------------------------------------------------------- 1 | prometheus: 2 | prometheusSpec: 3 | additionalScrapeConfigs: 4 | - job_name: postgres-exporter 5 | static_configs: 6 | - targets: ['prometheus-postgres-exporter:80'] 7 | grafana: 8 | persistence: 9 | enabled: true 10 | storageClassName: managed-premium 11 | size: "1Gi" 12 | service: 13 | type: LoadBalancer -------------------------------------------------------------------------------- /kubernetes/observability/tracing/README.md: -------------------------------------------------------------------------------- 1 | - [Jaeger](jaeger-distributed-tracing) -------------------------------------------------------------------------------- /kubernetes/observability/tracing/jaeger-distributed-tracing/readme.md: -------------------------------------------------------------------------------- 1 | ### Jaeger: open source, end-to-end distributed tracing 2 | Monitor and troubleshoot transactions in complex distributed systems 3 | 4 | ### Install on K8s using helm 5 | ``` 6 | helm repo add jaeger https://jaegertracing.github.io/helm-charts 7 | helm repo update 8 | # Default storage is cassandra you can override it by --set storage.type={storage type} 9 | helm install jaeger jaeger/jaeger --namespace observability 10 | kubectl port-forward -n jaeger service/jaeger-query 16686:16686 11 | # Load balancer expose 12 | helm upgrade --install jaeger jaeger/jaeger --namespace observability --set query.service.type=LoadBalancer --set storage.type=memory 13 | 14 | ``` -------------------------------------------------------------------------------- /kubernetes/postgres/README.md: -------------------------------------------------------------------------------- 1 | - [HA postgres cluster via patroni](patroni) -------------------------------------------------------------------------------- /kubernetes/postgres/patroni/readme.md: -------------------------------------------------------------------------------- 1 | ### Install Postgres HA cluster using Patroni 2 | 3 | ``` 4 | # First, clone the repository and change to the directory 5 | git clone https://github.com/zalando/postgres-operator.git 6 | cd postgres-operator 7 | 8 | # apply the manifests in the following order 9 | kubectl create -f manifests/configmap.yaml # configuration 10 | kubectl create -f manifests/operator-service-account-rbac.yaml # identity and permissions 11 | kubectl create -f manifests/postgres-operator.yaml # deployment 12 | kubectl create -f manifests/api-service.yaml # operator API to be used by UI 13 | 14 | ``` -------------------------------------------------------------------------------- /kubernetes/readme.md: -------------------------------------------------------------------------------- 1 | ## Kubernetes resources 2 | 3 | - [Argocd](argocd) 4 | - [Cert manager](cert-manager) 5 | - [Observability](observability) 6 | - [Postgres clustering](postgres) 7 | - [Traefik ingress](traefik) 8 | - [nginx ingress](nginx) 9 | - [K8s troubleshooting](trouble-shooting) 10 | - [K8s containous-delivery progressive canary deployment](containous-delivery/progrssive-canary/) -------------------------------------------------------------------------------- /kubernetes/traefik/readme.md: -------------------------------------------------------------------------------- 1 | ## Install traefik using helm 2 | ``` 3 | helm install traefik/traefik --namespace=traefik --version=23.1.0 4 | ``` -------------------------------------------------------------------------------- /kubernetes/trouble-shooting/README.md: -------------------------------------------------------------------------------- 1 | - [Delete suck namespaces](delete-terminating-stuck-resources) -------------------------------------------------------------------------------- /kubernetes/trouble-shooting/delete-terminating-stuck-resources/force-delete.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import atexit 3 | import json 4 | import requests 5 | import subprocess 6 | import sys 7 | 8 | namespace = sys.argv[1] 9 | proxy_process = subprocess.Popen(['kubectl', 'proxy']) 10 | atexit.register(proxy_process.kill) 11 | p = subprocess.Popen(['kubectl', 'get', 'namespace', namespace, '-o', 'json'], stdout=subprocess.PIPE) 12 | p.wait() 13 | data = json.load(p.stdout) 14 | data['spec']['finalizers'] = [] 15 | requests.put('http://127.0.0.1:8001/api/v1/namespaces/{}/finalize'.format(namespace), json=data).raise_for_status() -------------------------------------------------------------------------------- /kubernetes/trouble-shooting/delete-terminating-stuck-resources/readme.md: -------------------------------------------------------------------------------- 1 | ## Python script to delete stuck namespace 2 | ``` 3 | python3 force-delete.py {stuck-namespace} 4 | 5 | ``` -------------------------------------------------------------------------------- /roadmap.md: -------------------------------------------------------------------------------- 1 | ## My Devops roadmap 2 | ### Software Development Process 3 | - Git 4 | - GitHub, GitLab, or Azure DevOps 5 | 6 | ### Operating Systems 7 | - Linux Management 8 | - Process Management and Monitoring 9 | - Performance Monitoring 10 | - Text Manipulation 11 | - Bash Scripting 12 | - Networking and security 13 | - ubuntu , alpine , .. 14 | 15 | ### 16 | 17 | ### Web Servers and Reverse Proxies 18 | - Nginx 19 | - HAProxy 20 | - Load Balancing 21 | 22 | ### Programming Skills for Automation 23 | - Python or Go 24 | ### Containerization 25 | - Docker 26 | - containerd 27 | 28 | ### Orchestration 29 | - Kubernetes 30 | 31 | ### Cloud Providers 32 | - Azure or AWS 33 | 34 | ### Service Mesh 35 | - Linkerd 36 | - Istio 37 | 38 | ### Observability 39 | - Monitoring and Metrics (Prometheus , thanos, verctoria metric and Grafana) 40 | - Logging 41 | (ELK or Loki) 42 | fluentd 43 | - Distributed Tracing 44 | Open telemetry 45 | (Jaeger or honycomb or ..) 46 | 47 | ### CI/CD 48 | - Continuous Integration (Jenkins, GitHub Actions, GitLab, or Azure DevOps) 49 | - Continuous Deployment (ArgoCD) 50 | - ArgoRollout / Flagger (Canary , blue-green , experimentation , progressive delivery) 51 | 52 | ### Secret Management 53 | - HashiCorp Vault 54 | - External secrets 55 | - seald secret operator 56 | 57 | ### Infrastructure as Code 58 | - Terraform 59 | - Ansible 60 | 61 | ### Cloud Design Patterns 62 | - Elastic scale 63 | - Availability 64 | --------------------------------------------------------------------------------