├── .gitignore ├── LICENSE ├── README.md ├── aks-deploy.sh ├── aks2eks-mysql-policy.sh ├── aks2gke-mysql-policy.sh ├── aks2oke-mysql-policy.sh ├── az-location.sh ├── createapp.sh ├── deploy.sh ├── destroy.sh ├── k10-deploy.sh ├── k10-destroy.sh ├── k10status.sh ├── migration-oci-location.sh ├── migration-s3-location.sh ├── mysql-policy.sh ├── runonce.sh └── setenv.sh /.gitignore: -------------------------------------------------------------------------------- 1 | *token* 2 | *yong1app* 3 | *access* 4 | *key* 5 | *backupname* 6 | *exportname* 7 | *k10primer* 8 | 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Yongkang He 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | #### Follow [@YongkangHe](https://twitter.com/yongkanghe) on Twitter, Subscribe [K8s Data Management](https://www.youtube.com/channel/UCm-sw1b23K-scoVSCDo30YQ?sub_confirmation=1) Youtube Channel 2 | 3 | I just want to build an AKS Cluster to play with the various Data Management capabilities e.g. Container's Backup/Restore, Disaster Recovery and Application Mobility. 4 | 5 | It is challenging to create AKS Cluster from Azure Cloud if you are not familiar to it. After the AKS Cluster is up running, we still need to install Kasten, create a sample database, create location profile, backup policies etc.. The whole process is not that simple. 6 | 7 | ![image](https://blog.kasten.io/hs-fs/hubfs/Kasten_January2020/Images/microsoft-azure-with-kasten-k10-intro-blog.png?width=1226&name=microsoft-azure-with-kasten-k10-intro-blog.png) 8 | 9 | 10 | This script based automation allows you to build a ready-to-use Kasten K10 demo environment running on AKS in about 10 minutes. For simplicity and cost optimization, the AKS cluster will have only one worker node in the newly created vnet and subnet. This is bash shell based scripts which has been tested on Cloud Shell. Linux or MacOS terminal has not been tested though it might work as well. If you don't have an Azure account, please watch the video by Louisa below to sign up a free trial account in 5 minutes. 11 | 12 | # Sign up an Azure trial account 13 | [![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/FN0ARvEdrjg/0.jpg)](https://www.youtube.com/watch?v=FN0ARvEdrjg) 14 | #### Subscribe [K8s Data Management](https://www.youtube.com/channel/UCm-sw1b23K-scoVSCDo30YQ?sub_confirmation=1) Youtube Channel 15 | 16 | # Here're the prerequisities. 17 | 1. Go to open Azure Cloud Shell 18 | 2. Clone the github repo to your local host, run below command 19 | ```` 20 | git clone https://github.com/yongkanghe/aks-k10.git 21 | ```` 22 | 3. Create Azure App Registration first 23 | ```` 24 | cd aks-k10;./createapp.sh 25 | ```` 26 | 4. Optionally, you can customize the clustername, vm size, location, region, containername, etc. 27 | ```` 28 | vi setenv.sh 29 | ```` 30 | 31 | # Deploy based on your needs 32 | 33 | | Don't have an AKS cluster | Already have an AKS cluster | Have nothing | 34 | |---------------------------|----------------------------------|----------------------------------| 35 | | Deploy AKS only | Deploy K10 only | Deploy AKS and K10 | 36 | | ``` ./aks-deploy.sh ``` | ``` ./k10-deploy.sh ``` | ``` ./deploy.sh ``` | 37 | | 1. Create an AKS Cluster | | 1. Create an AKS Cluster | 38 | | | 1. Install Kasten K10 | 2. Install Kasten K10 | 39 | | | 2. Deploy a MySQL database | 3. Deploy a MySQL database | 40 | | | 3. Create an Azure Blob location | 4. Create an Azure Blob location | 41 | | | 4. Create a backup policy | 5. Create a backup policy | 42 | | | 5. Kick off on-demand backup job | 6. Kick off on-demand backup job | 43 | 44 | # Destroy based on your needs 45 | 46 | | Destroy AKS only | Destroy K10 only | Destroy AKS and K10 | 47 | |---------------------------|--------------------------------------|-------------------------------------| 48 | | ``` ./aks-destroy.sh ``` | ``` ./k10-destroy.sh ``` | ``` ./destroy.sh ``` | 49 | | 1. Remove the AKS Cluster | | 1. Remove the Resource Group | 50 | | | 1. Remove MySQL database | + Remove AKS Kubernetes Cluster | 51 | | | 2. Remove Kasten K10 | + Remove the disks and snapshots | 52 | | | 3. Remove Azure Blob storage bucket | + Remove the storage account etc.| 53 | 54 | # To kickoff a backup job manually, run 55 | ```` 56 | ./runonce.sh 57 | ```` 58 | + Take a snapshot of Application Components 59 | + Take a snapshot of Application Configurations 60 | + Take a snapshot of Workload MySQL 61 | + Export the snapshot to Azure Blob Storage 62 | 63 | # Automate AKS creation & protection 64 | [![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/308ZOMRaRDk/0.jpg)](https://www.youtube.com/watch?v=308ZOMRaRDk) 65 | #### Subscribe [K8s Data Management](https://www.youtube.com/channel/UCm-sw1b23K-scoVSCDo30YQ?sub_confirmation=1) Youtube Channel 66 | 67 | # AKS Backup and Restore 68 | https://blog.kasten.io/posts/backup-and-recovery-in-microsoft-azure-with-kasten-k10/ 69 | 70 | # Kasten - No. 1 Self Managed Kubernetes Backup 71 | https://kasten.io 72 | 73 | # CloudCasa - Best Kubernetes Backup as a Service 74 | https://cloudcasa.io 75 | 76 | # Contributors 77 | 78 | #### Follow [Yongkang He](http://yongkang.cloud) on LinkedIn, Join [Kubernetes Data Management](https://www.linkedin.com/groups/13983251) LinkedIn Group 79 | 80 | #### Follow [Louisa He](https://www.linkedin.com/in/louisahe/) on LinkedIn, Join [Kubernetes Data Management](http://t.k8sug.com) Telegram Group 81 | -------------------------------------------------------------------------------- /aks-deploy.sh: -------------------------------------------------------------------------------- 1 | echo '-------Creating an AKS Cluster only (~5 mins)' 2 | starttime=$(date +%s) 3 | . ./setenv.sh 4 | az group create --name $MY_PREFIX-$MY_GROUP --location $MY_LOCATION 5 | AKS_K8S_VERSION=$(az aks get-versions --location $MY_LOCATION --output table | awk '{print $1}' | grep $K8S_VERSION | head -1) 6 | az aks create \ 7 | --resource-group $MY_PREFIX-$MY_GROUP \ 8 | --name $MY_PREFIX-$MY_CLUSTER-$(date +%s) \ 9 | --location $MY_LOCATION \ 10 | --generate-ssh-keys \ 11 | --kubernetes-version $AKS_K8S_VERSION \ 12 | --node-count 2 \ 13 | --node-vm-size $MY_VMSIZE \ 14 | --enable-cluster-autoscaler \ 15 | --min-count 2 \ 16 | --max-count 3 \ 17 | --network-plugin azure 18 | 19 | az aks get-credentials -g $MY_PREFIX-$MY_GROUP -n $(az aks list -o table | grep $MY_PREFIX-$MY_CLUSTER | awk '{print $1}') 20 | 21 | echo '-------Create a Azure Storage account' 22 | AKS_RG=$(az group list -o table | grep $MY_PREFIX-$MY_GROUP | grep MC | awk '{print $1}') 23 | az storage account create -n $MY_PREFIX$AZURE_STORAGE_ACCOUNT_ID -g $AKS_RG -l $MY_LOCATION --sku Standard_LRS 24 | echo $(az storage account keys list -g $AKS_RG -n $MY_PREFIX$AZURE_STORAGE_ACCOUNT_ID --query [].value -o tsv | head -1) > az_storage_key 25 | 26 | echo "" | awk '{print $1}' 27 | endtime=$(date +%s) 28 | duration=$(( $endtime - $starttime )) 29 | echo "-------Total time to build an AKS Cluster is $(($duration / 60)) minutes $(($duration % 60)) seconds." 30 | echo "" | awk '{print $1}' 31 | echo "-------Created by Yongkang" 32 | echo "-------Email me if any suggestions or issues he@yongkang.cloud" 33 | echo "" | awk '{print $1}' 34 | -------------------------------------------------------------------------------- /aks2eks-mysql-policy.sh: -------------------------------------------------------------------------------- 1 | # export MY_REGION=us-east-2 2 | export MY_OBJECT_STORAGE_PROFILE=myaws3-migration 3 | 4 | echo '------Create backup policies' 5 | cat < aks4yong1app 4 | 5 | clear 6 | 7 | echo "-------App Registration has been created" 8 | echo "" | awk '{print $1}' 9 | echo "You are ready to deploy now!" 10 | echo "" | awk '{print $1}' -------------------------------------------------------------------------------- /deploy.sh: -------------------------------------------------------------------------------- 1 | echo '-------Creating an AKS Cluster + K10 (~8 mins)' 2 | starttime=$(date +%s) 3 | 4 | #Create an AKS cluster 5 | ./aks-deploy.sh 6 | 7 | #Deploy K10 + sample DB + backup policy 8 | ./k10-deploy.sh 9 | 10 | endtime=$(date +%s) 11 | duration=$(( $endtime - $starttime )) 12 | echo "-------Total time for AKS+K10 deployment is $(($duration / 60)) minutes $(($duration % 60)) seconds." 13 | echo "" | awk '{print $1}' 14 | -------------------------------------------------------------------------------- /destroy.sh: -------------------------------------------------------------------------------- 1 | starttime=$(date +%s) 2 | . ./setenv.sh 3 | 4 | echo '-------Deleting an AKS Cluster (typically in less than 10 mins)' 5 | az group delete -g $MY_PREFIX-$MY_GROUP --yes 6 | kubectl config delete-context $(kubectl config get-contexts | grep $MY_CLUSTER | awk '{print $2}') 7 | echo "" | awk '{print $1}' 8 | 9 | # echo '-------Deleting the app registration created by AKS' 10 | # MYID=$(az ad sp list --show-mine --query [].servicePrincipalNames -o table | grep $MY_PREFIX-$MY_GROUP | awk '{print $2}') 11 | # az ad app delete --id $MYID 12 | 13 | endtime=$(date +%s) 14 | duration=$(( $endtime - $starttime )) 15 | echo "-------Total time to clean up is $(($duration / 60)) minutes $(($duration % 60)) seconds." 16 | echo "" | awk '{print $1}' 17 | echo "-------Created by Yongkang" 18 | echo "-------Email me if any suggestions or issues he@yongkang.cloud" 19 | echo "" | awk '{print $1}' -------------------------------------------------------------------------------- /k10-deploy.sh: -------------------------------------------------------------------------------- 1 | echo '-------Deploying Kasten K10 and MySQL' 2 | starttime=$(date +%s) 3 | . ./setenv.sh 4 | 5 | echo '-------Exporting the Azure Tenant, Client, Secret' 6 | AZURE_TENANT_ID=$(cat aks4yong1app | grep tenant | awk '{print $2}' | sed -e 's/\"//g') 7 | AZURE_CLIENT_ID=$(cat aks4yong1app | grep appId | awk '{print $2}' | sed -e 's/\"//g' | sed -e 's/\,//g') 8 | AZURE_CLIENT_SECRET=$(cat aks4yong1app | grep password | awk '{print $2}' | sed -e 's/\"//g' | sed -e 's/\,//g') 9 | 10 | echo '-------Install K10' 11 | kubectl create ns kasten-io 12 | helm repo add kasten https://charts.kasten.io/ 13 | helm repo update 14 | 15 | #For Production, remove the lines ending with =1Gi from helm install 16 | #For Production, remove the lines ending with airgap from helm install 17 | helm install k10 kasten/k10 --namespace=kasten-io \ 18 | --set secrets.azureTenantId=$AZURE_TENANT_ID \ 19 | --set secrets.azureClientId=$AZURE_CLIENT_ID \ 20 | --set secrets.azureClientSecret=$AZURE_CLIENT_SECRET \ 21 | --set global.persistence.metering.size=1Gi \ 22 | --set prometheus.server.persistentVolume.size=1Gi \ 23 | --set global.persistence.catalog.size=1Gi \ 24 | --set global.persistence.jobs.size=1Gi \ 25 | --set global.persistence.logging.size=1Gi \ 26 | --set global.persistence.grafana.size=1Gi \ 27 | --set auth.tokenAuth.enabled=true \ 28 | --set externalGateway.create=true \ 29 | --set metering.mode=airgap 30 | 31 | echo '-------Set the default ns to k10' 32 | kubectl config set-context --current --namespace kasten-io 33 | 34 | echo '-------Creating an azure disk vsc' 35 | cat < aks_token 57 | echo My Cluster ID is $clusterid >> aks_token 58 | kubectl wait --for=condition=ready --timeout=180s -n kasten-io pod -l component=jobs 59 | k10ui=http://$(kubectl get svc gateway-ext | awk '{print $4}'|grep -v EXTERNAL)/k10/# 60 | echo -e "\nHere is the URL to log into K10 Web UI" >> aks_token 61 | echo -e "\n$k10ui" >> aks_token 62 | echo "" | awk '{print $1}' >> aks_token 63 | 64 | cat <> aks_token 76 | echo "" | awk '{print $1}' >> aks_token 77 | kubectl get secret k10-k10-yong --namespace kasten-io -ojsonpath="{.data.token}{'\n'}" | base64 --decode | awk '{print $1}' >> aks_token 78 | echo "" | awk '{print $1}' >> aks_token 79 | 80 | echo '-------Waiting for K10 services are up running in about 3 mins more or less' 81 | kubectl wait --for=condition=ready --timeout=300s -n kasten-io pod -l component=catalog 82 | 83 | ./az-location.sh 84 | 85 | ./mysql-policy.sh 86 | 87 | echo '-------Accessing K10 UI' 88 | cat aks_token 89 | 90 | endtime=$(date +%s) 91 | duration=$(( $endtime - $starttime )) 92 | echo "-------Total time for K10 deployment is $(($duration / 60)) minutes $(($duration % 60)) seconds." 93 | echo "" | awk '{print $1}' 94 | echo "-------Created by Yongkang" 95 | echo "-------Email me if any suggestions or issues he@yongkang.cloud" 96 | echo "" | awk '{print $1}' 97 | -------------------------------------------------------------------------------- /k10-destroy.sh: -------------------------------------------------------------------------------- 1 | starttime=$(date +%s) 2 | . ./setenv.sh 3 | 4 | echo '-------Deleting MySQL and Kasten K10' 5 | helm uninstall mysql -n yong-mysql 6 | helm uninstall k10 -n kasten-io 7 | kubectl delete ns yong-mysql 8 | kubectl delete ns kasten-io 9 | 10 | echo '-------Deleting objects from Azure Blob Storage Bucket' 11 | az storage blob delete-batch --account-name $MY_PREFIX$AZURE_STORAGE_ACCOUNT_ID -s $MY_PREFIX-$MY_CONTAINER --account-key $(cat az_storage_key) 12 | 13 | echo "" | awk '{print $1}' 14 | endtime=$(date +%s) 15 | duration=$(( $endtime - $starttime )) 16 | echo "-------Total time is $(($duration / 60)) minutes $(($duration % 60)) seconds." 17 | echo "" | awk '{print $1}' 18 | echo "-------Created by Yongkang" 19 | echo "-------Email me if any suggestions or issues he@yongkang.cloud" 20 | -------------------------------------------------------------------------------- /k10status.sh: -------------------------------------------------------------------------------- 1 | #This is the script for you to verify the backup and export status of the k10- namespace 2 | #Run ./k10status, shortly, the backup and and export status will be reported. 3 | 4 | my_namespace=$(kubectl get ns | grep yong- | awk '{print $1}') 5 | kubectl get backupactions.actions.kio.kasten.io -n $my_namespace | grep -v NAME | head -1 | awk '{print $1}' > backupname 6 | echo "The backup job status is $(kubectl get backupactions.actions.kio.kasten.io -n $my_namespace $(cat backupname) -ojsonpath="{.status.state}{'\n'}")" 7 | kubectl get exportactions.actions.kio.kasten.io -n $my_namespace | grep -v NAME | head -1 | awk '{print $1}' > exportname 8 | echo "The export job status is $(kubectl get exportactions.actions.kio.kasten.io -n $my_namespace $(cat exportname) -ojsonpath="{.status.state}{'\n'}")" 9 | echo '' | awk '{print $1}' 10 | -------------------------------------------------------------------------------- /migration-oci-location.sh: -------------------------------------------------------------------------------- 1 | echo '-------Creating a S3 profile secret' 2 | . ./setenv.sh 3 | 4 | if [ ! -f ociaccess ]; then 5 | echo -n "Enter your Object Storage Access Key ID and press [ENTER]: " 6 | read AWS_ACCESS_KEY_ID 7 | echo "" | awk '{print $1}' 8 | echo $AWS_ACCESS_KEY_ID > ociaccess 9 | echo -n "Enter your Object Storage Secret Access Key and press [ENTER]: " 10 | read AWS_SECRET_ACCESS_KEY 11 | echo $AWS_SECRET_ACCESS_KEY >> ociaccess 12 | fi 13 | 14 | export OCI_MY_REGION=ap-mumbai-1 15 | export OCI_MY_OBJECT_STORAGE_PROFILE=oci-migration 16 | export AWS_ACCESS_KEY_ID=$(cat ociaccess | head -1) 17 | export AWS_SECRET_ACCESS_KEY=$(cat ociaccess | tail -1) 18 | echo oci-k10migration4yong1 | awk '{print tolower($0)}' > k10_migration_bucketname 19 | 20 | #echo '-------Creating a S3 profile secret' 21 | kubectl create secret generic k10-oci-s3-secret \ 22 | --namespace kasten-io \ 23 | --type secrets.kanister.io/aws \ 24 | --from-literal=aws_access_key_id=$AWS_ACCESS_KEY_ID \ 25 | --from-literal=aws_secret_access_key=$AWS_SECRET_ACCESS_KEY 26 | 27 | echo '-------Creating an OCI OS S3 profile' 28 | cat < awsaccess 8 | echo -n "Enter your AWS Secret Access Key and press [ENTER]: " 9 | read AWS_SECRET_ACCESS_KEY 10 | echo $AWS_SECRET_ACCESS_KEY >> awsaccess 11 | fi 12 | 13 | export MY_REGION=us-east-2 14 | export MY_OBJECT_STORAGE_PROFILE=myaws3-migration 15 | export AWS_ACCESS_KEY_ID=$(cat awsaccess | head -1) 16 | export AWS_SECRET_ACCESS_KEY=$(cat awsaccess | tail -1) 17 | LAST4=$(echo -n $(cat awsaccess) | tail -c4) 18 | echo k10migration4yong1-$LAST4$(date +%m%d) | awk '{print tolower($0)}' > k10_migration_bucketname 19 | # aws s3 mb s3://$(cat k10_migration_bucketname) --region $MY_REGION 20 | 21 | export MY_OBJECT_STORAGE_PROFILE=myaws3-migration 22 | 23 | echo '-------Creating a S3 profile secret' 24 | kubectl create secret generic k10-s3-secret \ 25 | --namespace kasten-io \ 26 | --type secrets.kanister.io/aws \ 27 | --from-literal=aws_access_key_id=$AWS_ACCESS_KEY_ID \ 28 | --from-literal=aws_secret_access_key=$AWS_SECRET_ACCESS_KEY 29 | 30 | echo '-------Creating a S3 profile' 31 | cat <