├── .gitignore ├── 01_containerization ├── Dockerfile ├── README.md ├── build.sh ├── push.sh └── run.sh ├── 02_k8s_setup ├── README.md ├── cluster_spec.yaml ├── dashboard.yaml ├── install_dashboard.sh └── setup_kind.sh ├── 03_porting_to_k8s ├── README.md ├── deploy.sh ├── myapp_deployment.yaml ├── myapp_service.yaml └── myclient_deployment.yaml ├── 04_ray_on_k8s ├── README.md ├── example-ray-app.yaml └── ray-cluster.yaml ├── 05_eks_spot_autoscaling ├── README.md ├── autoscaler-spot-priority.yaml ├── eks-spot.yaml └── launch_eks.sh ├── Dockerfile ├── LICENSE ├── README.md ├── app ├── requirements.txt └── webserver.py ├── build_tutorial_docker.sh ├── motd └── run_tutorial.sh /.gitignore: -------------------------------------------------------------------------------- 1 | 01_containerization/app/ 2 | -------------------------------------------------------------------------------- /01_containerization/Dockerfile: -------------------------------------------------------------------------------- 1 | # Inherit from miniconda base image 2 | FROM continuumio/miniconda3:latest 3 | 4 | # RUN runs a command during your build process 5 | RUN echo "Welcome to your first container build!" 6 | 7 | # COPY copies a file in the local build context to the container image 8 | COPY app /app 9 | 10 | # Installing some dependencies.. 11 | RUN pip install -r /app/requirements.txt && \ 12 | apt-get update && apt-get install -y curl && \ 13 | rm -rf /var/lib/apt/lists/* && \ 14 | echo "Setup done!" 15 | 16 | # CMD specifies the default command to run when the container is launched. \ 17 | # This can be overridden by the docker run command 18 | CMD ["python", "/app/webserver.py", "8000"] 19 | -------------------------------------------------------------------------------- /01_containerization/README.md: -------------------------------------------------------------------------------- 1 | # 01 - Containerizing the App 2 | 3 | In this chapter, we will containerize the application by wrapping it in a docker container. 4 | 1. We need to setup the build context. Copy the application code to this folder. 5 | ```console 6 | cp -r ../app . 7 | ``` 8 | 2. Have a close look at `Dockerfile`. 9 | 3. Let's build the image! 10 | ```console 11 | docker build -t romilb/myapp:latest . 12 | # Change romilb to your dockerhub username if you want to push your image 13 | ``` 14 | 4. Run the container! Don't forget to forward port 8000. 15 | ```console 16 | docker run -it --rm -p 8000:8000 romilb/myapp:latest 17 | # Open http://localhost:8000 in your browser 18 | ``` 19 | 5. Push the image to your dockerhub account. 20 | ```console 21 | docker push romilb/myapp:latest 22 | # Change romilb with your username 23 | ``` 24 | 25 | That's it! Now you can access this image from anywhere. 26 | -------------------------------------------------------------------------------- /01_containerization/build.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | 4 | # Copy the app into the build context for Docker 5 | cp -r ../app . 6 | 7 | # Build the Docker image. -t specifies the tag to use. 8 | # Change romilb to your dockerhub username 9 | docker build -t romilb/myapp:latest . 10 | -------------------------------------------------------------------------------- /01_containerization/push.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Change romilb to your dockerhub username 4 | docker push romilb/myapp:latest 5 | -------------------------------------------------------------------------------- /01_containerization/run.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Runs your docker container with port 8000 published 3 | 4 | # Change romilb to your dockerhub username 5 | docker run -it --rm -p 8000:8000 romilb/myapp:latest 6 | -------------------------------------------------------------------------------- /02_k8s_setup/README.md: -------------------------------------------------------------------------------- 1 | # 02 - Setting up Kubernetes 2 | 3 | We will use Kind (Kubernetes in Docker) to set up a Kubernetes cluster. 4 | 5 | 1. Use kind to create a cluster 6 | ``` 7 | kind create cluster 8 | ``` 9 | The output should look like 10 | ```bash 11 | (base) root@7f251c104577:/kube-tutorial/02_k8s_setup# kind create cluster 12 | Creating cluster "kind" ... 13 | ✓ Ensuring node image (kindest/node:v1.23.4) 🖼 14 | ✓ Preparing nodes 📦 15 | ✓ Writing configuration 📜 16 | ✓ Starting control-plane 🕹️ 17 | ✓ Installing CNI 🔌 18 | ✓ Installing StorageClass 💾 19 | Set kubectl context to "kind-kind" 20 | ``` 21 | 22 | 2. Kind automatically configures your kubectl to use the newly created kubernetes cluster. We can check if our Kubernetes cluster by fetching the nodes in the cluster: 23 | ```console 24 | (base) root@docker-desktop:/kube-tutorial/02_k8s_setup# kubectl get nodes 25 | NAME STATUS ROLES AGE VERSION 26 | kind-control-plane Ready control-plane,master 26s v1.23.4 27 | ``` 28 | 29 | 3. All objects (Pods, Jobs, Deployments, Auth Permissions, Secrets etc.) in Kubernetes are created using YAMLs. We will now deploy an example YAML which deploys the Kuberenetes Dashboard app. The dashboard is a useful webapp that gives us a GUI to explore our cluster. 30 | ```console 31 | (base) root@docker-desktop:/kube-tutorial/02_k8s_setup# kubectl apply -f dashboard.yaml 32 | namespace/kubernetes-dashboard created 33 | serviceaccount/kubernetes-dashboard created 34 | service/kubernetes-dashboard created 35 | secret/kubernetes-dashboard-certs created 36 | secret/kubernetes-dashboard-csrf created 37 | secret/kubernetes-dashboard-key-holder created 38 | configmap/kubernetes-dashboard-settings created 39 | role.rbac.authorization.k8s.io/kubernetes-dashboard created 40 | clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created 41 | rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created 42 | clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created 43 | deployment.apps/kubernetes-dashboard created 44 | service/dashboard-metrics-scraper created 45 | deployment.apps/dashboard-metrics-scraper created 46 | ``` 47 | 48 | 4. To access the dashboard, run `kubectl proxy`, and then open 49 | ```console 50 | http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/deployment?namespace=_all 51 | ``` 52 | 53 | Press `skip` on the webpage to login. 54 | 55 | ![KubernetesDashboard](https://i.imgur.com/Ta766ZR.png) 56 | -------------------------------------------------------------------------------- /02_k8s_setup/cluster_spec.yaml: -------------------------------------------------------------------------------- 1 | kind: Cluster 2 | apiVersion: kind.x-k8s.io/v1alpha4 3 | networking: 4 | # By default the API server listens on a random open port. 5 | # Choosing a specific port so we can forward it from the host machine when creating the tutorial container. 6 | apiServerPort: 6443 7 | -------------------------------------------------------------------------------- /02_k8s_setup/dashboard.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2017 The Kubernetes Authors. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | apiVersion: v1 16 | kind: Namespace 17 | metadata: 18 | name: kubernetes-dashboard 19 | 20 | --- 21 | 22 | apiVersion: v1 23 | kind: ServiceAccount 24 | metadata: 25 | labels: 26 | k8s-app: kubernetes-dashboard 27 | name: kubernetes-dashboard 28 | namespace: kubernetes-dashboard 29 | 30 | --- 31 | 32 | kind: Service 33 | apiVersion: v1 34 | metadata: 35 | labels: 36 | k8s-app: kubernetes-dashboard 37 | name: kubernetes-dashboard 38 | namespace: kubernetes-dashboard 39 | spec: 40 | ports: 41 | - port: 443 42 | targetPort: 8443 43 | selector: 44 | k8s-app: kubernetes-dashboard 45 | 46 | --- 47 | 48 | apiVersion: v1 49 | kind: Secret 50 | metadata: 51 | labels: 52 | k8s-app: kubernetes-dashboard 53 | name: kubernetes-dashboard-certs 54 | namespace: kubernetes-dashboard 55 | type: Opaque 56 | 57 | --- 58 | 59 | apiVersion: v1 60 | kind: Secret 61 | metadata: 62 | labels: 63 | k8s-app: kubernetes-dashboard 64 | name: kubernetes-dashboard-csrf 65 | namespace: kubernetes-dashboard 66 | type: Opaque 67 | data: 68 | csrf: "" 69 | 70 | --- 71 | 72 | apiVersion: v1 73 | kind: Secret 74 | metadata: 75 | labels: 76 | k8s-app: kubernetes-dashboard 77 | name: kubernetes-dashboard-key-holder 78 | namespace: kubernetes-dashboard 79 | type: Opaque 80 | 81 | --- 82 | 83 | kind: ConfigMap 84 | apiVersion: v1 85 | metadata: 86 | labels: 87 | k8s-app: kubernetes-dashboard 88 | name: kubernetes-dashboard-settings 89 | namespace: kubernetes-dashboard 90 | 91 | --- 92 | 93 | kind: Role 94 | apiVersion: rbac.authorization.k8s.io/v1 95 | metadata: 96 | labels: 97 | k8s-app: kubernetes-dashboard 98 | name: kubernetes-dashboard 99 | namespace: kubernetes-dashboard 100 | rules: 101 | # Allow Dashboard to get, update and delete Dashboard exclusive secrets. 102 | - apiGroups: [""] 103 | resources: ["secrets"] 104 | resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] 105 | verbs: ["get", "update", "delete"] 106 | # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. 107 | - apiGroups: [""] 108 | resources: ["configmaps"] 109 | resourceNames: ["kubernetes-dashboard-settings"] 110 | verbs: ["get", "update"] 111 | # Allow Dashboard to get metrics. 112 | - apiGroups: [""] 113 | resources: ["services"] 114 | resourceNames: ["heapster", "dashboard-metrics-scraper"] 115 | verbs: ["proxy"] 116 | - apiGroups: [""] 117 | resources: ["services/proxy"] 118 | resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"] 119 | verbs: ["get"] 120 | 121 | --- 122 | 123 | kind: ClusterRole 124 | apiVersion: rbac.authorization.k8s.io/v1 125 | metadata: 126 | labels: 127 | k8s-app: kubernetes-dashboard 128 | name: kubernetes-dashboard 129 | rules: 130 | # Allow Metrics Scraper to get metrics from the Metrics server 131 | - apiGroups: ["metrics.k8s.io"] 132 | resources: ["pods", "nodes"] 133 | verbs: ["get", "list", "watch"] 134 | 135 | --- 136 | 137 | apiVersion: rbac.authorization.k8s.io/v1 138 | kind: RoleBinding 139 | metadata: 140 | labels: 141 | k8s-app: kubernetes-dashboard 142 | name: kubernetes-dashboard 143 | namespace: kubernetes-dashboard 144 | roleRef: 145 | apiGroup: rbac.authorization.k8s.io 146 | kind: Role 147 | name: kubernetes-dashboard 148 | subjects: 149 | - kind: ServiceAccount 150 | name: kubernetes-dashboard 151 | namespace: kubernetes-dashboard 152 | 153 | --- 154 | 155 | apiVersion: rbac.authorization.k8s.io/v1 156 | kind: ClusterRoleBinding 157 | metadata: 158 | name: kubernetes-dashboard 159 | labels: 160 | k8s-app: kubernetes-dashboard 161 | roleRef: 162 | apiGroup: rbac.authorization.k8s.io 163 | kind: ClusterRole 164 | name: cluster-admin 165 | subjects: 166 | - kind: ServiceAccount 167 | name: kubernetes-dashboard 168 | namespace: kubernetes-dashboard 169 | 170 | --- 171 | 172 | kind: Deployment 173 | apiVersion: apps/v1 174 | metadata: 175 | labels: 176 | k8s-app: kubernetes-dashboard 177 | name: kubernetes-dashboard 178 | namespace: kubernetes-dashboard 179 | spec: 180 | replicas: 1 181 | revisionHistoryLimit: 10 182 | selector: 183 | matchLabels: 184 | k8s-app: kubernetes-dashboard 185 | template: 186 | metadata: 187 | labels: 188 | k8s-app: kubernetes-dashboard 189 | spec: 190 | containers: 191 | - name: kubernetes-dashboard 192 | image: kubernetesui/dashboard:v2.3.1 193 | imagePullPolicy: Always 194 | ports: 195 | - containerPort: 8443 196 | protocol: TCP 197 | args: 198 | - --enable-skip-login 199 | - --disable-settings-authorizer 200 | - --auto-generate-certificates 201 | - --namespace=kubernetes-dashboard 202 | # Uncomment the following line to manually specify Kubernetes API server Host 203 | # If not specified, Dashboard will attempt to auto discover the API server and connect 204 | # to it. Uncomment only if the default does not work. 205 | # - --apiserver-host=http://my-address:port 206 | volumeMounts: 207 | - name: kubernetes-dashboard-certs 208 | mountPath: /certs 209 | # Create on-disk volume to store exec logs 210 | - mountPath: /tmp 211 | name: tmp-volume 212 | livenessProbe: 213 | httpGet: 214 | scheme: HTTPS 215 | path: / 216 | port: 8443 217 | initialDelaySeconds: 30 218 | timeoutSeconds: 30 219 | securityContext: 220 | allowPrivilegeEscalation: false 221 | readOnlyRootFilesystem: true 222 | runAsUser: 1001 223 | runAsGroup: 2001 224 | volumes: 225 | - name: kubernetes-dashboard-certs 226 | secret: 227 | secretName: kubernetes-dashboard-certs 228 | - name: tmp-volume 229 | emptyDir: {} 230 | serviceAccountName: kubernetes-dashboard 231 | nodeSelector: 232 | "kubernetes.io/os": linux 233 | # Comment the following tolerations if Dashboard must not be deployed on master 234 | tolerations: 235 | - key: node-role.kubernetes.io/master 236 | effect: NoSchedule 237 | 238 | --- 239 | 240 | kind: Service 241 | apiVersion: v1 242 | metadata: 243 | labels: 244 | k8s-app: dashboard-metrics-scraper 245 | name: dashboard-metrics-scraper 246 | namespace: kubernetes-dashboard 247 | spec: 248 | ports: 249 | - port: 8000 250 | targetPort: 8000 251 | selector: 252 | k8s-app: dashboard-metrics-scraper 253 | 254 | --- 255 | 256 | kind: Deployment 257 | apiVersion: apps/v1 258 | metadata: 259 | labels: 260 | k8s-app: dashboard-metrics-scraper 261 | name: dashboard-metrics-scraper 262 | namespace: kubernetes-dashboard 263 | spec: 264 | replicas: 1 265 | revisionHistoryLimit: 10 266 | selector: 267 | matchLabels: 268 | k8s-app: dashboard-metrics-scraper 269 | template: 270 | metadata: 271 | labels: 272 | k8s-app: dashboard-metrics-scraper 273 | annotations: 274 | seccomp.security.alpha.kubernetes.io/pod: 'runtime/default' 275 | spec: 276 | containers: 277 | - name: dashboard-metrics-scraper 278 | image: kubernetesui/metrics-scraper:v1.0.6 279 | ports: 280 | - containerPort: 8000 281 | protocol: TCP 282 | livenessProbe: 283 | httpGet: 284 | scheme: HTTP 285 | path: / 286 | port: 8000 287 | initialDelaySeconds: 30 288 | timeoutSeconds: 30 289 | volumeMounts: 290 | - mountPath: /tmp 291 | name: tmp-volume 292 | securityContext: 293 | allowPrivilegeEscalation: false 294 | readOnlyRootFilesystem: true 295 | runAsUser: 1001 296 | runAsGroup: 2001 297 | serviceAccountName: kubernetes-dashboard 298 | nodeSelector: 299 | "kubernetes.io/os": linux 300 | # Comment the following tolerations if Dashboard must not be deployed on master 301 | tolerations: 302 | - key: node-role.kubernetes.io/master 303 | effect: NoSchedule 304 | volumes: 305 | - name: tmp-volume 306 | emptyDir: {} -------------------------------------------------------------------------------- /02_k8s_setup/install_dashboard.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | kubectl apply -f dashboard.yaml 4 | echo "Dashboard installed, please run 'kubectl proxy' and visit http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/node?namespace=default" 5 | kubectl proxy 6 | -------------------------------------------------------------------------------- /02_k8s_setup/setup_kind.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | kind create cluster 4 | -------------------------------------------------------------------------------- /03_porting_to_k8s/README.md: -------------------------------------------------------------------------------- 1 | # 03 - Porting to Kubernetes 2 | 3 | Now that our app is containerized and our local kubernetes cluster is running, we can 4 | start porting our app to kubernetes. There are two keys things we must do: 5 | 6 | 1. Define the application (Container image, number of replicas, etc.) 7 | 2. Define the networking for the application (how to connect to the application, ports etc.) 8 | 9 | Let's start with the application. We will define it as a Kubernetes `Deployment`. A 10 | deployment is a Kubernetes object that describes a group of pods that are created 11 | together. `Deployment` is unique in that it always tries to achieve the declared 12 | state of the pods. For intsance, if you say `replicas: 2`, the deployment will 13 | try to always have 2 pods running. If one of the pods dies, it will automatically 14 | create a new one. 15 | 16 | The `Deployment` is defined in the `myapp_deployment.yaml` file. Let's deploy it: 17 | ```console 18 | kubectl apply -f myapp_deployment.yaml 19 | ``` 20 | 21 | That's it, your deployment is running. 22 | 23 | Now since we can't access it from outside the cluster (of course, you can set it up to be 24 | accessible from outside but that's out of scope here), we will create a client that 25 | we will use to get inside the cluster and access the deployment. 26 | 27 | Let's launch a client that sleeps and we will get a terminal inside it to mess around. 28 | 29 | ```console 30 | kubectl apply -f myclient_deployment.yaml 31 | kubectl get pods # Copy the name of the myclient pod from here 32 | kubectl exec -it /bin/bash 33 | ``` 34 | 35 | This will drop you inside a terminal. Now to access the myapp pod, we need its IP address. 36 | Let's get it: 37 | 38 | ```console 39 | kubectl describe pod 40 | ``` 41 | 42 | 43 | -------------------------------------------------------------------------------- /03_porting_to_k8s/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | kubectl apply -f myapp_deployment.yaml 3 | kubectl apply -f myclient_deployment.yaml 4 | kubectl apply -f myapp_services.yaml 5 | -------------------------------------------------------------------------------- /03_porting_to_k8s/myapp_deployment.yaml: -------------------------------------------------------------------------------- 1 | # Example deployment YAML 2 | # First we define the Kubernetes API and the object type 3 | apiVersion: apps/v1 4 | kind: Deployment 5 | 6 | # Define the metadata for the deployment object, optional 7 | metadata: 8 | name: myapp-deployment 9 | labels: 10 | rise_deployment: webserver 11 | is_k8s_easy: maybe 12 | 13 | # Define the spec of the deployment - this is the meat 14 | spec: 15 | # Number of replica pods that the deployment will try to maintain 16 | replicas: 1 17 | 18 | # The selector condition that defines the pods that will be 19 | # managed by the deployment. 20 | selector: 21 | matchLabels: 22 | app: myapp 23 | 24 | # Template for the pods that will be created as a part of this deployment 25 | template: 26 | # Metadata for each pod 27 | metadata: 28 | labels: 29 | app: myapp # This is the same as the selector above 30 | 31 | # Specification of the container(s) that will be run in each pod 32 | spec: 33 | containers: 34 | - name: myapp 35 | # The image that will be used - make sure it's a public image 36 | # (or configure your k8s to work with your image registry) 37 | image: romilb/myapp:latest 38 | # Optional - override the command that will be executed in the container 39 | # command: ["python", "/app/webserver.py", "8000"] 40 | -------------------------------------------------------------------------------- /03_porting_to_k8s/myapp_service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: myapp-service 5 | spec: 6 | selector: 7 | app: myapp 8 | ports: 9 | - protocol: TCP 10 | # Port on the service 11 | port: 10000 12 | # Port on the target pod(s) 13 | targetPort: 8000 14 | -------------------------------------------------------------------------------- /03_porting_to_k8s/myclient_deployment.yaml: -------------------------------------------------------------------------------- 1 | # Example deployment YAML 2 | # First we define the Kubernetes API and the object type 3 | apiVersion: apps/v1 4 | kind: Deployment 5 | 6 | # Define the metadata for the deployment object, optional 7 | metadata: 8 | name: myclient-deployment 9 | labels: 10 | rise_deployment: client 11 | is_k8s_easy: maybe 12 | 13 | spec: 14 | replicas: 1 15 | selector: 16 | matchLabels: 17 | app: myclient 18 | template: 19 | metadata: 20 | labels: 21 | app: myclient 22 | spec: 23 | containers: 24 | - name: myapp 25 | image: romilb/myapp:latest 26 | # Override the command that will be executed in the container 27 | command: ["sleep", "365d"] 28 | -------------------------------------------------------------------------------- /04_ray_on_k8s/README.md: -------------------------------------------------------------------------------- 1 | # 04 - Running Ray on Kubernetes 2 | 3 | Sorry no docs here yet, added this last minute. 4 | -------------------------------------------------------------------------------- /04_ray_on_k8s/example-ray-app.yaml: -------------------------------------------------------------------------------- 1 | # Job to submit a Ray program from a pod outside a running Ray cluster. 2 | apiVersion: batch/v1 3 | kind: Job 4 | metadata: 5 | name: ray-test-job 6 | spec: 7 | template: 8 | spec: 9 | restartPolicy: Never 10 | containers: 11 | - name: ray 12 | image: public.ecr.aws/cilantro/cray-workloads:latest 13 | imagePullPolicy: Always 14 | command: [ "/bin/bash", "-c", "--" ] 15 | args: 16 | - "wget https://raw.githubusercontent.com/romilbhardwaj/cilantro-workloads/main/cray_workloads/k8s/playground/print_clusterstats.py && 17 | python print_clusterstats.py" 18 | resources: 19 | requests: 20 | cpu: 100m 21 | memory: 512Mi 22 | -------------------------------------------------------------------------------- /04_ray_on_k8s/ray-cluster.yaml: -------------------------------------------------------------------------------- 1 | # Ray head node service, allowing worker pods to discover the head node. 2 | apiVersion: v1 3 | kind: Service 4 | metadata: 5 | name: ray-head 6 | spec: 7 | ports: 8 | - name: client 9 | protocol: TCP 10 | port: 10001 11 | targetPort: 10001 12 | - name: dashboard 13 | protocol: TCP 14 | port: 8265 15 | targetPort: 8265 16 | - name: redis 17 | protocol: TCP 18 | port: 6379 19 | targetPort: 6379 20 | selector: 21 | component: ray-head 22 | --- 23 | # Ray external connection service 24 | apiVersion: v1 25 | kind: Service 26 | metadata: 27 | name: ray-head-external 28 | spec: 29 | type: NodePort 30 | ports: 31 | - name: client 32 | protocol: TCP 33 | port: 10001 34 | targetPort: 10001 35 | nodePort: 30001 36 | - name: dashboard 37 | protocol: TCP 38 | port: 8265 39 | targetPort: 8265 40 | nodePort: 32001 41 | - name: redis 42 | protocol: TCP 43 | port: 6379 44 | targetPort: 6379 45 | nodePort: 32000 46 | selector: 47 | component: ray-head 48 | --- 49 | apiVersion: apps/v1 50 | kind: Deployment 51 | metadata: 52 | name: ray-head 53 | spec: 54 | # Do not change this - Ray currently only supports one head node per cluster. 55 | replicas: 1 56 | selector: 57 | matchLabels: 58 | component: ray-head 59 | type: ray 60 | template: 61 | metadata: 62 | labels: 63 | component: ray-head 64 | type: ray 65 | spec: 66 | # If the head node goes down, the entire cluster (including all worker 67 | # nodes) will go down as well. If you want Kubernetes to bring up a new 68 | # head node in this case, set this to "Always," else set it to "Never." 69 | restartPolicy: Always 70 | 71 | # This volume allocates shared memory for Ray to use for its plasma 72 | # object store. If you do not provide this, Ray will fall back to 73 | # /tmp which cause slowdowns if is not a shared memory volume. 74 | volumes: 75 | - name: dshm 76 | emptyDir: 77 | medium: Memory 78 | containers: 79 | - name: ray-head 80 | image: rayproject/ray 81 | imagePullPolicy: Always 82 | command: [ "/bin/bash", "-c", "--" ] 83 | args: 84 | - "ray start --head --port=6379 --redis-shard-ports=6380,6381 --num-cpus=0 --object-manager-port=22345 --node-manager-port=22346 --dashboard-host=0.0.0.0 --block" 85 | ports: 86 | - containerPort: 6379 # Redis port 87 | - containerPort: 10001 # Used by Ray Client 88 | - containerPort: 8265 # Used by Ray Dashboard 89 | 90 | # This volume allocates shared memory for Ray to use for its plasma 91 | # object store. If you do not provide this, Ray will fall back to 92 | # /tmp which cause slowdowns if is not a shared memory volume. 93 | volumeMounts: 94 | - mountPath: /dev/shm 95 | name: dshm 96 | env: 97 | # This is used in the ray start command so that Ray can spawn the 98 | # correct number of processes. Omitting this may lead to degraded 99 | # performance. 100 | - name: MY_CPU_REQUEST 101 | valueFrom: 102 | resourceFieldRef: 103 | resource: requests.cpu 104 | resources: 105 | requests: 106 | cpu: 100m 107 | memory: 512Mi 108 | --- 109 | apiVersion: apps/v1 110 | kind: Deployment 111 | metadata: 112 | name: ray-worker 113 | spec: 114 | # Change this to scale the number of worker nodes started in the Ray cluster. 115 | replicas: 1 116 | selector: 117 | matchLabels: 118 | component: ray-worker 119 | type: ray 120 | is_workload: "true" 121 | template: 122 | metadata: 123 | labels: 124 | component: ray-worker 125 | type: ray 126 | is_workload: "true" 127 | spec: 128 | restartPolicy: Always 129 | volumes: 130 | - name: dshm 131 | emptyDir: 132 | medium: Memory 133 | containers: 134 | - name: ray-worker 135 | image: rayproject/ray 136 | imagePullPolicy: Always 137 | command: ["/bin/bash", "-c", "--"] 138 | args: 139 | - "ray start --num-cpus=$MY_CPU_REQUEST --address=$RAY_HEAD_SERVICE_HOST:$RAY_HEAD_SERVICE_PORT_REDIS --object-manager-port=22345 --node-manager-port=22346 --block" 140 | # This volume allocates shared memory for Ray to use for its plasma 141 | # object store. If you do not provide this, Ray will fall back to 142 | # /tmp which cause slowdowns if is not a shared memory volume. 143 | volumeMounts: 144 | - mountPath: /dev/shm 145 | name: dshm 146 | env: 147 | # This is used in the ray start command so that Ray can spawn the 148 | # correct number of processes. Omitting this may lead to degraded 149 | # performance. 150 | - name: MY_CPU_REQUEST 151 | valueFrom: 152 | resourceFieldRef: 153 | resource: requests.cpu 154 | resources: 155 | requests: 156 | cpu: 100m 157 | memory: 512Mi 158 | -------------------------------------------------------------------------------- /05_eks_spot_autoscaling/README.md: -------------------------------------------------------------------------------- 1 | # 05 - Autoscaling Spot clusters with EKS 2 | 3 | Your best bet for using Kubernetes in the cloud is to use hosted services, such 4 | as EKS/AKS/GKS. Not only do they simplify kubernetes deployments, they also offer 5 | cool features like autoscaling and spot instances. 6 | 7 | In this particular example, we will use EKS to create a managed autoscaling node group 8 | of spot instances. What this means is, EKS will automatically add spot instances 9 | to your cluster if any pods are pending for some time, and it will use on-demand intsances 10 | if spot instances are not available. More interestingly, if any spot instances fail, 11 | Kubernetes `Deployment` will handle recovery by relaunching pods that were on the failing 12 | instances. 13 | 14 | Complete the EKS autoscaling prequisites [here](https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html). 15 | 16 | Setting this up is super simple (you need `eksctl` installed). Simply run 17 | ``` 18 | eksctl create cluster -f eks-spot.yaml --kubeconfig kubeconfig-eks.yaml 19 | ``` 20 | 21 | If you don't specify `--kubeconfig`, it will create a kubeconfig file in your home directory. 22 | Your kubectl should be configured to use this cluster. 23 | 24 | For more details, see [eksctl spot instances documentation](https://eksctl.io/usage/spot-instances/). 25 | 26 | Now, we want to prioritize autoscaling the spot instances. We can do this by creating a 27 | config map for the k8s autoscaler [expander](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/priority/readme.md). 28 | 29 | ``` 30 | kubectl create -f autoscaler-spot-priority.yaml 31 | ``` 32 | 33 | That's it. Launch a workload, go crazy. EKS will prioritize using spot instances, 34 | and if it can't find any, it will use on-demand instances. 35 | -------------------------------------------------------------------------------- /05_eks_spot_autoscaling/autoscaler-spot-priority.yaml: -------------------------------------------------------------------------------- 1 | 2 | --- 3 | # Config map to prioritize spot instances over on-demand instances for autoscaling 4 | apiVersion: v1 5 | kind: ConfigMap 6 | metadata: 7 | name: cluster-autoscaler-priority-expander 8 | namespace: kube-system 9 | data: 10 | priorities: |- 11 | 20: 12 | - eks-spot.* 13 | 10: 14 | - eks-on-demand.* 15 | -------------------------------------------------------------------------------- /05_eks_spot_autoscaling/eks-spot.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: eksctl.io/v1alpha5 2 | kind: ClusterConfig 3 | 4 | metadata: 5 | name: hybrid-cluster 6 | region: us-west-2 7 | 8 | managedNodeGroups: 9 | - name: spot 10 | instanceTypes: ["c3.large","c4.large","c5.large","c5d.large","c5n.large","c5a.large"] 11 | spot: true 12 | 13 | # On-Demand instances 14 | - name: on-demand 15 | instanceTypes: ["c3.large", "c4.large", "c5.large"] 16 | -------------------------------------------------------------------------------- /05_eks_spot_autoscaling/launch_eks.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | eksctl create cluster -f eks-spot.yaml 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM continuumio/anaconda3:5.3.0 2 | MAINTAINER romil.bhardwaj@berkeley.edu 3 | 4 | COPY motd /etc/motd 5 | RUN echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/issue && cat /etc/motd' >> /etc/bash.bashrc 6 | 7 | # Install DIND 8 | COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/ 9 | 10 | # Install kind and kubectl 11 | RUN wget --no-check-certificate https://kind.sigs.k8s.io/dl/v0.12.0/kind-linux-amd64 && \ 12 | chmod +x ./kind-linux-amd64 && \ 13 | mv ./kind-linux-amd64 /usr/local/bin/kind 14 | 15 | RUN wget https://dl.k8s.io/release/v1.23.0/bin/linux/amd64/kubectl && \ 16 | chmod +x kubectl && \ 17 | mv ./kubectl /usr/local/bin/kubectl 18 | 19 | # Clone repo 20 | RUN git clone https://github.com/romilbhardwaj/kube-tutorial && cd kube-tutorial 21 | 22 | WORKDIR /kube-tutorial/ 23 | 24 | # To keep the container running in background 25 | CMD tail -f /dev/null 26 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Romil Bhardwaj 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Kubernetes for Grad Students 2 | Kubernetes Tutorial for the PS2 group meetings at UC Berkeley. 3 | 4 | Slides can be found [here](https://docs.google.com/presentation/d/16CEIbD7eISnzQGbtKfxGvRldutecqiGTwAi3VffOU5w/edit?usp=sharing). 5 | 6 | ![Dilbert on k8s](https://i.imgur.com/uOcM31v.jpg) 7 | 8 | ## Prerequisites 9 | **Linux users** - you only need [docker installed](https://docs.docker.com/engine/install/). You can then use the tutorial docker image: 10 | ```console 11 | # Make sure you're logged in to DockerHub 12 | docker login 13 | 14 | # Run docker image for the tutorial 15 | ./run_tutorial.sh 16 | ``` 17 | If you do not wish to use the container, see instructions below. 18 | 19 | **Mac/WSL/Linux users** - Please install: 20 | 1. Docker ([Mac](https://docs.docker.com/desktop/mac/install/) / [Windows](https://docs.docker.com/desktop/windows/install/)). After installation: 21 | ```console 22 | # Make sure you're logged in to DockerHub 23 | docker login 24 | ``` 25 | 26 | 3. [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/) - utility that we use to run the Kubernetes cluster. 27 | ``` 28 | # Linux/WSL 29 | curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.12.0/kind-linux-amd64 30 | chmod +x ./kind 31 | mv ./kind /usr/local/bin/kind 32 | ``` 33 | ``` 34 | # Mac 35 | brew install kind 36 | ``` 37 | 4. [Kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) - utility to interact with Kubernetes clusters. 38 | ``` 39 | # Linux/WSL 40 | curl -LO https://dl.k8s.io/release/v1.23.0/bin/linux/amd64/kubectl 41 | chmod +x kubectl 42 | mv ./kubectl /usr/local/bin/kubectl 43 | 44 | kubectl version --client 45 | ``` 46 | ``` 47 | # Mac 48 | brew install kubectl 49 | ``` 50 | 5. (Optional, but highly recommended) [Setup bash autocompletion for kubectl](https://kubernetes.io/docs/tasks/tools/included/optional-kubectl-configs-bash-linux/). This will save you countless copy-pastes. 51 | 52 | If you would like to build your own images, please make sure you are signed into your DockerHub account by running `docker login`. 53 | -------------------------------------------------------------------------------- /app/requirements.txt: -------------------------------------------------------------------------------- 1 | Flask==2.1.0 2 | -------------------------------------------------------------------------------- /app/webserver.py: -------------------------------------------------------------------------------- 1 | # Runs a webserver on the specified port and returns hello world 2 | 3 | import argparse 4 | import socket 5 | 6 | from flask import Flask 7 | 8 | app = Flask(__name__) 9 | hostname = socket.gethostname() 10 | 11 | @app.route('/') 12 | def index(): 13 | return f'Hello Sky Computing Lab! My hostname is {hostname}' 14 | 15 | 16 | def parse_args(): 17 | parser = argparse.ArgumentParser( 18 | description='Run a simple webserver.') 19 | parser.add_argument('port', type=int, default=8000) 20 | return parser.parse_args() 21 | 22 | 23 | def main(): 24 | args = parse_args() 25 | print(f'I am {hostname}, Running webserver on port {args.port}') 26 | app.run(host='0.0.0.0', port=args.port) 27 | 28 | 29 | if __name__ == '__main__': 30 | main() 31 | -------------------------------------------------------------------------------- /build_tutorial_docker.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | docker build -t romilb/k8stutorial . 4 | -------------------------------------------------------------------------------- /motd: -------------------------------------------------------------------------------- 1 | =========================================================================== 2 | ______ _____ _____ _____ _ __ _____ _____ _____ _ 3 | | ___ \_ _/ ___| ___| | | / /| _ |/ ___| |_ _| | | 4 | | |_/ / | | \ `--.| |__ ______| |/ / \ V / \ `--. ______| |_ _| |_ 5 | | / | | `--. \ __|______| \ / _ \ `--. \______| | | | | __| 6 | | |\ \ _| |_/\__/ / |___ | |\ \| |_| |/\__/ / | | |_| | |_ 7 | \_| \_|\___/\____/\____/ \_| \_/\_____/\____/ \_/\__,_|\__| 8 | =========================================================================== 9 | 10 | Welcome to the RISE Kubernetes Tutorial Docker image! To get started, 11 | 12 | 1. Run 'docker login' and authenticate with your dockerhub credentials. 13 | 2. 'cd /kube-tutorial/01_containerization' 14 | 3. You may need to modify the build scripts to point to your docker registry instead of romilb. 15 | -------------------------------------------------------------------------------- /run_tutorial.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Runs a new docker container or connects to an existing one. 3 | 4 | if [ ! "$(docker ps -q -f name=k8stutorial)" ]; then 5 | # Container exists, check if its exited 6 | if [ "$(docker ps -aq -f status=exited -f name=k8stutorial)" ]; then 7 | # cleanup 8 | docker rm k8stutorial 9 | fi 10 | # run the container 11 | docker run -d \ 12 | --net host \ 13 | -v /var/run/docker.sock:/var/run/docker.sock \ 14 | --name k8stutorial \ 15 | romilb/k8stutorial:latest 16 | fi 17 | 18 | docker exec -it k8stutorial /bin/bash 19 | --------------------------------------------------------------------------------