75 | 76 | ### kubelet 77 | ```yaml 78 | vim /var/lib/kubelet/config.yaml 79 | authentication: 80 | anonymous: 81 | enabled: false 82 | webhook: 83 | enabled: true 84 | authorization: 85 | mode: Webhook 86 | protectKernelDefaults: true 87 | 88 | systemctl restart kubelet.service 89 | systemctl status kubelet.service 90 | ``` 91 | 92 | ### kube-apiserver 93 | ```bash 94 | vim /etc/kubernetes/manifests/kube-apiserver.yaml 95 | - --authorization-mode=Node,RBAC 96 | - --profiling=false 97 | ``` 98 | 99 | ### etcd 100 | ```bash 101 | mv /etc/kubernetes/manifests/etcd.yaml /etc/kubernetes/ 102 | vim /etc/kubernetes/etcd.yaml 103 | - --client-cert-auth=true 104 | ``` 105 | 106 |
107 |113 | 114 | ### admission-control.conf 115 | ```yaml 116 | vim /etc/kubernetes/admission-control/admission-control.conf 117 | apiVersion: apiserver.config.k8s.io/v1 118 | kind: AdmissionConfiguration 119 | plugins: 120 | - name: ImagePolicyWebhook 121 | path: imagepolicy.conf 122 | ``` 123 | 124 | ### imagepolicy.conf | imagepolicy.json 125 | ```bash 126 | vim /etc/kubernetes/admission-control/imagepolicy.conf 127 | { 128 | "imagePolicy": { 129 | "kubeConfigFile": "/etc/kubernetes/admission-control/imagepolicy_backend.kubeconfig", 130 | "allowTTL": 50, 131 | "denyTTL": 50, 132 | "retryBackoff": 500, 133 | "defaultAllow": false 134 | } 135 | } 136 | Note: Change true to false and Take note of kubeConfigFile 137 | ``` 138 | 139 | ### imagepolicy_backend.kubeconfig 140 | ```yaml 141 | vim /etc/kubernetes/admission-control/imagepolicy_backend.kubeconfig 142 | apiVersion: v1 143 | kind: Config 144 | clusters: 145 | - name: trivy-k8s-webhook 146 | cluster: 147 | certificate-authority: /etc/kubernetes/admission-control/imagepolicywebhook-ca.crt 148 | server: https://acg.trivy.k8s.webhook:8090/scan 149 | contexts: 150 | - name: trivy-k8s-webhook 151 | context: 152 | cluster: trivy-k8s-webhook 153 | user: api-server 154 | current-context: trivy-k8s-webhook 155 | preferences: {} 156 | users: 157 | - name: api-server 158 | user: 159 | client-certificate: /etc/kubernetes/admission-control/api-server-client.crt 160 | client-key: /etc/kubernetes/admission-control/api-server-client.key 161 | # Note: Edit server value 162 | ``` 163 | 164 | ### kube-apiserver 165 | ```bash 166 | vim /etc/kubernetes/manifests/kube-apiserver.yaml 167 | - --admission-control-config-file=/etc/kubernetes/admission-control/admission-control.conf 168 | - --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook 169 | ``` 170 | 171 |
172 |178 | 179 | ### audit-policy.yaml 180 | ```yaml 181 | apiVersion: audit.k8s.io/v1 182 | kind: Policy 183 | omitStages: 184 | - "RequestReceived" 185 | rules: 186 | - level: None 187 | resources: 188 | - group: "" 189 | resources: ["pods/log", "pods/status"] 190 | - level: RequestResponse 191 | resources: 192 | - group: "" 193 | resources: ["configmaps"] 194 | - level: Request 195 | resources: 196 | - group: "" 197 | resources: ["services", "pods"] 198 | namespaces: ["web"] 199 | - level: Metadata 200 | resources: 201 | - group: "" 202 | resources: ["secrets"] 203 | - level: Metadata 204 | ``` 205 | 206 | ### kube-apiserver.yaml 207 | ```bash 208 | vim /etc/kubernetes/manifests/kube-apiserver.yaml 209 | - --audit-policy-file=/etc/kubernetes/audit-policy.yaml 210 | - --audit-log-path=/var/log/kubernetes/audit.log 211 | - --audit-log-maxage=10 212 | - --audit-log-maxbackup=1 213 | ``` 214 | 215 |
216 |223 | 224 | ```yaml 225 | vim nopriv-psp.yml 226 | apiVersion: policy/v1beta1 227 | kind: PodSecurityPolicy 228 | metadata: 229 | name: nopriv-psp 230 | spec: 231 | privileged: false 232 | runAsUser: 233 | rule: "RunAsAny" 234 | fsGroup: 235 | rule: "RunAsAny" 236 | seLinux: 237 | rule: "RunAsAny" 238 | supplementalGroups: 239 | rule: "RunAsAny" 240 | k apply -f nopriv-psp.yml 241 | ``` 242 | 243 | ```yaml 244 | /home/cloud_user/use-nopriv-psp.yml 245 | apiVersion: rbac.authorization.k8s.io/v1 246 | kind: ClusterRole 247 | metadata: 248 | name: use-nopriv-psp 249 | rules: 250 | - apiGroups: ['policy'] 251 | resources: ['podsecuritypolicies'] 252 | verbs: ['use'] 253 | resourceNames: 254 | - nopriv-psp 255 | k apply -f /home/cloud_user/use-nopriv-psp.yml 256 | ``` 257 | 258 | ```yaml 259 | /home/cloud_user/hoth-sa-use-nopriv-psp.yml 260 | apiVersion: rbac.authorization.k8s.io/v1 261 | kind: ClusterRoleBinding 262 | metadata: 263 | name: hoth-sa-use-nopriv-psp 264 | roleRef: 265 | kind: ClusterRole 266 | name: use-nopriv-psp 267 | apiGroup: rbac.authorization.k8s.io 268 | subjects: 269 | - kind: ServiceAccount 270 | name: hoth-sa 271 | namespace: hoth 272 | k apply -f /home/cloud_user/hoth-sa-use-nopriv-psp.yml 273 | ``` 274 | 275 |
276 |283 | 284 | ### RuntimeClass 285 | ```yaml 286 | vim /home/cloud_user/sandbox.yml 287 | apiVersion: node.k8s.io/v1 288 | kind: RuntimeClass 289 | metadata: 290 | name: sandbox 291 | handler: runsc 292 | k apply -f /home/cloud_user/sandbox.yml 293 | ``` 294 | 295 | ### Edit deployment 296 | ```bash 297 | k -n sunnydale edit deployments.apps buffy # runtimeClassName: sandbox 298 | k -n sunnydale edit deployments.apps giles 299 | k -n sunnydale edit deployments.apps spike 300 | ``` 301 | 302 | ### Verification 303 | ```bash 304 | k -n sunnydale exec buffy-7bdbdfc554-ls5q5 -- dmesg 305 | 306 | [ 0.000000] Starting gVisor... 307 | [ 0.453650] Forking spaghetti code... 308 | [ 0.939306] Conjuring /dev/null black hole... 309 | [ 1.162591] Searching for socket adapter... 310 | [ 1.450979] Generating random numbers by fair dice roll... 311 | [ 1.907884] Waiting for children... 312 | [ 2.063679] Checking naughty and nice process list... 313 | [ 2.554570] Recruiting cron-ies... 314 | [ 3.023213] Gathering forks... 315 | [ 3.300373] Synthesizing system calls... 316 | [ 3.401099] Searching for needles in stacks... 317 | [ 3.521588] Setting up VFS2... 318 | [ 3.938928] Ready! 319 | ``` 320 | 321 |
322 |328 | 329 | - Fixing issues in Dockerfile 330 | - Fixing issues in Deployment 331 | 332 |
333 |339 | 340 | - runAsUser: 0 341 | - readOnlyRootFilesystem: false 342 | - priveledged: true 343 | 344 |
345 |352 | 353 | ```bash 354 | k -n development get pods 355 | k -n development get pods --output=custom-columns="NAME:.metadata.name,IMAGE:.spec.containers[*].image" 356 | NAME IMAGE 357 | work1 busybox:1.33.1 358 | work2 nginx:1.14.2 359 | work3 amazonlinux:2 360 | work4 amazonlinux:1 361 | work5 centos:7 362 | trivy image -s HIGH,CRITICAL busybox:1.33.1 363 | trivy image -s HIGH,CRITICAL nginx:1.14.2 #HIGH and CRITICAL 364 | trivy image -s HIGH,CRITICAL amazonlinux:2 365 | trivy image -s HIGH,CRITICAL amazonlinux:1 366 | trivy image -s HIGH,CRITICAL centos:7 #HIGH and CRITICAL 367 | ``` 368 | 369 |
370 |376 | 377 | ```bash 378 | sudo falco -M 45 -r /home/cloud_user/monitor_rules.yml > /home/cloud_user/falco_output.log 379 | ``` 380 | 381 | ```bash 382 | - /etc/falco/falco_rules.local.yaml 383 | - /etc/falco/falco_rules.yaml 384 | - /etc/falco/falco.yaml 385 | systemctl restart falco.service 386 | ``` 387 | 388 | 389 |
390 |
397 |
398 | ```bash
399 | cat k8s-deny-write
400 | #include