├── README.md
├── authentication.md
├── deploy
├── examples
    └── rbac
    │   └── pod-reader-role.yaml
└── src
    ├── addons
        ├── deploy
        ├── external-dns
        │   └── external-dns.yaml
        ├── fluentd
        │   ├── docker-image
        │   │   └── v0.12
        │   │   │   └── alpine-cloudwatch
        │   │   │       ├── Dockerfile
        │   │   │       ├── conf
        │   │   │           ├── fluent.conf
        │   │   │           ├── kubernetes.conf
        │   │   │           └── systemd.conf
        │   │   │       └── plugins
        │   │   │           └── parser_kubernetes.rb
        │   └── fluentd-kubernetes-cloudwatch
        │   │   ├── fluentd.cm.yaml
        │   │   ├── fluentd.ds.yaml
        │   │   ├── fluentd.rbac.yaml
        │   │   └── log.ns.yaml
        ├── ingress
        │   └── nginx
        │   │   ├── nginx.lb.yaml
        │   │   └── rbac.yaml
        └── kube-lego
        │   ├── kube-lego.yaml
        │   └── rbac.yaml
    ├── iam
        ├── deploy
        └── iam.yaml
    ├── kube-aws
        └── cluster.yaml
    ├── vpc
        ├── deploy
        └── vpc.yaml
    └── vpn
        ├── Docker
            ├── Dockerfile
            └── start-pritunl
        └── userdata.yaml


/README.md:
--------------------------------------------------------------------------------
 1 | # Secure Kubernets HA cluster in AWS using kube-aws
 2 | 
 3 | This repository contains an example of how to deploy a secure Kubernetes HA cluster in AWS using [kube-aws](https://github.com/kubernetes-incubator/kube-aws) automatically.
 4 | 
 5 | The fallowing setup use a base CloudFormation stack to configure Public and Private Subnets, IGW, NatGW, Route Tables, KMS and deploys automatically a VPN server in a public subnet. After the stack is created, the Kubernetes cluster is automatically deployed on top of it using `kube-aws`.
 6 | 
 7 | [![asciicast](https://asciinema.org/a/145270.png)](https://asciinema.org/a/145270)
 8 | 
 9 | **Features:**
10 | 
11 | * simple and interactive deployment
12 | * all the nodes are deployed in private subnets
13 | * 3 distinct availability zones
14 | * multi AZ masters
15 | * workers configured using node pools, similar to [GKE node pools](https://cloud.google.com/container-engine/docs/node-pools)
16 | * HA ETCD with encrypted partitions for data, automatic backups to S3 and automatic/manual recovery from failover
17 | * role based authentication using the RBAC plugin
18 | * users authentication using [OpenID Connect Identity](https://kubernetes.io/docs/admin/authentication/#openid-connect-tokens) (OIDC)
19 | * AWS IAM roles directly assigned to pods using [kube2iam](https://github.com/jtblin/kube2iam)
20 | * cluster autoscaling
21 | * VPN server automatically deployed to a public subnet
22 | 
23 | ![alt](https://www.camil.org/content/images/2017/05/kube-aws-secure.png)
24 | 
25 | 
26 | ### Deploy the Kubernetes cluster
27 | 
28 | 1. Clone this repository locally
29 | 
30 | 2. run `./deploy` and fallow the instructions
31 | 
32 | 3. Access your Kubernetes cluster. Since  all the resources are in private networks, in order to access it, you'll need a VPN placed in one of the public subnets.[Pritunl](https://docs.pritunl.com/docs/installation) is now automatically deployed to a public subnet with a Elastic IP and DNS reccord.
33 | 
34 | 
35 | *Optionally you can configure your `~/.kube/config`according to `kubeconfig` file to avoid passing the `--kubeconfig` flag on your commands.*
36 | 
37 | **Important**
38 | 
39 | *In order to expose public services using ELB or Ingress, the public subnets have to be tagged with the cluster name.*
40 | 
41 | *Ex. `KubeernetesCluser=cluster_name`*
42 | 
43 | *This is now set automatically*
44 | 
45 | 
46 | ### Add-ons
47 | 
48 | 
49 | *Note: all the addons can now be deployed automatically using addons/deploy script*
50 | #### Route53
51 | 
52 | This add-on is based on [ExternalDNS](https://github.com/kubernetes-incubator/external-dns) project which allows you to control Route53 DNS records dynamically via Kubernetes resources.
53 | 
54 | *Note: before deploying this addon, you have to create a IAM role and setup a trust relationship*
55 | 
56 | #### Nginx Ingress Controller
57 | 
58 | [Nginx ingress controller](https://github.com/kubernetes/ingress-nginx) is deployed behind a ELB configured with Proxy Protocol. This way the ingress external address will be always associated with your ELB. Also you don't have to expose your workers publicly and get better protection from your ELB.
59 | 
60 | #### kube-lego
61 | [Kube-Lego](https://github.com/jetstack/kube-lego) automatically requests certificates for Kubernetes Ingress resources from Let's Encrypt.
62 | 
63 | #### fluentd-cloudwatch
64 | This add-on is based on [fluentd-kubernetes-daemonset](https://github.com/fluent/fluentd-kubernetes-daemonset) and can forward the container logs to CloudWatchLogs.
65 | 
66 | #### Monitoring
67 | A easy to setup, in-cluster, monitoring solution using Prometheus is available [here](https://github.com/camilb/prometheus-kubernetes)
68 | 


--------------------------------------------------------------------------------
/authentication.md:
--------------------------------------------------------------------------------
 1 | ##[Work in progress]
 2 | 
 3 | #### OIDC
 4 | 
 5 | #### Kubernetes Dashboard
 6 | 
 7 | #### Dex
 8 | Dex runs natively on top of any Kubernetes cluster using Third Party Resources and can drive API server authentication through the OpenID Connect plugin.
 9 | 
10 | By default you have administrator rights using the TLS certificates. If you plan to grant restricted permissions to other users, Dex can facilitate users access using OpenID Connect Tokens.
11 | 
12 | In this example we use the [Github provider](https://github.com/coreos/dex/blob/master/Documentation/github-connector.md) to identify the users.
13 | 
14 | Please configure the`./addons/dex/elb/internal-elb.yaml` file then expose the service.
15 | 
16 |     kubectl create -f ./addons/dex/elb/internal-elb.yaml
17 | 
18 | The DNS is configured automatically by `ExternalDNS` add-on and should be available in  approximately 1 minute.
19 | 
20 | You can now use a client like dex's [example-app](https://github.com/coreos/dex/tree/master/cmd/example-app) to obtain a authentication token.
21 | 
22 | If you prefer, you can use this app as a always running service by configuring and deploying `./addons/kid/kid.yaml`
23 | 
24 |     kubectl create secret \
25 |     generic kid \
26 |     --from-literal=CLIENT_ID=your-client-id \
27 |     --from-literal=CLIENT_SECRET=your-client-secret \
28 |     -n kube-system    
29 | 
30 |     kubectl create -f ./addons/kid/kid.yaml
31 | 
32 | Please check the dex [documentation](https://github.com/coreos/dex/tree/master/Documentation) if you need more informations.
33 | 
34 | Make a quick test by granting a user permissions to list the pods in `kube-system` namespace.
35 | 
36 |     kubectl create -f `./examples/rbac/pod-reader.yaml`
37 |     kubectl create rolebinding pod-reader --role=pod-reader --user=user@example.com --namespace=kube-system
38 | 
39 | 
40 | Example of `~/.kube/config` for a user
41 | 
42 |     apiVersion: v1
43 |     clusters:
44 |     - cluster:
45 |         certificate-authority-data: ca.pem_base64_encoded
46 |         server: https://kubeapi.example.com
47 |       name: your_cluster_name
48 |     contexts:
49 |     - context:
50 |         cluster: your_cluster_name
51 |         user: user@example.com
52 |       name: your_cluster_name
53 |     current-context: your_cluster_name
54 |     kind: Config
55 |     preferences: {}
56 |     users:
57 |     - name: user@example.com
58 |       user:
59 |         auth-provider:
60 |           config:
61 |             access-token: id_token
62 |             client-id: client_id
63 |             client-secret: client_secret
64 |             extra-scopes: groups
65 |             id-token: id_token
66 |             idp-issuer-url: https://dex.example.com
67 |             refresh-token: refresh_token
68 |           name: oidc
69 | 
70 | If you already have the `~/.kube/config` set, you can use this example to configure the user authentication
71 | 
72 |     kubectl config set-credentials user@example.com \
73 |       --auth-provider=oidc \
74 |       --auth-provider-arg=idp-issuer-url=https://dex.example.com \
75 |       --auth-provider-arg=client-id=your_client_id \
76 |       --auth-provider-arg=client-secret=your_client_secret \
77 |       --auth-provider-arg=refresh-token=your_refresh_token \
78 |       --auth-provider-arg=id-token=your_id_token \
79 |       --auth-provider-arg=extra-scopes=groups
80 | 
81 | Once your `id_token` expires, `kubectl` will attempt to refresh your `id_token` using your `refresh_token` and `client_secret` storing the new values for the `refresh_token` and `id_token` in your `~/.kube/config`
82 | 


--------------------------------------------------------------------------------
/deploy:
--------------------------------------------------------------------------------
  1 | #!/bin/bash
  2 | 
  3 | RED='\033[0;31m'
  4 | GREEN='\033[0;32m'
  5 | ORANGE='\033[0;33m'
  6 | BLUE='\033[0;34m'
  7 | WHITE='\033[0;37m'
  8 | GREEN_PS3=$'\e[0;32m'
  9 | 
 10 | 
 11 | #########################################################################################
 12 | # Default CIDR
 13 | #########################################################################################
 14 | DEFAULT_VPC_CIDR=10.0.0.0/16
 15 | DEFAULT_PRIVATE_SUBNET_A_CIDR=10.0.1.0/24
 16 | DEFAULT_PRIVATE_SUBNET_B_CIDR=10.0.2.0/24
 17 | DEFAULT_PRIVATE_SUBNET_C_CIDR=10.0.3.0/24
 18 | DEFAULT_PUBLIC_SUBNET_A_CIDR=10.0.101.0/24
 19 | DEFAULT_PUBLIC_SUBNET_B_CIDR=10.0.102.0/24
 20 | DEFAULT_PUBLIC_SUBNET_C_CIDR=10.0.103.0/24
 21 | 
 22 | #########################################################################################
 23 | # check requirements
 24 | #########################################################################################
 25 | echo
 26 | echo -e "${ORANGE}Checking requirements"
 27 | tput sgr0
 28 | echo
 29 | #kube-aws
 30 | if [[ "$(which kube-aws)" != "" ]] > /dev/null 2>&1; then
 31 |   echo -e "kube-aws: ......... ${GREEN}OK"
 32 |   tput sgr0
 33 |   echo
 34 | else
 35 |   echo -e "${RED}Please install kube-aws and run this script again"
 36 |   tput sgr0
 37 |   echo
 38 |   exit
 39 | fi
 40 | 
 41 | #kubectl
 42 | if [[ "$(which kubectl)" != "" ]] > /dev/null 2>&1; then
 43 |   echo -e "kubectl: .......... ${GREEN}OK"
 44 |   tput sgr0
 45 |   echo
 46 | else
 47 |   echo -e "${RED}Please install kubectl and run this script again"
 48 |   tput sgr0
 49 |   echo
 50 |   exit
 51 | fi
 52 | 
 53 | #awscli
 54 | if [[ "$(which aws)" != "" ]] > /dev/null 2>&1; then
 55 |   echo -e "awscli: ........... ${GREEN}OK"
 56 |   tput sgr0
 57 |   echo
 58 | else
 59 |   echo -e "${RED}Please install awscli and run this script again"
 60 |   tput sgr0
 61 |   echo
 62 |   exit
 63 | fi
 64 | 
 65 | #aws config
 66 | if [[ -f ~/.aws/config ]] > /dev/null 2>&1; then
 67 |   echo -e "aws config: ....... ${GREEN}OK"
 68 |   tput sgr0
 69 |   echo
 70 | else
 71 |   echo -e "${RED}Please configure awscli using ${ORANGE}aws configure ${RED}and run this script again"
 72 |   tput sgr0
 73 |   echo
 74 |   exit
 75 | fi
 76 | 
 77 | #jq
 78 | if [[ "$(which jq)" != "" ]] > /dev/null 2>&1; then
 79 |   echo -e "jq: ............... ${GREEN}OK"
 80 |   tput sgr0
 81 |   echo
 82 | else
 83 |   echo -e "${RED}Please install jq and run this script again"
 84 |   tput sgr0
 85 |   echo
 86 |   exit
 87 | fi
 88 | 
 89 | #copy source
 90 | if [ -d "kube-aws" ] && [ -d "vpc" ] && [ -d "iam" ] && [ -d "addons" ]; then
 91 |   true
 92 | else
 93 |   mkdir vpc kube-aws iam addons
 94 | fi
 95 | 
 96 | cp src/kube-aws/cluster.yaml kube-aws/
 97 | cp src/vpc/* vpc/
 98 | cp src/iam/* iam/
 99 | cp -r src/addons/* addons
100 | 
101 | #########################################################################################
102 | # setup
103 | #########################################################################################
104 | 
105 | #aws profile
106 | PS3="${GREEN_PS3}Please select the AWS profile:"
107 | echo
108 | set -- $(cat ~/.aws/config | grep "\[" | tr -d '[]' | cut -d " " -f 2)
109 | select opt in "$@"
110 | do
111 |     case $opt in
112 |         $opt)
113 |             aws_profile="${opt}"
114 |             tput sgr0
115 |             echo "$aws_profile"
116 |             break
117 |             ;;
118 |         *) echo invalid option;;
119 |     esac
120 | done
121 | echo
122 | 
123 | #aws region
124 | PS3="${GREEN_PS3}Please select the AWS Region:"
125 | echo
126 | set -- $(aws --profile $aws_profile ec2 describe-regions | jq -r '.Regions[] | .RegionName')
127 | select opt in "$@"
128 | do
129 |     case $opt in
130 |         $opt)
131 |             aws_region="$opt"
132 |             tput sgr0
133 |             echo "$aws_region"
134 |             break
135 |             ;;
136 |         *) echo invalid option;;
137 |     esac
138 | done
139 | echo
140 | 
141 | #aws key pair
142 | PS3="${GREEN_PS3}Please select which key pair to use:"
143 | echo
144 | set -- $(aws --profile $aws_profile ec2 describe-key-pairs | jq -r '.KeyPairs[] | .KeyName')
145 | select opt in "$@"
146 | do
147 |     case $opt in
148 |         $opt)
149 |             key_name="$opt"
150 |             tput sgr0
151 |             echo "$key_name"
152 |             break
153 |             ;;
154 |         *) echo invalid option;;
155 |     esac
156 | done
157 | echo
158 | 
159 | #aws domain name
160 | PS3="${GREEN_PS3}Please select which domain name to use:"
161 | echo
162 | set -- $(aws --profile $aws_profile route53 list-hosted-zones | jq -r '.HostedZones[].Name')
163 | select opt in "$@"
164 | do
165 |     case $opt in
166 |         $opt)
167 |             domain_name="${opt%.}"
168 |             tput sgr0
169 |             echo "$domain_name"
170 |             break
171 |             ;;
172 |         *) echo invalid option;;
173 |     esac
174 | done
175 | echo
176 | 
177 | #####################################################################################################
178 | # pritunl vpn
179 | #####################################################################################################
180 | ami_id=$(curl -s https://coreos.com/dist/aws/aws-stable.json | jq '.["'$aws_region'"].hvm')
181 | 
182 | #detect external IP
183 | my_ip=$(dig +short myip.opendns.com @resolver1.opendns.com)
184 | 
185 | #curl -s https://coreos.com/dist/aws/aws-stable.json | jq '.release_info.version'
186 | 
187 | 
188 | #aws hosted zone id
189 | hosted_zone_id=$(aws --profile $aws_profile route53 list-hosted-zones | jq -r '.HostedZones[] | select(.Name="'$domain_name'") | .Id' | cut -d "/" -f3)
190 | 
191 | ##set aws region
192 | sed -i -e 's,aws_region,'"$aws_region"',g' kube-aws/cluster.yaml
193 | sed -i -e 's,aws_region,'"$aws_region"',g' vpc/vpc.yaml
194 | sed -i -e 's,aws_region,'"$aws_region"',g' vpc/deploy
195 | sed -i -e 's,aws_region,'"$aws_region"',g' iam/iam.yaml
196 | sed -i -e 's,aws_region,'"$aws_region"',g' iam/deploy
197 | sed -i -e 's,aws_region,'"$aws_region"',g' addons/fluentd/fluentd-kubernetes-cloudwatch/fluentd.ds.yaml
198 | 
199 | ##set key pair
200 | sed -i -e 's,key_name,'"$key_name"',g' kube-aws/cluster.yaml
201 | sed -i -e 's,key_name,'"$key_name"',g' vpc/vpc.yaml
202 | 
203 | ##set hosted_zone_id
204 | sed -i -e 's,hosted_zone_id,'"$hosted_zone_id"',g' kube-aws/cluster.yaml
205 | sed -i -e 's,hosted_zone_id,'"$hosted_zone_id"',g' iam/iam.yaml
206 | 
207 | ##set domain_name
208 | sed -i -e 's,domain_name,'"$domain_name"',g' kube-aws/cluster.yaml
209 | sed -i -e 's,domain_name,'"$domain_name"',g' addons/external-dns/external-dns.yaml
210 | sed -i -e 's,domain_name,'"$domain_name"',g' vpc/vpc.yaml
211 | 
212 | ##set ami_id
213 | sed -i -e 's,ami_id,'"$ami_id"',g' vpc/vpc.yaml
214 | 
215 | ##set allowed ip for vpn access SSH and Web ports
216 | sed -i -e 's,my_ip,'"$my_ip"',g' vpc/vpc.yaml
217 | 
218 | #clean sed generated files
219 | find . -name "*-e" -exec rm -rf {} \;
220 | 
221 | #####################################################################################################
222 | # VPC stack
223 | #####################################################################################################
224 | 
225 | #vpc stack name
226 | read -p "Set a name for the CloudFormation VPC stack: " stack_name
227 | echo
228 | 
229 | if [[ "$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].StackStatus')" != "CREATE_COMPLETE" ]] > /dev/null 2>&1; then
230 |   cd vpc
231 |   sed -i -e 's,stack_name,'"$stack_name"',g' deploy
232 |   echo -e "${BLUE}Use default CIDR settings?"
233 |   tput sgr0
234 |   echo
235 |   echo -e "${BLUE}VPC CIDR: ${WHITE}10.0.0.0/16"
236 |   echo
237 |   echo -e "${BLUE}Private Subnet A CIDR: ${WHITE}10.0.1.0/24"
238 |   echo -e "${BLUE}Private Subnet B CIDR: ${WHITE}10.0.2.0/24"
239 |   echo -e "${BLUE}Private Subnet C CIDR: ${WHITE}10.0.3.0/24"
240 |   echo
241 |   echo -e "${BLUE}Public Subnet A CIDR: ${WHITE}10.0.101.0/24"
242 |   echo -e "${BLUE}Public Subnet B CIDR: ${WHITE}10.0.102.0/24"
243 |   echo -e "${BLUE}Public Subnet C CIDR: ${WHITE}10.0.103.0/24"
244 |   tput sgr0
245 |   echo
246 |   read -p "Y/N [N]: " default_cidr
247 |   if [[ $default_cidr =~ ^([yY][eE][sS]|[yY])$ ]]; then
248 |     echo "Using default CIDR settings"
249 |     echo
250 | 
251 |     #Set default CIDR values
252 |     #VPC CIDR
253 |     sed -i -e 's,VPC_CIDR,'"$DEFAULT_VPC_CIDR"',g' ./vpc.yaml
254 | 
255 |     #Private Subnet A CIDR
256 |     sed -i -e 's,PRIVATE_SUBNET_A_CIDR,'"$DEFAULT_PRIVATE_SUBNET_A_CIDR"',g' ./vpc.yaml
257 | 
258 |     #Private Subnet B CIDR
259 |     sed -i -e 's,PRIVATE_SUBNET_B_CIDR,'"$DEFAULT_PRIVATE_SUBNET_B_CIDR"',g' ./vpc.yaml
260 | 
261 |     #Private Subnet C CIDR
262 |     sed -i -e 's,PRIVATE_SUBNET_C_CIDR,'"$DEFAULT_PRIVATE_SUBNET_C_CIDR"',g' ./vpc.yaml
263 | 
264 |     #Public Subnet A CIDR
265 |     sed -i -e 's,PUBLIC_SUBNET_A_CIDR,'"$DEFAULT_PUBLIC_SUBNET_A_CIDR"',g' ./vpc.yaml
266 | 
267 |     #Public Subnet B CIDR
268 |     sed -i -e 's,PUBLIC_SUBNET_B_CIDR,'"$DEFAULT_PUBLIC_SUBNET_B_CIDR"',g' ./vpc.yaml
269 | 
270 |     #Public Subnet C CIDR
271 |     sed -i -e 's,PUBLIC_SUBNET_C_CIDR,'"$DEFAULT_PUBLIC_SUBNET_C_CIDR"',g' ./vpc.yaml
272 | 
273 |     #clean sed generated files
274 |     find . -name "*-e" -exec rm -rf {} \;
275 | 
276 |   else
277 |     #VPC CIDR
278 |     echo
279 |     read -p "VPC CIDR [$DEFAULT_VPC_CIDR]:" VPC_CIDR
280 |     VPC_CIDR=${VPC_CIDR:-$DEFAULT_VPC_CIDR}
281 | 
282 |     #PRIVATE SUBNETS
283 |     #private Subnet A CIDR
284 |     echo
285 |     read -p "Private Subnet A CIDR [$DEFAULT_PRIVATE_SUBNET_A_CIDR]:" PRIVATE_SUBNET_A_CIDR
286 |     PRIVATE_SUBNET_A_CIDR=${PRIVATE_SUBNET_A_CIDR:-$DEFAULT_PRIVATE_SUBNET_A_CIDR}
287 | 
288 |     #private Subnet B CIDR
289 |     echo
290 |     read -p "Private Subnet B CIDR [$DEFAULT_PRIVATE_SUBNET_B_CIDR]:" PRIVATE_SUBNET_B_CIDR
291 |     PRIVATE_SUBNET_B_CIDR=${PRIVATE_SUBNET_B_CIDR:-$DEFAULT_PRIVATE_SUBNET_B_CIDR}
292 | 
293 |     #private Subnet C CIDR
294 |     echo
295 |     read -p "Private Subnet C CIDR [$DEFAULT_PRIVATE_SUBNET_C_CIDR]:" PRIVATE_SUBNET_C_CIDR
296 |     PRIVATE_SUBNET_C_CIDR=${PRIVATE_SUBNET_C_CIDR:-$DEFAULT_PRIVATE_SUBNET_C_CIDR}
297 | 
298 |     #PUBLIC SUBNETS
299 |     #public Subnet A CIDR
300 |     echo
301 |     read -p "Public Subnet A CIDR [$DEFAULT_PUBLIC_SUBNET_A_CIDR]:" PUBLIC_SUBNET_A_CIDR
302 |     PUBLIC_SUBNET_A_CIDR=${PUBLIC_SUBNET_A_CIDR:-$DEFAULT_PUBLIC_SUBNET_A_CIDR}
303 | 
304 |     #public Subnet B CIDR
305 |     echo
306 |     read -p "Public Subnet B CIDR [$DEFAULT_PUBLIC_SUBNET_B_CIDR]:" PUBLIC_SUBNET_B_CIDR
307 |     PUBLIC_SUBNET_B_CIDR=${PUBLIC_SUBNET_B_CIDR:-$DEFAULT_PUBLIC_SUBNET_B_CIDR}
308 | 
309 |     #public Subnet C CIDR
310 |     echo
311 |     read -p "Public Subnet C CIDR [$DEFAULT_PUBLIC_SUBNET_C_CIDR]:" PUBLIC_SUBNET_C_CIDR
312 |     PUBLIC_SUBNET_C_CIDR=${PUBLIC_SUBNET_C_CIDR:-$DEFAULT_PUBLIC_SUBNET_C_CIDR}
313 | 
314 |     #Replace default CIDR values
315 |     #VPC CIDR
316 |     sed -i -e 's,VPC_CIDR,'"$VPC_CIDR"',g' ./vpc.yaml
317 | 
318 |     #Private Subnet A CIDR
319 |     sed -i -e 's,PRIVATE_SUBNET_A_CIDR,'"$PRIVATE_SUBNET_A_CIDR"',g' ./vpc.yaml
320 | 
321 |     #Private Subnet B CIDR
322 |     sed -i -e 's,PRIVATE_SUBNET_B_CIDR,'"$PRIVATE_SUBNET_B_CIDR"',g' ./vpc.yaml
323 | 
324 |     #Private Subnet C CIDR
325 |     sed -i -e 's,PRIVATE_SUBNET_C_CIDR,'"$PRIVATE_SUBNET_C_CIDR"',g' ./vpc.yaml
326 | 
327 |     #Public Subnet A CIDR
328 |     sed -i -e 's,PUBLIC_SUBNET_A_CIDR,'"$PUBLIC_SUBNET_A_CIDR"',g' ./vpc.yaml
329 | 
330 |     #Public Subnet B CIDR
331 |     sed -i -e 's,PUBLIC_SUBNET_B_CIDR,'"$PUBLIC_SUBNET_B_CIDR"',g' ./vpc.yaml
332 | 
333 |     #Public Subnet C CIDR
334 |     sed -i -e 's,PUBLIC_SUBNET_C_CIDR,'"$PUBLIC_SUBNET_C_CIDR"',g' ./vpc.yaml
335 | 
336 |     #clean sed generated files
337 |     find . -name "*-e" -exec rm -rf {} \;
338 | 
339 |   fi
340 | 
341 | 
342 |   #Custom VPC settings
343 |   echo
344 |   echo -e "${BLUE}You can make additional changes in VPC config. Edit ${ORANGE}vpc/vpc.yaml ${BLUE}and press ENTER"
345 |   tput sgr0
346 |   read -p " [ENTER]: " vpc_yaml
347 |   echo
348 | 
349 |   echo -e "${BLUE}Creating CloudFormation VPC stack"
350 |   tput sgr0
351 |   echo
352 | 
353 |   ./deploy
354 | 
355 |   echo -e "${BLUE}Wait until the CloudFormation stack is created"
356 |   tput sgr0
357 |   echo
358 | 
359 |   while [[ "$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].StackStatus')" != "CREATE_COMPLETE" ]]
360 |     do sleep 2; printf ".";
361 |   done
362 |   echo
363 |   echo
364 | 
365 |   echo -e "${BLUE}CloudFormation VPC stack successfully  created"
366 |   tput sgr0
367 |   echo
368 |   cd ../
369 | 
370 | else
371 |   echo -e "${BLUE}Stack already exists, getting the outputs"
372 |   echo
373 | fi
374 | 
375 | echo -e "${BLUE}Getting the CloudFormation stack outputs"
376 | tput sgr0
377 | echo
378 | 
379 | #vpn DNS Reccord
380 | VPN_DNS_RECCORD=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="VpnDNSReccord") | .OutputValue')
381 | echo -e "${RED}VPN_DNS_RECCORD: ${GREEN}${VPN_DNS_RECCORD}"
382 | 
383 | #vpn IP address
384 | VPN_IP_ADDRESS=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="VpnIpAddress") | .OutputValue')
385 | echo -e "${RED}VPN_IP_ADDRESS: ${GREEN}${VPN_IP_ADDRESS}"
386 | 
387 | #kms key
388 | KMS_KEY_ARN=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="KMSKeyArn") | .OutputValue')
389 | echo -e "${RED}KMS_KEY_ARN: ${GREEN}${KMS_KEY_ARN}"
390 | 
391 | #vpc id
392 | VPC_ID=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="VpcId") | .OutputValue')
393 | echo -e "${RED}VPC_ID: ${GREEN}${VPC_ID}"
394 | 
395 | #vpc cidr
396 | VPC_CIDR=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="CidrBlock") | .OutputValue')
397 | echo -e "${RED}VPC_CIDR: ${GREEN}${VPC_CIDR}"
398 | 
399 | #route table id
400 | ROUTE_TABLE_ID=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="PrivateRouteTableId") | .OutputValue')
401 | echo -e "${RED}ROUTE_TABLE_ID: ${GREEN}${ROUTE_TABLE_ID}"
402 | 
403 | #private subnet a
404 | PRIVATE_SUBNET_A=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="PrivateSubnetAId") | .OutputValue')
405 | echo -e "${RED}PRIVATE_SUBNET_A: ${GREEN}${PRIVATE_SUBNET_A}"
406 | 
407 | #private subnet b
408 | PRIVATE_SUBNET_B=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="PrivateSubnetBId") | .OutputValue')
409 | echo -e "${RED}PRIVATE_SUBNET_B: ${GREEN}${PRIVATE_SUBNET_B}"
410 | 
411 | #private subnet c
412 | PRIVATE_SUBNET_C=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="PrivateSubnetCId") | .OutputValue')
413 | echo -e "${RED}PRIVATE_SUBNET_C: ${GREEN}${PRIVATE_SUBNET_C}"
414 | tput sgr0
415 | 
416 | #aws account id
417 | AWS_ACCOUNT_ID=$(aws --profile $aws_profile cloudformation describe-stacks --stack-name=$stack_name | jq -r '.Stacks[].Outputs[] | select(.OutputKey=="KMSKeyArn") | .OutputValue' | cut -d ':' -f5 )
418 | echo -e "${RED}AWS_ACCOUNT_ID: ${GREEN}${AWS_ACCOUNT_ID}"
419 | tput sgr0
420 | 
421 | echo
422 | echo -e "${GREEN}Please go to ${RED}https://k8svpn.$domain_name ${GREEN}and configure the VPN server using user: ${RED}pritunl ${GREEN}and password: ${RED}pritunl."
423 | echo
424 | echo -e "${GREEN}If you don't want to configure the Security Group manually, please configure the server to use the port ${RED}12777. ${GREEN}An option to set the port will be added soon."
425 | tput sgr0
426 | echo
427 | 
428 | #replace the values from the CloudFormation outputs
429 | ##kms key arn
430 | sed -i -e 's,kms_key_arn,'"$KMS_KEY_ARN"',g' kube-aws/cluster.yaml
431 | 
432 | ## vpc id
433 | sed -i -e 's,vpc_id,'"$VPC_ID"',g' kube-aws/cluster.yaml
434 | 
435 | ## vpc CIDR
436 | sed -i -e 's,vpc_cidr,'"$VPC_CIDR"',g' kube-aws/cluster.yaml
437 | 
438 | ## route table id
439 | sed -i -e 's,route_table_id,'"$ROUTE_TABLE_ID"',g' kube-aws/cluster.yaml
440 | 
441 | ## private subnet a
442 | sed -i -e 's,private_subnet_a,'"$PRIVATE_SUBNET_A"',g' kube-aws/cluster.yaml
443 | 
444 | ## private subnet b
445 | sed -i -e 's,private_subnet_b,'"$PRIVATE_SUBNET_B"',g' kube-aws/cluster.yaml
446 | 
447 | ## private subnet c
448 | sed -i -e 's,private_subnet_c,'"$PRIVATE_SUBNET_C"',g' kube-aws/cluster.yaml
449 | 
450 | ## aws accont id
451 | sed -i -e 's,aws_account_id,'"$AWS_ACCOUNT_ID"',g' iam/iam.yaml
452 | 
453 | #clean sed generated files
454 | find . -name "*-e" -exec rm -rf {} \;
455 | 
456 | #####################################################################################################
457 | # kube-aws setup
458 | #####################################################################################################
459 | 
460 | cd kube-aws
461 | 
462 | #select the S3 bucket to use for deployment
463 | PS3="${GREEN_PS3}Please select which S3 bucket to use:"
464 | echo
465 | set -- $(aws --profile $aws_profile s3 ls | cut -d " " -f3)
466 | select opt in "$@"
467 | do
468 |     case $opt in
469 |         $opt)
470 |             bucket="${opt%.}"
471 |             tput sgr0
472 |             echo "$bucket"
473 |             break
474 |             ;;
475 |         *) echo invalid option;;
476 |     esac
477 | done
478 | echo
479 | 
480 | #cluster.yaml config
481 | echo
482 | echo -e "${BLUE}Please make the desired changes in ${ORANGE}kube-aws/cluster.yaml ${BLUE}and press ENTER"
483 | tput sgr0
484 | read -p " [ENTER]: " cluster_yaml
485 | echo
486 | 
487 | #generate credentials
488 | echo -e "${ORANGE}Generate credentials"
489 | tput sgr0
490 | kube-aws render credentials --generate-ca
491 | echo
492 | 
493 | #render stack
494 | echo -e "${ORANGE}Render stack"
495 | tput sgr0
496 | kube-aws render stack
497 | echo
498 | 
499 | #validate stack
500 | echo -e "${ORANGE}Validate stack"
501 | tput sgr0
502 | AWS_PROFILE=$aws_profile kube-aws validate --s3-uri s3://$bucket
503 | echo
504 | 
505 | #export stack
506 | echo -e "${ORANGE}Export stack"
507 | tput sgr0
508 | AWS_PROFILE=$aws_profile kube-aws up --s3-uri s3://$bucket --export
509 | echo
510 | 
511 | #deploy
512 | echo -e "${ORANGE}Deploy stack"
513 | tput sgr0
514 | AWS_PROFILE=$aws_profile kube-aws up --s3-uri s3://$bucket
515 | echo
516 | 
517 | echo -e "${GREEN}If you didn't configure the VPN server yet, please go to ${RED}https://k8svpn.$domain_name ${GREEN}and set it up using the user: ${RED}pritunl ${GREEN}and the password: ${RED}pritunl"
518 | echo
519 | echo -e "${GREEN}If you don't want to configure the Security Group manually, please configure the server to use the port ${RED}12777. ${GREEN}An option to set the port will be added soon."
520 | tput sgr0
521 | echo
522 | 
523 | echo -e "${GREEN}Done"
524 | tput sgr0
525 | 


--------------------------------------------------------------------------------
/examples/rbac/pod-reader-role.yaml:
--------------------------------------------------------------------------------
 1 | kind: Role
 2 | apiVersion: rbac.authorization.k8s.io/v1beta1
 3 | metadata:
 4 |   namespace: kube-system
 5 |   name: pod-reader
 6 | rules:
 7 | - apiGroups: [""] # "" indicates the core API group
 8 |   resources: ["pods", "secrets"]
 9 |   verbs: ["get", "watch", "list"]
10 | 


--------------------------------------------------------------------------------
/src/addons/deploy:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | RED='\033[0;31m'
 4 | GREEN='\033[0;32m'
 5 | ORANGE='\033[0;33m'
 6 | BLUE='\033[0;34m'
 7 | GREEN_PS3=$'\e[0;32m'
 8 | 
 9 | #########################################################################################
10 | # external-dns
11 | #########################################################################################
12 | echo
13 | echo -e "${BLUE}Deploy external-dns?"
14 | tput sgr0
15 | read -p "Y/N [N]: " external_dns
16 | 
17 | if [[ $external_dns =~ ^([yY][eE][sS]|[yY])$ ]]; then
18 | 
19 |   kubectl apply -f external-dns/external-dns.yaml
20 | else
21 |   echo -e "Skipping"
22 | fi
23 | 
24 | 
25 | #########################################################################################
26 | # ingress
27 | #########################################################################################
28 | echo
29 | echo -e "${BLUE}Deploy nginx ingress controller?"
30 | tput sgr0
31 | read -p "Y/N [N]: " nginx_ingress
32 | 
33 | if [[ $nginx_ingress =~ ^([yY][eE][sS]|[yY])$ ]]; then
34 | 
35 |   kubectl apply -f ingress/nginx/rbac.yaml
36 |   kubectl apply -f ingress/nginx/nginx.lb.yaml
37 | 
38 | else
39 |   echo -e "Skipping"
40 | fi
41 | 
42 | #########################################################################################
43 | # kube-lego
44 | #########################################################################################
45 | echo
46 | echo -e "${BLUE}Deploy kube-lego?"
47 | tput sgr0
48 | read -p "Y/N [N]: " kube_lego
49 | 
50 | if [[ $kube_lego =~ ^([yY][eE][sS]|[yY])$ ]]; then
51 | 
52 |   kubectl apply -f kube-lego/rbac.yaml
53 |   kubectl apply -f kube-lego/kube-lego.yaml
54 | 
55 | else
56 |   echo -e "Skipping"
57 | fi
58 | 
59 | echo -e "${GREEN}Done"
60 | tput sgr0
61 | 
62 | #########################################################################################
63 | # fluentd-kubernetes-cloudwatch
64 | #########################################################################################
65 | echo
66 | echo -e "${BLUE}Deploy fluentd-kubernetes-cloudwatch?"
67 | tput sgr0
68 | read -p "Y/N [N]: " fluentd_cloudwatch
69 | 
70 | if [[ $fluentd_cloudwatch =~ ^([yY][eE][sS]|[yY])$ ]]; then
71 | 
72 |   kubectl apply -f fluentd-kubernetes-cloudwatch/log.ns.yaml
73 |   kubectl apply -f fluentd-kubernetes-cloudwatch/fluentd.rbac.yaml
74 |   #Custom Fluentd settings
75 |   echo
76 |   echo -e "${BLUE}Make sure you updated the configmap for fluentd with your custom configuration ${ORANGE}addons/fluentd/fluentd-kubernetes-cloudwatch/fluentd.cm.yaml ${BLUE}Press ENTER to continue"
77 |   tput sgr0
78 |   read -p " [ENTER]: " fluent_config
79 |   echo
80 |   kubectl apply -f fluentd-kubernetes-cloudwatch/fluentd.cm.yaml
81 |   kubectl apply -f fluentd-kubernetes-cloudwatch/fluentd.ds.yaml
82 | 
83 | else
84 |   echo -e "Skipping"
85 | fi
86 | 
87 | echo -e "${GREEN}Done"
88 | tput sgr0
89 | 


--------------------------------------------------------------------------------
/src/addons/external-dns/external-dns.yaml:
--------------------------------------------------------------------------------
 1 | apiVersion: extensions/v1beta1
 2 | kind: Deployment
 3 | metadata:
 4 |   name: external-dns
 5 |   namespace: kube-system
 6 | spec:
 7 |   strategy:
 8 |     type: Recreate
 9 |   template:
10 |     metadata:
11 |       labels:
12 |         app: external-dns
13 |       annotations:
14 |         iam.amazonaws.com/role: k8s-route53-external-dns
15 |     spec:
16 |       containers:
17 |       - name: external-dns
18 |         image: registry.opensource.zalan.do/teapot/external-dns:v0.4.7
19 |         args:
20 |         - --source=service
21 |         - --source=ingress
22 |         - --domain-filter=domain_name. # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
23 |         - --provider=aws
24 |         - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization
25 |         - --registry=txt
26 |         - --txt-owner-id=kube-aws # set a owner id
27 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/docker-image/v0.12/alpine-cloudwatch/Dockerfile:
--------------------------------------------------------------------------------
 1 | FROM fluent/fluentd:v0.12.33
 2 | USER root
 3 | WORKDIR /home/fluent
 4 | ENV PATH /home/fluent/.gem/ruby/2.3.0/bin:$PATH
 5 | 
 6 | RUN set -ex \
 7 |     && apk add --no-cache --virtual .build-deps \
 8 |         build-base \
 9 |         ruby-dev \
10 |         libffi-dev \
11 |     && echo 'gem: --no-document' >> /etc/gemrc \
12 |     && gem install fluent-plugin-secure-forward \
13 |     && gem install fluent-plugin-record-reformer \
14 |     && gem install aws-sdk-core -v 2.10.50 \
15 |     && gem install fluent-plugin-cloudwatch-logs -v 0.4.2 \
16 |     && gem install fluent-plugin-kubernetes_metadata_filter \
17 |     && apk del .build-deps \
18 |     && gem sources --clear-all \
19 |     && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
20 | 
21 | # Copy configuration files
22 | COPY ./conf/fluent.conf /fluentd/etc/
23 | COPY ./conf/kubernetes.conf /fluentd/etc/
24 | 
25 | # Copy plugins
26 | COPY plugins /fluentd/plugins/
27 | 
28 | # Environment variables
29 | ENV FLUENTD_OPT=""
30 | ENV FLUENTD_CONF="fluent.conf"
31 | 
32 | # jemalloc is memory optimization only available for td-agent
33 | # td-agent is provided and QA'ed by treasuredata as rpm/deb/.. package
34 | # -> td-agent (stable) vs fluentd (edge)
35 | #ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
36 | 
37 | # Run Fluentd
38 | CMD exec fluentd -c /fluentd/etc/$FLUENTD_CONF -p /fluentd/plugins $FLUENTD_OPT
39 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/docker-image/v0.12/alpine-cloudwatch/conf/fluent.conf:
--------------------------------------------------------------------------------
1 | @include kubernetes.conf
2 | 
3 | <match **>
4 |   type cloudwatch_logs
5 |   log_group_name "#{ENV['LOG_GROUP_NAME']}"
6 |   auto_create_stream true
7 |   use_tag_as_stream true
8 | </match>
9 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/docker-image/v0.12/alpine-cloudwatch/conf/kubernetes.conf:
--------------------------------------------------------------------------------
  1 | <match fluent.**>
  2 |   type null
  3 | </match>
  4 | 
  5 | <source>
  6 |   type tail
  7 |   path /var/log/containers/*.log
  8 |   pos_file /var/log/fluentd-containers.log.pos
  9 |   time_format %Y-%m-%dT%H:%M:%S.%NZ
 10 |   tag kubernetes.*
 11 |   format json
 12 |   read_from_head true
 13 | </source>
 14 | 
 15 | <source>
 16 |   type tail
 17 |   format /^(?<time>[^ ]* [^ ,]*)[^\[]*\[[^\]]*\]\[(?<severity>[^ \]]*) *\] (?<message>.*)$/
 18 |   time_format %Y-%m-%d %H:%M:%S
 19 |   path /var/log/salt/minion
 20 |   pos_file /var/log/fluentd-salt.pos
 21 |   tag salt
 22 | </source>
 23 | 
 24 | <source>
 25 |   type tail
 26 |   format syslog
 27 |   path /var/log/startupscript.log
 28 |   pos_file /var/log/fluentd-startupscript.log.pos
 29 |   tag startupscript
 30 | </source>
 31 | 
 32 | <source>
 33 |   type tail
 34 |   format /^time="(?<time>[^)]*)" level=(?<severity>[^ ]*) msg="(?<message>[^"]*)"( err="(?<error>[^"]*)")?( statusCode=($<status_code>\d+))?/
 35 |   path /var/log/docker.log
 36 |   pos_file /var/log/fluentd-docker.log.pos
 37 |   tag docker
 38 | </source>
 39 | 
 40 | <source>
 41 |   type tail
 42 |   format none
 43 |   path /var/log/etcd.log
 44 |   pos_file /var/log/fluentd-etcd.log.pos
 45 |   tag etcd
 46 | </source>
 47 | 
 48 | <source>
 49 |   type tail
 50 |   format kubernetes
 51 |   multiline_flush_interval 5s
 52 |   path /var/log/kubelet.log
 53 |   pos_file /var/log/fluentd-kubelet.log.pos
 54 |   tag kubelet
 55 | </source>
 56 | 
 57 | <source>
 58 |   type tail
 59 |   format kubernetes
 60 |   multiline_flush_interval 5s
 61 |   path /var/log/kube-proxy.log
 62 |   pos_file /var/log/fluentd-kube-proxy.log.pos
 63 |   tag kube-proxy
 64 | </source>
 65 | 
 66 | <source>
 67 |   type tail
 68 |   format kubernetes
 69 |   multiline_flush_interval 5s
 70 |   path /var/log/kube-apiserver.log
 71 |   pos_file /var/log/fluentd-kube-apiserver.log.pos
 72 |   tag kube-apiserver
 73 | </source>
 74 | 
 75 | <source>
 76 |   type tail
 77 |   format kubernetes
 78 |   multiline_flush_interval 5s
 79 |   path /var/log/kube-controller-manager.log
 80 |   pos_file /var/log/fluentd-kube-controller-manager.log.pos
 81 |   tag kube-controller-manager
 82 | </source>
 83 | 
 84 | <source>
 85 |   type tail
 86 |   format kubernetes
 87 |   multiline_flush_interval 5s
 88 |   path /var/log/kube-scheduler.log
 89 |   pos_file /var/log/fluentd-kube-scheduler.log.pos
 90 |   tag kube-scheduler
 91 | </source>
 92 | 
 93 | <source>
 94 |   type tail
 95 |   format kubernetes
 96 |   multiline_flush_interval 5s
 97 |   path /var/log/rescheduler.log
 98 |   pos_file /var/log/fluentd-rescheduler.log.pos
 99 |   tag rescheduler
100 | </source>
101 | 
102 | <source>
103 |   type tail
104 |   format kubernetes
105 |   multiline_flush_interval 5s
106 |   path /var/log/glbc.log
107 |   pos_file /var/log/fluentd-glbc.log.pos
108 |   tag glbc
109 | </source>
110 | 
111 | <source>
112 |   type tail
113 |   format kubernetes
114 |   multiline_flush_interval 5s
115 |   path /var/log/cluster-autoscaler.log
116 |   pos_file /var/log/fluentd-cluster-autoscaler.log.pos
117 |   tag cluster-autoscaler
118 | </source>
119 | 
120 | <filter kubernetes.**>
121 |   type kubernetes_metadata
122 | </filter>
123 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/docker-image/v0.12/alpine-cloudwatch/conf/systemd.conf:
--------------------------------------------------------------------------------
 1 | 
 2 | # Logs from systemd-journal for interesting services.
 3 | <source>
 4 |   @type systemd
 5 |   filters [{ "_SYSTEMD_UNIT": "kubelet.service" }]
 6 |   pos_file /var/log/fluentd-journald-kubelet.pos
 7 |   read_from_head true
 8 |   tag kubelet
 9 | </source>
10 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/docker-image/v0.12/alpine-cloudwatch/plugins/parser_kubernetes.rb:
--------------------------------------------------------------------------------
 1 | #
 2 | # Fluentd
 3 | #
 4 | #    Licensed under the Apache License, Version 2.0 (the "License");
 5 | #    you may not use this file except in compliance with the License.
 6 | #    You may obtain a copy of the License at
 7 | #
 8 | #        http://www.apache.org/licenses/LICENSE-2.0
 9 | #
10 | #    Unless required by applicable law or agreed to in writing, software
11 | #    distributed under the License is distributed on an "AS IS" BASIS,
12 | #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | #    See the License for the specific language governing permissions and
14 | #    limitations under the License.
15 | #
16 | 
17 | # The following Fluentd parser plugin, aims to simplify the parsing of multiline
18 | # logs found in Kubernetes nodes. Since many log files shared the same format and
19 | # in order to simplify the configuration, this plugin provides a 'kubernetes' format
20 | # parser (built on top of MultilineParser).
21 | #
22 | # When tailing files, this 'kubernetes' format should be applied to the following
23 | # log file sources:
24 | #
25 | #  - /var/log/kubelet.log
26 | #  - /var/log/kube-proxy.log
27 | #  - /var/log/kube-apiserver.log
28 | #  - /var/log/kube-controller-manager.log
29 | #  - /var/log/kube-scheduler.log
30 | #  - /var/log/rescheduler.log
31 | #  - /var/log/glbc.log
32 | #  - /var/log/cluster-autoscaler.log
33 | #
34 | # Usage:
35 | #
36 | # ---- fluentd.conf ----
37 | #
38 | # <source>
39 | #    type tail
40 | #    format kubernetes
41 | #    path ./kubelet.log
42 | #    read_from_head yes
43 | #    tag kubelet
44 | # </source>
45 | #
46 | # ----   EOF       ---
47 | 
48 | require 'fluent/parser'
49 | 
50 | module Fluent
51 |   class KubernetesParser < Fluent::TextParser::MultilineParser
52 |     Fluent::Plugin.register_parser("kubernetes", self)
53 | 
54 |     CONF_FORMAT_FIRSTLINE = %q{/^\w\d{4}/}
55 |     CONF_FORMAT1 = %q{/^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/}
56 |     CONF_TIME_FORMAT = "%m%d %H:%M:%S.%N"
57 | 
58 |     def configure(conf)
59 |       conf['format_firstline'] = CONF_FORMAT_FIRSTLINE
60 |       conf['format1'] = CONF_FORMAT1
61 |       conf['time_format'] = CONF_TIME_FORMAT
62 |       super
63 |     end
64 |   end
65 | end
66 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/fluentd-kubernetes-cloudwatch/fluentd.cm.yaml:
--------------------------------------------------------------------------------
 1 | apiVersion: v1
 2 | kind: ConfigMap
 3 | metadata:
 4 |   name: fluentd
 5 |   namespace: log
 6 | data:
 7 |   fluent.conf: |
 8 |     <match fluent.**>
 9 |       @type null
10 |     </match>
11 | 
12 |     ################################################################################################################
13 |     ##########################################   Custom Configuration    #######################################################
14 | 
15 |     ##### Example App 1 #####
16 |     <source>
17 |       @type tail
18 |       path /var/log/containers/example-app-1-*
19 |       pos_file /logpos/example-app-1.log.pos
20 |       time_format %Y-%m-%dT%H:%M:%S
21 |       tag example-app-1.*
22 |       format json
23 |       read_from_head true
24 |     </source>
25 | 
26 |     <filter example-app-1.**>
27 |       @type kubernetes_metadata
28 |     </filter>
29 | 
30 |     <filter example-app-1.**>
31 |       @type record_transformer
32 |       <record>
33 |         hostname ${hostname}
34 |       </record>
35 |     </filter>
36 | 
37 |     <match example-app-1.**>
38 |       @type cloudwatch_logs
39 |       log_group_name "Production/ExampleApp1"
40 |       use_tag_as_stream true
41 |       remove_log_group_name_key true
42 |       auto_create_stream true
43 |       put_log_events_retry_limit 20
44 |     </match>
45 |     ################################################################################################################
46 | 
47 | 
48 |     ##### Example App 2 #####
49 |     <source>
50 |       @type tail
51 |       path /var/log/containers/example-app-2-*
52 |       pos_file /logpos/example-app-2.log.pos
53 |       time_format %Y-%m-%dT%H:%M:%S
54 |       tag example-app-2.*
55 |       format json
56 |       read_from_head true
57 |     </source>
58 | 
59 |     <filter example-app-2.**>
60 |       @type kubernetes_metadata
61 |     </filter>
62 | 
63 |     <filter example-app-2.**>
64 |       @type record_transformer
65 |       <record>
66 |         hostname ${hostname}
67 |       </record>
68 |     </filter>
69 | 
70 |     <match example-app-2.**>
71 |       @type cloudwatch_logs
72 |       log_group_name "Production/ExampleApp2"
73 |       use_tag_as_stream true
74 |       remove_log_group_name_key true
75 |       auto_create_stream true
76 |       put_log_events_retry_limit 20
77 |     </match>
78 |     ################################################################################################################
79 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/fluentd-kubernetes-cloudwatch/fluentd.ds.yaml:
--------------------------------------------------------------------------------
 1 | apiVersion: extensions/v1beta1
 2 | kind: DaemonSet
 3 | metadata:
 4 |   labels:
 5 |     name: fluentd
 6 |   name: fluentd
 7 |   namespace: log
 8 | spec:
 9 |   updateStrategy:
10 |     type: RollingUpdate
11 |   template:
12 |     metadata:
13 |       labels:
14 |         app: fluentd
15 |       annotations:
16 |         iam.amazonaws.com/role: k8s-fluentd-cloudwatch
17 |     spec:
18 |       tolerations:
19 |       - operator: Exists
20 |         effect: NoSchedule
21 |       - operator: Exists
22 |         effect: NoExecute
23 |       - operator: Exists
24 |         key: CriticalAddonsOnly
25 |       serviceAccount: fluentd
26 |       containers:
27 |         - image: camil/fluentd-kubernetes-cloudwatch:0.1.0
28 |           name: fluentd
29 |           ports:
30 |           volumeMounts:
31 |             - mountPath: /var/log/containers
32 |               name: containerlogs
33 |               readOnly: true
34 |             - mountPath: /var/lib/docker/containers/
35 |               name: dockercontainerlogs
36 |               readOnly: true
37 |             - mountPath: /var/log/pods
38 |               name: podlogs
39 |               readOnly: true
40 |             - mountPath: /fluentd/etc
41 |               name: fluentd-config
42 |             - mountPath: /logpos
43 |               name: logpos
44 |           imagePullPolicy: Always
45 |           env:
46 |             - name: AWS_REGION
47 |               value: aws_region
48 |       volumes:
49 |         - name: containerlogs
50 |           hostPath:
51 |             path: /var/log/containers
52 |         - name: podlogs
53 |           hostPath:
54 |             path: /var/log/pods
55 |         - name: dockercontainerlogs
56 |           hostPath:
57 |             path: /var/lib/docker/containers/
58 |         - name: logpos
59 |           hostPath:
60 |             path: /var/log/fluentd/pos
61 |         - name: fluentd-config
62 |           configMap:
63 |             name: fluentd
64 |             items:
65 |               - key: fluent.conf
66 |                 path: fluent.conf
67 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/fluentd-kubernetes-cloudwatch/fluentd.rbac.yaml:
--------------------------------------------------------------------------------
 1 | ---
 2 | apiVersion: v1
 3 | kind: ServiceAccount
 4 | metadata:
 5 |   name: fluentd
 6 |   namespace: log
 7 | 
 8 | ---
 9 | apiVersion: rbac.authorization.k8s.io/v1beta1
10 | kind: ClusterRole
11 | metadata:
12 |   name: fluentd
13 |   namespace: log
14 | rules:
15 | - apiGroups:
16 |   - ""
17 |   resources:
18 |   - pods
19 |   verbs:
20 |   - get
21 |   - list
22 |   - watch
23 | 
24 | ---
25 | kind: ClusterRoleBinding
26 | apiVersion: rbac.authorization.k8s.io/v1beta1
27 | metadata:
28 |   name: fluentd
29 | roleRef:
30 |   kind: ClusterRole
31 |   name: fluentd
32 |   apiGroup: rbac.authorization.k8s.io
33 | subjects:
34 | - kind: ServiceAccount
35 |   name: fluentd
36 |   namespace: log
37 | 


--------------------------------------------------------------------------------
/src/addons/fluentd/fluentd-kubernetes-cloudwatch/log.ns.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: v1
2 | kind: Namespace
3 | metadata:
4 |   name: log
5 | 


--------------------------------------------------------------------------------
/src/addons/ingress/nginx/nginx.lb.yaml:
--------------------------------------------------------------------------------
  1 | kind: Service
  2 | apiVersion: v1
  3 | metadata:
  4 |   name: nginx-default-backend
  5 |   namespace: nginx-ingress
  6 |   labels:
  7 |     k8s-addon: ingress-nginx.addons.k8s.io
  8 | spec:
  9 |   ports:
 10 |   - port: 80
 11 |     targetPort: http
 12 |   selector:
 13 |     app: nginx-default-backend
 14 | ---
 15 | kind: Deployment
 16 | apiVersion: extensions/v1beta1
 17 | metadata:
 18 |   name: nginx-default-backend
 19 |   namespace: nginx-ingress
 20 |   labels:
 21 |     k8s-addon: ingress-nginx.addons.k8s.io
 22 | spec:
 23 |   replicas: 1
 24 |   template:
 25 |     metadata:
 26 |       labels:
 27 |         k8s-addon: ingress-nginx.addons.k8s.io
 28 |         app: nginx-default-backend
 29 |     spec:
 30 |       terminationGracePeriodSeconds: 60
 31 |       containers:
 32 |       - name: default-http-backend
 33 |         image: gcr.io/google_containers/defaultbackend:1.3
 34 |         livenessProbe:
 35 |           httpGet:
 36 |             path: /healthz
 37 |             port: 8080
 38 |             scheme: HTTP
 39 |           initialDelaySeconds: 30
 40 |           timeoutSeconds: 5
 41 |         resources:
 42 |           limits:
 43 |             cpu: 10m
 44 |             memory: 20Mi
 45 |           requests:
 46 |             cpu: 10m
 47 |             memory: 20Mi
 48 |         ports:
 49 |         - name: http
 50 |           containerPort: 8080
 51 |           protocol: TCP
 52 | ---
 53 | kind: ConfigMap
 54 | apiVersion: v1
 55 | metadata:
 56 |   name: ingress-nginx
 57 |   namespace: nginx-ingress
 58 |   labels:
 59 |     k8s-addon: ingress-nginx.addons.k8s.io
 60 | data:
 61 |   use-proxy-protocol: "true"
 62 | 
 63 | ---
 64 | 
 65 | kind: Service
 66 | apiVersion: v1
 67 | metadata:
 68 |   name: ingress-nginx
 69 |   namespace: nginx-ingress
 70 |   labels:
 71 |     k8s-addon: ingress-nginx.addons.k8s.io
 72 |   annotations:
 73 |     service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
 74 | spec:
 75 |   type: LoadBalancer
 76 |   #loadBalancerSourceRanges:
 77 |   #- 8.8.8.8/32
 78 |   #- 8.8.4.4/32
 79 |   selector:
 80 |     app: ingress-nginx
 81 |   ports:
 82 |   - name: http
 83 |     port: 80
 84 |     targetPort: http
 85 |   - name: https
 86 |     port: 443
 87 |     targetPort: https
 88 | ---
 89 | kind: Deployment
 90 | apiVersion: extensions/v1beta1
 91 | metadata:
 92 |   name: ingress-nginx
 93 |   namespace: nginx-ingress
 94 |   labels:
 95 |     k8s-addon: ingress-nginx.addons.k8s.io
 96 | spec:
 97 |   replicas: 1
 98 |   template:
 99 |     metadata:
100 |       labels:
101 |         app: ingress-nginx
102 |         k8s-addon: ingress-nginx.addons.k8s.io
103 |     spec:
104 |       terminationGracePeriodSeconds: 60
105 |       containers:
106 |       - image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0-beta.17
107 |         name: ingress-nginx
108 |         imagePullPolicy: Always
109 |         ports:
110 |           - name: http
111 |             containerPort: 80
112 |             protocol: TCP
113 |           - name: https
114 |             containerPort: 443
115 |             protocol: TCP
116 |         livenessProbe:
117 |           httpGet:
118 |             path: /healthz
119 |             port: 10254
120 |             scheme: HTTP
121 |           initialDelaySeconds: 30
122 |           timeoutSeconds: 5
123 |         env:
124 |           - name: POD_NAME
125 |             valueFrom:
126 |               fieldRef:
127 |                 fieldPath: metadata.name
128 |           - name: POD_NAMESPACE
129 |             valueFrom:
130 |               fieldRef:
131 |                 fieldPath: metadata.namespace
132 |         args:
133 |         - /nginx-ingress-controller
134 |         - --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
135 |         - --configmap=$(POD_NAMESPACE)/ingress-nginx
136 |         - --publish-service=$(POD_NAMESPACE)/ingress-nginx
137 | 


--------------------------------------------------------------------------------
/src/addons/ingress/nginx/rbac.yaml:
--------------------------------------------------------------------------------
  1 | apiVersion: v1
  2 | kind: Namespace
  3 | metadata:
  4 |   name: nginx-ingress
  5 | ---
  6 | apiVersion: rbac.authorization.k8s.io/v1beta1
  7 | kind: ClusterRole
  8 | metadata:
  9 |   name: nginx-ingress-clusterrole
 10 | rules:
 11 |   - apiGroups:
 12 |       - ""
 13 |     resources:
 14 |       - configmaps
 15 |       - endpoints
 16 |       - nodes
 17 |       - pods
 18 |       - secrets
 19 |     verbs:
 20 |       - list
 21 |       - watch
 22 |       - create
 23 |       - update
 24 |   - apiGroups:
 25 |       - ""
 26 |     resources:
 27 |       - services
 28 |     verbs:
 29 |       - get
 30 |       - list
 31 |       - watch
 32 |       - update
 33 |   - apiGroups:
 34 |       - "extensions"
 35 |     resources:
 36 |       - ingresses
 37 |     verbs:
 38 |       - get
 39 |       - list
 40 |       - watch
 41 |   - apiGroups:
 42 |       - ""
 43 |     resources:
 44 |         - events
 45 |     verbs:
 46 |         - create
 47 |         - patch
 48 |   - apiGroups:
 49 |       - "extensions"
 50 |     resources:
 51 |       - ingresses/status
 52 |     verbs:
 53 |       - update
 54 | ---
 55 | apiVersion: rbac.authorization.k8s.io/v1beta1
 56 | kind: Role
 57 | metadata:
 58 |   name: nginx-ingress-role
 59 |   namespace: nginx-ingress
 60 | rules:
 61 |   - apiGroups:
 62 |       - ""
 63 |     resources:
 64 |       - configmaps
 65 |       - pods
 66 |       - secrets
 67 |     verbs:
 68 |       - get
 69 |   - apiGroups:
 70 |       - ""
 71 |     resources:
 72 |       - endpoints
 73 |     verbs:
 74 |       - get
 75 |       - create
 76 |       - update
 77 | ---
 78 | apiVersion: rbac.authorization.k8s.io/v1beta1
 79 | kind: RoleBinding
 80 | metadata:
 81 |   name: nginx-ingress-role-nisa-binding
 82 |   namespace: nginx-ingress
 83 | roleRef:
 84 |   apiGroup: rbac.authorization.k8s.io
 85 |   kind: Role
 86 |   name: nginx-ingress-role
 87 | subjects:
 88 |   - kind: ServiceAccount
 89 |     name: default
 90 |     namespace: nginx-ingress
 91 | ---
 92 | apiVersion: rbac.authorization.k8s.io/v1beta1
 93 | kind: ClusterRoleBinding
 94 | metadata:
 95 |   name: nginx-ingress-clusterrole-nisa-binding
 96 | roleRef:
 97 |   apiGroup: rbac.authorization.k8s.io
 98 |   kind: ClusterRole
 99 |   name: nginx-ingress-clusterrole
100 | subjects:
101 |   - kind: ServiceAccount
102 |     name: default
103 |     namespace: nginx-ingress
104 | 


--------------------------------------------------------------------------------
/src/addons/kube-lego/kube-lego.yaml:
--------------------------------------------------------------------------------
 1 | apiVersion: v1
 2 | kind: Namespace
 3 | metadata:
 4 |   name: kube-lego
 5 | ---
 6 | apiVersion: v1
 7 | metadata:
 8 |   name: kube-lego
 9 |   namespace: kube-lego
10 | data:
11 |   # modify this to specify your address
12 |   lego.email: "root@example.com"
13 |   # configure letsencrypt's production api
14 |   lego.url: "https://acme-v01.api.letsencrypt.org/directory"
15 |   lego.port: "8080"
16 | kind: ConfigMap
17 | ---
18 | apiVersion: extensions/v1beta1
19 | kind: Deployment
20 | metadata:
21 |   name: kube-lego
22 |   namespace: kube-lego
23 | spec:
24 |   replicas: 1
25 |   template:
26 |     metadata:
27 |       labels:
28 |         app: kube-lego
29 |     spec:
30 |       containers:
31 |       - name: kube-lego
32 |         image: jetstack/kube-lego:0.1.4
33 |         imagePullPolicy: Always
34 |         ports:
35 |         - containerPort: 8080
36 |         env:
37 |         - name: LEGO_EMAIL
38 |           valueFrom:
39 |             configMapKeyRef:
40 |               name: kube-lego
41 |               key: lego.email
42 |         - name: LEGO_URL
43 |           valueFrom:
44 |             configMapKeyRef:
45 |               name: kube-lego
46 |               key: lego.url
47 |         - name: LEGO_NAMESPACE
48 |           valueFrom:
49 |             fieldRef:
50 |               fieldPath: metadata.namespace
51 |         - name: LEGO_POD_IP
52 |           valueFrom:
53 |             fieldRef:
54 |               fieldPath: status.podIP
55 |         readinessProbe:
56 |           httpGet:
57 |             path: /healthz
58 |             port: 8080
59 |           initialDelaySeconds: 5
60 |           timeoutSeconds: 1
61 | 


--------------------------------------------------------------------------------
/src/addons/kube-lego/rbac.yaml:
--------------------------------------------------------------------------------
 1 | kind: ClusterRole
 2 | apiVersion: rbac.authorization.k8s.io/v1beta1
 3 | metadata:
 4 |   name: ingress-secret-admin
 5 | rules:
 6 | - apiGroups: [""]
 7 |   resources: ["secrets"]
 8 |   verbs:
 9 |   - get
10 |   - watch
11 |   - list
12 |   - create
13 |   - update
14 |   - patch
15 | - apiGroups: [""]
16 |   resources: ["services"]
17 |   verbs:
18 |   - get
19 |   - create
20 | - apiGroups: ["extensions"]
21 |   resources: ["ingresses"]
22 |   verbs:
23 |   - get
24 |   - watch
25 |   - list
26 |   - create
27 |   - update
28 |   - patch
29 | ---
30 | kind: ClusterRoleBinding
31 | apiVersion: rbac.authorization.k8s.io/v1beta1
32 | metadata:
33 |   name: kube-lego
34 | roleRef:
35 |   kind: ClusterRole
36 |   name: ingress-secret-admin
37 |   apiGroup: rbac.authorization.k8s.io
38 | subjects:
39 | - kind: ServiceAccount
40 |   name: default
41 |   namespace: kube-lego
42 | 


--------------------------------------------------------------------------------
/src/iam/deploy:
--------------------------------------------------------------------------------
 1 | #!/bin/sh
 2 | set -e
 3 | 
 4 | ROLE="k8s-iam"
 5 | STACK_NAME="k8s-iam"
 6 | STACK_FILE="iam"
 7 | SERVICE_NAME_PREFIX="k8s"
 8 | REGION="aws_region"
 9 | 
10 | STACK_PARAMS="ParameterKey=ServiceNamePrefix,ParameterValue=${SERVICE_NAME_PREFIX} ParameterKey=Role,ParameterValue=${ROLE}"
11 | STACK_TAGS="Key=Role,Value=${ROLE}"
12 | 
13 | for TEMPLATE_FILE in *.yaml
14 | do
15 |     echo "### Validating -> ${TEMPLATE_FILE}"
16 | 
17 |     aws cloudformation validate-template \
18 |             --template-body file://${TEMPLATE_FILE}
19 | 
20 |     echo
21 | done
22 | 
23 | aws --region $REGION cloudformation create-stack \
24 |     --stack-name $STACK_NAME \
25 |     --template-body file://./${TEMPLATE_FILE} \
26 |     --parameters ${STACK_PARAMS} \
27 |     --capabilities CAPABILITY_NAMED_IAM \
28 |     --tags ${STACK_TAGS}
29 | 


--------------------------------------------------------------------------------
/src/iam/iam.yaml:
--------------------------------------------------------------------------------
 1 | AWSTemplateFormatVersion: '2010-09-09'
 2 | Description: IAM [kube-aws-secure]
 3 | 
 4 | Parameters:
 5 |   Role:
 6 |     Type: String
 7 |   ServiceNamePrefix:
 8 |     Type: String
 9 | 
10 | Resources:
11 | 
12 | #route53-external-dns
13 |   K8sRoute53ExternalDns:
14 |     Type: "AWS::IAM::Role"
15 |     Properties:
16 |       AssumeRolePolicyDocument:
17 |         Version: "2012-10-17"
18 |         Statement:
19 |           -
20 |             Effect: "Allow"
21 |             Principal:
22 |               Service:
23 |                 - "ec2.amazonaws.com"
24 |             Action:
25 |               - "sts:AssumeRole"
26 |           -
27 |             Effect: "Allow"
28 |             Principal:
29 |               AWS:
30 |                 - "arn:aws:iam::aws_account_id:role/aws_region-workerMR"
31 |             Action:
32 |               - "sts:AssumeRole"
33 |       Path: "/"
34 |       RoleName: "k8s-route53-external-dns"
35 |       Policies:
36 |         -
37 |           PolicyName: "k8s-route53-external-dns"
38 |           PolicyDocument:
39 |             Version: "2012-10-17"
40 |             Statement:
41 |               -
42 |                 Effect: "Allow"
43 |                 Action:
44 |                   - "route53:ListHostedZonesByName"
45 |                   - "route53:ListHostedZones"
46 |                   - "route53:ListResourceRecordSets"
47 |                   - "elasticloadbalancing:DescribeLoadBalancers"
48 |                 Resource:
49 |                   - "*"
50 |               -
51 |                 Effect: "Allow"
52 |                 Action:
53 |                   - "route53:ChangeResourceRecordSets"
54 |                 Resource:
55 |                   - "arn:aws:route53:::hostedzone/hosted_zone_id"
56 | 
57 | #cloudwatchlogs
58 |   K8sFluentdCloudwatchLogs:
59 |     Type: "AWS::IAM::Role"
60 |     Properties:
61 |       AssumeRolePolicyDocument:
62 |         Version: "2012-10-17"
63 |         Statement:
64 |           -
65 |             Effect: "Allow"
66 |             Principal:
67 |               Service:
68 |                 - "ec2.amazonaws.com"
69 |             Action:
70 |               - "sts:AssumeRole"
71 |           -
72 |             Effect: "Allow"
73 |             Principal:
74 |               AWS:
75 |                 - "arn:aws:iam::aws_account_id:role/aws_region-workerMR"
76 |             Action:
77 |               - "sts:AssumeRole"
78 |       Path: "/"
79 |       RoleName: "k8s-fluentd-cloudwatch"
80 |       Policies:
81 |         -
82 |           PolicyName: "k8s-fluentd-cloudwatch"
83 |           PolicyDocument:
84 |             Version: "2012-10-17"
85 |             Statement:
86 |               -
87 |                 Effect: "Allow"
88 |                 Action:
89 |                   - "logs:CreateLogGroup"
90 |                   - "logs:CreateLogStream"
91 |                   - "logs:PutLogEvents"
92 |                   - "logs:DescribeLogStreams"
93 |                   - "logs:describeLogGroups"
94 |                 Resource:
95 |                   - "arn:aws:logs:*:*:*"
96 | 
97 | # ------
98 | 


--------------------------------------------------------------------------------
/src/kube-aws/cluster.yaml:
--------------------------------------------------------------------------------
   1 | # Unique name of Kubernetes cluster. In order to deploy
   2 | # more than one cluster into the same AWS account, this
   3 | # name must not conflict with an existing cluster.
   4 | clusterName: demokube #has to be the same as KUBERNETES_CLUSTER var in vpc/deploy
   5 | 
   6 | # CoreOS release channel to use. Currently supported options: alpha, beta, stable
   7 | # See coreos.com/releases for more information
   8 | releaseChannel: stable
   9 | 
  10 | # The AMI ID of CoreOS.
  11 | # If omitted, the latest AMI for the releaseChannel is used.
  12 | # This happens every time 'kube-aws update' is run, potentially replacing all instances with a newer version
  13 | #amiId: ""
  14 | # The ID of hosted zone to add the externalDNSName to.
  15 | # Either specify hostedZoneId or hostedZone, but not both
  16 | #hostedZoneId: ""
  17 | 
  18 | # Network ranges of sources you'd like SSH accesses to be allowed from, in CIDR notation. Defaults to ["0.0.0.0/0"] which allows any sources.
  19 | # Explicitly set to an empty array to completely disable it.
  20 | # If you do that, probably you would like to set securityGroupIds to provide this worker node pool an existing SG with a SSH access allowed from specific ranges.
  21 | #sshAccessAllowedSourceCIDRs:
  22 | #- 0.0.0.0/0
  23 | 
  24 | # The name of one of API endpoints defined in `apiEndpoints` below to be written in kubeconfig and then used by admins
  25 | # to access k8s API from their laptops, CI servers, or etc.
  26 | # Required if there are 2 or more API endpoints defined in `apiEndpoints`
  27 | #adminAPIEndpointName: versionedPublic
  28 | 
  29 | # Kubernetes API endpoints with each one has a DNS name and is with/without a managed/unmanaged load balancer, Route 53 record set
  30 | # CAUTION: `externalDNSName` must be omitted when there are one or more items under `apiEndpoints`
  31 | apiEndpoints:
  32 | - # The unique name of this API endpoint used to identify it inside CloudFormation stacks or
  33 |   # to be referenced from other parts of cluster.yaml
  34 |   name: default
  35 | 
  36 |   # DNS name for this endpoint, added to the kube-apiserver TLS cert
  37 |   # It must be somehow routable to the Kubernetes controller nodes
  38 |   # from worker nodes and external clients. Configure the options
  39 |   # below if you'd like kube-aws to create a Route53 record sets/hosted zones
  40 |   # for you.  Otherwise the deployer is responsible for making this name routable
  41 |   dnsName: kubeapi.domain_name
  42 | 
  43 |   # Configuration for the load balancer serving this endpoint
  44 |   # Omit all the settings when you want kube-aws not to provision a load balancer for you
  45 |   loadBalancer:
  46 |     # Specifies an existing load-balancer used for load-balancing controller nodes and serving this endpoint
  47 |     # Setting id requires all the other settings excluding `name` to be omitted because reusing an ELB implies that configuring other resources
  48 |     # like a Route 53 record set for the endpoint is now your responsibility!
  49 |     # Also, don't forget to add controller.securityGroupIds to include a glue SG to allow your existing ELB to access controller nodes created by kube-aws
  50 |     #id: existing-elb
  51 | 
  52 |     # All the subnets assigned to this load-balancer. Specified only when this load balancer is not reused but managed one
  53 |     # Must be omitted when `id` is specified
  54 |     subnets:
  55 |     - name: PrivateSubnetA
  56 |     - name: PrivateSubnetB
  57 |     - name: PrivateSubnetC
  58 | 
  59 |     # Set to true so that the managed ELB becomes an `internal` one rather than `internet-facing` one
  60 |     # When set to true while subnets are omitted, one or more private subnets in the top-level `subnets` must exist
  61 |     # Must be omitted when `id` is specified
  62 |     private: true
  63 | 
  64 |     # Set to 'network' to provision a Network Load Balancer instead of a classic ELB; this will cause
  65 |     # the controller nodes SG to allow inbound traffic from the VPC CIDR to port 443 TCP, see:
  66 |     # http://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html
  67 |     # Must be omitted when `id` is specified
  68 |     type: network
  69 | 
  70 |     # TTL in seconds for the Route53 RecordSet created if hostedZone.id is set to a non-nil value.
  71 |     recordSetTTL: 300
  72 | 
  73 | 
  74 | 
  75 |     # The Route 53 hosted zone is where the resulting Alias record is created for this endpoint
  76 |     #
  77 |     # Omitting hostedZone.id implies you must configure Route 53 hosted zone and record set or your own DNS so that worker nodes
  78 |     # are able to resolve DNS names of this API endpoint.
  79 |     #
  80 |     # Must be omitted when `id` is specified because kube-aws doesn't deal with the existing ELB by design
  81 |     hostedZone:
  82 |       # The ID of hosted zone to add the dnsName to.
  83 |       id: hosted_zone_id
  84 | 
  85 | #    # Network ranges of sources you'd like Kubernetes API accesses to be allowed from, in CIDR notation. Defaults to ["0.0.0.0/0"] which allows any sources.
  86 | #    # Explicitly set to an empty array to completely disable it.
  87 | #    # If you do that, probably you would like to set securityGroupIds to provide this load balancer an existing SG with a Kubernetes API access allowed from specific ranges.
  88 | #    apiAccessAllowedSourceCIDRs:
  89 | #    #- 0.0.0.0/0
  90 | #
  91 | #    # Existing security groups attached to this load balancer which are typically used to
  92 | #    # allow Kubernetes API accesses from admins and/or CD systems when `apiAccessAllowedSourceCIDRs` are explicitly set to an empty array
  93 | #    securityGroupIds:
  94 | #    - sg-1234567
  95 | #
  96 | #  #
  97 | #  # Common configuration #1: Unversioned, internet-facing API endpoint
  98 | #  #
  99 | #  - name: unversionedPublic
 100 | #    dnsName: api.example.com
 101 | #    loadBalancer:
 102 | #      id: elb-abcdefg
 103 | #
 104 | #  #
 105 | #  # Common configuration #2: Versioned, internet-facing API endpoint
 106 | #  #
 107 | #  - name: versionedPublic
 108 | #    dnsName: v1api.example.com
 109 | #    loadBalancer:
 110 | #      hostedZone:
 111 | #        id: hostedzone-abcedfg
 112 | #
 113 | #  #
 114 | #  # Common configuration #3: Unmanaged endpoint a.k.a Extra DNS name added to the kube-apiserver TLS cert
 115 | #  #
 116 | #  - name: extra
 117 | #    dnsName: youralias.example.com
 118 | #    loadBalancer:
 119 | #      managed: false
 120 | #
 121 | #  #
 122 | #  # Uncommon configuration #1: Managed, internet-facing API endpoint, without a Route53 record set
 123 | #  #
 124 | #  - name: elbOnly
 125 | #    dnsName: youralias.example.com
 126 | #    loadBalancer:
 127 | #      # Setting this to false allows you to omit hostedZone.id and hence the creation of Route53 record set is skipped
 128 | #      recordSetManaged: false
 129 | #
 130 | 
 131 | # Name of the SSH keypair already loaded into the AWS
 132 | # account being used to deploy this cluster.
 133 | keyName: key_name
 134 | 
 135 | # Additional keys to preload on the coreos account (keep this to a minimum)
 136 | # sshAuthorizedKeys:
 137 | # - "ssh-rsa AAAAEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEYEXAMPLEKEY example@example.org"
 138 | 
 139 | # Region to provision Kubernetes cluster
 140 | region: aws_region
 141 | 
 142 | # Availability Zone to provision Kubernetes cluster when placing nodes in a single availability zone (not highly-available) Comment out for multi availability zone setting and use the below `subnets` section instead.
 143 | # availabilityZone: us-east-1d
 144 | 
 145 | # ARN of the KMS key used to encrypt TLS assets.
 146 | kmsKeyArn: "kms_key_arn"
 147 | 
 148 | controller:
 149 | #  # Number of controller nodes to create, for more control use `controller.autoScalingGroup` and do not use this setting
 150 |   count: 2
 151 | #
 152 | #  # Maximum time to wait for controller creation
 153 |   createTimeout: PT60M
 154 | #
 155 | #  # Instance type for controller node.
 156 | #  # CAUTION: Don't use t2.micro or the cluster won't work. See https://github.com/kubernetes/kubernetes/issues/18975
 157 |   instanceType: t2.medium
 158 | #
 159 | #  # EC2 instance tags for controller nodes
 160 |   instanceTags:
 161 |     Role: controller
 162 |     SubRole: Development
 163 |     Application: Kubernetes
 164 | #
 165 |   rootVolume:
 166 | #    # Disk size (GiB) for controller node
 167 |     size: 30
 168 | #    # Disk type for controller node (one of standard, io1, or gp2)
 169 |     type: gp2
 170 | #    # Number of I/O operations per second (IOPS) that the controller node disk supports. Leave blank if controller.rootVolume.type is not io1
 171 | #    iops: 0
 172 | #
 173 | #  # Existing security groups attached to controller nodes which are typically used to
 174 | #  # (1) allow access from controller nodes to services running on an existing infrastructure
 175 | #  # (2) allow ssh accesses from bastions when `sshAccessAllowedSourceCIDRs` are explicitly set to an empty array
 176 | #  securityGroupIds:
 177 | #    - sg-1234abcd
 178 | #    - sg-5678efab
 179 | #
 180 | #  # Auto Scaling Group definition for controllers. If only `controllerCount` is specified, min and max will be the set to that value and `rollingUpdateMinInstancesInService` will be one less.
 181 |   autoScalingGroup:
 182 |     minSize: 1
 183 |     maxSize: 2
 184 |     rollingUpdateMinInstancesInService: 1
 185 | #  # If you specify managedIamRoleName the role created for controller nodes will not suffix the random id at the end
 186 | #  # Role will be created with "Ref": {"AWS::StackName"}-{"AWS::Region"}-yourManagedRole
 187 | #  # to follow the recommendation in AWS documentation  http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
 188 | #  # This is also intended to be used in combination with .experimental.kube2IamSupport. See #297 for more information.
 189 | #  #
 190 | #  # ATTENTION: Consider limiting number of characters in clusterName and iam.role.name to avoid the resulting IAM
 191 | #  # role name's length from exceeding the AWS limit: 64
 192 | #  # See https://github.com/kubernetes-incubator/kube-aws/issues/347
 193 | #  # if you omit this block kube-aws would create an IAM Role and a customer managed policy with a random name.
 194 |   iam:
 195 |      role:
 196 | #       # This will create a named IAM Role managed by kube-aws with the name {AWS::Region}-$managedIamRoleName.
 197 | #       # It will have attached a customer Managed Policy that you can modify afterwards if you need more permissions for your cluster.
 198 | #       # Be careful with the Statements you modify because an update could overwrite your own.
 199 | #       # the Statements included in the ManagedPolicy are the minimun ones required for the Controllers to run.
 200 |         name: "controllerMR"
 201 | #       # if you set managedPolicies here it will be attached in addition to the created managedPolicy in kube-aws for the cluster.
 202 | #       # CAUTION: if you attach a more restrictive policy in some resources (i.e ec2:* Deny) you can make kube-aws fail.
 203 | #       # managedPolicies:
 204 | #       # -  arn: "arn:aws:iam::aws:policy/AdministratorAccess"
 205 | #       # -  arn: "arn:aws:iam::YOURACCOUNTID:policy/YOURPOLICYNAME"
 206 | 
 207 | #       # if you set an InstanceProfile kube-aws will NOT create any IAM Role and will use the configured instanceProfile.
 208 | #       # CAUTION: you must ensure that the IAM Role linked to the listed InstanceProfile has enough permissions to ensure kube-aws to run.
 209 | #       # if you dont know which permissions are required is recommended to create a cluster with a managed role.
 210 | #       # instanceProfile:
 211 | #       #   arn: "arn:aws:iam::YOURACCOUNTID:instance-profile/INSTANCEPROFILENAME"
 212 | 
 213 | #  # If omitted, public subnets are created by kube-aws and used for controller nodes
 214 |   subnets:
 215 | #    # References subnets defined under the top-level `subnets` key by their names
 216 |   - name: PrivateSubnetA
 217 |   - name: PrivateSubnetB
 218 |   - name: PrivateSubnetC
 219 | #
 220 | #   # Kubernetes node labels to be added to controller nodes
 221 |   nodeLabels:
 222 |     kube-aws.coreos.com/role: controller
 223 | #
 224 | #  # User defined files that will be added to the Controller cluster cloud-init configuration in the "write_files:" section.
 225 | #  customFiles:
 226 | #    - path: "/etc/rkt/auth.d/docker.json"
 227 | #      permissions: 0600
 228 | #      content: |
 229 | #        { "rktKind": "dockerAuth", "rktVersion": "v1", ... }
 230 | #
 231 | #  # User defined systemd units that will be added to the Controller cluster cloud-init configuration in the "units:" section.
 232 | #  customSystemdUnits:
 233 | #    - name: monitoring.service
 234 | #      command: start
 235 | #      enable: true
 236 | #      content: |
 237 | #        [Unit]
 238 | #        Description=Example Custom Service
 239 | #        [Service]
 240 | #        ExecStart=/bin/rkt run --set-env TAGS=Controller ...
 241 | 
 242 | worker:
 243 | #
 244 | #  # Settings that apply to all worker node pools
 245 | #
 246 | #  # The name of the API endpoint defined in the top level `apiEndpoints` that the worker node pools will use to access the k8s API
 247 | #  # Required if there are 2 or more API endpoints defined in `apiEndpoints`
 248 | #  # Can be overriden per node pool by specifying `worker.nodePools[].apiEndpointName`
 249 | #  apiEndpointName: versionedPublic
 250 | #
 251 |   nodePools:
 252 |     - # Name of this node pool. Must be unique among all the node pools in this cluster
 253 |       name: nodepool1
 254 | #      # Subnet(s) to which worker nodes in this node pool are deployed
 255 | #      # References subnets defined under the top-level `subnets` key by their names
 256 | #      # If omitted, public subnets are created by kube-aws and used for worker nodes
 257 |       subnets:
 258 |       - name: PrivateSubnetA
 259 |       - name: PrivateSubnetB
 260 |       - name: PrivateSubnetC
 261 | #
 262 | #      # Existing "glue" security groups attached to worker nodes which are typically used to allow
 263 | #      # access from worker nodes to services running on an existing infrastructure
 264 | #      securityGroupIds:
 265 | #        - sg-1234abcd
 266 | #        - sg-5678efab
 267 | #
 268 | #      # Configuration for external managed ELBs for worker nodes
 269 | #      # Use this with k8s load balancers with type=NodePort. See https://kubernetes.io/docs/user-guide/services/#type-nodeport
 270 | #      #
 271 | #      # NOTICE: This is generally recommended over k8s managed load-balancers with type=LoadBalancer because it allows
 272 | #      # a manual but no-downtime rotation of kube-aws clusters
 273 | #      loadBalancer:
 274 | #        enabled: true
 275 | #        # Names of ELBs attached to worker nodes
 276 | #        names: [ "manuallymanagedelb" ]
 277 | #        # IDs of "glue" security groups attached to worker nodes to allow the ELBs to communicate with worker nodes
 278 | #        securityGroupIds: [ "sg-87654321" ]
 279 | #
 280 | #      # The name of the API endpoint defined in the top level `apiEndpoints` that this worker node pool will use to access the k8s API
 281 | #      # If not specified, defaults to `worker.apiEndpointName`
 282 | #      apiEndpointName: versionedPublic
 283 | #
 284 | #     # if you omit this block kube-aws would create an IAM Role and a customer managed policy with a random name.
 285 |       iam:
 286 |         role:
 287 | #       # This will create a named IAM Role managed by kube-aws with the name {AWS::Region}-$managedIamRoleName.
 288 | #       # It will have attached a customer Managed Policy that you can modify afterwards if you need more permissions for your cluster.
 289 | #       # Be careful with the Statements you modify because an update could overwrite your own.
 290 | #       # the Statements included in the ManagedPolicy are the minimun ones required for the Controllers to run.
 291 |           name: "workerMR"
 292 | #       # if you set managedPolicies here it will be attached in addition to the created managedPolicy in kube-aws for the cluster.
 293 | #       # CAUTION: if you attach a more restrictive policy in some resources (i.e ec2:* Deny) you can make kube-aws fail.
 294 | #       #   managedPolicies:
 295 | #       #     - arn: "arn:aws:iam::aws:policy/AdministratorAccess"
 296 | #       #     - arn: "arn:aws:iam::YOURACCOUNTID:policy/YOURPOLICYNAME"
 297 | 
 298 | #       # if you set an InstanceProfile kube-aws will NOT create any IAM Role and will use the configured instanceProfile.
 299 | #       # CAUTION: you must ensure that the IAM Role linked to the listed InstanceProfile has enough permissions to ensure kube-aws to run.
 300 | #       # if you dont know which permissions are required is recommended to create a cluster with a managed role.
 301 | #       # instanceProfile:
 302 | #       #   arn: "arn:aws:iam::YOURACCOUNTID:instance-profile/INSTANCEPROFILENAME"
 303 | #      # Configuration for external managed ALBs Target Groups for worker nodes
 304 | #      targetGroup:
 305 | #        enabled: true
 306 | #        # ARNs of ALBs Target Groups attached to worker nodes
 307 | #        arns:
 308 | #        - arn:aws:elasticloadbalancing:eu-west-1:xxxxxxxxxxxx:targetgroup/manuallymanagedetg/xxxxxxxxxxxxxxxx
 309 | #        # IDs of "glue" security groups attached to worker nodes to allow the ALBs Target Groups to communicate with worker nodes
 310 | #        securityGroupIds: [ "sg-87654321" ]
 311 | #
 312 | #      # If you specify managedIamRoleName the role created for worker nodes will not suffix the random id at the end
 313 | #      # Role will be created with "Ref": {"AWS::StackName"}-{"AWS::Region"}-yourManagedRole
 314 | #      # to follow the recommendation in AWS documentation  http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
 315 | #      # This is also intended to be used in combination with kube2IamSupport. See #297 for more information.
 316 | #      #
 317 | #      # ATTENTION: Consider limiting number of characters in clusterName, nodePools[].name and managedIamRoleName to
 318 | #      # avoid the resulting IAM role name's length from exceeding the AWS limit: 64
 319 | #      # See https://github.com/kubernetes-incubator/kube-aws/issues/347
 320 | #      managedIamRoleName: "yourManagedRole"
 321 | #
 322 | #      # Additional EBS volumes mounted on the worker
 323 | #      # No additional EBS volumes by default. All parameter values do not default - they must be explicitly defined
 324 | #      volumeMounts:
 325 | #      - type: "gp2"
 326 | #        iops: 0
 327 | #        size: 30
 328 | #        # Follow the aws convention of '/dev/xvd*' where '*' is a single letter from 'f' to 'z'
 329 | #        device: "/dev/xvdf"
 330 | #        path: "/ebs"
 331 | #
 332 | #      # Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller.
 333 | #      # nodeStatusUpdateFrequency: "10s"
 334 | #
 335 | #      #
 336 | #      # Settings only for ASG-based node pools
 337 | #      #
 338 | #
 339 | #      # Number of worker nodes to create for an autoscaling group based pool
 340 | #      # Specify `worker.autoScalingGroup` instead for more control over its min/max/desired capacity.
 341 | #      count: 1
 342 | #
 343 | #      # Instance type for worker nodes
 344 | #      # CAUTION: Don't use t2.micro or the cluster won't work. See https://github.com/kubernetes/kubernetes/issues/16122
 345 |       instanceType: t2.medium
 346 | #
 347 | #      # EC2 instance tags for worker nodes
 348 |       instanceTags:
 349 |         Role: worker
 350 |         Environment: Development
 351 |         Application: Kubernetes
 352 | #
 353 |       rootVolume:
 354 | #        # Disk size (GiB) for worker nodes
 355 |         size: 30
 356 | #        # Disk type for worker node (one of standard, io1, or gp2)
 357 |         type: gp2
 358 | #        # Number of I/O operations per second (IOPS) that the worker node disk supports. Leave blank if worker.rootVolume.type is not io1
 359 | #        iops: 0
 360 | #
 361 | #      # Maximum time to wait for worker creation
 362 |       createTimeout: PT60M
 363 | #
 364 | #      # Tenancy of the worker nodes. Options are "default" and "dedicated"
 365 | #      # Documentation: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/dedicated-instance.html
 366 | #      tenancy: default
 367 | #
 368 | #      # (Experimental) GPU Driver installation support
 369 | #      # Currently, only Nvidia driver is supported.
 370 | #      # This setting takes effect only when configured instance type is GPU enabled (p2, p3, g2, or g3).
 371 | #      # Make sure to choose 'docker' as container runtime when enabled this feature.
 372 | #      # Ensure that automatic Container Linux is disabled(it is disabled by default btw).
 373 | #      # Otherwise the installed driver may stop working when an OS update resulted in an updated kernel
 374 | #      gpu:
 375 | #        nvidia:
 376 | #          enabled: true
 377 | #          version: "375.66"
 378 | #
 379 | #      # Price (Dollars) to bid for spot instances. Omit for on-demand instances.
 380 | #      spotPrice: "0.05"
 381 | #
 382 | #      # When enabled, autoscaling groups managing worker nodes wait for nodes to be up and running.
 383 | #      # This is enabled by default.
 384 |       waitSignal:
 385 |         enabled: true
 386 | #        # Max number of nodes concurrently updated
 387 |         maxBatchSize: 1
 388 | #
 389 | #      # Auto Scaling Group definition for workers. If only `workerCount` is specified, min and max will be the set to that value and `rollingUpdateMinInstancesInService` will be one less.
 390 |       autoScalingGroup:
 391 |         minSize: 2
 392 |         maxSize: 4
 393 |         rollingUpdateMinInstancesInService: 2
 394 | #
 395 | #      #
 396 | #      # Spot fleet config for worker nodes
 397 | #      #
 398 | #      spotFleet:
 399 | #        # Total desired number of units to maintain
 400 | #        # An unit is chosen by you and can be a vCPU, specific amount of memory, size of instance store, etc., according to your requirement.
 401 | #        # Omit or put zero to disable the spot fleet support and use an autoscaling group to manage nodes.
 402 | #        targetCapacity: 10
 403 | #
 404 | #        # IAM role to grant the Spot fleet permission to bid on, launch, and terminate instances on your behalf
 405 | #        # See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-requests.html#spot-fleet-prerequisites
 406 | #        #
 407 | #        # Defaults to "arn:aws:iam::youraccountid:role/aws-ec2-spot-fleet-tagging-role" assuming you've arrived "Spot Requests" in EC2 Dashboard
 408 | #        # hence the role is automatically created for you
 409 | #        iamFleetRoleArn: "arn:aws:iam::youraccountid:role/kube-aws-doesnt-create-this-for-you"
 410 | #
 411 | #        # Price per unit hour = Price per instance hour / num of weighted capacity for the instance
 412 | #        # Defaults to "0.06"
 413 | #        spotPrice: 0.06
 414 | #
 415 | #        # Disk size (GiB) per unit
 416 | #        # Disk size for each launch specification defaults to unitRootVolumeSize * weightedCapacity
 417 | #        unitRootVolumeSize: 30
 418 | #
 419 | #        # Disk type for worker node (one of standard, io1, or gp2)
 420 | #        # Can be overridden in each launch specification
 421 | #        rootVolumeType: gp2
 422 | #
 423 | #        # Number of I/O operations per second (IOPS) per unit. Leave blank if rootVolumeType is not io1
 424 | #        # IOPS for each launch specification defaults to unitRootVolumeIOPS * weightedCapacity
 425 | #        unitRootVolumeIOPS: 0
 426 | #
 427 | #        launchSpecifications:
 428 | #        - # Number of units provided by an EC2 instance of the instanceType.
 429 | #          weightedCapacity: 1
 430 | #          instanceType: c4.large
 431 | #
 432 | #          # Price per instance hour
 433 | #          # Defaults to worker.spotFleet.spotPrice * weightedCapacity if omitted
 434 | #          #spotPrice:
 435 | #
 436 | #          rootVolume:
 437 | #            # Defaults to worker.spotFleet.rootVolumeType if omitted
 438 | #            type:
 439 | #            # Defaults to worker.spotFleet.unitRootVolumeSize * weightedCapacity if omitted
 440 | #            size:
 441 | #            # Number of provisioned IOPS when rootVlumeType is io1
 442 | #            # Must be within the range between 100 and 20000
 443 | #            # Defaults to worker.spotFleet.unitRootVolumeIOPS * weightedCapacity if omitted
 444 | #            iops:
 445 | #
 446 | #        - weightedCapacity: 2
 447 | #          instanceType: c4.xlarge
 448 | #
 449 | #      #
 450 | #      # Optional settings for both ASG-based and SpotFleet-based node pools
 451 | #      #
 452 | #
 453 | #      # Autoscaling by adding/removing nodes according to resource usage
 454 |       autoscaling:
 455 | #        # Make this node pool an autoscaling-target of k8s cluster-autoscaler
 456 | #        #
 457 | #        # Beware that making this an autoscaing-target doesn't automatically deploy cluster-autoscaler itself -
 458 | #        # turn on `addons.clusterAutoscaler.enabled` to deploy it on controller nodes.
 459 |         clusterAutoscaler:
 460 |           enabled: true
 461 | #
 462 | #      # Used to provide `/etc/environment` env vars with values from arbitrary CloudFormation refs
 463 | #      awsEnvironment:
 464 | #        enabled: true
 465 | #        environment:
 466 | #          CFNSTACK: '{ "Ref" : "AWS::StackId" }'
 467 | #
 468 | #      # Add predefined set of labels to the nodes
 469 | #      # The set includes names of launch configurations and autoscaling groups
 470 |       awsNodeLabels:
 471 |         enabled: true
 472 | #
 473 | #      # Will provision worker nodes with IAM permissions to run cluster-autoscaler and add node labels so that
 474 | #      # cluster-autoscaler pods are scheduled to nodes in this node pool
 475 |       clusterAutoscalerSupport:
 476 |         enabled: true
 477 | #
 478 | #      # Mount an EFS only to a specific node pool.
 479 | #      # This is a NFS share that will be available across a node pool through a hostPath volume on the "/efs" mountpoint
 480 | #      #
 481 | #      # If you'd like to configure an EFS mounted to every node, use the top-level `elasticFileSystemId` instead
 482 | #      #
 483 | #      # Beware that currently it doesn't work for a node pool in subnets managed by kube-aws.
 484 | #      # In other words, this node pool must be in existing subnets managed by you by specifying subnets[].id to make this work.
 485 | #      elasticFileSystemId: fs-47a2c22e
 486 | #
 487 | #      # This option has not yet been tested with rkt as container runtime
 488 | #      ephemeralImageStorage:
 489 | #        enabled: true
 490 | #
 491 | #      # When enabled it will grant sts:assumeRole permission to the IAM roles for worker nodes.
 492 | #      # This is intended to be used in combination with managedIamRoleName. See #297 for more information.
 493 |       kube2IamSupport:
 494 |         enabled: true
 495 | #
 496 | #      # Propagate custom CLI options to kubelet
 497 | #      # Provided example overrides default docker image garbage collection limits
 498 | #      # as in https://kubernetes.io/docs/concepts/cluster-administration/kubelet-garbage-collection/
 499 | #      # Anything from https://kubernetes.io/docs/admin/kubelet/ can be used as a value
 500 | #
 501 | #      kubeletOpts: --image-gc-low-threshold=50 --image-gc-high-threshold=65
 502 | #
 503 | #      # Kubernetes node labels to be added to worker nodes
 504 |       nodeLabels:
 505 |         kube-aws.coreos.com/role: worker
 506 | #
 507 | #      # Kubernetes node taints to be added to worker nodes
 508 | #      taints:
 509 | #        - key: dedicated
 510 | #          value: search
 511 | #          effect: NoSchedule
 512 | #
 513 | #      # Other less common customizations per node pool
 514 | #      # All these settings default to the top-level ones
 515 | #      keyName:
 516 | #      releaseChannel: alpha
 517 | #      amiId:
 518 | #      kubernetesVersion: 1.6.0-alpha.1
 519 | #
 520 | #      # Images are taken from controlplane by default, but you can override values for node pools here. E.g.:
 521 | #      AwsCliImage:
 522 | #        repo: quay.io/coreos/awscli
 523 | #        tag: edge
 524 | #        rktPullDocker: false
 525 | #
 526 | #      sshAuthorizedKeys:
 527 | #      # User-provided YAML map available in control-plane's stack-template.json
 528 | #      customSettings:
 529 | #        key1: [ 1, 2, 3 ]
 530 | 
 531 | #      # User defined files that will be added to the NodePool cloud-init configuration in the "write_files:" section.
 532 | #      customFiles:
 533 | #        - path: "/etc/rkt/auth.d/docker.json"
 534 | #          permissions: 0600
 535 | #          content: |
 536 | #            { "rktKind": "dockerAuth", "rktVersion": "v1", ... }
 537 | #
 538 | #      # User defined systemd units that will be added to the NodePool cluster cloud-init configuration in the "units:" section.
 539 | #      customSystemdUnits:
 540 | #        - name: monitoring.service
 541 | #          command: start
 542 | #          enable: true
 543 | #          content: |
 544 | #            [Unit]
 545 | #            Description=Example Custom Service
 546 | #            [Service]
 547 | #            ExecStart=/bin/rkt run --set-env TAGS=Controller ...
 548 | #      # Enable PodPriority (it only makes sense if you enabled priority admission control in experimental section)
 549 | #      featureGates:
 550 | #        PodPriority: true
 551 | 
 552 | # Maximum time to wait for worker creation
 553 | #workerCreateTimeout: PT15M
 554 | 
 555 | # Instance type for worker nodes
 556 | # CAUTION: Don't use t2.micro or the cluster won't work. See https://github.com/kubernetes/kubernetes/issues/16122
 557 | #workerInstanceType: t2.medium
 558 | 
 559 | # Disk size (GiB) for worker nodes
 560 | #workerRootVolumeSize: 30
 561 | 
 562 | # Disk type for worker node (one of standard, io1, or gp2)
 563 | #workerRootVolumeType: gp2
 564 | 
 565 | # Number of I/O operations per second (IOPS) that the worker node disk supports. Leave blank if workerRootVolumeType is not io1
 566 | #workerRootVolumeIOPS: 0
 567 | 
 568 | # Tenancy of the worker nodes. Options are "default" and "dedicated"
 569 | # Documentation: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/dedicated-instance.html
 570 | # workerTenancy: default
 571 | 
 572 | # Price (Dollars) to bid for spot instances. Omit for on-demand instances.
 573 | # workerSpotPrice: "0.05"
 574 | 
 575 | ## Etcd Cluster config
 576 | ## WARNING: Any changes to etcd parameters after the cluster is first created will not be applied
 577 | ## during a cluster upgrade, due to concerns over data loss.
 578 | ## This situation is being rectified with work towards automated management of etcd clusters
 579 | 
 580 | etcd:
 581 | #  # Number of etcd nodes
 582 | #  # (Set to an odd number >= 3 for HA control plane)
 583 | #  # WARNING: You can't add etcd nodes after the first creation by modifying this and running `kube-aws update`
 584 | #  # See https://github.com/kubernetes-incubator/kube-aws/issues/631 for more information
 585 |   count: 1
 586 | #
 587 | #  # Instance type for etcd node
 588 |   instanceType: t2.medium
 589 | #
 590 | #  # EC2 instance tags for etcd nodes
 591 |   instanceTags:
 592 |     Role: etcd
 593 |     SubRole: Development
 594 |     Application: Kubernetes
 595 | #
 596 |   rootVolume:
 597 | #    # Root volume size (GiB) for etcd node
 598 |     size: 30
 599 | #    # Root volume type for etcd node (one of standard, io1, or gp2)
 600 |     type: gp2
 601 | #    # Number of I/O operations per second (IOPS) that the etcd node's root volume supports. Leave blank if etcdRootVolumeType is not io1
 602 | #    iops: 0
 603 | #
 604 |   dataVolume:
 605 | #    # Data volume size (GiB) for etcd node
 606 | #    # if etcdDataVolumeEphemeral=true, this value is ignored. The size of ephemeral volumes is not configurable.
 607 |     size: 30
 608 | #    # Data volume type for etcd node (one of standard, io1, or gp2)
 609 |     type: gp2
 610 | #    # Number of I/O operations per second (IOPS) that the etcd node's data volume supports. Leave blank if etcdDataVolumeType is not io1
 611 | #    iops: 0
 612 | #    # Encrypt the Etcd Data volume.  Set to true for encryption.  This does not work for etcdDataVolumeEphemeral.
 613 |     encrypted: true
 614 | #    # Use ephemeral instance storage for etcd data volume instead of EBS?
 615 | #    # (Recommended set to true for high-throughput control planes)
 616 | #    # CAUTION: Currently broken. Please don't turn this on until it is fixed in https://github.com/kubernetes-incubator/kube-aws/pull/417
 617 | #    ephemeral: false
 618 | #
 619 | #  # Tenancy of the etcd instances. Options are "default" and "dedicated"
 620 | #  # Documentation: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/dedicated-instance.html
 621 | #  tenancy: default
 622 | #
 623 | #  # If omitted, public subnets are created by kube-aws and used for etcd nodes
 624 |   subnets:
 625 | #    # References subnets defined under the top-level `subnets` key by their names
 626 |   - name: PrivateSubnetA
 627 |   - name: PrivateSubnetB
 628 |   - name: PrivateSubnetC
 629 | #
 630 | #  # Existing security groups attached to etcd nodes which are typically used to
 631 | #  # allow ssh accesses from bastions when `sshAccessAllowedSourceCIDRs` are explicitly set to an empty array
 632 | #  securityGroupIds:
 633 | #    - sg-1234abcd
 634 | #    - sg-5678efab
 635 | #
 636 | #  # If you omit this block kube-aws would create an IAM Role and a managed policy for etcd nodes with a random name.
 637 | #  iam:
 638 | #    role:
 639 | #      # If you set managedPolicies here they will be attached to the role in addition to the managed policy.
 640 | #      # CAUTION: if you attach a more restrictive policy in some resources (i.e ec2:* Deny) you can make kube-aws fail.
 641 | #      managedPolicies:
 642 | #      - arn: "arn:aws:iam::aws:policy/AdministratorAccess"
 643 | #      - arn: "arn:aws:iam::YOURACCOUNTID:policy/YOURPOLICYNAME"
 644 | #    # If you set an InstanceProfile kube-aws will NOT create any IAM Role and will use the existing instance profile.
 645 | #    # CAUTION: you must ensure that the IAM role linked to the existing instance profile has enough permissions to ensure etcd nodes to run.
 646 | #    # If you dont know which permissions are required, it is recommended to go ahead with a managed instance profile.
 647 | #    instanceProfile:
 648 | #      arn: "arn:aws:iam::YOURACCOUNTID:instance-profile/INSTANCEPROFILENAME"
 649 | #
 650 | #  # The version of etcd to be used. Set to e.g. "3.2.1" to use etcd3
 651 |   version: 3.2.13
 652 | #
 653 |   snapshot:
 654 | #    # Set to true to periodically take an etcd snapshot
 655 | #    # Beware that this can be enabled only for etcd 3+
 656 | #    # Please carefully test if it works as you've expected when being enabled for your production clusters
 657 |     automated: true
 658 | #
 659 |   disasterRecovery:
 660 | #    # Set to true to automatically execute a disaster-recovery process whenever etcd node(s) seemed to be broken for a while
 661 | #    # Beware that this can be enabled only for etcd 3+
 662 | #    # Please carefully test if it works as you've expected when being enabled for your production clusters
 663 |     automated: false
 664 | #
 665 | #  # The strategy to provide your etcd nodes, in combination with floating EBS volumes, stable member identities. Defaults to "eip".
 666 | #  #
 667 | #  # Available options: eip, eni
 668 | #  #
 669 | #  # With every option, etcd nodes communicate to each other via their private IPs.
 670 | #  #
 671 | #  # eip: Use EC2 public hostnames stabilized with EIPs and "resolved eventually to private IPs"
 672 | #  #      Requires Amazon DNS (at the second IP of your VPC, e.g. 10.0.0.2) to work.
 673 | #  #      1st recommendation because less moving parts and relatively easy disaster recovery for a single-AZ etcd cluster.
 674 | #  #      If you run a single-AZ etcd cluster and the AZ failed, EBS volumes created from latest snapshots and EIPs can be reused in an another AZ to reproduce your etcd cluster in the AZ.
 675 | #  #
 676 | #  # eni: [EXPERIMENTAL] Use secondary ENIs and Route53 record sets to provide etcd nodes stable hostnames
 677 | #  #      Requires Amazon DNS (at the second IP of your VPC, e.g. 10.0.0.2) or an another DNS which can resolve dns names in the hosted zone managed by kube-aws to work.
 678 | #  #      2nd recommendation because relatively easy disaster recovery for a single-AZ etcd cluster but more moving parts than "eip".
 679 | #  #      If you run a single-AZ etcd cluster and the AZ failed, EBS volumes created from latest snapshots and record sets can be reused in an another AZ to reproduce your etcd cluster in the AZ.
 680 | #  #
 681 | #  memberIdentityProvider: eip
 682 | #
 683 | #  # Domain of the hostname used for etcd peer discovery.
 684 | #  # Used only when `memberIdentityProvider: eni` for TLS key/cert generation
 685 | #  # If omitted, defaults to "ec2.internal" for us-east-1 region and "<region>.compute.internal" otherwise
 686 | #  internalDomainName:
 687 | #
 688 | #  # Set to `false` to disable creation of record sets.
 689 | #  # Used only when `memberIdentityProvider` is set to `eni`
 690 | #  # When disabled, it's your responsibility to configure all the etcd nodes so that
 691 | #  # they can resolve each other's FQDN(specified via the below `etcd.nods[].fqdn` settings) via your DNS(can be the Amazon DNS or your own DNS. Configure it with e.g. coroes-cloudinit)
 692 | #  manageRecordSets:
 693 | #
 694 | #  # Advanced configuration used only when `memberIdentityProvider: eni`
 695 | #  hostedZone:
 696 | #    # The hosted zone where record sets for etcd nodes managed by kube-aws are created
 697 | #    # If omitted, kube-aws creates a hosted zone for you
 698 | #    id:
 699 | #
 700 | #  # CAUTION: Advanced configuration. This should be omitted unless you have very deep knowledge of etcd and kube-aws
 701 | #  nodes:
 702 | #  - # The name of this etcd node. Specified only when you want to customize the etcd member's name shown in ETCD_INITIAL_CLUSTER and ETCD_NAME
 703 | #    name: etcd0
 704 | #    # The FQDN of this etcd node
 705 | #    # Usually this should be omitted so that kube-aws can choose a proper value.
 706 | #    # Specified only when `memberIdentityProvider: eni` and `manageRecordSets: false` i.e.
 707 | #    # it is your responsibility to properly configure EC2 instances to use a DNS which is able to resolve the FQDN.
 708 | #    fqdn: etcd0.<internalDomainName>
 709 | #  - name: etcd1
 710 | #    fqdn: etcd1.<internalDomainName>
 711 | #  - name: etcd2
 712 | #    fqdn: etcd2.<internalDomainName>
 713 | #
 714 | #  # ARN of the KMS key used to encrypt the etcd data volume. The account default key will be used if `etcdDataVolumeEncrypted`
 715 | #  # is enable and `etcd.kmsKeyArn` is omitted.
 716 | #  kmsKeyArn: ""
 717 | #
 718 | #  # User defined files that will be added to the Etcd cluster cloud-init configuration in the "write_files:" section.
 719 | #  customFiles:
 720 | #    - path: "/etc/rkt/auth.d/docker.json"
 721 | #      permissions: 0600
 722 | #      content: |
 723 | #        { "rktKind": "dockerAuth", "rktVersion": "v1", ... }
 724 | #
 725 | #  # User defined systemd units that will be added to the Etcd cluster cloud-init configuration in the "units:" section.
 726 | #  customSystemdUnits:
 727 | #    - name: monitoring.service
 728 | #      command: start
 729 | #      enable: true
 730 | #      content: |
 731 | #        [Unit]
 732 | #        Description=Example Custom Service
 733 | #        [Service]
 734 | #        ExecStart=/bin/rkt run --set-env TAGS=Controller ...
 735 | 
 736 | ## Networking config
 737 | 
 738 | # CAUTION: Deprecated and will be removed in v0.9.9. Please use vpc.id instead
 739 | # vpcId:
 740 | 
 741 | vpc:
 742 | #  # ID of existing VPC to create subnet in. Leave blank to create a new VPC
 743 |   id: vpc_id
 744 | #  # Exported output's name from another stack
 745 | #  # Only specify either id or idFromStackOutput but not both
 746 | #  #idFromStackOutput: myinfra-Vpc
 747 | 
 748 | # CAUTION: Deprecated and will be removed in v0.9.9. Please use internetGateway.id instead
 749 | # internetGatewayId:
 750 | 
 751 | #internetGateway:
 752 | #  # ID of existing Internet Gateway to associate subnet with. Leave blank to create a new Internet Gateway
 753 | #  id:
 754 | #  # Exported output's name from another stack
 755 | #  # Only specify either id or idFromStackOutput but not both
 756 | #  #idFromStackOutput: myinfra-igw
 757 | 
 758 | # Advanced: ID of existing route table in existing VPC to attach subnet to.
 759 | # Leave blank to use the VPC's main route table.
 760 | # This should be specified if and only if vpcId is specified.
 761 | #
 762 | # IMPORTANT NOTICE:
 763 | #
 764 | # If routeTableId is specified, it's your responsibility to add an appropriate route to
 765 | # an internet gateway(IGW) or a NAT gateway(NGW) to the route table.
 766 | #
 767 | # More concretely,
 768 | # * If you like to make all the subnets private, pre-configure an NGW yourself and add a route to the NGW beforehand
 769 | # * If you like to make all the subnets public, pre-configure an IGW yourself and add a route to the IGW beforehand
 770 | # * If you like to mix private and public subnets, omit routeTableId but specify subnets[].routeTable.id per subnet
 771 | #
 772 |   routeTableId: route_table_id
 773 | 
 774 | # CIDR for Kubernetes VPC. If vpcId is specified, must match the CIDR of existing vpc.
 775 |   vpcCIDR: "vpc_cidr"
 776 | 
 777 | # CIDR for Kubernetes subnet when placing nodes in a single availability zone (not highly-available) Leave commented out for multi availability zone setting and use the below `subnets` section instead.
 778 | # instanceCIDR: "10.0.0.0/24"
 779 | 
 780 | # Kubernetes subnets with their CIDRs and availability zones.
 781 | # Differentiating availability zone for 2 or more subnets result in high-availability (failures of a single availability zone won't result in immediate downtimes)
 782 | subnets:
 783 | - name: PrivateSubnetA
 784 |   availabilityZone: aws_regiona
 785 |   id: "private_subnet_a"
 786 | 
 787 | - name: PrivateSubnetB
 788 |   availabilityZone: aws_regionb
 789 |   id: "private_subnet_b"
 790 | 
 791 | - name: PrivateSubnetC
 792 |   availabilityZone: aws_regionc
 793 |   id: "private_subnet_c"
 794 | #   #
 795 | #   # Managed public subnet managed by kube-aws
 796 | #   #
 797 | #   - name: ManagedPublicSubnet1
 798 | #     # Set to false if this subnet is public
 799 | #     # private: false
 800 | #     availabilityZone: us-west-1a
 801 | #     instanceCIDR: "10.0.0.0/24"
 802 | #
 803 | #   #
 804 | #   # Managed private subnet managed by kube-aws
 805 | #   #
 806 | #   - name: ManagedPrivateSubnet1
 807 | #     # Set to true if this subnet is private
 808 | #     private: true
 809 | #     availabilityZone: us-west-1a
 810 | #     instanceCIDR: "10.0.1.0/24"
 811 | #
 812 | #   #
 813 | #   # Advanced: Unmanaged/existing public subnet reused but not managed by kube-aws
 814 | #   #
 815 | #   # An internet gateway(igw) and a route table contains the route to the igw must have been properly configured by YOU.
 816 | #   # kube-aws tries to reuse the subnet specified by id or idFromStackOutput but kube-aws never modify the subnet
 817 | #   #
 818 | #   - name: ExistingPublicSubnet1
 819 | #     # Beware that `availabilityZone` can't be omitted; it must be the one in which the subnet exists.
 820 | #     availabilityZone: us-west-1a
 821 | #     # ID of existing subnet to be reused.
 822 | #     # availabilityZone should still be provided but instanceCIDR can be omitted when id is specified.
 823 | #     id: "subnet-xxxxxxxx"
 824 | #     # Exported output's name from another stack
 825 | #     # Only specify either id or idFromStackOutput but not both
 826 | #     #idFromStackOutput: myinfra-PublicSubnet1
 827 | #
 828 | #   #
 829 | #   # Advanced: Unmanaged/existing private subnet reused but not managed by kube-aws
 830 | #   #
 831 | #   # A nat gateway(ngw) and a route table contains the route to the ngw must have been properly configured by YOU.
 832 | #   # kube-aws tries to reuse the subnet specified by id or idFromStackOutput but kube-aws never modify the subnet
 833 | #   #
 834 | #   - name: ExistingPrivateSubnet1
 835 | #     # Beware that `availabilityZone` can't be omitted; it must be the one in which the subnet exists.
 836 | #     availabilityZone: us-west-1a
 837 | #     # Existing subnet.
 838 | #     id: "subnet-xxxxxxxx"
 839 | #     # Exported output's name from another stack
 840 | #     # Only specify either id or idFromStackOutput but not both
 841 | #     #idFromStackOutput: myinfra-PrivateSubnet1
 842 | #
 843 | #   #
 844 | #   # Advanced: Managed private subnet with an existing NAT gateway
 845 | #   #
 846 | #   # kube-aws tries to reuse the ngw specified by id or idFromStackOutput
 847 | #   # by adding a route to the ngw to a route table managed by kube-aws
 848 | #   #
 849 | #   # Please be sure that the NGW is properly deployed. kube-aws will never modify ngw itself.
 850 | #   #
 851 | #   - name: ManagedPrivateSubnetWithExistingNGW
 852 | #     private: true
 853 | #     availabilityZone: us-west-1a
 854 | #     instanceCIDR: "10.0.1.0/24"
 855 | #     natGateway:
 856 | #       id: "ngw-xxxxxxxx"
 857 | #       # Exported output's name from another stack
 858 | #       # Only specify either id or idFromStackOutput but not both
 859 | #       #idFromStackOutput: myinfra-PrivateSubnet1
 860 | #
 861 | #   #
 862 | #   # Advanced: Managed private subnet with an existing NAT gateway
 863 | #   #
 864 | #   # kube-aws tries to reuse the ngw specified by id or idFromStackOutput
 865 | #   # by adding a route to the ngw to a route table managed by kube-aws
 866 | #   #
 867 | #   # Please be sure that the NGW is properly deployed. kube-aws will never modify ngw itself.
 868 | #   # For example, kube-aws won't assign a pre-allocated EIP to the existing ngw for you.
 869 | #   #
 870 | #   - name: ManagedPrivateSubnetWithExistingNGW
 871 | #     private: true
 872 | #     availabilityZone: us-west-1a
 873 | #     instanceCIDR: "10.0.1.0/24"
 874 | #     natGateway:
 875 | #       # Pre-allocated NAT Gateway. Used with private subnets.
 876 | #       id: "ngw-xxxxxxxx"
 877 | #       # Exported output's name from another stack
 878 | #       # Only specify either id or idFromStackOutput but not both
 879 | #       #idFromStackOutput: myinfra-PrivateSubnet1
 880 | #
 881 | #   #
 882 | #   # Advanced: Managed private subnet with an existing EIP for kube-aws managed NGW
 883 | #   #
 884 | #   # kube-aws tries to reuse the EIP specified by eipAllocationId
 885 | #   # by associating the EIP to a NGW managed by kube-aws.
 886 | #   # Please be sure that kube-aws won't assign an EIP to an existing NGW i.e.
 887 | #   # either natGateway.id or eipAllocationId can be specified but not both.
 888 | #   #
 889 | #   - name: ManagedPrivateSubnetWithManagedNGWWithExistingEIP
 890 | #     private: true
 891 | #     availabilityZone: us-west-1a
 892 | #     instanceCIDR: "10.0.1.0/24"
 893 | #     natGateway:
 894 | #       # Pre-allocated EIP for NAT Gateways. Used with private subnets.
 895 | #       eipAllocationId: eipalloc-xxxxxxxx
 896 | #
 897 | #   #
 898 | #   # Advanced: Managed private subnet with an existing route table
 899 | #   #
 900 | #   # kube-aws tries to reuse the route table specified by id or idFromStackOutput
 901 | #   # by assigning this subnet to the route table.
 902 | #   #
 903 | #   # Please be sure that it's your responsibility to:
 904 | #   # * Configure an AWS managed NAT or a NAT instance or an another NAT and
 905 | #   # * Add a route to the NAT to the route table being reused
 906 | #   #
 907 | #   # i.e. kube-aws neither modify route table nor create other related resources like
 908 | #   # ngw, route to nat gateway, eip for ngw, etc.
 909 | #   #
 910 | #   - name: ManagedPrivateSubnetWithExistingRouteTable
 911 | #     private: true
 912 | #     availabilityZone: us-west-1a
 913 | #     instanceCIDR: "10.0.1.0/24"
 914 | #     routeTable:
 915 | #       # Pre-allocated route table
 916 | #       id: "rtb-xxxxxxxx"
 917 | #       # Exported output's name from another stack
 918 | #       # Only specify either id or idFromStackOutput but not both
 919 | #       #idFromStackOutput: myinfra-PrivateRouteTable1
 920 | #
 921 | #   #
 922 | #   # Advanced: Managed public subnet with an existing route table
 923 | #   #
 924 | #   # kube-aws tries to reuse the route table specified by id or idFromStackOutput
 925 | #   # by assigning this subnet to the route table.
 926 | #   #
 927 | #   # Please be sure that it's your responsibility to:
 928 | #   # * Configure an internet gateway(IGW) and
 929 | #   # * Attach the IGW to the VPC you're deploying to
 930 | #   # * Add a route to the IGW to the route table being reused
 931 | #   #
 932 | #   # i.e. kube-aws neither modify route table nor create other related resources like
 933 | #   # igw, route to igw, igw vpc attachment, etc.
 934 | #   #
 935 | #   - name: ManagedPublicSubnetWithExistingRouteTable
 936 | #     availabilityZone: us-west-1a
 937 | #     instanceCIDR: "10.0.1.0/24"
 938 | #     routeTable:
 939 | #       # Pre-allocated route table
 940 | #       id: "rtb-xxxxxxxx"
 941 | #       # Exported output's name from another stack
 942 | #       # Only specify either id or idFromStackOutput but not both
 943 | #       #idFromStackOutput: myinfra-PublicRouteTable1
 944 | 
 945 | 
 946 | # CIDR for all service IP addresses
 947 | # serviceCIDR: "10.3.0.0/24"
 948 | 
 949 | # CIDR for all pod IP addresses
 950 | # podCIDR: "10.2.0.0/16"
 951 | 
 952 | # IP address of Kubernetes dns service (must be contained by serviceCIDR)
 953 | # dnsServiceIP: 10.3.0.10
 954 | 
 955 | # Uncomment to provision nodes without a public IP. This assumes your VPC route table is setup to route to the internet via a NAT gateway.
 956 | # If you did not set vpcId and routeTableId the cluster will not bootstrap.
 957 | # mapPublicIPs: false
 958 | 
 959 | # Expiration in days from creation time of TLS assets. By default, the CA will
 960 | # expire in 10 years and the server and client certificates will expire in 1
 961 | # year.
 962 | tlsCADurationDays: 3650
 963 | tlsCertDurationDays: 365
 964 | 
 965 | # Use custom images for kube-aws  and  kubernetes  components. Especially if you are deploying in cn-north-1 where gcr.io is blocked
 966 | # and pulling from quay or dockerhub is slow and you get many timeouts.
 967 | 
 968 | # Version of hyperkube image to use. This is the tag for the hyperkube image repository.
 969 | kubernetesVersion: v1.9.1
 970 | 
 971 | # Hyperkube image repository to use.
 972 | # hyperkubeImage:
 973 | #   repo: k8s.gcr.io/hyperkube-amd64
 974 | #   rktPullDocker: true
 975 | 
 976 | # AWS CLI image repository to use.
 977 | # awsCliImage:
 978 | #   repo: quay.io/coreos/awscli
 979 | #   tag: master
 980 | #   rktPullDocker: false
 981 | 
 982 | # Calico Node image repository to use.
 983 | #calicoNodeImage:
 984 | #  repo: quay.io/calico/node
 985 | #  tag: v2.6.5
 986 | #  rktPullDocker: false
 987 | 
 988 | # Calico CNI image repository to use.
 989 | #calicoCniImage:
 990 | #  repo: quay.io/calico/cni
 991 | #  tag: v1.11.2
 992 | #  rktPullDocker: false
 993 | 
 994 | # Calico Kube Controllers image repository to use.
 995 | #calicoKubeControllersImage:
 996 | #  repo: quay.io/calico/kube-controllers
 997 | #  tag: v1.0.2
 998 | #  rktPullDocker: false
 999 | 
1000 | # Cluster Autoscaler image repository to use.
1001 | #clusterAutoscalerImage:
1002 | #  repo: k8s.gcr.io/cluster-autoscaler
1003 | #  tag: v1.1.0
1004 | #  rktPullDocker: false
1005 | 
1006 | # Cluster Proportional Autoscaler image repository to use.
1007 | #clusterProportionalAutoscalerImage:
1008 | #  repo: k8s.gcr.io/cluster-proportional-autoscaler-amd64
1009 | #  tag: 1.1.2
1010 | #  rktPullDocker: false
1011 | 
1012 | # kube2iam image repository to use.
1013 | #kube2iamImage:
1014 | #  repo: jtblin/kube2iam
1015 | #  tag: 0.9.0
1016 | #  rktPullDocker: false
1017 | 
1018 | # kube DNS image repository to use.
1019 | #kubeDnsImage:
1020 | #  repo: k8s.gcr.io/k8s-dns-kube-dns-amd64
1021 | #  tag: 1.14.7
1022 | #  rktPullDocker: false
1023 | 
1024 | # kube DNS Masq image repository to use.
1025 | #kubeDnsMasqImage:
1026 | #  repo: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64
1027 | #  tag: 1.14.7
1028 | #  rktPullDocker: false
1029 | 
1030 | # kube rescheduler image repository to use.
1031 | #kubeReschedulerImage:
1032 | #  repo: k8s.gcr.io/rescheduler-amd64
1033 | #  tag: v0.3.2
1034 | #  rktPullDocker: false
1035 | 
1036 | # DNS Masq metrics image repository to use.
1037 | #dnsMasqMetricsImage:
1038 | #  repo: k8s.gcr.io/k8s-dns-sidecar-amd64
1039 | #  tag: 1.14.7
1040 | #  rktPullDocker: false
1041 | 
1042 | # Exec Healthz image repository to use.
1043 | #execHealthzImage:
1044 | #  repo: k8s.gcr.io/exechealthz-amd64
1045 | #  tag: 1.2
1046 | #  rktPullDocker: false
1047 | 
1048 | # Helm image repository to use.
1049 | #helmImage:
1050 | #  repo: quay.io/kube-aws/helm
1051 | #  tag: v2.6.0
1052 | #  rktPullDocker: false
1053 | 
1054 | # Tiller (Helm backend) image repository to use.
1055 | #tillerImage:
1056 | #  repo: gcr.io/kubernetes-helm/tiller
1057 | #  tag: v2.7.2
1058 | #  rktPullDocker: false
1059 | 
1060 | # Heapster image repository to use.
1061 | #heapsterImage:
1062 | #  repo: k8s.gcr.io/heapster
1063 | #  tag: v1.5.0
1064 | #  rktPullDocker: false
1065 | 
1066 | # Metrics Server image repository to use.
1067 | #metricsServerImage:
1068 | #  repo: k8s.gcr.io/metrics-server-amd64
1069 | #  tag: v0.2.1
1070 | #  rktPullDocker: false
1071 | 
1072 | # Addon Resizer image repository to use.
1073 | #addonResizerImage:
1074 | #  repo: k8s.gcr.io/addon-resizer
1075 | #  tag: 1.8.1
1076 | #  rktPullDocker: false
1077 | 
1078 | # Kube Dashboard image repository to use.
1079 | #kubeDashboardImage:
1080 | #  repo: k8s.gcr.io/kubernetes-dashboard-amd64
1081 | #  tag: v1.8.1
1082 | #  rktPullDocker: false
1083 | 
1084 | # Calico Controller image repository to use.
1085 | #calicoCtlImage:
1086 | #  repo: calico/ctl
1087 | #  tag: v1.6.3
1088 | #  rktPullDocker: false
1089 | 
1090 | # Pause image repository to use.This works only if you are deploying your cluster in "cn-north-1" region.
1091 | #pauseImage:
1092 | #  repo: k8s.gcr.io/pause-amd64
1093 | #  tag: 3.1
1094 | #  rktPullDocker: false
1095 | 
1096 | # Flannel image repository to use.This allow pulling flannel image from a docker repo.
1097 | #flannelImage:
1098 | #  repo: quay.io/coreos/flannel
1099 | #  tag: v0.9.1
1100 | #  rktPullDocker: false
1101 | 
1102 | # JournaldCloudWatchLogsImage image repository to use. This sends journald logs to CloudWatch.
1103 | #journaldCloudWatchLogsImage:
1104 | #  repo: "jollinshead/journald-cloudwatch-logs"
1105 | #  tag: "0.1"
1106 | #  rktPullDocker: true
1107 | 
1108 | # Use Calico for network policy.
1109 | # useCalico: false
1110 | 
1111 | # Create MountTargets to subnets managed by kube-aws for a pre-existing Elastic File System (Amazon EFS),
1112 | # and then mount to every node.
1113 | #
1114 | # This is for mounting an EFS to every node.
1115 | # If you'd like to mount an EFS only to a specific node pool, set `worker.nodePools[].elasticFileSystemId` instead.
1116 | #
1117 | # Enter the resource id, eg "fs-47a2c22e"
1118 | # This is a NFS share that will be available across the entire cluster through a hostPath volume on the "/efs" mountpoint
1119 | #
1120 | # You can create a new EFS volume using the CLI:
1121 | # $ aws efs create-file-system --creation-token $(uuidgen)
1122 | #
1123 | # Beware that:
1124 | # * an EFS file system can not be mounted from multiple subnets from the same availability zone and
1125 | #   therefore this feature won't work like you might have expected when you're deploying your cluster to an existing VPC and
1126 | #   wanted to mount an existing EFS to the subnet(s) created by kube-aws
1127 | #   See https://github.com/kubernetes-incubator/kube-aws/issues/208 for more information
1128 | # * kube-aws doesn't create MountTargets for existing subnets(=NOT managed by kube-aws). If you'd like to bring your own subnets
1129 | #   to be used by kube-aws, it is now your responsibility to configure the subnets, including creating MountTargets, accordingly
1130 | #   to your requirements
1131 | #elasticFileSystemId: fs-47a2c22e
1132 | 
1133 | # Create shared persistent volume
1134 | #sharedPersistentVolume: false
1135 | 
1136 | # Determines the container runtime for kubernetes to use. Accepts 'docker' or 'rkt'.
1137 | # containerRuntime: docker
1138 | 
1139 | # If you do not want kube-aws to manage certificaes, set it to false. If you do that
1140 | # you are responsible for making sure that nodes have correct certificates by the time
1141 | # daemons start up.
1142 | # manageCertificates: true
1143 | 
1144 | # When enabled, autoscaling groups managing controller nodes wait for nodes to be up and running.
1145 | # It is enabled by default.
1146 | #waitSignal:
1147 | #  enabled: true
1148 | #   maxBatchSize: 1
1149 | 
1150 | # Autosaves all Kubernetes resources (in .json format) to a bucket 's3:.../<your-cluster-name>/backup/*'.
1151 | # The autosave process executes on start-up and repeats every 24 hours.
1152 | kubeResourcesAutosave:
1153 |   enabled: true
1154 | 
1155 | # Accessing Dashboard:
1156 | # By default, the kubernetes dashboard has admin privileges and doesn't require authentication.
1157 | # Set 'adminPrivileges: false' to restrict access using a token or kubeconfig file.
1158 | # An example of how to generate a token is provided in the documentation: https://kubernetes-incubator.github.io/kube-aws/advanced-topics/kubernetes-dashboard.html
1159 | 
1160 | # Access the dashboard  using 'kubectl proxy:
1161 | # insecureLogin: false ==> "http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/"
1162 | # insecureLogin: true ==>  "http://localhost:8001/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/"
1163 | # Expose the dashboard
1164 | # Please check the documentation: https://kubernetes-incubator.github.io/kube-aws/advanced-topics/kubernetes-dashboard.html
1165 | 
1166 | kubernetesDashboard:
1167 |   adminPrivileges: true
1168 |   insecureLogin: false
1169 | 
1170 | # When enabled, all nodes will forward journald logs to AWS CloudWatch.
1171 | # It is disabled by default.
1172 | cloudWatchLogging:
1173 |  enabled: true
1174 |  retentionInDays: 7
1175 | # # When enabled, feedback from Journald logs (with an applied filter) will be outputted during kube-aws 'apply | up'.
1176 | # # It is enabled by default - provided cloudWatchLogging is enabled.
1177 |  localStreaming:
1178 |   enabled: true
1179 | #  filter:  `{ $.priority = "CRIT" || $.priority = "WARNING" && $.transport = "journal" && $.systemdUnit = "init.scope" }`
1180 | #  interval: 60
1181 | 
1182 | # When enabled, all nodes will have Amazon SSM Agent installed.
1183 | # It is disabled by default.
1184 | # You might want to set 'managedPolicies' up to run the agents properly. More details: http://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-configuring-access-policies.html
1185 | #amazonSsmAgent:
1186 | #  enabled: true
1187 | #  downloadUrl: https://github.com/DailyHotel/amazon-ssm-agent/releases/download/v2.0.805.1/ssm.linux-amd64.tar.gz
1188 | #  sha1sum: a6fff8f9839d5905934e7e3df3123b54020a1f5e
1189 | 
1190 | # When enabled, will enable a DNS-masq DaemonSet to make PODs to resolve DNS names via locally running dnsmasq
1191 | # It is disabled by default.
1192 | #kubeDns:
1193 | # nodeLocalResolver: false
1194 | 
1195 | kubeProxy:
1196 |   # Use IPVS kube-proxy mode instead of [default] iptables one (requires Kubernetes 1.9.0+ to work reliably)
1197 |   # This is intended to address performance issues of iptables mode for clusters with big number of nodes and services
1198 |   # FIXME For those who use hyperkube version 'v1.9.0' / 'v1.9.0_coreos.0', your image may lack `ipset` utility
1199 |   # FIXME Please see: https://github.com/kubernetes/kubernetes/issues/57321 (next Kubernetes release will have a fix)
1200 |   # FIXME https://github.com/kubernetes/kubernetes/commit/787a55bb67ccd2da14aa6e7f91289c859beecb5f#diff-bf0f8d724d18f700f3c821aa5a74f4cf
1201 |   # FIXME IPVS integration is still green, proceed with care! You may get fixed hyperkube image from 'ivanilves/hyperkube' Docker repo
1202 |   ipvsMode:
1203 |     enabled: true
1204 |     scheduler: rr
1205 |     syncPeriod: 300s
1206 |     minSyncPeriod: 60s
1207 | 
1208 | # When enabled, CloudFormation events will stream to stdout during kube-aws 'update | up'.
1209 | # It is enabled by default.
1210 | cloudFormationStreaming: true
1211 | 
1212 | # Addon features
1213 | addons:
1214 |   # Will provision controller nodes with IAM permissions to run cluster-autoscaler and
1215 |   # create a cluster-autoscaler deployment in the cluster
1216 |   # CA runs only on controller nodes by default. Turn on `worker.nodePools[].clusterAutoscalerSupport.enabled` to
1217 |   # add nodes from a node pool to be where CA runs
1218 |   clusterAutoscaler:
1219 |     enabled: true
1220 | 
1221 |   # When enabled, Kubernetes rescheduler is deployed to the cluster controller(s)
1222 |   # This feature is experimental currently so may not be production ready
1223 |   rescheduler:
1224 |     enabled: true
1225 | 
1226 |   # Metrics Server (https://github.com/kubernetes-incubator/metrics-server)
1227 |   metricsServer:
1228 |     enabled: true
1229 | 
1230 |   # When set to true this configures security groups for prometheus between nodes.
1231 |   # This includes the following ports: 10252, 10251, 10250, 9100, and 4194
1232 |   prometheus:
1233 |     securityGroupsEnabled: true
1234 | 
1235 | # Experimental features will change in backward-incompatible ways
1236 | experimental:
1237 |   # Enable admission controllers
1238 |   admission:
1239 |     podSecurityPolicy:
1240 |       enabled: false
1241 |     # alwaysPullImages Note
1242 |     # Recommended to turn this on when and only when docker registries are reliable enough.
1243 |     # Otherwise, automatic pod restarts can be delayed due to temporally docker registry outages.
1244 |     # Please see https://github.com/kubernetes-incubator/kube-aws/pull/1009#discussion_r151197787 for more info.
1245 |     alwaysPullImages:
1246 |       enabled: false
1247 |     denyEscalatingExec:
1248 |       enabled: false
1249 |     initializers:
1250 |       enabled: true
1251 |     # Priority enables PodPriority in the API server, scheduler and kubelet. you need to manually add the
1252 |     # featureGate PodPriority:true in the worker
1253 |     priority:
1254 |       enabled: false
1255 | 
1256 |   # Used to provide `/etc/environment` env vars with values from arbitrary CloudFormation refs
1257 |   awsEnvironment:
1258 |     enabled: false
1259 |     environment:
1260 |       CFNSTACK: '{ "Ref" : "AWS::StackId" }'
1261 | 
1262 |   # Enable audit log for apiserver. Recommended when `rbac` is enabled.
1263 |   auditLog:
1264 |     enabled: false
1265 |     maxage: 30
1266 |     logpath: /dev/stdout
1267 | 
1268 |   # See https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication for more information
1269 |   authentication:
1270 |     webhook:
1271 |       enabled: false
1272 |       cacheTTL: 1m0s
1273 |       configBase64: base64-encoded-webhook-yaml
1274 | 
1275 |   # Add predefined set of labels to the controller nodes
1276 |   # The set includes names of launch configurations and autoscaling groups
1277 |   awsNodeLabels:
1278 |     enabled: true
1279 | 
1280 |   # If enabled, instructs the controller manager to automatically issue TLS certificates to worker nodes via
1281 |   # certificate signing requests (csr) made to the API server using the bootstrap token. It's recommended to
1282 |   # also enable the rbac plugin in order to limit requests using the bootstrap token to only be able to make
1283 |   # requests related to certificate provisioning.
1284 |   # The bootstrap token is automatically generated in ./credentials/kubelet-tls-bootstrap-token.
1285 |   tlsBootstrap:
1286 |     enabled: true
1287 | 
1288 |   # Enables the Node authorization + node restriction admission controller (requires Kubernetes 1.7.4+)
1289 |   # Requires TLS bootstrapping to be enabled as well
1290 |   nodeAuthorizer:
1291 |     enabled: true
1292 | 
1293 |   # This option has not yet been tested with rkt as container runtime
1294 |   ephemeralImageStorage:
1295 |     enabled: false
1296 | 
1297 |   # When enabled it will grant sts:assumeRole permission to the IAM role for controller nodes.
1298 |   # This is intended to be used in combination with .controller.managedIamRoleName. See #297 for more information.
1299 |   kube2IamSupport:
1300 |     enabled: true
1301 | 
1302 |   # When enabled, `kubectl drain` is run when the instance is being replaced by the auto scaling group, or when
1303 |   # the instance receives a termination notice (in case of spot instances)
1304 |   nodeDrainer:
1305 |     enabled: true
1306 |     # Maximum time to wait, in minutes, for the node to be completely drained. Must be an integer between 1 and 60.
1307 |     drainTimeout: 5
1308 | 
1309 |   # Configure OpenID Connect token authenticator plugin in Kubernetes API server.
1310 |   # For using Dex as a custom OIDC provider, please check "contrib/dex/README.md".
1311 |   # WARNING: always use "https" for "issuerUrl", otherwise the Kubernetes API server will not start correctly.
1312 |   oidc:
1313 |     enabled: false
1314 |     issuerUrl: "https://accounts.google.com"
1315 |     clientId: "kubernetes"
1316 |     usernameClaim: "email"
1317 |     groupsClaim: "groups"
1318 | 
1319 |   # When set to true this configures the k8s aws provider so that it doesn't modify every node's security group
1320 |   # to include an additional ingress rule per an ELB created for a k8s service whose type is "LoadBalancer".
1321 |   # It requires that the user has setup a rule that allows inbound traffic on kubelet ports
1322 |   # from the local VPC subnet so that load balancers can access it.  Ref: https://github.com/kubernetes/kubernetes/issues/26670
1323 |   disableSecurityGroupIngress: false
1324 | 
1325 |   # Command line flag passed to the controller-manager. (default 40s)
1326 |   # This is the amount of time which we allow running Node to be unresponsive before marking it unhealthy.
1327 |   # Must be N times more than kubelet's nodeStatusUpdateFrequency (default 10s).
1328 | #   nodeMonitorGracePeriod: "40s"
1329 | 
1330 | kubelet:
1331 |   # Tell Kubelet to renew its certificate as its expiration time is approaching
1332 |   # Requires Experimental.tlsBootstrap to be enabled
1333 |   RotateCerts:
1334 |     enabled: true
1335 | 
1336 | # AWS Tags for cloudformation stack resources
1337 | stackTags:
1338 |   Name: "Kubernetes"
1339 |   Environment: "Development"
1340 | 
1341 | # User-provided YAML map available in control-plane's stack-template.json
1342 | #customSettings:
1343 | #  key1: [ 1, 2, 3 ]
1344 | 


--------------------------------------------------------------------------------
/src/vpc/deploy:
--------------------------------------------------------------------------------
 1 | #!/bin/sh
 2 | set -e
 3 | 
 4 | ROLE="kube-aws-vpc"
 5 | STACK_NAME="stack_name"
 6 | STACK_FILE="vpc"
 7 | SERVICE_NAME_PREFIX="k8s"
 8 | REGION="aws_region"
 9 | KUBERNETES_CLUSTER="demokube" #if you change this value, please also change the clusterName value in cluster.yaml
10 | 
11 | STACK_PARAMS="ParameterKey=ServiceNamePrefix,ParameterValue=${SERVICE_NAME_PREFIX} ParameterKey=Role,ParameterValue=${ROLE} ParameterKey=KubernetesCluster,ParameterValue=${KUBERNETES_CLUSTER}"
12 | STACK_TAGS="Key=Role,Value=${ROLE}"
13 | 
14 | for TEMPLATE_FILE in *.yaml
15 | do
16 |     echo "### Validating -> ${TEMPLATE_FILE}"
17 | 
18 |     aws cloudformation validate-template \
19 |             --template-body file://${TEMPLATE_FILE}
20 | 
21 |     echo
22 | done
23 | 
24 | aws --region $REGION cloudformation create-stack \
25 |     --stack-name $STACK_NAME \
26 |     --template-body file://./${TEMPLATE_FILE} \
27 |     --parameters ${STACK_PARAMS} \
28 |     --capabilities CAPABILITY_IAM \
29 |     --tags ${STACK_TAGS}
30 | 


--------------------------------------------------------------------------------
/src/vpc/vpc.yaml:
--------------------------------------------------------------------------------
  1 | AWSTemplateFormatVersion: '2010-09-09'
  2 | Description: kube-aws-secure VPC (Public and Private Subnets, IGW, Route Tables, KMS)
  3 | 
  4 | Parameters:
  5 |   Role:
  6 |     Type: String
  7 |   ServiceNamePrefix:
  8 |     Type: String
  9 |   KubernetesCluster:
 10 |     Type: String
 11 | 
 12 |   InstanceType:
 13 |     Description: EC2 HVM instance type.
 14 |     Type: String
 15 |     Default: t2.micro
 16 |     AllowedValues:
 17 |     - t2.micro
 18 |     - t2.small
 19 |     - t2.medium
 20 |     ConstraintDescription: Must be a valid EC2 HVM instance type.
 21 |   KeyPair:
 22 |     Description: The name of an EC2 Key Pair to allow SSH access to the instance.
 23 |     Default: key_name
 24 |     Type: String
 25 | 
 26 | Mappings:
 27 |   ConfigMap:
 28 |     Environment:
 29 |       VPC:            VPC_CIDR
 30 | 
 31 |       PublicSubnetA:  PUBLIC_SUBNET_A_CIDR
 32 |       PublicSubnetB:  PUBLIC_SUBNET_B_CIDR
 33 |       PublicSubnetC:  PUBLIC_SUBNET_C_CIDR
 34 | 
 35 |       PrivateSubnetA: PRIVATE_SUBNET_A_CIDR
 36 |       PrivateSubnetB: PRIVATE_SUBNET_B_CIDR
 37 |       PrivateSubnetC: PRIVATE_SUBNET_C_CIDR
 38 | 
 39 |       FnGetAZs:       ["aws_regiona", "aws_regionb", "aws_regionc"]
 40 | 
 41 | Resources:
 42 |   #KMS
 43 |   KubeAwsKey:
 44 |     Type: AWS::KMS::Key
 45 |     Properties:
 46 |       Description: "KubeAWS KMS key"
 47 |       KeyPolicy:
 48 |         Version: "2012-10-17"
 49 |         Id: !Sub "key-k8s"
 50 |         Statement:
 51 |           - Sid: "Enable IAM User Permissions"
 52 |             Effect: "Allow"
 53 |             Principal:
 54 |               AWS: !Sub "arn:aws:sts::${AWS::AccountId}:root"
 55 |             Action:
 56 |               - "kms:*"
 57 |             Resource: "*"
 58 |   # VPC
 59 |   KubeAwsVPC:
 60 |     Type: AWS::EC2::VPC
 61 |     Properties:
 62 |       CidrBlock:          !FindInMap [ConfigMap, Environment, VPC]
 63 |       EnableDnsSupport:   true
 64 |       EnableDnsHostnames: true
 65 |       InstanceTenancy:    default
 66 |       Tags:
 67 |         - Key:   Name
 68 |           Value: !Ref ServiceNamePrefix
 69 |         - Key:   Role
 70 |           Value: !Ref Role
 71 | 
 72 | #DHCP Options
 73 |   KubeDhcpOptions:
 74 |     Type: "AWS::EC2::DHCPOptions"
 75 |     Properties:
 76 |       DomainName: ec2.internal
 77 |       DomainNameServers:
 78 |         - AmazonProvidedDNS
 79 |       Tags:
 80 |         - Key:   Name
 81 |           Value: !Sub "${ServiceNamePrefix}-dhcp-options"
 82 |         - Key:   Role
 83 |           Value: !Ref Role
 84 | 
 85 |   KubeVPCDHCPOptionsAssociation:
 86 |     Type: AWS::EC2::VPCDHCPOptionsAssociation
 87 |     Properties:
 88 |       VpcId:
 89 |         Ref: KubeAwsVPC
 90 |       DhcpOptionsId:
 91 |         Ref: KubeDhcpOptions
 92 | 
 93 |   # IGW
 94 |   PublicInternetGateway:
 95 |     Type: AWS::EC2::InternetGateway
 96 |     Properties:
 97 |       Tags:
 98 |         - Key:   Name
 99 |           Value: !Sub "${ServiceNamePrefix}-igw"
100 |         - Key:   Role
101 |           Value: !Ref Role
102 | 
103 |   PublicVPCGatewayAttachment:
104 |     Type: AWS::EC2::VPCGatewayAttachment
105 |     Properties:
106 |       VpcId:             !Ref KubeAwsVPC
107 |       InternetGatewayId: !Ref PublicInternetGateway
108 | 
109 |   # Nat GW
110 | 
111 |   NATGatewayEIP:
112 |     Type: AWS::EC2::EIP
113 |     Properties:
114 |       Domain: vpc
115 | 
116 |   NATGateway:
117 |     Type: AWS::EC2::NatGateway
118 |     Properties:
119 |       AllocationId: !GetAtt NATGatewayEIP.AllocationId
120 |       SubnetId:     !Ref PublicSubnetA
121 | 
122 |   PublicVPCGatewayAttachment:
123 |     Type: AWS::EC2::VPCGatewayAttachment
124 |     Properties:
125 |       VpcId:             !Ref KubeAwsVPC
126 |       InternetGatewayId: !Ref PublicInternetGateway
127 | 
128 |   ### Public Subnets
129 |   PublicSubnetA:
130 |     Type: AWS::EC2::Subnet
131 |     Properties:
132 |       VpcId:               !Ref KubeAwsVPC
133 |       CidrBlock:           !FindInMap [ConfigMap, Environment, PublicSubnetA]
134 |       MapPublicIpOnLaunch: true
135 |       AvailabilityZone:    !Select [0, !FindInMap [ConfigMap, Environment, FnGetAZs]]
136 |       Tags:
137 |         - Key:   Name
138 |           Value: !Sub "${ServiceNamePrefix}-public-subnet-a"
139 |         - Key:   Role
140 |           Value: !Ref Role
141 |         - Key:   KubernetesCluster
142 |           Value: !Ref KubernetesCluster
143 | 
144 |   PublicSubnetB:
145 |     Type: AWS::EC2::Subnet
146 |     Properties:
147 |       VpcId:               !Ref KubeAwsVPC
148 |       CidrBlock:           !FindInMap [ConfigMap, Environment, PublicSubnetB]
149 |       MapPublicIpOnLaunch: true
150 |       AvailabilityZone:    !Select [1, !FindInMap [ConfigMap, Environment, FnGetAZs]]
151 |       Tags:
152 |         - Key:   Name
153 |           Value: !Sub "${ServiceNamePrefix}-public-subnet-b"
154 |         - Key:   Role
155 |           Value: !Ref Role
156 |         - Key:   KubernetesCluster
157 |           Value: !Ref KubernetesCluster
158 | 
159 |   PublicSubnetC:
160 |     Type: AWS::EC2::Subnet
161 |     Properties:
162 |       VpcId:               !Ref KubeAwsVPC
163 |       CidrBlock:           !FindInMap [ConfigMap, Environment, PublicSubnetC]
164 |       MapPublicIpOnLaunch: true
165 |       AvailabilityZone:    !Select [2, !FindInMap [ConfigMap, Environment, FnGetAZs]]
166 |       Tags:
167 |         - Key:   Name
168 |           Value: !Sub "${ServiceNamePrefix}-public-subnet-c"
169 |         - Key:   Role
170 |           Value: !Ref Role
171 |         - Key:   KubernetesCluster
172 |           Value: !Ref KubernetesCluster
173 | 
174 |   ### Private Subnets
175 |   PrivateSubnetA:
176 |     Type: AWS::EC2::Subnet
177 |     Properties:
178 |       VpcId:            !Ref KubeAwsVPC
179 |       CidrBlock:        !FindInMap [ConfigMap, Environment, PrivateSubnetA]
180 |       AvailabilityZone: !Select [0, !FindInMap [ConfigMap, Environment, FnGetAZs]]
181 |       Tags:
182 |         - Key:   Name
183 |           Value: !Sub "${ServiceNamePrefix}-private-subnet-a"
184 |         - Key:   Role
185 |           Value: !Ref Role
186 | 
187 |   PrivateSubnetB:
188 |     Type: AWS::EC2::Subnet
189 |     Properties:
190 |       VpcId:            !Ref KubeAwsVPC
191 |       CidrBlock:        !FindInMap [ConfigMap, Environment, PrivateSubnetB]
192 |       AvailabilityZone: !Select [1, !FindInMap [ConfigMap, Environment, FnGetAZs]]
193 |       Tags:
194 |         - Key:   Name
195 |           Value: !Sub "${ServiceNamePrefix}-private-subnet-b"
196 |         - Key:   Role
197 |           Value: !Ref Role
198 | 
199 |   PrivateSubnetC:
200 |     Type: AWS::EC2::Subnet
201 |     Properties:
202 |       VpcId:            !Ref KubeAwsVPC
203 |       CidrBlock:        !FindInMap [ConfigMap, Environment, PrivateSubnetC]
204 |       AvailabilityZone: !Select [2, !FindInMap [ConfigMap, Environment, FnGetAZs]]
205 |       Tags:
206 |         - Key:   Name
207 |           Value: !Sub "${ServiceNamePrefix}-private-subnet-c"
208 |         - Key:   Role
209 |           Value: !Ref Role
210 | 
211 |   ### Public Route Table
212 |   PublicRouteTable:
213 |     Type: AWS::EC2::RouteTable
214 |     Properties:
215 |       VpcId: !Ref KubeAwsVPC
216 | 
217 |       Tags:
218 |         - Key:   Name
219 |           Value: !Sub "${ServiceNamePrefix}-public-routes"
220 |         - Key:   Role
221 |           Value: !Ref Role
222 | 
223 |   PublicSubnetARouteTableAssociation:
224 |     Type: AWS::EC2::SubnetRouteTableAssociation
225 |     Properties:
226 |       RouteTableId: !Ref PublicRouteTable
227 |       SubnetId:     !Ref PublicSubnetA
228 | 
229 |   PublicSubnetBRouteTableAssociation:
230 |     Type: AWS::EC2::SubnetRouteTableAssociation
231 |     Properties:
232 |       RouteTableId: !Ref PublicRouteTable
233 |       SubnetId:     !Ref PublicSubnetB
234 | 
235 |   PublicSubnetCRouteTableAssociation:
236 |     Type: AWS::EC2::SubnetRouteTableAssociation
237 |     Properties:
238 |       RouteTableId: !Ref PublicRouteTable
239 |       SubnetId:     !Ref PublicSubnetC
240 | 
241 |   PublicRoute:
242 |     Type: AWS::EC2::Route
243 |     Properties:
244 |       RouteTableId:         !Ref PublicRouteTable
245 |       DestinationCidrBlock: 0.0.0.0/0
246 |       GatewayId:            !Ref PublicInternetGateway
247 | 
248 |   ### Private Route Table
249 |   PrivateRouteTable:
250 |     Type: AWS::EC2::RouteTable
251 |     Properties:
252 |       VpcId: !Ref KubeAwsVPC
253 | 
254 |       Tags:
255 |         - Key:   Name
256 |           Value: !Sub "${ServiceNamePrefix}-private-routes"
257 |         - Key:   Role
258 |           Value: !Ref Role
259 | 
260 |   PrivateSubnetARouteTableAssociation:
261 |     Type: AWS::EC2::SubnetRouteTableAssociation
262 |     Properties:
263 |       RouteTableId: !Ref PrivateRouteTable
264 |       SubnetId:     !Ref PrivateSubnetA
265 | 
266 |   PrivateSubnetBRouteTableAssociation:
267 |     Type: AWS::EC2::SubnetRouteTableAssociation
268 |     Properties:
269 |       RouteTableId: !Ref PrivateRouteTable
270 |       SubnetId:     !Ref PrivateSubnetB
271 | 
272 |   PrivateSubnetCRouteTableAssociation:
273 |     Type: AWS::EC2::SubnetRouteTableAssociation
274 |     Properties:
275 |       RouteTableId: !Ref PrivateRouteTable
276 |       SubnetId:     !Ref PrivateSubnetC
277 | 
278 |   PrivateRoute:
279 |     Type: AWS::EC2::Route
280 |     Properties:
281 |       DestinationCidrBlock: 0.0.0.0/0
282 |       RouteTableId:         !Ref PrivateRouteTable
283 |       NatGatewayId:         !Ref NATGateway
284 | 
285 | 
286 | #VPN
287 |   VpnAddress:
288 |     Type: AWS::EC2::EIP
289 |     Properties:
290 |       Domain: vpc
291 |   AssociateVpnAddress:
292 |     Type: AWS::EC2::EIPAssociation
293 |     Properties:
294 |       AllocationId: !GetAtt VpnAddress.AllocationId
295 |       NetworkInterfaceId: !Ref eth0
296 |   SSHSecurityGroup:
297 |     Type: AWS::EC2::SecurityGroup
298 |     Properties:
299 |       VpcId: !Ref KubeAwsVPC
300 |       GroupDescription: Enable SSH access via port 22 from current IP
301 |       SecurityGroupIngress:
302 |       - CidrIp: my_ip/32
303 |         FromPort: 22
304 |         IpProtocol: tcp
305 |         ToPort: 22
306 |   WebSecurityGroup:
307 |     Type: AWS::EC2::SecurityGroup
308 |     Properties:
309 |       VpcId: !Ref KubeAwsVPC
310 |       GroupDescription: Enable HTTPS access from current IP
311 |       SecurityGroupIngress:
312 |       - CidrIp: my_ip/32
313 |         FromPort: 443
314 |         IpProtocol: tcp
315 |         ToPort: 443
316 |   VPNSecurityGroup:
317 |     Type: AWS::EC2::SecurityGroup
318 |     Properties:
319 |       VpcId: !Ref KubeAwsVPC
320 |       GroupDescription: Enable VPN access via port 12777 from everywhere
321 |       SecurityGroupIngress:
322 |       - CidrIp: 0.0.0.0/0
323 |         FromPort: 12777
324 |         IpProtocol: udp
325 |         ToPort: 12777
326 |   LESecurityGroup:
327 |     Type: AWS::EC2::SecurityGroup
328 |     Properties:
329 |       VpcId: !Ref KubeAwsVPC
330 |       GroupDescription: Allow LetsEncrypt to verify the host
331 |       SecurityGroupIngress:
332 |       - CidrIp: 0.0.0.0/0
333 |         FromPort: 80
334 |         IpProtocol: tcp
335 |         ToPort: 80
336 |   eth0:
337 |     Type: AWS::EC2::NetworkInterface
338 |     Properties:
339 |       SubnetId: !Ref PublicSubnetA
340 |       Description: Vpn interface
341 |       GroupSet:
342 |       - !Ref SSHSecurityGroup
343 |       - !Ref WebSecurityGroup
344 |       - !Ref VPNSecurityGroup
345 |       - !Ref LESecurityGroup
346 |       SourceDestCheck: true
347 |       Tags:
348 |         -
349 |           Key: Network
350 |           Value: Control
351 |   Ec2Instance:
352 |     Type: AWS::EC2::Instance
353 |     Properties:
354 |       ImageId: ami_id
355 |       KeyName: !Ref KeyPair
356 |       InstanceType: !Ref InstanceType
357 |       NetworkInterfaces:
358 |         -
359 |           NetworkInterfaceId: !Ref eth0
360 |           DeviceIndex: 0
361 |       Tags:
362 |         -
363 |           Key: Role
364 |           Value: Vpn
365 |       UserData: H4sICKHlBVoAA3VzZXJkYXRhLnlhbWwAvVVJb9QwFL7Pr3ikPcDBk6mEQArygYoicQBVlOVQKuQ4LxNrYjvYTmZG4sdjZ9eQ0sKBHLK89fNbvpzxUtcZ4VrlYrvi2qC2yQqgrjLmMLwBGEy1dsQ640XbYwKRzvMoGCnhbGdDQDGJCVRGuFqVa4umERxbHQAqlpZe60w9iLiWkqksAeuYcaNQOVQugZ+9AOD2s09yN36+QcuNqJzQil53ueDL9YdR/xF/1MKgpZnmOzQnOABe5w4NbZghpUjJgFbqWrnVlPOm85rSXh2Q3wSg1wYpiWtr4lSoWO4yYYBUEPuIsY8Y9xGH578HkFptdZb+HoCOzt0RwdQKvo12w0VIaAjtwy0aZOgYLxZV3q0RJW4xWw6Nbq/NjhbaukUD34EWKyv37GiXTJp7j5yMil7wSPf+mZwqFtw5k2JyuFhvZvMzB34qvkFOX2zsbFLeKa8oy2lSvjI/wtnlkcq6dILUfgDX3nWL/XwNi7I8gl2MxXV5aDMuMffbS08XcDJ/HzLMgBbMz1KGTXxoMj4To49yWsNR/elYIcWDe/6nGvRLGKqwCGeogQcsmRtKQBpd1hL/C3W8bTPbga4AUwtd+hOm8PUhoT5r/zLnkYln7jHo2/HXTNMWWCu0hXazbJIJ1SK6OghHjziN5owYdOVaYgj9IX1xu1MRrTjC1O3HNe8e8Hv/id9zzw8t+ROomCsSeCh9m6FCI4W1vgc2gc3LTbd5eq/QJGD8byYJt9ViR8+etOFTZgsgQ6VFDre3EJ0/3YsKcwtEBTo9v4CfsDVYQQDzLAJKIYrg7u6VK1CNZ5W73K6DhXcY5q20UxORFxqi7gwhprDASoMsO/az61c96q1zsfoFsiL7OE0HAAA=
366 |   DataVolume:
367 |     Type: AWS::EC2::Volume
368 |     Properties:
369 |       Size: 5
370 |       AvailabilityZone: !GetAtt Ec2Instance.AvailabilityZone
371 |   MountPoint:
372 |     Type: AWS::EC2::VolumeAttachment
373 |     Properties:
374 |       InstanceId: !Ref Ec2Instance
375 |       VolumeId: !Ref DataVolume
376 |       Device: /dev/xvdc
377 |   VpnDNSRecord:
378 |     Type: AWS::Route53::RecordSet
379 |     Properties:
380 |       HostedZoneName: domain_name.
381 |       Name: k8svpn.domain_name.
382 |       Type: A
383 |       TTL: '300'
384 |       ResourceRecords:
385 |       - !Ref VpnAddress
386 | 
387 | Outputs:
388 |   VpnDNSReccord:
389 |     Value: !Ref VpnDNSRecord
390 | 
391 |   VpnIpAddress:
392 |     Value: !Ref VpnAddress
393 | 
394 |   VpcId:
395 |     Value: !Ref KubeAwsVPC
396 | 
397 |   CidrBlock:
398 |     Value: !GetAtt KubeAwsVPC.CidrBlock
399 | 
400 |   PublicRouteTableId:
401 |     Value: !Ref PublicRouteTable
402 |   PrivateRouteTableId:
403 |     Value: !Ref PrivateRouteTable
404 | 
405 |   PublicSubnetAId:
406 |     Value: !Ref PublicSubnetA
407 |   PublicSubnetBId:
408 |     Value: !Ref PublicSubnetB
409 |   PublicSubnetCId:
410 |     Value: !Ref PublicSubnetC
411 | 
412 |   PrivateSubnetAId:
413 |     Value: !Ref PrivateSubnetA
414 |   PrivateSubnetBId:
415 |     Value: !Ref PrivateSubnetB
416 |   PrivateSubnetCId:
417 |     Value: !Ref PrivateSubnetC
418 | 
419 |   KMSKeyArn:
420 |     Value: !GetAtt KubeAwsKey.Arn
421 | 


--------------------------------------------------------------------------------
/src/vpn/Docker/Dockerfile:
--------------------------------------------------------------------------------
 1 | FROM ubuntu:16.04
 2 | 
 3 | RUN echo "deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" > /etc/apt/sources.list.d/mongodb-org-3.4.list \
 4 |     && echo "deb http://repo.pritunl.com/stable/apt xenial main" > /etc/apt/sources.list.d/pritunl.list \
 5 |     && echo "deb http://build.openvpn.net/debian/openvpn/stable xenial main" > /etc/apt/sources.list.d/openvpn-aptrepo.list \
 6 |     && apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv 0C49F3730359A14518585931BC711F9BA15703C6 \
 7 |     && apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv 7568D9BB55FF9E5287D586017AE645C0CF8E292A \
 8 |     && apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv 8E6DA8B4E158C569 \
 9 |     && apt-get update -q \
10 |     && apt-get install locales \
11 |     && locale-gen en_US en_US.UTF-8 \
12 |     && dpkg-reconfigure locales \
13 |     && ln -sf /usr/share/zoneinfo/UTC /etc/localtime \
14 |     && apt-get upgrade -y -q \
15 |     && apt-get dist-upgrade -y -q \
16 |     && apt-get -y install pritunl mongodb-org iptables \
17 |     && apt-get clean \
18 |     && apt-get -y -q autoclean \
19 |     && apt-get -y -q autoremove \
20 |     && rm -rf /tmp/*
21 | 
22 | ADD start-pritunl /bin/start-pritunl
23 | 
24 | EXPOSE 80
25 | EXPOSE 443
26 | EXPOSE 1194
27 | EXPOSE 1194/udp
28 | 
29 | ENTRYPOINT ["/bin/start-pritunl"]
30 | 
31 | CMD ["/usr/bin/tail", "-f","/var/log/pritunl.log"]
32 | 


--------------------------------------------------------------------------------
/src/vpn/Docker/start-pritunl:
--------------------------------------------------------------------------------
 1 | #!/bin/sh
 2 | set -e
 3 | 
 4 | [ -d /dev/net ] ||
 5 |     mkdir -p /dev/net
 6 | [ -c /dev/net/tun ] ||
 7 |     mknod /dev/net/tun c 10 200
 8 | 
 9 | touch /var/log/pritunl.log
10 | touch /var/run/pritunl.pid
11 | /bin/rm /var/run/pritunl.pid
12 | 
13 | if [ "$1" = "bash" ]; then
14 |     exec "$@"
15 |     exit $?
16 | fi
17 | 
18 | # allow changing debug mode
19 | if [ -z "$PRITUNL_DEBUG" ]; then
20 |     PRITUNL_DEBUG="false"
21 | fi
22 | 
23 | # allow changing bind addr
24 | if [ -z "$PRITUNL_BIND_ADDR" ]; then
25 |     PRITUNL_BIND_ADDR="0.0.0.0"
26 | fi
27 | 
28 | 
29 | ## start a local mongodb instance if no mongodb specified through env
30 | if [ -z "$PRITUNL_MONGODB_URI" ]; then
31 |   /usr/bin/mongod -f /etc/mongod.conf &
32 |   PRITUNL_MONGODB_URI="mongodb://localhost:27017/pritunl"
33 | fi
34 | 
35 | if [ -z "$PRITUNL_DONT_WRITE_CONFIG" ]; then
36 |     cat << EOF > /etc/pritunl.conf
37 |     {
38 |         "mongodb_uri": "$PRITUNL_MONGODB_URI",
39 |         "server_key_path": "/var/lib/pritunl/pritunl.key",
40 |         "log_path": "/var/log/pritunl.log",
41 |         "static_cache": true,
42 |         "server_cert_path": "/var/lib/pritunl/pritunl.crt",
43 |         "temp_path": "/tmp/pritunl_%r",
44 |         "bind_addr": "$PRITUNL_BIND_ADDR",
45 |         "debug": $PRITUNL_DEBUG,
46 |         "www_path": "/usr/share/pritunl/www",
47 |         "local_address_interface": "auto"
48 |     }
49 | EOF
50 | 
51 | fi
52 | 
53 | exec /usr/bin/pritunl start -c /etc/pritunl.conf
54 | 


--------------------------------------------------------------------------------
/src/vpn/userdata.yaml:
--------------------------------------------------------------------------------
 1 | #cloud-config
 2 | coreos:
 3 |   update:
 4 |     reboot-strategy: "off"
 5 |   units:
 6 |     - name: pritunl.service
 7 |       enable: true
 8 |       command: start
 9 |       content: |
10 |         [Unit]
11 |         Description=Pritunl VPN
12 |         Requires=docker.service
13 |         After=var-lib-pritunl.mount
14 | 
15 |         [Service]
16 |         ExecStartPre=-/usr/bin/mkdir -p /var/lib/pritunl/pritunl
17 |         ExecStartPre=-/usr/bin/mkdir -p /var/lib/pritunl/mongodb
18 |         ExecStart=/usr/bin/docker run \
19 |                 --name=pritunl \
20 |                 --detach \
21 |                 --privileged \
22 |                 --network=host \
23 |                 --restart=always \
24 |                 -v /var/lib/pritunl/mongodb:/var/lib/mongodb \
25 |                 -v /var/lib/pritunl/pritunl:/var/lib/pritunl \
26 |                 camil/pritunl:1.0
27 |         Restart=always
28 |         RestartSec=60s
29 | 
30 |         [Install]
31 |         WantedBy=multi-user.target
32 | 
33 |     - name: var-lib-pritunl.mount
34 |       enable: true
35 |       content: |
36 |         [Unit]
37 |         Before=pritunl.service
38 | 
39 |         [Mount]
40 |         What=/dev/xvdc
41 |         Where=/var/lib/pritunl
42 |         Type=ext4
43 | 
44 |         [Install]
45 |         RequiredBy=pritunl.service
46 | 
47 |     - name: format-pritunl-volume.service
48 |       enable: true
49 |       command: start
50 |       content: |
51 |         [Unit]
52 |         Description=Formats pritunl ebs volume
53 |         After=dev-xvdc.device
54 |         Requires=dev-xvdc.device
55 |         Before=var-lib-pritunl.mount
56 | 
57 |         [Service]
58 |         Type=oneshot
59 |         RemainAfterExit=yes
60 |         ExecStart=/opt/bin/ext4-format-volume-once /dev/xvdc
61 | 
62 |         [Install]
63 |         RequiredBy=var-lib-pritunl.mount
64 | 
65 | write_files:
66 |   - path: /opt/bin/ext4-format-volume-once
67 |     permissions: 0700
68 |     owner: root:root
69 |     content: |
70 |       #!/bin/bash -e
71 |       if [[ "$(wipefs -n -p $1 | grep ext4)" == "" ]];then
72 |         mkfs.ext4 $1
73 |       else
74 |         echo "volume $1 is already formatted"
75 |       fi
76 | 


--------------------------------------------------------------------------------