├── README-jumpbox.md ├── README.md ├── c9-instance-profile.yaml ├── ec2-role-trust-policy.json ├── images ├── 2048.png ├── alb-address.png ├── alb-ingress-controller.png ├── alb-ingress.png ├── alb-ip.png ├── c9-config.png ├── c9-create.png ├── c9-credential.png ├── cfn-complete.png ├── cfn-confirm.png └── cfn-ip.png ├── jumpbox-eks.yaml └── spot ├── README.md └── images ├── 01.png ├── 02.png └── 03.png /README-jumpbox.md: -------------------------------------------------------------------------------- 1 | # EKS Workshop 动手实验说明 2 | 3 | 本文档目的是指导用户熟悉建立Amazon EKS集群的过程，并利用AWS相关的服务和插件部署一套常见应用程序。 4 | 5 | 部署流程如下： 6 | 7 | ## 1. 准备工作 8 | ### a. 请确保您有海外AWS账号和足够的资源配额 9 | - 请确保您有海外AWS账号的IAM user，该用户有创建相关资源的权限。 10 | ### b. 准备跳板机所需要的VPC 11 | - 可以使用默认的VPC或者创建新的VPC, 可以参考[这个文档](https://docs.aws.amazon.com/zh_cn/vpc/latest/userguide/VPC_Scenario1.html#VPC_Scenario1_Implementation)创建新的VPC。 12 | 13 | ## 2. 创建跳板机 14 | 请点击如下链接在各AWS区域创建跳板机： 15 | 16 | [Virginia](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 17 | 18 | [Ohio](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 19 | 20 | [N California](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 21 | 22 | [Oregon](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 23 | 24 | [Mumbai](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 25 | 26 | [Seoul](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 27 | 28 | [Singapore](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 29 | 30 | [Sydney](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 31 | 32 | [Tokyo](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 33 | 34 | [Montreal](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 35 | 36 | [Frankfurt](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 37 | 38 | [Ireland](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 39 | 40 | [London](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 41 | 42 | [Paris](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 43 | 44 | [Sao Paulo](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/quickcreate?stackName=eks-jumpbox&templateURL=https://s3-ap-southeast-1.amazonaws.com/whe-pub/eks-workshop/jumpbox-eks.yaml) 45 | 46 | 在Quick create stack页面填入相应参数： 47 | - **InstanceType**: 跳板机实例类型，可以保持默认参数。 48 | - **SubnetId**: 跳板机所在的子网，私有子网或公有子网都可以，如果是私有子网，请确保有NAT网关或NAT实例可访问外网。 49 | - **VpcId**: 跳板机所在VPC，请确保SubnetId是在VpcId所在的VPC内。 50 | 51 | 勾选 "我确认，AWS CloudFormation 可能创建 IAM 资源" 52 | ![](./images/cfn-confirm.png) 53 | 约5分钟后，CloudFormation堆栈创建完成: 54 | ![](./images/cfn-complete.png) 55 | 56 | 57 | ## 3. 登录跳板机创建EKS集群 58 | - 登录跳板机的Web SSH控制台，控制台的URL可以从CloudFormation的输出界面中找到： 59 | ![](./images/cfn-ip.png) 60 | - 创建eksctl config文件。在此配置文件中，node group 使用了mixed instances nodegroup，按需实例和竞价实例搭配使用。更多信息请参考此[文档](https://docs.aws.amazon.com/zh_cn/AWSEC2/latest/UserGuide/spot-fleet.html) 61 | ``` 62 | cd ~ 63 | REGION=`curl -sS http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region` 64 | 65 | cat <>eks-config.yaml 66 | apiVersion: eksctl.io/v1alpha5 67 | kind: ClusterConfig 68 | 69 | metadata: 70 | name: eks-workshop 71 | region: $REGION 72 | 73 | nodeGroups: 74 | - name: ng-1 75 | minSize: 3 76 | maxSize: 5 77 | privateNetworking: true 78 | iam: 79 | attachPolicyARNs: 80 | - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy 81 | - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy 82 | - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy 83 | withAddonPolicies: 84 | autoScaler: true 85 | albIngress: true 86 | instancesDistribution: 87 | instanceTypes: ["t3.small", "t3.medium"] 88 | onDemandBaseCapacity: 0 89 | onDemandPercentageAboveBaseCapacity: 50 90 | spotInstancePools: 2 91 | EOF 92 | ``` 93 | - 运行eksctl创建新的EKS集群： 94 | ``` 95 | eksctl create cluster -f ./eks-config.yaml 96 | ``` 97 | 98 | 更多参数可参考[eksctl官方文档](https://eksctl.io/)。 99 | 100 | 所有资源创建可能需要15分钟左右。 101 | 102 | ## 4. 创建示例应用程序 2048小游戏 103 | 在本部分中，您将创建一个2048小游戏来测试新集群。 104 | ```bash 105 | kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/2048/2048-namespace.yaml 106 | kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/2048/2048-deployment.yaml 107 | kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/2048/2048-service.yaml 108 | ``` 109 | 110 | ## 5. 创建AWS ALB Ingress controller 111 | ### a. 部署 ALB ingress controller 112 | 部署 AWS ALB Ingress controller所需的RBAC Roles 和 RoleBindings: 113 | ```bash 114 | kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/rbac-role.yaml 115 | ``` 116 | 下载 AWS ALB Ingress controller YAML: 117 | ```bash 118 | curl -sS "https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/alb-ingress-controller.yaml" > alb-ingress-controller.yaml 119 | ``` 120 | 修改 –cluster-name 为上述步骤创建的EKS集群名字: 121 | ![](./images/alb-ingress-controller.png) 122 | 123 | 部署ingress controller: 124 | ```bash 125 | kubectl apply -f alb-ingress-controller.yaml 126 | ``` 127 | 128 | ### b. 部署 ingress 129 | 下载ingress YAML: 130 | ```bash 131 | curl -sS https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.0.1/docs/examples/2048/2048-ingress.yaml -o 2048-ingress.yaml 132 | ``` 133 | 编辑 `2048-ingress.yaml` 文件, 在annotations中增加 `alb.ingress.kubernetes.io/target-type: ip`. 本示例中我们将用`ip mode` 作为目标类型，这样ALB的流量将会直接转发到Pod上。 134 | ![](./images/alb-ip.png) 135 | 然后创建ingress 136 | ```bash 137 | kubectl apply -f 2048-ingress.yaml 138 | ``` 139 | 140 | 获取ingress资源的`Address`属性： 141 | ```bash 142 | kubectl -n 2048-game describe ing/2048-ingress 143 | ``` 144 | ![](./images/alb-address.png) 145 | 146 | ### c. 用浏览器打开上一步的ALB地址，开始您的2048游戏之旅！ 147 | ![](./images/2048.png) 148 | 149 | ## 6. 清理环境 150 | ### a. 删除ingress 151 | ```bash 152 | kubectl delete -f 2048-ingress.yaml 153 | ``` 154 | ### b. 删除EKS集群 155 | ```bash 156 | eksctl delete cluster eks-workshop --region $REGION 157 | ``` 158 | ### c. 删除跳板机CloudFormation stack 159 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # EKS Workshop 动手实验说明 2 | 3 | 本文档目的是指导用户熟悉建立Amazon EKS集群的过程，并利用AWS相关的服务和插件部署一套常见应用程序。 4 | 5 | 部署流程如下： 6 | 7 | ## 1. 准备工作 8 | ### a. 请确保您有海外AWS账号和足够的资源配额 9 | - 请确保您有创建海外AWS账号的IAM user，该用户有Administrator权限。 10 | ### b. 准备Cloud9所需要的VPC 11 | - 可以使用默认的VPC或者创建新的VPC, 可以参考[这个文档](https://docs.aws.amazon.com/zh_cn/vpc/latest/userguide/VPC_Scenario1.html#VPC_Scenario1_Implementation)创建新的VPC。 12 | 13 | ## 2. 创建Cloud9 14 | 点击[这里](https://ap-southeast-1.console.aws.amazon.com/cloud9/home/create)启动Cloud9开发环境 15 | 填入Cloud9名称，Next Step进入到下一步： 16 | ![](./images/c9-create.png) 17 | 18 | 其它步骤可以保持默认参数不变，最后一步点击Create Environment. Cloud9新环境创建需要一分钟左右。 19 | 20 | ## 3. 配置Cloud9环境 21 | - 在Cloud9 terminal窗口中执行如下命令，创建新的EC2 instance profile并配置给当前Cloud9实例。请注意下面第二行命令需要等待两分钟左右完成。 22 | ```bash 23 | $ aws cloudformation create-stack --stack-name eksworkshop-instprofile --capabilities CAPABILITY_IAM --template-url https://whe-pub.s3-ap-southeast-1.amazonaws.com/c9-instance-profile.yaml 24 | $ aws cloudformation wait stack-create-complete --stack-name eksworkshop-instprofile 25 | $ PROFILENAME=$(aws cloudformation describe-stacks --stack-name eksworkshop-instprofile --output text --query 'Stacks[0].Outputs[0].OutputValue') 26 | $ MYINSTID=$(curl -sS http://169.254.169.254/latest/meta-data/instance-id) 27 | $ aws ec2 associate-iam-instance-profile --instance-id $MYINSTID --iam-instance-profile Name=$PROFILENAME 28 | ``` 29 | 30 | - 禁用掉managed temporary credentials： 31 | ![](./images/c9-config.png) 32 | 33 | 正确配置完成后，运行`aws sts get-caller-identity`将会看到如下类似json输出。 34 | ![](./images/c9-credential.png) 35 | 36 | - 下载 `kubectl` 37 | 38 | ```bash 39 | $ sudo curl -o /usr/local/bin/kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.13.7/2019-06-11/bin/linux/amd64/kubectl 40 | $ sudo chmod +x /usr/local/bin/kubectl 41 | ``` 42 | 43 | - 下载 `eksctl` 44 | 45 | ```bash 46 | $ curl -sL "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp 47 | $ sudo mv /tmp/eksctl /usr/local/bin 48 | ``` 49 | 50 | - 下载 `aws-iam-authenticator`。*目前eksctl还[不支持使用aws cli get-token配置kube config](https://github.com/weaveworks/eksctl/issues/788)* 51 | 52 | ```bash 53 | $ sudo curl -o /usr/local/bin/aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.13.7/2019-06-11/bin/linux/amd64/aws-iam-authenticator 54 | $ sudo chmod +x /usr/local/bin/aws-iam-authenticator 55 | ``` 56 | 57 | ## 4. 创建EKS集群 58 | - 运行如下命令，用eksctl创建一个包含默认node group的集群： 59 | ```bash 60 | $ sudo yum install -y jq 61 | $ REGION=`curl -sS http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region` 62 | $ aws configure set default.region $REGION 63 | $ eksctl create cluster --region $REGION --name eks-workshop --alb-ingress-access 64 | ``` 65 | 66 | 更多参数可参考[eksctl官方文档](https://eksctl.io/)。 67 | 68 | 所有资源创建需要15分钟左右。 69 | 70 | ## 5. 配置Helm环境 71 | - 安装helm cli，创建tiller namespace: 72 | ```bash 73 | $ curl -L https://git.io/get_helm.sh | bash 74 | $ kubectl create namespace tiller 75 | ``` 76 | - 启动本地tiller server: 77 | ```bash 78 | $ export TILLER_NAMESPACE=tiller 79 | $ tiller -listen=localhost:44134 -storage=secret -logtostderr 80 | ``` 81 | - 不要关闭上述命令行窗口，在旁边启动一个新tab页命令行窗口，运行下面的命令以初始化本地heml环境: 82 | ```bash 83 | $ export HELM_HOST=:44134 84 | $ helm init --client-only 85 | $ helm repo update 86 | ``` 87 | 88 | ## 5. 创建示例应用程序 2048小游戏 89 | 在本部分中，您将创建一个2048小游戏来测试新集群。 90 | ```bash 91 | $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/2048/2048-namespace.yaml 92 | $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/2048/2048-deployment.yaml 93 | $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/2048/2048-service.yaml 94 | ``` 95 | 96 | ## 6. 创建AWS ALB Ingress controller 97 | ### a. 部署 ALB ingress controller 98 | ```bash 99 | $ helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator 100 | $ helm repo update 101 | $ helm install incubator/aws-alb-ingress-controller \ 102 | --set clusterName=eks-workshop \ 103 | --set autoDiscoverAwsRegion=true \ 104 | --set autoDiscoverAwsVpcID=true 105 | ``` 106 | 107 | ### b. 部署 ingress 108 | 下载ingress YAML: 109 | ```bash 110 | $ curl -sS https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/2048/2048-ingress.yaml -o 2048-ingress.yaml 111 | ``` 112 | 编辑 `2048-ingress.yaml` 文件, 在annotations中增加 `alb.ingress.kubernetes.io/target-type: ip`. 本示例中我们将用`ip mode` 作为目标类型，这样ALB的流量将会直接转发到Pod上。 113 | ![](./images/alb-ip.png) 114 | 然后创建ingress 115 | ```bash 116 | $ kubectl apply -f 2048-ingress.yaml 117 | ``` 118 | 119 | 获取ingress资源的`Address`属性： 120 | ```bash 121 | $ kubectl -n 2048-game describe ing/2048-ingress 122 | ``` 123 | ![](./images/alb-address.png) 124 | 125 | ### c. 用浏览器打开上一步的ALB地址。如果不能正常显示，等待一分钟左右，或者进入EC2控制台查看ALB的状态。 126 | ![](./images/2048.png) 127 | 128 | ## 7. Amazon EKS结合Amazon EC2 Spot 129 | 参考[此文档](https://walkley.github.io/eks-workshop/spot/)以创建基于spot fleet的EKS集群。 130 | 131 | ## 8. 清理环境 132 | ### a. 删除ingress 133 | ```bash 134 | $ kubectl delete -f 2048-ingress.yaml 135 | ``` 136 | ### b. 删除EKS集群 137 | ```bash 138 | $ eksctl delete cluster eks-workshop 139 | ``` 140 | -------------------------------------------------------------------------------- /c9-instance-profile.yaml: -------------------------------------------------------------------------------- 1 | Metadata: 2 | License: Apache-2.0 3 | AWSTemplateFormatVersion: '2010-09-09' 4 | Description: 'AWS CloudFormation template to create instance profile for Cloud9.' 5 | Resources: 6 | InstanceRole: 7 | Type: AWS::IAM::Role 8 | Properties: 9 | AssumeRolePolicyDocument: 10 | Version: '2012-10-17' 11 | Statement: 12 | - Effect: Allow 13 | Principal: 14 | Service: ec2.amazonaws.com 15 | Action: sts:AssumeRole 16 | Path: "/" 17 | ManagedPolicyArns: 18 | - arn:aws:iam::aws:policy/AdministratorAccess 19 | NodeInstanceProfile: 20 | Type: AWS::IAM::InstanceProfile 21 | Properties: 22 | Path: "/" 23 | Roles: 24 | - !Ref InstanceRole 25 | Outputs: 26 | InstanceProfile: 27 | Description: InstanceProfile ID 28 | Value: !Ref NodeInstanceProfile 29 | -------------------------------------------------------------------------------- /ec2-role-trust-policy.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Principal": { 7 | "Service": "ec2.amazonaws.com" 8 | }, 9 | "Action": "sts:AssumeRole" 10 | } 11 | ] 12 | } -------------------------------------------------------------------------------- /images/2048.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/2048.png -------------------------------------------------------------------------------- /images/alb-address.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/alb-address.png -------------------------------------------------------------------------------- /images/alb-ingress-controller.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/alb-ingress-controller.png -------------------------------------------------------------------------------- /images/alb-ingress.png: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | kops-cn/aws-alb-ingress-04.png at master · nwcdlabs/kops-cn 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 |
114 | Skip to content 115 |
116 | 117 | 118 | 119 | 120 | 121 | 122 | 597 | 598 | 599 | 600 |
601 | 602 |
603 | 604 | 605 |
606 | 607 |
608 | 609 | 610 | 611 |
612 |
613 |
614 | 615 | 616 | 617 | 618 | 619 | 620 | 621 | 622 | 623 | 624 |
625 |
626 | 627 |
    628 | 629 | 630 | 631 |
  • 632 | 633 |
    634 | 635 |
    636 | 637 | 638 | Watch 639 | 640 | 643 |
    644 | Notifications 645 |
    646 |
    647 | 658 | 659 | 670 | 671 | 682 | 683 | 694 |
    695 |     696 |
    697 | 702 |
    703 |
    • 704 | 705 |
  • 706 |
    707 |
    708 | 709 | 715 |
    716 |
    717 | 718 | 724 |
    725 | 726 |
    • 727 | 728 |
  • 729 |
    730 | 731 | Fork 732 | 736 |
    737 | 740 |

    Fork kops-cn

    741 |
    742 |
    743 | 744 | 745 |

    If this dialog fails to load, you can visit the fork page directly.

    746 |     747 |
    748 |     749 |
    750 | 751 | 755 |
    • 756 |
757 | 758 |

759 | 760 | /kops-cn 763 | 764 | 765 |

766 | 767 |
768 | 769 | 816 | 817 |
818 | 865 |
866 | 867 | 868 |
869 |
870 |
871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | Permalink 879 | 880 | 881 | 882 | 883 |
884 | 885 | 886 |
887 | 891 | Branch: 892 | master 893 | 894 | 895 | 896 | 897 | 898 | 899 | 900 |
901 | 902 |
903 | 907 | Find file 908 | 909 | 910 | Copy path 911 | 912 |
913 | 914 |

915 | kops-cn/images/aws-alb-ingress-04.png 916 |

917 | 918 |
919 | 923 | Find file 924 | 925 | 926 | Copy path 927 | 928 |
929 |
930 | 931 | 932 | 933 | 934 |
935 | Fetching contributors… 936 |
937 | 938 |
939 | 940 | Cannot retrieve contributors at this time 941 |
942 | 943 | 944 | 945 | 946 | 947 |
948 | 949 |
950 | 951 |
952 | 112 KB 953 |
954 | 955 |
956 | 957 |
958 | Download 959 | History 960 |
961 | 962 | 963 |
964 | 968 | 969 | 970 | 971 |
972 | 976 |
977 |
978 |
979 | 980 | 981 | 982 |
983 |
984 | aws-alb-ingress-04.png 985 |
986 |
987 | 988 |
989 | 990 | 991 | 992 |
993 | 994 | 995 |
996 | 997 | 998 |
999 |
1000 | 1001 | 1002 | 1003 |
1004 |
1005 |
1006 | 1007 |
1008 |
1009 | 1010 | 1011 |
1012 | 1013 | 1014 | 1042 | 1043 | 1044 | 1045 |
1046 | 1047 | 1050 | You can’t perform that action at this time. 1051 |
1052 | 1053 | 1054 | 1055 | 1056 | 1057 | 1058 | 1059 | 1060 | 1061 | 1067 | 1078 | 1079 | 1083 | 1084 |
1085 | 1086 | 1087 | 1088 | 1089 | -------------------------------------------------------------------------------- /images/alb-ip.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/alb-ip.png -------------------------------------------------------------------------------- /images/c9-config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/c9-config.png -------------------------------------------------------------------------------- /images/c9-create.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/c9-create.png -------------------------------------------------------------------------------- /images/c9-credential.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/c9-credential.png -------------------------------------------------------------------------------- /images/cfn-complete.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/cfn-complete.png -------------------------------------------------------------------------------- /images/cfn-confirm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/cfn-confirm.png -------------------------------------------------------------------------------- /images/cfn-ip.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/images/cfn-ip.png -------------------------------------------------------------------------------- /jumpbox-eks.yaml: -------------------------------------------------------------------------------- 1 | Metadata: 2 | License: Apache-2.0 3 | AWSTemplateFormatVersion: '2010-09-09' 4 | Description: 'AWS CloudFormation template to create jumpbox node for EKS.' 5 | Parameters: 6 | InstanceType: 7 | Description: Enter a valid instance type for your selected region. 8 | Type: String 9 | Default: 't2.small' 10 | AllowedValues: 11 | - t2.small 12 | - t2.medium 13 | - t2.large 14 | - m4.large 15 | - m4.xlarge 16 | VpcId: 17 | Type: AWS::EC2::VPC::Id 18 | Description: VPC for jumpbox instance. 19 | SubnetId: 20 | Type: AWS::EC2::Subnet::Id 21 | Description: Public subnet for jumpbox instance. 22 | Mappings: 23 | AWSRegionToAMI: 24 | us-east-1: 25 | AMI: ami-0de53d8956e8dcf80 # Virginia 26 | us-east-2: 27 | AMI: ami-02bcbb802e03574ba # Ohio 28 | us-west-1: 29 | AMI: ami-0019ef04ac50be30f # N California 30 | us-west-2: 31 | AMI: ami-061392db613a6357b # Oregon 32 | ap-south-1: 33 | AMI: ami-0889b8a448de4fc44 # Mumbai 34 | ap-northeast-2: 35 | AMI: ami-047f7b46bd6dd5d84 # Seoul 36 | ap-southeast-1: 37 | AMI: ami-0b419c3a4b01d1859 # Singapore 38 | ap-southeast-2: 39 | AMI: ami-04481c741a0311bbb # Sydney 40 | ap-northeast-1: 41 | AMI: ami-0f9ae750e8274075b # Tokyo 42 | ca-central-1: 43 | AMI: ami-03338e1f67dae0168 # Montreal 44 | eu-central-1: 45 | AMI: ami-09def150731bdbcc2 # Frankfurt 46 | eu-west-1: 47 | AMI: ami-07683a44e80cd32c5 # Ireland 48 | eu-west-2: 49 | AMI: ami-09ead922c1dad67e4 # London 50 | eu-west-3: 51 | AMI: ami-0451ae4fd8dd178f7 # Paris 52 | sa-east-1: 53 | AMI: ami-0669a96e355eac82f # Sao Paulo 54 | Resources: 55 | InstanceRole: 56 | Type: AWS::IAM::Role 57 | Properties: 58 | AssumeRolePolicyDocument: 59 | Version: '2012-10-17' 60 | Statement: 61 | - Effect: Allow 62 | Principal: 63 | Service: ec2.amazonaws.com 64 | Action: sts:AssumeRole 65 | Path: "/" 66 | Policies: 67 | - PolicyName: "K8sClusterPolicy" 68 | PolicyDocument: 69 | Version: "2012-10-17" 70 | Statement: 71 | - Effect: "Allow" 72 | Action: ["iam:*", "ec2:*", "autoscaling:*", "cloudformation:*", "eks:*"] 73 | Resource: "*" 74 | ManagedPolicyArns: 75 | - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM 76 | NodeInstanceProfile: 77 | Type: AWS::IAM::InstanceProfile 78 | Properties: 79 | Path: "/" 80 | Roles: 81 | - !Ref InstanceRole 82 | EC2Instance: 83 | Type: 'AWS::EC2::Instance' 84 | Metadata: 85 | Comment: Install application 86 | AWS::CloudFormation::Init: 87 | config: 88 | packages: 89 | yum: 90 | jq: [] 91 | files: 92 | /usr/local/bin/kubectl: 93 | source: https://amazon-eks.s3-us-west-2.amazonaws.com/1.12.7/2019-03-27/bin/linux/amd64/kubectl 94 | mode: 755 95 | owner: root 96 | group: root 97 | /usr/local/bin/aws-iam-authenticator: 98 | source: https://amazon-eks.s3-us-west-2.amazonaws.com/1.12.7/2019-03-27/bin/linux/amd64/aws-iam-authenticator 99 | mode: 755 100 | owner: root 101 | group: root 102 | sources: 103 | /usr/local/bin: https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_Linux_amd64.tar.gz 104 | Properties: 105 | ImageId: !FindInMap [AWSRegionToAMI, !Ref 'AWS::Region', AMI] 106 | InstanceType: !Ref InstanceType 107 | IamInstanceProfile: !Ref NodeInstanceProfile 108 | Tags: 109 | - Key: Name 110 | Value: !Ref AWS::StackName 111 | NetworkInterfaces: 112 | - DeviceIndex: "0" 113 | SubnetId: !Ref SubnetId 114 | AssociatePublicIpAddress: True 115 | UserData: 116 | Fn::Base64: !Sub | 117 | #!/bin/bash 118 | yum install -y aws-cfn-bootstrap 119 | /opt/aws/bin/cfn-init -v --stack '${AWS::StackName}' --resource EC2Instance --region '${AWS::Region}' 120 | /opt/aws/bin/cfn-signal -e $? --stack !Ref '${AWS::StackName}' --resource EC2Instance --region '${AWS::Region}' 121 | 122 | Outputs: 123 | SSHConsole: 124 | Description: Jumpbox SSH console URL 125 | Value: !Sub https://${AWS::Region}.console.aws.amazon.com/systems-manager/session-manager/${EC2Instance}?region=${AWS::Region} 126 | -------------------------------------------------------------------------------- /spot/README.md: -------------------------------------------------------------------------------- 1 | # Amazon EKS结合Amazon EC2 Spot 2 | 3 | 这个章节说明**Amazon EKS**结合**AWS EC2 Spot Instances**的通用做法与最佳实践。 4 | 5 | 6 | 7 | ## 什么是Amazon EC2 Spot 8 | 9 | Amazon EC2 Spot 实例让您可以利用 AWS 云中未使用的 EC2 容量。与按需实例的价格相比，使用 Spot 实例最高可以享受90% 的折扣。您可以将 Spot 实例用于各种无状态、容错或者灵活的应用程序，例如大数据、容器化工作负载、CI/CD、Web服务器、高性能计算 (HPC) 以及其他测试和开发工作负载。Spot 实例与 Auto Scaling、EMR、ECS、CloudFormation、Data Pipeline 和 AWS Batch 等 AWS 服务紧密集成，因此您可以选择如何启动和维护 Spot 实例上运行的应用程序。 10 | 11 | 更多介绍请参考[官方文件](https://aws.amazon.com/cn/ec2/spot/?nc1=f_ls)说明。 12 | 13 | 14 | 15 | ## Amazon EKS如何结合Amazon EC2 Spot 16 | 17 | 由于Kubernetes能维护部署的状态，因此特别适合搭配Spot来打造一个兼具高可用与费用优化的Kubernetes集群，我们会利用**Auto Scaling Group**[多个实例类型与购买选项的新特性](https://docs.aws.amazon.com/zh_cn/autoscaling/ec2/userguide/asg-purchase-options.html)，将一定数量的on-demand实例与spot实例混合在一个**Auto Scaling Group**当中，除了保证具有一定的基线数量(baseline)由on-demand支撑之外，整个集群的横向扩展则由spot与指定比例的on-demand来进行扩展。 18 | 19 | 20 | 21 | ## 范例架构 22 | 23 | 下面这张图说明整个Amazon EKS结合Spot的整体架构 24 | 25 | ![](https://raw.githubusercontent.com/aws-samples/amazon-eks-refarch-cloudformation/master/images/arch-overview.png) 26 | 27 | 28 | 29 | ### 其中包含了几个重点： 30 | 31 | 1. 这里我们用一个ASG(Auto Scaling Group)来创建一个nodegroup，并且注册到Amazon EKS控制平面中(control plane) 32 | 2. 这个ASG由若干数量的on-demand与spot组成 33 | 3. spot一旦面临关机事件，会触发一个关机事件处理机制(termination handling)，提前对spot实例进行node draining，确保上面运行的pod可以提前被重新调度(reschedule)到其他节点。 34 | 35 | 接下来我们就来实际操作 36 | 37 | ## Amazon EKS集群创建 38 | 39 | 我们可以使用eksctl来创建一个混合的集群 40 | 41 | 首先，创建一个`cluster.yaml`文档，内容如下： 42 | 43 | ```yaml 44 | --- 45 | apiVersion: eksctl.io/v1alpha5 46 | kind: ClusterConfig 47 | 48 | metadata: 49 | name: eksdemo 50 | region: ap-northeast-1 51 | 52 | nodeGroups: 53 | - name: ng-1 54 | minSize: 4 55 | maxSize: 8 56 | instancesDistribution: 57 | instanceTypes: ["t3.small", "t3.medium", "t3.large"] # At least two instance types should be specified 58 | onDemandBaseCapacity: 2 59 | onDemandPercentageAboveBaseCapacity: 0 60 | spotInstancePools: 2 61 | ``` 62 | 63 | 接着执行： 64 | 65 | ```bash 66 | $ eksctl create cluster -f cluster.yaml 67 | ``` 68 | 69 | 如此就可以在`ap-northeast-1` region创建一个名为`eksdemo`的EKS集群，同时建立起一个名为`ng-1`的nodegroup，当中由多种instance types组成，其中OnDemand基线(baseline)为2，意味着保证会有两台on-demand实例，而其他的实例则由ondemand+spot组成，但由于我们定义了`onDemandPercentageAboveBaseCapacity`为0, 意味着额外需要的两台实例(minSize-onDemandBaseCapacity)全部由spot来满足。 70 | 71 | 更多相关参数的说明请参考 [AWS文件](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html) 72 | 73 | 更多cluster YAML配置范例请参考 `eksctl` [github](https://github.com/weaveworks/eksctl/tree/master/examples) 74 | 75 | 76 | 77 | ## 使用amazon-eks-refarch模版创建 78 | 79 | 除了使用`eksctl`之外，我们也可以透过[aws-samples/amazon-eks-refarch-cloudformation](https://github.com/aws-samples/amazon-eks-refarch-cloudformation)专案来创建这样的混合集群，**amazon-eks-refarch-cloudformation** 使用100%的cloudformation模版，并且提供丰富的特性。 80 | 81 | https://github.com/aws-samples/amazon-eks-refarch-cloudformation 82 | 83 | 84 | ## node labels, taints and tolerations 85 | 透过eks-templates模版创建的混合集群，会自动将ondemand于spot实例打上不同的node labels，同时也会对spot打上相应的taints，这部分eksctl专案也有对应的[issue](https://github.com/weaveworks/eksctl/issues/929)，预计很快也能实现。 86 | 87 | 88 | ## 验证集群 89 | 90 | 透过`kubectl get no —show-labels`列出所有节点，可以看到ondemand与spot各自带上了不同的`lifecycle` label。 91 | 92 | ![](images/01.png) 93 | 94 | 因此可以透过label select的方式列出指定label的节点。 95 | 96 | ![](images/02.png) 97 | 98 | 打开EC2 console，查看Auto Scaling Group，会看到相应的配置。 99 | 100 | ![](images/03.png) 101 | 102 | ## 部署eks-lambda-drainer 103 | 104 | Spot实例当面临资源回收，强迫被关机的时候，系统会提前两分钟收到通知，我们可以借由CloudWatch Event捕捉到这个通知信息出发一个外部调度的Lambda function对这个节点进行node draining。 eks-lambda-drainer专案可以帮助我们部署一个完全serverless独立于Kubernetes集群之外的无服务器时间响应handler，并且监听整个VPC内的spot关机信号，一旦提前两分钟获得关机信号就会对这个节点进行`kubectl drain`操作，确保节点上吗的Pod能被及时重新调度到其他节点上运行。 105 | 106 | ![](https://raw.github.com/pahud/eks-lambda-drainer/master/images/eks-lambda-drainer.png) 107 | 108 | 关于**eks-lambda-drainer**的部署方式，请参考Github 109 | 110 | https://github.com/pahud/eks-lambda-drainer 111 | 112 | 113 | 114 | ## 测试spot关机事件处理 115 | 116 | 一旦**eks-lambda-drainer**部署完成，当spot准备关机的时候，eks-lambda-drainer会开始进行相应处理，从Lambda log可以看到如下的信息，表示确实对node节点进行了kubectl drain的操作。 117 | 118 | ![](https://raw.github.com/pahud/eks-lambda-drainer/master/images/11.png) 119 | 120 | ![](https://raw.github.com/pahud/eks-lambda-drainer/master/images/12.png) 121 | 122 | 123 | 124 | ## 结语 125 | 126 | Amazon EKS结合Amazon EC2 Spot是一个非常好的组合，常见的部署模式是把重要的controller或agent指定部署到on demand节点，而其他应用则部署到spot节点，甚至也可以全部都部署到spot节点。只要在ASG里面选择多种节点类型，并且同时选择至少三个AZ，如此ASG就会尽可能维护所需要的节点数量保证一定的高可用性。 127 | 128 | 129 | 130 | ## 公开案例 131 | 132 | 知名在线旅游服务商**Skyscanner**在**This is My Architecture**系列影片当中揭露了他们在AWS上面构建Kubernetes集群，横跨全球多个地理区，多个集群，尖峰时间支撑每秒60-75K QPS，以及每个月8千万个月独立用户，**全部100%使用EC2 Spot构建K8S节点**。 133 | 134 | **SkyScanner: Building Highly-Available, Multi-Region Kubernetes Clusters on 100% Amazon EC2 Spot** 135 | https://www.youtube.com/watch?v=99nNHsbwBpg 136 | ![](https://i.ytimg.com/vi/99nNHsbwBpg/maxresdefault.jpg) 137 | 138 | -------------------------------------------------------------------------------- /spot/images/01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/spot/images/01.png -------------------------------------------------------------------------------- /spot/images/02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/spot/images/02.png -------------------------------------------------------------------------------- /spot/images/03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/walkley/eks-workshop/45fe9df97a90d18817099b05d0c8b6423bfd8934/spot/images/03.png --------------------------------------------------------------------------------