├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── amazon-eks-ami.tf ├── diagrams ├── amazon-eks-on-aws-outposts-architecture.svg └── amazon-eks-self-managed-node-group.svg ├── examples ├── outposts │ ├── .terraform.lock.hcl │ ├── README.md │ ├── aws-auth-config-map.tf │ ├── providers.tf │ └── self-managed-node-group.tf └── region │ ├── .terraform.lock.hcl │ ├── aws-auth-config-map.tf │ ├── providers.tf │ └── self-managed-node-group.tf ├── iam.tf ├── iam_ec2_assume_role_policy.json ├── main.tf ├── outputs.tf ├── providers.tf ├── user_data.sh └── variables.tf /.gitignore: -------------------------------------------------------------------------------- 1 | # Local .terraform directories 2 | **/.terraform/* 3 | 4 | # .tfstate files 5 | *.tfstate 6 | *.tfstate.* 7 | 8 | # Crash log files 9 | crash.log 10 | 11 | # Exclude all .tfvars files, which are likely to contain sentitive data, such as 12 | # password, private keys, and other secrets. These should not be part of version 13 | # control as they are data points which are potentially sensitive and subject 14 | # to change depending on the environment. 15 | # 16 | *.tfvars 17 | 18 | # Ignore override files as they are usually used to override resources locally and so 19 | # are not checked in 20 | override.tf 21 | override.tf.json 22 | *_override.tf 23 | *_override.tf.json 24 | 25 | # Include override files you do wish to add to version control using negated pattern 26 | # 27 | # !example_override.tf 28 | 29 | # Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan 30 | # example: *tfplan* 31 | 32 | # Ignore CLI configuration files 33 | .terraformrc 34 | terraform.rc 35 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazon EKS Self-Managed Node Group Terraform Module 2 | 3 | *Create [Amazon Elastic Kubernetes Service (Amazon EKS)](https://aws.amazon.com/eks) self-managed node groups on AWS using [HashiCorp Terraform](https://www.hashicorp.com/products/terraform).* 4 | 5 | This Terraform module is as a simple example that illustrates the AWS resources involved in deploying Amazon EKS self-managed node groups. You can deploy Amazon EKS self-managed node groups in AWS Regions and on [AWS Outposts](https://aws.amazon.com/outposts/). 6 | 7 | There is a more sophisticated [terraform-aws-eks](https://registry.terraform.io/modules/terraform-aws-modules/eks/aws) module on the Terraform registry, which allows you to create and manage Amazon EKS clusters along with managed and self-managed node groups and Fargate profiles. The terraform-aws-eks module's [code](https://github.com/terraform-aws-modules/terraform-aws-eks) is more complex to read and understand. With that complexity, the registry module provides more functionality and flexibility than this sample code module. 8 | 9 | You should using this sample code module for learning and educational purposes. Once you understand how to construct self-managed node groups from the basic AWS resources, you can create your own Terraform configurations or use the terraform-aws-eks module to save time. 10 | 11 | ## Usage 12 | 13 | This module contains the required resources to deploy an *Amazon EKS self-managed node group* on AWS. 14 | 15 | To allow the nodes to register with your EKS cluster, you will need to configure the [AWS IAM Authenticator](https://github.com/kubernetes-sigs/aws-iam-authenticator) (`aws-auth`) ConfigMap with the node group's IAM role and add the role to the `system:bootstrappers` and `system:nodes` Kubernetes RBAC groups. You can use the [kubernetes]() Terraform provider and the [`kubernetes_config_map`]() resource to manage the `aws-auth` ConfigMap for your cluster (see the [`aws-auth-config-map.tf`]() file in the example configurations). 16 | 17 | ### Deploy an Amazon EKS self-managed node group in an AWS Region 18 | 19 | ```terraform 20 | module "eks_self_managed_node_group" { 21 | source = "github.com/aws-samples/amazon-eks-self-managed-node-group" 22 | 23 | eks_cluster_name = "cmluns-eks-cluster" 24 | instance_type = "m5.2xlarge" 25 | desired_capacity = 2 26 | min_size = 1 27 | max_size = 4 28 | subnets = ["subnet-0aeebfca3d1a6da83", "subnet-0e407d26b34566b16"] # Region subnet(s) 29 | 30 | node_labels = { 31 | "node.kubernetes.io/node-group" = "node-group-a" # (Optional) node-group name label 32 | } 33 | } 34 | ``` 35 | 36 | See the [`examples/region/`](./examples/region/) directory for the full configuration. 37 | 38 | ### Deploy an Amazon EKS self-managed node group on an AWS Outpost 39 | 40 | ```terraform 41 | module "eks_self_managed_node_group" { 42 | source = "../.." 43 | 44 | eks_cluster_name = "cmluns-eks-cluster" 45 | instance_type = "m5.2xlarge" 46 | desired_capacity = 1 47 | min_size = 1 48 | max_size = 1 49 | subnets = ["subnet-0afb721a5cc5bd01f"] # Outposts subnet(s) 50 | 51 | node_labels = { 52 | "node.kubernetes.io/outpost" = "op-0d4579457ff2dc345" # (Optional) Outpost ID label 53 | "node.kubernetes.io/node-group" = "node-group-a" # (Optional) node-group name label 54 | } 55 | 56 | # Outposts require that you encrypt all EBS volumes 57 | ebs_encrypted = true 58 | ebs_kms_key_arn = "arn:aws:kms:us-west-2:799838960553:key/0e8f15cc-d3fc-4da4-ae03-5fadf45cc0fb" 59 | } 60 | ``` 61 | 62 | See the [`examples/outposts/`](./examples/outposts) directory for the full configuration. 63 | 64 | ## Terraform resources 65 | 66 | ![anatomy of an EKS self-managed node group](./diagrams/amazon-eks-self-managed-node-group.svg) 67 | 68 | | Name | Type | 69 | | ---------------------------------------------------------------------------------------------------------------------------------------- | ----------- | 70 | | [aws_autoscaling_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group) | resource | 71 | | [aws_launch_template](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) | resource | 72 | | [aws_iam_instance_profile](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) | resource | 73 | | [aws_iam_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | 74 | | [aws_iam_role_policy_attachment](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | 75 | | [aws_ami](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source | 76 | | [aws_eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source | 77 | | [aws_ec2_instance_type](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ec2_instance_type) | data source | 78 | 79 | ## Module inputs 80 | 81 | ### Required 82 | 83 | - `eks_cluster_name` - The name of the Amazon EKS cluster. 84 | - `instance_type` - The EC2 instance type to use for the worker nodes. 85 | - `desired_capacity` - The desired number of nodes to create in the node group. 86 | - `min_size` - The minimum number of nodes to create in the node group. 87 | - `max_size` - The maximum number of nodes to create in the node group. 88 | - `subnets` - A list of subnet IDs to launch nodes in. Subnets automatically determine which availability zones the node group will reside. 89 | 90 | ### Optional 91 | 92 | - `name` - The name to be used for the self-managed node group. By default, the module will generate a unique name. 93 | - `name_prefix` - Creates a unique name beginning with the specified prefix. Conflicts with `name`. 94 | - `tags` - Tags to apply to all tag-able resources. 95 | - `node_labels` - Kubernetes labels to apply to all nodes in the node group. 96 | - `key_name` - The name of the EC2 key pair to configure on the nodes. 97 | - `security_group_ids` - A list of security group IDs to associate with the worker nodes. The module automatically associates the EKS cluster security group with the nodes. 98 | - `ebs_encrypted` - Enables EBS encryption on the volume. By default, the module uses the setting from the selected AMI. 99 | - `ebs_kms_key_arn` - The ARN of the AWS Key Management Service (AWS KMS) to use when creating the encrypted volume. `encrypted` must be set to true when this is set. 100 | - `ebs_volume_size` - The EBS volume size for a worker node. By default, the module uses the setting from the selected AMI. 101 | - `ebs_volume_type` - The EBS volume type for a worker node. By default, the module uses the setting from the selected AMI. 102 | - `ebs_iops` - The amount of provisioned IOPS for a worker node. This must be set with an `ebs_volume_type` of `io1` or `io2`. 103 | - `ebs_throughput` - The throughput to provision for a `gp3` volume in MiB/s (specified as an integer). 104 | - `ebs_delete_on_termination` - Whether the worker node EBS volumes should be destroyed on instance termination. By default, the module uses the setting from the selected AMI. 105 | 106 | ## Module outputs 107 | 108 | - `name` - The full name of the self-managed node group. 109 | - `role_arn` - The ARN of the node group IAM role. 110 | - `ami_id` - The ID of the selected Amazon EKS optimized AMI. 111 | - `ami_name` - The name of the selected Amazon EKS optimized AMI. 112 | - `ami_description` - The description of the selected Amazon EKS optimized AMI. 113 | - `ami_creation_date` - The creation date of the selected Amazon EKS optimized AMI. 114 | 115 | 116 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 117 | -------------------------------------------------------------------------------- /amazon-eks-ami.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # Most recent optimizaed Amazon EKS AMI 5 | data "aws_ami" "selected_eks_optimized_ami" { 6 | owners = ["amazon"] 7 | most_recent = true 8 | 9 | filter { 10 | name = "name" 11 | values = ["amazon-eks-node-${data.aws_eks_cluster.selected.version}*"] 12 | } 13 | 14 | filter { 15 | name = "architecture" 16 | values = data.aws_ec2_instance_type.selected.supported_architectures 17 | } 18 | 19 | filter { 20 | name = "virtualization-type" 21 | values = data.aws_ec2_instance_type.selected.supported_virtualization_types 22 | } 23 | 24 | filter { 25 | name = "root-device-type" 26 | values = data.aws_ec2_instance_type.selected.supported_root_device_types 27 | } 28 | } 29 | -------------------------------------------------------------------------------- /diagrams/amazon-eks-on-aws-outposts-architecture.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 |
AWS Cloud
AWS Cloud
Region
Region
VPC
VPC
Corporate data center
Corporate data centerAvailability Zone
Private subnet
Private subnetAvailability Zone
Private subnet
Private subnet
Service Link
Service Link
AWS Outpost
AWS Outpost
Private subnet
Private subnet
kubectl
kubectlOn-premises
Self-Managed
Node Group
Self-Managed...
Local Gateway
(LGW)
Local...
Applications
App...
Clients
Clien...
Databases
Dat...
Private EKS endpoint
Privat...
Private EKS endpoint
Privat...
Public subnet
Public subnet
Public subnet
Public subnet
NAT Gateway
(NAT)
NAT Ga...
NAT Gateway
(NAT)
NAT Ga...
Amazon EKS
Amazon EKS
Public endpoint
Public endpoint
Kubernetes
Master Nodes
Kubernetes...
API
Servers
API...
control plane
control planeOther AWS resourceson AWS Outposts
EC2
EC2
RDS
RDS
S3
S3
Internet Gateway
(IGW)
Intern...
Amazon Elastic
Container Registry
(ECR)
Amazon E...
AWS Key
Management Service
(KMS)
AWS Key...
AWS Identity and
Access Management
(IAM)
AWS Iden...Viewer does not support full SVG 1.1 -------------------------------------------------------------------------------- /diagrams/amazon-eks-self-managed-node-group.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 |
Data
Source
Data...
EC2 Auto Scaling group
EC2 Auto Scaling group
Amazon EKS Cluster
Amazon...
Launch Template
Launc...
IAM Role
IAM R...
Amazon EC2
Instance Type
Amazo...
Amazon EKS
Optimized AMI
Amazo...
User Data Script
User...
Tags
Tags
KMS Key
KMS...
Amazon EKS
Worker Node Policy
Amaz...
Amazon EKS
CNI Policy
Amaz...
Amazon EC2
Container Registry
Read Only
Amaz...
EKS self-managed node group
EKS s...
VPC Zone Identifier
VPC Zone Identifier
IAM Instance Profile
IAM I...
Security Group(s)
Secur...
kubernetes.io/cluster/<cluster-name> = owned
kubernetes.io/cluster/<cluster-name> = owned
[
[
Subnet
Subnet
]
]Viewer does not support full SVG 1.1 -------------------------------------------------------------------------------- /examples/outposts/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "3.46.0" 6 | constraints = "~> 3.27" 7 | hashes = [ 8 | "h1:aChsFp4Zfrt0z+6IEbQYHWqrQYbDPpBd59bX2I6QK1w=", 9 | "zh:3ec89dba1d9ed494c5a8069b98d230289c736f5d7abb0d47d6d657d1c9a22a38", 10 | "zh:47dd0ba54897a43aa22a9009d9eddec30d2e656a6153219335af23c5be609e47", 11 | "zh:482164d6d7782d574d6ef3740d02a3b3566c9e3f03021b497675aa4aa6855ef9", 12 | "zh:5b068dd406e0989cb1b1ce390b8dc33eb77997a594b500dea3d39595e67086b3", 13 | "zh:7bb6dbe99cd483db05d28e0e3109dac6be233961f816b1145035f0f49b30bbde", 14 | "zh:7c245831b5e062b0207b988821d6ed674516c78b81afe0fc015a58e40b973d05", 15 | "zh:7f3fb2457ff59c6e3795acd0995cb3ec3b0f22fce5ab8b261e8480bc752787a6", 16 | "zh:8dcbb64802f38dc20fccedaf93dbfbf367859eba81fe7fa4dc734323f287cf4a", 17 | "zh:da6c412927a514e46ff81e4044ce29617b7c11d33db99ff959a761f97ca09fce", 18 | "zh:e670cda0e9ffcd791d94bb1822c26e2a1d26cb0e7a7b655019f4375a14e04e90", 19 | "zh:ebf9c5ef3eceebc1c21bcd31e535e5c323c3bf6ca5918959e297e9a6617d8094", 20 | ] 21 | } 22 | 23 | provider "registry.terraform.io/hashicorp/kubernetes" { 24 | version = "1.13.4" 25 | constraints = "~> 1.13.3" 26 | hashes = [ 27 | "h1:g7jKtArUeyDDevYvh+Xx2U6XIS90WgBCjfbWTrfbBgk=", 28 | "zh:0658034b1b0e241f6d6fc8dac2073755dcbab8f82645c0a46cec052469c518b2", 29 | "zh:11a08ffa9b86670711cb8f2754ac8034b0cdf3d9bad4f3c22695f749a892c630", 30 | "zh:3e90e15a58f699f22bcbe27d3cf45064f9e1a2f1fb50992afc6ea55a59100d4c", 31 | "zh:5e5a335655e40ceb4576af3790aead62646942972c206f49a3dc52275d925f11", 32 | "zh:6bbf068c35380e75fbd7f5186c37175c6058bd6160d59957a023af3e4c9f43c5", 33 | "zh:6bd839cce4ce786201b3d0d43b6ad80e3bf9642f74b1490b9cf72ca8d8c90575", 34 | "zh:804ba2f1d03f315b071434fd7201eeb1e705fcb82f9a1dc4bec760e4231becfa", 35 | "zh:957963a9f287589836a56be24bb9a172919f5a3f18098adb9f185f2a6699680b", 36 | "zh:b099aea7f5213450f3b0d4e439aeb83aba965920b89474aa94f2bc0d6f698fe7", 37 | "zh:b8d610a387f0df4b4c5c27b9319749d1bf60b01c69ea65d2d129c2a61afa0c7b", 38 | "zh:cbf56221840b360befc00fe2336a9236d1ff0f32456453030ed6f58b49deb8df", 39 | ] 40 | } 41 | 42 | provider "registry.terraform.io/hashicorp/random" { 43 | version = "3.1.0" 44 | constraints = "~> 3.1.0" 45 | hashes = [ 46 | "h1:rKYu5ZUbXwrLG1w81k7H3nce/Ys6yAxXhWcbtk36HjY=", 47 | "zh:2bbb3339f0643b5daa07480ef4397bd23a79963cc364cdfbb4e86354cb7725bc", 48 | "zh:3cd456047805bf639fbf2c761b1848880ea703a054f76db51852008b11008626", 49 | "zh:4f251b0eda5bb5e3dc26ea4400dba200018213654b69b4a5f96abee815b4f5ff", 50 | "zh:7011332745ea061e517fe1319bd6c75054a314155cb2c1199a5b01fe1889a7e2", 51 | "zh:738ed82858317ccc246691c8b85995bc125ac3b4143043219bd0437adc56c992", 52 | "zh:7dbe52fac7bb21227acd7529b487511c91f4107db9cc4414f50d04ffc3cab427", 53 | "zh:a3a9251fb15f93e4cfc1789800fc2d7414bbc18944ad4c5c98f466e6477c42bc", 54 | "zh:a543ec1a3a8c20635cf374110bd2f87c07374cf2c50617eee2c669b3ceeeaa9f", 55 | "zh:d9ab41d556a48bd7059f0810cf020500635bfc696c9fc3adab5ea8915c1d886b", 56 | "zh:d9e13427a7d011dbd654e591b0337e6074eef8c3b9bb11b2e39eaaf257044fd7", 57 | "zh:f7605bd1437752114baf601bdf6931debe6dc6bfe3006eb7e9bb9080931dca8a", 58 | ] 59 | } 60 | -------------------------------------------------------------------------------- /examples/outposts/README.md: -------------------------------------------------------------------------------- 1 | # Amazon EKS on AWS Outposts 2 | 3 | You deploy Amazon EKS worker nodes to Outposts *using self-managed node groups*. The worker nodes run on Outposts and register with the Kubernetes control plane in the AWS Region. The worker nodes, and containers running on the nodes, can communicate with AWS services and resources running on the Outpost and in the region (via the Service Link) and with on-premises networks (via the Local Gateway). 4 | 5 | ![Amazon EKS on AWS Outposts architecture](../../diagrams/amazon-eks-on-aws-outposts-architecture.svg) 6 | 7 | You use the same AWS and Kubernetes tools and APIs to work with EKS on Outposts nodes that you use to work with EKS nodes in the Region. 8 | 9 | ## Additional requirements 10 | 11 | To deploy Amazon EKS worker nodes on AWS Outposts, you must: 12 | 13 | * Encrypt all EBS volumes with an [AWS Key Management Service (KMS)](https://aws.amazon.com/kms/) key (you can use the `alias/aws/ebs` AWS managed key) 14 | * Launch worker node instances in Outposts VPC subnets 15 | -------------------------------------------------------------------------------- /examples/outposts/aws-auth-config-map.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # ----------------------------------------------------------------------------- 5 | # Configure Kubernetes to permit the nodes to register 6 | # ----------------------------------------------------------------------------- 7 | data "aws_eks_cluster" "selected" { 8 | name = "cmluns-eks-cluster" 9 | } 10 | 11 | data "aws_eks_cluster_auth" "selected" { 12 | name = "cmluns-eks-cluster" 13 | } 14 | 15 | provider "kubernetes" { 16 | load_config_file = false 17 | host = data.aws_eks_cluster.selected.endpoint 18 | cluster_ca_certificate = base64decode(data.aws_eks_cluster.selected.certificate_authority[0].data) 19 | token = data.aws_eks_cluster_auth.selected.token 20 | } 21 | 22 | resource "kubernetes_config_map" "aws_auth" { 23 | metadata { 24 | name = "aws-auth" 25 | namespace = "kube-system" 26 | } 27 | 28 | data = { 29 | mapRoles = <<-EOT 30 | - rolearn: ${module.eks_self_managed_node_group.role_arn} 31 | username: system:node:{{EC2PrivateDNSName}} 32 | groups: 33 | - system:bootstrappers 34 | - system:nodes 35 | EOT 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /examples/outposts/providers.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | terraform { 5 | required_providers { 6 | aws = { 7 | source = "hashicorp/aws" 8 | version = "~> 3.27" 9 | } 10 | 11 | kubernetes = { 12 | source = "hashicorp/kubernetes" 13 | version = "~> 1.13.3" 14 | } 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /examples/outposts/self-managed-node-group.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # ----------------------------------------------------------------------------- 5 | # Deploy a self-managed node group on an Outpost 6 | # ----------------------------------------------------------------------------- 7 | provider "aws" { 8 | region = "us-west-2" 9 | profile = "aws-outposts-dev+sea19.04-Admin" 10 | } 11 | 12 | module "eks_self_managed_node_group" { 13 | source = "../.." 14 | 15 | eks_cluster_name = "cmluns-eks-cluster" 16 | instance_type = "m5.2xlarge" 17 | desired_capacity = 1 18 | min_size = 1 19 | max_size = 1 20 | subnets = ["subnet-0afb721a5cc5bd01f"] # Outposts subnet(s) 21 | 22 | node_labels = { 23 | "node.kubernetes.io/outpost" = "op-0d4579457ff2dc345" # (Optional) Outpost ID label 24 | "node.kubernetes.io/node-group" = "node-group-a" # (Optional) node-group name label 25 | } 26 | 27 | # Outposts require that you encrypt all EBS volumes 28 | ebs_encrypted = true 29 | ebs_kms_key_arn = "arn:aws:kms:us-west-2:799838960553:key/0e8f15cc-d3fc-4da4-ae03-5fadf45cc0fb" 30 | } 31 | -------------------------------------------------------------------------------- /examples/region/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "3.46.0" 6 | constraints = "~> 3.27" 7 | hashes = [ 8 | "h1:aChsFp4Zfrt0z+6IEbQYHWqrQYbDPpBd59bX2I6QK1w=", 9 | "zh:3ec89dba1d9ed494c5a8069b98d230289c736f5d7abb0d47d6d657d1c9a22a38", 10 | "zh:47dd0ba54897a43aa22a9009d9eddec30d2e656a6153219335af23c5be609e47", 11 | "zh:482164d6d7782d574d6ef3740d02a3b3566c9e3f03021b497675aa4aa6855ef9", 12 | "zh:5b068dd406e0989cb1b1ce390b8dc33eb77997a594b500dea3d39595e67086b3", 13 | "zh:7bb6dbe99cd483db05d28e0e3109dac6be233961f816b1145035f0f49b30bbde", 14 | "zh:7c245831b5e062b0207b988821d6ed674516c78b81afe0fc015a58e40b973d05", 15 | "zh:7f3fb2457ff59c6e3795acd0995cb3ec3b0f22fce5ab8b261e8480bc752787a6", 16 | "zh:8dcbb64802f38dc20fccedaf93dbfbf367859eba81fe7fa4dc734323f287cf4a", 17 | "zh:da6c412927a514e46ff81e4044ce29617b7c11d33db99ff959a761f97ca09fce", 18 | "zh:e670cda0e9ffcd791d94bb1822c26e2a1d26cb0e7a7b655019f4375a14e04e90", 19 | "zh:ebf9c5ef3eceebc1c21bcd31e535e5c323c3bf6ca5918959e297e9a6617d8094", 20 | ] 21 | } 22 | 23 | provider "registry.terraform.io/hashicorp/kubernetes" { 24 | version = "1.13.4" 25 | constraints = "~> 1.13.3" 26 | hashes = [ 27 | "h1:g7jKtArUeyDDevYvh+Xx2U6XIS90WgBCjfbWTrfbBgk=", 28 | "zh:0658034b1b0e241f6d6fc8dac2073755dcbab8f82645c0a46cec052469c518b2", 29 | "zh:11a08ffa9b86670711cb8f2754ac8034b0cdf3d9bad4f3c22695f749a892c630", 30 | "zh:3e90e15a58f699f22bcbe27d3cf45064f9e1a2f1fb50992afc6ea55a59100d4c", 31 | "zh:5e5a335655e40ceb4576af3790aead62646942972c206f49a3dc52275d925f11", 32 | "zh:6bbf068c35380e75fbd7f5186c37175c6058bd6160d59957a023af3e4c9f43c5", 33 | "zh:6bd839cce4ce786201b3d0d43b6ad80e3bf9642f74b1490b9cf72ca8d8c90575", 34 | "zh:804ba2f1d03f315b071434fd7201eeb1e705fcb82f9a1dc4bec760e4231becfa", 35 | "zh:957963a9f287589836a56be24bb9a172919f5a3f18098adb9f185f2a6699680b", 36 | "zh:b099aea7f5213450f3b0d4e439aeb83aba965920b89474aa94f2bc0d6f698fe7", 37 | "zh:b8d610a387f0df4b4c5c27b9319749d1bf60b01c69ea65d2d129c2a61afa0c7b", 38 | "zh:cbf56221840b360befc00fe2336a9236d1ff0f32456453030ed6f58b49deb8df", 39 | ] 40 | } 41 | 42 | provider "registry.terraform.io/hashicorp/random" { 43 | version = "3.1.0" 44 | constraints = "~> 3.1.0" 45 | hashes = [ 46 | "h1:rKYu5ZUbXwrLG1w81k7H3nce/Ys6yAxXhWcbtk36HjY=", 47 | "zh:2bbb3339f0643b5daa07480ef4397bd23a79963cc364cdfbb4e86354cb7725bc", 48 | "zh:3cd456047805bf639fbf2c761b1848880ea703a054f76db51852008b11008626", 49 | "zh:4f251b0eda5bb5e3dc26ea4400dba200018213654b69b4a5f96abee815b4f5ff", 50 | "zh:7011332745ea061e517fe1319bd6c75054a314155cb2c1199a5b01fe1889a7e2", 51 | "zh:738ed82858317ccc246691c8b85995bc125ac3b4143043219bd0437adc56c992", 52 | "zh:7dbe52fac7bb21227acd7529b487511c91f4107db9cc4414f50d04ffc3cab427", 53 | "zh:a3a9251fb15f93e4cfc1789800fc2d7414bbc18944ad4c5c98f466e6477c42bc", 54 | "zh:a543ec1a3a8c20635cf374110bd2f87c07374cf2c50617eee2c669b3ceeeaa9f", 55 | "zh:d9ab41d556a48bd7059f0810cf020500635bfc696c9fc3adab5ea8915c1d886b", 56 | "zh:d9e13427a7d011dbd654e591b0337e6074eef8c3b9bb11b2e39eaaf257044fd7", 57 | "zh:f7605bd1437752114baf601bdf6931debe6dc6bfe3006eb7e9bb9080931dca8a", 58 | ] 59 | } 60 | -------------------------------------------------------------------------------- /examples/region/aws-auth-config-map.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # ----------------------------------------------------------------------------- 5 | # Configure Kubernetes to permit the nodes to register 6 | # ----------------------------------------------------------------------------- 7 | data "aws_eks_cluster" "selected" { 8 | name = "cmluns-eks-cluster" 9 | } 10 | 11 | data "aws_eks_cluster_auth" "selected" { 12 | name = "cmluns-eks-cluster" 13 | } 14 | 15 | provider "kubernetes" { 16 | load_config_file = false 17 | host = data.aws_eks_cluster.selected.endpoint 18 | cluster_ca_certificate = base64decode(data.aws_eks_cluster.selected.certificate_authority[0].data) 19 | token = data.aws_eks_cluster_auth.selected.token 20 | } 21 | 22 | resource "kubernetes_config_map" "aws_auth" { 23 | metadata { 24 | name = "aws-auth" 25 | namespace = "kube-system" 26 | } 27 | 28 | data = { 29 | mapRoles = <<-EOT 30 | - rolearn: ${module.eks_self_managed_node_group.role_arn} 31 | username: system:node:{{EC2PrivateDNSName}} 32 | groups: 33 | - system:bootstrappers 34 | - system:nodes 35 | EOT 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /examples/region/providers.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | terraform { 5 | required_providers { 6 | aws = { 7 | source = "hashicorp/aws" 8 | version = "~> 3.27" 9 | } 10 | 11 | kubernetes = { 12 | source = "hashicorp/kubernetes" 13 | version = "~> 1.13.3" 14 | } 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /examples/region/self-managed-node-group.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # ----------------------------------------------------------------------------- 5 | # Deploy a self-managed node group in an AWS Region 6 | # ----------------------------------------------------------------------------- 7 | provider "aws" { 8 | region = "us-west-2" 9 | profile = "aws-outposts-dev+sea19.04-Admin" 10 | } 11 | 12 | module "eks_self_managed_node_group" { 13 | source = "../.." 14 | 15 | eks_cluster_name = "cmluns-eks-cluster" 16 | instance_type = "m5.2xlarge" 17 | desired_capacity = 2 18 | min_size = 1 19 | max_size = 4 20 | subnets = ["subnet-0aeebfca3d1a6da83", "subnet-0e407d26b34566b16"] # Region subnet(s) 21 | 22 | node_labels = { 23 | "node.kubernetes.io/node-group" = "node-group-a" # (Optional) node-group name label 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /iam.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | resource "aws_iam_role" "eks_self_managed_node_group" { 5 | name = "${var.eks_cluster_name}-${local.node_group_name}-role" 6 | assume_role_policy = file("${path.module}/iam_ec2_assume_role_policy.json") 7 | tags = var.tags 8 | } 9 | 10 | resource "aws_iam_role_policy_attachment" "amazon_eks_worker_node_policy" { 11 | policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy" 12 | role = aws_iam_role.eks_self_managed_node_group.name 13 | } 14 | 15 | resource "aws_iam_role_policy_attachment" "amazon_eks_cni_policy" { 16 | policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy" 17 | role = aws_iam_role.eks_self_managed_node_group.name 18 | } 19 | 20 | resource "aws_iam_role_policy_attachment" "amazon_ec2_container_registry_read_only" { 21 | policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly" 22 | role = aws_iam_role.eks_self_managed_node_group.name 23 | } 24 | 25 | resource "aws_iam_instance_profile" "eks_self_managed_node_group" { 26 | name = "${var.eks_cluster_name}-${local.node_group_name}-instance-profile" 27 | role = aws_iam_role.eks_self_managed_node_group.name 28 | tags = var.tags 29 | } 30 | -------------------------------------------------------------------------------- /iam_ec2_assume_role_policy.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Principal": {"Service": "ec2.amazonaws.com"}, 7 | "Action": "sts:AssumeRole" 8 | } 9 | ] 10 | } 11 | -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # Amazon EKS cluster data 5 | data "aws_eks_cluster" "selected" { 6 | name = var.eks_cluster_name 7 | } 8 | 9 | 10 | # EC2 instance type data 11 | data "aws_ec2_instance_type" "selected" { 12 | instance_type = var.instance_type 13 | } 14 | 15 | 16 | resource "aws_autoscaling_group" "eks_self_managed_node_group" { 17 | name = "${var.eks_cluster_name}-${local.node_group_name}" 18 | 19 | desired_capacity = var.desired_capacity 20 | min_size = var.min_size 21 | max_size = var.max_size 22 | 23 | vpc_zone_identifier = var.subnets 24 | 25 | launch_template { 26 | id = aws_launch_template.eks_self_managed_nodes.id 27 | version = "$Latest" 28 | } 29 | 30 | tags = concat( 31 | [ 32 | for tag, value in var.tags : { 33 | key = tag 34 | value = value 35 | propagate_at_launch = true 36 | } 37 | ], 38 | [ 39 | { 40 | key = "Name" 41 | value = "${var.eks_cluster_name}-${local.node_group_name}" 42 | propagate_at_launch = true 43 | }, 44 | { 45 | key = "kubernetes.io/cluster/${var.eks_cluster_name}" 46 | value = "owned" 47 | propagate_at_launch = true 48 | }, 49 | ] 50 | ) 51 | 52 | 53 | # Ensure the IAM role has been created (and the policies have been attached) 54 | # before creating the auto-scaling group. 55 | depends_on = [ 56 | aws_iam_role_policy_attachment.amazon_eks_worker_node_policy, 57 | aws_iam_role_policy_attachment.amazon_eks_cni_policy, 58 | aws_iam_role_policy_attachment.amazon_ec2_container_registry_read_only, 59 | ] 60 | } 61 | 62 | 63 | resource "aws_launch_template" "eks_self_managed_nodes" { 64 | name_prefix = "${var.eks_cluster_name}-${local.node_group_name}" 65 | description = "Amazon EKS self-managed nodes" 66 | 67 | instance_type = var.instance_type 68 | image_id = data.aws_ami.selected_eks_optimized_ami.id 69 | ebs_optimized = data.aws_ec2_instance_type.selected.ebs_optimized_support == "default" ? true : false 70 | key_name = var.key_name 71 | update_default_version = true 72 | 73 | vpc_security_group_ids = concat( 74 | [data.aws_eks_cluster.selected.vpc_config[0].cluster_security_group_id], 75 | var.security_group_ids 76 | ) 77 | 78 | iam_instance_profile { 79 | arn = aws_iam_instance_profile.eks_self_managed_node_group.arn 80 | } 81 | 82 | block_device_mappings { 83 | device_name = local.root_block_device_mapping.device_name 84 | 85 | ebs { 86 | snapshot_id = local.root_block_device_mapping.ebs.snapshot_id 87 | 88 | encrypted = coalesce(var.ebs_encrypted, local.root_block_device_mapping.ebs.encrypted) 89 | kms_key_id = var.ebs_kms_key_arn 90 | 91 | volume_size = coalesce(var.ebs_volume_size, local.root_block_device_mapping.ebs.volume_size) 92 | volume_type = coalesce(var.ebs_volume_type, local.root_block_device_mapping.ebs.volume_type) 93 | iops = contains(["io1", "io2"], var.ebs_volume_type) ? var.ebs_iops : null 94 | throughput = var.ebs_volume_type == "gp3" ? var.ebs_throughput : null 95 | 96 | delete_on_termination = coalesce(var.ebs_delete_on_termination, local.root_block_device_mapping.ebs.delete_on_termination) 97 | } 98 | } 99 | 100 | user_data = base64encode(templatefile("${path.module}/user_data.sh", { 101 | cluster_name = var.eks_cluster_name 102 | node_labels = var.node_labels 103 | })) 104 | 105 | tags = var.tags 106 | } 107 | 108 | locals { 109 | ami_block_device_mappings = { 110 | for bdm in data.aws_ami.selected_eks_optimized_ami.block_device_mappings : bdm.device_name => bdm 111 | } 112 | root_block_device_mapping = local.ami_block_device_mappings[data.aws_ami.selected_eks_optimized_ami.root_device_name] 113 | } 114 | -------------------------------------------------------------------------------- /outputs.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | output "name" { 5 | value = "${var.eks_cluster_name}-${local.node_group_name}" 6 | } 7 | 8 | output "role_arn" { 9 | value = aws_iam_role.eks_self_managed_node_group.arn 10 | } 11 | 12 | output "ami_id" { 13 | value = data.aws_ami.selected_eks_optimized_ami.id 14 | } 15 | 16 | output "ami_name" { 17 | value = data.aws_ami.selected_eks_optimized_ami.name 18 | } 19 | 20 | output "ami_description" { 21 | value = data.aws_ami.selected_eks_optimized_ami.description 22 | } 23 | 24 | output "ami_creation_date" { 25 | value = data.aws_ami.selected_eks_optimized_ami.creation_date 26 | } 27 | -------------------------------------------------------------------------------- /providers.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | terraform { 5 | required_providers { 6 | aws = { 7 | source = "hashicorp/aws" 8 | version = "~> 3.27" 9 | } 10 | 11 | random = { 12 | source = "hashicorp/random" 13 | version = "~> 3.1.0" 14 | } 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /user_data.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | # SPDX-License-Identifier: MIT-0 4 | 5 | set -o xtrace 6 | systemctl stop kubelet 7 | /etc/eks/bootstrap.sh \ 8 | --kubelet-extra-args '--node-labels=${join(",", [for label, value in node_labels : "${label}=${value}"])}' \ 9 | ${cluster_name} 10 | -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | // SPDX-License-Identifier: MIT-0 3 | 4 | # ----------------------------------------------------------------------------- 5 | # Required input variables 6 | # ----------------------------------------------------------------------------- 7 | variable "eks_cluster_name" { 8 | type = string 9 | description = "(Required) The name of the Amazon EKS cluster." 10 | } 11 | 12 | variable "instance_type" { 13 | type = string 14 | description = "(Required) The EC2 instance type to use for the worker nodes." 15 | } 16 | 17 | variable "desired_capacity" { 18 | type = number 19 | description = "(Required) The desired number of nodes to create in the node group." 20 | } 21 | 22 | variable "min_size" { 23 | type = number 24 | description = "(Required) The minimum number of nodes to create in the node group." 25 | } 26 | 27 | variable "max_size" { 28 | type = number 29 | description = "(Required) The maximum number of nodes to create in the node group." 30 | } 31 | 32 | variable "subnets" { 33 | type = list(string) 34 | description = "(Required) A list of subnet IDs to launch nodes in. Subnets automatically determine which availability zones the node group will reside." 35 | } 36 | 37 | 38 | # ----------------------------------------------------------------------------- 39 | # Optional input variables 40 | # ----------------------------------------------------------------------------- 41 | variable "name" { 42 | type = string 43 | description = "(Optional) The name to be used for the self-managed node group. By default, the module will generate a unique name." 44 | default = "" 45 | } 46 | 47 | variable "name_prefix" { 48 | type = string 49 | description = "(Optional) Creates a unique name beginning with the specified prefix. Conflicts with `name`." 50 | default = "node-group" 51 | } 52 | 53 | variable "tags" { 54 | type = map(any) 55 | description = "(Optional) Tags to apply to all tag-able resources." 56 | default = {} 57 | } 58 | 59 | variable "node_labels" { 60 | type = map(any) 61 | description = "(Optional) Kubernetes labels to apply to all nodes in the node group." 62 | default = {} 63 | } 64 | 65 | variable "key_name" { 66 | type = string 67 | description = "(Optional) The name of the EC2 key pair to configure on the nodes." 68 | default = null 69 | } 70 | 71 | variable "security_group_ids" { 72 | type = list(string) 73 | description = "(Optional) A list of security group IDs to associate with the worker nodes. The module automatically associates the EKS cluster security group with the nodes." 74 | default = [] 75 | } 76 | 77 | variable "ebs_encrypted" { 78 | type = bool 79 | description = "(Optional) Enables EBS encryption on the volume. By default, the module uses the setting from the selected AMI." 80 | default = null 81 | } 82 | 83 | variable "ebs_kms_key_arn" { 84 | type = string 85 | description = "(Optional) The ARN of the AWS Key Management Service (AWS KMS) to use when creating the encrypted volume. `encrypted` must be set to true when this is set." 86 | default = null 87 | } 88 | 89 | variable "ebs_volume_size" { 90 | type = number 91 | description = "(Optional) The EBS volume size for a worker node. By default, the module uses the setting from the selected AMI." 92 | default = null 93 | } 94 | 95 | variable "ebs_volume_type" { 96 | type = string 97 | description = "(Optional) The EBS volume type for a worker node. By default, the module uses the setting from the selected AMI." 98 | default = "" 99 | } 100 | 101 | variable "ebs_iops" { 102 | type = number 103 | description = "(Optional) The amount of provisioned IOPS for a worker node. This must be set with an `ebs_volume_type` of `io1` or `io2`." 104 | default = null 105 | } 106 | 107 | variable "ebs_throughput" { 108 | type = number 109 | description = "(Optional) The throughput to provision for a `gp3` volume in MiB/s (specified as an integer)." 110 | default = null 111 | } 112 | 113 | variable "ebs_delete_on_termination" { 114 | type = number 115 | description = "(Optional) Whether the worker node EBS volumes should be destroyed on instance termination. By default, the module uses the setting from the selected AMI." 116 | default = null 117 | } 118 | 119 | 120 | # ----------------------------------------------------------------------------- 121 | # Local variables 122 | # ----------------------------------------------------------------------------- 123 | resource "random_id" "name_suffix" { 124 | byte_length = 8 125 | } 126 | 127 | locals { 128 | node_group_name = coalesce(var.name, "${var.name_prefix}-${random_id.name_suffix.hex}") 129 | } 130 | --------------------------------------------------------------------------------