├── .github └── lockdown.yml ├── .gitignore ├── LICENSE ├── OWNERS ├── README.md ├── example ├── README.md ├── main.tf ├── terraform.tfvars.example └── variables.tf ├── hack └── verify.sh ├── iam.tf ├── main.tf └── variables.tf /.github/lockdown.yml: -------------------------------------------------------------------------------- 1 | # Configuration for Repo Lockdown - https://github.com/dessant/repo-lockdown-app 2 | 3 | # Skip issues and pull requests created before a given timestamp. Timestamp must 4 | # follow ISO 8601 (`YYYY-MM-DD`). Set to `false` to disable 5 | skipCreatedBefore: false 6 | 7 | # Issues and pull requests with these labels will be ignored. Set to `[]` to disable 8 | exemptLabels: [] 9 | 10 | # Comment to post before closing or locking. Set to `false` to disable 11 | comment: This project is now [deprecated](https://github.com/jetstack/terraform-google-gke-cluster#deprecation) so we aren't accepting any further changes. 12 | 13 | # Label to add before closing or locking. Set to `false` to disable 14 | label: false 15 | 16 | # Close issues and pull requests 17 | close: true 18 | 19 | # Lock issues and pull requests 20 | lock: true 21 | 22 | # Limit to only `issues` or `pulls` 23 | # only: issues 24 | 25 | # Optionally, specify configuration settings just for `issues` or `pulls` 26 | # issues: 27 | # label: wontfix 28 | 29 | # pulls: 30 | # comment: > 31 | # This repository does not accept pull requests, see the README for details. 32 | # lock: false 33 | 34 | # Repository to extend settings from 35 | # _extends: repo -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .terraform 2 | terraform.tfvars 3 | terraform.tfstate 4 | terraform.tfstate.backup 5 | key.json 6 | verify-terraform/ 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. -------------------------------------------------------------------------------- /OWNERS: -------------------------------------------------------------------------------- 1 | approvers: 2 | - wwwil 3 | - zee-ahmed 4 | reviewers: 5 | - wwwil 6 | - zee-ahmed 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Terraform Google GKE Cluster 2 | 3 | A Terraform module to create a best-practice Google Kubernetes Engine (GKE) cluster. 4 | 5 | This module is available on [Terraform registry](https://registry.terraform.io/modules/jetstack/gke-cluster/google/). 6 | 7 | The module is designed to be used by Jetstack customers to make it easier for them to create clusters that are secure and follow Jetstack recommendations. 8 | It gives them flexibility with certain properties so the cluster can be customised to their needs, but gives fixed values for properties that could lead to issues or insecurity. 9 | 10 | ## Deprecation 11 | 12 | :warning: 13 | **The `0.3` release of this module is planned to be the final release.** 14 | :warning: 15 | 16 | After this the module will be deprecated in favour of [Google's GKE module](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine). 17 | Jetstack will be producing an example project using Google's module as well as migration guidance. 18 | 19 | ## Requirements 20 | 21 | The module requires an existing Google Cloud project, with VPC network and subnetwork for the cluster to use. 22 | The subnetwork must be in the same region as the cluster and have pod and service ranges specified. 23 | 24 | ## Customisable Properties 25 | 26 | The module allows the cluster to be extensively customised using input variables. 27 | These can be found with documentation in [`variables.tf`](variables.tf). 28 | 29 | The customisable properties include: 30 | - Release channel or minimum master version 31 | - Private nodes 32 | - Master private endpoint 33 | - Master authorised network CIDR blocks 34 | - Master CIDR block 35 | - Node service account container registry access 36 | - Google security group for RBAC 37 | - Workload identity namespace 38 | - Enable Stackdriver logging and monitoring 39 | - Enable Google Cloud HTTP load balancing 40 | - Enable pod security policy controller 41 | - Daily maintenance window start time 42 | - Node pools 43 | - Name 44 | - Inital node count 45 | - Minimum and maximum number of nodes for autoscaling 46 | - Enable automatic repair and upgrade 47 | - Machine type 48 | - Disk size and type 49 | - Use preemptible nodes 50 | - Kubernetes version 51 | 52 | ## Fixed Arguments 53 | 54 | Some of the properties of the cluster are fixed based on Jetstack's recommended best-practice settings: 55 | - Enabling network policy for nodes and master using Calico. 56 | - Disabling basic authentication and client certificate issuing. 57 | - Disabling Kubernetes dashboard (Google Cloud Console should be used instead). 58 | - Use of VPC native networking (using a specified network and subnetwork). 59 | - Removing the default node pool and creating one or more new pools with Terraform for easier management. 60 | - Setting the OAuth scope of nodes to `cloud-platform` to manage permissions with IAM. 61 | - Disabling node legacy endpoints. 62 | - Creating an IAM service account for nodes with the minimum required roles: 63 | - Logging log writer 64 | - Monitoring metric writer 65 | - Monitoring viewer 66 | 67 | ## Usage 68 | 69 | The module itself is located in the root of this repo, and is designed to be used as part of a larger Terraform project. 70 | It can be used directly from the Terraform Registry like so: 71 | 72 | ``` 73 | module "gke-cluster" { 74 | source = "jetstack/gke-cluster/google" 75 | version = "0.3.0" 76 | 77 | # insert the 9 required variables here 78 | } 79 | ``` 80 | 81 | ## Example 82 | 83 | There is an [example project](https://github.com/jetstack/terraform-google-gke-cluster/tree/master/example) in the `example/` directory which can be used to test and demonstrate the module. It could also be used as the basis for your own Terraform project. 84 | 85 | ## Limitations 86 | 87 | If private nodes are used then **nodes will not have direct access to the internet**. 88 | This means they cannot pull images hosted outside of the container registry in the same project as the cluster. 89 | The example project features a [Cloud NAT](https://cloud.google.com/nat/docs/overview) to give the nodes to access the internet. 90 | 91 | ## License 92 | 93 | This project is licensed under the [Apache 2.0 License](https://choosealicense.com/licenses/apache-2.0/). 94 | For full details see the `LICENSE` file. 95 | -------------------------------------------------------------------------------- /example/README.md: -------------------------------------------------------------------------------- 1 | # Terraform Google GKE Cluster Example 2 | 3 | This example project gives the minimum Terraform setup required to use the [`terraform-google-gke-cluster`](https://registry.terraform.io/modules/jetstack/gke-cluster/google/) module implemented in this repo. 4 | 5 | ## Prerequisites 6 | 7 | Check that Terraform is installed and up to date on your machine by running `terraform version`. 8 | At the time of writing the version of binary distributed by HashiCorp is `v0.12.24`. 9 | Installation instructions can be found [here](https://learn.hashicorp.com/terraform/getting-started/install.html). 10 | 11 | This guide uses the Google Cloud Platform (GCP) utility `gcloud`, which is part of the [Cloud SDK](https://cloud.google.com/sdk/). 12 | Installation instructions for this can be found [here](https://cloud.google.com/sdk/install). 13 | The Google Cloud Storage (GCS) command line utility `gsutil` is also used to create and delete storage buckets. 14 | This is installed as part of the Cloud SDK. 15 | Much of what these commands do can be achieved using the GCP Console, but using the commands allows this guide to be concise and precise. 16 | 17 | Once the Cloud SDK is installed you can authenticate, set the GCP project, and choose a compute zone with the interactive command: 18 | 19 | ``` 20 | gcloud init 21 | ``` 22 | 23 | Next ensure the required APIs are enabled in your GCP project: 24 | 25 | ``` 26 | gcloud services enable storage-api.googleapis.com 27 | gcloud services enable cloudresourcemanager.googleapis.com 28 | gcloud services enable compute.googleapis.com 29 | gcloud services enable container.googleapis.com 30 | gcloud services enable iam.googleapis.com 31 | ``` 32 | 33 | **The following guide assumes commands are run from the `example` directory.** 34 | 35 | ## Input Variables 36 | 37 | Values for input variables are set with a `terraform.tfvars` file. 38 | This can be created from the example file provided: 39 | 40 | ``` 41 | cp terraform.tfvars.example terraform.tfvars 42 | ``` 43 | 44 | The values set in this file should be edited according to your environment and requirements. 45 | **You must update the `gcp_project_id` value in this file.** 46 | Other values can be left as default, or adjusted as required. 47 | 48 | ## Service Account 49 | 50 | Terraform needs some [credentials](https://www.terraform.io/docs/providers/google/guides/getting_started.html#adding-credentials) 51 | to make requests against the GCP API. 52 | Using a dedicated GCP service account is recommended as allows permissions to be limited to those required. 53 | Create a service account for Terraform to use, generate a key file for it, and save the key location as an environment variable: 54 | 55 | ``` 56 | PROJECT_ID=[ID OF YOUR PROJECT] 57 | gcloud iam service-accounts create terraform 58 | gcloud projects add-iam-policy-binding ${PROJECT_ID} --member "serviceAccount:terraform@${PROJECT_ID}.iam.gserviceaccount.com" --role "roles/owner" 59 | gcloud iam service-accounts keys create key.json --iam-account terraform@${PROJECT_ID}.iam.gserviceaccount.com 60 | export GOOGLE_APPLICATION_CREDENTIALS="$PWD/key.json" 61 | ``` 62 | 63 | **Keep the key file in a safe place, and do not share or publicise it.** 64 | 65 | ## Remote Terraform State 66 | 67 | Next create a GCS Bucket that will be used to hold Terraform state information. 68 | Using [remote Terraform state](https://www.terraform.io/docs/state/remote.html) 69 | makes it easier to collaborate on Terraform projects. 70 | The storage bucket name must be globally unique, the suggested name is `{PROJECT_ID}-terraform-state`. 71 | After creating the bucket update the IAM permissions to make the `terraform` service account an `objectAdmin`. 72 | 73 | ``` 74 | REGION="europe-west1" 75 | BUCKET_NAME="${PROJECT_ID}-terraform-state" 76 | gsutil mb -l ${REGION} gs://${BUCKET_NAME} 77 | gsutil iam ch serviceAccount:terraform@${PROJECT_ID}.iam.gserviceaccount.com:objectAdmin gs://${BUCKET_NAME} 78 | ``` 79 | 80 | Next, initialise Terraform with the name of the GCS Bucket just created: 81 | 82 | ``` 83 | terraform init -backend-config=bucket=${BUCKET_NAME} 84 | ``` 85 | 86 | ### Applying 87 | 88 | Now that Terraform is setup check that the configuration is valid: 89 | 90 | ``` 91 | terraform validate 92 | ``` 93 | 94 | If the configuration is valid then apply it with: 95 | 96 | ``` 97 | terraform apply 98 | ``` 99 | 100 | Inspect the output of apply to ensure that what Terraform is going to do what you want, if so then enter `yes` at the prompt. 101 | The infrastructure will then be created, this make take a some time. 102 | 103 | ### Clean Up 104 | 105 | Remove the infrastructure created by Terraform with: 106 | 107 | ``` 108 | terraform destroy 109 | ``` 110 | 111 | Sometimes Terraform may report a failure to destroy some resources due to dependencies and timing contention. 112 | In this case wait a few seconds and run the above command again. 113 | If it is still unable to remove everything it may be necessary to remove resources manually using the `gcloud` command or the Cloud Console. 114 | 115 | The GCS Bucket used for Terraform state storage can be removed with: 116 | 117 | ``` 118 | gsutil rm -r gs://${BUCKET_NAME} 119 | ``` 120 | -------------------------------------------------------------------------------- /example/main.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | terraform { 16 | required_version = "~> 0.12" 17 | 18 | # Use a GCS Bucket as a backend 19 | backend "gcs" {} 20 | } 21 | 22 | # Local values assign a name to an expression, that can then be used multiple 23 | # times within a module. They are used here to determine the GCP region from 24 | # the given location, which can be either a region or zone. 25 | locals { 26 | gcp_location_parts = split("-", var.gcp_location) 27 | gcp_region = format("%s-%s", local.gcp_location_parts[0], local.gcp_location_parts[1]) 28 | } 29 | 30 | # https://www.terraform.io/docs/providers/google/index.html 31 | provider "google" { 32 | version = "3.5.0" 33 | project = var.gcp_project_id 34 | region = local.gcp_region 35 | } 36 | 37 | resource "google_compute_network" "vpc_network" { 38 | name = var.vpc_network_name 39 | auto_create_subnetworks = "false" 40 | project = var.gcp_project_id 41 | } 42 | 43 | # https://www.terraform.io/docs/providers/google/r/compute_subnetwork.html 44 | resource "google_compute_subnetwork" "vpc_subnetwork" { 45 | # The name of the resource, provided by the client when initially creating 46 | # the resource. The name must be 1-63 characters long, and comply with 47 | # RFC1035. Specifically, the name must be 1-63 characters long and match the 48 | # regular expression [a-z]([-a-z0-9]*[a-z0-9])? which means the first 49 | # character must be a lowercase letter, and all following characters must be 50 | # a dash, lowercase letter, or digit, except the last character, which 51 | # cannot be a dash. 52 | #name = "default-${var.gcp_cluster_region}" 53 | name = var.vpc_subnetwork_name 54 | region = local.gcp_region 55 | project = var.gcp_project_id 56 | 57 | ip_cidr_range = var.vpc_subnetwork_cidr_range 58 | 59 | # The network this subnet belongs to. Only networks that are in the 60 | # distributed mode can have subnetworks. 61 | network = var.vpc_network_name 62 | 63 | # Configurations for secondary IP ranges for VM instances contained in this 64 | # subnetwork. The primary IP of such VM must belong to the primary ipCidrRange 65 | # of the subnetwork. The alias IPs may belong to either primary or secondary 66 | # ranges. 67 | secondary_ip_range { 68 | range_name = var.cluster_secondary_range_name 69 | ip_cidr_range = var.cluster_secondary_range_cidr 70 | } 71 | secondary_ip_range { 72 | range_name = var.services_secondary_range_name 73 | ip_cidr_range = var.services_secondary_range_cidr 74 | } 75 | 76 | # When enabled, VMs in this subnetwork without external IP addresses can 77 | # access Google APIs and services by using Private Google Access. This is 78 | # set explicitly to prevent Google's default from fighting with Terraform. 79 | private_ip_google_access = true 80 | 81 | depends_on = [ 82 | google_compute_network.vpc_network, 83 | ] 84 | } 85 | 86 | # https://www.terraform.io/docs/providers/google/r/compute_router.html 87 | # This Cloud Router is used only for the Cloud NAT. 88 | resource "google_compute_router" "router" { 89 | # Only create the Cloud NAT if it is enabled. 90 | count = var.enable_cloud_nat ? 1 : 0 91 | name = format("%s-router", var.cluster_name) 92 | region = local.gcp_region 93 | network = google_compute_network.vpc_network.self_link 94 | } 95 | 96 | # https://www.terraform.io/docs/providers/google/r/compute_router_nat.html 97 | resource "google_compute_router_nat" "nat" { 98 | # Only create the Cloud NAT if it is enabled. 99 | count = var.enable_cloud_nat ? 1 : 0 100 | name = format("%s-nat", var.cluster_name) 101 | // Because router has the count attribute set we have to use [0] here to 102 | // refer to its attributes. 103 | router = google_compute_router.router[0].name 104 | region = google_compute_router.router[0].region 105 | # For this example project just use IPs allocated automatically by GCP. 106 | nat_ip_allocate_option = "AUTO_ONLY" 107 | # Apply NAT to all IP ranges in the subnetwork. 108 | source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES" 109 | 110 | log_config { 111 | enable = var.enable_cloud_nat_logging 112 | filter = var.cloud_nat_logging_filter 113 | } 114 | } 115 | 116 | module "cluster" { 117 | source = "jetstack/gke-cluster/google" 118 | version = "0.3.0" 119 | 120 | # These values are set from the terrafrom.tfvas file 121 | gcp_project_id = var.gcp_project_id 122 | cluster_name = var.cluster_name 123 | gcp_location = var.gcp_location 124 | daily_maintenance_window_start_time = var.daily_maintenance_window_start_time 125 | node_pools = var.node_pools 126 | cluster_secondary_range_name = var.cluster_secondary_range_name 127 | services_secondary_range_name = var.services_secondary_range_name 128 | master_ipv4_cidr_block = var.master_ipv4_cidr_block 129 | access_private_images = var.access_private_images 130 | http_load_balancing_disabled = var.http_load_balancing_disabled 131 | master_authorized_networks_cidr_blocks = var.master_authorized_networks_cidr_blocks 132 | private_nodes = var.private_nodes 133 | private_endpoint = var.private_endpoint 134 | pod_security_policy_enabled = var.pod_security_policy_enabled 135 | identity_namespace = var.identity_namespace 136 | 137 | # Refer to the vpc-network and vpc-subnetwork by the name value on the 138 | # resource, rather than the variable used to assign the name, so that 139 | # Terraform knows they must be created before creating the cluster 140 | 141 | vpc_network_name = google_compute_network.vpc_network.name 142 | vpc_subnetwork_name = google_compute_subnetwork.vpc_subnetwork.name 143 | } 144 | -------------------------------------------------------------------------------- /example/terraform.tfvars.example: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | gcp_project_id = "my-project" 16 | 17 | cluster_name = "my-cluster" 18 | 19 | # The location (region or zone) in which the cluster will be created. If you 20 | # specify a zone (such as us-central1-a), the cluster will be a zonal cluster. 21 | # If you specify a region (such as us-west1), the cluster will be a regional 22 | # cluster. 23 | gcp_location = "europe-west1" 24 | 25 | daily_maintenance_window_start_time = "03:00" 26 | 27 | node_pools = [ 28 | { 29 | name = "default" 30 | initial_node_count = 1 31 | autoscaling_min_node_count = 2 32 | autoscaling_max_node_count = 3 33 | management_auto_upgrade = true 34 | management_auto_repair = true 35 | node_config_machine_type = "n1-standard-1" 36 | node_config_disk_type = "pd-standard" 37 | node_config_disk_size_gb = 100 38 | node_config_preemptible = false 39 | }, 40 | ] 41 | 42 | vpc_network_name = "vpc-network" 43 | 44 | vpc_subnetwork_name = "vpc-subnetwork" 45 | 46 | vpc_subnetwork_cidr_range = "10.0.16.0/20" 47 | 48 | cluster_secondary_range_name = "pods" 49 | 50 | cluster_secondary_range_cidr = "10.16.0.0/12" 51 | 52 | services_secondary_range_name = "services" 53 | 54 | services_secondary_range_cidr = "10.1.0.0/20" 55 | 56 | master_ipv4_cidr_block = "172.16.0.0/28" 57 | 58 | access_private_images = "false" 59 | 60 | http_load_balancing_disabled = "false" 61 | 62 | master_authorized_networks_cidr_blocks = [ 63 | { 64 | cidr_block = "0.0.0.0/0" 65 | 66 | display_name = "default" 67 | }, 68 | ] 69 | 70 | identity_namespace = "my-project.svc.id.goog" 71 | -------------------------------------------------------------------------------- /example/variables.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | variable "gcp_project_id" { 16 | type = string 17 | 18 | description = <=0 and <= max_node_count. Defaults to 2. 71 | 72 | autoscaling_max_node_count - Maximum number of nodes in the NodePool. Must be 73 | >= min_node_count. Defaults to 3. 74 | 75 | management_auto_repair - Whether the nodes will be automatically repaired. 76 | Defaults to 'true'. 77 | 78 | management_auto_upgrade - Whether the nodes will be automatically upgraded. 79 | Defaults to 'true'. 80 | 81 | version - The Kubernetes version for the nodes in this pool. Note that if this 82 | field is set the 'management_auto_upgrade' will be overriden and set to 'false'. 83 | This is to avoid both options fighting on what the node version should be. While 84 | a fuzzy version can be specified, it's recommended that you specify explicit 85 | versions as Terraform will see spurious diffs when fuzzy versions are used. See 86 | the 'google_container_engine_versions' data source's 'version_prefix' field to 87 | approximate fuzzy versions in a Terraform-compatible way. 88 | 89 | node_config_machine_type - The name of a Google Compute Engine machine type. 90 | Defaults to n1-standard-1. To create a custom machine type, value should be 91 | set as specified here: 92 | https://cloud.google.com/compute/docs/reference/rest/v1/instances#machineType 93 | 94 | node_config_disk_type - Type of the disk attached to each node (e.g. 95 | 'pd-standard' or 'pd-ssd'). Defaults to 'pd-standard' 96 | 97 | node_config_disk_size_gb - Size of the disk attached to each node, specified 98 | in GB. The smallest allowed disk size is 10GB. Defaults to 100GB. 99 | 100 | node_config_preemptible - Whether or not the underlying node VMs are 101 | preemptible. See the official documentation for more information. Defaults to 102 | false. https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms 103 | EOF 104 | } 105 | 106 | variable "vpc_network_name" { 107 | type = string 108 | 109 | description = </dev/null 2>&1 && pwd )" 23 | 24 | # Make temporary directory to use for testing and enter it 25 | VERIFY_DIR="${REPO_ROOT}/verify" 26 | mkdir -p "$VERIFY_DIR" 27 | pushd "$VERIFY_DIR" 28 | 29 | # Determine OS type and architecture to get the correct Terraform binary. 30 | # Terrafom supports more platforms than are listed here, but only those used by 31 | # the developers are included. We're open to PRs to add more. 32 | if [[ "$OSTYPE" == "linux-gnu" ]]; then 33 | TERRAFORM_OS="linux" 34 | elif [[ "$OSTYPE" == "darwin"* ]]; then 35 | TERRAFORM_OS="darwin" 36 | else 37 | echo "OS type not supported" 38 | exit 1 39 | fi 40 | ARCH=$(uname -m) 41 | if [[ "$ARCH" == "x86_64" ]]; then 42 | TERRAFORM_ARCH="amd64" 43 | elif [[ "$ARCH" == "i386" ]]; then 44 | TERRAFORM_ARCH="386" 45 | else 46 | echo "Architecture not supported" 47 | exit 1 48 | fi 49 | 50 | # Checks the Terraform version used by the module, download the Terraform binary 51 | # for that version. 52 | if grep "required_version.*0.12.*" "${REPO_ROOT}/main.tf"; then 53 | TERRAFORM_VERSION="0.12.24" 54 | else 55 | echo "Terraform version is not supported or could not be found." 56 | exit 1 57 | fi 58 | 59 | TERRAFORM_ZIP="terraform_${TERRAFORM_VERSION}_${TERRAFORM_OS}_${TERRAFORM_ARCH}.zip" 60 | TERRAFORM_URL="https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/${TERRAFORM_ZIP}" 61 | # If the zip is already present don't download it again. 62 | if [ ! -f ${TERRAFORM_ZIP} ]; then 63 | curl $TERRAFORM_URL -o $TERRAFORM_ZIP 64 | fi 65 | unzip -o $TERRAFORM_ZIP 66 | chmod +x terraform 67 | TERRAFORM="${VERIFY_DIR}/terraform" 68 | $TERRAFORM version 69 | 70 | # Capture the output of terraform fmt so that we can trigger the script to 71 | # fail if formatting changes were made. terraform fmt does not consider 72 | # applying formatting changes to be failure, however we want the files to be 73 | # correctly formatted in version control. 74 | FMT=$(${TERRAFORM} fmt $REPO_ROOT) 75 | if [ "$FMT" != "" ]; then 76 | echo "$FMT" 77 | exit 1 78 | fi 79 | 80 | # Copy files from the example project to use as a test project for the module. 81 | # Only copy these files if they don't already exist so they can be edited during 82 | # iterative testing. 83 | if [ ! -f main.tf ]; then 84 | cp "${REPO_ROOT}/example/main.tf" main.tf 85 | # Remove the requirement for a GCS backend so we can init and validate locally 86 | perl -i -0pe 's/(\s*)backend "gcs" \{\n?\s*\n?\s*\}/\1# GCS bucket not used for testing/gms' main.tf 87 | # Use the local version of the module, not the Terraform Registry version, and remove the version specification 88 | perl -i -0pe 's/(\s*)source*\s*= "jetstack\/gke-cluster\/google"\n\s*version = ".*?"/\1source = "..\/"/gms' main.tf 89 | fi 90 | if [ ! -f variables.tf ]; then 91 | cp "${REPO_ROOT}/example/variables.tf" variables.tf 92 | fi 93 | if [ ! -f terraform.tfvars ]; then 94 | cp "${REPO_ROOT}/example/terraform.tfvars.example" terraform.tfvars 95 | fi 96 | 97 | # Initialise and validate the generated test project 98 | $TERRAFORM init 99 | $TERRAFORM validate 100 | 101 | # TODO: Set up a GCP project and service account to run the following section 102 | # in automated testing. 103 | 104 | # If SKIP_DESTROY is true then exit without destroying, this can be used to 105 | # conduct more manual testing and experiments. 106 | SKIP_DESTROY="${SKIP_DESTROY:-false}" 107 | 108 | # To make Terraform plan and apply the generated test project the following 109 | # environment variables are required: 110 | # GOOGLE_APPLICATION_CREDENTIALS is the path of a key.json for a service account 111 | # GCP_PROJECT_ID is the ID of a GCP project to use 112 | if [ ! -z ${GCP_PROJECT_ID+x} ] || [ ! -z ${GOOGLE_APPLICATION_CREDENTIALS+x} ]; then 113 | echo $GCP_PROJECT_ID 114 | echo $GOOGLE_APPLICATION_CREDENTIALS 115 | sed -i.bak "s|my-project|$GCP_PROJECT_ID|g" terraform.tfvars 116 | $TERRAFORM plan 117 | $TERRAFORM apply -auto-approve 118 | if [ ! "$SKIP_DESTROY" == "true" ]; then 119 | $TERRAFORM destroy -auto-approve 120 | fi 121 | else 122 | echo "Skipping terraform plan and apply as GCP_PROJECT_ID and GOOGLE_APPLICATION_CREDENTIALS not set." 123 | fi 124 | 125 | popd > /dev/null 126 | # If SKIP_DESTROY is true then don't delete the project directory so that 127 | # terraform destroy can still be run later for clean up. 128 | if [ ! "$SKIP_DESTROY" == "true" ]; then 129 | rm -rf "$VERIFY_DIR" 130 | fi 131 | -------------------------------------------------------------------------------- /iam.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # https://www.terraform.io/docs/providers/random/r/id.html 16 | provider "random" { 17 | version = "~> 2.1" 18 | } 19 | 20 | resource "random_id" "entropy" { 21 | byte_length = 6 22 | } 23 | 24 | # https://www.terraform.io/docs/providers/google/r/google_service_account.html 25 | resource "google_service_account" "default" { 26 | provider = google 27 | 28 | account_id = "cluster-minimal-${random_id.entropy.hex}" 29 | display_name = "Minimal service account for GKE cluster ${var.cluster_name}" 30 | project = var.gcp_project_id 31 | } 32 | 33 | # https://www.terraform.io/docs/providers/google/r/google_project_iam.html 34 | resource "google_project_iam_member" "logging-log-writer" { 35 | provider = google 36 | 37 | role = "roles/logging.logWriter" 38 | member = "serviceAccount:${google_service_account.default.email}" 39 | project = var.gcp_project_id 40 | } 41 | 42 | resource "google_project_iam_member" "monitoring-metric-writer" { 43 | provider = google 44 | 45 | role = "roles/monitoring.metricWriter" 46 | member = "serviceAccount:${google_service_account.default.email}" 47 | project = var.gcp_project_id 48 | } 49 | 50 | resource "google_project_iam_member" "monitoring-viewer" { 51 | provider = google 52 | 53 | role = "roles/monitoring.viewer" 54 | member = "serviceAccount:${google_service_account.default.email}" 55 | project = var.gcp_project_id 56 | } 57 | 58 | resource "google_project_iam_member" "storage-object-viewer" { 59 | provider = google 60 | 61 | count = var.access_private_images == "true" ? 1 : 0 62 | role = "roles/storage.objectViewer" 63 | member = "serviceAccount:${google_service_account.default.email}" 64 | project = var.gcp_project_id 65 | } 66 | -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # Resource definitions for cluster and node pools 16 | 17 | # Each argument is explained, most details are pulled from the Terraform 18 | # documentation. Arguments set by input variables are documented in the 19 | # variables.tf file. 20 | 21 | terraform { 22 | required_version = "~> 0.12" 23 | } 24 | 25 | # Local values assign a name to an expression, that can then be used multiple 26 | # times within a module. They are used here to determine the GCP region from 27 | # the given location, which can be either a region or zone. 28 | locals { 29 | gcp_location_parts = split("-", var.gcp_location) 30 | gcp_region = format("%s-%s", local.gcp_location_parts[0], local.gcp_location_parts[1]) 31 | } 32 | 33 | # https://www.terraform.io/docs/providers/google/index.html 34 | provider "google" { 35 | version = "~> 3.5" 36 | project = var.gcp_project_id 37 | region = local.gcp_region 38 | } 39 | 40 | # To make use of beta features the google-beta provider is also used. Only 41 | # resources that use beta features use the beta provider. All resources have 42 | # the provider set explicitly for clarity. 43 | # https://www.terraform.io/docs/providers/google/guides/provider_versions.html#using-the-google-beta-provider 44 | provider "google-beta" { 45 | version = "~> 3.5" 46 | project = var.gcp_project_id 47 | region = local.gcp_region 48 | } 49 | 50 | locals { 51 | release_channel = var.release_channel == "" ? [] : [var.release_channel] 52 | min_master_version = var.release_channel == "" ? var.min_master_version : "" 53 | identity_namespace = var.identity_namespace == "" ? [] : [var.identity_namespace] 54 | } 55 | 56 | locals { 57 | authenticator_security_group = var.authenticator_security_group == "" ? [] : [var.authenticator_security_group] 58 | } 59 | 60 | # https://www.terraform.io/docs/providers/google/r/container_cluster.html 61 | resource "google_container_cluster" "cluster" { 62 | provider = google-beta 63 | 64 | location = var.gcp_location 65 | 66 | name = var.cluster_name 67 | 68 | min_master_version = local.min_master_version 69 | 70 | dynamic "release_channel" { 71 | for_each = toset(local.release_channel) 72 | 73 | content { 74 | channel = release_channel.value 75 | } 76 | } 77 | 78 | dynamic "authenticator_groups_config" { 79 | for_each = toset(local.authenticator_security_group) 80 | 81 | content { 82 | security_group = authenticator_groups_config.value 83 | } 84 | } 85 | 86 | # Configure workload identity if set 87 | dynamic "workload_identity_config" { 88 | for_each = toset(local.identity_namespace) 89 | 90 | content { 91 | identity_namespace = workload_identity_config.value 92 | } 93 | } 94 | 95 | maintenance_policy { 96 | daily_maintenance_window { 97 | start_time = var.daily_maintenance_window_start_time 98 | } 99 | } 100 | 101 | # A set of options for creating a private cluster. 102 | private_cluster_config { 103 | enable_private_endpoint = var.private_endpoint 104 | enable_private_nodes = var.private_nodes 105 | 106 | master_ipv4_cidr_block = var.master_ipv4_cidr_block 107 | } 108 | 109 | # Enable the PodSecurityPolicy admission controller for the cluster. 110 | pod_security_policy_config { 111 | enabled = var.pod_security_policy_enabled 112 | } 113 | 114 | # Configuration options for the NetworkPolicy feature. 115 | network_policy { 116 | # Whether network policy is enabled on the cluster. Defaults to false. 117 | # In GKE this also enables the ip masquerade agent 118 | # https://cloud.google.com/kubernetes-engine/docs/how-to/ip-masquerade-agent 119 | enabled = true 120 | 121 | # The selected network policy provider. Defaults to PROVIDER_UNSPECIFIED. 122 | provider = "CALICO" 123 | } 124 | 125 | master_auth { 126 | # Setting an empty username and password explicitly disables basic auth 127 | username = "" 128 | password = "" 129 | 130 | # Whether client certificate authorization is enabled for this cluster. 131 | client_certificate_config { 132 | issue_client_certificate = false 133 | } 134 | } 135 | 136 | # The configuration for addons supported by GKE. 137 | addons_config { 138 | http_load_balancing { 139 | disabled = var.http_load_balancing_disabled 140 | } 141 | 142 | # Whether we should enable the network policy addon for the master. This must be 143 | # enabled in order to enable network policy for the nodes. It can only be disabled 144 | # if the nodes already do not have network policies enabled. Defaults to disabled; 145 | # set disabled = false to enable. 146 | network_policy_config { 147 | disabled = false 148 | } 149 | } 150 | 151 | network = var.vpc_network_name 152 | subnetwork = var.vpc_subnetwork_name 153 | 154 | # Configuration for cluster IP allocation. As of now, only pre-allocated 155 | # subnetworks (custom type with secondary ranges) are supported. This will 156 | # activate IP aliases. 157 | ip_allocation_policy { 158 | cluster_secondary_range_name = var.cluster_secondary_range_name 159 | services_secondary_range_name = var.services_secondary_range_name 160 | } 161 | 162 | # It's not possible to create a cluster with no node pool defined, but we 163 | # want to only use separately managed node pools. So we create the smallest 164 | # possible default node pool and immediately delete it. 165 | remove_default_node_pool = true 166 | 167 | # The number of nodes to create in this cluster (not including the Kubernetes master). 168 | initial_node_count = 1 169 | 170 | # The desired configuration options for master authorized networks. Omit the 171 | # nested cidr_blocks attribute to disallow external access (except the 172 | # cluster node IPs, which GKE automatically whitelists). 173 | master_authorized_networks_config { 174 | dynamic "cidr_blocks" { 175 | for_each = var.master_authorized_networks_cidr_blocks 176 | content { 177 | cidr_block = cidr_blocks.value.cidr_block 178 | display_name = cidr_blocks.value.display_name 179 | } 180 | } 181 | } 182 | 183 | # The loggingservice that the cluster should write logs to. Using the 184 | # 'logging.googleapis.com/kubernetes' option makes use of new Stackdriver 185 | # Kubernetes integration. 186 | logging_service = var.stackdriver_logging != "false" ? "logging.googleapis.com/kubernetes" : "" 187 | 188 | # The monitoring service that the cluster should write metrics to. Using the 189 | # 'monitoring.googleapis.com/kubernetes' option makes use of new Stackdriver 190 | # Kubernetes integration. 191 | monitoring_service = var.stackdriver_monitoring != "false" ? "monitoring.googleapis.com/kubernetes" : "" 192 | 193 | # Change how long update operations on the cluster are allowed to take 194 | # before being considered to have failed. The default is 10 mins. 195 | # https://www.terraform.io/docs/configuration/resources.html#operation-timeouts 196 | timeouts { 197 | update = "20m" 198 | } 199 | } 200 | 201 | # https://www.terraform.io/docs/providers/google/r/container_node_pool.html 202 | resource "google_container_node_pool" "node_pool" { 203 | provider = google 204 | 205 | # The location (region or zone) in which the cluster resides 206 | location = google_container_cluster.cluster.location 207 | 208 | count = length(var.node_pools) 209 | 210 | # The name of the node pool. Instance groups created will have the cluster 211 | # name prefixed automatically. 212 | name = format("%s-pool", lookup(var.node_pools[count.index], "name", format("%03d", count.index + 1))) 213 | 214 | # The cluster to create the node pool for. 215 | cluster = google_container_cluster.cluster.name 216 | 217 | initial_node_count = lookup(var.node_pools[count.index], "initial_node_count", 1) 218 | 219 | # Configuration required by cluster autoscaler to adjust the size of the node pool to the current cluster usage. 220 | autoscaling { 221 | # Minimum number of nodes in the NodePool. Must be >=0 and <= max_node_count. 222 | min_node_count = lookup(var.node_pools[count.index], "autoscaling_min_node_count", 2) 223 | 224 | # Maximum number of nodes in the NodePool. Must be >= min_node_count. 225 | max_node_count = lookup(var.node_pools[count.index], "autoscaling_max_node_count", 3) 226 | } 227 | 228 | # Target a specific Kubernetes version. 229 | version = lookup(var.node_pools[count.index], "version", "") 230 | 231 | # Node management configuration, wherein auto-repair and auto-upgrade is configured. 232 | management { 233 | # Whether the nodes will be automatically repaired. 234 | auto_repair = lookup(var.node_pools[count.index], "auto_repair", true) 235 | 236 | # Whether the nodes will be automatically upgraded. 237 | auto_upgrade = lookup(var.node_pools[count.index], "version", "") == "" ? lookup(var.node_pools[count.index], "auto_upgrade", true) : false 238 | } 239 | 240 | # Parameters used in creating the cluster's nodes. 241 | node_config { 242 | # The name of a Google Compute Engine machine type. Defaults to 243 | # n1-standard-1. 244 | machine_type = lookup( 245 | var.node_pools[count.index], 246 | "node_config_machine_type", 247 | "n1-standard-1", 248 | ) 249 | 250 | service_account = google_service_account.default.email 251 | 252 | # Size of the disk attached to each node, specified in GB. The smallest 253 | # allowed disk size is 10GB. Defaults to 100GB. 254 | disk_size_gb = lookup( 255 | var.node_pools[count.index], 256 | "node_config_disk_size_gb", 257 | 100 258 | ) 259 | 260 | # Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd'). 261 | # If unspecified, the default disk type is 'pd-standard' 262 | disk_type = lookup( 263 | var.node_pools[count.index], 264 | "node_config_disk_type", 265 | "pd-standard", 266 | ) 267 | 268 | # A boolean that represents whether or not the underlying node VMs are 269 | # preemptible. See the official documentation for more information. 270 | # Defaults to false. 271 | preemptible = lookup( 272 | var.node_pools[count.index], 273 | "node_config_preemptible", 274 | false, 275 | ) 276 | 277 | # The set of Google API scopes to be made available on all of the node VMs 278 | # under the "default" service account. These can be either FQDNs, or scope 279 | # aliases. The cloud-platform access scope authorizes access to all Cloud 280 | # Platform services, and then limit the access by granting IAM roles 281 | # https://cloud.google.com/compute/docs/access/service-accounts#service_account_permissions 282 | oauth_scopes = [ 283 | "https://www.googleapis.com/auth/cloud-platform", 284 | ] 285 | 286 | # The metadata key/value pairs assigned to instances in the cluster. 287 | metadata = { 288 | # https://cloud.google.com/kubernetes-engine/docs/how-to/protecting-cluster-metadata 289 | disable-legacy-endpoints = "true" 290 | } 291 | } 292 | 293 | # Change how long update operations on the node pool are allowed to take 294 | # before being considered to have failed. The default is 10 mins. 295 | # https://www.terraform.io/docs/configuration/resources.html#operation-timeouts 296 | timeouts { 297 | update = "20m" 298 | } 299 | } 300 | -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Jetstack Ltd. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | variable "gcp_project_id" { 16 | type = string 17 | 18 | description = <=0 and <= max_node_count. Defaults to 2. 71 | 72 | autoscaling_max_node_count - Maximum number of nodes in the NodePool. Must be 73 | >= min_node_count. Defaults to 3. 74 | 75 | management_auto_repair - Whether the nodes will be automatically repaired. 76 | Defaults to 'true'. 77 | 78 | management_auto_upgrade - Whether the nodes will be automatically upgraded. 79 | Defaults to 'true'. 80 | 81 | version - The Kubernetes version for the nodes in this pool. Note that if this 82 | field is set the 'management_auto_upgrade' will be overriden and set to 'false'. 83 | This is to avoid both options fighting on what the node version should be. While 84 | a fuzzy version can be specified, it's recommended that you specify explicit 85 | versions as Terraform will see spurious diffs when fuzzy versions are used. See 86 | the 'google_container_engine_versions' data source's 'version_prefix' field to 87 | approximate fuzzy versions in a Terraform-compatible way. 88 | 89 | node_config_machine_type - The name of a Google Compute Engine machine type. 90 | Defaults to n1-standard-1. To create a custom machine type, value should be 91 | set as specified here: 92 | https://cloud.google.com/compute/docs/reference/rest/v1/instances#machineType 93 | 94 | node_config_disk_type - Type of the disk attached to each node (e.g. 95 | 'pd-standard' or 'pd-ssd'). Defaults to 'pd-standard' 96 | 97 | node_config_disk_size_gb - Size of the disk attached to each node, specified 98 | in GB. The smallest allowed disk size is 10GB. Defaults to 100GB. 99 | 100 | node_config_preemptible - Whether or not the underlying node VMs are 101 | preemptible. See the official documentation for more information. Defaults to 102 | false. https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms 103 | EOF 104 | } 105 | 106 | variable "vpc_network_name" { 107 | type = string 108 | 109 | description = <