├── .gitignore ├── README.md ├── infra ├── backend.tf ├── backend │ ├── firewall │ │ ├── main.tf │ │ └── variables.tf │ ├── subnet │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ └── vpc │ │ ├── main.tf │ │ └── outputs.tf ├── cloudsql │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── gke │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── main.tf ├── outputs.tf └── variables.tf └── project ├── backend.tf ├── main.tf ├── outputs.tf └── variables.tf /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.tfstate 3 | *.tfstate.backup 4 | *.tfvars 5 | .terraform 6 | tfplan 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Google GKE and SQL with terraform 2 | 3 | ## Initial tooling setup gcloud, kubectl and terraform 4 | 5 | Assuming you already have Google Cloud account we will need additional binaries for gcloud CLI,, terraform and kubectl. 6 | Gcloud deployment differs from Linux distribution and you can follow the [link](https://cloud.google.com/sdk/docs/quickstarts) to deploy for OSX and diff Linux distributions 7 | 8 | ### Deploying terraform 9 | 10 | #### OS X 11 | 12 | ```sh 13 | curl -o terraform_0.11.10_darwin_amd64.zip \ 14 | https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.10_darwin_amd64.zip 15 | 16 | unzip terraform_0.11.10_linux_amd64.zip -d /usr/local/bin/ 17 | ``` 18 | 19 | #### Linux 20 | 21 | ```sh 22 | curl https://releases.hashicorp.com/terraform/0.11.10/terraform_0.11.7_linux_amd64.zip \ 23 | > terraform_0.11.10_linux_amd64.zip 24 | 25 | unzip terraform_0.11.10_linux_amd64.zip -d /usr/local/bin/ 26 | ``` 27 | 28 | #### Verification 29 | 30 | Verify terraform version 0.11.10 or higher is installed: 31 | 32 | ```sh 33 | terraform version 34 | ``` 35 | 36 | ### Deploying kubectl 37 | 38 | #### OS X 39 | 40 | ```sh 41 | curl -o kubectl \ 42 | https://storage.googleapis.com/kubernetes-release/release/v1.12.0/bin/darwin/amd64/kubectl 43 | 44 | chmod +x kubectl 45 | 46 | sudo mv kubectl /usr/local/bin/ 47 | ``` 48 | 49 | #### Linux 50 | 51 | ```sh 52 | wget \ 53 | https://storage.googleapis.com/kubernetes-release/release/v1.12.0/bin/linux/amd64/kubectl 54 | 55 | chmod +x kubectl 56 | 57 | sudo mv kubectl /usr/local/bin/ 58 | ``` 59 | 60 | #### Verification 61 | 62 | ```sh 63 | kubectl version --client 64 | ``` 65 | 66 | ### Authenticate to gcloud 67 | 68 | Before configuring gcloud CLI you can check available Zones and Regions nearest to your location 69 | 70 | ```sh 71 | gcloud compute regions list 72 | 73 | gcloud compute zones list 74 | ``` 75 | 76 | Follow gcloud init and select default Zone Ex. asia-south1 77 | 78 | ```sh 79 | gcloud init 80 | ``` 81 | 82 | ## Creating Google Cloud project and service account for terraform 83 | 84 | Best practice to use separate account "technical account" to manage infrastructure, this account can be used in automated code deployment like in Jenkins, CirceleCI or any other tool you may choose. 85 | 86 | ### Set up environment 87 | 88 | ```sh 89 | export TF_VAR_org_id=YOUR_ORG_ID 90 | export TF_VAR_billing_account=YOUR_BILLING_ACCOUNT_ID 91 | export TF_ADMIN=terraform-admin-demo 92 | export TF_CREDS=~/.config/gcloud/terraform-admin-demo.json 93 | ``` 94 | 95 | > NOTE: value of YOUR_ORG_ID and YOUR_BILLING_ACCOUNT_ID you can find by running 96 | 97 | ```sh 98 | gcloud organizations list 99 | 100 | gcloud beta billing accounts list 101 | ``` 102 | 103 | ### Create the Terraform Admin Project 104 | 105 | Create a new project and link it to your billing account 106 | 107 | ```sh 108 | gcloud projects create ${TF_ADMIN} \ 109 | --organization ${TF_VAR_org_id} \ 110 | --set-as-default 111 | 112 | gcloud beta billing projects link ${TF_ADMIN} \ 113 | --billing-account ${TF_VAR_billing_account} 114 | ``` 115 | 116 | ### Create the Terraform service account 117 | 118 | Create the service account in the Terraform admin project and download the JSON credentials: 119 | 120 | ```sh 121 | gcloud iam service-accounts create terraform \ 122 | --display-name "Terraform admin account" 123 | 124 | gcloud iam service-accounts keys create ${TF_CREDS} \ 125 | --iam-account terraform@${TF_ADMIN}.iam.gserviceaccount.com 126 | ``` 127 | 128 | Grant the service account permission to view the Admin Project and manage Cloud Storage 129 | 130 | ```sh 131 | gcloud projects add-iam-policy-binding ${TF_ADMIN} \ 132 | --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \ 133 | --role roles/viewer 134 | 135 | gcloud projects add-iam-policy-binding ${TF_ADMIN} \ 136 | --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \ 137 | --role roles/storage.admin 138 | ``` 139 | 140 | Enabled API for newly created projects 141 | 142 | ```sh 143 | gcloud services enable cloudresourcemanager.googleapis.com && \ 144 | gcloud services enable cloudbilling.googleapis.com && \ 145 | gcloud services enable iam.googleapis.com && \ 146 | gcloud services enable compute.googleapis.com && \ 147 | gcloud services enable sqladmin.googleapis.com && \ 148 | gcloud services enable container.googleapis.com 149 | ``` 150 | 151 | ### Add organization/folder-level permissions 152 | 153 | Grant the service account permission to create projects and assign billing accounts 154 | 155 | ```sh 156 | gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \ 157 | --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \ 158 | --role roles/resourcemanager.projectCreator 159 | 160 | gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \ 161 | --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \ 162 | --role roles/billing.user 163 | ``` 164 | 165 | ## Creating back-end storage to tfstate file in Cloud Storage 166 | 167 | Terraform stores the state about infrastructure and configuration by default local file "terraform.tfstate. State is used by Terraform to map resources to configuration, track metadata. 168 | 169 | Terraform allows state file to be stored remotely, which works better in a team environment or automated deployments. 170 | We will used Google Storage and create new bucket where we can store state files. 171 | 172 | Create the remote back-end bucket in Cloud Storage for storage of the terraform.tfstate file 173 | 174 | ```sh 175 | gsutil mb -p ${TF_ADMIN} -l asia-southeast1 gs://${TF_ADMIN} 176 | ``` 177 | 178 | Enable versioning for said remote bucket: 179 | 180 | ```sh 181 | gsutil versioning set on gs://${TF_ADMIN} 182 | ``` 183 | 184 | Configure your environment for the Google Cloud terraform provider 185 | 186 | ```sh 187 | export GOOGLE_APPLICATION_CREDENTIALS=${TF_CREDS} 188 | ``` 189 | 190 | ## Setting up separate projects for Development and Production environments 191 | 192 | In order to segregate Development environment we will use Google cloud projects that allows us to segregate infrastructure bur maintain same time same code base for terraform. 193 | 194 | Terraform allow us to use separate tfstate file for different environment by using terraform functionality workspaces. 195 | Let's see current file structure 196 | 197 | ```sh 198 | . 199 | ├── backend.tf 200 | ├── main.tf 201 | ├── outputs.tf 202 | ├── terraform.tfvars 203 | └── variables.tf 204 | ``` 205 | 206 | * 1st step is to keep sensitive information outside external git repository. Best practice is to create terraform.tfvars and keep sensitive information and add .tfvars to .gitignore 207 | 208 | ```sh 209 | .gitignore 210 | *.tfstate 211 | *.tfstate.backup 212 | *.tfvars 213 | .terraform 214 | tfplan 215 | ``` 216 | 217 | * Create terraform.tfvars file in project folder and replace "XXXXXX" with proper data. In our case tfvars files data is reference in variables.tf where we keep variables for main.tf 218 | 219 | ```sh 220 | billing_account = "XXXXXX-XXXXXX-XXXXXX" 221 | org_id = "XXXXXXXXXXX" 222 | ``` 223 | 224 | * backend.tf allows us to use newly created Google storage bucket to keep our tfstate files. 225 | 226 | ```go 227 | terraform { 228 | backend "gcs" { 229 | bucket = "terraform-admin-demo" 230 | prefix = "terraform-project" 231 | } 232 | } 233 | ``` 234 | 235 | * Variable used in terraform main.tf file 236 | 237 | ```go 238 | # GCP variables 239 | variable "region" { 240 | default = "asia-southeast1" 241 | description = "Region of resources" 242 | } 243 | 244 | variable "project_name" { 245 | # default = "test" 246 | 247 | default = { 248 | prod = "prod" 249 | dev = "dev" 250 | } 251 | 252 | description = "The NAME of the Google Cloud project" 253 | } 254 | 255 | variable "billing_account" { 256 | description = "Billing account STRING." 257 | } 258 | 259 | variable "org_id" { 260 | description = "Organisation account NR." 261 | } 262 | ``` 263 | 264 | * Outputs, once terraform will deploy new infrastructure we will need some outputs that we can reuse for GKE and SQL setup 265 | 266 | ```go 267 | # project creation output 268 | output "project_id" { 269 | value = "${google_project.project.project_id}" 270 | } 271 | ``` 272 | 273 | * Finally main source of the gcloud project creation 274 | 275 | ```go 276 | provider "google" { 277 | version = "~> 1.16" 278 | region = "${var.region}" 279 | } 280 | 281 | provider "random" {} 282 | 283 | resource "random_id" "id" { 284 | byte_length = 4 285 | prefix = "terraform-${var.project_name[terraform.workspace]}-" 286 | } 287 | 288 | resource "google_project" "project" { 289 | name = "terraform-${var.project_name[terraform.workspace]}" 290 | project_id = "${random_id.id.hex}" 291 | billing_account = "${var.billing_account}" 292 | org_id = "${var.org_id}" 293 | } 294 | 295 | resource "google_project_services" "project" { 296 | project = "${google_project.project.project_id}" 297 | 298 | services = [ 299 | "bigquery-json.googleapis.com", 300 | "compute.googleapis.com", 301 | "container.googleapis.com", 302 | "containerregistry.googleapis.com", 303 | "deploymentmanager.googleapis.com", 304 | "dns.googleapis.com", 305 | "logging.googleapis.com", 306 | "monitoring.googleapis.com", 307 | "oslogin.googleapis.com", 308 | "pubsub.googleapis.com", 309 | "replicapool.googleapis.com", 310 | "replicapoolupdater.googleapis.com", 311 | "resourceviews.googleapis.com", 312 | "servicemanagement.googleapis.com", 313 | "sql-component.googleapis.com", 314 | "sqladmin.googleapis.com", 315 | "storage-api.googleapis.com", 316 | ] 317 | } 318 | ``` 319 | 320 | ### Initialize and pull terraform cloud specific dependencies 321 | 322 | Terraform uses modular setup and in order to download specific plugin for cloud provider, terraform will need to be 1st initiated. 323 | 324 | ```sh 325 | terraform init 326 | ``` 327 | 328 | ### Workspace creation for dev and prod 329 | 330 | Once we have our project code and our tfvar secretes secure we can create workspaces for terraform 331 | 332 | > NOTE: in below example we will use only dev workspace but you can use both following same logic 333 | 334 | * Create dev workspace 335 | 336 | ```sh 337 | terraform workspace new dev 338 | ``` 339 | 340 | * List available workspaces 341 | 342 | ```sh 343 | terraform workspace list 344 | ``` 345 | 346 | * Switch between workspaces 347 | 348 | ```sh 349 | terraform workspace select dev 350 | ``` 351 | 352 | ### Terraform plan 353 | 354 | Terraform plan will simulate what changes terraform will be done on cloud provider 355 | 356 | ```sh 357 | terraform plan 358 | ``` 359 | 360 | ### Apply terraform plan for selected environment 361 | 362 | ```sh 363 | terraform apply 364 | ``` 365 | 366 | With above code we only created new project in Google Cloud and this dependent of the in what terraform workspace we are in. 367 | 368 | [![asciicast](https://asciinema.org/a/195482.png)](https://asciinema.org/a/195482) 369 | 370 | 371 | ## Creating Kubernetes cluster on GKE and PostgreSQL on Cloud SQL 372 | 373 | Once we have project ready for dev and prod we can move into deploying our gke and sql infrastructure. 374 | 375 | Code structure 376 | 377 | ```sh 378 | . 379 | ├── backend 380 | │ ├── firewall 381 | │ │ ├── main.tf 382 | │ │ └── variables.tf 383 | │ ├── subnet 384 | │ │ ├── main.tf 385 | │ │ ├── outputs.tf 386 | │ │ └── variables.tf 387 | │ └── vpc 388 | │ ├── main.tf 389 | │ └── outputs.tf 390 | ├── backend.tf 391 | ├── cloudsql 392 | │ ├── main.tf 393 | │ ├── outputs.tf 394 | │ └── variables.tf 395 | ├── gke 396 | │ ├── main.tf 397 | │ ├── outputs.tf 398 | │ └── variables.tf 399 | ├── main.tf 400 | ├── outputs.tf 401 | └── variables.tf 402 | ``` 403 | 404 | Now is time to deploy our infrastructure, noticeable differences between prod and dev workspaces you can find in the terraform files. 405 | 406 | * dev - single instance of PostgreSQL without replication and read replica 407 | * prod - single instance in multi AZ for high availablity and additional one read replica for PostgreSQL 408 | * dev - single kubernetes node will be added to GKE 409 | * prod - two nodes will be created and added to kubernetes GKE 410 | 411 | In order to keep our code clean I decided to use modules for every segment Networking(vpc, subnets and firewall), cloudsql and gke. All this modules can be maintained in separate git repositories and can be called by root main.tf file. 412 | 413 | main.tf 414 | 415 | ```go 416 | # Configure the Google Cloud provider 417 | 418 | data "terraform_remote_state" "project_id" { 419 | backend = "gcs" 420 | workspace = "${terraform.workspace}" 421 | 422 | config { 423 | bucket = "${var.bucket_name}" 424 | prefix = "terraform-project" 425 | } 426 | } 427 | 428 | provider "google" { 429 | version = "~> 1.16" 430 | project = "${data.terraform_remote_state.project_id.project_id}" 431 | region = "${var.region}" 432 | } 433 | 434 | module "vpc" { 435 | source = "./backend/vpc" 436 | } 437 | 438 | module "subnet" { 439 | source = "./backend/subnet" 440 | region = "${var.region}" 441 | vpc_name = "${module.vpc.vpc_name}" 442 | subnet_cidr = "${var.subnet_cidr}" 443 | } 444 | 445 | module "firewall" { 446 | source = "./backend/firewall" 447 | vpc_name = "${module.vpc.vpc_name}" 448 | ip_cidr_range = "${module.subnet.ip_cidr_range}" 449 | } 450 | 451 | module "cloudsql" { 452 | source = "./cloudsql" 453 | region = "${var.region}" 454 | availability_type = "${var.availability_type}" 455 | sql_instance_size = "${var.sql_instance_size}" 456 | sql_disk_type = "${var.sql_disk_type}" 457 | sql_disk_size = "${var.sql_disk_size}" 458 | sql_require_ssl = "${var.sql_require_ssl}" 459 | sql_master_zone = "${var.sql_master_zone}" 460 | sql_connect_retry_interval = "${var.sql_connect_retry_interval}" 461 | sql_replica_zone = "${var.sql_replica_zone}" 462 | sql_user = "${var.sql_user}" 463 | sql_pass = "${var.sql_pass}" 464 | } 465 | 466 | module "gke" { 467 | source = "./gke" 468 | region = "${var.region}" 469 | min_master_version = "${var.min_master_version}" 470 | node_version = "${var.node_version}" 471 | gke_num_nodes = "${var.gke_num_nodes}" 472 | vpc_name = "${module.vpc.vpc_name}" 473 | subnet_name = "${module.subnet.subnet_name}" 474 | gke_master_user = "${var.gke_master_user}" 475 | gke_master_pass = "${var.gke_master_pass}" 476 | gke_node_machine_type = "${var.gke_node_machine_type}" 477 | gke_label = "${var.gke_label}" 478 | } 479 | ``` 480 | 481 | All variables that is consumed by modules I keep in single variable.tf file. 482 | 483 | We will use same google storage bucket but with different prefix not to conflict with project creation terraform plan. 484 | 485 | ```go 486 | # Configure the Google Cloud tfstate file location 487 | terraform { 488 | backend "gcs" { 489 | bucket = "terraform-admin-mmm" 490 | prefix = "terraform" 491 | } 492 | } 493 | ``` 494 | 495 | As terraform needs to be aware of the new projects we created in previous step we will import state from terraform 1st run. 496 | 497 | ```go 498 | data "terraform_remote_state" "project_id" { 499 | backend = "gcs" 500 | workspace = "${terraform.workspace}" 501 | config { 502 | bucket = "${var.bucket_name}" 503 | prefix = "terraform-project" 504 | } 505 | } 506 | ``` 507 | 508 | ### Running terraform changes for infrastructure 509 | 510 | We are now ready to to run our plan and create infrastructure. 511 | 512 | As we are in separate code base will need to follow same sequence as in project creation. 513 | 514 | > NOTE: Just make sure you have new terraform.tfvars 515 | 516 | ```sh 517 | bucket_name = "terraform-admin-demo" 518 | gke_master_pass = "your-gke-password" 519 | sql_pass = "your-sql-password" 520 | ``` 521 | 522 | * Initialize and pull terraform cloud specific dependencies 523 | 524 | ```sh 525 | terraform init 526 | ``` 527 | 528 | * Create dev workspace 529 | 530 | ```sh 531 | terraform workspace new dev 532 | ``` 533 | 534 | * List available workspaces 535 | 536 | ```sh 537 | terraform workspace list 538 | ``` 539 | 540 | * Switch between workspaces 541 | 542 | ```sh 543 | terraform workspace select dev 544 | ``` 545 | 546 | * Terraform plan will simulate what changes terraform will be done on cloud provider 547 | 548 | ```sh 549 | terraform plan 550 | ``` 551 | 552 | * Apply terraform 553 | 554 | ```sh 555 | terraform apply 556 | ``` 557 | 558 | [![asciicast](https://asciinema.org/a/195490.png)](https://asciinema.org/a/195490) 559 | 560 | * To check what terraform deployed use 561 | 562 | ```sh 563 | terraform show 564 | ``` 565 | 566 | * Once test is completed you can remove "destroy" all buildup infrastructure. 567 | 568 | ```sh 569 | terraform destroy -auto-approve 570 | ``` 571 | 572 | ## Terraform Tips 573 | 574 | * Refresh terraform 575 | 576 | ```sh 577 | terraform refresh 578 | ``` 579 | 580 | * List and show terraform state file 581 | 582 | ```sh 583 | terraform state list 584 | terraform state show 585 | ``` 586 | 587 | * Use tflint to check syntax of the tf files 588 | 589 | ```sh 590 | tflint 591 | ``` 592 | 593 | * Destroy only selected module Ex. 594 | 595 | ```sh 596 | terraform destroy -target=module.cloudsql 597 | ``` 598 | -------------------------------------------------------------------------------- /infra/backend.tf: -------------------------------------------------------------------------------- 1 | # Configure the Google Cloud tfstate file location 2 | terraform { 3 | backend "gcs" { 4 | bucket = "terraform-admin-demo" 5 | prefix = "terraform" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /infra/backend/firewall/main.tf: -------------------------------------------------------------------------------- 1 | # VPC firewall configuration 2 | # Create a firewall rule that allows internal communication across all protocols: 3 | resource "google_compute_firewall" "firewalli-int" { 4 | name = "${terraform.workspace}-firewall-int" 5 | network = "${var.vpc_name}" 6 | 7 | allow { 8 | protocol = "icmp" 9 | } 10 | 11 | allow { 12 | protocol = "tcp" 13 | } 14 | 15 | allow { 16 | protocol = "udp" 17 | } 18 | 19 | source_ranges = ["${var.ip_cidr_range}"] 20 | } 21 | 22 | # Create a firewall rule that allows external SSH, ICMP, and HTTPS: 23 | resource "google_compute_firewall" "firewalli-ext" { 24 | name = "${terraform.workspace}-firewall-ext" 25 | network = "${var.vpc_name}" 26 | 27 | allow { 28 | protocol = "icmp" 29 | } 30 | 31 | allow { 32 | protocol = "tcp" 33 | ports = ["22", "6443"] 34 | } 35 | 36 | source_ranges = ["0.0.0.0/0"] 37 | } 38 | -------------------------------------------------------------------------------- /infra/backend/firewall/variables.tf: -------------------------------------------------------------------------------- 1 | # variable needed for subnetwork creation 2 | 3 | variable "vpc_name" { 4 | description = "Netwrok name" 5 | } 6 | 7 | variable "ip_cidr_range" { 8 | description = "Subnet range" 9 | } 10 | -------------------------------------------------------------------------------- /infra/backend/subnet/main.tf: -------------------------------------------------------------------------------- 1 | # Create Subnet 2 | 3 | resource "google_compute_subnetwork" "subnet" { 4 | name = "${terraform.workspace}-subnet" 5 | ip_cidr_range = "${var.subnet_cidr[terraform.workspace]}" 6 | network = "${var.vpc_name}" 7 | region = "${var.region}" 8 | } 9 | -------------------------------------------------------------------------------- /infra/backend/subnet/outputs.tf: -------------------------------------------------------------------------------- 1 | # network subnet output 2 | 3 | output "ip_cidr_range" { 4 | value = "${google_compute_subnetwork.subnet.ip_cidr_range}" 5 | description = "Export created CICDR range" 6 | } 7 | 8 | output "subnet_name" { 9 | value = "${google_compute_subnetwork.subnet.name}" 10 | description = "Export created CICDR range" 11 | } -------------------------------------------------------------------------------- /infra/backend/subnet/variables.tf: -------------------------------------------------------------------------------- 1 | # Subnet variables 2 | 3 | variable "region" { 4 | description = "Region of resources" 5 | } 6 | 7 | variable "vpc_name" { 8 | description = "Netwrok name" 9 | } 10 | 11 | variable "subnet_cidr" { 12 | type = "map" 13 | description = "Subnet range" 14 | } 15 | -------------------------------------------------------------------------------- /infra/backend/vpc/main.tf: -------------------------------------------------------------------------------- 1 | # Create VPC 2 | resource "google_compute_network" "vpc" { 3 | name = "${terraform.workspace}-vpc" 4 | auto_create_subnetworks = "false" 5 | } 6 | -------------------------------------------------------------------------------- /infra/backend/vpc/outputs.tf: -------------------------------------------------------------------------------- 1 | # network VPC output 2 | 3 | output "vpc_name" { 4 | value = "${google_compute_network.vpc.name}" 5 | description = "The unique name of the network" 6 | } 7 | 8 | output "self_link" { 9 | value = "${google_compute_network.vpc.self_link}" 10 | description = "The URL of the created resource" 11 | } 12 | -------------------------------------------------------------------------------- /infra/cloudsql/main.tf: -------------------------------------------------------------------------------- 1 | provider "random" {} 2 | 3 | resource "random_id" "id" { 4 | byte_length = 4 5 | prefix = "sql-${terraform.workspace}-" 6 | } 7 | 8 | resource "google_sql_database_instance" "master" { 9 | # name = "sql-${terraform.workspace}-master" 10 | name = "${random_id.id.hex}" 11 | region = "${var.region}" 12 | database_version = "POSTGRES_9_6" 13 | 14 | settings { 15 | availability_type = "${var.availability_type[terraform.workspace]}" 16 | tier = "${var.sql_instance_size}" 17 | disk_type = "${var.sql_disk_type}" 18 | disk_size = "${var.sql_disk_size}" 19 | disk_autoresize = true 20 | 21 | ip_configuration { 22 | authorized_networks = { 23 | value = "0.0.0.0/0" 24 | } 25 | 26 | require_ssl = "${var.sql_require_ssl}" 27 | ipv4_enabled = true 28 | } 29 | 30 | location_preference { 31 | zone = "${var.region}-${var.sql_master_zone}" 32 | } 33 | 34 | backup_configuration { 35 | # binary_log_enabled = true 36 | enabled = true 37 | start_time = "00:00" 38 | } 39 | } 40 | } 41 | 42 | resource "google_sql_database_instance" "replica" { 43 | depends_on = [ 44 | "google_sql_database_instance.master", 45 | ] 46 | 47 | name = "metest-${terraform.workspace}-replica" 48 | count = "${terraform.workspace == "prod" ? 1 : 0}" 49 | region = "${var.region}" 50 | database_version = "POSTGRES_9_6" 51 | master_instance_name = "${google_sql_database_instance.master.name}" 52 | 53 | settings { 54 | tier = "${var.sql_instance_size}" 55 | disk_type = "${var.sql_disk_type}" 56 | disk_size = "${var.sql_disk_size}" 57 | disk_autoresize = true 58 | 59 | location_preference { 60 | zone = "${var.region}-${var.sql_replica_zone}" 61 | } 62 | } 63 | } 64 | 65 | resource "google_sql_user" "user" { 66 | depends_on = [ 67 | "google_sql_database_instance.master", 68 | "google_sql_database_instance.replica", 69 | ] 70 | 71 | instance = "${google_sql_database_instance.master.name}" 72 | name = "${var.sql_user}" 73 | password = "${var.sql_pass}" 74 | } 75 | -------------------------------------------------------------------------------- /infra/cloudsql/outputs.tf: -------------------------------------------------------------------------------- 1 | # cloud SQL postgresql outputs 2 | 3 | output "master_instance_sql_ipv4" { 4 | value = "${google_sql_database_instance.master.ip_address.0.ip_address}" 5 | description = "The IPv4 address assigned for master" 6 | } 7 | -------------------------------------------------------------------------------- /infra/cloudsql/variables.tf: -------------------------------------------------------------------------------- 1 | # GCP variables 2 | variable "region" { 3 | description = "Region of resources" 4 | } 5 | 6 | # Cloud SQL variables 7 | 8 | variable "availability_type" { 9 | type = "map" 10 | description = "Availability type for HA" 11 | } 12 | 13 | variable "sql_instance_size" { 14 | description = "Size of Cloud SQL instances" 15 | } 16 | 17 | variable "sql_disk_type" { 18 | description = "Cloud SQL instance disk type" 19 | } 20 | 21 | variable "sql_disk_size" { 22 | description = "Storage size in GB" 23 | } 24 | 25 | variable "sql_require_ssl" { 26 | description = "Enforce SSL connections" 27 | } 28 | 29 | variable "sql_connect_retry_interval" { 30 | description = "The number of seconds between connect retries." 31 | } 32 | 33 | variable "sql_master_zone" { 34 | description = "Zone of the Cloud SQL master (a, b, ...)" 35 | } 36 | 37 | variable "sql_replica_zone" { 38 | description = "Zone of the Cloud SQL replica (a, b, ...)" 39 | } 40 | 41 | variable "sql_user" { 42 | description = "Username of the host to access the database" 43 | } 44 | 45 | variable "sql_pass" { 46 | description = "Password of the host to access the database" 47 | } 48 | -------------------------------------------------------------------------------- /infra/gke/main.tf: -------------------------------------------------------------------------------- 1 | resource "google_container_cluster" "primary" { 2 | name = "gke-${terraform.workspace}-cluster" 3 | zone = "${var.region}-a" 4 | 5 | additional_zones = [ 6 | "${var.region}-b", 7 | "${var.region}-c", 8 | ] 9 | 10 | // region = "${var.region}" 11 | min_master_version = "${var.min_master_version}" 12 | node_version = "${var.node_version}" 13 | enable_legacy_abac = false 14 | initial_node_count = "${var.gke_num_nodes[terraform.workspace]}" 15 | network = "${var.vpc_name}" 16 | subnetwork = "${var.subnet_name}" 17 | 18 | addons_config { 19 | http_load_balancing { 20 | disabled = false 21 | } 22 | 23 | horizontal_pod_autoscaling { 24 | disabled = false 25 | } 26 | 27 | kubernetes_dashboard { 28 | disabled = false 29 | } 30 | } 31 | 32 | master_auth { 33 | username = "${var.gke_master_user}" 34 | password = "${var.gke_master_pass}" 35 | } 36 | 37 | node_config { 38 | oauth_scopes = [ 39 | "https://www.googleapis.com/auth/compute", 40 | "https://www.googleapis.com/auth/devstorage.read_only", 41 | "https://www.googleapis.com/auth/logging.write", 42 | "https://www.googleapis.com/auth/monitoring", 43 | ] 44 | 45 | labels { 46 | env = "${var.gke_label[terraform.workspace]}" 47 | } 48 | 49 | disk_size_gb = 10 50 | machine_type = "${var.gke_node_machine_type}" 51 | tags = ["gke-node"] 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /infra/gke/outputs.tf: -------------------------------------------------------------------------------- 1 | # GKE outputs 2 | 3 | output "endpoint" { 4 | value = "${google_container_cluster.primary.endpoint}" 5 | description = "Endpoint for accessing the master node" 6 | } 7 | -------------------------------------------------------------------------------- /infra/gke/variables.tf: -------------------------------------------------------------------------------- 1 | # GKE variables 2 | 3 | variable "region" { 4 | description = "Region of resources" 5 | } 6 | 7 | variable "min_master_version" { 8 | description = "Number of nodes in each GKE cluster zone" 9 | } 10 | 11 | variable "node_version" { 12 | description = "Number of nodes in each GKE cluster zone" 13 | } 14 | 15 | variable "gke_num_nodes" { 16 | type = "map" 17 | description = "Number of nodes in each GKE cluster zone" 18 | } 19 | 20 | variable "vpc_name" { 21 | description = "vpc name" 22 | } 23 | variable "subnet_name" { 24 | description = "subnet name" 25 | } 26 | 27 | variable "gke_master_user" { 28 | description = "Username to authenticate with the k8s master" 29 | } 30 | 31 | variable "gke_master_pass" { 32 | description = "Username to authenticate with the k8s master" 33 | } 34 | 35 | variable "gke_node_machine_type" { 36 | description = "Machine type of GKE nodes" 37 | } 38 | 39 | # k8s variables 40 | variable gke_label { 41 | type = "map" 42 | description = "label" 43 | } 44 | -------------------------------------------------------------------------------- /infra/main.tf: -------------------------------------------------------------------------------- 1 | # Configure the Google Cloud provider 2 | 3 | data "terraform_remote_state" "project_id" { 4 | backend = "gcs" 5 | workspace = "${terraform.workspace}" 6 | 7 | config { 8 | bucket = "${var.bucket_name}" 9 | prefix = "terraform-project" 10 | } 11 | } 12 | 13 | provider "google" { 14 | version = "~> 1.16" 15 | project = "${data.terraform_remote_state.project_id.project_id}" 16 | region = "${var.region}" 17 | } 18 | 19 | module "vpc" { 20 | source = "./backend/vpc" 21 | } 22 | 23 | module "subnet" { 24 | source = "./backend/subnet" 25 | region = "${var.region}" 26 | vpc_name = "${module.vpc.vpc_name}" 27 | subnet_cidr = "${var.subnet_cidr}" 28 | } 29 | 30 | module "firewall" { 31 | source = "./backend/firewall" 32 | vpc_name = "${module.vpc.vpc_name}" 33 | ip_cidr_range = "${module.subnet.ip_cidr_range}" 34 | } 35 | 36 | module "cloudsql" { 37 | source = "./cloudsql" 38 | region = "${var.region}" 39 | availability_type = "${var.availability_type}" 40 | sql_instance_size = "${var.sql_instance_size}" 41 | sql_disk_type = "${var.sql_disk_type}" 42 | sql_disk_size = "${var.sql_disk_size}" 43 | sql_require_ssl = "${var.sql_require_ssl}" 44 | sql_master_zone = "${var.sql_master_zone}" 45 | sql_connect_retry_interval = "${var.sql_connect_retry_interval}" 46 | sql_replica_zone = "${var.sql_replica_zone}" 47 | sql_user = "${var.sql_user}" 48 | sql_pass = "${var.sql_pass}" 49 | } 50 | 51 | module "gke" { 52 | source = "./gke" 53 | region = "${var.region}" 54 | min_master_version = "${var.min_master_version}" 55 | node_version = "${var.node_version}" 56 | gke_num_nodes = "${var.gke_num_nodes}" 57 | vpc_name = "${module.vpc.vpc_name}" 58 | subnet_name = "${module.subnet.subnet_name}" 59 | gke_master_user = "${var.gke_master_user}" 60 | gke_master_pass = "${var.gke_master_pass}" 61 | gke_node_machine_type = "${var.gke_node_machine_type}" 62 | gke_label = "${var.gke_label}" 63 | } 64 | -------------------------------------------------------------------------------- /infra/outputs.tf: -------------------------------------------------------------------------------- 1 | # remote state output 2 | output "data_out" { 3 | value = "${data.terraform_remote_state.project_id.project_id}" 4 | } 5 | 6 | # network VPC output 7 | output "vpc_name" { 8 | value = "${module.vpc.vpc_name}" 9 | description = "The unique name of the network" 10 | } 11 | 12 | # subnet cidr ip range 13 | output "ip_cidr_range" { 14 | value = "${module.subnet.ip_cidr_range}" 15 | description = "Export created CICDR range" 16 | } 17 | 18 | # Cloud SQL postgresql outputs 19 | output "master_instance_sql_ipv4" { 20 | value = "${module.cloudsql.master_instance_sql_ipv4}" 21 | description = "The IPv4 address assigned for master" 22 | } 23 | 24 | # GKE outputs 25 | output "endpoint" { 26 | value = "${module.gke.endpoint}" 27 | description = "Endpoint for accessing the master node" 28 | } 29 | 30 | 31 | -------------------------------------------------------------------------------- /infra/variables.tf: -------------------------------------------------------------------------------- 1 | # GCP variables 2 | 3 | variable "region" { 4 | default = "asia-southeast1" 5 | description = "Region of resources" 6 | } 7 | 8 | variable "bucket_name" { 9 | description = "Name of the google storage bucket" 10 | } 11 | 12 | variable "name" { 13 | default = { 14 | prod = "prod" 15 | dev = "dev" 16 | } 17 | 18 | description = "Name for vpc" 19 | } 20 | 21 | # Network variables 22 | 23 | variable "subnet_cidr" { 24 | default = { 25 | prod = "10.10.0.0/24" 26 | dev = "10.240.0.0/24" 27 | } 28 | 29 | description = "Subnet range" 30 | } 31 | 32 | # Cloud SQL variables 33 | 34 | variable "availability_type" { 35 | default = { 36 | prod = "REGIONAL" 37 | dev = "ZONAL" 38 | } 39 | 40 | description = "Availability type for HA" 41 | } 42 | 43 | variable "sql_instance_size" { 44 | default = "db-f1-micro" 45 | description = "Size of Cloud SQL instances" 46 | } 47 | 48 | variable "sql_disk_type" { 49 | default = "PD_SSD" 50 | description = "Cloud SQL instance disk type" 51 | } 52 | 53 | variable "sql_disk_size" { 54 | default = "10" 55 | description = "Storage size in GB" 56 | } 57 | 58 | variable "sql_require_ssl" { 59 | default = "false" 60 | description = "Enforce SSL connections" 61 | } 62 | 63 | variable "sql_master_zone" { 64 | default = "a" 65 | description = "Zone of the Cloud SQL master (a, b, ...)" 66 | } 67 | 68 | variable "sql_replica_zone" { 69 | default = "b" 70 | description = "Zone of the Cloud SQL replica (a, b, ...)" 71 | } 72 | 73 | variable "sql_connect_retry_interval" { 74 | default = 60 75 | description = "The number of seconds between connect retries." 76 | } 77 | 78 | variable "sql_user" { 79 | default = "admin" 80 | description = "Username of the host to access the database" 81 | } 82 | 83 | variable "sql_pass" { 84 | description = "Password of the host to access the database" 85 | } 86 | 87 | # GKE variables 88 | 89 | variable "min_master_version" { 90 | default = "1.10.7-gke.6" 91 | description = "Number of nodes in each GKE cluster zone" 92 | } 93 | 94 | variable "node_version" { 95 | default = "1.10.7-gke.6" 96 | description = "Number of nodes in each GKE cluster zone" 97 | } 98 | 99 | variable "gke_num_nodes" { 100 | default = { 101 | prod = 2 102 | dev = 1 103 | } 104 | 105 | description = "Number of nodes in each GKE cluster zone" 106 | } 107 | 108 | variable "gke_master_user" { 109 | default = "k8s_admin" 110 | description = "Username to authenticate with the k8s master" 111 | } 112 | 113 | variable "gke_master_pass" { 114 | description = "Username to authenticate with the k8s master" 115 | } 116 | 117 | variable "gke_node_machine_type" { 118 | default = "n1-standard-1" 119 | description = "Machine type of GKE nodes" 120 | } 121 | 122 | variable gke_label { 123 | default = { 124 | prod = "prod" 125 | dev = "dev" 126 | } 127 | 128 | description = "label" 129 | } 130 | -------------------------------------------------------------------------------- /project/backend.tf: -------------------------------------------------------------------------------- 1 | # Configure the Google Cloud tfstate file location 2 | terraform { 3 | backend "gcs" { 4 | bucket = "terraform-admin-demo" 5 | prefix = "terraform-project" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /project/main.tf: -------------------------------------------------------------------------------- 1 | provider "google" { 2 | version = "~> 1.16" 3 | region = "${var.region}" 4 | } 5 | 6 | provider "random" {} 7 | 8 | resource "random_id" "id" { 9 | byte_length = 4 10 | prefix = "terraform-${var.project_name[terraform.workspace]}-" 11 | } 12 | 13 | resource "google_project" "project" { 14 | name = "terraform-${var.project_name[terraform.workspace]}" 15 | project_id = "${random_id.id.hex}" 16 | billing_account = "${var.billing_account}" 17 | org_id = "${var.org_id}" 18 | } 19 | 20 | resource "google_project_services" "project" { 21 | project = "${google_project.project.project_id}" 22 | 23 | services = [ 24 | "bigquery-json.googleapis.com", 25 | "compute.googleapis.com", 26 | "container.googleapis.com", 27 | "containerregistry.googleapis.com", 28 | "deploymentmanager.googleapis.com", 29 | "dns.googleapis.com", 30 | "logging.googleapis.com", 31 | "monitoring.googleapis.com", 32 | "oslogin.googleapis.com", 33 | "pubsub.googleapis.com", 34 | "replicapool.googleapis.com", 35 | "replicapoolupdater.googleapis.com", 36 | "resourceviews.googleapis.com", 37 | "servicemanagement.googleapis.com", 38 | "sql-component.googleapis.com", 39 | "sqladmin.googleapis.com", 40 | "storage-api.googleapis.com", 41 | ] 42 | } 43 | -------------------------------------------------------------------------------- /project/outputs.tf: -------------------------------------------------------------------------------- 1 | # project creation output 2 | output "project_id" { 3 | value = "${google_project.project.project_id}" 4 | } 5 | -------------------------------------------------------------------------------- /project/variables.tf: -------------------------------------------------------------------------------- 1 | # GCP variables 2 | variable "region" { 3 | default = "asia-southeast1" 4 | description = "Region of resources" 5 | } 6 | 7 | variable "project_name" { 8 | # default = "test" 9 | 10 | default = { 11 | prod = "prod" 12 | dev = "dev" 13 | } 14 | 15 | description = "The NAME of the Google Cloud project" 16 | } 17 | 18 | variable "billing_account" { 19 | description = "Billing account STRING." 20 | } 21 | 22 | variable "org_id" { 23 | description = "Organisation account NR." 24 | } 25 | --------------------------------------------------------------------------------