├── .gitignore
├── LICENSE
├── README.md
├── config.tf
├── docs
    └── CreateAzureSpn.md
├── infra.tf
├── main.tf
├── master.tf
├── network.tf
├── node.tf
├── scripts
    ├── deployOpenShift.sh
    ├── masterPrep.sh
    └── nodePrep.sh
└── terraform.tfvars.example


/.gitignore:
--------------------------------------------------------------------------------
1 | *.tfstate
2 | *.tfstate.backup
3 | .terraform.tfstate.lock.info
4 | terraform.tfvars


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2017 Sertaç Özercan
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # OpenShift-Azure-Terraform
 2 | Deploy OpenShift Origin v3.6.0 on Azure using Terraform and Ansible
 3 | ==================
 4 | 
 5 | This script allow you to deploy an OpenShift Origin v3.6.0 in best practices on Microsoft Azure.
 6 | 
 7 | # Terraform Usage #
 8 | 
 9 | #### WARNING: Be sure that you are not overriding existing Azure resources that are in use. This Terraform process will create a resource group to contain all dependent resources within. This makes it easy to cleanup.
10 | 
11 | #### NOTE: This deployment is not meant to obviate the need to understand the install process or read the docs. Please spend some time to understand both [OpenShift](https://www.openshift.com/) and the [install process](https://install.openshift.com/).
12 | 
13 | ## Preperation Steps ##
14 | 
15 | * It is assumed that you have a functioning Azure client installed. You can do so [here](https://github.com/Azure/azure-cli)
16 | 
17 | * Install [Terraform](https://www.terraform.io/downloads.html) and create credentials for Terraform to access Azure. To do so, you will need to following environment variables :
18 | 
19 |   * ARM_SUBSCRIPTION_ID=<subscription id>
20 |   * ARM_CLIENT_ID=<client id>
21 |   * ARM_CLIENT_SECRET=<cient secret>
22 |   * ARM_TENANT_ID=<tenant id>
23 | 
24 | * You can also fill the following values in the tfvars file if you prefeer.
25 | 
26 | * The values for the above environment variables can be obtained through the Azure CLI.
27 | 
28 | [Click here to get the step by step about it](/docs/CreateAzureSpn.md)
29 | 
30 | ## Deploy the Azure infrastructure and OpenShift
31 | 
32 | * First rename the `terraform.tfvars.example` to `terraform.tfvars` and review the default configuration. Most common options are available inside. The full list of available options are in `config.tf`. CentOS is the default as it has pre-requirements built in.
33 | 
34 | * Update `terraform.tfvars` with the path to your passwordless SSH public and private keys. (openshift_azure_public_key and openshift_azure_private_key)
35 | 
36 | * Change `openshift_azure_resource_prefix` (and optionally `openshift_azure_resource_suffix`) to something unique
37 | 
38 | * Optionally, customize the `openshift_azure_master_vm_count` (default 1), the `openshift_azure_node_vm_count` (default 1) and `openshift_azure_infra_vm_count` for master (default 1), the agents size is Standard_D2_V2 per default, but you can change it for your need.
39 | 
40 | * Create the OpenShift cluster by executing:
41 | ```bash
42 | $ EXPORT ARM_SUBSCRIPTION_ID=<your subscription id>
43 | $ EXPORT ARM_CLIENT_ID=<your client id>
44 | $ EXPORT ARM_CLIENT_SECRET=<your cient secret>
45 | $ EXPORT ARM_TENANT_ID=<your tenant id>
46 | 
47 | $ cd <repo> && terraform apply
48 | ```
49 | ### Connection to console
50 | 
51 | After your deployment your should be able to reach the OS console
52 | 
53 | ```https://<masterFQDN>.<location>.cloudapp.azure.com:8443/console```
54 | 
55 | The cluster will use self-signed certificates. Accept the warning and proceed to the login page.
56 | 
57 | * If you didn't change it, the default username/password is `ocpadmin/password123`.
58 | 
59 | ## ADDITIONAL ##
60 | 
61 | ### Cleanup ###
62 | 
63 | To restart and cleanup the Azure assets run the following commands from the <repo> directory
64 | 
65 | ```bash
66 | $ az group delete <yourResourceGroup>
67 | info:    Executing command group delete
68 | Delete resource group <yourResourceGroup>? [y/n] y
69 | + Deleting resource group <yourResourceGroup>
70 | info:    group delete command OK
71 | 
72 | $ cd <repo> && rm *terraform.tfstate
73 | 
74 | ```
75 | 
76 | ### Troubleshooting ###
77 | 
78 | If the deployment gets in an inconsistent state (repeated `terraform apply` commands fail, or output references to leases that no longer exist), you may need to manually reconcile. Destroy the `<yourResourceGroup>` resource group, run `terraform remote config -disable` and delete all `terraform.tfstate*` files from `os`, follow the above instructions again.
79 | 
80 | * You could also check this repo : [Microsoft/openshift-origin](https://github.com/Microsoft/openshift-origin) to get more informations


--------------------------------------------------------------------------------
/config.tf:
--------------------------------------------------------------------------------
  1 | variable "azure_client_id" {
  2 |   type        = "string"
  3 |   description = "Azure Client ID"
  4 |   default     = ""
  5 | }
  6 | 
  7 | variable "azure_client_secret" {
  8 |   type        = "string"
  9 |   description = "Azure Client Secret"
 10 |   default     = ""
 11 | }
 12 | 
 13 | variable "azure_tenant_id" {
 14 |   type        = "string"
 15 |   description = "Azure Tenant ID"
 16 |   default     = ""
 17 | }
 18 | 
 19 | variable "azure_subscription_id" {
 20 |   type        = "string"
 21 |   description = "Azure Subscription ID"
 22 |   default     = ""
 23 | }
 24 | 
 25 | variable "openshift_azure_resource_prefix" {
 26 |   type        = "string"
 27 |   description = "Prefix for all the resources"
 28 |   default     = "os"
 29 | }
 30 | 
 31 | variable "openshift_azure_resource_suffix" {
 32 |   type        = "string"
 33 |   description = "Suffix for all the resources"
 34 |   default     = "tf"
 35 | }
 36 | 
 37 | variable "openshift_azure_resource_group" {
 38 |   type        = "string"
 39 |   description = "Azure resource group"
 40 |   default     = "azure"
 41 | }
 42 | 
 43 | variable "openshift_azure_region" {
 44 |   type        = "string"
 45 |   description = "Azure region for deployment"
 46 |   default     = "East US"
 47 | }
 48 | 
 49 | variable "openshift_azure_public_key" {
 50 |   type        = "string"
 51 |   description = "SSH Public key"
 52 |   default     = ""
 53 | }
 54 | 
 55 | variable "openshift_azure_private_key" {
 56 |   type        = "string"
 57 |   description = "SSH Private key"
 58 |   default     = ""
 59 | }
 60 | 
 61 | variable "openshift_azure_master_vm_count" {
 62 |   description = "Master VM count"
 63 |   default     = 1
 64 | }
 65 | 
 66 | variable "openshift_azure_infra_vm_count" {
 67 |   description = "Infra VM count"
 68 |   default     = 1
 69 | }
 70 | 
 71 | variable "openshift_azure_node_vm_count" {
 72 |   description = "Node VM count"
 73 |   default     = 1
 74 | }
 75 | 
 76 | variable "openshift_azure_master_vm_size" {
 77 |   type        = "string"
 78 |   description = "Master VM size"
 79 |   default     = "Standard_DS2_v2"
 80 | }
 81 | 
 82 | variable "openshift_azure_infra_vm_size" {
 83 |   type        = "string"
 84 |   description = "Infra VM size"
 85 |   default     = "Standard_DS2_v2"
 86 | }
 87 | 
 88 | variable "openshift_azure_node_vm_size" {
 89 |   type        = "string"
 90 |   description = "Node VM size"
 91 |   default     = "Standard_DS2_v2"
 92 | }
 93 | 
 94 | variable "openshift_azure_data_disk_size" {
 95 |   description = "Size of Datadisk in GB for Docker volume"
 96 |   default     = 128
 97 | }
 98 | 
 99 | variable "openshift_azure_vm_os" {
100 |   type = "map"
101 | 
102 |   default = {
103 |     publisher = "OpenLogic"
104 |     offer     = "CentOS"
105 |     sku       = "7.3"
106 |     version   = "latest"
107 |   }
108 | }
109 | 
110 | variable "openshift_azure_vm_username" {
111 |   type        = "string"
112 |   description = "VM Username"
113 |   default     = "ocpadmin"
114 | }
115 | 
116 | variable "openshift_master_dns_name" {
117 |   type        = "string"
118 |   description = "DNS prefix name for the master"
119 |   default     = "osmaster"
120 | }
121 | 
122 | variable "openshift_infra_dns_name" {
123 |   type        = "string"
124 |   description = "DNS prefix name for the infra"
125 |   default     = "osinfra"
126 | }
127 | 
128 | variable "openshift_initial_password" {
129 |   type        = "string"
130 |   description = "initial password for OpenShift"
131 |   default     = "password123"
132 | }
133 | 
134 | variable "openshift_azure_default_subdomain" {
135 |   type        = "string"
136 |   description = "The wildcard DNS name you would like to use for routing"
137 |   default     = "xip.io"
138 | }
139 | 
140 | variable "openshift_azure_master_prep_script" {
141 |   type        = "string"
142 |   description = "URL for Master Prep script"
143 |   default     = "https://raw.githubusercontent.com/sozercan/OpenShift-Azure-Terraform/master/scripts/masterPrep.sh"
144 | }
145 | 
146 | variable "openshift_azure_node_prep_script" {
147 |   type        = "string"
148 |   description = "URL for Node Prep script"
149 |   default     = "https://raw.githubusercontent.com/sozercan/OpenShift-Azure-Terraform/master/scripts/nodePrep.sh"
150 | }
151 | 
152 | variable "openshift_azure_deploy_openshift_script" {
153 |   type        = "string"
154 |   description = "URL for Deploy Openshift script"
155 |   default     = "https://raw.githubusercontent.com/sozercan/OpenShift-Azure-Terraform/master/scripts/deployOpenShift.sh"
156 | }
157 | 
158 | variable "openshift_ansible_url" {
159 |   type        = "string"
160 |   description = "URL for openshift-ansible repo"
161 |   default     = "https://github.com/openshift/openshift-ansible.git"
162 | }
163 | 
164 | variable "openshift_ansible_branch" {
165 |   type        = "string"
166 |   description = "Branch of the openshift-ansible repo"
167 |   default     = "master"
168 | }


--------------------------------------------------------------------------------
/docs/CreateAzureSpn.md:
--------------------------------------------------------------------------------
 1 | # Create a Service Principal for your Subscription
 2 | 
 3 | *NOTE: A more detailed overview can be found on the [Terraform Site](https://www.terraform.io/docs/providers/azurerm/index.html)*
 4 | 
 5 | * Login with the [Azure CLI 2.0](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
 6 | 
 7 | ```bash
 8 |  az login
 9 | ```
10 | 
11 | * After a succesfull login, you will get a list of all the subscriptions related to your account.
12 | 
13 | ```bash
14 | {
15 |   "environmentName": "AzureCloud",
16 |   "id": "a97d7ca2-18ca-426f-b7c4-1a2cdaa4d9d1",
17 |   "isDefault": true,
18 |   "name": "My_Azure_Subscription",
19 |   "state": "Enabled",
20 |   "tenantId": "34a934ff-86a1-34af-34cd-2d7cd0134bd34",
21 |   "user": {
22 |     "name": "juliens@microsoft.com",
23 |     "type": "user"
24 |   }
25 | }
26 | ```
27 | 
28 | * Store the subscription ID from the default subsciption.
29 | 
30 | ```bash
31 | export SUBSCRIPTIONID=`az account show --output tsv | cut -f2`
32 | ```
33 | 
34 | * Create a Service Principal as a contributor to your Subscription
35 | 
36 | ```bash
37 | az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/${SUBSCRIPTIONID}"
38 | ```
39 | 
40 | * Store Service Principal, you will use it in your tfvars file.
41 | 
42 | ```json
43 | {
44 |   "appId": "23xxxxx-xxxx-xxxx-xxxx-7e83xxxb1",
45 |   "displayName": "azure-cli-2017-07-31-xx-xx-xx",
46 |   "name": "http://azure-cli-2017-07-31-xx-xx-xx",
47 |   "password": "1219e938-72ad-439c-xxx-517ab8b60xxx",
48 |   "tenant": "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
49 | }
50 | ```
51 | 
52 | Now, you can fill up your `terraform.tfvars`
53 | 
54 | ```csharp
55 | azure_client_id => appId
56 | azure_client_secret => password
57 | azure_tenant_id => tenant
58 | azure_subscription_id => SUBSCRIPTIONID
59 | ```
60 | 
61 | *NOTE: A more detailed overview can be found on the [Terraform Site](https://www.terraform.io/docs/providers/azurerm/index.html)*
62 | 
63 | ### Create a Client ID / Password using Docker
64 | 
65 | You just have to run the following command :
66 | 
67 | ```docker run -it julienstroheker/add-azure-spn <NameApp> <PasswordApp>```
68 | 
69 | For example :
70 | 
71 | ```$ docker run -it julienstroheker/add-azure-spn MyAwesomeApplication MyAw3s0meP@ssw0rd!```
72 | 
73 | After less than 1 minute, you will have a nice output like this, ready to be copy and paste :
74 | 
75 | ```
76 | ================== Informations about your new App ==============================
77 | Subscription ID                    XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX
78 | Subscription Name                  Your Subscription Name
79 | Service Principal Client ID:       XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX
80 | Service Principal Key:             YourPasswordOrGeneratingARandomOne
81 | Tenant ID:                         XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX
82 | =================================================================================
83 | ```


--------------------------------------------------------------------------------
/infra.tf:
--------------------------------------------------------------------------------
  1 | resource "azurerm_availability_set" "osinfraas" {
  2 |   name                = "${var.openshift_azure_resource_prefix}-as-infra-${var.openshift_azure_resource_suffix}"
  3 |   location            = "${azurerm_resource_group.osrg.location}"
  4 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
  5 |   managed             = true
  6 | }
  7 | 
  8 | resource "azurerm_network_interface" "osinfranic" {
  9 |   name                      = "${var.openshift_azure_resource_prefix}-nic-infra-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
 10 |   count                     = "${var.openshift_azure_infra_vm_count}"
 11 |   location                  = "${azurerm_resource_group.osrg.location}"
 12 |   resource_group_name       = "${azurerm_resource_group.osrg.name}"
 13 |   network_security_group_id = "${azurerm_network_security_group.osinfransg.id}"
 14 | 
 15 |   ip_configuration {
 16 |     name                                    = "configuration-${count.index}"
 17 |     subnet_id                               = "${azurerm_subnet.osmastersubnet.id}"
 18 |     private_ip_address_allocation           = "dynamic"
 19 |     load_balancer_backend_address_pools_ids = ["${azurerm_lb_backend_address_pool.osinfralbbepool.id}"]
 20 |   }
 21 | }
 22 | 
 23 | resource "azurerm_lb_backend_address_pool" "osinfralbbepool" {
 24 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 25 |   loadbalancer_id     = "${azurerm_lb.osinfralb.id}"
 26 |   name                = "BackEndAddressPool"
 27 | }
 28 | 
 29 | resource "azurerm_public_ip" "osinfraip" {
 30 |   name                         = "${var.openshift_azure_resource_prefix}-vip-infra-${var.openshift_azure_resource_suffix}"
 31 |   location                     = "${azurerm_resource_group.osrg.location}"
 32 |   resource_group_name          = "${azurerm_resource_group.osrg.name}"
 33 |   public_ip_address_allocation = "static"
 34 |   domain_name_label            = "${var.openshift_azure_resource_prefix}-${var.openshift_infra_dns_name}-${var.openshift_azure_resource_suffix}"
 35 | }
 36 | 
 37 | resource "azurerm_lb" "osinfralb" {
 38 |   name                = "${var.openshift_azure_resource_prefix}-nlb-infra-${var.openshift_azure_resource_suffix}"
 39 |   location            = "${azurerm_resource_group.osrg.location}"
 40 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 41 | 
 42 |   frontend_ip_configuration {
 43 |     name                 = "PublicIPAddress"
 44 |     public_ip_address_id = "${azurerm_public_ip.osinfraip.id}"
 45 |   }
 46 | }
 47 | 
 48 | resource "azurerm_lb_rule" "osinfralbrule80" {
 49 |   resource_group_name            = "${azurerm_resource_group.osrg.name}"
 50 |   loadbalancer_id                = "${azurerm_lb.osinfralb.id}"
 51 |   name                           = "OpenShiftRouterHTTP"
 52 |   protocol                       = "Tcp"
 53 |   frontend_port                  = 80
 54 |   backend_port                   = 80
 55 |   frontend_ip_configuration_name = "PublicIPAddress"
 56 |   probe_id                       = "${azurerm_lb_probe.osinfralbprobe80.id}"
 57 |   backend_address_pool_id        = "${azurerm_lb_backend_address_pool.osinfralbbepool.id}"
 58 | }
 59 | 
 60 | resource "azurerm_lb_probe" "osinfralbprobe80" {
 61 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 62 |   loadbalancer_id     = "${azurerm_lb.osinfralb.id}"
 63 |   name                = "httpProbe"
 64 |   port                = 80
 65 |   number_of_probes    = 2
 66 | }
 67 | 
 68 | resource "azurerm_lb_rule" "osinfralbrule443" {
 69 |   resource_group_name            = "${azurerm_resource_group.osrg.name}"
 70 |   loadbalancer_id                = "${azurerm_lb.osinfralb.id}"
 71 |   name                           = "OpenShiftRouterHTTPS"
 72 |   protocol                       = "Tcp"
 73 |   frontend_port                  = 443
 74 |   backend_port                   = 443
 75 |   frontend_ip_configuration_name = "PublicIPAddress"
 76 |   probe_id                       = "${azurerm_lb_probe.osinfralbprobe443.id}"
 77 |   backend_address_pool_id        = "${azurerm_lb_backend_address_pool.osinfralbbepool.id}"
 78 | }
 79 | 
 80 | resource "azurerm_lb_probe" "osinfralbprobe443" {
 81 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 82 |   loadbalancer_id     = "${azurerm_lb.osinfralb.id}"
 83 |   name                = "httpsProbe"
 84 |   port                = 443
 85 |   number_of_probes    = 2
 86 | }
 87 | 
 88 | resource "azurerm_virtual_machine" "osinfravm" {
 89 |   name                  = "${var.openshift_azure_resource_prefix}-vm-infra-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
 90 |   count                 = "${var.openshift_azure_infra_vm_count}"
 91 |   location              = "${azurerm_resource_group.osrg.location}"
 92 |   resource_group_name   = "${azurerm_resource_group.osrg.name}"
 93 |   network_interface_ids = ["${element(azurerm_network_interface.osinfranic.*.id, count.index)}"]
 94 |   availability_set_id   = "${azurerm_availability_set.osinfraas.id}"
 95 |   vm_size               = "${var.openshift_azure_infra_vm_size}"
 96 | 
 97 |   storage_image_reference {
 98 |     publisher = "${var.openshift_azure_vm_os["publisher"]}"
 99 |     offer     = "${var.openshift_azure_vm_os["offer"]}"
100 |     sku       = "${var.openshift_azure_vm_os["sku"]}"
101 |     version   = "${var.openshift_azure_vm_os["version"]}"
102 |   }
103 | 
104 |   storage_os_disk {
105 |     name              = "${var.openshift_azure_resource_prefix}-disk-os-infra-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
106 |     caching           = "ReadWrite"
107 |     create_option     = "FromImage"
108 |     managed_disk_type = "Standard_LRS"
109 |   }
110 | 
111 |   storage_data_disk {
112 |     name              = "${var.openshift_azure_resource_prefix}-disk-data-infra-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
113 |     managed_disk_type = "Standard_LRS"
114 |     create_option     = "Empty"
115 |     lun               = 0
116 |     disk_size_gb      = "${var.openshift_azure_data_disk_size}"
117 |   }
118 | 
119 |   os_profile {
120 |     computer_name  = "${var.openshift_azure_resource_prefix}-vm-infra-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
121 |     admin_username = "${var.openshift_azure_vm_username}"
122 |     admin_password = "${uuid()}"
123 |   }
124 | 
125 |   os_profile_linux_config {
126 |     disable_password_authentication = true
127 | 
128 |     ssh_keys {
129 |       path     = "/home/${var.openshift_azure_vm_username}/.ssh/authorized_keys"
130 |       key_data = "${file(var.openshift_azure_public_key)}"
131 |     }
132 |   }
133 | }
134 | 
135 | resource "azurerm_virtual_machine_extension" "osinfravmextension" {
136 |   name                 = "osinfravmextension"
137 |   count                = "${var.openshift_azure_infra_vm_count}"
138 |   location             = "${azurerm_resource_group.osrg.location}"
139 |   resource_group_name  = "${azurerm_resource_group.osrg.name}"
140 |   virtual_machine_name = "${element(azurerm_virtual_machine.osinfravm.*.name, count.index)}"
141 |   publisher            = "Microsoft.Azure.Extensions"
142 |   type                 = "CustomScript"
143 |   type_handler_version = "2.0"
144 | 
145 |   settings = <<SETTINGS
146 |     {
147 |         "fileUris": [
148 |             "${var.openshift_azure_node_prep_script}"
149 |         ],
150 |         "commandToExecute": "bash nodePrep.sh ${azurerm_storage_account.osstoragepv.name} ${var.openshift_azure_vm_username}"
151 |     }
152 | SETTINGS
153 | }
154 | 


--------------------------------------------------------------------------------
/main.tf:
--------------------------------------------------------------------------------
 1 | provider "azurerm" {
 2 |   subscription_id = "${var.azure_subscription_id}"
 3 |   client_id       = "${var.azure_client_id}"
 4 |   client_secret   = "${var.azure_client_secret}"
 5 |   tenant_id       = "${var.azure_tenant_id}"
 6 | }
 7 | 
 8 | resource "azurerm_resource_group" "osrg" {
 9 |   name     = "${format("%s-%s-%s",var.openshift_azure_resource_prefix,var.openshift_azure_resource_group,var.openshift_azure_resource_suffix)}"
10 |   location = "${var.openshift_azure_region}"
11 | }
12 | 
13 | resource "random_id" "randomId" {
14 |   byte_length = 4
15 | }
16 | 
17 | resource "azurerm_storage_account" "osstorageregistry" {
18 |   name                = "${var.openshift_azure_resource_prefix}sa${random_id.randomId.hex}"
19 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
20 |   depends_on          = ["azurerm_resource_group.osrg"]
21 |   location            = "${azurerm_resource_group.osrg.location}"
22 |   account_type        = "Standard_LRS"
23 | }
24 | 
25 | resource "azurerm_storage_account" "osstoragepv" {
26 |   name                = "${var.openshift_azure_resource_prefix}pv${random_id.randomId.hex}"
27 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
28 |   depends_on          = ["azurerm_resource_group.osrg"]
29 |   location            = "${azurerm_resource_group.osrg.location}"
30 |   account_type        = "Standard_LRS"
31 | }
32 | 


--------------------------------------------------------------------------------
/master.tf:
--------------------------------------------------------------------------------
  1 | resource "azurerm_availability_set" "osmasteras" {
  2 |   name                = "${var.openshift_azure_resource_prefix}-as-master-${var.openshift_azure_resource_suffix}"
  3 |   location            = "${azurerm_resource_group.osrg.location}"
  4 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
  5 |   managed             = true
  6 | }
  7 | 
  8 | resource "azurerm_network_interface" "osmasternic" {
  9 |   name                      = "${var.openshift_azure_resource_prefix}-nic-master-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
 10 |   count                     = "${var.openshift_azure_master_vm_count}"
 11 |   location                  = "${azurerm_resource_group.osrg.location}"
 12 |   resource_group_name       = "${azurerm_resource_group.osrg.name}"
 13 |   network_security_group_id = "${azurerm_network_security_group.osmasternsg.id}"
 14 | 
 15 |   ip_configuration {
 16 |     name                                    = "configuration-${count.index}"
 17 |     subnet_id                               = "${azurerm_subnet.osmastersubnet.id}"
 18 |     private_ip_address_allocation           = "dynamic"
 19 |     load_balancer_backend_address_pools_ids = ["${azurerm_lb_backend_address_pool.osmasterlbbepool.id}"]
 20 |     load_balancer_inbound_nat_rules_ids     = ["${element(azurerm_lb_nat_rule.osmasterlbnatrule22.*.id, count.index)}"]
 21 |   }
 22 | }
 23 | 
 24 | resource "azurerm_lb_backend_address_pool" "osmasterlbbepool" {
 25 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 26 |   loadbalancer_id     = "${azurerm_lb.osmasterlb.id}"
 27 |   name                = "BackEndAddressPool"
 28 | }
 29 | 
 30 | resource "azurerm_lb_nat_rule" "osmasterlbnatrule22" {
 31 |   resource_group_name            = "${azurerm_resource_group.osrg.name}"
 32 |   loadbalancer_id                = "${azurerm_lb.osmasterlb.id}"
 33 |   name                           = "SSH-${format("%01d", count.index)}"
 34 |   count                          = "${var.openshift_azure_master_vm_count}"
 35 |   protocol                       = "Tcp"
 36 |   frontend_port                  = "${22 + count.index}"
 37 |   backend_port                   = 22
 38 |   frontend_ip_configuration_name = "PublicIPAddress"
 39 | }
 40 | 
 41 | resource "azurerm_lb_rule" "osmasterlbrule8443" {
 42 |   resource_group_name            = "${azurerm_resource_group.osrg.name}"
 43 |   loadbalancer_id                = "${azurerm_lb.osmasterlb.id}"
 44 |   name                           = "OpenShiftAdminConsole"
 45 |   protocol                       = "Tcp"
 46 |   frontend_port                  = 8443
 47 |   backend_port                   = 8443
 48 |   frontend_ip_configuration_name = "PublicIPAddress"
 49 |   probe_id                       = "${azurerm_lb_probe.osmasterlbprobe8443.id}"
 50 |   idle_timeout_in_minutes        = 30
 51 |   backend_address_pool_id        = "${azurerm_lb_backend_address_pool.osmasterlbbepool.id}"
 52 | }
 53 | 
 54 | resource "azurerm_lb_probe" "osmasterlbprobe8443" {
 55 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 56 |   loadbalancer_id     = "${azurerm_lb.osmasterlb.id}"
 57 |   name                = "8443Probe"
 58 |   port                = 8443
 59 |   number_of_probes    = 2
 60 | }
 61 | 
 62 | resource "azurerm_public_ip" "osmasterip" {
 63 |   name                         = "${var.openshift_azure_resource_prefix}-vip-master-${var.openshift_azure_resource_suffix}"
 64 |   location                     = "${azurerm_resource_group.osrg.location}"
 65 |   resource_group_name          = "${azurerm_resource_group.osrg.name}"
 66 |   public_ip_address_allocation = "static"
 67 |   domain_name_label            = "${var.openshift_azure_resource_prefix}-${var.openshift_master_dns_name}-${var.openshift_azure_resource_suffix}"
 68 | }
 69 | 
 70 | resource "azurerm_lb" "osmasterlb" {
 71 |   name                = "${var.openshift_azure_resource_prefix}-nlb-master-${var.openshift_azure_resource_suffix}"
 72 |   location            = "${azurerm_resource_group.osrg.location}"
 73 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 74 | 
 75 |   frontend_ip_configuration {
 76 |     name                 = "PublicIPAddress"
 77 |     public_ip_address_id = "${azurerm_public_ip.osmasterip.id}"
 78 |   }
 79 | }
 80 | 
 81 | resource "azurerm_virtual_machine" "osmastervm" {
 82 |   name                  = "${var.openshift_azure_resource_prefix}-vm-master-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
 83 |   count                 = "${var.openshift_azure_master_vm_count}"
 84 |   location              = "${azurerm_resource_group.osrg.location}"
 85 |   resource_group_name   = "${azurerm_resource_group.osrg.name}"
 86 |   network_interface_ids = ["${element(azurerm_network_interface.osmasternic.*.id, count.index)}"]
 87 |   availability_set_id   = "${azurerm_availability_set.osmasteras.id}"
 88 |   vm_size               = "${var.openshift_azure_master_vm_size}"
 89 | 
 90 |   storage_image_reference {
 91 |     publisher = "${var.openshift_azure_vm_os["publisher"]}"
 92 |     offer     = "${var.openshift_azure_vm_os["offer"]}"
 93 |     sku       = "${var.openshift_azure_vm_os["sku"]}"
 94 |     version   = "${var.openshift_azure_vm_os["version"]}"
 95 |   }
 96 | 
 97 |   storage_os_disk {
 98 |     name              = "${var.openshift_azure_resource_prefix}-disk-os-master-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
 99 |     caching           = "ReadWrite"
100 |     create_option     = "FromImage"
101 |     managed_disk_type = "Standard_LRS"
102 |   }
103 | 
104 |   storage_data_disk {
105 |     name              = "${var.openshift_azure_resource_prefix}-disk-data-master-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
106 |     managed_disk_type = "Standard_LRS"
107 |     create_option     = "Empty"
108 |     lun               = 0
109 |     disk_size_gb      = "${var.openshift_azure_data_disk_size}"
110 |   }
111 | 
112 |   os_profile {
113 |     computer_name  = "${var.openshift_azure_resource_prefix}-vm-master-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
114 |     admin_username = "${var.openshift_azure_vm_username}"
115 |     admin_password = "${uuid()}"
116 |   }
117 | 
118 |   os_profile_linux_config {
119 |     disable_password_authentication = true
120 | 
121 |     ssh_keys {
122 |       path     = "/home/${var.openshift_azure_vm_username}/.ssh/authorized_keys"
123 |       key_data = "${file(var.openshift_azure_public_key)}"
124 |     }
125 |   }
126 | }
127 | 
128 | resource "azurerm_virtual_machine_extension" "osmastervmextension" {
129 |   name                 = "osmastervmextension"
130 |   count                = "${var.openshift_azure_master_vm_count}"
131 |   location             = "${azurerm_resource_group.osrg.location}"
132 |   resource_group_name  = "${azurerm_resource_group.osrg.name}"
133 |   depends_on           = ["azurerm_virtual_machine.osmastervm"]
134 |   virtual_machine_name = "${element(azurerm_virtual_machine.osmastervm.*.name, count.index)}"
135 |   publisher            = "Microsoft.Azure.Extensions"
136 |   type                 = "CustomScript"
137 |   type_handler_version = "2.0"
138 | 
139 |   settings = <<SETTINGS
140 |     {
141 |         "fileUris": [
142 |             "${var.openshift_azure_master_prep_script}", "${var.openshift_azure_deploy_openshift_script}"
143 |         ],
144 |         "commandToExecute": "bash masterPrep.sh ${azurerm_storage_account.osstoragepv.name} ${var.openshift_azure_vm_username} && bash deployOpenShift.sh ${var.openshift_azure_vm_username} ${var.openshift_initial_password} ${base64encode(file(var.openshift_azure_private_key))} '${var.openshift_azure_resource_prefix}-vm-master-${var.openshift_azure_resource_suffix}' ${azurerm_public_ip.osmasterip.fqdn} ${azurerm_public_ip.osmasterip.ip_address} '${var.openshift_azure_resource_prefix}-vm-infra-${var.openshift_azure_resource_suffix}' '${var.openshift_azure_resource_prefix}-vm-node-${var.openshift_azure_resource_suffix}' ${var.openshift_azure_node_vm_count} ${var.openshift_azure_infra_vm_count} ${var.openshift_azure_master_vm_count} ${azurerm_public_ip.osinfraip.ip_address}.${var.openshift_azure_default_subdomain} ${azurerm_storage_account.osstorageregistry.name} ${azurerm_storage_account.osstorageregistry.primary_access_key} ${var.azure_tenant_id} ${var.azure_subscription_id} ${var.azure_client_id} ${var.azure_client_secret} ${azurerm_resource_group.osrg.name} '${azurerm_resource_group.osrg.location}' ${azurerm_storage_account.osstoragepv.name} ${azurerm_storage_account.osstoragepv.primary_access_key} ${var.openshift_ansible_url} ${var.openshift_ansible_branch}"
145 |     }
146 | SETTINGS
147 | }
148 | 
149 | output "OpenShift Web Console" {
150 |   value = "https://${azurerm_public_ip.osmasterip.fqdn}:8443/console"
151 | }
152 | 


--------------------------------------------------------------------------------
/network.tf:
--------------------------------------------------------------------------------
  1 | resource "azurerm_virtual_network" "osvnet" {
  2 |   name                = "${var.openshift_azure_resource_prefix}-vnet-${var.openshift_azure_resource_suffix}"
  3 |   depends_on          = ["azurerm_resource_group.osrg"]
  4 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
  5 |   location            = "${azurerm_resource_group.osrg.location}"
  6 |   address_space       = ["10.0.0.0/8"]
  7 | }
  8 | 
  9 | resource "azurerm_subnet" "osmastersubnet" {
 10 |   name                      = "osmastersubnet"
 11 |   depends_on                = ["azurerm_virtual_network.osvnet"]
 12 |   resource_group_name       = "${azurerm_resource_group.osrg.name}"
 13 |   virtual_network_name      = "${azurerm_virtual_network.osvnet.name}"
 14 |   network_security_group_id = "${azurerm_network_security_group.osmasternsg.id}"
 15 |   address_prefix            = "10.1.0.0/16"
 16 | }
 17 | 
 18 | resource "azurerm_subnet" "osnodesubnet" {
 19 |   name                      = "osnodesubnet"
 20 |   depends_on                = ["azurerm_virtual_network.osvnet"]
 21 |   resource_group_name       = "${azurerm_resource_group.osrg.name}"
 22 |   virtual_network_name      = "${azurerm_virtual_network.osvnet.name}"
 23 |   network_security_group_id = "${azurerm_network_security_group.osnodensg.id}"
 24 |   address_prefix            = "10.2.0.0/16"
 25 | }
 26 | 
 27 | resource "azurerm_network_security_group" "osmasternsg" {
 28 |   name                = "${var.openshift_azure_resource_prefix}-nsg-master-${var.openshift_azure_resource_suffix}"
 29 |   location            = "${azurerm_resource_group.osrg.location}"
 30 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 31 | 
 32 |   security_rule {
 33 |     name                       = "allowSSHin_all"
 34 |     description                = "Allow SSH in from all locations"
 35 |     priority                   = 100
 36 |     direction                  = "Inbound"
 37 |     access                     = "Allow"
 38 |     protocol                   = "Tcp"
 39 |     source_port_range          = "*"
 40 |     destination_port_range     = "22"
 41 |     source_address_prefix      = "*"
 42 |     destination_address_prefix = "*"
 43 |   }
 44 | 
 45 |   security_rule {
 46 |     name                       = "allowHTTPS_all"
 47 |     description                = "Allow HTTPS connections from all locations"
 48 |     priority                   = 200
 49 |     direction                  = "Inbound"
 50 |     access                     = "Allow"
 51 |     protocol                   = "Tcp"
 52 |     source_port_range          = "*"
 53 |     destination_port_range     = "443"
 54 |     source_address_prefix      = "*"
 55 |     destination_address_prefix = "*"
 56 |   }
 57 | 
 58 |   security_rule {
 59 |     name                       = "allowOpenShiftConsoleIn_all"
 60 |     description                = "Allow OpenShift Console connections from all locations"
 61 |     priority                   = 300
 62 |     direction                  = "Inbound"
 63 |     access                     = "Allow"
 64 |     protocol                   = "TCP"
 65 |     source_port_range          = "*"
 66 |     destination_port_range     = "8443"
 67 |     source_address_prefix      = "*"
 68 |     destination_address_prefix = "*"
 69 |   }
 70 | }
 71 | 
 72 | resource "azurerm_network_security_group" "osnodensg" {
 73 |   name                = "${var.openshift_azure_resource_prefix}-nsg-node-${var.openshift_azure_resource_suffix}"
 74 |   location            = "${azurerm_resource_group.osrg.location}"
 75 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 76 | 
 77 |   security_rule {
 78 |     name                       = "allowSSHin_all"
 79 |     description                = "Allow SSH in from all locations"
 80 |     priority                   = 100
 81 |     direction                  = "Inbound"
 82 |     access                     = "Allow"
 83 |     protocol                   = "*"
 84 |     source_port_range          = "*"
 85 |     destination_port_range     = "22"
 86 |     source_address_prefix      = "*"
 87 |     destination_address_prefix = "*"
 88 |   }
 89 | 
 90 |   security_rule {
 91 |     name                       = "allowHTTPSIn_all"
 92 |     description                = "Allow HTTPS traffic from the Internet to Public Agents"
 93 |     priority                   = 200
 94 |     direction                  = "Inbound"
 95 |     access                     = "Allow"
 96 |     protocol                   = "*"
 97 |     source_port_range          = "*"
 98 |     destination_port_range     = "443"
 99 |     source_address_prefix      = "*"
100 |     destination_address_prefix = "*"
101 |   }
102 | 
103 |   security_rule {
104 |     name                       = "allowHTTPIn_all"
105 |     description                = "Allow HTTP connections from all locations"
106 |     priority                   = 300
107 |     direction                  = "Inbound"
108 |     access                     = "Allow"
109 |     protocol                   = "*"
110 |     source_port_range          = "*"
111 |     destination_port_range     = "80"
112 |     source_address_prefix      = "*"
113 |     destination_address_prefix = "*"
114 |   }
115 | }
116 | 
117 | resource "azurerm_network_security_group" "osinfransg" {
118 |   name                = "${var.openshift_azure_resource_prefix}-nsg-infra-${var.openshift_azure_resource_suffix}"
119 |   location            = "${azurerm_resource_group.osrg.location}"
120 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
121 | 
122 |   security_rule {
123 |     name                       = "allowSSHin_all"
124 |     description                = "Allow SSH in from all locations"
125 |     priority                   = 100
126 |     direction                  = "Inbound"
127 |     access                     = "Allow"
128 |     protocol                   = "*"
129 |     source_port_range          = "*"
130 |     destination_port_range     = "22"
131 |     source_address_prefix      = "*"
132 |     destination_address_prefix = "*"
133 |   }
134 | 
135 |   security_rule {
136 |     name                       = "allowHTTPSIn_all"
137 |     description                = "Allow HTTPS traffic from the Internet to Public Agents"
138 |     priority                   = 200
139 |     direction                  = "Inbound"
140 |     access                     = "Allow"
141 |     protocol                   = "*"
142 |     source_port_range          = "*"
143 |     destination_port_range     = "443"
144 |     source_address_prefix      = "*"
145 |     destination_address_prefix = "*"
146 |   }
147 | 
148 |   security_rule {
149 |     name                       = "allowHTTPIn_all"
150 |     description                = "Allow HTTP connections from all locations"
151 |     priority                   = 300
152 |     direction                  = "Inbound"
153 |     access                     = "Allow"
154 |     protocol                   = "*"
155 |     source_port_range          = "*"
156 |     destination_port_range     = "80"
157 |     source_address_prefix      = "*"
158 |     destination_address_prefix = "*"
159 |   }
160 | }
161 | 


--------------------------------------------------------------------------------
/node.tf:
--------------------------------------------------------------------------------
 1 | resource "azurerm_availability_set" "osnodeas" {
 2 |   name                = "${var.openshift_azure_resource_prefix}-as-node-${var.openshift_azure_resource_suffix}"
 3 |   location            = "${azurerm_resource_group.osrg.location}"
 4 |   resource_group_name = "${azurerm_resource_group.osrg.name}"
 5 |   managed             = true
 6 | }
 7 | 
 8 | resource "azurerm_network_interface" "osnodenic" {
 9 |   name                      = "${var.openshift_azure_resource_prefix}-nic-node-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
10 |   count                     = "${var.openshift_azure_node_vm_count}"
11 |   location                  = "${azurerm_resource_group.osrg.location}"
12 |   resource_group_name       = "${azurerm_resource_group.osrg.name}"
13 |   network_security_group_id = "${azurerm_network_security_group.osnodensg.id}"
14 | 
15 |   ip_configuration {
16 |     name                          = "configuration-${count.index}"
17 |     subnet_id                     = "${azurerm_subnet.osnodesubnet.id}"
18 |     private_ip_address_allocation = "dynamic"
19 |   }
20 | }
21 | 
22 | resource "azurerm_virtual_machine" "osnodevm" {
23 |   name                  = "${var.openshift_azure_resource_prefix}-vm-node-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
24 |   count                 = "${var.openshift_azure_node_vm_count}"
25 |   location              = "${azurerm_resource_group.osrg.location}"
26 |   resource_group_name   = "${azurerm_resource_group.osrg.name}"
27 |   network_interface_ids = ["${element(azurerm_network_interface.osnodenic.*.id, count.index)}"]
28 |   availability_set_id   = "${azurerm_availability_set.osnodeas.id}"
29 |   vm_size               = "${var.openshift_azure_node_vm_size}"
30 | 
31 |   storage_image_reference {
32 |     publisher = "${var.openshift_azure_vm_os["publisher"]}"
33 |     offer     = "${var.openshift_azure_vm_os["offer"]}"
34 |     sku       = "${var.openshift_azure_vm_os["sku"]}"
35 |     version   = "${var.openshift_azure_vm_os["version"]}"
36 |   }
37 | 
38 |   storage_os_disk {
39 |     name              = "${var.openshift_azure_resource_prefix}-disk-os-node-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
40 |     caching           = "ReadWrite"
41 |     create_option     = "FromImage"
42 |     managed_disk_type = "Standard_LRS"
43 |   }
44 | 
45 |   storage_data_disk {
46 |     name              = "${var.openshift_azure_resource_prefix}-disk-data-node-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
47 |     managed_disk_type = "Standard_LRS"
48 |     create_option     = "Empty"
49 |     lun               = 0
50 |     disk_size_gb      = "${var.openshift_azure_data_disk_size}"
51 |   }
52 | 
53 |   os_profile {
54 |     computer_name  = "${var.openshift_azure_resource_prefix}-vm-node-${var.openshift_azure_resource_suffix}-${format("%01d", count.index)}"
55 |     admin_username = "${var.openshift_azure_vm_username}"
56 |     admin_password = "${uuid()}"
57 |   }
58 | 
59 |   os_profile_linux_config {
60 |     disable_password_authentication = true
61 | 
62 |     ssh_keys {
63 |       path     = "/home/${var.openshift_azure_vm_username}/.ssh/authorized_keys"
64 |       key_data = "${file(var.openshift_azure_public_key)}"
65 |     }
66 |   }
67 | }
68 | 
69 | resource "azurerm_virtual_machine_extension" "osnodevmextension" {
70 |   name                 = "osnodevmextension"
71 |   count                = "${var.openshift_azure_node_vm_count}"
72 |   location             = "${azurerm_resource_group.osrg.location}"
73 |   resource_group_name  = "${azurerm_resource_group.osrg.name}"
74 |   virtual_machine_name = "${element(azurerm_virtual_machine.osnodevm.*.name, count.index)}"
75 |   publisher            = "Microsoft.Azure.Extensions"
76 |   type                 = "CustomScript"
77 |   type_handler_version = "2.0"
78 | 
79 |   settings = <<SETTINGS
80 |     {
81 |         "fileUris": [
82 |             "${var.openshift_azure_node_prep_script}"
83 |         ],
84 |         "commandToExecute": "bash nodePrep.sh ${azurerm_storage_account.osstoragepv.name} ${var.openshift_azure_vm_username}"
85 |     }
86 | SETTINGS
87 | }
88 | 


--------------------------------------------------------------------------------
/scripts/deployOpenShift.sh:
--------------------------------------------------------------------------------
  1 | #!/bin/bash
  2 | 
  3 | echo $(date) " - Starting Script"
  4 | 
  5 | set -e
  6 | 
  7 | SUDOUSER=$1
  8 | PASSWORD="$2"
  9 | PRIVATEKEY=$3
 10 | MASTER=$4
 11 | MASTERPUBLICIPHOSTNAME=$5
 12 | MASTERPUBLICIPADDRESS=$6
 13 | INFRA=$7
 14 | NODE=$8
 15 | NODECOUNT=$9
 16 | INFRACOUNT=${10}
 17 | MASTERCOUNT=${11}
 18 | ROUTING=${12}
 19 | REGISTRYSA=${13}
 20 | ACCOUNTKEY="${14}"
 21 | TENANTID=${15}
 22 | SUBSCRIPTIONID=${16}
 23 | AADCLIENTID=${17}
 24 | AADCLIENTSECRET="${18}"
 25 | RESOURCEGROUP=${19}
 26 | LOCATION=${20}
 27 | STORAGEACCOUNT1=${21}
 28 | SAKEY1=${22}
 29 | OANSIBLEURL="${23}"
 30 | OANSIBLEBRANCH="${24}"
 31 | 
 32 | MASTERLOOP=$((MASTERCOUNT - 1))
 33 | INFRALOOP=$((INFRACOUNT - 1))
 34 | NODELOOP=$((NODECOUNT - 1))
 35 | 
 36 | # Generate private keys for use by Ansible
 37 | echo $(date) " - Generating Private keys for use by Ansible for OpenShift Installation"
 38 | 
 39 | runuser -l $SUDOUSER -c "(echo \"$PRIVATEKEY\" | base64 -d) > ~/.ssh/id_rsa"
 40 | runuser -l $SUDOUSER -c "chmod 600 ~/.ssh/id_rsa*"
 41 | 
 42 | echo $(date) "- Configuring SSH ControlPath to use shorter path name"
 43 | 
 44 | sed -i -e "s/^# control_path = %(directory)s\/%%h-%%r/control_path = %(directory)s\/%%h-%%r/" /etc/ansible/ansible.cfg
 45 | sed -i -e "s/^#host_key_checking = False/host_key_checking = False/" /etc/ansible/ansible.cfg
 46 | sed -i -e "s/^#pty=False/pty=False/" /etc/ansible/ansible.cfg
 47 | 
 48 | # Temporary workaround: https://access.redhat.com/solutions/3165971
 49 | mkdir -p /etc/origin/node/
 50 | touch /etc/origin/node/resolv.conf
 51 | 
 52 | # Create playbook to update ansible.cfg file
 53 | 
 54 | cat > updateansiblecfg.yaml <<EOF
 55 | #!/usr/bin/ansible-playbook
 56 | 
 57 | - hosts: localhost
 58 |   gather_facts: no
 59 |   tasks:
 60 |   - lineinfile:
 61 |       dest: /etc/ansible/ansible.cfg
 62 |       regexp: '^library '
 63 |       insertafter: '#library        = /usr/share/my_modules/'
 64 |       line: 'library = /home/${SUDOUSER}/openshift-ansible/library/'
 65 | EOF
 66 | 
 67 | # Run Ansible Playbook to update ansible.cfg file
 68 | 
 69 | echo $(date) " - Updating ansible.cfg file"
 70 | 
 71 | ansible-playbook ./updateansiblecfg.yaml
 72 | 
 73 | # Create Ansible Playbooks for Post Installation tasks
 74 | echo $(date) " - Create Ansible Playbooks for Post Installation tasks"
 75 | 
 76 | # Run on all masters - Create Inital OpenShift User on all Masters
 77 | 
 78 | cat > /home/${SUDOUSER}/addocpuser.yml <<EOF
 79 | ---
 80 | - hosts: masters
 81 |   gather_facts: no
 82 |   remote_user: ${SUDOUSER}
 83 |   become: yes
 84 |   become_method: sudo
 85 |   vars:
 86 |     description: "Create OpenShift Users"
 87 |   tasks:
 88 |   - name: create directory
 89 |     file: path=/etc/origin/master state=directory
 90 |   - name: add initial OpenShift user
 91 |     shell: htpasswd -cb /etc/origin/master/htpasswd ${SUDOUSER} "${PASSWORD}"
 92 | EOF
 93 | 
 94 | # Run on MASTER-0 - Make initial OpenShift User a Cluster Admin
 95 | 
 96 | cat > /home/${SUDOUSER}/assignclusteradminrights.yml <<EOF
 97 | ---
 98 | - hosts: master0
 99 |   gather_facts: no
100 |   remote_user: ${SUDOUSER}
101 |   become: yes
102 |   become_method: sudo
103 |   vars:
104 |     description: "Make user cluster admin"
105 |   tasks:
106 |   - name: make OpenShift user cluster admin
107 |     shell: oadm policy add-cluster-role-to-user cluster-admin $SUDOUSER --config=/etc/origin/master/admin.kubeconfig
108 | EOF
109 | 
110 | # Run on MASTER-0 - configure registry to use Azure Storage
111 | 
112 | cat > /home/${SUDOUSER}/dockerregistry.yml <<EOF
113 | ---
114 | - hosts: master0
115 |   gather_facts: no
116 |   remote_user: ${SUDOUSER}
117 |   become: yes
118 |   become_method: sudo
119 |   vars:
120 |     description: "Set registry to use Azure Storage"
121 |   tasks:
122 |   - name: Configure docker-registry to use Azure Storage
123 |     shell: oc env dc docker-registry -e REGISTRY_STORAGE=azure -e REGISTRY_STORAGE_AZURE_ACCOUNTNAME=$REGISTRYSA -e REGISTRY_STORAGE_AZURE_ACCOUNTKEY=$ACCOUNTKEY -e REGISTRY_STORAGE_AZURE_CONTAINER=registry
124 | EOF
125 | 
126 | # Run on MASTER-0 - configure Storage Class
127 | 
128 | cat > /home/${SUDOUSER}/configurestorageclass.yml <<EOF
129 | ---
130 | - hosts: master0
131 |   gather_facts: no
132 |   remote_user: ${SUDOUSER}
133 |   become: yes
134 |   become_method: sudo
135 |   vars:
136 |     description: "Create Storage Class"
137 |   tasks:
138 |   - name: Create Storage Class with StorageAccountPV1
139 |     shell: oc create -f /home/${SUDOUSER}/scgeneric1.yml
140 | EOF
141 | 
142 | # Create vars.yml file for use by setup-azure-config.yml playbook
143 | 
144 | cat > /home/${SUDOUSER}/vars.yml <<EOF
145 | g_tenantId: $TENANTID
146 | g_subscriptionId: $SUBSCRIPTIONID
147 | g_aadClientId: $AADCLIENTID
148 | g_aadClientSecret: $AADCLIENTSECRET
149 | g_resourceGroup: $RESOURCEGROUP
150 | g_location: $LOCATION
151 | EOF
152 | 
153 | # Create Azure Cloud Provider configuration Playbook for Master Config
154 | 
155 | if [ $MASTERCOUNT -eq 1 ]
156 | then
157 | 
158 | # Single Master Configuration
159 | 
160 | cat > /home/${SUDOUSER}/setup-azure-master.yml <<EOF
161 | #!/usr/bin/ansible-playbook
162 | - hosts: masters
163 |   gather_facts: no
164 |   serial: 1
165 |   vars_files:
166 |   - vars.yml
167 |   become: yes
168 |   vars:
169 |     azure_conf_dir: /etc/azure
170 |     azure_conf: "{{ azure_conf_dir }}/azure.conf"
171 |     master_conf: /etc/origin/master/master-config.yaml
172 |   handlers:
173 |   - name: restart origin-master-api
174 |     systemd:
175 |       state: restarted
176 |       name: origin-master-api
177 | 
178 |   - name: restart origin-master-controllers
179 |     systemd:
180 |       state: restarted
181 |       name: origin-master-controllers
182 | 
183 |   post_tasks:
184 |   - name: make sure /etc/azure exists
185 |     file:
186 |       state: directory
187 |       path: "{{ azure_conf_dir }}"
188 | 
189 |   - name: populate /etc/azure/azure.conf
190 |     copy:
191 |       dest: "{{ azure_conf }}"
192 |       content: |
193 |         {
194 |           "aadClientID" : "{{ g_aadClientId }}",
195 |           "aadClientSecret" : "{{ g_aadClientSecret }}",
196 |           "subscriptionID" : "{{ g_subscriptionId }}",
197 |           "tenantID" : "{{ g_tenantId }}",
198 |           "resourceGroup": "{{ g_resourceGroup }}",
199 |         }
200 |     notify:
201 |     - restart origin-master-api
202 |     - restart origin-master-controllers
203 | 
204 |   - name: insert the azure disk config into the master
205 |     modify_yaml:
206 |       dest: "{{ master_conf }}"
207 |       yaml_key: "{{ item.key }}"
208 |       yaml_value: "{{ item.value }}"
209 |     with_items:
210 |     - key: kubernetesMasterConfig.apiServerArguments.cloud-config
211 |       value:
212 |       - "{{ azure_conf }}"
213 | 
214 |     - key: kubernetesMasterConfig.apiServerArguments.cloud-provider
215 |       value:
216 |       - azure
217 | 
218 |     - key: kubernetesMasterConfig.controllerArguments.cloud-config
219 |       value:
220 |       - "{{ azure_conf }}"
221 | 
222 |     - key: kubernetesMasterConfig.controllerArguments.cloud-provider
223 |       value:
224 |       - azure
225 |     notify:
226 |     - restart origin-master-api
227 |     - restart origin-master-controllers
228 | EOF
229 | 
230 | else
231 | 
232 | # Multiple Master Configuration
233 | 
234 | cat > /home/${SUDOUSER}/setup-azure-master.yml <<EOF
235 | #!/usr/bin/ansible-playbook
236 | - hosts: masters
237 |   gather_facts: no
238 |   serial: 1
239 |   vars_files:
240 |   - vars.yml
241 |   become: yes
242 |   vars:
243 |     azure_conf_dir: /etc/azure
244 |     azure_conf: "{{ azure_conf_dir }}/azure.conf"
245 |     master_conf: /etc/origin/master/master-config.yaml
246 |   handlers:
247 |   - name: restart origin-master-api
248 |     systemd:
249 |       state: restarted
250 |       name: origin-master-api
251 | 
252 |   - name: restart origin-master-controllers
253 |     systemd:
254 |       state: restarted
255 |       name: origin-master-controllers
256 | 
257 |   post_tasks:
258 |   - name: make sure /etc/azure exists
259 |     file:
260 |       state: directory
261 |       path: "{{ azure_conf_dir }}"
262 | 
263 |   - name: populate /etc/azure/azure.conf
264 |     copy:
265 |       dest: "{{ azure_conf }}"
266 |       content: |
267 |         {
268 |           "aadClientID" : "{{ g_aadClientId }}",
269 |           "aadClientSecret" : "{{ g_aadClientSecret }}",
270 |           "subscriptionID" : "{{ g_subscriptionId }}",
271 |           "tenantID" : "{{ g_tenantId }}",
272 |           "resourceGroup": "{{ g_resourceGroup }}",
273 |         }
274 |     notify:
275 |     - restart origin-master-api
276 |     - restart origin-master-controllers
277 | 
278 |   - name: insert the azure disk config into the master
279 |     modify_yaml:
280 |       dest: "{{ master_conf }}"
281 |       yaml_key: "{{ item.key }}"
282 |       yaml_value: "{{ item.value }}"
283 |     with_items:
284 |     - key: kubernetesMasterConfig.apiServerArguments.cloud-config
285 |       value:
286 |       - "{{ azure_conf }}"
287 | 
288 |     - key: kubernetesMasterConfig.apiServerArguments.cloud-provider
289 |       value:
290 |       - azure
291 | 
292 |     - key: kubernetesMasterConfig.controllerArguments.cloud-config
293 |       value:
294 |       - "{{ azure_conf }}"
295 | 
296 |     - key: kubernetesMasterConfig.controllerArguments.cloud-provider
297 |       value:
298 |       - azure
299 |     notify:
300 |     - restart origin-master-api
301 |     - restart origin-master-controllers
302 | EOF
303 | 
304 | fi
305 | 
306 | # Create Azure Cloud Provider configuration Playbook for Node Config (Master Nodes)
307 | 
308 | cat > /home/${SUDOUSER}/setup-azure-node-master.yml <<EOF
309 | #!/usr/bin/ansible-playbook
310 | - hosts: masters
311 |   serial: 1
312 |   gather_facts: no
313 |   vars_files:
314 |   - vars.yml
315 |   become: yes
316 |   vars:
317 |     azure_conf_dir: /etc/azure
318 |     azure_conf: "{{ azure_conf_dir }}/azure.conf"
319 |     node_conf: /etc/origin/node/node-config.yaml
320 |   handlers:
321 |   - name: restart origin-node
322 |     systemd:
323 |       state: restarted
324 |       name: origin-node
325 |   post_tasks:
326 |   - name: make sure /etc/azure exists
327 |     file:
328 |       state: directory
329 |       path: "{{ azure_conf_dir }}"
330 | 
331 |   - name: populate /etc/azure/azure.conf
332 |     copy:
333 |       dest: "{{ azure_conf }}"
334 |       content: |
335 |         {
336 |           "aadClientID" : "{{ g_aadClientId }}",
337 |           "aadClientSecret" : "{{ g_aadClientSecret }}",
338 |           "subscriptionID" : "{{ g_subscriptionId }}",
339 |           "tenantID" : "{{ g_tenantId }}",
340 |           "resourceGroup": "{{ g_resourceGroup }}",
341 |         }
342 |     notify:
343 |     - restart origin-node
344 |   - name: insert the azure disk config into the node
345 |     modify_yaml:
346 |       dest: "{{ node_conf }}"
347 |       yaml_key: "{{ item.key }}"
348 |       yaml_value: "{{ item.value }}"
349 |     with_items:
350 |     - key: kubeletArguments.cloud-config
351 |       value:
352 |       - "{{ azure_conf }}"
353 | 
354 |     - key: kubeletArguments.cloud-provider
355 |       value:
356 |       - azure
357 |     notify:
358 |     - restart origin-node
359 | EOF
360 | 
361 | # Create Azure Cloud Provider configuration Playbook for Node Config (Non-Master Nodes)
362 | 
363 | cat > /home/${SUDOUSER}/setup-azure-node.yml <<EOF
364 | #!/usr/bin/ansible-playbook
365 | - hosts: nodes:!masters
366 |   serial: 1
367 |   gather_facts: no
368 |   vars_files:
369 |   - vars.yml
370 |   become: yes
371 |   vars:
372 |     azure_conf_dir: /etc/azure
373 |     azure_conf: "{{ azure_conf_dir }}/azure.conf"
374 |     node_conf: /etc/origin/node/node-config.yaml
375 |   handlers:
376 |   - name: restart origin-node
377 |     systemd:
378 |       state: restarted
379 |       name: origin-node
380 |   post_tasks:
381 |   - name: make sure /etc/azure exists
382 |     file:
383 |       state: directory
384 |       path: "{{ azure_conf_dir }}"
385 | 
386 |   - name: populate /etc/azure/azure.conf
387 |     copy:
388 |       dest: "{{ azure_conf }}"
389 |       content: |
390 |         {
391 |           "aadClientID" : "{{ g_aadClientId }}",
392 |           "aadClientSecret" : "{{ g_aadClientSecret }}",
393 |           "subscriptionID" : "{{ g_subscriptionId }}",
394 |           "tenantID" : "{{ g_tenantId }}",
395 |           "resourceGroup": "{{ g_resourceGroup }}",
396 |         }
397 |     notify:
398 |     - restart origin-node
399 |   - name: insert the azure disk config into the node
400 |     modify_yaml:
401 |       dest: "{{ node_conf }}"
402 |       yaml_key: "{{ item.key }}"
403 |       yaml_value: "{{ item.value }}"
404 |     with_items:
405 |     - key: kubeletArguments.cloud-config
406 |       value:
407 |       - "{{ azure_conf }}"
408 | 
409 |     - key: kubeletArguments.cloud-provider
410 |       value:
411 |       - azure
412 |     notify:
413 |     - restart origin-node
414 |   - name: delete the node so it can recreate itself
415 |     command: oc delete node {{inventory_hostname}}
416 |     delegate_to: ${MASTER}-0
417 |   - name: sleep to let node come back to life
418 |     pause:
419 |        seconds: 90
420 | EOF
421 | 
422 | # Create Playbook to delete stuck Master nodes and set as not schedulable
423 | 
424 | cat > /home/${SUDOUSER}/deletestucknodes.yml <<EOF
425 | - hosts: masters
426 |   gather_facts: no
427 |   become: yes
428 |   vars:
429 |     description: "Delete stuck nodes"
430 |   tasks:
431 |   - name: Delete stuck nodes so it can recreate itself
432 |     command: oc delete node {{inventory_hostname}}
433 |     delegate_to: ${MASTER}-0
434 |   - name: sleep between deletes
435 |     pause:
436 |       seconds: 25
437 |   - name: set masters as unschedulable
438 |     command: oadm manage-node {{inventory_hostname}} --schedulable=false
439 | EOF
440 | 
441 | # Create Ansible Hosts File
442 | echo $(date) " - Create Ansible Hosts file"
443 | 
444 | if [ $MASTERCOUNT -eq 1 ]
445 | then
446 | 
447 | # Ansible Host file for Single Master Configuration
448 | 
449 | cat > /etc/ansible/hosts <<EOF
450 | # Create an OSEv3 group that contains the masters and nodes groups
451 | [OSEv3:children]
452 | masters
453 | nodes
454 | master0
455 | new_nodes
456 | 
457 | # Set variables common for all OSEv3 hosts
458 | [OSEv3:vars]
459 | ansible_ssh_user=$SUDOUSER
460 | ansible_become=yes
461 | openshift_install_examples=true
462 | openshift_deployment_type=origin
463 | openshift_release=v3.6
464 | docker_udev_workaround=True
465 | openshift_use_dnsmasq=True
466 | openshift_master_default_subdomain=$ROUTING
467 | openshift_override_hostname_check=true
468 | osm_use_cockpit=false
469 | #os_sdn_network_plugin_name='redhat/openshift-ovs-multitenant'
470 | #console_port=443
471 | openshift_cloudprovider_kind=azure
472 | osm_default_node_selector='type=app'
473 | openshift_disable_check=disk_availability,memory_availability
474 | os_firewall_use_firewalld=False
475 | 
476 | # default selectors for router and registry services
477 | openshift_router_selector='type=infra'
478 | openshift_registry_selector='type=infra'
479 | 
480 | openshift_master_cluster_hostname=$MASTERPUBLICIPHOSTNAME
481 | openshift_master_cluster_public_hostname=$MASTERPUBLICIPHOSTNAME
482 | openshift_master_cluster_public_vip=$MASTERPUBLICIPADDRESS
483 | 
484 | # Enable HTPasswdPasswordIdentityProvider for username / password authentication for OpenShift Cluster
485 | openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]
486 | 
487 | # host group for masters
488 | [masters]
489 | $MASTER-0
490 | 
491 | [master0]
492 | $MASTER-0
493 | 
494 | # host group for etcd
495 | [etcd]
496 | $MASTER-0
497 | 
498 | # host group for nodes
499 | [nodes]
500 | $MASTER-0 openshift_node_labels="{'type': 'master', 'zone': 'default'}" openshift_hostname=$MASTER-0
501 | EOF
502 | 
503 | # Loop to add Infra Nodes
504 | 
505 | for (( c=0; c<$INFRACOUNT; c++ ))
506 | do
507 |   echo "$INFRA-$c openshift_node_labels=\"{'type': 'infra', 'zone': 'default'}\" openshift_hostname=$INFRA-$c" >> /etc/ansible/hosts
508 | done
509 | 
510 | # Loop to add Nodes
511 | 
512 | for (( c=0; c<$NODECOUNT; c++ ))
513 | do
514 |   echo "$NODE-$c openshift_node_labels=\"{'type': 'app', 'zone': 'default'}\" openshift_hostname=$NODE-$c" >> /etc/ansible/hosts
515 | done
516 | 
517 | # Create new_nodes group
518 | 
519 | cat >> /etc/ansible/hosts <<EOF
520 | 
521 | # host group for adding new nodes
522 | [new_nodes]
523 | EOF
524 | 
525 | else
526 | 
527 | # Ansible Host file for Multiple Master Configuration
528 | 
529 | cat > /etc/ansible/hosts <<EOF
530 | # Create an OSEv3 group that contains the masters and nodes groups
531 | [OSEv3:children]
532 | masters
533 | nodes
534 | etcd
535 | master0
536 | new_nodes
537 | 
538 | # Set variables common for all OSEv3 hosts
539 | [OSEv3:vars]
540 | ansible_ssh_user=$SUDOUSER
541 | ansible_become=yes
542 | openshift_install_examples=true
543 | openshift_deployment_type=origin
544 | openshift_release=v3.6
545 | #openshift_image_tag=v1.5.0
546 | docker_udev_workaround=True
547 | openshift_use_dnsmasq=True
548 | openshift_master_default_subdomain=$ROUTING
549 | openshift_override_hostname_check=true
550 | osm_use_cockpit=false
551 | #os_sdn_network_plugin_name='redhat/openshift-ovs-multitenant'
552 | #console_port=443
553 | openshift_cloudprovider_kind=azure
554 | osm_default_node_selector='type=app'
555 | openshift_disable_check=disk_availability,memory_availability
556 | os_firewall_use_firewalld=False
557 | 
558 | # default selectors for router and registry services
559 | openshift_router_selector='type=infra'
560 | openshift_registry_selector='type=infra'
561 | 
562 | openshift_master_cluster_method=native
563 | openshift_master_cluster_hostname=$MASTERPUBLICIPHOSTNAME
564 | openshift_master_cluster_public_hostname=$MASTERPUBLICIPHOSTNAME
565 | openshift_master_cluster_public_vip=$MASTERPUBLICIPADDRESS
566 | 
567 | # Enable HTPasswdPasswordIdentityProvider
568 | openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]
569 | 
570 | # host group for masters
571 | [masters]
572 | $MASTER-[0:${MASTERLOOP}]
573 | 
574 | # host group for etcd
575 | [etcd]
576 | $MASTER-[0:${MASTERLOOP}]
577 | 
578 | [master0]
579 | $MASTER-0
580 | 
581 | # host group for nodes
582 | [nodes]
583 | EOF
584 | 
585 | # Loop to add Masters
586 | 
587 | for (( c=0; c<$MASTERCOUNT; c++ ))
588 | do
589 |   echo "$MASTER-$c openshift_node_labels=\"{'type': 'master', 'zone': 'default'}\" openshift_hostname=$MASTER-$c" >> /etc/ansible/hosts
590 | done
591 | 
592 | # Loop to add Infra Nodes
593 | 
594 | for (( c=0; c<$INFRACOUNT; c++ ))
595 | do
596 |   echo "$INFRA-$c openshift_node_labels=\"{'type': 'infra', 'zone': 'default'}\" openshift_hostname=$INFRA-$c" >> /etc/ansible/hosts
597 | done
598 | 
599 | # Loop to add Nodes
600 | 
601 | for (( c=0; c<$NODECOUNT; c++ ))
602 | do
603 |   echo "$NODE-$c openshift_node_labels=\"{'type': 'app', 'zone': 'default'}\" openshift_hostname=$NODE-$c" >> /etc/ansible/hosts
604 | done
605 | 
606 | # Create new_nodes group
607 | 
608 | cat >> /etc/ansible/hosts <<EOF
609 | 
610 | # host group for adding new nodes
611 | [new_nodes]
612 | EOF
613 | 
614 | fi
615 | 
616 | # Initiating installation of OpenShift Container Platform using Ansible Playbook
617 | echo $(date) " - Installing OpenShift Container Platform via Ansible Playbook"
618 | 
619 | runuser -l $SUDOUSER -c "git clone ${OANSIBLEURL} /home/$SUDOUSER/openshift-ansible && cd /home/$SUDOUSER/openshift-ansible && git checkout ${OANSIBLEBRANCH}"
620 | 
621 | echo $(date) " - Running network_manager.yml playbook"
622 | DOMAIN=`domainname -d`
623 | # Setup NetworkManager to manage eth0
624 | runuser -l $SUDOUSER -c "ansible-playbook openshift-ansible/playbooks/byo/openshift-node/network_manager.yml"
625 | 
626 | echo $(date) " - Setting up NetworkManager on eth0"
627 | # Configure resolv.conf on all hosts through NetworkManager
628 | runuser -l $SUDOUSER -c "ansible all -b -m service -a \"name=NetworkManager state=restarted\""
629 | sleep 5
630 | runuser -l $SUDOUSER -c "ansible all -b -m command -a \"nmcli con modify eth0 ipv4.dns-search $DOMAIN\""
631 | runuser -l $SUDOUSER -c "ansible all -b -m service -a \"name=NetworkManager state=restarted\""
632 | 
633 | echo $(date) " - Running config.yml playbook"
634 | runuser -l $SUDOUSER -c "ansible-playbook openshift-ansible/playbooks/byo/config.yml"
635 | 
636 | echo $(date) " - Modifying sudoers"
637 | 
638 | sed -i -e "s/Defaults    requiretty/# Defaults    requiretty/" /etc/sudoers
639 | sed -i -e '/Defaults    env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY"/aDefaults    env_keep += "PATH"' /etc/sudoers
640 | 
641 | # Deploying Registry
642 | echo $(date) "- Registry automatically deployed to infra nodes"
643 | 
644 | # Deploying Router
645 | echo $(date) "- Router automaticaly deployed to infra nodes"
646 | 
647 | echo $(date) "- Re-enabling requiretty"
648 | 
649 | sed -i -e "s/# Defaults    requiretty/Defaults    requiretty/" /etc/sudoers
650 | 
651 | # Adding user to OpenShift authentication file
652 | echo $(date) "- Adding OpenShift user"
653 | 
654 | runuser -l $SUDOUSER -c "ansible-playbook ~/addocpuser.yml"
655 | 
656 | # Assigning cluster admin rights to OpenShift user
657 | echo $(date) "- Assigning cluster admin rights to user"
658 | 
659 | runuser -l $SUDOUSER -c "ansible-playbook ~/assignclusteradminrights.yml"
660 | 
661 | # Create Storage Class
662 | echo $(date) "- Creating Storage Class"
663 | 
664 | runuser -l $SUDOUSER -c "ansible-playbook ~/configurestorageclass.yml"
665 | 
666 | # Configure Docker Registry to use Azure Storage Account
667 | echo $(date) "- Configuring Docker Registry to use Azure Storage Account"
668 | 
669 | runuser -l $SUDOUSER -c "ansible-playbook ~/dockerregistry.yml"
670 | 
671 | echo $(date) "- Sleep for 120"
672 | 
673 | sleep 120
674 | 
675 | # Execute setup-azure-master and setup-azure-node playbooks to configure Azure Cloud Provider
676 | echo $(date) "- Configuring OpenShift Cloud Provider to be Azure"
677 | 
678 | runuser -l $SUDOUSER -c "ansible-playbook ~/setup-azure-master.yml"
679 | runuser -l $SUDOUSER -c "ansible-playbook ~/setup-azure-node-master.yml"
680 | runuser -l $SUDOUSER -c "ansible-playbook ~/setup-azure-node.yml"
681 | runuser -l $SUDOUSER -c "ansible-playbook ~/deletestucknodes.yml"
682 | 
683 | # Delete postinstall files
684 | echo $(date) "- Deleting post installation files"
685 | 
686 | rm /home/${SUDOUSER}/addocpuser.yml
687 | rm /home/${SUDOUSER}/assignclusteradminrights.yml
688 | rm /home/${SUDOUSER}/dockerregistry.yml
689 | rm /home/${SUDOUSER}/vars.yml
690 | rm /home/${SUDOUSER}/setup-azure-master.yml
691 | rm /home/${SUDOUSER}/setup-azure-node-master.yml
692 | rm /home/${SUDOUSER}/setup-azure-node.yml
693 | rm /home/${SUDOUSER}/deletestucknodes.yml
694 | 
695 | echo $(date) "- Script complete"
696 | 


--------------------------------------------------------------------------------
/scripts/masterPrep.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | echo $(date) " - Starting Script"
 3 | 
 4 | STORAGEACCOUNT1=$1
 5 | SUDOUSER=$2
 6 | 
 7 | # Update system to latest packages and install dependencies
 8 | echo $(date) " - Update system to latest packages and install dependencies"
 9 | 
10 | yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion httpd-tools
11 | yum -y update --exclude=WALinuxAgent
12 | 
13 | # Install EPEL repository
14 | echo $(date) " - Installing EPEL"
15 | 
16 | yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
17 | 
18 | sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo
19 | 
20 | # Only install Ansible and pyOpenSSL on Master-0 Node
21 | 
22 | if hostname -f|grep -- "-0" >/dev/null
23 | then
24 |    echo $(date) " - Installing Ansible and pyOpenSSL"
25 |    yum -y --enablerepo=epel install ansible pyOpenSSL
26 | fi
27 | 
28 | # Install Docker 1.12.x
29 | echo $(date) " - Installing Docker 1.12.x"
30 | 
31 | yum -y install docker
32 | sed -i -e "s#^OPTIONS='--selinux-enabled'#OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0/16'#" /etc/sysconfig/docker
33 | 
34 | # Create thin pool logical volume for Docker
35 | echo $(date) " - Creating thin pool logical volume for Docker and staring service"
36 | 
37 | DOCKERVG=$( parted -m /dev/sda print all 2>/dev/null | grep unknown | grep /dev/sd | cut -d':' -f1 )
38 | 
39 | echo "DEVS=${DOCKERVG}" >> /etc/sysconfig/docker-storage-setup
40 | echo "VG=docker-vg" >> /etc/sysconfig/docker-storage-setup
41 | docker-storage-setup
42 | if [ $? -eq 0 ]
43 | then
44 |    echo "Docker thin pool logical volume created successfully"
45 | else
46 |    echo "Error creating logical volume for Docker"
47 |    exit 5
48 | fi
49 | 
50 | # Enable and start Docker services
51 | 
52 | systemctl enable docker
53 | systemctl start docker
54 | 
55 | # Create Storage Class yml files on MASTER-0
56 | 
57 | if hostname -f|grep -- "-0" >/dev/null
58 | then
59 | cat <<EOF > /home/${SUDOUSER}/scgeneric1.yml
60 | kind: StorageClass
61 | apiVersion: storage.k8s.io/v1beta1
62 | metadata:
63 |   name: generic
64 |   annotations:
65 |     storageclass.beta.kubernetes.io/is-default-class: "true"
66 | provisioner: kubernetes.io/azure-disk
67 | parameters:
68 |   storageAccount: ${STORAGEACCOUNT1}
69 | EOF
70 | 
71 | fi
72 | 
73 | echo $(date) " - Script Complete"
74 | 


--------------------------------------------------------------------------------
/scripts/nodePrep.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | echo $(date) " - Starting Script"
 3 | 
 4 | # Update system to latest packages and install dependencies
 5 | echo $(date) " - Update system to latest packages and install dependencies"
 6 | 
 7 | yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion
 8 | yum -y update --exclude=WALinuxAgent
 9 | 
10 | # Install EPEL repository
11 | echo $(date) " - Installing EPEL"
12 | 
13 | yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
14 | 
15 | sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo
16 | 
17 | # Install Docker 1.12.x
18 | echo $(date) " - Installing Docker 1.12.x"
19 | 
20 | yum -y install docker
21 | sed -i -e "s#^OPTIONS='--selinux-enabled'#OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0/16'#" /etc/sysconfig/docker
22 | 
23 | # Create thin pool logical volume for Docker
24 | echo $(date) " - Creating thin pool logical volume for Docker and staring service"
25 | 
26 | DOCKERVG=$( parted -m /dev/sda print all 2>/dev/null | grep unknown | grep /dev/sd | cut -d':' -f1 )
27 | 
28 | echo "DEVS=${DOCKERVG}" >> /etc/sysconfig/docker-storage-setup
29 | echo "VG=docker-vg" >> /etc/sysconfig/docker-storage-setup
30 | docker-storage-setup
31 | if [ $? -eq 0 ]
32 | then
33 |    echo "Docker thin pool logical volume created successfully"
34 | else
35 |    echo "Error creating logical volume for Docker"
36 |    exit 5
37 | fi
38 | 
39 | # Enable and start Docker services
40 | 
41 | systemctl enable docker
42 | systemctl start docker
43 | 
44 | # Temporary workaround: https://access.redhat.com/solutions/3165971
45 | 
46 | mkdir -p /etc/origin/node/
47 | touch /etc/origin/node/resolv.conf
48 | 
49 | echo $(date) " - Script Complete"
50 | 


--------------------------------------------------------------------------------
/terraform.tfvars.example:
--------------------------------------------------------------------------------
 1 | azure_client_id = ""
 2 | azure_client_secret = ""
 3 | azure_tenant_id = ""
 4 | azure_subscription_id = ""
 5 | 
 6 | openshift_azure_resource_prefix = "os"
 7 | openshift_azure_resource_suffix = "test"
 8 | 
 9 | openshift_azure_resource_group = "osrg123"
10 | 
11 | openshift_azure_region = "East US"
12 | 
13 | openshift_azure_vm_username = "ocpadmin"
14 | openshift_initial_password = "password123"
15 | 
16 | openshift_azure_public_key = "~/.ssh/randomKey.pub"
17 | openshift_azure_private_key = "~/.ssh/randomKey"
18 | 
19 | openshift_azure_master_vm_count = 1
20 | openshift_azure_node_vm_count = 1
21 | openshift_azure_infra_vm_count = 1
22 | 
23 | openshift_azure_master_vm_size = "Standard_DS2_v2"
24 | openshift_azure_data_disk_size = 128
25 | 
26 | openshift_azure_vm_os = {
27 |   publisher = "OpenLogic"
28 |   offer     = "CentOS"
29 |   sku       = "7.3"
30 |   version   = "latest"
31 | }
32 | 
33 | openshift_master_dns_name = "osmaster"
34 | openshift_infra_dns_name = "osinfra"
35 | 
36 | openshift_azure_default_subdomain = "xip.io"


--------------------------------------------------------------------------------