├── .gitignore ├── README.md ├── deploy ├── auth │ ├── kubeadmin-password │ └── kubeconfig ├── bootstrap.ign ├── master.ign ├── metadata.json └── worker.ign ├── dns ├── 56.10.in-addr.arpa.db └── ocp4.sni.com.mx.db ├── go.sh ├── haproxy └── haproxy.conf ├── images ├── bootstrap.jpg ├── final_ip.jpg ├── manual.jpg ├── master.jpg ├── ova_template.jpg └── temp_ip.jpg └── terraform ├── apply ├── destroy ├── main.tf ├── modules ├── clone_from_template │ ├── main.tf │ ├── outputs.tf │ └── vars.tf ├── create_ignitions │ ├── main.tf │ ├── outputs.tf │ ├── templates │ │ ├── ifcfg-ens192.tpl │ │ └── systemd_restart │ └── vars.tf └── create_template │ ├── main.tf │ ├── outputs.tf │ └── vars.tf ├── outputs └── tf.output ├── plan └── vars └── common.tfvars /.gitignore: -------------------------------------------------------------------------------- 1 | openshift-install 2 | ocp-key 3 | ocp-key.pub 4 | .python-version 5 | deploy/.openshift* 6 | terraform/.terraform 7 | terraform/terraform.* 8 | terraform/*.ova 9 | terraform/*.ovf 10 | terraform/*.vmdk 11 | terraform/govc_env 12 | deploy/auth/* 13 | install-config.yaml 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ocp4-vmware-terraform 2 | # Update 05/13/2020 100% terraform 3 | 4 | The following procedure is intended to create VM's from an **OVA** template booting with **static IP's** when the DHCP server **can not** reserve the IP addresses. 5 | 6 | #### The problem 7 | OCP requires that all DNS configurations be in place. VMWare requires that the DHCP assign the correct IPs to the VM. Since many real installations require the coordination with different teams in an organization, many times we don't have control of DNS, DHCP or Loadbalancer configurations. 8 | 9 | Sometimes we need to do a **["bare metal"](https://docs.openshift.com/container-platform/4.3/installing/installing_bare_metal/installing-bare-metal.html)** installation over vmware to set the network configuration with kernel parameters at boot (ip, gateway, nameserver, etc.). 10 | 11 | The coreos [documentation](https://coreos.com/ignition/docs/latest/network-configuration.html) explain how to create configurations using ignition files. I created a python script to put the network configuration using the ignition files created by the openshift-install program. 12 | 13 | #### How does it work? 14 | When the VM boots, it will take the first IP provided by the DHCP server (probably will not be the IP set by the script). ``(...dhclient[836]: bound to 10.56.240.99 -- renewal in 18253 seconds.)`` 15 | 16 | ![Temporary IP](images/temp_ip.jpg "Temporary IP") 17 | 18 | At the end of the boot process the VM will take the IP provided by the ignition file. 19 | 20 | ![Final IP](images/final_ip.jpg "Final IP") 21 | 22 | # Start: 23 | ## Pre-requirements 24 | - Terraform latest version (v0.12.24) you can download [here](https://www.terraform.io/downloads.html) 25 | - Have the ignitions files already created (bootstrap.ign, master.ign, worker.ign). 26 | - Copy the 3 ignitions files to your webserver. 27 | - Download the OVA image from the [mirror repository](https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/latest/latest/rhcos-4.4.3-x86_64-vmware.x86_64.ova) to the ``terraform/`` folder. 28 | - **Optional** ``install-config.yaml`` file. 29 | - **Optional:** The fist section of **script** ``go.sh`` create the ignition files and copy to the webserver, if you already did this process you can delete that part and start from the **Terraforn deployment** section of the script. 30 | 31 | 32 | ## Procedure 33 | If you want to create all running the ``go.sh`` script. 34 | 1. Place the ``install-config.yaml`` in the root of this git repo. 35 | 36 | 2. Update the ``go.sh`` script with the path or webserver to copy the ignition files to the webserver. 37 | ```bash 38 | scp deploy/*.ign root@bastion.ocp4.sni.com.mx:/var/www/html/ 39 | ssh root@bastion.ocp4.sni.com.mx 'chmod 644 /var/www/html/*.ign' 40 | # cp -f deploy/*.ign /var/www/html/ 41 | # chmod 644 chmod 644 /var/www/html/*.ign 42 | ``` 43 | 44 | 3. Edit the file ``terraform/vars/common.tfvars`` with the values of your installation; vmware configuration and Openshift configuration. 45 | ```bash 46 | ## VSPHERE CONFIGURATIONS, SHOULD BE THE SAME IN YOU install-config.yaml 47 | vsphere_user = "administrator@sni.com.mx" 48 | vsphere_password = "Password123!" 49 | vsphere_server = "vcenter.sni.com.mx" 50 | # 51 | datacenter = "BHM" 52 | datastore = "SAS-6K" 53 | network = "VM Network" 54 | resource_pool = "Resources" 55 | host = "esxi67.sni.com.mx" 56 | . 57 | . 58 | . 59 | ``` 60 | 61 | 4. Extract the contents from the [OVA](https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/latest/latest/rhcos-4.4.3-x86_64-vmware.x86_64.ova) file to the ``terraform/`` folder, wi will get 2 files: ``coreos.ovf`` and ``disk.vmdk``. 62 | ```bash 63 | ../terraform$ tar xvf rhcos-4.4.3-x86_64-vmware.x86_64.ova 64 | coreos.ovf 65 | disk.vmdk 66 | ``` 67 | 68 | 5. Run the script ``go.sh``, and wait to finish. 69 | 70 | ```bash 71 | terraform$ ./go.sh 72 | +++ dirname ./go.sh 73 | ++ cd . 74 | ++ pwd 75 | + DIR=/home/alex/work/RedHat/ocp4-vmware-terraform 76 | + rm -fr deploy 77 | + mkdir deploy 78 | + cp install-config.yaml deploy/ 79 | + openshift-install create manifests --dir=deploy 80 | INFO Consuming Install Config from target directory 81 | WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings 82 | + sed -i s/true/false/g deploy/manifests/cluster-scheduler-02-config.yml 83 | + openshift-install create ignition-configs --dir=deploy 84 | INFO Consuming OpenShift Install (Manifests) from target directory 85 | INFO Consuming Openshift Manifests from target directory 86 | INFO Consuming Master Machines from target directory 87 | INFO Consuming Worker Machines from target directory 88 | INFO Consuming Common Manifests from target directory 89 | + scp deploy/bootstrap.ign deploy/master.ign deploy/worker.ign root@bastion.ocp4.sni.com.mx:/var/www/html/ 90 | Warning: Permanently added 'bastion.ocp4.sni.com.mx,10.56.241.10' (ECDSA) to the list of known hosts. 91 | bootstrap.ign 100% 299KB 20.5MB/s 00:00 92 | master.ign 100% 1820 994.2KB/s 00:00 93 | worker.ign 100% 1820 1.1MB/s 00:00 94 | + ssh root@bastion.ocp4.sni.com.mx 'chmod 644 /var/www/html/*.ign' 95 | Warning: Permanently added 'bastion.ocp4.sni.com.mx,10.56.241.10' (ECDSA) to the list of known hosts. 96 | ``` 97 | 98 | 6. Continue with the **oc commands** to complete the installation, approving CRS, etc. etc. 99 | 100 | #### Destroy 101 | You can easily destroy the bootstrap once you don't need it anymore. 102 | ```bash 103 | alex@:/../terraform $ ./destroy --target=module.bootstrap 104 | data.vsphere_datacenter.dc: Refreshing state... 105 | data.vsphere_host.esxi67: Refreshing state... 106 | Plan: 0 to add, 0 to change, 1 to destroy. 107 | . 108 | Warning: Resource targeting is in effect 109 | . 110 | Do you really want to destroy all resources? 111 | Terraform will destroy all your managed infrastructure, as shown above. 112 | There is no undo. Only 'yes' will be accepted to confirm. 113 | . 114 | Enter a value: yes 115 | . 116 | module.bootstrap.vsphere_virtual_machine.clone[0]: Destroying... [id=423147d7-b7ff-11d2-1762-a2e10dd37584] 117 | module.bootstrap.vsphere_virtual_machine.clone[0]: Destruction complete after 9s 118 | . 119 | Destroy complete! Resources: 1 destroyed. 120 | ``` 121 | Or you can destroy everything 122 | ```bash 123 | alex@:/../terraform $ ./destroy 124 | Plan: 0 to add, 0 to change, 4 to destroy. 125 | . 126 | Do you really want to destroy all resources? 127 | Terraform will destroy all your managed infrastructure, as shown above. 128 | There is no undo. Only 'yes' will be accepted to confirm. 129 | . 130 | Enter a value: yes 131 | module.master.vsphere_virtual_machine.clone[1]: Destroying... [id=4231f92a-08c9-6afe-7f0c-b17105e7991a] 132 | module.master.vsphere_virtual_machine.clone[0]: Destroying... [id=4231da99-3018-40dd-2532-157990aea86f] 133 | module.master.vsphere_virtual_machine.clone[2]: Destroying... [id=423188fc-571c-f243-367c-c6edaf8906b2] 134 | module.master.vsphere_virtual_machine.clone[2]: Destruction complete after 4s 135 | module.master.vsphere_virtual_machine.clone[1]: Destruction complete after 4s 136 | module.master.vsphere_virtual_machine.clone[0]: Destruction complete after 5s 137 | vsphere_folder.cluster: Destroying... [id=group-v185] 138 | vsphere_folder.cluster: Destruction complete after 0s 139 | . 140 | Destroy complete! Resources: 4 destroyed. 141 | ``` 142 | 143 | References 144 | - https://docs.openshift.com/container-platform/4.3/installing/installing_vsphere/installing-vsphere.html 145 | - https://docs.openshift.com/container-platform/4.3/installing/installing_bare_metal/installing-bare-metal.html 146 | - https://www.terraform.io/ 147 | - https://github.com/terraform-providers/terraform-provider-vsphere 148 | - https://coreos.com/ignition/docs/latest/network-configuration.html 149 | - https://coreos.com/ignition/docs/latest/examples.html 150 | -------------------------------------------------------------------------------- /deploy/auth/kubeadmin-password: -------------------------------------------------------------------------------- 1 | CHScu-aE5wM-vNdM4-nB9Mi -------------------------------------------------------------------------------- /deploy/auth/kubeconfig: -------------------------------------------------------------------------------- 1 | clusters: 2 | - cluster: 3 | certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURRRENDQWlpZ0F3SUJBZ0lJSUV6MHc5d1lRVTR3RFFZSktvWklodmNOQVFFTEJRQXdQakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1TZ3dKZ1lEVlFRREV4OXJkV0psTFdGd2FYTmxjblpsY2kxc2IyTmhiR2h2YzNRdApjMmxuYm1WeU1CNFhEVEl3TURVeE5ERTJOREl6TmxvWERUTXdNRFV4TWpFMk5ESXpObG93UGpFU01CQUdBMVVFCkN4TUpiM0JsYm5Ob2FXWjBNU2d3SmdZRFZRUURFeDlyZFdKbExXRndhWE5sY25abGNpMXNiMk5oYkdodmMzUXQKYzJsbmJtVnlNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXJDK2k0akJKTHl1dwpwYjA3MndVbXFZc2Rva09iNUdNTW14OFovWDNzc0szTWFaRThlbGdFZTQ3ZTVLanVjZWFhaWIwL21XaHRJZUZzCmVaNVpHZWxiSjdnSGNjUmpDeTF4WlVCaVVEa0toM29JUXpHOUIzcTMzdk8xY1J5dzVuTmpJSlRkNzQ3OU50VHIKZU1tQXYrSnJqa0ZwOVRhSkxNd2RvMDlOZ0NFc1gvVE1rdlpWejVscDN3VGw0dUE2QlpzUDMycUxWWHgvaUVMMQpWVjNKaVdjZnhHTHFnM1l2RHNlWUpTU0hhbmlvUDR1TThrNlM5WVk4bXNIL2taT0luUGR3NThUSGZGV1ZqREVrClNqd0dzb21OdTUzZkVUcE1qQ2RMUVBCeVVTV0ZDc3J2OHhiSWFIRTltT1JqS2gxMHk4VFlwSGFleDVESGFRYlEKSEJRb1Arem56d0lEQVFBQm8wSXdRREFPQmdOVkhROEJBZjhFQkFNQ0FxUXdEd1lEVlIwVEFRSC9CQVV3QXdFQgovekFkQmdOVkhRNEVGZ1FVbU91Uyt6eEEvUUQya2trWHRCMi9WdTdqZlhnd0RRWUpLb1pJaHZjTkFRRUxCUUFECmdnRUJBS2pqZ05uNExabk9heDBJR2Q2ZUtiVU4vMjd4Z2FIWUtiYXlNK3h3T1pTTkkzNTBPdXhIdlgxWGNOMzcKajhLaWN6K1hvUUZhWnZIeEdGemZGUkRXZUdGOWlqUjBoTkRJbUFvRzZVbUduM012cXY2T0NyTWY5UVhya3N1MApXcmdiaS9SRmVsbHZ6WGM3SDd1c2dnZFUwR1ZIT2ZNVTc4UUp6MEZZem43U3RQeThweTR4eWVxM0pqT0NOSnQ0Ck1CMHpZaHZ4emQ4VVYrMGpRK3NONGxhNDgyaG9yaWFVOExOQmYrYnNYMjZUVVF6Vm56eUhJa3J0Q1dBalN0VVAKZFRiblZ2ZXpDcjZuSGM2WUxpZHZZRWVRTzNhUGRyM0RaaHViK3JEeGRDdkt3Y2IzYWVhc1VmbW1WSEZ5cnN3aApLdkdPdHJlR2JVbzhXeXNiQVluSkRXZENnSGc9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURURENDQWpTZ0F3SUJBZ0lJTFo3WGRQa0JudUF3RFFZSktvWklodmNOQVFFTEJRQXdSREVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1TNHdMQVlEVlFRREV5VnJkV0psTFdGd2FYTmxjblpsY2kxelpYSjJhV05sTFc1bApkSGR2Y21zdGMybG5ibVZ5TUI0WERUSXdNRFV4TkRFMk5ESXpObG9YRFRNd01EVXhNakUyTkRJek5sb3dSREVTCk1CQUdBMVVFQ3hNSmIzQmxibk5vYVdaME1TNHdMQVlEVlFRREV5VnJkV0psTFdGd2FYTmxjblpsY2kxelpYSjIKYVdObExXNWxkSGR2Y21zdGMybG5ibVZ5TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQwpBUUVBdzhzdVpleHJSMmQvYU0wa1VWWGtXOG1TS2swREdpVDhCaDArZFJPMUJTZ0poS1F4c0hjeTdOdmhUWkhZCkpZeGwvbGZFTWtqUnhRYWRyNlhxSXV0aUY2NVNaZGdiL2Z0Z3Nsa3B2cE9kSXNNaW4yNGNjRXZaMVIyYlFTTUkKaGE3Q0ZXWVJyNHV0b3lZSS9mQ0ZmRzBZdHVCREpKQmc1Qkk3UjZHRDdzV3J1amZDQ1VxVUFuY0N4ZWl3blVOdwpqc0Y2eUM0MzVMeGQ3RW1tVUJuYUpCTVBUa3JzcERHY2NhUkl0UGtZbjNVd1BkRU1VWHY5VzhhU2JTdU1CSVpTCm1yY0JsWURHTDNqbWVneUE2Q3BhMkdqUjdXQkFHUFJWOWo5L1VNYkFETkhMMTlFM21vQUY4dERpODA0MzFGWDcKLzVBaEN6dTZpVjZ5UmpkMnBndGl4V29kOVFJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRApWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVU1oZHhFbmdQMEtueER1K29BaEJYNXd3ajBuUXdEUVlKCktvWklodmNOQVFFTEJRQURnZ0VCQUNEUzdXQkNNVGEvdUVYMC9SMHJDRDV6cS9aRTFmVkx1VmQ5RDFDM2JxeU8KRkl0eVQxL0s4NnNCVG9makdxNXNQNk5qSHdZRTlTSzFHTko4bVhodW52OVNXelU0V3BobVU3NkhmNlJNV2VRZAp0aFhTWVJXRHorM1FWWW5JclFUVW5DRWN6NnRYUnA2UUpiU0lIVW9oSERsV0tTQVhXV1llSjhaNnZ5VE5icTg3ClNSdHZzWGNtakM3UmpWSHdpTU90cWFJd2F3TzM4dURLa1IwR3owT2REMHNHaVNncWpFSkZndEpGWm1QTmx6TzEKVVprak5DL1BGTEJzNVI3YkRmNVpKRy9nZ0E1RUhpNzlkZUhRMklIWEtNZEtwRmVVVmVlbEwvOVRDTkxVY3RqdApnb2t2K3AwTlI2bWt6Y2hKSWxmMHZ2SndzTWQyVGJNV0hxVEVWSFc2UkQwPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tQkVHSU4gQ0VSVElGSUNBVEUtLS0tLQpNSUlETWpDQ0FocWdBd0lCQWdJSURJS1ZiZ0F0c2lZd0RRWUpLb1pJaHZjTkFRRUxCUUF3TnpFU01CQUdBMVVFCkN4TUpiM0JsYm5Ob2FXWjBNU0V3SHdZRFZRUURFeGhyZFdKbExXRndhWE5sY25abGNpMXNZaTF6YVdkdVpYSXcKSGhjTk1qQXdOVEUwTVRZME1qTTNXaGNOTXpBd05URXlNVFkwTWpNM1dqQTNNUkl3RUFZRFZRUUxFd2x2Y0dWdQpjMmhwWm5ReElUQWZCZ05WQkFNVEdHdDFZbVV0WVhCcGMyVnlkbVZ5TFd4aUxYTnBaMjVsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUpyK0szVDNvM1RkVkowRXpuc0JGd1ZNUWh1SlE1OHIKV0FmRHdwdlN2SUNIcU5RQ3doZERCMnJiRHlWT2hVS3Zhcit2VWc1UURUZDBGdVY4dlpxRDV1UVFaU1JUUW40UQo0di9DRHpUYWQzQ0c3ckxsZXYxQTAvZ3ZONFNrRTkvNG1HR0FpWW0xbU1KYUZMK2hCNXVJWEF1K0RUVWFBWnltCnQyWW1tVXp4VXErNWtoREdkK2hrcUl1OTlGWHJ6N0h3ZExRTmVCdGpDOGxaeENXbXpLTTdrOHVXWXB5bjRVT2EKL2hqR3lRQU9xWnVsanV4Z20xVlJuNmpvRlF4WUY4cWFCQlpQQzl0NUhzdktEQmJXa1VERlFtKytOc3AxaWIyWgp3V0NDN215dDl1cXZ3cCtJdy95MUl5VTFZVHhldVZ4VjZnNjl3blU0Rmx4ZDZWVUxhc01hL1cwQ0F3RUFBYU5DCk1FQXdEZ1lEVlIwUEFRSC9CQVFEQWdLa01BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOV28KV2NMM002QlY0dXZCMU1haTNneWErS2xQTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFBQ0JrVTBobUdJRzNZTQo5V1JzQVJGL2wrY2RwOXl5bTRIRC96cmU0WDNtL2V5UDd1aXBhNllTVEFmSnVmbDJjZFYyOWE2WHA0eGpNczZTCmFaNUlFWjBzMDBXTnhUYjVpRytYUHB4bVkxSVlUcGl6R3FYd2ZVZ3JUS1k0a3R6enYzSE1JYVRVa0tIcWxReU8KOEVUeVI5ZGY5Z0VrOWhqSzcrKzZYRndudjZ3aTRycmlmenpicUtyck1ENStrME9IWnpQbW0rcThEcTEvVit4ZwpWZ1FmN0VTMjJmSTMrcGNOQ2d0b0xGdGpBUjBieEpnRDNNSmR1b3hsYXFIbkgzZlRvd3hTUUxZS1FjOXpjcWRFCkg3WFcvNjNWU1NOT3pvQjJ1ZG1iU2NaT0x0dXUzMXZNZDVDcGk5bHNuS0p0MllIZ01TaUYwUzhmTjVqZ1F3cmwKMzhCSGpLUjkKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= 4 | server: https://api.ocp4.sni.com.mx:6443 5 | name: ocp4 6 | contexts: 7 | - context: 8 | cluster: ocp4 9 | user: admin 10 | name: admin 11 | current-context: admin 12 | preferences: {} 13 | users: 14 | - name: admin 15 | user: 16 | client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURaekNDQWsrZ0F3SUJBZ0lJRngvRzdxamRPVzR3RFFZSktvWklodmNOQVFFTEJRQXdOakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1TQXdIZ1lEVlFRREV4ZGhaRzFwYmkxcmRXSmxZMjl1Wm1sbkxYTnBaMjVsY2pBZQpGdzB5TURBMU1UUXhOalF5TXpaYUZ3MHpNREExTVRJeE5qUXlNelphTURBeEZ6QVZCZ05WQkFvVERuTjVjM1JsCmJUcHRZWE4wWlhKek1SVXdFd1lEVlFRREV3eHplWE4wWlcwNllXUnRhVzR3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUNkNjVTY3FiV1M4NkIxRmVEZm4rZlRSdDdXMUttaHdSZHhGcy83WEhjOQorNlhqaURrS3d1NzB3RDE1RWZFMGhFRzc3WllEZEorMVNjYUxyc3lqS0hPQWVJMXlMVTg3bEhyOG1vOHJhVytOCk1SUE93VVJIT0l0MFlhNlhGV1hPNEdpK3B6LzVuQ280aTRWakpjN2tlWlJrUS8raTdDUE5renNsZlRqTm9GTEkKMlRNVXlTT0FJcFJtak5WNkNJcSt3SFdnaDlZa1kyR0dSNzhXbUoxdUhQbmZqWVJGNWl5OTNmcGhidHNuUi9hQwpzZ2JpKzREcXQwYktvTm5CRHcvdXkvWWtRYVNJeEl6T08xd1IrYkp0N1poUFB2bnI2dFhuUEZCdnZQeXVRci9SCjNuRWZNT050eXFldHpLWUtLazNFODhmbmdDK3JqMlA2ZWQvczJvKzJrekJOQWdNQkFBR2pmekI5TUE0R0ExVWQKRHdFQi93UUVBd0lGb0RBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFRWUlLd1lCQlFVSEF3SXdEQVlEVlIwVApBUUgvQkFJd0FEQWRCZ05WSFE0RUZnUVViN1pUOUlVUnVDUkNnblU3YWFZcHJNNzNtMnN3SHdZRFZSMGpCQmd3CkZvQVViN1pUOUlVUnVDUkNnblU3YWFZcHJNNzNtMnN3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUk2SENNcTkKbjZCMWI2c0xNSGd3NzAyTldYdmVGR0t4a1lzVk1FeWtYSjN5TzFkYjErbEpTaWZwR1p1SHk5WDVNOVI0YW1LNgoyK1RXQlIrYTVZMWE2Umk4RFNpcVRIN1B0MkpIM0Evc0hWRGRQVHRlQ1pxcHNLOGtBaU5RWWpJU1F2OGE1eDVaCm1iMnlaQTV5U25jS0JGWW5rOGVFRXlZb013UjdNT3JwQSt2V21aR25jL2xNeW94REZib1pVdU9HVENyZUtlWm0KRnE2V1RMS2xCMi80NXVpOHh5ckFlOURwRmxHRHpqK282Z1VUTUpGMzVROUl6N3VEdmJMdlFxcHVvcXJZVTFZdQp3aTJoTVlpVWpCMStCLzBVcGJPT09NemxabjNuV2dqb21tWkc3OUc0M3JvZTZGYXFOSXVjMmNkVlBJbXpvL2xsCjNoWUtRZUFXcHFKTzZCST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= 17 | client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb2dJQkFBS0NBUUVBbmV1VW5LbTFrdk9nZFJYZzM1L24wMGJlMXRTcG9jRVhjUmJQKzF4M1BmdWw0NGc1CkNzTHU5TUE5ZVJIeE5JUkJ1KzJXQTNTZnRVbkdpNjdNb3loemdIaU5jaTFQTzVSNi9KcVBLMmx2alRFVHpzRkUKUnppTGRHR3VseFZsenVCb3ZxYy8rWndxT0l1Rll5WE81SG1VWkVQL291d2p6Wk03SlgwNHphQlN5Tmt6Rk1ragpnQ0tVWm96VmVnaUt2c0Ixb0lmV0pHTmhoa2UvRnBpZGJoejUzNDJFUmVZc3ZkMzZZVzdiSjBmMmdySUc0dnVBCjZyZEd5cURad1E4UDdzdjJKRUdraU1TTXpqdGNFZm15YmUyWVR6NzU2K3JWNXp4UWI3ejhya0svMGQ1eEh6RGoKYmNxbnJjeW1DaXBOeFBQSDU0QXZxNDlqK25uZjdOcVB0cE13VFFJREFRQUJBb0lCQUIxVzkzVmowbWhLK3pURgpxOUhnVTZIZE9PbVErVE1rUE1hMkJrNGREOHBrUnAzUFlVbXVqZGJMSUtDdHhMS05IK2xhYXRTWmhsS1lNbHNoClRzUEJjRFFlTWFsVTRiT2dFbmw3L2IyRENVYWlKNXZ5TEUxUEpCbXJURlNsbFJTcWZNdHZKMk9vcis5eGZJYS8KTDRFalZYNkxZNk5SWEJXRmxTZVI3RXZIelptaTlpMEwvT2lMS2lqNFM5WSt6VU9qLzlVWmw4eElJeWxxQlJqdgpOd3hoczZnVUl0eFlNVnhYRDJyOTA0dytmd0xyYkxaNmxqU3Z6NEU4eDN6VkdJcFI2anErTFpyS3c1MDVmcGtECm5jeUR2VytqaWhNZjl3VmFjKzlwVnFTdGZZQ3ZmNVVYR2FvOXdnemlySWxyOXJhYkxxSlk4QUtPRHYwMVlOMDcKSjNNbFFjRUNnWUVBenFGczJGRGhHQUtUTEFUU1R1N3JtUExVMkVBVnFwYUVVdlBBYmZsRW5sK1B6bHNaLytBTwpqTFhVZThmKzNaVjlPamZJMzArVXZra0luNWtNOUhIOUlqdWlTRlB4OGlmQUx3MzIyTEJMT2V6RkYycWwrMEtQCjkrMk9LRnZKMzU1SVd5NTVqcTdWTEwvUWdWOG11QVVIS0lUY1VQNU9kNVdsS29zMk5GamxSM3NDZ1lFQXc2YkkKdmMweG5rZXFnY3NiUDNYOGUvREg2ZXUwcWg3ckpocFlCSDFvVmVsMDV3VjZKRngwc0pJWU4zY2dmaWN2RnFMNApnUDJTelhJeXlaakdIWVNwWkV0WTRJOGpYNXpKVHZCMEV1MEl3dVdtd2hJTHlneEExc21PazNCVEp6aGI0Z25PCk8zemVXTHJBdlNPYVNnRjdlM0ZJVWRIMk5kUVdXTkVTN3hTLytOY0NnWUJwcW04bzU2TlVUZXpJOCtaQWF6bWIKeTRKQTIvUzRlYVJOYjAwV2NBUkJJUWsvTTZOR0VHUWdYTk8rUVBYNmhubTFqZFZ4TFJjWmYrdWpBZmQ0QWRxNgo2TmdDT0x0ZlpDaUpaNUMyR3VFOVMvRTJHbG1xOWN6OXNrT2lLOTQrTm9rMTU3eE4xT1k2cmhTSU5BcHV6Rk51Cnl4S0hjS1Riem5EYUlycWpabHBKaFFLQmdIYTA4K2tNM1lVT01HcXRvNnVlVjRHUGRvdjhBZEtTRGRSQmVaUEgKencwSkoyK2Z5QnJWcWNzVDZQT0dLTVhCMm5VZWppNGRLL3ZReFBYQjVybzBLbDU2eE1NQ3RkQ21IUkh6SEVuNwpYU3g3RC9HQjUzdEFjZnFmc2d6TXd2YngzL1hHUTBXZW5uMHhXZzJuMDA0QmxqMHdYMU9kVTRnWFZUT3dMbysyCmhEOWpBb0dBRXp2Q1BYZUsrTFRRYStLaGxneHViZTc0RVk0aXJGWkw5MkZGYVFUYWtmeUhrazJnbFprL1QybmkKVXlwbExCaGc0NGN2MGIxRE9BVFd3YytsRHV5Mk9SSmJJSzFFMG0xT0krWWVyUHF1NXp4QUdEeFpwWVhyZzNSOApPY0h0czJSdklvZGsvY3B5YkZyN2l4UDhEUm9yTzBzK1dPdlU2eVhPTkFoV0NGK0tnZUk9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== 18 | -------------------------------------------------------------------------------- /deploy/master.ign: -------------------------------------------------------------------------------- 1 | {"ignition":{"config":{"append":[{"source":"https://api-int.ocp4.sni.com.mx:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUkFmbFBPaC9uZ1V3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEl3TURVeE5ERTJOREl6TWxvWApEVE13TURVeE1qRTJOREl6TWxvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFyeDZ1TzlNSXhGcEgKN1dLV1Rja3FKeGoxWk1HeFJYUzVPZGtselR3dTJiYzIzd3JHZ2NQTDhLbmtHQkk2dDIyTWRkcjhKa0o0aTVhVQpYTUZNWUMvcHZ2WnJOeVZJbHh6VllKT0ptOERvaHc1TzJsZ010bzJDYXdmWTFjWk1ZV050OExHV1dGVThtV1JwCmpIaG8zN0Q3NElJYUZ6eC9SMkRKZ0d3azNpQmd3RVJRdjVwR1ZYZ0dEN1ZBTFAreVk1c3l2aU9LanAyeXRadkQKUGVrRjFLUjJ2SXBMVFFzcXVZZmFEK0NqblZOdDhuYW04YkRJdXR4Z01VT2QydzU4RXVkbXJwNmpZbXo5QlUxbApxa0JtNytUV1MvUWxvWUJ2WGs0d1VQbldseWEyUldRTTlaRjUrTng0dW5CV3h0amM2TVJmYUljbXFvUUJ3eHBuCng5S1RncGwvZlFJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVUZDTXlGMWxJL3hCWURkZkNJa3JXRGJSbzkySXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUh1OE5sd0I0K0ZxRmhrREtrM1RmQ3lrcExtVFA4Y0huVEkwZ0xUWTQ4L05sdWUvOCt5VDZUT1lKc2RhCmw2T003V2N3QW04eDdoY2hYSmhrQXRMcmxCdHF4OG9WV1JQdDhPRGZjYWQrZndmM2g1VzIwY25vMGhFeWUzQmgKUjJybk42UUNhTXErSUJZMjZMTXNzMmFYcVVsQ28wOXQ5aTVLUVJiVzJjZWZMN1cwOXpBSG5FQXo2eU5oTmhXMgpSOXFYL3RMQ3dSaDRiYnlPY3piWjJ4R2dHSWRHWTNwbWdPRi9vcldJMEYzZVVWYkJxVjFnWm0wM1ZEbnNWbXNiCmp5cW9iektRMmw4SXRNUnVDTnNQVFdMS3Zpb0ZMamVhSzZsVEUvWmR6L25DTWkvRlUwVnh4WnpnWVpRdUR4YjEKZW1PN2UxZk1TM1g0K2FEQkRNWk9iL2RzTlFVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} -------------------------------------------------------------------------------- /deploy/metadata.json: -------------------------------------------------------------------------------- 1 | {"clusterName":"ocp4","clusterID":"6e6ffa1d-8a30-44c7-b28a-1a92e3946510","infraID":"ocp4-xhklp","vsphere":{"vCenter":"vcenter.sni.com.mx","username":"administrator@sni.com.mx","password":"Password123!"}} -------------------------------------------------------------------------------- /deploy/worker.ign: -------------------------------------------------------------------------------- 1 | {"ignition":{"config":{"append":[{"source":"https://api-int.ocp4.sni.com.mx:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUkFmbFBPaC9uZ1V3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEl3TURVeE5ERTJOREl6TWxvWApEVE13TURVeE1qRTJOREl6TWxvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFyeDZ1TzlNSXhGcEgKN1dLV1Rja3FKeGoxWk1HeFJYUzVPZGtselR3dTJiYzIzd3JHZ2NQTDhLbmtHQkk2dDIyTWRkcjhKa0o0aTVhVQpYTUZNWUMvcHZ2WnJOeVZJbHh6VllKT0ptOERvaHc1TzJsZ010bzJDYXdmWTFjWk1ZV050OExHV1dGVThtV1JwCmpIaG8zN0Q3NElJYUZ6eC9SMkRKZ0d3azNpQmd3RVJRdjVwR1ZYZ0dEN1ZBTFAreVk1c3l2aU9LanAyeXRadkQKUGVrRjFLUjJ2SXBMVFFzcXVZZmFEK0NqblZOdDhuYW04YkRJdXR4Z01VT2QydzU4RXVkbXJwNmpZbXo5QlUxbApxa0JtNytUV1MvUWxvWUJ2WGs0d1VQbldseWEyUldRTTlaRjUrTng0dW5CV3h0amM2TVJmYUljbXFvUUJ3eHBuCng5S1RncGwvZlFJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVUZDTXlGMWxJL3hCWURkZkNJa3JXRGJSbzkySXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUh1OE5sd0I0K0ZxRmhrREtrM1RmQ3lrcExtVFA4Y0huVEkwZ0xUWTQ4L05sdWUvOCt5VDZUT1lKc2RhCmw2T003V2N3QW04eDdoY2hYSmhrQXRMcmxCdHF4OG9WV1JQdDhPRGZjYWQrZndmM2g1VzIwY25vMGhFeWUzQmgKUjJybk42UUNhTXErSUJZMjZMTXNzMmFYcVVsQ28wOXQ5aTVLUVJiVzJjZWZMN1cwOXpBSG5FQXo2eU5oTmhXMgpSOXFYL3RMQ3dSaDRiYnlPY3piWjJ4R2dHSWRHWTNwbWdPRi9vcldJMEYzZVVWYkJxVjFnWm0wM1ZEbnNWbXNiCmp5cW9iektRMmw4SXRNUnVDTnNQVFdMS3Zpb0ZMamVhSzZsVEUvWmR6L25DTWkvRlUwVnh4WnpnWVpRdUR4YjEKZW1PN2UxZk1TM1g0K2FEQkRNWk9iL2RzTlFVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} -------------------------------------------------------------------------------- /dns/56.10.in-addr.arpa.db: -------------------------------------------------------------------------------- 1 | $ORIGIN 56.10.in-addr.arpa. 2 | $TTL 1; 1 seconds (for testing only) 3 | @ IN SOA ns1.sni.com.mx. hostmaster.sni.com.mx. ( 4 | 2011112904 ; serial 5 | 60 ; refresh (1 minute) 6 | 15 ; retry (15 seconds) 7 | 1800 ; expire (30 minutes) 8 | 10 ; minimum (10 seconds) 9 | ) 10 | $ORIGIN 56.10.in-addr.arpa. 11 | IN NS ns1.sni.com.mx. 12 | 1.240 IN PTR dns01.sni.com.mx. 13 | 14 | 5.240 IN PTR esxi67.sni.com.mx. 15 | 6.240 IN PTR vcenter.sni.com.mx. 16 | 17 | 5.241 IN PTR esxi67.ocp4.sni.com.mx. 18 | 19 | 10.241 IN PTR bastion.ocp4.sni.com.mx. 20 | 21 | 11.241 IN PTR lb.ocp4.sni.com.mx. 22 | 11.241 IN PTR api.ocp4.sni.com.mx. 23 | 11.241 IN PTR api-int.ocp4.sni.com.mx. 24 | 25 | 12.241 IN PTR bootstrap.ocp4.sni.com.mx. 26 | 27 | 13.241 IN PTR master01.ocp4.sni.com.mx. 28 | 14.241 IN PTR master02.ocp4.sni.com.mx. 29 | 15.241 IN PTR master03.ocp4.sni.com.mx. 30 | 31 | 13.241 IN PTR etcd-0.ocp4.sni.com.mx. 32 | 14.241 IN PTR etcd-1.ocp4.sni.com.mx. 33 | 15.241 IN PTR etcd-2.ocp4.sni.com.mx. 34 | 35 | 16.241 IN PTR worker01.ocp4.sni.com.mx. 36 | 17.241 IN PTR worker02.ocp4.sni.com.mx. 37 | 38 | 18.241 IN PTR infra01.ocp4.sni.com.mx. 39 | 19.241 IN PTR infra02.ocp4.sni.com.mx. 40 | 41 | 20.241 IN PTR logging01.ocp4.sni.com.mx. 42 | 21.241 IN PTR logging02.ocp4.sni.com.mx. 43 | -------------------------------------------------------------------------------- /dns/ocp4.sni.com.mx.db: -------------------------------------------------------------------------------- 1 | $ORIGIN . 2 | $TTL 1; 1 seconds (for testing only) 3 | ocp4.sni.com.mx IN SOA ns1.ocp4.sni.com.mx. hostmaster.ocp4.sni.com.mx. ( 4 | 2011112904 ; serial 5 | 60 ; refresh (1 minute) 6 | 15 ; retry (15 seconds) 7 | 1800 ; expire (30 minutes) 8 | 10 ; minimum (10 seconds) 9 | ) 10 | NS ns1.ocp4.sni.com.mx. 11 | ;MX 10 mail.ocp4sni.com.mx. 12 | $ORIGIN ocp4.sni.com.mx. 13 | ns1 A 10.56.241.1 14 | 15 | esxi67 A 10.56.241.5 16 | 17 | bastion A 10.56.241.10 18 | 19 | lb A 10.56.241.11 20 | api A 10.56.241.11 21 | api-int A 10.56.241.11 22 | *.apps A 10.56.241.11 23 | 24 | bootstrap A 10.56.241.12 25 | 26 | master01 A 10.56.241.13 27 | master02 A 10.56.241.14 28 | master03 A 10.56.241.15 29 | 30 | etcd-0 A 10.56.241.13 31 | etcd-1 A 10.56.241.14 32 | etcd-2 A 10.56.241.15 33 | 34 | _etcd-server-ssl._tcp 86400 IN SRV 0 10 2380 etcd-0 35 | _etcd-server-ssl._tcp 86400 IN SRV 0 10 2380 etcd-1 36 | _etcd-server-ssl._tcp 86400 IN SRV 0 10 2380 etcd-2 37 | 38 | worker01 A 10.56.241.16 39 | worker02 A 10.56.241.17 40 | 41 | infra01 A 10.56.241.18 42 | infra02 A 10.56.241.19 43 | 44 | logging01 A 10.56.241.20 45 | logging02 A 10.56.241.21 46 | -------------------------------------------------------------------------------- /go.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -xe 3 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | 5 | ## Create the ignitions files 6 | rm -fr deploy 7 | mkdir deploy 8 | 9 | cp install-config.yaml deploy/ 10 | 11 | openshift-install create manifests --dir=deploy 12 | 13 | sed -i 's/true/false/g' deploy/manifests/cluster-scheduler-02-config.yml 14 | 15 | openshift-install create ignition-configs --dir=deploy 16 | 17 | scp deploy/*.ign root@bastion.ocp4.sni.com.mx:/var/www/html/ 18 | ssh root@bastion.ocp4.sni.com.mx 'chmod 644 /var/www/html/*.ign' 19 | #cp -f deploy/*.ign /var/www/html/ 20 | 21 | ## Terraforn deployment. 22 | cd terraform 23 | 24 | ./plan -target=module.template 25 | ./apply 26 | 27 | ./plan -target=module.bootstrap 28 | ./apply 29 | 30 | echo "Waiting 2 minutes to bootstrap node finish the boot process" 31 | sleep 120 32 | ./plan -target=module.master 33 | ./apply 34 | 35 | export KUBECONFIG="${DIR}/deploy/auth/kubeconfig" 36 | openshift-install wait-for bootstrap-complete --dir ${DIR}/deploy/ --log-level debug 37 | 38 | ./plan 39 | ./apply 40 | 41 | ./destroy -target=module.bootstrap 42 | 43 | cd .. 44 | -------------------------------------------------------------------------------- /haproxy/haproxy.conf: -------------------------------------------------------------------------------- 1 | global 2 | log 127.0.0.1 local2 3 | chroot /var/lib/haproxy 4 | pidfile /var/run/haproxy.pid 5 | maxconn 4000 6 | user haproxy 7 | group haproxy 8 | daemon 9 | stats socket /var/lib/haproxy/stats 10 | 11 | defaults 12 | mode tcp 13 | log global 14 | option tcplog 15 | option dontlognull 16 | option redispatch 17 | retries 3 18 | timeout queue 1m 19 | timeout connect 10s 20 | timeout client 1m 21 | timeout server 1m 22 | timeout check 10s 23 | maxconn 3000 24 | # 6443 points to control plan 25 | frontend ocp4-api *:6443 26 | default_backend master-api 27 | backend master-api 28 | balance source 29 | server bootstrap bootstrap.ocp4.sni.com.mx:6443 check 30 | server master01 master01.ocp4.sni.com.mx:6443 check 31 | server master02 master02.ocp4.sni.com.mx:6443 check 32 | server master03 master03.ocp4.sni.com.mx:6443 check 33 | 34 | # 22623 points to control plane 35 | frontend ocp4-mapi *:22623 36 | default_backend master-mapi 37 | backend master-mapi 38 | balance source 39 | server bootstrap bootstrap.ocp4.sni.com.mx:22623 check 40 | server master01 master01.ocp4.sni.com.mx:22623 check 41 | server master02 master02.ocp4.sni.com.mx:22623 check 42 | server master03 master03.ocp4.sni.com.mx:22623 check 43 | 44 | # 80 points to worker nodes 45 | frontend ocp4-http *:80 46 | default_backend ingress-http 47 | backend ingress-http 48 | balance source 49 | server worker01 worker01.ocp4.sni.com.mx:80 check 50 | server worker02 worker02.ocp4.sni.com.mx:80 check 51 | server infra01 infra01.ocp4.sni.com.mx:80 check 52 | server infra02 infra02.ocp4.sni.com.mx:80 check 53 | # server logging01 logging01.ocp4.sni.com.mx:80 check 54 | # server logging02 logging02.ocp4.sni.com.mx:80 check 55 | 56 | # 443 points to worker nodes 57 | frontend ocp4-https *:443 58 | default_backend infra-https 59 | backend infra-https 60 | balance source 61 | server worker01 worker01.ocp4.sni.com.mx:443 check 62 | server worker02 worker02.ocp4.sni.com.mx:443 check 63 | server infra01 infra01.ocp4.sni.com.mx:443 check 64 | server infra02 infra02.ocp4.sni.com.mx:443 check 65 | # server logging01 logging01.ocp4.sni.com.mx:443 check 66 | # server logging02 logging02.ocp4.sni.com.mx:443 check 67 | -------------------------------------------------------------------------------- /images/bootstrap.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/bootstrap.jpg -------------------------------------------------------------------------------- /images/final_ip.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/final_ip.jpg -------------------------------------------------------------------------------- /images/manual.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/manual.jpg -------------------------------------------------------------------------------- /images/master.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/master.jpg -------------------------------------------------------------------------------- /images/ova_template.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/ova_template.jpg -------------------------------------------------------------------------------- /images/temp_ip.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/images/temp_ip.jpg -------------------------------------------------------------------------------- /terraform/apply: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | 5 | terraform init 6 | terraform apply -auto-approve "$DIR/outputs/tf.output" 7 | -------------------------------------------------------------------------------- /terraform/destroy: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | 5 | terraform destroy -var="full_path=${DIR}" -var-file=${DIR}/vars/common.tfvars $1 6 | -------------------------------------------------------------------------------- /terraform/main.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | # Deploy ocp4 infrastructure to vmware 11 | 12 | variable vsphere_user {} 13 | variable vsphere_password {} 14 | variable vsphere_server {} 15 | variable ocp_cluster_name {} 16 | variable full_path {} 17 | variable datacenter {} 18 | variable datastore {} 19 | variable network {} 20 | variable resource_pool {} 21 | variable host {} 22 | variable template {} 23 | variable node_network {} 24 | variable url_ignition {} 25 | variable node_configs {} 26 | 27 | provider "vsphere" { 28 | user = var.vsphere_user 29 | password = var.vsphere_password 30 | vsphere_server = var.vsphere_server 31 | 32 | allow_unverified_ssl = true 33 | } 34 | 35 | data "vsphere_datacenter" "dc" { 36 | name = var.datacenter 37 | } 38 | 39 | data "vsphere_datastore" "datastore" { 40 | name = var.datastore 41 | datacenter_id = data.vsphere_datacenter.dc.id 42 | } 43 | 44 | resource "vsphere_folder" "cluster" { 45 | path = var.ocp_cluster_name 46 | type = "vm" 47 | datacenter_id = data.vsphere_datacenter.dc.id 48 | } 49 | 50 | data "vsphere_network" "network" { 51 | name = var.network 52 | datacenter_id = data.vsphere_datacenter.dc.id 53 | } 54 | 55 | data "vsphere_resource_pool" "sni" { 56 | name = var.resource_pool 57 | datacenter_id = data.vsphere_datacenter.dc.id 58 | } 59 | 60 | data "vsphere_host" "esxi67" { 61 | name = var.host 62 | datacenter_id = data.vsphere_datacenter.dc.id 63 | } 64 | 65 | module "template" { 66 | source = "./modules/create_template" 67 | name = var.template.name 68 | local_ovf = "${path.module}/${var.template.ovf_name}" 69 | resource_pool_id = data.vsphere_resource_pool.sni.id 70 | host = var.host 71 | datastore = var.datastore 72 | datacenter = var.datacenter 73 | folder = vsphere_folder.cluster.path 74 | } 75 | 76 | module "create_ignitions_bootstrap" { 77 | source = "./modules/create_ignitions" 78 | node_network = var.node_network 79 | cluster_name = var.ocp_cluster_name 80 | node_config = var.node_configs.bootstrap 81 | root_path = var.full_path 82 | url_ignition = var.url_ignition 83 | type = "bootstrap" 84 | } 85 | 86 | module "create_ignitions_master" { 87 | source = "./modules/create_ignitions" 88 | node_network = var.node_network 89 | cluster_name = var.ocp_cluster_name 90 | node_config = var.node_configs.master 91 | root_path = var.full_path 92 | url_ignition = var.url_ignition 93 | type = "master" 94 | } 95 | module "create_ignitions_worker" { 96 | source = "./modules/create_ignitions" 97 | node_network = var.node_network 98 | cluster_name = var.ocp_cluster_name 99 | node_config = var.node_configs.worker 100 | root_path = var.full_path 101 | url_ignition = var.url_ignition 102 | type = "worker" 103 | } 104 | module "create_ignitions_infra" { 105 | source = "./modules/create_ignitions" 106 | node_network = var.node_network 107 | cluster_name = var.ocp_cluster_name 108 | node_config = var.node_configs.infra 109 | root_path = var.full_path 110 | url_ignition = var.url_ignition 111 | type = "worker" 112 | } 113 | module "create_ignitions_logging" { 114 | source = "./modules/create_ignitions" 115 | node_network = var.node_network 116 | cluster_name = var.ocp_cluster_name 117 | node_config = var.node_configs.logging 118 | root_path = var.full_path 119 | url_ignition = var.url_ignition 120 | type = "worker" 121 | } 122 | 123 | module "bootstrap" { 124 | source = "./modules/clone_from_template" 125 | folder = vsphere_folder.cluster.path 126 | resource_pool_id = data.vsphere_resource_pool.sni.id 127 | host_system_id = data.vsphere_host.esxi67.id 128 | datastore_id = data.vsphere_datastore.datastore.id 129 | network_id = data.vsphere_network.network.id 130 | guest_id = module.template.guest_id 131 | adapter_type = module.template.adapter_type[0].adapter_type 132 | template_uuid = module.template.template_id 133 | vm_data = module.create_ignitions_bootstrap.data 134 | machine_config = var.node_configs.bootstrap 135 | } 136 | 137 | module "master" { 138 | source = "./modules/clone_from_template" 139 | folder = vsphere_folder.cluster.path 140 | resource_pool_id = data.vsphere_resource_pool.sni.id 141 | host_system_id = data.vsphere_host.esxi67.id 142 | datastore_id = data.vsphere_datastore.datastore.id 143 | network_id = data.vsphere_network.network.id 144 | guest_id = module.template.guest_id 145 | adapter_type = module.template.adapter_type[0].adapter_type 146 | template_uuid = module.template.template_id 147 | vm_data = module.create_ignitions_master.data 148 | machine_config = var.node_configs.master 149 | } 150 | 151 | module "worker" { 152 | source = "./modules/clone_from_template" 153 | folder = vsphere_folder.cluster.path 154 | resource_pool_id = data.vsphere_resource_pool.sni.id 155 | host_system_id = data.vsphere_host.esxi67.id 156 | datastore_id = data.vsphere_datastore.datastore.id 157 | network_id = data.vsphere_network.network.id 158 | guest_id = module.template.guest_id 159 | adapter_type = module.template.adapter_type[0].adapter_type 160 | template_uuid = module.template.template_id 161 | vm_data = module.create_ignitions_worker.data 162 | machine_config = var.node_configs.worker 163 | } 164 | 165 | module "infra" { 166 | source = "./modules/clone_from_template" 167 | folder = vsphere_folder.cluster.path 168 | resource_pool_id = data.vsphere_resource_pool.sni.id 169 | host_system_id = data.vsphere_host.esxi67.id 170 | datastore_id = data.vsphere_datastore.datastore.id 171 | network_id = data.vsphere_network.network.id 172 | guest_id = module.template.guest_id 173 | adapter_type = module.template.adapter_type[0].adapter_type 174 | template_uuid = module.template.template_id 175 | vm_data = module.create_ignitions_infra.data 176 | machine_config = var.node_configs.infra 177 | } 178 | 179 | module "logging" { 180 | source = "./modules/clone_from_template" 181 | folder = vsphere_folder.cluster.path 182 | resource_pool_id = data.vsphere_resource_pool.sni.id 183 | host_system_id = data.vsphere_host.esxi67.id 184 | datastore_id = data.vsphere_datastore.datastore.id 185 | network_id = data.vsphere_network.network.id 186 | guest_id = module.template.guest_id 187 | adapter_type = module.template.adapter_type[0].adapter_type 188 | template_uuid = module.template.template_id 189 | vm_data = module.create_ignitions_logging.data 190 | machine_config = var.node_configs.logging 191 | } 192 | 193 | output machine { 194 | value = module.template.machine 195 | } 196 | -------------------------------------------------------------------------------- /terraform/modules/clone_from_template/main.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | 11 | resource "vsphere_virtual_machine" "clone" { 12 | count = length(var.vm_data) 13 | name = var.machine_config.hostname[count.index] 14 | folder = var.folder 15 | resource_pool_id = var.resource_pool_id 16 | host_system_id = var.host_system_id 17 | datastore_id = var.datastore_id 18 | enable_disk_uuid = true 19 | wait_for_guest_net_timeout = -1 20 | wait_for_guest_net_routable = false 21 | 22 | num_cpus = var.machine_config.cpu 23 | memory = var.machine_config.memory 24 | guest_id = var.guest_id 25 | 26 | network_interface { 27 | network_id = var.network_id 28 | adapter_type = var.adapter_type 29 | } 30 | 31 | disk { 32 | # eagerly_scrub = false 33 | thin_provisioned = true 34 | label = "disk0" 35 | size = var.machine_config.disk 36 | } 37 | 38 | clone { 39 | template_uuid = var.template_uuid 40 | linked_clone = false 41 | } 42 | 43 | vapp { 44 | properties = { 45 | "guestinfo.ignition.config.data" = base64encode(var.vm_data[count.index]) 46 | "guestinfo.ignition.config.data.encoding" = "base64" 47 | } 48 | } 49 | } 50 | -------------------------------------------------------------------------------- /terraform/modules/clone_from_template/outputs.tf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/terraform/modules/clone_from_template/outputs.tf -------------------------------------------------------------------------------- /terraform/modules/clone_from_template/vars.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | variable "clone_name" { 11 | default = "" 12 | } 13 | 14 | variable "folder" { 15 | default = "" 16 | } 17 | 18 | variable "resource_pool_id" { 19 | default = "" 20 | } 21 | 22 | variable "host_system_id" { 23 | default = "" 24 | } 25 | 26 | variable "datastore_id" { 27 | default = "" 28 | } 29 | 30 | variable "guest_id" { 31 | default = "" 32 | } 33 | 34 | variable "network_id" { 35 | default = "" 36 | } 37 | 38 | variable "adapter_type" { 39 | default = "" 40 | } 41 | 42 | variable "template_uuid" { 43 | default = "" 44 | } 45 | 46 | variable "ignition_config_data" { 47 | default = "" 48 | } 49 | 50 | variable "vm_data" {} 51 | 52 | variable machine_config { 53 | default = [] 54 | } 55 | -------------------------------------------------------------------------------- /terraform/modules/create_ignitions/main.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | 11 | data "ignition_file" "hostname" { 12 | count = length(var.node_config.hostname) 13 | 14 | filesystem = "root" 15 | path = "/etc/hostname" 16 | mode = "420" 17 | 18 | content { 19 | content = "${element(var.node_config.hostname, count.index)}.${var.cluster_name}.${var.node_network.ocp_domain}" 20 | } 21 | } 22 | 23 | data "template_file" "ifcfg-ens192" { 24 | count = length(var.node_config.ip) 25 | template = file("${path.module}/templates/ifcfg-ens192.tpl") 26 | 27 | vars = { 28 | ip_address = element(var.node_config.ip, count.index) 29 | net_mask = var.node_network.prefix 30 | gateway = var.node_network.gateway 31 | domain = "${var.cluster_name}.${var.node_network.ocp_domain}" 32 | dns = var.node_network.dns 33 | } 34 | } 35 | 36 | data "ignition_file" "static_ip" { 37 | count = length(var.node_config.ip) 38 | 39 | filesystem = "root" 40 | path = "/etc/sysconfig/network-scripts/ifcfg-ens192" 41 | mode = "420" 42 | 43 | source { 44 | source = "data:text/plain;charset=utf-8;base64,${base64encode(data.template_file.ifcfg-ens192[count.index].rendered)}" 45 | } 46 | } 47 | 48 | data "ignition_systemd_unit" "restart" { 49 | name = "restart.service" 50 | content = file("${path.module}/templates/systemd_restart") 51 | } 52 | 53 | 54 | data "ignition_config" "ign" { 55 | count = length(var.node_config.ip) 56 | 57 | append { 58 | source = "${var.url_ignition}/${var.type}.ign" 59 | } 60 | 61 | systemd = [ 62 | data.ignition_systemd_unit.restart.rendered 63 | ] 64 | 65 | files = [ 66 | data.ignition_file.hostname[count.index].rendered, 67 | data.ignition_file.static_ip[count.index].rendered 68 | ] 69 | } 70 | 71 | 72 | output data { 73 | value = data.ignition_config.ign.*.rendered 74 | } 75 | -------------------------------------------------------------------------------- /terraform/modules/create_ignitions/outputs.tf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/terraform/modules/create_ignitions/outputs.tf -------------------------------------------------------------------------------- /terraform/modules/create_ignitions/templates/ifcfg-ens192.tpl: -------------------------------------------------------------------------------- 1 | TYPE=Ethernet 2 | BOOTPROTO=none 3 | NAME=ens192 4 | DEVICE=ens192 5 | ONBOOT=yes 6 | IPADDR=${ip_address} 7 | PREFIX=${net_mask} 8 | GATEWAY=${gateway} 9 | DOMAIN=${domain} 10 | DNS1=${dns} 11 | -------------------------------------------------------------------------------- /terraform/modules/create_ignitions/templates/systemd_restart: -------------------------------------------------------------------------------- 1 | [Unit] 2 | ConditionFirstBoot=yes 3 | Before=NetworkManager.service 4 | [Service] 5 | Type=idle 6 | ExecStart=/sbin/reboot 7 | [Install] 8 | #WantedBy=multi-user.target 9 | WantedBy=network.target 10 | -------------------------------------------------------------------------------- /terraform/modules/create_ignitions/vars.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | variable node_network { 11 | default = {} 12 | } 13 | 14 | variable cluster_name { 15 | default = "" 16 | } 17 | 18 | variable node_config { 19 | default = {} 20 | } 21 | 22 | variable root_path { 23 | default = "" 24 | } 25 | 26 | variable type { 27 | default = "bootstrap" 28 | } 29 | 30 | variable url_ignition { 31 | default = "" 32 | } 33 | -------------------------------------------------------------------------------- /terraform/modules/create_template/main.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | data "vsphere_datacenter" "dc" { 11 | name = var.datacenter 12 | } 13 | 14 | data "vsphere_datastore" "datastore" { 15 | name = var.datastore 16 | datacenter_id = data.vsphere_datacenter.dc.id 17 | } 18 | 19 | data "vsphere_host" "esxi67" { 20 | name = var.host 21 | datacenter_id = data.vsphere_datacenter.dc.id 22 | } 23 | 24 | resource "vsphere_virtual_machine" "template" { 25 | name = var.name 26 | datacenter_id = data.vsphere_datacenter.dc.id 27 | resource_pool_id = var.resource_pool_id 28 | datastore_id = data.vsphere_datastore.datastore.id 29 | host_system_id = data.vsphere_host.esxi67.id 30 | folder = var.folder 31 | enable_disk_uuid = true 32 | wait_for_guest_net_timeout = 0 33 | wait_for_guest_net_routable = false 34 | boot_delay = 10000 35 | 36 | ovf_deploy { 37 | local_ovf_path = var.local_ovf 38 | disk_provisioning = "thin" 39 | } 40 | } 41 | 42 | resource "vsphere_virtual_machine_snapshot" "template" { 43 | virtual_machine_uuid = vsphere_virtual_machine.template.uuid 44 | snapshot_name = "Snapshot for clone" 45 | description = "rhcos snapshot for clone ocp vm" 46 | memory = "false" 47 | quiesce = "false" 48 | remove_children = "true" 49 | consolidate = "true" 50 | } 51 | -------------------------------------------------------------------------------- /terraform/modules/create_template/outputs.tf: -------------------------------------------------------------------------------- 1 | 2 | output template_id { 3 | value = vsphere_virtual_machine.template.id 4 | } 5 | 6 | output guest_id { 7 | value = vsphere_virtual_machine.template.guest_id 8 | } 9 | 10 | output adapter_type { 11 | value = vsphere_virtual_machine.template.network_interface 12 | } 13 | 14 | output machine { 15 | value = vsphere_virtual_machine.template.id 16 | } 17 | 18 | output template { 19 | value = vsphere_virtual_machine_snapshot.template.id 20 | } 21 | -------------------------------------------------------------------------------- /terraform/modules/create_template/vars.tf: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | variable "name" { 11 | default = "" 12 | } 13 | 14 | variable "folder" { 15 | default = "" 16 | } 17 | 18 | variable "resource_pool_id" { 19 | default = "" 20 | } 21 | 22 | variable "host" { 23 | default = "" 24 | } 25 | 26 | variable "datastore" { 27 | default = "" 28 | } 29 | 30 | variable "datacenter" { 31 | default = "" 32 | } 33 | 34 | variable "local_ovf" {} 35 | -------------------------------------------------------------------------------- /terraform/outputs/tf.output: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexgdmx/ocp4-vmware-terraform/798ba2cd15c565f4c7d393ded02c80893cbcf23c/terraform/outputs/tf.output -------------------------------------------------------------------------------- /terraform/plan: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 3 | 4 | terraform fmt . 5 | terraform init 6 | terraform plan -var="full_path=${DIR}" -var-file=${DIR}/vars/common.tfvars -out "$DIR/outputs/tf.output" $1 7 | -------------------------------------------------------------------------------- /terraform/vars/common.tfvars: -------------------------------------------------------------------------------- 1 | # __author__ = "Alejandro Guadarrama Dominguez" 2 | # __copyright__ = "Copyright 2020, Alejandro Guadarrama Dominguez" 3 | # __credits__ = ["Alejandro Guadarrama Dominguez"] 4 | # __license__ = "GPL" 5 | # __version__ = "0.0.1" 6 | # __maintainer__ = "Alejandro Guadarrama Dominguez" 7 | # __email__ = "alexgd.devops@gmail.com" 8 | # __status__ = "Dev" 9 | 10 | ## VSPHERE CONFIGURATIONS, SHOULD BE THE SAME IN YOU install-config.yaml 11 | vsphere_user = "administrator@sni.com.mx" 12 | vsphere_password = "Password123!" 13 | vsphere_server = "vcenter.sni.com.mx" 14 | 15 | datacenter = "BHM" 16 | datastore = "SAS-6K" 17 | network = "VM Network" 18 | resource_pool = "Resources" 19 | host = "esxi67.sni.com.mx" 20 | 21 | ## TEMPLATE NAME DESIRED 22 | template = { 23 | name = "rhcos-4.4.3-template" 24 | ovf_name = "coreos.ovf" 25 | } 26 | 27 | ## OCP NODES NETWORKING 28 | node_network = { 29 | netmask = "255.255.254.0" 30 | prefix = 23 31 | gateway = "10.56.240.254" 32 | dns = "10.56.240.1" 33 | ocp_domain = "sni.com.mx" 34 | } 35 | 36 | ocp_cluster_name = "ocp4" 37 | 38 | ## WEBSERVER CONTAINING THE IGNINTIO FILES 39 | url_ignition = "http://10.56.241.10" 40 | 41 | ## NODE INFORMATIOM 42 | node_configs = { 43 | bootstrap = { 44 | ip = ["10.56.241.12"] 45 | hostname = ["bootstrap"] 46 | cpu = 4 47 | memory = 16384 48 | disk = 120 49 | } 50 | master = { 51 | ip = ["10.56.241.13", "10.56.241.14", "10.56.241.15"] 52 | hostname = ["master01", "master02", "master03"] 53 | cpu = 4 54 | memory = 16384 55 | disk = 120 56 | } 57 | worker = { 58 | ip = ["10.56.241.16", "10.56.241.17"] 59 | hostname = ["worker01", "worker02"] 60 | cpu = 4 61 | memory = 16384 62 | disk = 120 63 | } 64 | infra = { 65 | ip = ["10.56.241.18", "10.56.241.19"] 66 | hostname = ["infra01", "infra02"] 67 | cpu = 2 68 | memory = 8192 69 | disk = 120 70 | } 71 | logging = { 72 | ip = ["10.56.241.20", "10.56.241.21"] 73 | hostname = ["logging01", "logging02"] 74 | cpu = 2 75 | memory = 8192 76 | disk = 120 77 | } 78 | } 79 | --------------------------------------------------------------------------------