├── README.md ├── pks-logo.png ├── screenshot-vmware-pks-lab-deployment-0.png ├── screenshot-vmware-pks-lab-deployment-1.png ├── screenshot-vmware-pks-lab-deployment-2.png └── vmware-pks-lab-deployment.ps1 /README.md: -------------------------------------------------------------------------------- 1 | # VMware Pivotal Container Service (PKS) Automated Lab Deployment 2 | 3 | ![](pks-logo.png) 4 | 5 | ## Table of Contents 6 | 7 | * [Description](#description) 8 | * [Changelog](#changelog) 9 | * [Requirements](#requirements) 10 | * [Configuration](#configuration) 11 | * [Logging](#logging) 12 | * [Sample Execution](#sample-execution) 13 | 14 | ## Description 15 | 16 | Automated deployment of the infrastructure (Nested ESXi including VSAN storage, NSX-T & Pivotal Ops Manager) required for configuring Pivotal Container Service (PKS) using PowerCLI. You just need a vCenter Server 6.5 or greater endpoint along with a single physical ESXi host to deploy the infrastructure to. For information, you can refer to this blog post [here](http://www.virtuallyghetto.com/2018/06/vghetto-automated-pivotal-container-service-pks-lab-deployment.html) for more details. 17 | 18 | Below is an screenshot of an example deployment: 19 | 20 | ![](screenshot-vmware-pks-lab-deployment-1.png) 21 | 22 | ## Changelog 23 | 24 | * 06/19/28 25 | * Removed Windows-specific dependencies, can now run on MacOS & Linux using PowerShell Core 26 | * Automate to deployment and configuration all PKS components (Ops Manager, BOSH Director & Harbor) 27 | 28 | 29 | ## Requirements 30 | * vCenter Server endpoint running at least vSphere 6.5 or later 31 | * Windows system that supports PowerCLI 32 | * [PowerCLI 10.1](https://code.vmware.com/web/dp/tool/vmware-powercli/10.1.0) or greater 33 | * Download [vmware-pks-lab-deployment.ps1](vmware-pks-lab-deployment.ps1) script 34 | * Nested ESXi [6.5d](https://download3.vmware.com/software/vmw-tools/nested-esxi/Nested_ESXi6.5d_Appliance_Template_v1.0.ova) or [6.5 Update 1](https://download3.vmware.com/software/vmw-tools/nested-esxi/Nested_ESXi6.5u1_Appliance_Template_v1.0.ova) Virtual Appliance OVA 35 | * Enable either [MAC Learning](https://williamlam.com/2018/04/native-mac-learning-in-vsphere-6-7-removes-the-need-for-promiscuous-mode-for-nested-esxi.html) or [Promiscuous Mode](https://kb.vmware.com/kb/1004099) on your physical ESXi host networking to ensure proper network connectivity for Nested ESXi workloads 36 | * NSX-T 2.1 37 | * [NSX-T 2.1 Manager for ESXi OVA](https://my.vmware.com/group/vmware/details?downloadGroup=NSX-T-210&productId=673&download=true&fileId=ceb09d4c2a044e745be0e5ab7f698295&secureParam=4faa362cd30eda7241a9471ee692774b&uuId=e268e430-59fd-45f2-a7c4-08629fe084ef&downloadType=) 38 | * [NSX-T 2.1 Controller for ESXi OVA](https://my.vmware.com/group/vmware/details?downloadGroup=NSX-T-210&productId=673&download=true&fileId=46f4cf5fa854bcd4f2c74f849e7acb69&secureParam=852f862526f99ea5f73fda3fb455ca14&uuId=e7c7f09a-3567-4aa4-9c80-d30bdae030fe&downloadType=) 39 | * [NSX-T 2.1 Edge For ESXi OVA](https://my.vmware.com/group/vmware/details?downloadGroup=NSX-T-210&productId=673&download=true&fileId=46bdb3be8b48cdea045eed660a9d68d0&secureParam=35cf3c9f5f143fd48a74afd1773cde7f&uuId=dd993136-275e-43b6-83e7-6ba10fec89ac&downloadType=) 40 | * [Ops Manager CLI (OM)](https://github.com/pivotal-cf/om/releases) 41 | * Windows - om-windows.exe 42 | * MacOS - om-darn 43 | * Linux - om-linux 44 | * [Pivotal Ops Manager 2.1.5 OVA](https://network.pivotal.io/products/ops-manager) 45 | * [Pivotal Container Service 1.0.4 Tile](https://network.pivotal.io/products/pivotal-container-service) 46 | * [Harbor 1.4.2 Tile](https://network.pivotal.io/products/harbor-container-registry) 47 | * [Stemcell 3468.42](https://network.pivotal.io/products/stemcells#/releases/97389) 48 | 49 | ## Configuration 50 | 51 | This section describes the credentials to your vCenter Server in which the PKS lab environment will be deployed to: 52 | ```console 53 | $VIServer = "vcenter.primp-industries.com" 54 | $VIUsername = "primp" 55 | $VIPassword = "-->MySuperDuperSecurePassword<--" 56 | ``` 57 | 58 | This section describes the location of the OVA files that are required for deployment. If you are using MacOS or Linux, simply update the path to reflect those systems 59 | 60 | ```console 61 | $NestedESXiApplianceOVA = "C:\Users\primp\Desktop\Nested_ESXi6.5u1_Appliance_Template_v1.0.ova" 62 | $NSXTManagerOVA = "C:\Users\primp\Desktop\nsx-unified-appliance-2.1.0.0.0.7395503.ova" 63 | $NSXTControllerOVA = "C:\Users\primp\Desktop\nsx-controller-2.1.0.0.0.7395493.ova" 64 | $NSXTEdgeOVA = "C:\Users\primp\Desktop\nsx-edge-2.1.0.0.0.7395502.ova" 65 | $PKSOpsMgrOVA = "C:\Users\primp\Desktop\pcf-vsphere-2.1-build.318.ova" 66 | ``` 67 | 68 | This section describes the location fo the PKS specific binaries that must also be downloaded locally to your system. If you are using MacOS or Linux, simply update the path to reflect those systems. In addition, make sure that your OM CLI is for your specific platform or you will have issues :) 69 | 70 | ```console 71 | $OMCLI = "C:\Users\primp\Desktop\om-windows.exe" 72 | $PKSTile = "C:\Users\primp\Desktop\pivotal-container-service-1.0.4-build.5.pivotal" 73 | $HarborTile = "C:\Users\primp\Desktop\harbor-container-registry-1.4.2-build.14.pivotal" 74 | $Stemcell = "C:\Users\primp\Desktop\bosh-stemcell-3468.42-vsphere-esxi-ubuntu-trusty-go_agent.tgz" 75 | ``` 76 | 77 | This section describes the Ops Manager deployment configuration 78 | ```console 79 | $OpsManagerDisplayName = "pks-opsmgr" 80 | $OpsManagerHostname = "pks-opsmgr.primp-industries.com" 81 | $OpsManagerIPAddress = "172.30.51.19" 82 | $OpsManagerNetmask = "255.255.255.0" 83 | $OpsManagerGateway = "172.30.51.1" 84 | $OpsManagerAdminPassword = "VMware1!" 85 | ``` 86 | 87 | This section defines the number of Nested ESXi VMs to deploy along with their associated IP Address(s). The names are merely the display name of the VMs when deployed. At a minimum, you should deploy at least three hosts, but you can always add additional hosts and the script will automatically take care of provisioning them correctly. 88 | ```console 89 | $NestedESXiHostnameToIPs = @{ 90 | "esxi-01" = "172.30.51.10" 91 | "esxi-02" = "172.30.51.11" 92 | "esxi-03" = "172.30.51.12" 93 | } 94 | ``` 95 | 96 | This section describes the resources allocated to each of the Nested ESXi VM(s). Depending on the deployment type, you may need to increase the resources. For Memory and Disk configuration, the unit is in GB. 97 | ```console 98 | $NestedESXivCPU = "2" 99 | $NestedESXivMEM = "24" #GB 100 | $NestedESXiCachingvDisk = "4" #GB 101 | $NestedESXiCapacityvDisk = "60" #GB 102 | ``` 103 | 104 | This section describes the location as well as the generic networking settings applied to the Nested ESXi, NSX-T & Ops Manager VMs. 105 | ```console 106 | $VirtualSwitchType = "VDS" # VSS or VDS 107 | $VMNetwork = "dv-vlan3251" 108 | $VMDatastore = "himalaya-local-SATA-dc3500-3" 109 | $VMNetmask = "255.255.255.0" 110 | $VMGateway = "172.30.51.1" 111 | $VMDNS = "172.30.0.100" 112 | $VMNTP = "pool.ntp.org" 113 | $VMPassword = "VMware1!" 114 | $VMDomain = "primp-industries.com" 115 | $VMSyslog = "172.30.51.170" 116 | # Applicable to Nested ESXi only 117 | $VMSSH = "true" 118 | $VMVMFS = "false" 119 | # Applicable to VC Deployment Target only 120 | $RootDatacenterName = "Production" 121 | $VMCluster = "Primp-Cluster" 122 | ``` 123 | 124 | This section describes the new vSphere Cluster which will be created for PKS Compute Cluster 125 | ```console 126 | $NewVCVSANClusterName = "PKS-Cluster" 127 | ``` 128 | 129 | This section describes the Ops Manager application configuration 130 | 131 | ```console 132 | $OpsmanAdminUsername = "admin" 133 | $OpsmanAdminPassword = "VMware1!" 134 | $OpsmanDecryptionPassword = "VMware1!" 135 | ``` 136 | 137 | This section describes the BOSH Director application configuration 138 | You can refer to this [blog post](https://www.virtuallyghetto.com/2018/03/getting-started-with-vmware-pivotal-container-service-pks-part-4-ops-manager-bosh.html) for more details on the individual settings 139 | 140 | ```console 141 | $BOSHvCenterUsername = "pks" 142 | $BOSHvCenterPassword = "VMware1!" 143 | $BOSHvCenterDatacenter = "Production" 144 | $BOSHvCenterPersistentDatastores = "himalaya-local-SATA-re4gp4T:storage,vsanDatastore" 145 | $BOSHvCenterEpemeralDatastores = "himalaya-local-SATA-re4gp4T:storage,vsanDatastore" 146 | $BOSHvCenterVMFolder = "PKS-VMS" 147 | $BOSHvCenterTemplateFolder = "PKS-TEMPLATES" 148 | $BOSHvCenterDiskFolder = "PKS-DISKS" 149 | 150 | # AZ Defintions 151 | $BOSHManagementAZ = @{ 152 | "AZ-Management" = "Primp-Cluster" 153 | } 154 | $BOSHComputeAZ = @{ 155 | "AZ-Compute" = "PKS-Cluster" 156 | } 157 | # Network Definitions 158 | $BOSHManagementNetwork = @{ 159 | "pks-mgmt-network" = @{ 160 | portgroupname = "dv-vlan3251" #represents the vSphere Portgroup or NSX-T Logical Switch Name 161 | cidr = "172.30.51.0/24" 162 | reserved_range = "172.30.51.1-172.30.51.30" 163 | dns = "172.30.0.100" 164 | gateway = "172.30.51.1" 165 | az = "AZ-Management" 166 | } 167 | } 168 | $BOSHServiceNetwork = @{ 169 | "k8s-mgmt-cluster-network" = @{ 170 | portgroupname = "K8S-Mgmt-Cluster-LS" #represents the vSphere Portgroup or NSX-T Logical Switch Name 171 | cidr = "10.10.0.0/24" 172 | reserved_range = "10.10.0.1" 173 | dns = "172.30.0.100" 174 | gateway = "10.10.0.1" 175 | az = "AZ-Compute" 176 | } 177 | } 178 | $BOSHManagementNetworkAssignment = "pks-mgmt-network" 179 | $BOSHManagementAZAssignment = "AZ-Management" 180 | ``` 181 | 182 | This section describes the PKS Control Plane application configuration 183 | You can refer to this [blog post](https://www.virtuallyghetto.com/2018/04/getting-started-with-vmware-pivotal-container-service-pks-part-5-pks-control-plane.html) for more details on the individual settings 184 | 185 | ```console 186 | $PKSDatacenter = $RootDatacenterName 187 | $PKSDatastore = "vsanDatastore" 188 | $PKSCluster = $NewVCVSANClusterName 189 | $PKSvCenter = $VIServer 190 | $PKSCPIMasterUsername = $BOSHvCenterUsername 191 | $PKSCPIMasterPassword = $BOSHvCenterPassword 192 | $PKSCPIWorkerUsername = $BOSHvCenterUsername 193 | $PKSCPIWorkerPassword = $BOSHvCenterPassword 194 | $PKSNSX = "nsx-mgr.primp-industries.com" 195 | $PKSNSXUsername = "admin" 196 | $PKSNSXPassword = "VMware1!" 197 | $PKSManagementNetworkAssignment = $BOSHManagementNetworkAssignment 198 | $PKSManagementAZAssignment = $BOSHManagementAZAssignment 199 | $PKSServiceNetworkAssignment = "k8s-mgmt-cluster-network" 200 | $PKSServiceAZAssignment = "AZ-Compute" 201 | $PKSPlan1AZ = "AZ-Compute" 202 | $PKSPlan2AZ = "AZ-Compute" 203 | $PKSUAAURL = "uaa.primp-industries.com" 204 | 205 | # The SSL Certificates are generated by the customer but you can use the following snippets to generate 206 | # the certs using openssl. Be sure to specify wildcard for your domain, this is whats asked of in PKS UIs 207 | # Copy the *.key and *.pem to the respective fields 208 | 209 | # openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -out pks.csr -newkey rsa:2048 -nodes -keyout pks.key 210 | # openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -x509 -sha256 -nodes -newkey rsa:2048 -keyout pks.key -out pks.pem 211 | # openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -x509 -sha256 -nodes -days 730 -newkey rsa:2048 -keyout pks.key -out pks.pem 212 | 213 | $pksCertPEM = @' 214 | -----BEGIN CERTIFICATE----- 215 | MIIDyzCCArOgAwIBAgIJAOkdUNyqSIk+MA0GCSqGSIb3DQEBCwUAMHwxCzAJBgNV 216 | BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FudGEgQmFyYmFyYTEZMBcG 217 | A1UECgwQUHJpbXAtSW5kdXN0cmllczEMMAoGA1UECwwDUiZEMR8wHQYDVQQDDBYq 218 | LnByaW1wLWluZHVzdHJpZXMuY29tMB4XDTE4MDYxNzAyMjM0M1oXDTIwMDYxNjAy 219 | MjM0M1owfDELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1TYW50 220 | YSBCYXJiYXJhMRkwFwYDVQQKDBBQcmltcC1JbmR1c3RyaWVzMQwwCgYDVQQLDANS 221 | JkQxHzAdBgNVBAMMFioucHJpbXAtaW5kdXN0cmllcy5jb20wggEiMA0GCSqGSIb3 222 | DQEBAQUAA4IBDwAwggEKAoIBAQDFHe3d1DfNmkAmLLmWnCXwjHa58FiPwHt30bje 223 | KsJ1Bbn4qx51Y8Rjp7jQ9zipFF8EaWfK0weym1PHyr2Pxq0EshYvWKl+in5rKshY 224 | qtLvsu3wZe5QpFQrbNgqsjpZ6/Vo1mjSgxYtZs1NyPNIv/ertM9iaTyileenUtnk 225 | XgUzRgUXYgzRzNsCd9zaKc6I11N2g8/EKa0WXN1x+908BLAvyAlDX5Hqa66tDjZE 226 | pIkRwRvmPWnoGj/wssfbw4wosfHaaCKvtv0AiAruheFy8Tmah19Zy6Jfuhc1sjzl 227 | YO3GXFHpePls5U+oYjurL/2VAdY7Y4ZR9dDnpIs7vQm1HqdjAgMBAAGjUDBOMB0G 228 | A1UdDgQWBBQXb8OC3C0S7ScHz76AtJQlv13HmjAfBgNVHSMEGDAWgBQXb8OC3C0S 229 | 7ScHz76AtJQlv13HmjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCy 230 | eiypyNOeW6l1wEiIyXZJT8TqZKqLzPlIUWWci/mcJwzmvi3S1pXxI0Ke2o98sikX 231 | hb3JEfLfUogXTLcRo2taUv2iDtvrRGhoJte8a2S51VxuPRFuhDWmaYUTwNHDf8uP 232 | 3S8WlkD8Foc2K6kK1UnfHN2CI43KC2Boce3jtyOz5Y4zABVMHoP0l4LxBUFnbhWu 233 | iV/ib/mMEoU4X3pX+JAjkVFSC2v0s9qhf4ws1shXgibx487hgZn8PqoQwXkPsaF/ 234 | n5U2jUtSM6d3Hxd2Jo1U+C6qfUSWBboI44w8rnyzI4u4hjwBE3S4NNh46+jkpv24 235 | iC9imeTm7r/IXWZBfs87 236 | -----END CERTIFICATE----- 237 | '@ 238 | 239 | $pksCertPrivateKey = @' 240 | -----BEGIN PRIVATE KEY----- 241 | MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDFHe3d1DfNmkAm 242 | LLmWnCXwjHa58FiPwHt30bjeKsJ1Bbn4qx51Y8Rjp7jQ9zipFF8EaWfK0weym1PH 243 | yr2Pxq0EshYvWKl+in5rKshYqtLvsu3wZe5QpFQrbNgqsjpZ6/Vo1mjSgxYtZs1N 244 | yPNIv/ertM9iaTyileenUtnkXgUzRgUXYgzRzNsCd9zaKc6I11N2g8/EKa0WXN1x 245 | +908BLAvyAlDX5Hqa66tDjZEpIkRwRvmPWnoGj/wssfbw4wosfHaaCKvtv0AiAru 246 | heFy8Tmah19Zy6Jfuhc1sjzlYO3GXFHpePls5U+oYjurL/2VAdY7Y4ZR9dDnpIs7 247 | vQm1HqdjAgMBAAECggEBAK0acHbbVDohmO4tXrnt3L+XivgVIqDzJzp9GX05TdXY 248 | to2zMKdkeuYNN5eDU+Xf9uV372dF1b+6+mM9HyVxEyZJgoQHt6lh1E0moBSFx4Iq 249 | vxvbV+LHvQb5qggsxmOLfNOZXypnZgVu/yKtM0ETHFxVB75jrpUVUf82GhWbn7N6 250 | 4PvyFLi13FCu9LOP1hTkp5zHbMPm7K+Bmiyc5VHZVoPQp4Nsg6nspJkA7jXHZRLk 251 | QPNl581L80duAzKcKi3rptZsoPiYpsnWK7sp3HkoT6Dzn0PTupmJ3LUS2VEDliG+ 252 | qeKaEnGEWv/sxini01GDpqowo7H95MW2dGDaFwDn8BkCgYEA9dmXh0nyIX7F1nps 253 | EnPvkO85UpEbGRkXhcpSxyfW5YPIKsFRIxi9LAUpFgtBoWCbUYwXt54G6MlgQihw 254 | iZqx2sn8gKFz/CPJ5CaOmi36Eo0h6AfiUycMOsa/M/Cd4g47GOl0lETtg3JoiRue 255 | BP6wFgiKDAYNJZuQxLNilK0C/gcCgYEAzUFGDhQeC9CR/Pfs12Tfu7m21BB+Ci1c 256 | tsjwz84J6XKJ3D06DOEiB2tIgPiXz6PVRIUhL2mr2564gB6tSf2JW9Xht/8/KJTX 257 | ABccwUnz43wjkSFQ8XfZQD1UIDTwa4Cq6zuznQxJ2nznxpwb/5Gzzsn9QQwIKr/l 258 | Kw7H7MYadMUCgYEAncmvlScCfjjtJMChyB4crbq74aA78hnGnRnDkwqgw+GWgMpe 259 | FtZz42LUgc9rqfVk+iudtT15VcKZQxzNTaO5bqCgrLXyyOr3UrTkZVQI4gsurcsR 260 | mSjAkqCoat+NlV5o045SQi8S+YBeU1EkVDRaM2n7n8fqfC6h9XzkUmPQPdUCgYAn 261 | n+5SUXfrd/x3BbXnb0XyC8xL7FMoy9EWSHyU4YXwV3hd2EQYsG3NWNzKaTOFlm9Z 262 | pwndCV1wLJgZw9JYcmXOIOBOkSw0PWe0UMHwXsKCrDiBkBj8RNLgH/bZsN6pIlHc 263 | z83BB9pKH8rvALw2/n3j8gK+SABboGgxg8z83NHGsQKBgQDiPKp4weX0cP1/ablO 264 | MeMFIms+ISxjFi+f7iuXNs2LahbP/gK0fHyqM22ZLNsK3sb8KLc3PhykNZCpvE/s 265 | mlahgq98CyVapmK97GDLFEdbUXY1JY8XakcrDDUA8/GBM5IL0Vi6uLIeW8+pTa0L 266 | gr6moJLrg6EMbw1C7xWrzhxR0g== 267 | -----END PRIVATE KEY----- 268 | '@ 269 | ``` 270 | 271 | This section describes the Harbor application configuration 272 | You can refer to this [blog post](https://www.virtuallyghetto.com/2018/04/getting-started-with-vmware-pivotal-container-service-pks-part-7-harbor.html) for more details on the individual settings 273 | 274 | ```console 275 | $HarborAdminPassword = "VMware1!" 276 | $HarborHostname = "pks-harbor.primp-industries.com" 277 | $HarborManagementNetworkAssignment = $BOSHManagementNetworkAssignment 278 | $HarborManagementAZAssignment = $BOSHManagementAZAssignment 279 | 280 | # The SSL Certificates are generated by the customer but you can use the following snippets to generate 281 | # the certs using openssl. Be sure to specify wildcard for your domain, this is whats asked of in PKS UIs 282 | # Copy the *.key and *.pem to the respective fields 283 | 284 | #openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -out harbor.csr -newkey rsa:2048 -nodes -keyout harbor.key 285 | #openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -x509 -sha256 -nodes -newkey rsa:2048 -keyout harbor.key -out harbor.pem 286 | #openssl req -subj '/C=US/ST=CA/L=Santa Barbara/O=Primp-Industries/OU=R&D/CN=*.primp-industries.com' -x509 -sha256 -nodes -days 730 -newkey rsa:2048 -keyout harbor.key -out harbor.pem 287 | 288 | $harborCertPEM = @' 289 | -----BEGIN CERTIFICATE----- 290 | MIIDyzCCArOgAwIBAgIJANKigLa4gSLgMA0GCSqGSIb3DQEBCwUAMHwxCzAJBgNV 291 | BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FudGEgQmFyYmFyYTEZMBcG 292 | A1UECgwQUHJpbXAtSW5kdXN0cmllczEMMAoGA1UECwwDUiZEMR8wHQYDVQQDDBYq 293 | LnByaW1wLWluZHVzdHJpZXMuY29tMB4XDTE4MDYxNzAyMjIwN1oXDTIwMDYxNjAy 294 | MjIwN1owfDELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1TYW50 295 | YSBCYXJiYXJhMRkwFwYDVQQKDBBQcmltcC1JbmR1c3RyaWVzMQwwCgYDVQQLDANS 296 | JkQxHzAdBgNVBAMMFioucHJpbXAtaW5kdXN0cmllcy5jb20wggEiMA0GCSqGSIb3 297 | DQEBAQUAA4IBDwAwggEKAoIBAQCzwhsWPQOb6wTX+wlfkhtllNHAoz6pswJbhezO 298 | JDvxw1L2sECUvvb0SFIPcKQnyIyaaS5IFlG03unFC/IbIKRVduTloc5gLEErfPy3 299 | QKLlbEMzA/44K1vhqubY0568rbqJ4oVRSe/o7aaSaM68F7Nw5M+M9G5Yv6Ib9PAM 300 | CyLrFt8sg0u0uGvcNe4oz4ZrvZVcXf6XTH6RlQZsGZjBs6OMX5Svn2DimtcpLEsv 301 | Thzq6J3IeFNO5cbkksn0l7YVC6KW/wYxEHAN847Pf0dVnrQej2W7+W804RH0McJa 302 | Tupuzz0MDdZLMoe95UcvwXwkYrPg5smL0+dtckTKbH4bpUDfAgMBAAGjUDBOMB0G 303 | A1UdDgQWBBQjsCzrbQDpXmeFHVMDCnuq5GCbSzAfBgNVHSMEGDAWgBQjsCzrbQDp 304 | XmeFHVMDCnuq5GCbSzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAC 305 | kXYq0EPs7GYZznwscEcwi8DMb8gr5xsIZ7fU1akwaDPwT9a3mCyGeOEsnmf/wW55 306 | LfQqXP1c3X9auTZ/N1GpP2qW/+PQA1hn825OcdLi6d+PFLPFS/Je+vr90GfTeZEG 307 | X0cljtAlHDEYTrh0m3Bp5QpxRto3Z7cTLermsIldBKuvc2SEuja4BYTLhot+urkN 308 | XvcJsPFO57D4f21qVVHXelVBEFbzn2Q+mWAb+emNi7tM45IMxWOh8pUl18888wKG 309 | dvLltkwohryDBtBICQB8JpZmdgLfSarCIQp13H0koMyR40DbZyachT8ftlcXIxAQ 310 | s3zI5HH8qWO16K17hjyz 311 | -----END CERTIFICATE----- 312 | '@ 313 | 314 | $harborCertPrivateKey = @' 315 | -----BEGIN PRIVATE KEY----- 316 | MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCzwhsWPQOb6wTX 317 | +wlfkhtllNHAoz6pswJbhezOJDvxw1L2sECUvvb0SFIPcKQnyIyaaS5IFlG03unF 318 | C/IbIKRVduTloc5gLEErfPy3QKLlbEMzA/44K1vhqubY0568rbqJ4oVRSe/o7aaS 319 | aM68F7Nw5M+M9G5Yv6Ib9PAMCyLrFt8sg0u0uGvcNe4oz4ZrvZVcXf6XTH6RlQZs 320 | GZjBs6OMX5Svn2DimtcpLEsvThzq6J3IeFNO5cbkksn0l7YVC6KW/wYxEHAN847P 321 | f0dVnrQej2W7+W804RH0McJaTupuzz0MDdZLMoe95UcvwXwkYrPg5smL0+dtckTK 322 | bH4bpUDfAgMBAAECggEAV6Uis9sX8WPLvssFrPV+Ki8/fh+aI//F/H32EiSUnbJQ 323 | tzsEogHiQwUoDaMsRsF/3KHAESHgwMGXVZ4Xc6acuZb40AXuq/Gn7N5KEceQJTB+ 324 | K1edEiIB8Kv1Vm8IDJLgSu6JdjMIqJeHCgfUFN2xfi/yCpX7X4ZAMkVg7V5Yript 325 | d3qWGXPHDZg0LOc5CkaZhYO0pRCVTZfi7pM8Dxo+C8Mw/hLBwrEziL2kgyFMvUoU 326 | F5dTtMoy+yqyX2mEzg7YfA7kv70OhpyRQNZ+B1guXhgd+WDaanqxh7bRrRIG5RHd 327 | h3q/nyLrXdrSWLoHgM8KkYE/TYisW6fBsTGYSl1M8QKBgQDawUKVBbxEbqvjGPxC 328 | jQj3r5gHM0r1fdIASnyji2+c8zwZykN6s1E5pC6Tqz7GRwE6dlo6X4O3udGvlr8D 329 | RSb0IfzYRcZ2+Xyu/WPahzcRt+NL5Y70wmduH9xN2LaOGAupmRYyOLS5+8hWcf+O 330 | HuPRCLU7ICibjKkIfrSkLTU6WwKBgQDSXR67bRE0SIxJzJTW/1jq7+JLN4xbJ11O 331 | NxlCbJ0KYS0esHPzeOPBTKUJDQa8MIP0OD651mNLe5fU02RvAYkSzVUbrgjSOguu 332 | aRQD8+3YegIYk2amhA8/N7CZWBT0wdsXzfnX4CijJZUgp1q1IgtZWLMk5xQm6Ql6 333 | IxbncGtyzQKBgQDD5oJ79hDtr7aav0tZRfgb59Jb0GF2i2C/BfWseDhR87mE1w+r 334 | GF7LIe7cK2UiJ4BAHLEcyWCp7eyMNJGGmi0SQEWwYHwlG7O++gisMJ7ubSFOXJuz 335 | MU1y33Fo/YQup/X5wbCQ9RtT2tlEIP7dBWi7T/MMqfXzpvnRM7cNt7aNNQKBgBwI 336 | 6PWVfXt4R6n2J8fXU+RLf98CUiQ7xMWNtkIR84PUm4zBe1JxQ/kY282u/LzLwmoj 337 | rMhbd/QxTnTAj1vz2m61CqibsvVBYxklS9OTCJmW+PyJeF6srtN/+nsVMAXGaApu 338 | GuPYLdJASfWGGCKXnOeVWJqMaTUeTXMHhh/l7YvpAoGAaobtDUTH78O27lVZ1IQ0 339 | /LsT9S72DhcK3n5a40WUMXgdxoQwa+3YaURspr20eJzt4xenY+4dKaJ+VHPPOfiK 340 | 4UeazY1fbefUkA+ElzV8uaQEc0OVak8fbP9+5b05tiNgBpPP3Vo81Nb55FmbLJJe 341 | Z/bFvEcEbEabMJwxMyPl5ys= 342 | -----END PRIVATE KEY----- 343 | '@ 344 | ``` 345 | 346 | This section describes the general NSX-T configurations and most of the defaults can be left alone outside of the environmental specific configurations such as **NSXPrivatePortgroup** and **$NSXIntermediateNetworkPortgroup** property. You will also need an additional VLAN-backed portgroup (e.g. dv-vlan3250) to act as an intermediate network to bridge you management traffic to the NSX-T Logical Networks. This is only required if you are using static routes which I am using in my environment. If you are using BGP, this is not required but will require some modification to the NSX-T configuration post-deployment. 347 | ```console 348 | $NSXRootPassword = "VMware1!" 349 | $NSXAdminUsername = "admin" 350 | $NSXAdminPassword = "VMware1!" 351 | $NSXAuditUsername = "audit" 352 | $NSXAuditPassword = "VMware1!" 353 | $NSXSSHEnable = "true" 354 | $NSXEnableRootLogin = "true" 355 | $NSXPrivatePortgroup = "dv-private" 356 | $NSXIntermediateNetworkPortgroup = "dv-vlan3250" 357 | ``` 358 | 359 | This section describes the two IP Pools: Tunnel Endpoint for ESXi host and Load Balancer Pool for K8s 360 | ```console 361 | $TunnelEndpointPoolName = "ESXi-VTEP-Pool" 362 | $TunnelEndpointPoolDescription = "Tunnel Endpoint for ESXi Transport Nodes" 363 | $TunnelEndpointPoolIPRangeStart = "192.168.1.10" 364 | $TunnelEndpointPoolIPRangeEnd = "192.168.1.30" 365 | $TunnelEndpointPoolCIDR = "192.168.1.0/24" 366 | $TunnelEndpointPoolGateway = "192.168.1.1" 367 | 368 | $LoadBalancerPoolName = "Load-Balancer-Pool" 369 | $LoadBalancerPoolDescription = "Load Balancer IP Pool" 370 | $LoadBalancerPoolIPRangeStart = "10.20.0.10" 371 | $LoadBalancerPoolIPRangeEnd = "10.20.0.50" 372 | $LoadBalancerPoolCIDR = "10.20.0.0/24" 373 | ``` 374 | 375 | This section describes the IP Block used for K8s workload 376 | ```console 377 | $ipBlockName = "PKS-IP-Block" 378 | $ipBlockNetwork = "172.16.0.0/16" 379 | ``` 380 | 381 | This section describes the Transport Zones that will be created 382 | ```console 383 | $OverlayTransportZoneName = "TZ-Overlay" 384 | $OverlayTransportZoneHostSwitchName = "Hostswitch1-OVERLAY" 385 | $VlanTransportZoneName = "TZ-VLAN" 386 | $VlanTransportZoneNameHostSwitchName = "Hostswitch2-VLAN" 387 | ``` 388 | 389 | This section describes the Uplink Profile that will be used for Edge VMs (just make sure you properly map the VM Networks when you deploy the Edge VM. Please refer to this [blog post](https://www.virtuallyghetto.com/2018/03/getting-started-with-vmware-pivotal-container-service-pks-part-3-nsx-t.html) for more details) 390 | ```console 391 | $ESXiUplinkProfileName = "ESXi-Uplink-Profile" 392 | $ESXiUplinkProfilePolicy = "FAILOVER_ORDER" 393 | $ESXiUplinkProfileActivepNIC = "vmnic2" 394 | $ESXiUplinkProfileTransportVLAN = "0" 395 | $ESXiUplinkProfileMTU = "1600" 396 | 397 | $EdgeUplinkProfileName = "Edge-Uplink-Profile" 398 | $EdgeUplinkProfilePolicy = "FAILOVER_ORDER" 399 | $EdgeUplinkProfileActivepNIC = "uplink-1" 400 | $EdgeUplinkProfileTransportVLAN = "0" 401 | $EdgeUplinkProfileMTU = "1600" 402 | $EdgeUplinkProfileOverlayvNIC = "fp-eth0" 403 | $EdgeUplinkProfileVlanvNIC = "fp-eth1" 404 | ``` 405 | 406 | This section describes the Logical Switches that will be created for both K8s Mgmt & Uplink 407 | ```console 408 | $K8SMgmtClusterLogicalSwitchName = "K8S-Mgmt-Cluster-LS" 409 | $K8SMgmtClusterLogicalSwitchReplicationMode = "MTEP" 410 | $UplinkLogicalSwitchName = "Uplink-LS" 411 | $UplinkLogicalSwitchVlan = "0" 412 | ``` 413 | 414 | This section describes the name fo the Edge Cluster 415 | ```console 416 | $EdgeClusterName = "Edge-Cluster-01" 417 | ``` 418 | 419 | This section describes the T0 Router that will be created. This assumes you will be using static route to provide connectivity to NSX env rather than BGP 420 | ```console 421 | $T0LogicalRouterName = "T0-LR" 422 | $T0LogicalRouterEdgeCluster = $EdgeClusterName 423 | $T0LogicalRouterHAMode = "ACTIVE_STANDBY" 424 | $T0UplinkRouterPortName = "Uplink-1" 425 | $T0UplinkRouterPortLS = $UplinkLogicalSwitchName 426 | $T0UplinkRouterPortSwitchPortName = "Uplink-1-Port" 427 | $T0UplinkRouterPortIP = "172.30.50.2" 428 | $T0UplinkRouterPortIPPrefix = "24" 429 | $T0UplinkRouterStaticRouteNetwork = "0.0.0.0/0" 430 | $T0UplinkRouterStaticRouteNextHop = "172.30.50.1" 431 | ``` 432 | 433 | This section describes the T1 Router that will be created for K8s Mgmt along with the up/down links 434 | ```console 435 | $T1LogicalRouterName = "T1-K8S-Mgmt-Cluster" 436 | $T1LogicalRouterEdgeCluster = $EdgeClusterName 437 | $T1LogicalRouterHAMode = "ACTIVE_STANDBY" 438 | $T1LogicalRouterFailOverMode = "PREEMPTIVE" 439 | $T1LinkedRouterPortLS = $K8SMgmtClusterLogicalSwitchName 440 | $T1LinkedRouterPortNameOnT0 = "LinkedPort_K8S-Mgmt-Cluster" 441 | $T1LinkedRouterPortNameOnT1 = "LinkedPort_T0-LR" 442 | $T1DownlinkRouterPortSwitchPortName = "Downlink-1-Port" 443 | $T1DownlinkRouterPortNameOnT1 = "Downlink-1" 444 | $T1DownlinkRouterPortIP = "10.10.0.1" 445 | $T1DownlinkRouterPortIPPrefix = "24" 446 | ``` 447 | 448 | This section describes the NSX-T Manager configuration. Currently, the re-sizing of the VM has not been enabled but will be possible in the future. For now, you can simply adjust the Hostname/IP Address. 449 | ```console 450 | $NSXTMgrDeploymentSize = "small" 451 | $NSXTMgrvCPU = "2" 452 | $NSXTMgrvMEM = "8" 453 | $NSXTMgrDisplayName = "nsx-mgr" 454 | $NSXTMgrHostname = "nsx-mgr.primp-industries.com" 455 | $NSXTMgrIPAddress = "172.30.51.13" 456 | ``` 457 | 458 | This section describes the NSX-T Controller configuration. You can deploy N-number of Controllers by simply adding entries to the hash table. Currently, the re-sizing of the VM has not been enabled but will be possible in the future. 459 | ```console 460 | $NSXTCtrvCPU = "2" 461 | $NSXTCtrvMEM = "6" 462 | $NSXControllerSharedSecret = "s3cR3ctz" 463 | $NSXTControllerHostnameToIPs = @{ 464 | "nsx-ctr01" = "172.30.51.14" 465 | "nsx-ctr02" = "172.30.51.15" 466 | "nsx-ctr03" = "172.30.51.16" 467 | } 468 | ``` 469 | 470 | This section describes the NSX-T Edge configuration. You can deploy N-number of Edges by simply adding entries to the hash table. Currently, the re-sizing of the VM has not been enabled but will be possible in the future. 471 | ```console 472 | $NSXTEdgevCPU = "8" 473 | $NSXTEdgevMEM = "16" 474 | $NSXTEdgeHostnameToIPs = @{ 475 | "nsx-edge01" = "172.30.51.17" 476 | } 477 | ``` 478 | 479 | Once you have saved your changes, you can now run the PowerCLI script as you normally would. 480 | 481 | ## Logging 482 | 483 | There is additional verbose logging that outputs as a log file in your current working directory called **vmare-pks-lab-deployment.log** which can be useful for debugging purposes. 484 | 485 | For advanced debugging, you can set `$pksDebug` flag to $true ($false by default) which will also print out the OM CLI commands if you are running into issues 486 | 487 | ## Sample Execution 488 | 489 | Here is an example deployment of just the PKS infrastructure: 490 | 491 | ![](screenshot-vmware-pks-lab-deployment-0.png) 492 | 493 | Here is an example deployment including the PKS components (deployment and configuration) 494 | 495 | ![](screenshot-vmware-pks-lab-deployment-2.png) 496 | 497 | If everything is succesful, you should be able to login to the NSX-T Manager to confirm all the network deployments. You should be able to login to your Ops Manager instance and start your PKS deployment workflows or the default will automatically configure the solution end-2-end. -------------------------------------------------------------------------------- /pks-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lamw/vmware-pks-automated-lab-deployment/2bf03335d0231191e2e57f5ff5a1c329f22cbd99/pks-logo.png -------------------------------------------------------------------------------- /screenshot-vmware-pks-lab-deployment-0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lamw/vmware-pks-automated-lab-deployment/2bf03335d0231191e2e57f5ff5a1c329f22cbd99/screenshot-vmware-pks-lab-deployment-0.png -------------------------------------------------------------------------------- /screenshot-vmware-pks-lab-deployment-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lamw/vmware-pks-automated-lab-deployment/2bf03335d0231191e2e57f5ff5a1c329f22cbd99/screenshot-vmware-pks-lab-deployment-1.png -------------------------------------------------------------------------------- /screenshot-vmware-pks-lab-deployment-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lamw/vmware-pks-automated-lab-deployment/2bf03335d0231191e2e57f5ff5a1c329f22cbd99/screenshot-vmware-pks-lab-deployment-2.png -------------------------------------------------------------------------------- /vmware-pks-lab-deployment.ps1: -------------------------------------------------------------------------------- 1 | # Author: William Lam 2 | # Website: www.virtuallyghetto.com 3 | # Description: PowerCLI script to deploy the required infrastructure for setting up Piovtal Container Service (PKS) and configuring PKS components e2e 4 | 5 | # vCenter Server to deploy PKS Lab 6 | $VIServer = "vcenter.primp-industries.com" 7 | $VIUsername = "primp" 8 | $VIPassword = "-->MySuperDuperSecurePassword<--" 9 | 10 | # Full Path to Nested ESXi 6.5u1 VA, NSX-T & PKS OVAs 11 | $NestedESXiApplianceOVA = "C:\Users\primp\Desktop\Nested_ESXi6.5u1_Appliance_Template_v1.0.ova" 12 | $NSXTManagerOVA = "C:\Users\primp\Desktop\nsx-unified-appliance-2.1.0.0.0.7395503.ova" 13 | $NSXTControllerOVA = "C:\Users\primp\Desktop\nsx-controller-2.1.0.0.0.7395493.ova" 14 | $NSXTEdgeOVA = "C:\Users\primp\Desktop\nsx-edge-2.1.0.0.0.7395502.ova" 15 | $PKSOpsMgrOVA = "C:\Users\primp\Desktop\pcf-vsphere-2.1-build.318.ova" 16 | 17 | # PKS Binaries 18 | $OMCLI = "C:\Users\primp\Desktop\om-windows.exe" 19 | $PKSTile = "C:\Users\primp\Desktop\pivotal-container-service-1.0.4-build.5.pivotal" 20 | $HarborTile = "C:\Users\primp\Desktop\harbor-container-registry-1.4.2-build.14.pivotal" 21 | $Stemcell = "C:\Users\primp\Desktop\bosh-stemcell-3468.42-vsphere-esxi-ubuntu-trusty-go_agent.tgz" 22 | 23 | # Ops Manager VM 24 | $OpsManagerDisplayName = "pks-opsmgr" 25 | $OpsManagerHostname = "pks-opsmgr.primp-industries.com" 26 | $OpsManagerIPAddress = "172.30.51.19" 27 | $OpsManagerNetmask = "255.255.255.0" 28 | $OpsManagerGateway = "172.30.51.1" 29 | $OpsManagerOSPassword = "VMware1!" 30 | 31 | # Nested ESXi VMs to deploy 32 | $NestedESXiHostnameToIPs = @{ 33 | "esxi-01" = "172.30.51.10" 34 | "esxi-02" = "172.30.51.11" 35 | "esxi-03" = "172.30.51.12" 36 | } 37 | 38 | # Nested ESXi VM Resources 39 | $NestedESXivCPU = "2" 40 | $NestedESXivMEM = "24" #GB 41 | $NestedESXiCachingvDisk = "4" #GB 42 | $NestedESXiCapacityvDisk = "60" #GB 43 | 44 | # General Deployment Configuration for Nested ESXi, NSX-T & PKS VMs 45 | $VirtualSwitchType = "VSS" # VSS or VDS 46 | $VMNetwork = "vlan3251" 47 | $VMDatastore = "himalaya-local-SATA-dc3500-3" 48 | $VMNetmask = "255.255.255.0" 49 | $VMGateway = "172.30.51.1" 50 | $VMDNS = "172.30.0.100" 51 | $VMNTP = "pool.ntp.org" 52 | $VMPassword = "VMware1!" 53 | $VMDomain = "primp-industries.com" 54 | $VMSyslog = "172.30.51.170" 55 | # Applicable to Nested ESXi only 56 | $VMSSH = "true" 57 | $VMVMFS = "false" 58 | # Applicable to VC Deployment Target only 59 | $RootDatacenterName = "Production" 60 | $VMCluster = "Primp-Cluster" 61 | 62 | # Name of new vSphere Cluster for the Compute ESXi Cluster 63 | $NewVCVSANClusterName = "PKS-Cluster" 64 | 65 | ######## Ops Manager Configuration ######## 66 | $OpsmanAdminUsername = "admin" 67 | $OpsmanAdminPassword = "VMware1!" 68 | $OpsmanDecryptionPassword = "VMware1!" 69 | 70 | ######## BOSH Director Configuration ######## 71 | $BOSHvCenterUsername = "pks" 72 | $BOSHvCenterPassword = "VMware1!" 73 | $BOSHvCenterDatacenter = "Production" 74 | $BOSHvCenterPersistentDatastores = "himalaya-local-SATA-re4gp4T:storage,vsanDatastore" 75 | $BOSHvCenterEpemeralDatastores = "himalaya-local-SATA-re4gp4T:storage,vsanDatastore" 76 | $BOSHvCenterVMFolder = "PKS-VMS" 77 | $BOSHvCenterTemplateFolder = "PKS-TEMPLATES" 78 | $BOSHvCenterDiskFolder = "PKS-DISKS" 79 | 80 | # AZ Defintions 81 | $BOSHManagementAZ = @{ 82 | "AZ-Management" = "Primp-Cluster" 83 | } 84 | $BOSHComputeAZ = @{ 85 | "AZ-Compute" = "PKS-Cluster" 86 | } 87 | # Network Definitions 88 | $BOSHManagementNetwork = @{ 89 | "pks-mgmt-network" = @{ 90 | portgroupname = "dv-vlan3251" #represents the vSphere Portgroup or NSX-T Logical Switch Name 91 | cidr = "172.30.51.0/24" 92 | reserved_range = "172.30.51.1-172.30.51.30" 93 | dns = "172.30.0.100" 94 | gateway = "172.30.51.1" 95 | az = "AZ-Management" 96 | } 97 | } 98 | $BOSHServiceNetwork = @{ 99 | "k8s-mgmt-cluster-network" = @{ 100 | portgroupname = "K8S-Mgmt-Cluster-LS" #represents the vSphere Portgroup or NSX-T Logical Switch Name 101 | cidr = "10.10.0.0/24" 102 | reserved_range = "10.10.0.1" 103 | dns = "172.30.0.100" 104 | gateway = "10.10.0.1" 105 | az = "AZ-Compute" 106 | } 107 | } 108 | $BOSHManagementNetworkAssignment = "pks-mgmt-network" 109 | $BOSHManagementAZAssignment = "AZ-Management" 110 | 111 | ######## PKS Control Plane Configuration ######## 112 | $PKSDatacenter = $RootDatacenterName 113 | $PKSDatastore = "vsanDatastore" 114 | $PKSCluster = $NewVCVSANClusterName 115 | $PKSvCenter = $VIServer 116 | $PKSCPIMasterUsername = $BOSHvCenterUsername 117 | $PKSCPIMasterPassword = $BOSHvCenterPassword 118 | $PKSCPIWorkerUsername = $BOSHvCenterUsername 119 | $PKSCPIWorkerPassword = $BOSHvCenterPassword 120 | $PKSNSX = "nsx-mgr.primp-industries.com" 121 | $PKSNSXUsername = "admin" 122 | $PKSNSXPassword = "VMware1!" 123 | $PKSManagementNetworkAssignment = $BOSHManagementNetworkAssignment 124 | $PKSManagementAZAssignment = $BOSHManagementAZAssignment 125 | $PKSServiceNetworkAssignment = "k8s-mgmt-cluster-network" 126 | $PKSServiceAZAssignment = "AZ-Compute" 127 | $PKSPlan1AZ = "AZ-Compute" 128 | $PKSPlan2AZ = "AZ-Compute" 129 | $PKSUAAURL = "uaa.primp-industries.com" 130 | 131 | $pksCertPEM = @' 132 | -----BEGIN CERTIFICATE----- 133 | MIIDyzCCArOgAwIBAgIJAOkdUNyqSIk+MA0GCSqGSIb3DQEBCwUAMHwxCzAJBgNV 134 | BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FudGEgQmFyYmFyYTEZMBcG 135 | A1UECgwQUHJpbXAtSW5kdXN0cmllczEMMAoGA1UECwwDUiZEMR8wHQYDVQQDDBYq 136 | LnByaW1wLWluZHVzdHJpZXMuY29tMB4XDTE4MDYxNzAyMjM0M1oXDTIwMDYxNjAy 137 | MjM0M1owfDELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1TYW50 138 | YSBCYXJiYXJhMRkwFwYDVQQKDBBQcmltcC1JbmR1c3RyaWVzMQwwCgYDVQQLDANS 139 | JkQxHzAdBgNVBAMMFioucHJpbXAtaW5kdXN0cmllcy5jb20wggEiMA0GCSqGSIb3 140 | DQEBAQUAA4IBDwAwggEKAoIBAQDFHe3d1DfNmkAmLLmWnCXwjHa58FiPwHt30bje 141 | KsJ1Bbn4qx51Y8Rjp7jQ9zipFF8EaWfK0weym1PHyr2Pxq0EshYvWKl+in5rKshY 142 | qtLvsu3wZe5QpFQrbNgqsjpZ6/Vo1mjSgxYtZs1NyPNIv/ertM9iaTyileenUtnk 143 | XgUzRgUXYgzRzNsCd9zaKc6I11N2g8/EKa0WXN1x+908BLAvyAlDX5Hqa66tDjZE 144 | pIkRwRvmPWnoGj/wssfbw4wosfHaaCKvtv0AiAruheFy8Tmah19Zy6Jfuhc1sjzl 145 | YO3GXFHpePls5U+oYjurL/2VAdY7Y4ZR9dDnpIs7vQm1HqdjAgMBAAGjUDBOMB0G 146 | A1UdDgQWBBQXb8OC3C0S7ScHz76AtJQlv13HmjAfBgNVHSMEGDAWgBQXb8OC3C0S 147 | 7ScHz76AtJQlv13HmjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCy 148 | eiypyNOeW6l1wEiIyXZJT8TqZKqLzPlIUWWci/mcJwzmvi3S1pXxI0Ke2o98sikX 149 | hb3JEfLfUogXTLcRo2taUv2iDtvrRGhoJte8a2S51VxuPRFuhDWmaYUTwNHDf8uP 150 | 3S8WlkD8Foc2K6kK1UnfHN2CI43KC2Boce3jtyOz5Y4zABVMHoP0l4LxBUFnbhWu 151 | iV/ib/mMEoU4X3pX+JAjkVFSC2v0s9qhf4ws1shXgibx487hgZn8PqoQwXkPsaF/ 152 | n5U2jUtSM6d3Hxd2Jo1U+C6qfUSWBboI44w8rnyzI4u4hjwBE3S4NNh46+jkpv24 153 | iC9imeTm7r/IXWZBfs87 154 | -----END CERTIFICATE----- 155 | '@ 156 | 157 | $pksCertPrivateKey = @' 158 | -----BEGIN PRIVATE KEY----- 159 | MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDFHe3d1DfNmkAm 160 | LLmWnCXwjHa58FiPwHt30bjeKsJ1Bbn4qx51Y8Rjp7jQ9zipFF8EaWfK0weym1PH 161 | yr2Pxq0EshYvWKl+in5rKshYqtLvsu3wZe5QpFQrbNgqsjpZ6/Vo1mjSgxYtZs1N 162 | yPNIv/ertM9iaTyileenUtnkXgUzRgUXYgzRzNsCd9zaKc6I11N2g8/EKa0WXN1x 163 | +908BLAvyAlDX5Hqa66tDjZEpIkRwRvmPWnoGj/wssfbw4wosfHaaCKvtv0AiAru 164 | heFy8Tmah19Zy6Jfuhc1sjzlYO3GXFHpePls5U+oYjurL/2VAdY7Y4ZR9dDnpIs7 165 | vQm1HqdjAgMBAAECggEBAK0acHbbVDohmO4tXrnt3L+XivgVIqDzJzp9GX05TdXY 166 | to2zMKdkeuYNN5eDU+Xf9uV372dF1b+6+mM9HyVxEyZJgoQHt6lh1E0moBSFx4Iq 167 | vxvbV+LHvQb5qggsxmOLfNOZXypnZgVu/yKtM0ETHFxVB75jrpUVUf82GhWbn7N6 168 | 4PvyFLi13FCu9LOP1hTkp5zHbMPm7K+Bmiyc5VHZVoPQp4Nsg6nspJkA7jXHZRLk 169 | QPNl581L80duAzKcKi3rptZsoPiYpsnWK7sp3HkoT6Dzn0PTupmJ3LUS2VEDliG+ 170 | qeKaEnGEWv/sxini01GDpqowo7H95MW2dGDaFwDn8BkCgYEA9dmXh0nyIX7F1nps 171 | EnPvkO85UpEbGRkXhcpSxyfW5YPIKsFRIxi9LAUpFgtBoWCbUYwXt54G6MlgQihw 172 | iZqx2sn8gKFz/CPJ5CaOmi36Eo0h6AfiUycMOsa/M/Cd4g47GOl0lETtg3JoiRue 173 | BP6wFgiKDAYNJZuQxLNilK0C/gcCgYEAzUFGDhQeC9CR/Pfs12Tfu7m21BB+Ci1c 174 | tsjwz84J6XKJ3D06DOEiB2tIgPiXz6PVRIUhL2mr2564gB6tSf2JW9Xht/8/KJTX 175 | ABccwUnz43wjkSFQ8XfZQD1UIDTwa4Cq6zuznQxJ2nznxpwb/5Gzzsn9QQwIKr/l 176 | Kw7H7MYadMUCgYEAncmvlScCfjjtJMChyB4crbq74aA78hnGnRnDkwqgw+GWgMpe 177 | FtZz42LUgc9rqfVk+iudtT15VcKZQxzNTaO5bqCgrLXyyOr3UrTkZVQI4gsurcsR 178 | mSjAkqCoat+NlV5o045SQi8S+YBeU1EkVDRaM2n7n8fqfC6h9XzkUmPQPdUCgYAn 179 | n+5SUXfrd/x3BbXnb0XyC8xL7FMoy9EWSHyU4YXwV3hd2EQYsG3NWNzKaTOFlm9Z 180 | pwndCV1wLJgZw9JYcmXOIOBOkSw0PWe0UMHwXsKCrDiBkBj8RNLgH/bZsN6pIlHc 181 | z83BB9pKH8rvALw2/n3j8gK+SABboGgxg8z83NHGsQKBgQDiPKp4weX0cP1/ablO 182 | MeMFIms+ISxjFi+f7iuXNs2LahbP/gK0fHyqM22ZLNsK3sb8KLc3PhykNZCpvE/s 183 | mlahgq98CyVapmK97GDLFEdbUXY1JY8XakcrDDUA8/GBM5IL0Vi6uLIeW8+pTa0L 184 | gr6moJLrg6EMbw1C7xWrzhxR0g== 185 | -----END PRIVATE KEY----- 186 | '@ 187 | 188 | ######## Harbor Configuration ######## 189 | $HarborAdminPassword = "VMware1!" 190 | $HarborHostname = "pks-harbor.primp-industries.com" 191 | $HarborManagementNetworkAssignment = $BOSHManagementNetworkAssignment 192 | $HarborManagementAZAssignment = $BOSHManagementAZAssignment 193 | 194 | $harborCertPEM = @' 195 | -----BEGIN CERTIFICATE----- 196 | MIIDyzCCArOgAwIBAgIJANKigLa4gSLgMA0GCSqGSIb3DQEBCwUAMHwxCzAJBgNV 197 | BAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FudGEgQmFyYmFyYTEZMBcG 198 | A1UECgwQUHJpbXAtSW5kdXN0cmllczEMMAoGA1UECwwDUiZEMR8wHQYDVQQDDBYq 199 | LnByaW1wLWluZHVzdHJpZXMuY29tMB4XDTE4MDYxNzAyMjIwN1oXDTIwMDYxNjAy 200 | MjIwN1owfDELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1TYW50 201 | YSBCYXJiYXJhMRkwFwYDVQQKDBBQcmltcC1JbmR1c3RyaWVzMQwwCgYDVQQLDANS 202 | JkQxHzAdBgNVBAMMFioucHJpbXAtaW5kdXN0cmllcy5jb20wggEiMA0GCSqGSIb3 203 | DQEBAQUAA4IBDwAwggEKAoIBAQCzwhsWPQOb6wTX+wlfkhtllNHAoz6pswJbhezO 204 | JDvxw1L2sECUvvb0SFIPcKQnyIyaaS5IFlG03unFC/IbIKRVduTloc5gLEErfPy3 205 | QKLlbEMzA/44K1vhqubY0568rbqJ4oVRSe/o7aaSaM68F7Nw5M+M9G5Yv6Ib9PAM 206 | CyLrFt8sg0u0uGvcNe4oz4ZrvZVcXf6XTH6RlQZsGZjBs6OMX5Svn2DimtcpLEsv 207 | Thzq6J3IeFNO5cbkksn0l7YVC6KW/wYxEHAN847Pf0dVnrQej2W7+W804RH0McJa 208 | Tupuzz0MDdZLMoe95UcvwXwkYrPg5smL0+dtckTKbH4bpUDfAgMBAAGjUDBOMB0G 209 | A1UdDgQWBBQjsCzrbQDpXmeFHVMDCnuq5GCbSzAfBgNVHSMEGDAWgBQjsCzrbQDp 210 | XmeFHVMDCnuq5GCbSzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAC 211 | kXYq0EPs7GYZznwscEcwi8DMb8gr5xsIZ7fU1akwaDPwT9a3mCyGeOEsnmf/wW55 212 | LfQqXP1c3X9auTZ/N1GpP2qW/+PQA1hn825OcdLi6d+PFLPFS/Je+vr90GfTeZEG 213 | X0cljtAlHDEYTrh0m3Bp5QpxRto3Z7cTLermsIldBKuvc2SEuja4BYTLhot+urkN 214 | XvcJsPFO57D4f21qVVHXelVBEFbzn2Q+mWAb+emNi7tM45IMxWOh8pUl18888wKG 215 | dvLltkwohryDBtBICQB8JpZmdgLfSarCIQp13H0koMyR40DbZyachT8ftlcXIxAQ 216 | s3zI5HH8qWO16K17hjyz 217 | -----END CERTIFICATE----- 218 | '@ 219 | 220 | $harborCertPrivateKey = @' 221 | -----BEGIN PRIVATE KEY----- 222 | MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCzwhsWPQOb6wTX 223 | +wlfkhtllNHAoz6pswJbhezOJDvxw1L2sECUvvb0SFIPcKQnyIyaaS5IFlG03unF 224 | C/IbIKRVduTloc5gLEErfPy3QKLlbEMzA/44K1vhqubY0568rbqJ4oVRSe/o7aaS 225 | aM68F7Nw5M+M9G5Yv6Ib9PAMCyLrFt8sg0u0uGvcNe4oz4ZrvZVcXf6XTH6RlQZs 226 | GZjBs6OMX5Svn2DimtcpLEsvThzq6J3IeFNO5cbkksn0l7YVC6KW/wYxEHAN847P 227 | f0dVnrQej2W7+W804RH0McJaTupuzz0MDdZLMoe95UcvwXwkYrPg5smL0+dtckTK 228 | bH4bpUDfAgMBAAECggEAV6Uis9sX8WPLvssFrPV+Ki8/fh+aI//F/H32EiSUnbJQ 229 | tzsEogHiQwUoDaMsRsF/3KHAESHgwMGXVZ4Xc6acuZb40AXuq/Gn7N5KEceQJTB+ 230 | K1edEiIB8Kv1Vm8IDJLgSu6JdjMIqJeHCgfUFN2xfi/yCpX7X4ZAMkVg7V5Yript 231 | d3qWGXPHDZg0LOc5CkaZhYO0pRCVTZfi7pM8Dxo+C8Mw/hLBwrEziL2kgyFMvUoU 232 | F5dTtMoy+yqyX2mEzg7YfA7kv70OhpyRQNZ+B1guXhgd+WDaanqxh7bRrRIG5RHd 233 | h3q/nyLrXdrSWLoHgM8KkYE/TYisW6fBsTGYSl1M8QKBgQDawUKVBbxEbqvjGPxC 234 | jQj3r5gHM0r1fdIASnyji2+c8zwZykN6s1E5pC6Tqz7GRwE6dlo6X4O3udGvlr8D 235 | RSb0IfzYRcZ2+Xyu/WPahzcRt+NL5Y70wmduH9xN2LaOGAupmRYyOLS5+8hWcf+O 236 | HuPRCLU7ICibjKkIfrSkLTU6WwKBgQDSXR67bRE0SIxJzJTW/1jq7+JLN4xbJ11O 237 | NxlCbJ0KYS0esHPzeOPBTKUJDQa8MIP0OD651mNLe5fU02RvAYkSzVUbrgjSOguu 238 | aRQD8+3YegIYk2amhA8/N7CZWBT0wdsXzfnX4CijJZUgp1q1IgtZWLMk5xQm6Ql6 239 | IxbncGtyzQKBgQDD5oJ79hDtr7aav0tZRfgb59Jb0GF2i2C/BfWseDhR87mE1w+r 240 | GF7LIe7cK2UiJ4BAHLEcyWCp7eyMNJGGmi0SQEWwYHwlG7O++gisMJ7ubSFOXJuz 241 | MU1y33Fo/YQup/X5wbCQ9RtT2tlEIP7dBWi7T/MMqfXzpvnRM7cNt7aNNQKBgBwI 242 | 6PWVfXt4R6n2J8fXU+RLf98CUiQ7xMWNtkIR84PUm4zBe1JxQ/kY282u/LzLwmoj 243 | rMhbd/QxTnTAj1vz2m61CqibsvVBYxklS9OTCJmW+PyJeF6srtN/+nsVMAXGaApu 244 | GuPYLdJASfWGGCKXnOeVWJqMaTUeTXMHhh/l7YvpAoGAaobtDUTH78O27lVZ1IQ0 245 | /LsT9S72DhcK3n5a40WUMXgdxoQwa+3YaURspr20eJzt4xenY+4dKaJ+VHPPOfiK 246 | 4UeazY1fbefUkA+ElzV8uaQEc0OVak8fbP9+5b05tiNgBpPP3Vo81Nb55FmbLJJe 247 | Z/bFvEcEbEabMJwxMyPl5ys= 248 | -----END PRIVATE KEY----- 249 | '@ 250 | 251 | # NSX-T Configuration 252 | $NSXRootPassword = "VMware1!" 253 | $NSXAdminUsername = "admin" 254 | $NSXAdminPassword = "VMware1!" 255 | $NSXAuditUsername = "audit" 256 | $NSXAuditPassword = "VMware1!" 257 | $NSXSSHEnable = "true" 258 | $NSXEnableRootLogin = "true" 259 | $NSXPrivatePortgroup = "vm-private-network" 260 | $NSXIntermediateNetworkPortgroup = "vlan3250" 261 | 262 | $TunnelEndpointPoolName = "ESXi-VTEP-Pool" 263 | $TunnelEndpointPoolDescription = "Tunnel Endpoint for ESXi Transport Nodes" 264 | $TunnelEndpointPoolIPRangeStart = "192.168.1.10" 265 | $TunnelEndpointPoolIPRangeEnd = "192.168.1.30" 266 | $TunnelEndpointPoolCIDR = "192.168.1.0/24" 267 | $TunnelEndpointPoolGateway = "192.168.1.1" 268 | 269 | $LoadBalancerPoolName = "Load-Balancer-Pool" 270 | $LoadBalancerPoolDescription = "Load Balancer IP Pool" 271 | $LoadBalancerPoolIPRangeStart = "10.20.0.10" 272 | $LoadBalancerPoolIPRangeEnd = "10.20.0.50" 273 | $LoadBalancerPoolCIDR = "10.20.0.0/24" 274 | 275 | $ipBlockName = "PKS-IP-Block" 276 | $ipBlockNetwork = "172.16.0.0/16" 277 | 278 | $OverlayTransportZoneName = "TZ-Overlay" 279 | $OverlayTransportZoneHostSwitchName = "Hostswitch1-OVERLAY" 280 | $VlanTransportZoneName = "TZ-VLAN" 281 | $VlanTransportZoneNameHostSwitchName = "Hostswitch2-VLAN" 282 | 283 | $ESXiUplinkProfileName = "ESXi-Uplink-Profile" 284 | $ESXiUplinkProfilePolicy = "FAILOVER_ORDER" 285 | $ESXiUplinkProfileActivepNIC = "vmnic2" 286 | $ESXiUplinkProfileTransportVLAN = "0" 287 | $ESXiUplinkProfileMTU = "1600" 288 | 289 | $EdgeUplinkProfileName = "Edge-Uplink-Profile" 290 | $EdgeUplinkProfilePolicy = "FAILOVER_ORDER" 291 | $EdgeUplinkProfileActivepNIC = "uplink-1" 292 | $EdgeUplinkProfileTransportVLAN = "0" 293 | $EdgeUplinkProfileMTU = "1600" 294 | $EdgeUplinkProfileOverlayvNIC = "fp-eth0" 295 | $EdgeUplinkProfileVlanvNIC = "fp-eth1" 296 | 297 | $K8SMgmtClusterLogicalSwitchName = "K8S-Mgmt-Cluster-LS" 298 | $K8SMgmtClusterLogicalSwitchReplicationMode = "MTEP" 299 | $UplinkLogicalSwitchName = "Uplink-LS" 300 | $UplinkLogicalSwitchVlan = "0" 301 | 302 | $EdgeClusterName = "Edge-Cluster-01" 303 | 304 | # T0 Router Configuration 305 | $T0LogicalRouterName = "T0-LR" 306 | $T0LogicalRouterEdgeCluster = $EdgeClusterName 307 | $T0LogicalRouterHAMode = "ACTIVE_STANDBY" 308 | $T0UplinkRouterPortName = "Uplink-1" 309 | $T0UplinkRouterPortLS = $UplinkLogicalSwitchName 310 | $T0UplinkRouterPortSwitchPortName = "Uplink-1-Port" 311 | $T0UplinkRouterPortIP = "172.30.50.2" 312 | $T0UplinkRouterPortIPPrefix = "24" 313 | $T0UplinkRouterStaticRouteNetwork = "0.0.0.0/0" 314 | $T0UplinkRouterStaticRouteNextHop = "172.30.50.1" 315 | 316 | # T1 Router Configuration 317 | $T1LogicalRouterName = "T1-K8S-Mgmt-Cluster" 318 | $T1LogicalRouterEdgeCluster = $EdgeClusterName 319 | $T1LogicalRouterHAMode = "ACTIVE_STANDBY" 320 | $T1LogicalRouterFailOverMode = "PREEMPTIVE" 321 | $T1LinkedRouterPortLS = $K8SMgmtClusterLogicalSwitchName 322 | $T1LinkedRouterPortNameOnT0 = "LinkedPort_K8S-Mgmt-Cluster" 323 | $T1LinkedRouterPortNameOnT1 = "LinkedPort_T0-LR" 324 | $T1DownlinkRouterPortSwitchPortName = "Downlink-1-Port" 325 | $T1DownlinkRouterPortNameOnT1 = "Downlink-1" 326 | $T1DownlinkRouterPortIP = "10.10.0.1" 327 | $T1DownlinkRouterPortIPPrefix = "24" 328 | 329 | # NSX-T Manager Configurations 330 | $NSXTMgrDeploymentSize = "small" 331 | $NSXTMgrvCPU = "2" 332 | $NSXTMgrvMEM = "8" 333 | $NSXTMgrDisplayName = "nsx-mgr" 334 | $NSXTMgrHostname = "nsx-mgr.primp-industries.com" 335 | $NSXTMgrIPAddress = "172.30.51.13" 336 | 337 | # NSX-T Controller Configurations 338 | $NSXTCtrvCPU = "2" 339 | $NSXTCtrvMEM = "6" 340 | $NSXControllerSharedSecret = "s3cR3ctz" 341 | $NSXTControllerHostnameToIPs = @{ 342 | "nsx-ctr01" = "172.30.51.14" 343 | "nsx-ctr02" = "172.30.51.15" 344 | "nsx-ctr03" = "172.30.51.16" 345 | } 346 | 347 | # NSX-T Edge Configuration 348 | $NSXTEdgevCPU = "8" 349 | $NSXTEdgevMEM = "16" 350 | $NSXTEdgeHostnameToIPs = @{ 351 | "nsx-edge01" = "172.30.51.17" 352 | } 353 | 354 | # Advanced Configurations 355 | # Set to 1 only if you have DNS (forward/reverse) for ESXi hostnames 356 | $addHostByDnsName = 1 357 | 358 | #### DO NOT EDIT BEYOND HERE #### 359 | 360 | $debug = $true 361 | $pksDebug = $false 362 | $verboseLogFile = "vmware-pks-lab-deployment.log" 363 | $random_string = -join ((65..90) + (97..122) | Get-Random -Count 8 | % {[char]$_}) 364 | $VAppName = "Nested-VMware-PKS-Lab-$random_string" 365 | 366 | $nsxStorageMap = @{ 367 | "manager"="160"; 368 | "controller"="120"; 369 | "edge"="120" 370 | } 371 | 372 | $esxiTotalCPU = 0 373 | $nsxTotalCPU = 0 374 | $esxiTotalMemory = 0 375 | $nsxTotalMemory = 0 376 | $esxiTotalStorage = 0 377 | $nsxTotalStorage = 0 378 | 379 | $preCheck = 1 380 | $confirmDeployment = 1 381 | $deployNestedESXiVMs = 1 382 | $deployOpsManager = 1 383 | $setupNewVC = 1 384 | $addESXiHostsToVC = 1 385 | $configureVSANDiskGroups = 1 386 | $deployNSX = 1 387 | $initialNSXConfig = 1 388 | $postDeployNSXConfig = 1 389 | $moveVMsIntovApp = 1 390 | $uploadStemcell = 1 391 | $setupOpsManager = 1 392 | $setupBOSHDirector = 1 393 | $setupPKS = 1 394 | $setupHarbor = 1 395 | 396 | $StartTime = Get-Date 397 | 398 | Function Set-VMKeystrokes { 399 | <# 400 | Please see http://www.virtuallyghetto.com/2017/09/automating-vm-keystrokes-using-the-vsphere-api-powercli.html for more details 401 | #> 402 | param( 403 | [Parameter(Mandatory=$true)][String]$VMName, 404 | [Parameter(Mandatory=$true)][String]$StringInput, 405 | [Parameter(Mandatory=$false)][Boolean]$ReturnCarriage, 406 | [Parameter(Mandatory=$false)][Boolean]$DebugOn 407 | ) 408 | 409 | # Map subset of USB HID keyboard scancodes 410 | # https://gist.github.com/MightyPork/6da26e382a7ad91b5496ee55fdc73db2 411 | $hidCharacterMap = @{ 412 | "a"="0x04"; 413 | "b"="0x05"; 414 | "c"="0x06"; 415 | "d"="0x07"; 416 | "e"="0x08"; 417 | "f"="0x09"; 418 | "g"="0x0a"; 419 | "h"="0x0b"; 420 | "i"="0x0c"; 421 | "j"="0x0d"; 422 | "k"="0x0e"; 423 | "l"="0x0f"; 424 | "m"="0x10"; 425 | "n"="0x11"; 426 | "o"="0x12"; 427 | "p"="0x13"; 428 | "q"="0x14"; 429 | "r"="0x15"; 430 | "s"="0x16"; 431 | "t"="0x17"; 432 | "u"="0x18"; 433 | "v"="0x19"; 434 | "w"="0x1a"; 435 | "x"="0x1b"; 436 | "y"="0x1c"; 437 | "z"="0x1d"; 438 | "1"="0x1e"; 439 | "2"="0x1f"; 440 | "3"="0x20"; 441 | "4"="0x21"; 442 | "5"="0x22"; 443 | "6"="0x23"; 444 | "7"="0x24"; 445 | "8"="0x25"; 446 | "9"="0x26"; 447 | "0"="0x27"; 448 | "!"="0x1e"; 449 | "@"="0x1f"; 450 | "#"="0x20"; 451 | "$"="0x21"; 452 | "%"="0x22"; 453 | "^"="0x23"; 454 | "&"="0x24"; 455 | "*"="0x25"; 456 | "("="0x26"; 457 | ")"="0x27"; 458 | "_"="0x2d"; 459 | "+"="0x2e"; 460 | "{"="0x2f"; 461 | "}"="0x30"; 462 | "|"="0x31"; 463 | ":"="0x33"; 464 | "`""="0x34"; 465 | "~"="0x35"; 466 | "<"="0x36"; 467 | ">"="0x37"; 468 | "?"="0x38"; 469 | "-"="0x2d"; 470 | "="="0x2e"; 471 | "["="0x2f"; 472 | "]"="0x30"; 473 | "\"="0x31"; 474 | "`;"="0x33"; 475 | "`'"="0x34"; 476 | ","="0x36"; 477 | "."="0x37"; 478 | "/"="0x38"; 479 | " "="0x2c"; 480 | } 481 | 482 | $vm = Get-View -ViewType VirtualMachine -Filter @{"Name"=$VMName} 483 | 484 | # Verify we have a VM or fail 485 | if(!$vm) { 486 | Write-host "Unable to find VM $VMName" 487 | return 488 | } 489 | 490 | $hidCodesEvents = @() 491 | foreach($character in $StringInput.ToCharArray()) { 492 | # Check to see if we've mapped the character to HID code 493 | if($hidCharacterMap.ContainsKey([string]$character)) { 494 | $hidCode = $hidCharacterMap[[string]$character] 495 | 496 | $tmp = New-Object VMware.Vim.UsbScanCodeSpecKeyEvent 497 | 498 | # Add leftShift modifer for capital letters and/or special characters 499 | if( ($character -cmatch "[A-Z]") -or ($character -match "[!|@|#|$|%|^|&|(|)|_|+|{|}|||:|~|<|>|?]") ) { 500 | $modifer = New-Object Vmware.Vim.UsbScanCodeSpecModifierType 501 | $modifer.LeftShift = $true 502 | $tmp.Modifiers = $modifer 503 | } 504 | 505 | # Convert to expected HID code format 506 | $hidCodeHexToInt = [Convert]::ToInt64($hidCode,"16") 507 | $hidCodeValue = ($hidCodeHexToInt -shl 16) -bor 0007 508 | 509 | $tmp.UsbHidCode = $hidCodeValue 510 | $hidCodesEvents+=$tmp 511 | } else { 512 | My-Logger Write-Host "The following character `"$character`" has not been mapped, you will need to manually process this character" 513 | break 514 | } 515 | } 516 | 517 | # Add return carriage to the end of the string input (useful for logins or executing commands) 518 | if($ReturnCarriage) { 519 | # Convert return carriage to HID code format 520 | $hidCodeHexToInt = [Convert]::ToInt64("0x28","16") 521 | $hidCodeValue = ($hidCodeHexToInt -shl 16) + 7 522 | 523 | $tmp = New-Object VMware.Vim.UsbScanCodeSpecKeyEvent 524 | $tmp.UsbHidCode = $hidCodeValue 525 | $hidCodesEvents+=$tmp 526 | } 527 | 528 | # Call API to send keystrokes to VM 529 | $spec = New-Object Vmware.Vim.UsbScanCodeSpec 530 | $spec.KeyEvents = $hidCodesEvents 531 | $results = $vm.PutUsbScanCodes($spec) 532 | } 533 | 534 | Function My-Logger { 535 | param( 536 | [Parameter(Mandatory=$true)] 537 | [String]$message 538 | ) 539 | 540 | $timeStamp = Get-Date -Format "MM-dd-yyyy_hh:mm:ss" 541 | 542 | Write-Host -NoNewline -ForegroundColor White "[$timestamp]" 543 | Write-Host -ForegroundColor Green " $message" 544 | $logMessage = "[$timeStamp] $message" 545 | $logMessage | Out-File -Append -LiteralPath $verboseLogFile 546 | } 547 | 548 | Function URL-Check([string] $url) { 549 | $isWorking = $true 550 | 551 | try { 552 | $request = [System.Net.WebRequest]::Create($url) 553 | $request.Method = "HEAD" 554 | $request.UseDefaultCredentials = $true 555 | 556 | $response = $request.GetResponse() 557 | $httpStatus = $response.StatusCode 558 | 559 | $isWorking = ($httpStatus -eq "OK") 560 | } 561 | catch { 562 | $isWorking = $false 563 | } 564 | return $isWorking 565 | } 566 | 567 | if($preCheck -eq 1) { 568 | if(!(Test-Path $NestedESXiApplianceOVA)) { 569 | Write-Host -ForegroundColor Red "`nUnable to find $NestedESXiApplianceOVA ...`nexiting" 570 | exit 571 | } 572 | 573 | if(!(Test-Path $NSXTManagerOVA)) { 574 | Write-Host -ForegroundColor Red "`nUnable to find $NSXTManagerOVA ...`nexiting" 575 | exit 576 | } 577 | 578 | if(!(Test-Path $NSXTControllerOVA)) { 579 | Write-Host -ForegroundColor Red "`nUnable to find $NSXTControllerOVA ...`nexiting" 580 | exit 581 | } 582 | 583 | if(!(Test-Path $NSXTEdgeOVA)) { 584 | Write-Host -ForegroundColor Red "`nUnable to find $NSXTEdgeOVA ...`nexiting" 585 | exit 586 | } 587 | 588 | if(!(Test-Path $OMCLI)) { 589 | Write-Host -ForegroundColor Red "`nUnable to find $OMCLI ...`nexiting" 590 | exit 591 | } 592 | 593 | if(!(Test-Path $PKSOpsMgrOVA)) { 594 | Write-Host -ForegroundColor Red "`nUnable to find $PKSOpsMgrOVA ...`nexiting" 595 | exit 596 | } 597 | 598 | if(!(Test-Path $PKSTile)) { 599 | Write-Host -ForegroundColor Red "`nUnable to find $PKSTile ...`nexiting" 600 | exit 601 | } 602 | 603 | if(!(Test-Path $HarborTile)) { 604 | Write-Host -ForegroundColor Red "`nUnable to find $HarborTile ...`nexiting" 605 | exit 606 | } 607 | } 608 | 609 | if($confirmDeployment -eq 1) { 610 | Write-Host -ForegroundColor Magenta "`nPlease confirm the following configuration will be deployed:`n" 611 | 612 | Write-Host -ForegroundColor Yellow "---- VMware PKS Automated Lab Deployment Configuration ---- " 613 | Write-Host -NoNewline -ForegroundColor Green "Nested ESXi Image Path: " 614 | Write-Host -ForegroundColor White $NestedESXiApplianceOVA 615 | 616 | if($DeployNSX -eq 1) { 617 | Write-Host -NoNewline -ForegroundColor Green "NSX-T Manager Image Path: " 618 | Write-Host -ForegroundColor White $NSXTManagerOVA 619 | Write-Host -NoNewline -ForegroundColor Green "NSX-T Controller Image Path: " 620 | Write-Host -ForegroundColor White $NSXTControllerOVA 621 | Write-Host -NoNewline -ForegroundColor Green "NSX-T Edge Image Path: " 622 | Write-Host -ForegroundColor White $NSXTEdgeOVA 623 | } 624 | 625 | Write-Host -NoNewline -ForegroundColor Green "Ops Manager Image Path: " 626 | Write-Host -ForegroundColor White $PKSOpsMgrOVA 627 | Write-Host -NoNewline -ForegroundColor Green "PKS Tile Path: " 628 | Write-Host -ForegroundColor White $PKSTile 629 | Write-Host -NoNewline -ForegroundColor Green "Harbor Tile Path: " 630 | Write-Host -ForegroundColor White $HarborTile 631 | 632 | Write-Host -ForegroundColor Yellow "`n---- vCenter Server Deployment Target Configuration ----" 633 | Write-Host -NoNewline -ForegroundColor Green "vCenter Server Address: " 634 | Write-Host -ForegroundColor White $VIServer 635 | Write-Host -NoNewline -ForegroundColor Green "VM Network: " 636 | Write-Host -ForegroundColor White $VMNetwork 637 | 638 | if($DeployNSX -eq 1) { 639 | Write-Host -NoNewline -ForegroundColor Green "NSX-T Private VM Network: " 640 | Write-Host -ForegroundColor White $NSXPrivatePortgroup 641 | } 642 | 643 | Write-Host -NoNewline -ForegroundColor Green "VM Storage: " 644 | Write-Host -ForegroundColor White $VMDatastore 645 | Write-Host -NoNewline -ForegroundColor Green "VM Cluster: " 646 | Write-Host -ForegroundColor White $VMCluster 647 | Write-Host -NoNewline -ForegroundColor Green "VM vApp: " 648 | Write-Host -ForegroundColor White $VAppName 649 | 650 | Write-Host -ForegroundColor Yellow "`n---- vESXi Configuration ----" 651 | Write-Host -NoNewline -ForegroundColor Green "# of Nested ESXi VMs: " 652 | Write-Host -ForegroundColor White $NestedESXiHostnameToIPs.count 653 | Write-Host -NoNewline -ForegroundColor Green "vCPU: " 654 | Write-Host -ForegroundColor White $NestedESXivCPU 655 | Write-Host -NoNewline -ForegroundColor Green "vMEM: " 656 | Write-Host -ForegroundColor White "$NestedESXivMEM GB" 657 | Write-Host -NoNewline -ForegroundColor Green "Caching VMDK: " 658 | Write-Host -ForegroundColor White "$NestedESXiCachingvDisk GB" 659 | Write-Host -NoNewline -ForegroundColor Green "Capacity VMDK: " 660 | Write-Host -ForegroundColor White "$NestedESXiCapacityvDisk GB" 661 | Write-Host -NoNewline -ForegroundColor Green "IP Address(s): " 662 | Write-Host -ForegroundColor White $NestedESXiHostnameToIPs.Values 663 | Write-Host -NoNewline -ForegroundColor Green "Netmask " 664 | Write-Host -ForegroundColor White $VMNetmask 665 | Write-Host -NoNewline -ForegroundColor Green "Gateway: " 666 | Write-Host -ForegroundColor White $VMGateway 667 | Write-Host -NoNewline -ForegroundColor Green "DNS: " 668 | Write-Host -ForegroundColor White $VMDNS 669 | Write-Host -NoNewline -ForegroundColor Green "NTP: " 670 | Write-Host -ForegroundColor White $VMNTP 671 | Write-Host -NoNewline -ForegroundColor Green "Syslog: " 672 | Write-Host -ForegroundColor White $VMSyslog 673 | Write-Host -NoNewline -ForegroundColor Green "Enable SSH: " 674 | Write-Host -ForegroundColor White $VMSSH 675 | Write-Host -NoNewline -ForegroundColor Green "Create VMFS Volume: " 676 | Write-Host -ForegroundColor White $VMVMFS 677 | Write-Host -NoNewline -ForegroundColor Green "Root Password: " 678 | Write-Host -ForegroundColor White $VMPassword 679 | 680 | if($DeployNSX -eq 1) { 681 | Write-Host -ForegroundColor Yellow "`n---- NSX-T Configuration ----" 682 | Write-Host -NoNewline -ForegroundColor Green "NSX Manager Hostname: " 683 | Write-Host -ForegroundColor White $NSXTMgrHostname 684 | Write-Host -NoNewline -ForegroundColor Green "NSX Manager IP Address: " 685 | Write-Host -ForegroundColor White $NSXTMgrIPAddress 686 | Write-Host -NoNewline -ForegroundColor Green "# of NSX Controller VMs: " 687 | Write-Host -ForegroundColor White $NSXTControllerHostnameToIPs.count 688 | Write-Host -NoNewline -ForegroundColor Green "IP Address(s): " 689 | Write-Host -ForegroundColor White $NSXTControllerHostnameToIPs.Values 690 | Write-Host -NoNewline -ForegroundColor Green "# of NSX Edge VMs: " 691 | Write-Host -ForegroundColor White $NSXTEdgeHostnameToIPs.count 692 | Write-Host -NoNewline -ForegroundColor Green "IP Address(s): " 693 | Write-Host -ForegroundColor White $NSXTEdgeHostnameToIPs.Values 694 | Write-Host -NoNewline -ForegroundColor Green "Netmask: " 695 | Write-Host -ForegroundColor White $VMNetmask 696 | Write-Host -NoNewline -ForegroundColor Green "Gateway: " 697 | Write-Host -ForegroundColor White $VMGateway 698 | Write-Host -NoNewline -ForegroundColor Green "Enable SSH: " 699 | Write-Host -ForegroundColor White $NSXSSHEnable 700 | Write-Host -NoNewline -ForegroundColor Green "Enable Root Login: " 701 | Write-Host -ForegroundColor White $NSXEnableRootLogin 702 | } 703 | 704 | $esxiTotalCPU = $NestedESXiHostnameToIPs.count * [int]$NestedESXivCPU 705 | $esxiTotalMemory = $NestedESXiHostnameToIPs.count * [int]$NestedESXivMEM 706 | $esxiTotalStorage = ($NestedESXiHostnameToIPs.count * [int]$NestedESXiCachingvDisk) + ($NestedESXiHostnameToIPs.count * [int]$NestedESXiCapacityvDisk) 707 | 708 | Write-Host -ForegroundColor Yellow "`n---- Resource Requirements ----" 709 | Write-Host -NoNewline -ForegroundColor Green "ESXi VM CPU: " 710 | Write-Host -NoNewline -ForegroundColor White $esxiTotalCPU 711 | Write-Host -NoNewline -ForegroundColor Green " ESXi VM Memory: " 712 | Write-Host -NoNewline -ForegroundColor White $esxiTotalMemory "GB " 713 | Write-Host -NoNewline -ForegroundColor Green "ESXi VM Storage: " 714 | Write-Host -ForegroundColor White $esxiTotalStorage "GB" 715 | 716 | if($DeployNSX -eq 1) { 717 | $nsxTotalCPU += $NSXTControllerHostnameToIPs.count * [int]$NSXTCtrvCPU 718 | $nsxTotalMemory += $NSXTControllerHostnameToIPs.count * [int]$NSXTCtrvMEM 719 | $nsxTotalStorage += $NSXTControllerHostnameToIPs.count * [int]$nsxStorageMap["controller"] 720 | 721 | $nsxTotalCPU += [int]$NSXTMgrvCPU 722 | $nsxTotalMemory += [int]$NSXTMgrvMEM 723 | $nsxTotalStorage += [int]$nsxStorageMap["manager"] 724 | 725 | $nsxTotalCPU += $NSXTEdgeHostnameToIPs.count * [int]$NSXTEdgevCPU 726 | $nsxTotalMemory += $NSXTEdgeHostnameToIPs.count * [int]$NSXTEdgevMEM 727 | $nsxTotalStorage += $NSXTEdgeHostnameToIPs.count * [int]$nsxStorageMap["edge"] 728 | 729 | Write-Host -NoNewline -ForegroundColor Green "NSX VM CPU: " 730 | Write-Host -NoNewline -ForegroundColor White $nsxTotalCPU 731 | Write-Host -NoNewline -ForegroundColor Green " NSX VM Memory: " 732 | Write-Host -NoNewline -ForegroundColor White $nsxTotalMemory "GB " 733 | Write-Host -NoNewline -ForegroundColor Green " NSX VM Storage: " 734 | Write-Host -ForegroundColor White $nsxTotalStorage "GB" 735 | } 736 | 737 | Write-Host -ForegroundColor White "---------------------------------------------" 738 | Write-Host -NoNewline -ForegroundColor Green "Total CPU: " 739 | Write-Host -ForegroundColor White ($esxiTotalCPU + $nsxTotalCPU) 740 | Write-Host -NoNewline -ForegroundColor Green "Total Memory: " 741 | Write-Host -ForegroundColor White ($esxiTotalMemory + $nsxTotalMemory) "GB" 742 | Write-Host -NoNewline -ForegroundColor Green "Total Storage: " 743 | Write-Host -ForegroundColor White ($esxiTotalStorage + $nsxTotalStorage) "GB" 744 | 745 | Write-Host -ForegroundColor Magenta "`nWould you like to proceed with this deployment?`n" 746 | $answer = Read-Host -Prompt "Do you accept (Y or N)" 747 | if($answer -ne "Y" -or $answer -ne "y") { 748 | exit 749 | } 750 | Clear-Host 751 | } 752 | 753 | if(($isWindows) -or ($Env:OS -eq "Windows_NT")) { 754 | $DestinationCtrThumprintStore = "$ENV:TMP\controller-thumbprint" 755 | $DestinationVCThumbprintStore = "$ENV:TMP\vc-thumbprint" 756 | } else { 757 | $DestinationCtrThumprintStore = "/tmp/controller-thumbprint" 758 | $DestinationVCThumbprintStore = "/tmp/vc-thumbprint" 759 | } 760 | 761 | if( ($deployNestedESXiVMs -eq 1) -or ($deployOpsManager -eq 1) -or ($setupNewVC -eq 1) -or ($addESXiHostsToVC -eq 1) -or ($configureVSANDiskGroups -eq 1) -or ($deployNSX -eq 1) -or ($moveVMsIntovApp -eq 1) ) { 762 | My-Logger "Connecting to Management vCenter Server $VIServer ..." 763 | $viConnection = Connect-VIServer $VIServer -User $VIUsername -Password $VIPassword -WarningAction SilentlyContinue 764 | 765 | $datastore = Get-Datastore -Server $viConnection -Name $VMDatastore | Select -First 1 766 | if($VirtualSwitchType -eq "VSS") { 767 | $network = Get-VirtualPortGroup -Server $viConnection -Name $VMNetwork | Select -First 1 768 | #if($DeployNSX -eq 1) { 769 | $privateNetwork = Get-VirtualPortGroup -Server $viConnection -Name $NSXPrivatePortgroup | Select -First 1 770 | $NSXIntermediateNetwork = Get-VirtualPortgroup -Server $viConnection -Name $NSXIntermediateNetworkPortgroup | Select -First 1 771 | #} 772 | } else { 773 | $network = Get-VDPortgroup -Server $viConnection -Name $VMNetwork | Select -First 1 774 | if($DeployNSX -eq 1) { 775 | $privateNetwork = Get-VDPortgroup -Server $viConnection -Name $NSXPrivatePortgroup | Select -First 1 776 | $NSXIntermediateNetwork = Get-VDPortgroup -Server $viConnection -Name $NSXIntermediateNetworkPortgroup | Select -First 1 777 | } 778 | } 779 | $cluster = Get-Cluster -Server $viConnection -Name $VMCluster 780 | $datacenter = $cluster | Get-Datacenter 781 | $vmhost = $cluster | Get-VMHost | Select -First 1 782 | 783 | if($datastore.Type -eq "vsan") { 784 | My-Logger "VSAN Datastore detected, enabling Fake SCSI Reservations ..." 785 | Get-AdvancedSetting -Entity $vmhost -Name "VSAN.FakeSCSIReservations" | Set-AdvancedSetting -Value 1 -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 786 | } 787 | } 788 | 789 | if($deployNestedESXiVMs -eq 1) { 790 | $NestedESXiHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 791 | $VMName = $_.Key 792 | $VMIPAddress = $_.Value 793 | 794 | $ovfconfig = Get-OvfConfiguration $NestedESXiApplianceOVA 795 | $networkMapLabel = ($ovfconfig.ToHashTable().keys | where {$_ -Match "NetworkMapping"}).replace("NetworkMapping.","").replace("-","_").replace(" ","_") 796 | $ovfconfig.NetworkMapping.$networkMapLabel.value = $VMNetwork 797 | 798 | $ovfconfig.common.guestinfo.hostname.value = $VMName 799 | $ovfconfig.common.guestinfo.ipaddress.value = $VMIPAddress 800 | $ovfconfig.common.guestinfo.netmask.value = $VMNetmask 801 | $ovfconfig.common.guestinfo.gateway.value = $VMGateway 802 | $ovfconfig.common.guestinfo.dns.value = $VMDNS 803 | $ovfconfig.common.guestinfo.domain.value = $VMDomain 804 | $ovfconfig.common.guestinfo.ntp.value = $VMNTP 805 | $ovfconfig.common.guestinfo.syslog.value = $VMSyslog 806 | $ovfconfig.common.guestinfo.password.value = $VMPassword 807 | if($VMSSH -eq "true") { 808 | $VMSSHVar = $true 809 | } else { 810 | $VMSSHVar = $false 811 | } 812 | $ovfconfig.common.guestinfo.ssh.value = $VMSSHVar 813 | 814 | My-Logger "Deploying Nested ESXi VM $VMName ..." 815 | $vm = Import-VApp -Source $NestedESXiApplianceOVA -OvfConfiguration $ovfconfig -Name $VMName -Location $cluster -VMHost $vmhost -Datastore $datastore -DiskStorageFormat thin 816 | 817 | My-Logger "Adding vmnic2/vmnic3 to $NSXPrivatePortgroup ..." 818 | New-NetworkAdapter -VM $vm -Type Vmxnet3 -Portgroup $privateNetwork -StartConnected -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 819 | New-NetworkAdapter -VM $vm -Type Vmxnet3 -Portgroup $privateNetwork -StartConnected -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 820 | 821 | My-Logger "Updating vCPU Count to $NestedESXivCPU & vMEM to $NestedESXivMEM GB ..." 822 | Set-VM -Server $viConnection -VM $vm -NumCpu $NestedESXivCPU -MemoryGB $NestedESXivMEM -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 823 | 824 | My-Logger "Updating vSAN Caching VMDK size to $NestedESXiCachingvDisk GB ..." 825 | Get-HardDisk -Server $viConnection -VM $vm -Name "Hard disk 2" | Set-HardDisk -CapacityGB $NestedESXiCachingvDisk -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 826 | 827 | My-Logger "Updating vSAN Capacity VMDK size to $NestedESXiCapacityvDisk GB ..." 828 | Get-HardDisk -Server $viConnection -VM $vm -Name "Hard disk 3" | Set-HardDisk -CapacityGB $NestedESXiCapacityvDisk -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 829 | 830 | My-Logger "Powering On $vmname ..." 831 | $vm | Start-Vm -RunAsync | Out-Null 832 | } 833 | } 834 | 835 | if($deployOpsManager -eq 1) { 836 | # Deploy Ops Manager 837 | $opsMgrOvfCOnfig = Get-OvfConfiguration $PKSOpsMgrOVA 838 | $opsMgrOvfCOnfig.Common.ip0.Value = $OpsManagerIPAddress 839 | $opsMgrOvfCOnfig.Common.custom_hostname.Value = $OpsManagerHostname 840 | $opsMgrOvfCOnfig.Common.netmask0.Value = $OpsManagerNetmask 841 | $opsMgrOvfCOnfig.Common.gateway.Value = $OpsManagerGateway 842 | $opsMgrOvfCOnfig.Common.ntp_servers.Value = $VMNTP 843 | $opsMgrOvfCOnfig.Common.DNS.Value = $VMDNS 844 | $opsMgrOvfCOnfig.Common.admin_password.Value = $OpsManagerOSPassword 845 | $opsMgrOvfCOnfig.NetworkMapping.Network_1.Value = $VMNetwork 846 | 847 | My-Logger "Deploying PKS Ops Manager $OpsManagerDisplayName ..." 848 | $opsmgr_vm = Import-VApp -Source $PKSOpsMgrOVA -OvfConfiguration $opsMgrOvfCOnfig -Name $OpsManagerDisplayName -Location $cluster -VMHost $vmhost -Datastore $datastore -DiskStorageFormat thin 849 | 850 | My-Logger "Powering On $OpsManagerDisplayName ..." 851 | $opsmgr_vm | Start-Vm -RunAsync | Out-Null 852 | } 853 | 854 | if($DeployNSX -eq 1) { 855 | # Deploy NSX Manager 856 | $nsxMgrOvfConfig = Get-OvfConfiguration $NSXTManagerOVA 857 | $nsxMgrOvfConfig.DeploymentOption.Value = $NSXTMgrDeploymentSize 858 | $nsxMgrOvfConfig.NetworkMapping.Network_1.value = $VMNetwork 859 | 860 | $nsxMgrOvfConfig.Common.nsx_role.Value = "nsx-manager" 861 | $nsxMgrOvfConfig.Common.nsx_hostname.Value = $NSXTMgrHostname 862 | $nsxMgrOvfConfig.Common.nsx_ip_0.Value = $NSXTMgrIPAddress 863 | $nsxMgrOvfConfig.Common.nsx_netmask_0.Value = $VMNetmask 864 | $nsxMgrOvfConfig.Common.nsx_gateway_0.Value = $VMGateway 865 | $nsxMgrOvfConfig.Common.nsx_dns1_0.Value = $VMDNS 866 | $nsxMgrOvfConfig.Common.nsx_domain_0.Value = $VMDomain 867 | $nsxMgrOvfConfig.Common.nsx_ntp_0.Value = $VMNTP 868 | 869 | if($NSXSSHEnable -eq "true") { 870 | $NSXSSHEnableVar = $true 871 | } else { 872 | $NSXSSHEnableVar = $false 873 | } 874 | $nsxMgrOvfConfig.Common.nsx_isSSHEnabled.Value = $NSXSSHEnableVar 875 | if($NSXEnableRootLogin -eq "true") { 876 | $NSXRootPasswordVar = $true 877 | } else { 878 | $NSXRootPasswordVar = $false 879 | } 880 | $nsxMgrOvfConfig.Common.nsx_allowSSHRootLogin.Value = $NSXRootPasswordVar 881 | 882 | $nsxMgrOvfConfig.Common.nsx_passwd_0.Value = $NSXRootPassword 883 | $nsxMgrOvfConfig.Common.nsx_cli_username.Value = $NSXAdminUsername 884 | $nsxMgrOvfConfig.Common.nsx_cli_passwd_0.Value = $NSXAdminPassword 885 | $nsxMgrOvfConfig.Common.nsx_cli_audit_username.Value = $NSXAuditUsername 886 | $nsxMgrOvfConfig.Common.nsx_cli_audit_passwd_0.Value = $NSXAuditPassword 887 | 888 | My-Logger "Deploying NSX Manager VM $NSXTMgrDisplayName ..." 889 | $nsxmgr_vm = Import-VApp -Source $NSXTManagerOVA -OvfConfiguration $nsxMgrOvfConfig -Name $NSXTMgrDisplayName -Location $cluster -VMHost $vmhost -Datastore $datastore -DiskStorageFormat thin -Force 890 | 891 | My-Logger "Updating vCPU Count to $NSXTMgrvCPU & vMEM to $NSXTMgrvMEM GB ..." 892 | Set-VM -Server $viConnection -VM $nsxmgr_vm -NumCpu $NSXTMgrvCPU -MemoryGB $NSXTMgrvMEM -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 893 | 894 | My-Logger "Powering On $NSXTMgrDisplayName ..." 895 | $nsxmgr_vm | Start-Vm -RunAsync | Out-Null 896 | 897 | # Deploy Controllers 898 | $nsxCtrOvfConfig = Get-OvfConfiguration $NSXTControllerOVA 899 | $NSXTControllerHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 900 | $VMName = $_.Key 901 | $VMIPAddress = $_.Value 902 | $VMHostname = "$VMName" + "@" + $VMDomain 903 | 904 | $nsxCtrOvfConfig.NetworkMapping.Network_1.value = $VMNetwork 905 | $nsxCtrOvfConfig.Common.nsx_hostname.Value = $VMHostname 906 | $nsxCtrOvfConfig.Common.nsx_ip_0.Value = $VMIPAddress 907 | $nsxCtrOvfConfig.Common.nsx_netmask_0.Value = $VMNetmask 908 | $nsxCtrOvfConfig.Common.nsx_gateway_0.Value = $VMGateway 909 | $nsxCtrOvfConfig.Common.nsx_dns1_0.Value = $VMDNS 910 | $nsxCtrOvfConfig.Common.nsx_domain_0.Value = $VMDomain 911 | $nsxCtrOvfConfig.Common.nsx_ntp_0.Value = $VMNTP 912 | 913 | if($NSXSSHEnable -eq "true") { 914 | $NSXSSHEnableVar = $true 915 | } else { 916 | $NSXSSHEnableVar = $false 917 | } 918 | $nsxCtrOvfConfig.Common.nsx_isSSHEnabled.Value = $NSXSSHEnableVar 919 | if($NSXEnableRootLogin -eq "true") { 920 | $NSXRootPasswordVar = $true 921 | } else { 922 | $NSXRootPasswordVar = $false 923 | } 924 | $nsxCtrOvfConfig.Common.nsx_allowSSHRootLogin.Value = $NSXRootPasswordVar 925 | 926 | $nsxCtrOvfConfig.Common.nsx_passwd_0.Value = $NSXRootPassword 927 | $nsxCtrOvfConfig.Common.nsx_cli_username.Value = $NSXAdminUsername 928 | $nsxCtrOvfConfig.Common.nsx_cli_passwd_0.Value = $NSXAdminPassword 929 | $nsxCtrOvfConfig.Common.nsx_cli_audit_username.Value = $NSXAuditUsername 930 | $nsxCtrOvfConfig.Common.nsx_cli_audit_passwd_0.Value = $NSXAuditPassword 931 | 932 | My-Logger "Deploying NSX Controller VM $VMName ..." 933 | $nsxctr_vm = Import-VApp -Source $NSXTControllerOVA -OvfConfiguration $nsxCtrOvfConfig -Name $VMName -Location $cluster -VMHost $vmhost -Datastore $datastore -DiskStorageFormat thin -Force 934 | 935 | My-Logger "Updating vCPU Count to $NSXTCtrvCPU & vMEM to $NSXTCtrvMEM GB ..." 936 | Set-VM -Server $viConnection -VM $nsxctr_vm -NumCpu $NSXTCtrvCPU -MemoryGB $NSXTCtrvMEM -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 937 | 938 | My-Logger "Powering On $VMName ..." 939 | $nsxctr_vm | Start-Vm -RunAsync | Out-Null 940 | } 941 | 942 | # Deploy Edges 943 | $nsxEdgeOvfConfig = Get-OvfConfiguration $NSXTEdgeOVA 944 | $NSXTEdgeHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 945 | $VMName = $_.Key 946 | $VMIPAddress = $_.Value 947 | $VMHostname = "$VMName" + "@" + $VMDomain 948 | 949 | $nsxEdgeOvfConfig.DeploymentOption.Value = $NSXTMgrDeploymentSize 950 | $nsxEdgeOvfConfig.NetworkMapping.Network_0.value = $VMNetwork 951 | $nsxEdgeOvfConfig.NetworkMapping.Network_1.value = $NSXPrivatePortgroup 952 | $nsxEdgeOvfConfig.NetworkMapping.Network_2.value = $NSXPrivatePortgroup 953 | $nsxEdgeOvfConfig.NetworkMapping.Network_3.value = $NSXPrivatePortgroup 954 | 955 | $nsxEdgeOvfConfig.Common.nsx_hostname.Value = $VMHostname 956 | $nsxEdgeOvfConfig.Common.nsx_ip_0.Value = $VMIPAddress 957 | $nsxEdgeOvfConfig.Common.nsx_netmask_0.Value = $VMNetmask 958 | $nsxEdgeOvfConfig.Common.nsx_gateway_0.Value = $VMGateway 959 | $nsxEdgeOvfConfig.Common.nsx_dns1_0.Value = $VMDNS 960 | $nsxEdgeOvfConfig.Common.nsx_domain_0.Value = $VMDomain 961 | $nsxEdgeOvfConfig.Common.nsx_ntp_0.Value = $VMNTP 962 | 963 | if($NSXSSHEnable -eq "true") { 964 | $NSXSSHEnableVar = $true 965 | } else { 966 | $NSXSSHEnableVar = $false 967 | } 968 | $nsxEdgeOvfConfig.Common.nsx_isSSHEnabled.Value = $NSXSSHEnableVar 969 | if($NSXEnableRootLogin -eq "true") { 970 | $NSXRootPasswordVar = $true 971 | } else { 972 | $NSXRootPasswordVar = $false 973 | } 974 | $nsxEdgeOvfConfig.Common.nsx_allowSSHRootLogin.Value = $NSXRootPasswordVar 975 | 976 | $nsxEdgeOvfConfig.Common.nsx_passwd_0.Value = $NSXRootPassword 977 | $nsxEdgeOvfConfig.Common.nsx_cli_username.Value = $NSXAdminUsername 978 | $nsxEdgeOvfConfig.Common.nsx_cli_passwd_0.Value = $NSXAdminPassword 979 | $nsxEdgeOvfConfig.Common.nsx_cli_audit_username.Value = $NSXAuditUsername 980 | $nsxEdgeOvfConfig.Common.nsx_cli_audit_passwd_0.Value = $NSXAuditPassword 981 | 982 | My-Logger "Deploying NSX Edge VM $NSXTEdgeDisplayName ..." 983 | $nsxedge_vm = Import-VApp -Source $NSXTEdgeOVA -OvfConfiguration $nsxEdgeOvfConfig -Name $VMName -Location $cluster -VMHost $vmhost -Datastore $datastore -DiskStorageFormat thin -Force 984 | 985 | My-Logger "Updating vCPU Count to $NSXTEdgevCPU & vMEM to $NSXTEdgevMEM GB ..." 986 | Set-VM -Server $viConnection -VM $nsxedge_vm -NumCpu $NSXTEdgevCPU -MemoryGB $NSXTEdgevMEM -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 987 | 988 | My-Logger "Reconfiguring Network Adapter 2 to $privateNetwork ..." 989 | $nsxedge_vm | Get-NetworkAdapter -Name "Network adapter 2" | Set-NetworkAdapter -Portgroup $privateNetwork -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 990 | 991 | My-Logger "Reconfiguring Network Adapter 3 to $NSXIntermediateNetwork ..." 992 | $nsxedge_vm | Get-NetworkAdapter -Name "Network adapter 3" | Set-NetworkAdapter -Portgroup $NSXIntermediateNetwork -confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 993 | 994 | My-Logger "Powering On $NSXTEdgeDisplayName ..." 995 | $nsxedge_vm | Start-Vm -RunAsync | Out-Null 996 | } 997 | } 998 | 999 | if($moveVMsIntovApp -eq 1) { 1000 | My-Logger "Creating vApp $VAppName ..." 1001 | $VApp = New-VApp -Name $VAppName -Server $viConnection -Location $cluster 1002 | 1003 | if($deployNestedESXiVMs -eq 1) { 1004 | My-Logger "Moving Nested ESXi VMs into $VAppName vApp ..." 1005 | $NestedESXiHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1006 | $vm = Get-VM -Name $_.Key -Server $viConnection 1007 | Move-VM -VM $vm -Server $viConnection -Destination $VApp -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1008 | } 1009 | } 1010 | 1011 | if($deployOpsManager -eq 1) { 1012 | My-Logger "Moving Ops Manager VM into $VAppName vApp ..." 1013 | $opsMgrVM = Get-VM -Name $OpsManagerDisplayName -Server $viConnection 1014 | Move-VM -VM $opsMgrVM -Server $viConnection -Destination $VApp -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1015 | } 1016 | 1017 | if($DeployNSX -eq 1) { 1018 | $nsxMgrVM = Get-VM -Name $NSXTMgrDisplayName -Server $viConnection 1019 | My-Logger "Moving $NSXTMgrDisplayName into $VAppName vApp ..." 1020 | Move-VM -VM $nsxMgrVM -Server $viConnection -Destination $VApp -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1021 | 1022 | My-Logger "Moving NSX Controller VMs into $VAppName vApp ..." 1023 | $NSXTControllerHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1024 | $nsxCtrVM = Get-VM -Name $_.Key -Server $viConnection 1025 | Move-VM -VM $nsxCtrVM -Server $viConnection -Destination $VApp -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1026 | } 1027 | 1028 | My-Logger "Moving NSX Edge VMs into $VAppName vApp ..." 1029 | $NSXTEdgeHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1030 | $nsxEdgeVM = Get-VM -Name $_.Key -Server $viConnection 1031 | Move-VM -VM $nsxEdgeVM -Server $viConnection -Destination $VApp -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1032 | } 1033 | } 1034 | } 1035 | 1036 | if( ($deployNestedESXiVMs -eq 1) -or ($deployOpsManager -eq 1) -or ($setupNewVC -eq 1) -or ($addESXiHostsToVC -eq 1) -or ($configureVSANDiskGroups -eq 1) -or ($deployNSX -eq 1) -or ($moveVMsIntovApp -eq 1) ) { 1037 | My-Logger "Disconnecting from $VIServer ..." 1038 | Disconnect-VIServer -Server $viConnection -Confirm:$false 1039 | } 1040 | 1041 | if($setupNewVC -eq 1) { 1042 | My-Logger "Connecting to Management vCenter Server $VIServer ..." 1043 | $viConnection = Connect-VIServer $VIServer -User $VIUsername -Password $VIPassword -WarningAction SilentlyContinue 1044 | 1045 | My-Logger "Creating new VSAN Cluster $NewVCVSANClusterName ..." 1046 | New-Cluster -Server $viConnection -Name $NewVCVSANClusterName -Location (Get-Datacenter -Name $RootDatacenterName -Server $viConnection) -DrsEnabled -VsanEnabled -VsanDiskClaimMode 'Manual' | Out-File -Append -LiteralPath $verboseLogFile 1047 | 1048 | if($addESXiHostsToVC -eq 1) { 1049 | $NestedESXiHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1050 | $VMName = $_.Key 1051 | $VMIPAddress = $_.Value 1052 | 1053 | $targetVMHost = $VMIPAddress 1054 | if($addHostByDnsName -eq 1) { 1055 | $targetVMHost = $VMName 1056 | } 1057 | My-Logger "Adding ESXi host $targetVMHost to Cluster ..." 1058 | Add-VMHost -Server $viConnection -Location (Get-Cluster -Name $NewVCVSANClusterName) -User "root" -Password $VMPassword -Name $targetVMHost -Force | Out-File -Append -LiteralPath $verboseLogFile 1059 | } 1060 | } 1061 | 1062 | if($configureVSANDiskGroups -eq 1) { 1063 | My-Logger "Enabling VSAN & disabling VSAN Health Check ..." 1064 | Get-VsanClusterConfiguration -Server $viConnection -Cluster $NewVCVSANClusterName | Set-VsanClusterConfiguration -HealthCheckIntervalMinutes 0 | Out-File -Append -LiteralPath $verboseLogFile 1065 | 1066 | 1067 | foreach ($vmhost in Get-Cluster -Server $viConnection -Name $NewVCVSANClusterName | Get-VMHost) { 1068 | $luns = $vmhost | Get-ScsiLun | select CanonicalName, CapacityGB 1069 | 1070 | My-Logger "Querying ESXi host disks to create VSAN Diskgroups ..." 1071 | foreach ($lun in $luns) { 1072 | if(([int]($lun.CapacityGB)).toString() -eq "$NestedESXiCachingvDisk") { 1073 | $vsanCacheDisk = $lun.CanonicalName 1074 | } 1075 | if(([int]($lun.CapacityGB)).toString() -eq "$NestedESXiCapacityvDisk") { 1076 | $vsanCapacityDisk = $lun.CanonicalName 1077 | } 1078 | } 1079 | My-Logger "Creating VSAN DiskGroup for $vmhost ..." 1080 | New-VsanDiskGroup -Server $viConnection -VMHost $vmhost -SsdCanonicalName $vsanCacheDisk -DataDiskCanonicalName $vsanCapacityDisk | Out-File -Append -LiteralPath $verboseLogFile 1081 | } 1082 | } 1083 | 1084 | # Exit maintanence mode in case patching was done earlier 1085 | foreach ($vmhost in Get-Cluster -Server $viConnection -Name $NewVCVSANClusterName | Get-VMHost) { 1086 | if($vmhost.ConnectionState -eq "Maintenance") { 1087 | Set-VMHost -VMhost $vmhost -State Connected -RunAsync -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile 1088 | } 1089 | } 1090 | 1091 | My-Logger "Disconnecting from management vCenter Server ..." 1092 | Disconnect-VIServer -Server $viConnection -Confirm:$false 1093 | } 1094 | 1095 | if($initialNSXConfig -eq 1) { 1096 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAdminUsername -Password $NSXAdminPassword -WarningAction SilentlyContinue)) { 1097 | Write-Host -ForegroundColor Red "Unable to connect to NSX Manager, please check the deployment" 1098 | exit 1099 | } else { 1100 | My-Logger "Successfully logged into NSX Manager $NSXTMgrHostname ..." 1101 | } 1102 | 1103 | My-Logger "Connecting back to Management vCenter Server $VIServer ..." 1104 | Connect-VIServer $VIServer -User $VIUsername -Password $VIPassword -WarningAction SilentlyContinue | Out-Null 1105 | 1106 | # Retrieve NSX Manager Thumbprint which will be needed later 1107 | My-Logger "Retrieving NSX Manager Thumbprint ..." 1108 | $nsxMgrID = (Get-NsxtService -Name "com.vmware.nsx.cluster.nodes").list().results.id 1109 | $nsxMgrCertThumbprint = (Get-NsxtService -Name "com.vmware.nsx.cluster.nodes").get($nsxMgrID).manager_role.api_listen_addr.certificate_sha256_thumbprint 1110 | 1111 | ### Setup NSX Controllers 1112 | $ctrCount=0 1113 | $firstNSXController = "" 1114 | $nsxControllerCertThumbprint = "" 1115 | $grabbedVCThumbprint = 0 1116 | $NSXTControllerHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1117 | $nsxCtrName = $_.name 1118 | $nsxCtrIp = $_.value 1119 | 1120 | if($ctrCount -eq 0) { 1121 | My-Logger "Configuring NSX Controller $nsxCtrName as control-cluster master ..." 1122 | # Store the first NSX Controller for later use 1123 | $firstNSXController = $nsxCtrName 1124 | 1125 | # Login by passing in admin username 1126 | if($debug) { My-Logger "Sending admin username ..." } 1127 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminUsername -ReturnCarriage $true 1128 | Start-Sleep 2 1129 | 1130 | # Login by passing in admin password 1131 | if($debug) { My-Logger "Sending admin password ..." } 1132 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminPassword -ReturnCarriage $true 1133 | Start-Sleep 5 1134 | 1135 | # Join Controller to NSX Manager 1136 | if($debug) { My-Logger "Sending join management plane command ..." } 1137 | $joinMgmtCmd1 = "join management-plane $NSXTMgrIPAddress username $NSXAdminUsername thumbprint $nsxMgrCertThumbprint" 1138 | $joinMgmtCmd2 = "$NSXAdminPassword" 1139 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $joinMgmtCmd1 -ReturnCarriage $true 1140 | Start-Sleep 5 1141 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $joinMgmtCmd2 -ReturnCarriage $true 1142 | Start-Sleep 25 1143 | 1144 | # Setup shared secret 1145 | if($debug) { My-Logger "Sending shared secret command ..." } 1146 | $sharedSecretCmd = "set control-cluster security-model shared-secret secret $NSXControllerSharedSecret" 1147 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $sharedSecretCmd -ReturnCarriage $true 1148 | Start-Sleep 5 1149 | 1150 | # Initialize NSX Controller Cluster 1151 | if($debug) { My-Logger "Sending control cluster init command ..." } 1152 | $initCmd = "initialize control-cluster" 1153 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $initCmd -ReturnCarriage $true 1154 | Start-Sleep 25 1155 | } else { 1156 | My-Logger "Configuring additional NSX Controller $nsxCtrName ..." 1157 | 1158 | # Login by passing in admin username 1159 | if($debug) { My-Logger "Sending admin username ..." } 1160 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminUsername -ReturnCarriage $true 1161 | Start-Sleep 2 1162 | 1163 | # Login by passing in admin password 1164 | if($debug) { My-Logger "Sending admin password ..." } 1165 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminPassword -ReturnCarriage $true 1166 | Start-Sleep 5 1167 | 1168 | # Join Controller to NSX Manager 1169 | if($debug) { My-Logger "Sending join management plane command ..." } 1170 | $joinMgmtCmd1 = "join management-plane $NSXTMgrIPAddress username $NSXAdminUsername thumbprint $nsxMgrCertThumbprint" 1171 | $joinMgmtCmd2 = "$NSXAdminPassword" 1172 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $joinMgmtCmd1 -ReturnCarriage $true 1173 | Start-Sleep 5 1174 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $joinMgmtCmd2 -ReturnCarriage $true 1175 | Start-Sleep 25 1176 | 1177 | # Setup shared secret 1178 | if($debug) { My-Logger "Sending shared secret command ..." } 1179 | $sharedSecretCmd = "set control-cluster security-model shared-secret secret $NSXControllerSharedSecret" 1180 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $sharedSecretCmd -ReturnCarriage $true 1181 | Start-Sleep 5 1182 | 1183 | ### --- (stupid hack because we don't have an API) --- ### 1184 | # Exit from nsxcli 1185 | if($debug) { My-Logger "Sending exit command ..." } 1186 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput "exit" -ReturnCarriage $true 1187 | Start-Sleep 10 1188 | 1189 | # Login using root 1190 | if($debug) { My-Logger "Sending root username ..." } 1191 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput "root" -ReturnCarriage $true 1192 | Start-Sleep 2 1193 | 1194 | # Login by passing in root password 1195 | if($debug) { My-Logger "Sending root password ..." } 1196 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXRootPassword -ReturnCarriage $true 1197 | Start-Sleep 10 1198 | 1199 | # Retrieve VC SHA256 Thumbprint by running openssl in the shell and 1200 | # storing the thumbprint to a file which we will download later 1201 | # only need to do this once 1202 | if($grabbedVCThumbprint -eq 0) { 1203 | if($debug) { My-Logger "Sending openssl to get VC Thumbprint ..." } 1204 | $vcThumbprintCmd = "echo -n | openssl s_client -connect ${VIServer}:443 2>/dev/null | openssl x509 -noout -fingerprint -sha256 | awk -F `'=`' `'{print `$2}`' > /tmp/vc-thumbprint" 1205 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $vcThumbprintCmd -ReturnCarriage $true 1206 | Start-Sleep 25 1207 | 1208 | if($debug) { My-Logger "Processing certificate thumbprint ..." } 1209 | Copy-VMGuestFile -vm (Get-VM -Name $nsxCtrName) -GuestToLocal -GuestUser "root" -GuestPassword $NSXRootPassword -Source /tmp/vc-thumbprint -Destination $DestinationVCThumbprintStore | Out-Null 1210 | $vcCertThumbprint = Get-Content -Path $DestinationVCThumbprintStore 1211 | 1212 | $grabbedVCThumbprint = 1 1213 | } 1214 | 1215 | # Retrieve Control Cluster Thumbprint by running nsxcli in the shell and 1216 | # storing the thumbprint to a file which we will download later 1217 | if($debug) { My-Logger "Sending get control cluster cert ..." } 1218 | $ctrClusterThumbprintCmd = "nsxcli -c `"get control-cluster certificate thumbprint`" > /tmp/controller-thumbprint" 1219 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $ctrClusterThumbprintCmd -ReturnCarriage $true 1220 | Start-Sleep 25 1221 | 1222 | Copy-VMGuestFile -vm (Get-VM -Name $nsxCtrName) -GuestToLocal -GuestUser "root" -GuestPassword $NSXRootPassword -Source /tmp/controller-thumbprint -Destination $DestinationCtrThumprintStore | Out-Null 1223 | $nsxControllerCertThumbprint = Get-Content -Path $DestinationCtrThumprintStore | ? {$_.trim() -ne "" } 1224 | 1225 | # Exit from shell 1226 | if($debug) { My-Logger "Sending exit command ..." } 1227 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput "exit" -ReturnCarriage $true 1228 | Start-Sleep 10 1229 | ### --- (stupid hack because we don't have an API) --- ### 1230 | 1231 | # Login by passing in admin username 1232 | if($debug) { My-Logger "Sending admin username ..." } 1233 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminUsername -ReturnCarriage $true 1234 | Start-Sleep 2 1235 | 1236 | # Login by passing in admin password 1237 | if($debug) { My-Logger "Sending admin password ..." } 1238 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $NSXAdminPassword -ReturnCarriage $true 1239 | Start-Sleep 5 1240 | 1241 | # Join NSX Controller to NSX Controller Cluster 1242 | if($debug) { My-Logger "Sending join control cluster command ..." } 1243 | $joinCtrCmd = "join control-cluster $nsxCtrIp thumbprint $nsxControllerCertThumbprint" 1244 | Set-VMKeystrokes -VMName $firstNSXController -StringInput $joinCtrCmd -ReturnCarriage $true 1245 | Start-Sleep 30 1246 | 1247 | # Activate NSX Controller 1248 | if($debug) { My-Logger "Sending control cluster activate command ..." } 1249 | $initCmd = "activate control-cluster" 1250 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput $initCmd -ReturnCarriage $true 1251 | Start-Sleep 30 1252 | 1253 | # Exit Console 1254 | if($debug) { My-Logger "Sending final exit ..." } 1255 | Set-VMKeystrokes -VMName $nsxCtrName -StringInput "exit" -ReturnCarriage $true 1256 | } 1257 | $ctrCount++ 1258 | } 1259 | 1260 | ### Setup NSX Edges 1261 | $NSXTEdgeHostnameToIPs.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1262 | $nsxEdgeName = $_.name 1263 | $nsxEdgeIp = $_.value 1264 | 1265 | My-Logger "Configuring NSX Edge $nsxEdgeName ..." 1266 | 1267 | # Login by passing in admin username 1268 | if($debug) { My-Logger "Sending admin username ..." } 1269 | Set-VMKeystrokes -VMName $nsxEdgeName -StringInput $NSXAdminUsername -ReturnCarriage $true 1270 | Start-Sleep 2 1271 | 1272 | # Login by passing in admin password 1273 | if($debug) { My-Logger "Sending admin password ..." } 1274 | Set-VMKeystrokes -VMName $nsxEdgeName -StringInput $NSXAdminPassword -ReturnCarriage $true 1275 | Start-Sleep 5 1276 | 1277 | # Join NSX Edge to NSX Manager 1278 | if($debug) { My-Logger "Sending join management plane command ..." } 1279 | $joinMgmtCmd1 = "join management-plane $NSXTMgrIPAddress username $NSXAdminUsername thumbprint $nsxMgrCertThumbprint" 1280 | $joinMgmtCmd2 = "$NSXAdminPassword" 1281 | Set-VMKeystrokes -VMName $nsxEdgeName -StringInput $joinMgmtCmd1 -ReturnCarriage $true 1282 | Start-Sleep 5 1283 | Set-VMKeystrokes -VMName $nsxEdgeName -StringInput $joinMgmtCmd2 -ReturnCarriage $true 1284 | Start-Sleep 20 1285 | 1286 | # Exit Console 1287 | if($debug) { My-Logger "Sending final exit ..." } 1288 | Set-VMKeystrokes -VMName $nsxEdgeName -StringInput "exit" -ReturnCarriage $true 1289 | } 1290 | 1291 | # Exit Console for first NSX Controller 1292 | if($debug) { My-Logger "Sending final exit to initial Controller ..." } 1293 | Set-VMKeystrokes -VMName $firstNSXController -StringInput "exit" -ReturnCarriage $true 1294 | 1295 | My-Logger "Disconnecting from NSX Manager ..." 1296 | Disconnect-NsxtServer -Confirm:$false 1297 | 1298 | My-Logger "Disconnecting from Management vCenter ..." 1299 | Disconnect-VIServer * -Confirm:$false 1300 | } 1301 | 1302 | if($postDeployNSXConfig -eq 1) { 1303 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAdminUsername -Password $NSXAdminPassword -WarningAction SilentlyContinue)) { 1304 | Write-Host -ForegroundColor Red "Unable to connect to NSX Manager, please check the deployment" 1305 | exit 1306 | } else { 1307 | My-Logger "Successfully logged into NSX Manager $NSXTMgrHostname ..." 1308 | } 1309 | 1310 | $runHealth=$true 1311 | $runEULA=$true 1312 | $runIPPool=$true 1313 | $runIPBlock=$true 1314 | $runTransportZone=$true 1315 | $runAddVC=$true 1316 | $runLogicalSwitch=$true 1317 | $runHostPrep=$true 1318 | $runUplinkProfile=$true 1319 | $runAddESXiTransportNode=$true 1320 | $runAddEdgeTransportNode=$true 1321 | $runAddEdgeCluster=$true 1322 | $runT0Router = $true 1323 | $runT0RouterPort = $true 1324 | $runT0StaticRoute = $true 1325 | $runT1Router = $true 1326 | $runT1RouterPort = $true 1327 | $runT1RouterAdvertisement = $true 1328 | 1329 | ### Verify Health for all Nodes 1330 | if($runHealth) { 1331 | My-Logger "Verifying health of all NSX Manager/Controller Nodes ..." 1332 | $clusterNodeService = Get-NsxtService -Name "com.vmware.nsx.cluster.nodes" 1333 | $clusterNodeStatusService = Get-NsxtService -Name "com.vmware.nsx.cluster.nodes.status" 1334 | $nodes = $clusterNodeService.list().results 1335 | $mgmtNodes = $nodes | where { $_.controller_role -eq $null } 1336 | $controllerNodes = $nodes | where { $_.manager_role -eq $null } 1337 | 1338 | foreach ($mgmtNode in $mgmtNodes) { 1339 | $mgmtNodeId = $mgmtNode.id 1340 | $mgmtNodeName = $mgmtNode.appliance_mgmt_listen_addr 1341 | 1342 | if($debug) { My-Logger "Check health status of Mgmt Node $mgmtNodeName ..." } 1343 | while ( $clusterNodeStatusService.get($mgmtNodeId).mgmt_cluster_status.mgmt_cluster_status -ne "CONNECTED") { 1344 | if($debug) { My-Logger "$mgmtNodeName is not ready, sleeping 20 seconds ..." } 1345 | Start-Sleep 20 1346 | } 1347 | } 1348 | 1349 | foreach ($controllerNode in $controllerNodes) { 1350 | $controllerNodeId = $controllerNode.id 1351 | $controllerNodeName = $controllerNode.controller_role.control_plane_listen_addr.ip_address 1352 | 1353 | if($debug) { My-Logger "Checking health of Ctrl Node $controllerNodeName ..." } 1354 | while ( $clusterNodeStatusService.get($controllerNodeId).control_cluster_status.control_cluster_status -ne "CONNECTED") { 1355 | if($debug) { My-Logger "$controllerNodeName is not ready, sleeping 20 seconds ..." } 1356 | Start-Sleep 20 1357 | } 1358 | } 1359 | } 1360 | 1361 | if($runEULA) { 1362 | My-Logger "Accepting NSX Manager EULA ..." 1363 | $eulaService = Get-NsxtService -Name "com.vmware.nsx.eula.accept" 1364 | $eulaService.create() 1365 | } 1366 | 1367 | if($runIPPool) { 1368 | My-Logger "Creating Tunnel Endpoint IP Pool for ESXi ..." 1369 | $ipPoolService = Get-NsxtService -Name "com.vmware.nsx.pools.ip_pools" 1370 | $ipPoolSpec = $ipPoolService.help.create.ip_pool.Create() 1371 | $subNetSpec = $ipPoolService.help.create.ip_pool.subnets.Element.Create() 1372 | $allocationRangeSpec = $ipPoolService.help.create.ip_pool.subnets.Element.allocation_ranges.Element.Create() 1373 | 1374 | $allocationRangeSpec.start = $TunnelEndpointPoolIPRangeStart 1375 | $allocationRangeSpec.end = $TunnelEndpointPoolIPRangeEnd 1376 | $addResult = $subNetSpec.allocation_ranges.Add($allocationRangeSpec) 1377 | $subNetSpec.cidr = $TunnelEndpointPoolCIDR 1378 | $subNetSpec.gateway_ip = $TunnelEndpointPoolGateway 1379 | $ipPoolSpec.display_name = $TunnelEndpointPoolName 1380 | $ipPoolSpec.description = $TunnelEndpointPoolDescription 1381 | $addResult = $ipPoolSpec.subnets.Add($subNetSpec) 1382 | $ipPool = $ipPoolService.create($ipPoolSpec) 1383 | 1384 | My-Logger "Creating Load Balancer IP Pool for K8S ..." 1385 | $ipPoolService = Get-NsxtService -Name "com.vmware.nsx.pools.ip_pools" 1386 | $ipPoolSpec = $ipPoolService.help.create.ip_pool.Create() 1387 | $subNetSpec = $ipPoolService.help.create.ip_pool.subnets.Element.Create() 1388 | $allocationRangeSpec = $ipPoolService.help.create.ip_pool.subnets.Element.allocation_ranges.Element.Create() 1389 | 1390 | $allocationRangeSpec.start = $LoadBalancerPoolIPRangeStart 1391 | $allocationRangeSpec.end = $LoadBalancerPoolIPRangeEnd 1392 | $addResult = $subNetSpec.allocation_ranges.Add($allocationRangeSpec) 1393 | $subNetSpec.cidr = $LoadBalancerPoolCIDR 1394 | $ipPoolSpec.display_name = $LoadBalancerPoolName 1395 | $ipPoolSpec.description = $LoadBalancerPoolDescription 1396 | $addResult = $ipPoolSpec.subnets.Add($subNetSpec) 1397 | $ipPool = $ipPoolService.create($ipPoolSpec) 1398 | } 1399 | 1400 | if($runIPBlock) { 1401 | My-Logger "Creating PKS IP Block ..." 1402 | $ipBlockService = Get-NsxtService -Name "com.vmware.nsx.pools.ip_blocks" 1403 | $ipBlockSpec = $ipBlockService.Help.create.ip_block.Create() 1404 | $ipBlockSpec.display_name = $ipBlockName 1405 | $ipBlockSpec.cidr = $ipBlockNetwork 1406 | $ipBlockAdd = $ipBlockService.create($ipBlockSpec) 1407 | } 1408 | 1409 | if($runTransportZone) { 1410 | My-Logger "Creating Overlay & VLAN Transport Zones ..." 1411 | $transportZoneService = Get-NsxtService -Name "com.vmware.nsx.transport_zones" 1412 | $overlayTZSpec = $transportZoneService.help.create.transport_zone.Create() 1413 | $overlayTZSpec.display_name = $OverlayTransportZoneName 1414 | $overlayTZSpec.host_switch_name = $OverlayTransportZoneHostSwitchName 1415 | $overlayTZSpec.transport_type = "OVERLAY" 1416 | $overlayTZ = $transportZoneService.create($overlayTZSpec) 1417 | 1418 | $vlanTZSpec = $transportZoneService.help.create.transport_zone.Create() 1419 | $vlanTZSpec.display_name = $VLANTransportZoneName 1420 | $vlanTZSpec.host_switch_name = $VlanTransportZoneNameHostSwitchName 1421 | $vlanTZSpec.transport_type = "VLAN" 1422 | $vlanTZ = $transportZoneService.create($vlanTZSpec) 1423 | } 1424 | 1425 | if($runAddVC) { 1426 | My-Logger "Adding vCenter Server Compute Manager ..." 1427 | $computeManagerSerivce = Get-NsxtService -Name "com.vmware.nsx.fabric.compute_managers" 1428 | $computeManagerStatusService = Get-NsxtService -Name "com.vmware.nsx.fabric.compute_managers.status" 1429 | 1430 | $computeManagerSpec = $computeManagerSerivce.help.create.compute_manager.Create() 1431 | $credentialSpec = $computeManagerSerivce.help.create.compute_manager.credential.username_password_login_credential.Create() 1432 | $credentialSpec.username = $VIUsername 1433 | $credentialSpec.password = $VIPassword 1434 | $credentialSpec.thumbprint = $vcCertThumbprint 1435 | $computeManagerSpec.server = $VIServer 1436 | $computeManagerSpec.origin_type = "vCenter" 1437 | $computeManagerSpec.display_name = $VIServer 1438 | $computeManagerSpec.credential = $credentialSpec 1439 | $computeManagerResult = $computeManagerSerivce.create($computeManagerSpec) 1440 | 1441 | if($debug) { My-Logger "Waiting for VC registration to complete ..." } 1442 | while ( $computeManagerStatusService.get($computeManagerResult.id).registration_status -ne "REGISTERED") { 1443 | if($debug) { My-Logger "$VIServer is not ready, sleeping 30 seconds ..." } 1444 | Start-Sleep 30 1445 | } 1446 | } 1447 | 1448 | if($runLogicalSwitch) { 1449 | $transportZoneService = Get-NsxtService -Name "com.vmware.nsx.transport_zones" 1450 | $overlayTZ = $transportZoneService.list().results | where { $_.display_name -eq $OverlayTransportZoneName } 1451 | $vlanTZ = $transportZoneService.list().results | where { $_.display_name -eq $VlanTransportZoneName } 1452 | 1453 | My-Logger "Adding Logical Switch for K8S Management Cluster ..." 1454 | $logicalSwitchService = Get-NsxtService -Name "com.vmware.nsx.logical_switches" 1455 | $logicalSwitchSpec = $logicalSwitchService.help.create.logical_switch.Create() 1456 | $logicalSwitchSpec.display_name = $K8SMgmtClusterLogicalSwitchName 1457 | $logicalSwitchSpec.admin_state = "UP" 1458 | $logicalSwitchSpec.replication_mode = $K8SMgmtClusterLogicalSwitchReplicationMode 1459 | $logicalSwitchSpec.transport_zone_id = $overlayTZ.id 1460 | $uplinkLogicalSwitch = $logicalSwitchService.create($logicalSwitchSpec) 1461 | 1462 | My-Logger "Adding Logical Switch for Uplink ..." 1463 | $logicalSwitchService = Get-NsxtService -Name "com.vmware.nsx.logical_switches" 1464 | $logicalSwitchSpec = $logicalSwitchService.help.create.logical_switch.Create() 1465 | $logicalSwitchSpec.display_name = $UplinkLogicalSwitchName 1466 | $logicalSwitchSpec.admin_state = "UP" 1467 | $logicalSwitchSpec.vlan = $UplinkLogicalSwitchVlan 1468 | $logicalSwitchSpec.transport_zone_id = $vlanTZ.id 1469 | $uplinkLogicalSwitch = $logicalSwitchService.create($logicalSwitchSpec) 1470 | } 1471 | 1472 | if($runHostPrep) { 1473 | My-Logger "Preparing ESXi hosts & Installing NSX VIBs ..." 1474 | $computeCollectionService = Get-NsxtService -Name "com.vmware.nsx.fabric.compute_collections" 1475 | $computeId = $computeCollectionService.list().results[0].external_id 1476 | 1477 | $computeCollectionFabricTemplateService = Get-NsxtService -Name "com.vmware.nsx.fabric.compute_collection_fabric_templates" 1478 | $computeFabricTemplateSpec = $computeCollectionFabricTemplateService.help.create.compute_collection_fabric_template.Create() 1479 | $computeFabricTemplateSpec.auto_install_nsx = $true 1480 | $computeFabricTemplateSpec.compute_collection_id = $computeId 1481 | $computeCollectionFabric = $computeCollectionFabricTemplateService.create($computeFabricTemplateSpec) 1482 | 1483 | My-Logger "Waiting for ESXi hosts to finish host prep ..." 1484 | $fabricNodes = (Get-NsxtService -Name "com.vmware.nsx.fabric.nodes").list().results | where { $_.resource_type -eq "HostNode" } 1485 | foreach ($fabricNode in $fabricNodes) { 1486 | $fabricNodeName = $fabricNode.display_name 1487 | while ((Get-NsxtService -Name "com.vmware.nsx.fabric.nodes.status").get($fabricNode.external_id).host_node_deployment_status -ne "INSTALL_SUCCESSFUL") { 1488 | if($debug) { My-Logger "ESXi hosts are still being prepped, sleeping for 30 seconds ..." } 1489 | Start-Sleep 30 1490 | } 1491 | } 1492 | } 1493 | 1494 | if($runUplinkProfile) { 1495 | $hostSwitchProfileService = Get-NsxtService -Name "com.vmware.nsx.host_switch_profiles" 1496 | 1497 | My-Logger "Creating ESXi Uplink Profile ..." 1498 | $ESXiUplinkProfileSpec = $hostSwitchProfileService.help.create.base_host_switch_profile.uplink_host_switch_profile.Create() 1499 | $activeUplinkSpec = $hostSwitchProfileService.help.create.base_host_switch_profile.uplink_host_switch_profile.teaming.active_list.Element.Create() 1500 | $activeUplinkSpec.uplink_name = $ESXiUplinkProfileActivepNIC 1501 | $activeUplinkSpec.uplink_type = "PNIC" 1502 | $ESXiUplinkProfileSpec.display_name = $ESXiUplinkProfileName 1503 | $ESXiUplinkProfileSpec.mtu = $ESXiUplinkProfileMTU 1504 | $ESXiUplinkProfileSpec.transport_vlan = $ESXiUplinkProfileTransportVLAN 1505 | $addActiveUplink = $ESXiUplinkProfileSpec.teaming.active_list.Add($activeUplinkSpec) 1506 | $ESXiUplinkProfileSpec.teaming.policy = $ESXiUplinkProfilePolicy 1507 | $ESXiUplinkProfile = $hostSwitchProfileService.create($ESXiUplinkProfileSpec) 1508 | 1509 | My-Logger "Creating Edge Uplink Profile ..." 1510 | $EdgeUplinkProfileSpec = $hostSwitchProfileService.help.create.base_host_switch_profile.uplink_host_switch_profile.Create() 1511 | $activeUplinkSpec = $hostSwitchProfileService.help.create.base_host_switch_profile.uplink_host_switch_profile.teaming.active_list.Element.Create() 1512 | $activeUplinkSpec.uplink_name = $EdgeUplinkProfileActivepNIC 1513 | $activeUplinkSpec.uplink_type = "PNIC" 1514 | $EdgeUplinkProfileSpec.display_name = $EdgeUplinkProfileName 1515 | $EdgeUplinkProfileSpec.mtu = $EdgeUplinkProfileMTU 1516 | $EdgeUplinkProfileSpec.transport_vlan = $EdgeUplinkProfileTransportVLAN 1517 | $addActiveUplink = $EdgeUplinkProfileSpec.teaming.active_list.Add($activeUplinkSpec) 1518 | $EdgeUplinkProfileSpec.teaming.policy = $EdgeUplinkProfilePolicy 1519 | $EdgeUplinkProfile = $hostSwitchProfileService.create($EdgeUplinkProfileSpec) 1520 | } 1521 | 1522 | if($runAddESXiTransportNode) { 1523 | $transportNodeService = Get-NsxtService -Name "com.vmware.nsx.transport_nodes" 1524 | $transportNodeStateService = Get-NsxtService -Name "com.vmware.nsx.transport_nodes.state" 1525 | 1526 | # Retrieve all ESXi Host Nodes 1527 | $hostNodes = (Get-NsxtService -Name "com.vmware.nsx.fabric.nodes").list().results | where { $_.resource_type -eq "HostNode" } 1528 | $ESXiUplinkProfile = (Get-NsxtService -Name "com.vmware.nsx.host_switch_profiles").list().results | where { $_.display_name -eq $ESXiUplinkProfileName} 1529 | $ipPool = (Get-NsxtService -Name "com.vmware.nsx.pools.ip_pools").list().results | where { $_.display_name -eq $TunnelEndpointPoolName } 1530 | $overlayTransportZone = (Get-NsxtService -Name "com.vmware.nsx.transport_zones").list().results | where { $_.transport_type -eq "OVERLAY" } 1531 | 1532 | foreach ($hostNode in $hostNodes) { 1533 | $hostNodeName = $hostNode.display_name 1534 | My-Logger "Adding $hostNodeName Transport Node ..." 1535 | 1536 | # Create all required empty specs 1537 | $transportNodeSpec = $transportNodeService.help.create.transport_node.Create() 1538 | $hostSwitchSpec = $transportNodeService.help.create.transport_node.host_switches.Element.Create() 1539 | $hostSwitchProfileSpec = $transportNodeService.help.create.transport_node.host_switches.Element.host_switch_profile_ids.Element.Create() 1540 | $pnicSpec = $transportNodeService.help.create.transport_node.host_switches.Element.pnics.Element.Create() 1541 | $transportZoneEPSpec = $transportNodeService.help.create.transport_node.transport_zone_endpoints.Element.Create() 1542 | 1543 | $transportNodeSpec.display_name = $hostNodeName 1544 | $hostSwitchSpec.host_switch_name = $OverlayTransportZoneHostSwitchName 1545 | $hostSwitchProfileSpec.key = "UplinkHostSwitchProfile" 1546 | $hostSwitchProfileSpec.value = $ESXiUplinkProfile.id 1547 | $pnicSpec.device_name = $ESXiUplinkProfileActivepNIC 1548 | $pnicSpec.uplink_name = $ESXiUplinkProfileActivepNIC 1549 | $hostSwitchSpec.static_ip_pool_id = $ipPool.id 1550 | $pnicAddResult = $hostSwitchSpec.pnics.Add($pnicSpec) 1551 | $switchProfileAddResult = $hostSwitchSpec.host_switch_profile_ids.Add($hostSwitchProfileSpec) 1552 | $switchAddResult = $transportNodeSpec.host_switches.Add($hostSwitchSpec) 1553 | $transportZoneEPSpec.transport_zone_id = $overlayTransportZone.id 1554 | $transportZoneAddResult = $transportNodeSpec.transport_zone_endpoints.Add($transportZoneEPSpec) 1555 | $transportNodeSpec.node_id = $hostNode.id 1556 | $transportNode = $transportNodeService.create($transportNodeSpec) 1557 | 1558 | My-Logger "Waiting for transport node configurations to complete ..." 1559 | while ($transportNodeStateService.get($transportNode.id).state -ne "success") { 1560 | if($debug) { My-Logger "ESXi transport node still being configured, sleeping for 30 seconds ..." } 1561 | Start-Sleep 30 1562 | } 1563 | } 1564 | } 1565 | 1566 | if($runAddEdgeTransportNode) { 1567 | $transportNodeService = Get-NsxtService -Name "com.vmware.nsx.transport_nodes" 1568 | $transportNodeStateService = Get-NsxtService -Name "com.vmware.nsx.transport_nodes.state" 1569 | 1570 | # Retrieve all Edge Host Nodes 1571 | $edgeNodes = (Get-NsxtService -Name "com.vmware.nsx.fabric.nodes").list().results | where { $_.resource_type -eq "EdgeNode" } 1572 | $EdgeUplinkProfile = (Get-NsxtService -Name "com.vmware.nsx.host_switch_profiles").list().results | where { $_.display_name -eq $EdgeUplinkProfileName} 1573 | $ipPool = (Get-NsxtService -Name "com.vmware.nsx.pools.ip_pools").list().results | where { $_.display_name -eq $TunnelEndpointPoolName } 1574 | $overlayTransportZone = (Get-NsxtService -Name "com.vmware.nsx.transport_zones").list().results | where { $_.transport_type -eq "OVERLAY" } 1575 | $vlanTransportZone = (Get-NsxtService -Name "com.vmware.nsx.transport_zones").list().results | where { $_.transport_type -eq "VLAN" } 1576 | 1577 | foreach ($edgeNode in $edgeNodes) { 1578 | $edgeNodeName = $edgeNode.display_name 1579 | My-Logger "Adding $edgeNodeName Edge Transport Node ..." 1580 | 1581 | # Create all required empty specs 1582 | $transportNodeSpec = $transportNodeService.help.create.transport_node.Create() 1583 | $hostSwitchOverlaySpec = $transportNodeService.help.create.transport_node.host_switches.Element.Create() 1584 | $hostSwitchVlanSpec = $transportNodeService.help.create.transport_node.host_switches.Element.Create() 1585 | $hostSwitchProfileSpec = $transportNodeService.help.create.transport_node.host_switches.Element.host_switch_profile_ids.Element.Create() 1586 | $pnicOverlaySpec = $transportNodeService.help.create.transport_node.host_switches.Element.pnics.Element.Create() 1587 | $pnicVlanSpec = $transportNodeService.help.create.transport_node.host_switches.Element.pnics.Element.Create() 1588 | $transportZoneEPOverlaySpec = $transportNodeService.help.create.transport_node.transport_zone_endpoints.Element.Create() 1589 | $transportZoneEPVlanSpec = $transportNodeService.help.create.transport_node.transport_zone_endpoints.Element.Create() 1590 | 1591 | $transportNodeSpec.display_name = $edgeNodeName 1592 | 1593 | $hostSwitchOverlaySpec.host_switch_name = $OverlayTransportZoneHostSwitchName 1594 | $hostSwitchProfileSpec.key = "UplinkHostSwitchProfile" 1595 | $hostSwitchProfileSpec.value = $EdgeUplinkProfile.id 1596 | $pnicOverlaySpec.device_name = $EdgeUplinkProfileOverlayvNIC 1597 | $pnicOverlaySpec.uplink_name = $EdgeUplinkProfileActivepNIC 1598 | $hostSwitchOverlaySpec.static_ip_pool_id = $ipPool.id 1599 | $pnicAddResult = $hostSwitchOverlaySpec.pnics.Add($pnicOverlaySpec) 1600 | $switchProfileAddResult = $hostSwitchOverlaySpec.host_switch_profile_ids.Add($hostSwitchProfileSpec) 1601 | $switchAddResult = $transportNodeSpec.host_switches.Add($hostSwitchOverlaySpec) 1602 | 1603 | $hostSwitchVlanSpec.host_switch_name = $VlanTransportZoneNameHostSwitchName 1604 | $hostSwitchProfileSpec.key = "UplinkHostSwitchProfile" 1605 | $hostSwitchProfileSpec.value = $EdgeUplinkProfile.id 1606 | $pnicVlanSpec.device_name = $EdgeUplinkProfileVlanvNIC 1607 | $pnicVlanSpec.uplink_name = $EdgeUplinkProfileActivepNIC 1608 | $pnicAddResult = $hostSwitchVlanSpec.pnics.Add($pnicVlanSpec) 1609 | $switchProfileAddResult = $hostSwitchVlanSpec.host_switch_profile_ids.Add($hostSwitchProfileSpec) 1610 | $switchAddResult = $transportNodeSpec.host_switches.Add($hostSwitchVlanSpec) 1611 | 1612 | $transportZoneEPOverlaySpec.transport_zone_id = $overlayTransportZone.id 1613 | $transportZoneAddResult = $transportNodeSpec.transport_zone_endpoints.Add($transportZoneEPOverlaySpec) 1614 | 1615 | $transportZoneEPVlanSpec.transport_zone_id = $vlanTransportZone.id 1616 | $transportZoneAddResult = $transportNodeSpec.transport_zone_endpoints.Add($transportZoneEPVlanSpec) 1617 | 1618 | $transportNodeSpec.node_id = $edgeNode.id 1619 | $transportNode = $transportNodeService.create($transportNodeSpec) 1620 | 1621 | My-Logger "Waiting for transport node configurations to complete ..." 1622 | while ($transportNodeStateService.get($transportNode.id).state -ne "success") { 1623 | if($debug) { My-Logger "ESXi transport node still being configured, sleeping for 30 seconds ..." } 1624 | Start-Sleep 30 1625 | } 1626 | } 1627 | } 1628 | 1629 | if($runAddEdgeCluster) { 1630 | $edgeNodes = (Get-NsxtService -Name "com.vmware.nsx.fabric.nodes").list().results | where { $_.resource_type -eq "EdgeNode" } 1631 | $edgeClusterService = Get-NsxtService -Name "com.vmware.nsx.edge_clusters" 1632 | $edgeNodeMembersSpec = $edgeClusterService.help.create.edge_cluster.members.Create() 1633 | 1634 | My-Logger "Creating Edge Cluster $EdgeClusterName and adding Edge Hosts ..." 1635 | 1636 | foreach ($edgeNode in $edgeNodes) { 1637 | $edgeNodeName = $edgeNode.display_name 1638 | $edgeNodeMemberSpec = $edgeClusterService.help.create.edge_cluster.members.Element.Create() 1639 | $edgeNodeMemberSpec.transport_node_id = $edgeNode.id 1640 | $edgeNodeMemberAddResult = $edgeNodeMembersSpec.Add($edgeNodeMemberSpec) 1641 | } 1642 | 1643 | $edgeClusterSpec = $edgeClusterService.help.create.edge_cluster.Create() 1644 | $edgeClusterSpec.display_name = $EdgeClusterName 1645 | $edgeClusterSpec.members = $edgeNodeMembersSpec 1646 | $edgeCluster = $edgeClusterService.Create($edgeClusterSpec) 1647 | } 1648 | 1649 | if($runT0Router) { 1650 | My-Logger "Creating T0 Router $T0LogicalRouterName ..." 1651 | # T0-LR 1652 | $edgeClusterService = Get-NsxtService -Name "com.vmware.nsx.edge_clusters" 1653 | $edgeCluster = $edgeClusterService.list().results | where { $_.display_name -eq $EdgeClusterName } 1654 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1655 | $lrSpec = $logicalRouterService.Help.create.logical_router.Create() 1656 | $lrSpec.display_name = $T0LogicalRouterName 1657 | $lrSpec.router_type = "TIER0" 1658 | $lrSpec.high_availability_mode = $T0LogicalRouterHAMode 1659 | $lrSpec.edge_cluster_id = $edgeCluster.id 1660 | $lrAdd = $logicalRouterService.create($lrSpec) 1661 | } 1662 | 1663 | if($runT0RouterPort) { 1664 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1665 | $lr = $logicalRouterService.list().results | where { $_.display_name -eq $T0LogicalRouterName } 1666 | $logicalSwitchService = Get-NsxtService -Name "com.vmware.nsx.logical_switches" 1667 | $ls = $logicalSwitchService.list().results | where { $_.display_name -eq $UplinkLogicalSwitchName} 1668 | 1669 | # Must add Logical Switch Port before adding Logical Router Port (gah) 1670 | My-Logger "Creating Logical Switch Port $T0UplinkRouterPortSwitchPortName ..." 1671 | # Uplink-1-Port 1672 | $logicalPortSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_ports" 1673 | $portSpec = $logicalPortSerivce.Help.create.logical_port.Create() 1674 | $portSpec.display_name = $T0UplinkRouterPortSwitchPortName 1675 | $portSpec.admin_state = "UP" 1676 | $portSpec.logical_switch_id = $ls.id 1677 | $logicalSwitchPort = $logicalPortSerivce.create($portSpec) 1678 | 1679 | My-Logger "Creating Logical Router Port $T0UplinkRouterPortName ..." 1680 | # Uplink-1 1681 | $logicalRouterPortSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_router_ports" 1682 | $lrPortSpec = $logicalRouterPortSerivce.help.create.logical_router_port.logical_router_up_link_port.Create() 1683 | $subnetSpec = $logicalRouterPortSerivce.help.create.logical_router_port.logical_router_up_link_port.subnets.Element.Create() 1684 | $memberIndex = $logicalRouterPortSerivce.help.create.logical_router_port.logical_router_up_link_port.edge_cluster_member_index.Element.Create() 1685 | $subnetAdd = $subnetSpec.ip_addresses.Add($T0UplinkRouterPortIP) 1686 | $subNetSpec.prefix_length = $T0UplinkRouterPortIPPrefix 1687 | $lrPortSpec.display_name = $T0UplinkRouterPortName 1688 | $lrPortSpec.linked_logical_switch_port_id.target_id = $logicalSwitchPort.id 1689 | $lrPortSpec.linked_logical_switch_port_id.target_type = "LogicalPort" 1690 | $subnetAdd = $lrPortSpec.subnets.Add($subNetSpec) 1691 | $memberIndex = @(0) 1692 | $lrPortSpec.edge_cluster_member_index = $memberIndex 1693 | $lrPortSpec.logical_router_id = $lr.id 1694 | $logicalRouterPortAdd = $logicalRouterPortSerivce.create($lrPortSpec) 1695 | } 1696 | 1697 | if($runT0StaticRoute) { 1698 | My-Logger "Creating Static Route on $T0LogicalRouterName from $T0UplinkRouterStaticRouteNetwork to $T0UplinkRouterstaticRouteNextHop ..." 1699 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1700 | $lr = $logicalRouterService.list().results | where { $_.display_name -eq $T0LogicalRouterName } 1701 | 1702 | $staticRouteSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_routers.routing.static_routes" 1703 | $staticRouteSpec = $staticRouteSerivce.Help.create.static_route.Create() 1704 | $nextHopeSpec = $staticRouteSerivce.Help.create.static_route.next_hops.Element.Create() 1705 | $staticRouteSpec.network = $T0UplinkRouterStaticRouteNetwork 1706 | $nextHopeSpec.ip_address = $T0UplinkRouterstaticRouteNextHop 1707 | $nextHopAdd = $staticRouteSpec.next_hops.Add($nextHopeSpec) 1708 | $staticRouteAdd = $staticRouteSerivce.create($lr.id, $staticRouteSpec) 1709 | } 1710 | 1711 | if($runT1Router) { 1712 | My-Logger "Creating T1 Router $T1LogicalRouterName ..." 1713 | # T1-K8S-Mgmt-Cluster 1714 | $edgeClusterService = Get-NsxtService -Name "com.vmware.nsx.edge_clusters" 1715 | $edgeCluster = $edgeClusterService.list().results | where { $_.display_name -eq $EdgeClusterName } 1716 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1717 | $lrSpec = $logicalRouterService.Help.create.logical_router.Create() 1718 | $lrSpec.display_name = $T1LogicalRouterName 1719 | $lrSpec.router_type = "TIER1" 1720 | $lrSpec.high_availability_mode = $T1LogicalRouterHAMode 1721 | $lrSpec.failover_mode = $T1LogicalRouterFailOverMode 1722 | $lrSpec.edge_cluster_id = $edgeCluster.id 1723 | $lrAdd = $logicalRouterService.create($lrSpec) 1724 | } 1725 | 1726 | if($runT1RouterPort) { 1727 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1728 | $t0lr = $logicalRouterService.list().results | where { $_.display_name -eq $T0LogicalRouterName } 1729 | $t1lr = $logicalRouterService.list().results | where { $_.display_name -eq $T1LogicalRouterName } 1730 | 1731 | $logicalSwitchService = Get-NsxtService -Name "com.vmware.nsx.logical_switches" 1732 | $ls = $logicalSwitchService.list().results | where { $_.display_name -eq $K8SMgmtClusterLogicalSwitchName} 1733 | 1734 | $logicalRouterPortSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_router_ports" 1735 | $portOnT0 = $logicalRouterPortSerivce.help.create.logical_router_port.logical_router_link_port_on_TIE_r0.Create() 1736 | $portOnT1 = $logicalRouterPortSerivce.Help.create.logical_router_port.logical_router_link_port_on_TIE_r1.Create() 1737 | $downlinkPortOnT1 = $logicalRouterPortSerivce.Help.create.logical_router_port.logical_router_down_link_port.Create() 1738 | $subnetSpec = $logicalRouterPortSerivce.help.create.logical_router_port.logical_router_down_link_port.subnets.Element.Create() 1739 | $memberIndex = $logicalRouterPortSerivce.Help.create.logical_router_port.logical_router_link_port_on_TIE_r1.edge_cluster_member_index.Create() 1740 | 1741 | My-Logger "Creating T0 Logical Router Port $T1LinkedRouterPortNameOnT0 ..." 1742 | # LinkedPort_K8S-Mgmt-Cluster 1743 | $portOnT0.display_name = $T1LinkedRouterPortNameOnT0 1744 | $portOnT0.logical_router_id = $t0lr.id 1745 | $portOnT0Add = $logicalRouterPortSerivce.create($portOnT0) 1746 | 1747 | My-Logger "Creating T1 Logical Router Port $T1LinkedRouterPortNameOnT1 ..." 1748 | # LinkedPort_T0-LR 1749 | $portOnT1.display_name = $T1LinkedRouterPortNameOnT1 1750 | $portOnT1.logical_router_id = $t1lr.id 1751 | $portOnT1.linked_logical_router_port_id.target_type = "LogicalRouterLinkPortOnTIER0" 1752 | $portOnT1.linked_logical_router_port_id.target_id = $portOnT0Add.id 1753 | $memberIndex = @(0) 1754 | $portOnT1.edge_cluster_member_index = $memberIndex 1755 | $portOnT1Add = $logicalRouterPortSerivce.create($portOnT1) 1756 | 1757 | My-Logger "Creating T1 Logical Downlink Router Switch Port $T1DownlinkRouterPortSwitchPortName" 1758 | # Downlink-1-Port 1759 | $logicalPortSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_ports" 1760 | $portSpec = $logicalPortSerivce.Help.create.logical_port.Create() 1761 | $portSpec.display_name = $T1DownlinkRouterPortSwitchPortName 1762 | $portSpec.admin_state = "UP" 1763 | $portSpec.logical_switch_id = $ls.id 1764 | $logicalSwitchPort = $logicalPortSerivce.create($portSpec) 1765 | 1766 | My-Logger "Creating T1 Logical Downlink Router Port $T1DownlinkRouterPortNameOnT1 ..." 1767 | # Downlink-1 1768 | $subnetAdd = $subnetSpec.ip_addresses.Add($T1DownlinkRouterPortIP) 1769 | $subNetSpec.prefix_length = $T1DownlinkRouterPortIPPrefix 1770 | $subnetAdd = $downlinkPortOnT1.subnets.Add($subNetSpec) 1771 | $downlinkPortOnT1.display_name = $T1DownlinkRouterPortNameOnT1 1772 | $downlinkPortOnT1.linked_logical_switch_port_id.target_type = "LogicalPort" 1773 | $downlinkPortOnT1.linked_logical_switch_port_id.target_id = $logicalSwitchPort.id 1774 | $downlinkPortOnT1.logical_router_id = $t1lr.id 1775 | $downlinkRouterPortAdd = $logicalRouterPortSerivce.create($downlinkPortOnT1) 1776 | } 1777 | 1778 | if($runT1RouterAdvertisement) { 1779 | $routingAdvertisementSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_routers.routing.advertisement" 1780 | 1781 | $logicalRouterSerivce = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1782 | $lr = $logicalRouterSerivce.list().results | where { $_.display_name -eq $T1LogicalRouterName } 1783 | $currentRevision = [int]$routingAdvertisementSerivce.get($lr.id).toString() 1784 | 1785 | My-Logger "Updating T1 Route Advertisement on $T1LogicalRouterName ..." 1786 | $advertisementSpec = $routingAdvertisementSerivce.Help.update.advertisement_config.Create() 1787 | $advertisementSpec.enabled = $true 1788 | $advertisementSpec.advertise_nsx_connected_routes = $true 1789 | $advertisementSpec.advertise_nat_routes = $true 1790 | $advertisementSpec.revision = $currentRevision 1791 | $routeAdvUpdate = $routingAdvertisementSerivce.update($lr.id,$advertisementSpec) 1792 | } 1793 | 1794 | My-Logger "Disconnecting from NSX Manager ..." 1795 | Disconnect-NsxtServer * -Confirm:$false 1796 | } 1797 | 1798 | if($setupOpsManager -eq 1) { 1799 | My-Logger "Setting up Ops Manager ..." 1800 | 1801 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword configure-authentication --username $OpsmanAdminUsername --password $OpsmanAdminPassword --decryption-passphrase $OpsmanDecryptionPassword" 1802 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 1803 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 1804 | } 1805 | 1806 | if($uploadStemcell -eq 1) { 1807 | $StemcellProduct = (Split-Path -Path $Stemcell -Leaf).ToString() 1808 | 1809 | My-Logger "Uploading Stemcell $StemcellProduct ..." 1810 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword upload-stemcell --force --stemcell $Stemcell" 1811 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 1812 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 1813 | } 1814 | 1815 | if($setupBOSHDirector -eq 1) { 1816 | My-Logger "Setting up BOSH Director ..." 1817 | 1818 | # Create BOSH YAML manually since we don't have native support for YAML in PS 1819 | $boshPayloadStart = @" 1820 | --- 1821 | az-configuration: 1822 | 1823 | "@ 1824 | # Process Mgmt & Compute AZ 1825 | $mgmtAZString = "" 1826 | $BOSHManagementAZ.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1827 | $mgmtAZString += " - name: `'"+$_.Name+"`'`n" 1828 | $mgmtAZString += " cluster: `'"+$_.Value+"`'`n" 1829 | } 1830 | 1831 | $computeAZString = "" 1832 | $BOSHComputeAZ.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1833 | $computeAZString += " - name: `'"+$_.Name+"`'`n" 1834 | $computeAZString += " cluster: `'"+$_.Value+"`'`n" 1835 | } 1836 | 1837 | # Process Networks 1838 | $boshPayloadNetwork = @" 1839 | networks-configuration: 1840 | icmp_checks_enabled: false 1841 | networks: 1842 | 1843 | "@ 1844 | $mgmtNetworkString = "" 1845 | $BOSHManagementNetwork.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1846 | $mgmtNetworkString += " - name: `'"+$_.Name+"`'`n" 1847 | $mgmtNetworkString += " service-network: false`'`n" 1848 | $mgmtNetworkString += " subnets:`n" 1849 | $mgmtNetworkString += " - iaas_identifier: `'"+$_.Value['portgroupname']+"`'`n" 1850 | $mgmtNetworkString += " cidr: `'"+$_.Value['cidr']+"`'`n" 1851 | $mgmtNetworkString += " gateway: `'"+$_.Value['gateway']+"`'`n" 1852 | $mgmtNetworkString += " dns: `'"+$_.Value['dns']+"`'`n" 1853 | $mgmtNetworkString += " cidr: `'"+$_.Value['cidr']+"`'`n" 1854 | $mgmtNetworkString += " reserved_ip_ranges: `'"+$_.Value['reserved_range']+"`'`n" 1855 | $mgmtNetworkString += " availability_zone_names:`n" 1856 | $mgmtNetworkString += " - `'"+$_.Value['az']+"`'`n" 1857 | } 1858 | 1859 | $computeNetworkString = "" 1860 | $BOSHServiceNetwork.GetEnumerator() | Sort-Object -Property Value | Foreach-Object { 1861 | $computeNetworkString += " - name: `'"+$_.Name+"`'`n" 1862 | $computeNetworkString += " service-network: true`'`n" 1863 | $computeNetworkString += " subnets:`n" 1864 | $computeNetworkString += " - iaas_identifier: `'"+$_.Value['portgroupname']+"`'`n" 1865 | $computeNetworkString += " cidr: `'"+$_.Value['cidr']+"`'`n" 1866 | $computeNetworkString += " gateway: `'"+$_.Value['gateway']+"`'`n" 1867 | $computeNetworkString += " dns: `'"+$_.Value['dns']+"`'`n" 1868 | $computeNetworkString += " cidr: `'"+$_.Value['cidr']+"`'`n" 1869 | $computeNetworkString += " reserved_ip_ranges: `'"+$_.Value['reserved_range']+"`'`n" 1870 | $computeNetworkString += " availability_zone_names:`n" 1871 | $computeNetworkString += " - `'"+$_.Value['az']+"`'`n" 1872 | } 1873 | 1874 | # Concat Mgmt & Service Network 1875 | $boshPayloadNetwork += $mgmtNetworkString 1876 | $boshPayloadNetwork += $computeNetworkString 1877 | 1878 | # Process remainder configs 1879 | $boshPayloadEnd = @" 1880 | director-configuration: 1881 | post_deploy_enabled: true 1882 | bosh_recreate_on_next_deploy: true 1883 | database_type: 'internal' 1884 | resurrector_enabled: true 1885 | director_worker_count: '5' 1886 | ntp_servers_string: $VMNTP 1887 | blobstore_type: 'local' 1888 | network-assignment: 1889 | network: 1890 | name: $BOSHManagementNetworkAssignment 1891 | singleton_availability_zone: 1892 | name: $BOSHManagementAZAssignment 1893 | iaas-configuration: 1894 | bosh_disk_path: $BOSHvCenterDiskFolder 1895 | bosh_vm_folder: $BOSHvCenterVMFolder 1896 | bosh_template_folder: $BOSHvCenterTemplateFolder 1897 | disk_type: 'thin' 1898 | datacenter: $BOSHvCenterDatacenter 1899 | persistent_datastores_string: $BOSHvCenterPersistentDatastores 1900 | ephemeral_datastores_string: $BOSHvCenterEpemeralDatastores 1901 | vcenter_host: $VIServer 1902 | vcenter_username: $BOSHvCenterUsername 1903 | vcenter_password: $BOSHvCenterPassword 1904 | security-configuration: 1905 | vm_password_type: 'generate' 1906 | trusted_certificates: '' 1907 | "@ 1908 | 1909 | # Concat configuration to form final YAML 1910 | $boshPayload = $boshPayloadStart + $mgmtAZString + $computeAZString + $boshPayloadNetwork + $boshPayloadEnd 1911 | 1912 | $boshYaml = "pks-bosh.yaml" 1913 | $boshPayload > $boshYaml 1914 | 1915 | My-Logger "Applying BOSH configuration ..." 1916 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword configure-director --config $boshYaml" 1917 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 1918 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 1919 | 1920 | My-Logger "Installing BOSH tile, this will take some time. Please grab some coffee or beer while you wait ..." 1921 | $installArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword apply-changes" 1922 | if($pksDebug) { My-Logger "${OMCLI} $installArgs"} 1923 | $output = Start-Process -FilePath $OMCLI -ArgumentList $installArgs -Wait -RedirectStandardOutput $verboseLogFile 1924 | } 1925 | 1926 | if($setupPKS -eq 1) { 1927 | My-Logger "Setting up PKS Control Plane ..." 1928 | 1929 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAdminUsername -Password $NSXAdminPassword -WarningAction SilentlyContinue)) { 1930 | Write-Host -ForegroundColor Red "Unable to connect to NSX Manager, please check the deployment" 1931 | exit 1932 | } else { 1933 | My-Logger "Successfully logged into NSX Manager $NSXTMgrHostname to retrieve some information ..." 1934 | } 1935 | 1936 | $ipPoolService = Get-NsxtService -Name "com.vmware.nsx.pools.ip_pools" 1937 | $ipPool = $ipPoolService.list().results | where { $_.display_name -eq $LoadBalancerPoolName} 1938 | $IpPoolID = $ipPool.Id 1939 | 1940 | $ipBlockService = Get-NsxtService -Name "com.vmware.nsx.pools.ip_blocks" 1941 | $ipBlock = $ipBlockService.list().results | where { $_.display_name -eq $ipBlockName} 1942 | $IpBlockID = $ipBlock.Id 1943 | 1944 | $logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers" 1945 | $logicalRouter = $logicalRouterService.list().results | where { $_.display_name -eq $T0LogicalRouterName} 1946 | $T0RouterID = $logicalRouter.Id 1947 | 1948 | My-Logger "Disconnecting from NSX Manager ..." 1949 | Disconnect-NsxtServer -Confirm:$false 1950 | 1951 | $PKSProduct = (Split-Path -Path $PKSTile -Leaf).ToString() 1952 | $PKSProductName = "pivotal-container-service" 1953 | $PKSVersion = $PKSProduct.Replace("pivotal-container-service-","").Replace(".pivotal","") 1954 | 1955 | # Upload PKS Tile 1956 | My-Logger "Uploading PKS Tile $PKSProduct ..." 1957 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword upload-product --product $PKSTile" 1958 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 1959 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 1960 | 1961 | # Stage PKS Tile 1962 | My-Logger "Adding PKS Tile to Ops Manager ..." 1963 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword stage-product --product-name $PKSProductName --product-version $PKSVersion" 1964 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 1965 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 1966 | 1967 | # Process PEM certs into single stringe encoded with \r\n 1968 | $certPEMString = "" 1969 | $certPrivateString = "" 1970 | $pksCertPEM -split "`r`n" | ForEach-Object { 1971 | $s = $_ + "\r\n" 1972 | $certPEMString += $s 1973 | } 1974 | $pksCertPrivateKey -split "`r`n" | ForEach-Object { 1975 | $s = $_ + "\r\n" 1976 | $certPrivateString += $s 1977 | } 1978 | 1979 | # Create PKS YAML manually since we don't have native support for YAML in PS 1980 | $pksPayload = @" 1981 | product-properties: 1982 | .pivotal-container-service.pks_tls: 1983 | value: 1984 | cert_pem: "$certPemString" 1985 | private_key_pem: "$certPrivateString" 1986 | .properties.cloud_provider: 1987 | value: vSphere 1988 | .properties.cloud_provider.vsphere.vcenter_dc: 1989 | value: $PKSDatacenter 1990 | .properties.cloud_provider.vsphere.vcenter_ds: 1991 | value: $PKSDatastore 1992 | .properties.cloud_provider.vsphere.vcenter_ip: 1993 | value: $PKSvCenter 1994 | .properties.cloud_provider.vsphere.vcenter_master_creds: 1995 | value: 1996 | identity: $PKSCPIMasterUsername 1997 | password: $PKSCPIMasterPassword 1998 | .properties.cloud_provider.vsphere.vcenter_vms: 1999 | value: $PKSDatastore 2000 | .properties.cloud_provider.vsphere.vcenter_worker_creds: 2001 | value: 2002 | identity: $PKSCPIWorkerUsername 2003 | password: $PKSCPIWorkerPassword 2004 | .properties.network_selector: 2005 | value: nsx 2006 | .properties.network_selector.nsx.credentials: 2007 | value: 2008 | identity: $PKSNSXUsername 2009 | password: $PKSNSXPassword 2010 | .properties.network_selector.nsx.floating-ip-pool-ids: 2011 | value: $IpPoolID 2012 | .properties.network_selector.nsx.ip-block-id: 2013 | value: $IpBlockID 2014 | .properties.network_selector.nsx.nsx-t-host: 2015 | value: $PKSNSX 2016 | .properties.network_selector.nsx.nsx-t-insecure: 2017 | value: true 2018 | .properties.network_selector.nsx.t0-router-id: 2019 | value: $T0RouterID 2020 | .properties.network_selector.nsx.vcenter_cluster: 2021 | value: $PKSCluster 2022 | .properties.plan1_selector: 2023 | value: Plan Active 2024 | .properties.plan1_selector.active.allow_privileged_containers: 2025 | value: false 2026 | .properties.plan1_selector.active.authorization_mode: 2027 | value: rbac 2028 | .properties.plan1_selector.active.az_placement: 2029 | value: $PKSPlan1AZ 2030 | .properties.plan1_selector.active.description: 2031 | value: Default plan for K8s cluster 2032 | .properties.plan1_selector.active.errand_vm_type: 2033 | value: micro 2034 | .properties.plan1_selector.active.master_persistent_disk_type: 2035 | value: "10240" 2036 | .properties.plan1_selector.active.master_vm_type: 2037 | value: medium 2038 | .properties.plan1_selector.active.name: 2039 | value: small 2040 | .properties.plan1_selector.active.persistent_disk_type: 2041 | value: "10240" 2042 | .properties.plan1_selector.active.worker_instances: 2043 | value: 3 2044 | .properties.plan1_selector.active.worker_vm_type: 2045 | value: medium 2046 | .properties.plan2_selector: 2047 | value: Plan Active 2048 | .properties.plan2_selector.active.allow_privileged_containers: 2049 | value: false 2050 | .properties.plan2_selector.active.authorization_mode: 2051 | value: rbac 2052 | .properties.plan2_selector.active.az_placement: 2053 | value: $PKSPlan2AZ 2054 | .properties.plan2_selector.active.description: 2055 | value: For Large Workloads 2056 | .properties.plan2_selector.active.errand_vm_type: 2057 | value: micro 2058 | .properties.plan2_selector.active.master_persistent_disk_type: 2059 | value: "10240" 2060 | .properties.plan2_selector.active.master_vm_type: 2061 | value: large 2062 | .properties.plan2_selector.active.name: 2063 | value: medium 2064 | .properties.plan2_selector.active.persistent_disk_type: 2065 | value: "10240" 2066 | .properties.plan2_selector.active.worker_instances: 2067 | value: 5 2068 | .properties.plan2_selector.active.worker_vm_type: 2069 | value: medium 2070 | .properties.plan3_selector: 2071 | value: Plan Inactive 2072 | .properties.plan3_selector.active.allow_privileged_containers: 2073 | value: false 2074 | .properties.plan3_selector.active.errand_vm_type: 2075 | value: micro 2076 | .properties.plan3_selector.active.master_persistent_disk_type: 2077 | value: "10240" 2078 | .properties.plan3_selector.active.name: 2079 | value: large 2080 | .properties.plan3_selector.active.persistent_disk_type: 2081 | value: "10240" 2082 | .properties.syslog_migration_selector: 2083 | value: disabled 2084 | .properties.syslog_migration_selector.enabled.tls_enabled: 2085 | value: true 2086 | .properties.syslog_migration_selector.enabled.transport_protocol: 2087 | value: tcp 2088 | .properties.uaa_pks_cli_access_token_lifetime: 2089 | value: 86400 2090 | .properties.uaa_pks_cli_refresh_token_lifetime: 2091 | value: 172800 2092 | .properties.uaa_url: 2093 | value: $PKSUAAURL 2094 | network-properties: 2095 | network: 2096 | name: $BOSHManagementNetworkAssignment 2097 | other_availability_zones: 2098 | - name: $BOSHManagementAZAssignment 2099 | service_network: 2100 | name: $PKSServiceNetworkAssignment 2101 | singleton_availability_zone: 2102 | name: $BOSHManagementAZAssignment 2103 | resource-config: 2104 | pivotal-container-service: 2105 | instances: automatic 2106 | persistent_disk: 2107 | size_mb: automatic 2108 | instance_type: 2109 | id: automatic 2110 | "@ 2111 | $pksYaml = "pks-pks.yaml" 2112 | $pksPayload > $pksYaml 2113 | 2114 | My-Logger "Applying PKS configuration ..." 2115 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword configure-product --product-name $PKSProductName --config $pksYaml" 2116 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 2117 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 2118 | 2119 | # setup errands 2120 | My-Logger "Applying PKS Errands ..." 2121 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword set-errand-state --product-name $PKSProductName --errand-name pks-nsx-t-precheck --post-deploy-state enabled" 2122 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 2123 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 2124 | 2125 | My-Logger "Installing PKS tile, this will take some time. Please grab some coffee or beer while you wait ..." 2126 | $installArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword apply-changes" 2127 | if($pksDebug) { My-Logger "${OMCLI} $installArgs"} 2128 | $output = Start-Process -FilePath $OMCLI -ArgumentList $installArgs -Wait -RedirectStandardOutput $verboseLogFile 2129 | } 2130 | 2131 | if($setupHarbor -eq 1) { 2132 | $HarborProduct = (Split-Path -Path $HarborTile -Leaf).ToString() 2133 | $HarborProductName = "harbor-container-registry" 2134 | $HarborVersion = $HarborProduct.Replace("harbor-container-registry-","").Replace(".pivotal","") 2135 | 2136 | # Upload Harbor Tile 2137 | My-Logger "Uploading Harbor Tile $HarborProduct ..." 2138 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword upload-product --product $HarborTile" 2139 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 2140 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 2141 | 2142 | # Stage Harbor Tile 2143 | My-Logger "Adding Harbor Tile to Ops Manager ..." 2144 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword stage-product --product-name $HarborProductName --product-version $HarborVersion" 2145 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 2146 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 2147 | 2148 | # Process PEM certs into single stringe encoded with \r\n 2149 | $certPEMString = "" 2150 | $certPrivateString = "" 2151 | $harborCertPEM -split "`r`n" | ForEach-Object { 2152 | $s = $_ + "\r\n" 2153 | $certPEMString += $s 2154 | } 2155 | $harborCertPrivateKey -split "`r`n" | ForEach-Object { 2156 | $s = $_ + "\r\n" 2157 | $certPrivateString += $s 2158 | } 2159 | 2160 | # Create Harbor YAML manually since we don't have native support for YAML in PS 2161 | $HarborPayload = @" 2162 | product-properties: 2163 | .properties.admin_password: 2164 | value: 2165 | secret: $HarborAdminPassword 2166 | .properties.auth_mode: 2167 | value: uaa_auth_pks 2168 | .properties.hostname: 2169 | value: $HarborHostname 2170 | .properties.no_proxy: 2171 | value: 127.0.0.1,localhost,ui 2172 | .properties.registry_storage: 2173 | value: filesystem 2174 | .properties.server_cert_key: 2175 | value: 2176 | cert_pem: "$certPEMString" 2177 | private_key_pem: "$certPrivateString" 2178 | .properties.with_clair: 2179 | value: true 2180 | .properties.with_notary: 2181 | value: true 2182 | network-properties: 2183 | network: 2184 | name: $HarborManagementNetworkAssignment 2185 | other_availability_zones: 2186 | - name: $HarborManagementAZAssignment 2187 | singleton_availability_zone: 2188 | name: $HarborManagementAZAssignment 2189 | resource-config: 2190 | harbor-app: 2191 | instances: automatic 2192 | persistent_disk: 2193 | size_mb: automatic 2194 | instance_type: 2195 | id: automatic 2196 | smoke-testing: 2197 | instances: automatic 2198 | instance_type: 2199 | id: automatic 2200 | "@ 2201 | 2202 | $harborYaml = "pks-harbor.yaml" 2203 | $harborPayload > $harborYaml 2204 | 2205 | My-Logger "Applying Harbor configuration ..." 2206 | $configArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword configure-product --product-name $HarborProductName --config $harborYaml" 2207 | if($pksDebug) { My-Logger "${OMCLI} $configArgs"} 2208 | $output = Start-Process -FilePath $OMCLI -ArgumentList $configArgs -Wait -RedirectStandardOutput $verboseLogFile 2209 | 2210 | My-Logger "Installing Harbor tile, this will take some time. Please grab some coffee or beer while you wait ..." 2211 | $installArgs = "-k -t $OpsManagerHostname -u $OpsmanAdminUsername -p $OpsmanAdminPassword apply-changes" 2212 | if($pksDebug) { My-Logger "${OMCLI} $installArgs"} 2213 | $output = Start-Process -FilePath $OMCLI -ArgumentList $installArgs -Wait -RedirectStandardOutput $verboseLogFile 2214 | } 2215 | 2216 | $EndTime = Get-Date 2217 | $duration = [math]::Round((New-TimeSpan -Start $StartTime -End $EndTime).TotalMinutes,2) 2218 | 2219 | My-Logger "PKS Lab Deployment Complete!" 2220 | My-Logger "StartTime: $StartTime" 2221 | My-Logger " EndTime: $EndTime" 2222 | My-Logger " Duration: $duration minutes" --------------------------------------------------------------------------------