├── README.adoc ├── common-further-resources.adoc ├── common-workshop-links.adoc ├── deploy.adoc ├── ignition.adoc ├── images ├── 01-vcenter-create-folder.png ├── 02-vcenter-deploy-ova.png ├── 03-vcenter-ova-url.png ├── 04-vcenter-ova-folder.png ├── 05-vcenter-ova-compute.png ├── 06-vcenter-ova-storage.png ├── 07-vcenter-ova-network.png ├── 08-vcenter-ova-template.png ├── 09-vcenter-ova-finish.png ├── 10-vcenter-clone-menu.png ├── 11-vcenter-clone-folder.png ├── 12-vcenter-clone-compute.png ├── 13-vcenter-clone-datastore.png ├── 14-vcenter-clone-customize-1.png ├── 15-vcenter-clone-customize-2.png ├── 16-vcenter-clone-advanced.png ├── 17-vcenter-clone-conf-params.png ├── 18-vcenter-clone-finish.png ├── 7.readytocompleteova.png ├── further-resources-deploying-to-openshift.png ├── further-resources-devops-with-openshift.png ├── further-resources-openshift-for-developers.png ├── overview-lab-env.png └── overview-workshop-vsphere-env.png ├── overview.adoc ├── post-deployment.adoc ├── prerequisites.adoc ├── troubleshooting.adoc └── vsphere-import-ova.adoc /README.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Modules: 4 | 5 | link:overview.adoc[Deployment Overview] 6 | 7 | link:prerequisites.adoc[Prerequisites Preparation] 8 | 9 | link:ignition.adoc[Generating Install Files] 10 | 11 | link:vsphere-import-ova.adoc[Import OVA on vSphere] 12 | 13 | link:deploy.adoc[Deploy Cluster on vSphere] 14 | 15 | link:post-deployment.adoc[Post deployment configuration] 16 | 17 | link:troubleshooting.adoc[Troubleshooting tips] 18 | 19 | link:common-further-resources.adoc[Further Resources] 20 | 21 | link:common-workshop-links.adoc[Workshop Links] 22 | 23 | -------------------------------------------------------------------------------- /common-further-resources.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | Below you will find further resources for learning about OpenShift and 4 | running OpenShift on your own computer, as well as details about OpenShift 5 | Online or other OpenShift related products and services. 6 | 7 | * *link:https://mojo.redhat.com/people/rpecora/blog/2019/12/04/dicas-para-implementa%C3%A7%C3%A3o-do-cluster-openshift-42-em-vmware-vsphere[Dicas para Implementação do cluster Openshift 4.2 em Vmware vSphere]* - Artigo detalhado com passo a passo e dicas de implementação do OpenShift em vSphere feito pelo nosso colega *Rafael Pécora*. 8 | 9 | * *link:https://blog.openshift.com/openshift-4-2-vsphere-install-quickstart/[OpenShift 4.2 vSphere Install Quickstart]* - Quick guide to install OpenShift on vSphere. 10 | 11 | * *link:https://docs.openshift.com[OpenShift Documentation]* - The landing page for OpenShift documentation. 12 | 13 | * *link:https://chat.google.com/room/AAAA2bt6nL0[OpenShift 4 on Google Chat]* - Google chat forum about OpenShift 4 14 | 15 | -------------------------------------------------------------------------------- /common-workshop-links.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | You can find all urls, hostnames, usernames and passwords that are needed during the workshop in this page. Note that the urls are also embedded inside the labs so that you don't have to go back and forth between this page and the labs. 4 | 5 | == Workshop Guides 6 | 7 | Web: https://github.com/giofontana/ocp4-vsphere-workshop/ 8 | 9 | == VCenter URL 10 | 11 | https://vcsa.rhbr-labs.com 12 | 13 | == Teams info 14 | 15 | *TEAM 1* 16 | ---- 17 | ssh user1@infra-services.rhbr-labs.com 18 | GUID=1 19 | datastore-id=esx1-lab-ds 20 | esx host=esx1.rhbr-labs.com 21 | vcenter vm folder=ocp1 22 | 23 | MAC ADDRESS: 24 | bootstrap-0 --> 00:50:56:01:01:01 25 | master-0 --> 00:50:56:01:01:02 26 | worker-0 --> 00:50:56:01:01:03 27 | 28 | URL Console: https://console-openshift-console.apps.ocp1.rhbr-labs.com/ 29 | ---- 30 | 31 | *TEAM 2* 32 | ---- 33 | ssh user2@infra-services.rhbr-labs.com 34 | GUID=2 35 | datastore-id=esx2-lab-ds 36 | esx host=esx2.rhbr-labs.com 37 | vcenter vm folder=ocp2 38 | 39 | MAC ADDRESS: 40 | bootstrap-0 --> 00:50:56:01:02:01 41 | master-0 --> 00:50:56:01:02:02 42 | worker-0 --> 00:50:56:01:02:03 43 | 44 | URL Console: https://console-openshift-console.apps.ocp2.rhbr-labs.com/ 45 | ---- 46 | 47 | *TEAM 3* 48 | ---- 49 | ssh user3@infra-services.rhbr-labs.com 50 | GUID=3 51 | datastore-id=esx3-lab-ds 52 | esx host=esx3.rhbr-labs.com 53 | vcenter vm folder=ocp3 54 | 55 | MAC ADDRESS: 56 | bootstrap-0 --> 00:50:56:01:03:01 57 | master-0 --> 00:50:56:01:03:02 58 | worker-0 --> 00:50:56:01:03:03 59 | 60 | URL Console: https://console-openshift-console.apps.ocp3.rhbr-labs.com/ 61 | ---- 62 | 63 | *TEAM 4* 64 | ---- 65 | ssh user4@infra-services.rhbr-labs.com 66 | GUID=4 67 | datastore-id=esx4-lab-ds 68 | esx host=esx4.rhbr-labs.com 69 | vcenter vm folder=ocp4 70 | 71 | MAC ADDRESS: 72 | bootstrap-0 --> 00:50:56:01:04:01 73 | master-0 --> 00:50:56:01:04:02 74 | worker-0 --> 00:50:56:01:04:03 75 | 76 | URL Console: https://console-openshift-console.apps.ocp4.rhbr-labs.com/ 77 | ---- 78 | 79 | *TEAM 5* 80 | ---- 81 | ssh user5@infra-services.rhbr-labs.com 82 | GUID=5 83 | datastore-id=esx5-lab-ds 84 | esx host=esx5.rhbr-labs.com 85 | vcenter vm folder=ocp5 86 | 87 | MAC ADDRESS: 88 | bootstrap-0 --> 00:50:56:01:05:01 89 | master-0 --> 00:50:56:01:05:02 90 | worker-0 --> 00:50:56:01:05:03 91 | 92 | URL Console: https://console-openshift-console.apps.ocp5.rhbr-labs.com/ 93 | ---- 94 | 95 | *TEAM 6* 96 | ---- 97 | ssh user6@infra-services.rhbr-labs.com 98 | GUID=6 99 | datastore-id=esx6-lab-ds 100 | esx host=esx6.rhbr-labs.com 101 | vcenter vm folder=ocp6 102 | 103 | MAC ADDRESS: 104 | bootstrap-0 --> 00:50:56:01:06:01 105 | master-0 --> 00:50:56:01:06:02 106 | worker-0 --> 00:50:56:01:06:03 107 | 108 | URL Console: https://console-openshift-console.apps.ocp6.rhbr-labs.com/ 109 | ---- -------------------------------------------------------------------------------- /deploy.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Deploying the Cluster on vSphere 4 | 5 | Access the vCenter web ui: 6 | ---- 7 | https://vcsa.rhbr-labs.com 8 | User: administrator@vsphere.local 9 | Password: 10 | ---- 11 | 12 | #### Provision OpenShift Servers 13 | 14 | Right click on the OVA and select *Clone -> Clone to VM* 15 | 16 | image::images/10-vcenter-clone-menu.png[Clone to VM] 17 | 18 | Select the folder you created before, input the vm name and click NEXT. Note that the vm name must be the same you have in your DHCP and DNS. 19 | 20 | ---- 21 | Folder: ocp 22 | VM Name: bootstrap-0 23 | ---- 24 | 25 | image::images/11-vcenter-clone-folder.png[Clone to VM] 26 | 27 | Select the compute resource and click NEXT: 28 | 29 | image::images/12-vcenter-clone-compute.png[Clone to VM] 30 | 31 | Select the datastore and select disk format as *"Thin Provision"*: 32 | 33 | image::images/13-vcenter-clone-datastore.png[Clone to VM] 34 | 35 | Enable the option *"Customize this virtual machine's hardware"* 36 | 37 | image::images/14-vcenter-clone-customize-1.png[Clone to VM] 38 | 39 | In the next screen input the following parameters: 40 | 41 | ---- 42 | CPU: 4 43 | Memory: 16 GB 44 | - Enable "Reserve all guest memory" option 45 | Hard Disk 1: 120 GB 46 | Network Adapter 1: 47 | - MAC Address: Manual - 48 | ---- 49 | 50 | image::images/15-vcenter-clone-customize-2.png[Clone to VM] 51 | 52 | Click in *"VM Options"* tab and expand *"Advanced"* accordion: 53 | 54 | image::images/16-vcenter-clone-advanced.png[Clone to VM] 55 | 56 | In *"Latency Sensitivity"* select High and click in *"Edit Configuration..."* button. 57 | 58 | Click in the *"ADD CONFIGURATION PARAMS"* button and add the following paramters: 59 | 60 | ---- 61 | guestinfo.ignition.config.data= 62 | guestinfo.ignition.config.data.encoding=base64 63 | disk.EnableUUID=TRUE 64 | ---- 65 | 66 | image::images/17-vcenter-clone-conf-params.png[Clone to VM] 67 | 68 | Click in NEXT and FINISH to create the bootstrap machine. 69 | 70 | image::images/18-vcenter-clone-finish.png[Clone to VM] 71 | 72 | *Repeat the same process above for VMs master-0 and worker-0. Use the following data:* 73 | 74 | [cols="3,2,2,2,5",options=header] 75 | |=== 76 | |MACHINE 77 | |vCPU 78 | |RAM 79 | |STORAGE 80 | |guestinfo.ignition.config.data 81 | 82 | |master-0 83 | |4 84 | |16 GB 85 | |120 GB 86 | |Output of: cat master.64 87 | 88 | |worker-0 89 | |2 90 | |8 GB 91 | |120 GB 92 | |Output of: cat worker.64 93 | 94 | |worker-1 95 | |2 96 | |8 GB 97 | |120 GB 98 | |Output of: cat worker.64 99 | 100 | |=== 101 | 102 | *Now boot the all the VMs.* 103 | 104 | #### Following the installation process 105 | 106 | After you boot the VMs, run the command below to follow the bootstrap process: 107 | 108 | ---- 109 | [user0@infra-services ocp]$ openshift-install wait-for bootstrap-complete --log-level debug 110 | DEBUG OpenShift Installer v4.2.0 111 | DEBUG Built from commit 90ccb37ac1f85ae811c50a29f9bb7e779c5045fb 112 | INFO Waiting up to 30m0s for the Kubernetes API at https://api.ocp.rhbr-labs.com:6443... 113 | INFO API v1.14.6+2e5ed54 up 114 | INFO Waiting up to 30m0s for bootstrapping to complete... 115 | DEBUG Bootstrap status: complete 116 | INFO It is now safe to remove the bootstrap resources 117 | ---- 118 | 119 | After you get the above INFO message, the bootstrap machine can be safely removed. 120 | 121 | This process can take up to 20 minutes. If you don't get this message, see some troubleshooting tips here link:troubleshooting.adoc[]! 122 | 123 | After bootstrap competition, run the following command to check the installation progress: 124 | 125 | ---- 126 | [user0@infra-services ocp]$ openshift-install wait-for install-complete --log-level debug 127 | DEBUG OpenShift Installer v4.2.10 128 | DEBUG Built from commit 6ed04f65b0f6a1e11f10afe658465ba8195ac459 129 | INFO Waiting up to 30m0s for the cluster at https://api.ocp.rhbr-labs.com:6443 to initialize... 130 | DEBUG Still waiting for the cluster to initialize: Working towards 4.2.10: 99% complete, waiting on authentication, console, image-registry 131 | DEBUG Still waiting for the cluster to initialize: Working towards 4.2.10: 99% complete, waiting on authentication, console, image-registry 132 | DEBUG Still waiting for the cluster to initialize: Working towards 4.2.10: 100% complete 133 | DEBUG Cluster is initialized 134 | INFO Waiting up to 10m0s for the openshift-console route to be created... 135 | DEBUG Route found in openshift-console namespace: console 136 | DEBUG Route found in openshift-console namespace: downloads 137 | DEBUG OpenShift console route is created 138 | INFO Install complete! 139 | INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/user0/ocp/auth/kubeconfig' 140 | INFO Access the OpenShift web-console here: https://console-openshift-console.apps.ocp.rhbr-labs.com 141 | INFO Login to the console with user: kubeadmin, password: YsviS-yGfBx-t6FsV-BZ58B 142 | ---- 143 | 144 | 145 | #### Running oc commands 146 | 147 | Configure kube config in your profile to be able to run oc commands: 148 | 149 | ---- 150 | mkdir ~/.kube/ 151 | cp auth/kubeconfig ~/.kube/config 152 | ---- 153 | 154 | #### Following cluster operators deployment 155 | 156 | During 157 | 158 | 159 | ---- 160 | [user0@infra-services ocp]$ watch -n 10 'oc get clusteroperators' 161 | Every 10.0s: oc get clusteroperators infra-services.rhbr-labs.com: Mon Dec 16 20:43:44 2019 162 | 163 | NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE 164 | authentication Unknown Unknown True 3m1s 165 | cloud-credential 4.2.10 True False False 6m54s 166 | console 4.2.10 Unknown True False 11s 167 | dns 4.2.10 True False False 6m20s 168 | image-registry False False True 11s 169 | ingress unknown False True False 11s 170 | insights 4.2.10 True False False 6m53s 171 | kube-apiserver 4.2.10 True False False 4m24s 172 | kube-controller-manager 4.2.10 True False False 4m18s 173 | kube-scheduler 4.2.10 True False False 4m16s 174 | machine-api 4.2.10 True False False 6m56s 175 | machine-config 4.2.10 True False False 6m18s 176 | marketplace False True False 12s 177 | monitoring Unknown True Unknown 14s 178 | network 4.2.10 True False False 5m57s 179 | node-tuning 4.2.10 True False False 2m50s 180 | openshift-apiserver 4.2.10 True False False 2m9s 181 | openshift-controller-manager 4.2.10 True False False 3m7s 182 | openshift-samples False False 9s 183 | operator-lifecycle-manager 4.2.10 True False False 5m52s 184 | operator-lifecycle-manager-catalog 4.2.10 True False False 5m52s 185 | operator-lifecycle-manager-packageserver 4.2.10 True False False 3m7s 186 | service-ca 4.2.10 True False False 6m46s 187 | service-catalog-apiserver 4.2.10 True False False 2m57s 188 | service-catalog-controller-manager 4.2.10 True False False 3m 189 | ---- -------------------------------------------------------------------------------- /ignition.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Generating Install Files 4 | 5 | Create ssh keys 6 | 7 | ---- 8 | ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa 9 | ---- 10 | 11 | Download install bin 12 | 13 | ---- 14 | mkdir ~/bin 15 | 16 | OCP4_BASEURL=https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest 17 | LATEST_VERSION=$(curl -s ${OCP4_BASEURL}/release.txt | grep 'Version: ' | awk '{print $2}') 18 | curl -s ${OCP4_BASEURL}/openshift-client-linux-$LATEST_VERSION.tar.gz | tar -xzf - -C ~/bin oc kubectl 19 | curl -s ${OCP4_BASEURL}/openshift-install-linux-$LATEST_VERSION.tar.gz | tar -xzf - -C ~/bin/ openshift-install 20 | ---- 21 | 22 | Generate Pull Secret at https://cloud.redhat.com/openshift/install/vsphere/user-provisioned. Store it in ~/ocp4_pull_secret 23 | 24 | 25 | Create install file: 26 | 27 | [NOTE] 28 | ==== 29 | Use the parameters given by the instructor! 30 | ==== 31 | 32 | ---- 33 | export GUID="" 34 | mkdir ~/ocp$GUID 35 | cd ~/ocp$GUID 36 | 37 | export DOMAIN=rhbr-labs.com # <1> 38 | export CLUSTERID=ocp$GUID # <2> 39 | export VCENTER_SERVER=vcsa.rhbr-labs.com # <3> 40 | export VCENTER_USER="administrator@vsphere.local" # <4> 41 | export VCENTER_PASS='' # <4> 42 | export VCENTER_DC='rhbr-labs-dc' # <5> 43 | export VCENTER_DS='' # <6> 44 | export PULL_SECRET=$(cat ~/ocp4_pull_secret) # <7> 45 | export OCP_SSH_KEY=$(cat ~/.ssh/id_rsa.pub) # <8> 46 | 47 | cat < install-config.yaml 48 | apiVersion: v1 49 | baseDomain: ${DOMAIN} 50 | compute: 51 | - hyperthreading: Enabled 52 | name: worker 53 | replicas: 1 54 | controlPlane: 55 | hyperthreading: Enabled 56 | name: master 57 | replicas: 1 58 | metadata: 59 | name: ${CLUSTERID} 60 | networking: 61 | clusterNetworks: 62 | - cidr: 10.254.0.0/16 63 | hostPrefix: 24 64 | networkType: OpenShiftSDN 65 | serviceNetwork: 66 | - 172.30.0.0/16 67 | platform: 68 | vsphere: 69 | vcenter: ${VCENTER_SERVER} 70 | username: ${VCENTER_USER} 71 | password: ${VCENTER_PASS} 72 | datacenter: ${VCENTER_DC} 73 | defaultDatastore: ${VCENTER_DS} 74 | pullSecret: '${PULL_SECRET}' 75 | sshKey: '${OCP_SSH_KEY}' 76 | EOF 77 | ---- 78 | 79 | <1> The base domain of the cluster. All DNS records must be sub-domains of this base and include the cluster name. 80 | <2> The cluster name that you specified in your DNS records. 81 | <3> The fully-qualified host name or IP address of the vCenter server. 82 | <4> vCenter credentials. This user must have at least the roles and privileges that are required for [dynamic persistent volume provisioning](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) in vSphere. 83 | <5> The vSphere Datacenter. 84 | <6> Default Datastore to use. 85 | <7> Pull secret obtained in cloud.redhat.com. 86 | <8> The public portion of the default SSH key for the core user in Red Hat Enterprise Linux CoreOS (RHCOS). 87 | 88 | 89 | Before next step, create a backup of install file (it will be automatically deleted in the next step): 90 | ---- 91 | cp install-config.yaml ../install-config.yaml.bkp 92 | ---- 93 | 94 | Create openshift-install manifests: 95 | 96 | ---- 97 | openshift-install create manifests 98 | ---- 99 | 100 | Manifests are yaml files that contains a lot of configuration that will be applied in the new cluster. Inspect the manifests files: 101 | ---- 102 | ls manifests/ 103 | cat manifests/* | more 104 | ---- 105 | 106 | Let's change you of the manifests to ensure that masters are no schedulable: 107 | ---- 108 | sed -i 's/mastersSchedulable: true/mastersSchedulable: false/g' manifests/cluster-scheduler-02-config.yml 109 | ---- 110 | 111 | Create ignition configs: 112 | ---- 113 | openshift-install create ignition-configs 114 | 115 | cat < append-bootstrap.ign 116 | { 117 | "ignition": { 118 | "config": { 119 | "append": [ 120 | { 121 | "source": "http://10.0.0.5:8080/ocp$GUID/ignition/bootstrap.ign", 122 | "verification": {} 123 | } 124 | ] 125 | }, 126 | "timeouts": {}, 127 | "version": "2.1.0" 128 | }, 129 | "networkd": {}, 130 | "passwd": {}, 131 | "storage": {}, 132 | "systemd": {} 133 | } 134 | EOF 135 | ---- 136 | 137 | 138 | Upload your bootstrap.ign to web server: 139 | ---- 140 | sudo mkdir -p /var/www/html/ocp$GUID/ignition/ 141 | sudo cp bootstrap.ign /var/www/html/ocp$GUID/ignition/ 142 | ---- 143 | 144 | Let's confirm that our webserver is hosting the bootstrap ignition file: 145 | ---- 146 | curl http://10.0.0.5:8080/ocp$GUID/ignition/bootstrap.ign 147 | ---- 148 | 149 | Generate files in base64: 150 | ---- 151 | for i in append-bootstrap master worker 152 | do 153 | base64 -w0 < $i.ign > $i.64 154 | done 155 | ---- -------------------------------------------------------------------------------- /images/01-vcenter-create-folder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/01-vcenter-create-folder.png -------------------------------------------------------------------------------- /images/02-vcenter-deploy-ova.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/02-vcenter-deploy-ova.png -------------------------------------------------------------------------------- /images/03-vcenter-ova-url.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/03-vcenter-ova-url.png -------------------------------------------------------------------------------- /images/04-vcenter-ova-folder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/04-vcenter-ova-folder.png -------------------------------------------------------------------------------- /images/05-vcenter-ova-compute.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/05-vcenter-ova-compute.png -------------------------------------------------------------------------------- /images/06-vcenter-ova-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/06-vcenter-ova-storage.png -------------------------------------------------------------------------------- /images/07-vcenter-ova-network.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/07-vcenter-ova-network.png -------------------------------------------------------------------------------- /images/08-vcenter-ova-template.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/08-vcenter-ova-template.png -------------------------------------------------------------------------------- /images/09-vcenter-ova-finish.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/09-vcenter-ova-finish.png -------------------------------------------------------------------------------- /images/10-vcenter-clone-menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/10-vcenter-clone-menu.png -------------------------------------------------------------------------------- /images/11-vcenter-clone-folder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/11-vcenter-clone-folder.png -------------------------------------------------------------------------------- /images/12-vcenter-clone-compute.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/12-vcenter-clone-compute.png -------------------------------------------------------------------------------- /images/13-vcenter-clone-datastore.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/13-vcenter-clone-datastore.png -------------------------------------------------------------------------------- /images/14-vcenter-clone-customize-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/14-vcenter-clone-customize-1.png -------------------------------------------------------------------------------- /images/15-vcenter-clone-customize-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/15-vcenter-clone-customize-2.png -------------------------------------------------------------------------------- /images/16-vcenter-clone-advanced.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/16-vcenter-clone-advanced.png -------------------------------------------------------------------------------- /images/17-vcenter-clone-conf-params.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/17-vcenter-clone-conf-params.png -------------------------------------------------------------------------------- /images/18-vcenter-clone-finish.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/18-vcenter-clone-finish.png -------------------------------------------------------------------------------- /images/7.readytocompleteova.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/7.readytocompleteova.png -------------------------------------------------------------------------------- /images/further-resources-deploying-to-openshift.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/further-resources-deploying-to-openshift.png -------------------------------------------------------------------------------- /images/further-resources-devops-with-openshift.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/further-resources-devops-with-openshift.png -------------------------------------------------------------------------------- /images/further-resources-openshift-for-developers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/further-resources-openshift-for-developers.png -------------------------------------------------------------------------------- /images/overview-lab-env.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/overview-lab-env.png -------------------------------------------------------------------------------- /images/overview-workshop-vsphere-env.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redhat-cop/ocp4-vsphere-workshop/c5ac4d39f82e4e78de94de4fc436968ab6caa912/images/overview-workshop-vsphere-env.png -------------------------------------------------------------------------------- /overview.adoc: -------------------------------------------------------------------------------- 1 | 2 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 3 | 4 | ### Overview 5 | 6 | The OpenShift installation on vSphere is comprised of the following: 7 | 8 | 1. Prepare pre-requisites 9 | 2. Pre-requisites validation 10 | 3. Ignition files creation 11 | 4. Deploy on vSphere 12 | 5. Post deployment configuration 13 | 14 | In this lab you will inspect in detail the pre-requisites and deploy a small cluster on vSphere from scratch. 15 | 16 | ### Lab Environment 17 | 18 | This lab environment is built on Ravello and contains 6 ESXi with 16 vCPU and 64 GB. Each team will use its own ESXi and datastore to avoid concurrence problems. 19 | 20 | image::images/overview-lab-env.png[Lab Environment on Ravello] 21 | 22 | ### Infra Services 23 | 24 | A VM named infra-services has been set up with the following services for each cluster you will deploy: 25 | 26 | - DHCP: DHCP is configured to provide ip and hostname for VMs of each cluster. 27 | - DNS: Infra-services is the internal DNS, providing A/PTR and SRV records for each cluster. 28 | - Webserver: A apache webserver is already configured in infra-services machine to host the ignition file for each cluster. 29 | - HAProxy: Infra-services is configured with one public ip for each cluster. HAProxy is set up to balance requisitions from each ip to the corresponding cluster. 30 | 31 | image::images/overview-workshop-vsphere-env.png[Workshop environment] 32 | -------------------------------------------------------------------------------- /post-deployment.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Post deployment configuration 4 | 5 | Some steps is needed after cluster is up. 6 | 7 | #### Check the certificates: 8 | 9 | ---- 10 | [root@infra-services ocp]# oc get csr 11 | NAME AGE REQUESTOR CONDITION 12 | csr-2zjlx 33m system:serviceaccount:openshift-machine-config-operator:node-bootstrapper Approved,Issued 13 | csr-7f8kq 4m42s system:serviceaccount:openshift-machine-config-operator:node-bootstrapper Pending 14 | csr-8cqn7 19m system:node:worker-0.ocp.rhbr-labs.com Pending 15 | csr-8ks7p 4m46s system:node:worker-0.ocp.rhbr-labs.com Pending 16 | csr-f4j47 23m system:serviceaccount:openshift-machine-config-operator:node-bootstrapper Approved,Issued 17 | csr-rthw9 4m44s system:node:worker-1.ocp.rhbr-labs.com Pending 18 | csr-sppgx 38m system:serviceaccount:openshift-machine-config-operator:node-bootstrapper Approved,Issued 19 | csr-zmtj4 19m system:node:worker-1.ocp.rhbr-labs.com 20 | ---- 21 | 22 | Approve the certificates with "oc adm certificate approve" command. 23 | 24 | ---- 25 | [root@infra-services ocp]# oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs oc adm certificate approve 26 | certificatesigningrequest.certificates.k8s.io/csr-7f8kq approved 27 | certificatesigningrequest.certificates.k8s.io/csr-8cqn7 approved 28 | certificatesigningrequest.certificates.k8s.io/csr-8ks7p approved 29 | certificatesigningrequest.certificates.k8s.io/csr-rthw9 approved 30 | certificatesigningrequest.certificates.k8s.io/csr-zmtj4 approved 31 | ---- 32 | 33 | #### Set registry storage storage: 34 | 35 | ---- 36 | oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}' 37 | ---- 38 | 39 | [NOTE] 40 | ==== 41 | In production environments set the registry storage with some supported persistent storage such OCS or NFS. 42 | ==== 43 | 44 | #### Set ingress configuration 45 | 46 | ---- 47 | oc get -n openshift-ingress-operator ingresscontrollers/default -o jsonpath='{$.spec.replicas}' 48 | 2 49 | oc patch -n openshift-ingress-operator ingresscontroller/default --patch '{"spec":{"replicas": 1}}' --type=merge 50 | ingresscontroller.operator.openshift.io/default patched 51 | ---- 52 | 53 | #### Config authentication with httpd 54 | 55 | ---- 56 | sudo yum install httpd-tools 57 | htpasswd -c -B -b users.htpasswd admin '******' 58 | oc create secret generic htpass-secret --from-file=htpasswd=users.htpasswd -n openshift-config 59 | 60 | cat < htpasswd-conf.yml 61 | apiVersion: config.openshift.io/v1 62 | kind: OAuth 63 | metadata: 64 | name: cluster 65 | spec: 66 | identityProviders: 67 | - name: my_htpasswd_provider 68 | mappingMethod: claim 69 | type: HTPasswd 70 | htpasswd: 71 | fileData: 72 | name: htpass-secret 73 | EOF 74 | 75 | oc apply -f htpasswd-conf.yml 76 | oc adm policy add-cluster-role-to-user cluster-admin admin 77 | 78 | oc delete secrets kubeadmin -n kube-system 79 | ---- 80 | -------------------------------------------------------------------------------- /prerequisites.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Prerequisites Preparation 4 | 5 | The following services are pre-requisites for OpenShift 4.2 installation on vSphere (https://docs.openshift.com/container-platform/4.2/installing/installing_vsphere/installing-vsphere.html#installation-infrastructure-user-infra_installing-vsphere[see more on the official docs for OpenShift]) : 6 | 7 | - DHCP 8 | - DNS 9 | - Load Balancer 10 | - Webserver (to host bootstrap ignition file) 11 | - vSphere 6.7U2 12 | 13 | ### Inspect pre-requisites 14 | 15 | All pre-requisites already has been prepared for you, however it is important to understand well these configurations. 16 | 17 | Access the infra-services node and inspect each pre-requisites below. 18 | 19 | *_DNS Settings:_* 20 | ---- 21 | [user0@infra-services ~]$ sudo cat /etc/named.conf 22 | acl internal_nets { 10.0.0.0/16; }; 23 | options { 24 | listen-on port 53 { 127.0.0.1; 10.0.0.5; }; 25 | listen-on-v6 port 53 { none; }; 26 | directory "/var/named"; 27 | dump-file "/var/named/data/cache_dump.db"; 28 | statistics-file "/var/named/data/named_stats.txt"; 29 | memstatistics-file "/var/named/data/named_mem_stats.txt"; 30 | secroots-file "/var/named/data/named.secroots"; 31 | (...) 32 | zone "ocp1.rhbr-labs.com" { 33 | type master; 34 | file "ocp1.rhbr-labs.com.zone"; 35 | allow-query { any; }; 36 | allow-transfer { none; }; 37 | allow-update { none; }; 38 | }; 39 | zone "1.0.10.in-addr.arpa" { 40 | type master; 41 | file "1.0.10.in-addr.arpa.zone"; 42 | allow-query { any; }; 43 | allow-transfer { none; }; 44 | allow-update { none; }; 45 | }; 46 | 47 | [user0@infra-services ~]# sudo cat /var/named/ocp1.rhbr-labs.com.zone 48 | $TTL 604800 49 | @ IN SOA infra-services.ocp1.rhbr-labs.com. admin.ocp1.rhbr-labs.com. ( 50 | 3 ; Serial 51 | 604800 ; Refresh 52 | 86400 ; Retry 53 | 2419200 ; Expire 54 | 604800 ) ; Negative Cache TTL 55 | ; 56 | ; name servers - NS records 57 | IN NS infra-services.ocp1.rhbr-labs.com. 58 | 59 | ; name servers - A records 60 | infra-services IN A 10.0.0.5 61 | bootstrap-0 IN A 10.0.1.100 62 | master-0 IN A 10.0.1.101 63 | worker-0 IN A 10.0.1.102 64 | api IN A 10.0.0.5 65 | api-int IN A 10.0.0.5 66 | apps IN A 10.0.0.5 67 | *.apps IN A 10.0.0.5 68 | etcd-0 IN A 10.0.1.101 69 | ; The SRV records ...note the trailing dot at the end. 70 | _etcd-server-ssl._tcp IN SRV 0 10 2380 etcd-0.ocp1.rhbr-labs.com. 71 | 72 | [user0@infra-services ~]# sudo cat /var/named/1.0.10.in-addr.arpa.zone 73 | $TTL 604800 74 | @ IN SOA infra-services.rhbr-labs.com. admin.rhbr-labs.com. ( 75 | 3 ; Serial 76 | 604800 ; Refresh 77 | 86400 ; Retry 78 | 2419200 ; Expire 79 | 604800 ) ; Negative Cache TTL 80 | ; name servers 81 | IN NS infra-services.rhbr-labs.com. 82 | 83 | ; PTR Records 84 | 100 IN PTR bootstrap-0.ocp1.rhbr-labs.com. 85 | 101 IN PTR master-0.ocp1.rhbr-labs.com. 86 | 102 IN PTR worker-0.ocp1.rhbr-labs.com. 87 | ---- 88 | 89 | *_DHCP:_* 90 | 91 | ---- 92 | [user0@infra-services ~]# sudo cat /etc/dhcp/dhcpd.conf 93 | default-lease-time 900; 94 | max-lease-time 7200; 95 | subnet 10.0.0.0 netmask 255.255.0.0 { 96 | option routers 10.0.0.2; 97 | option subnet-mask 255.255.0.0; 98 | option domain-name-servers 10.0.0.5; 99 | next-server 10.0.0.5; 100 | } 101 | 102 | #### CLUSTER OCP 103 | host bootstrap-0 { 104 | hardware ethernet 00:50:56:01:00:01; 105 | fixed-address 10.0.0.100; 106 | option host-name "bootstrap-0.ocp.rhbr-labs.com"; 107 | } 108 | host master-0 { 109 | hardware ethernet 00:50:56:01:00:02; 110 | fixed-address 10.0.0.101; 111 | option host-name "master-0.ocp.rhbr-labs.com"; 112 | } 113 | (...) 114 | ---- 115 | 116 | *_HAProxy Load Balancer:_* 117 | 118 | ---- 119 | [root@infra-services ~]# sudo cat /etc/haproxy/haproxy.cfg 120 | global 121 | log 127.0.0.1 local2 122 | chroot /var/lib/haproxy 123 | pidfile /var/run/haproxy.pid 124 | maxconn 4000 125 | user haproxy 126 | group haproxy 127 | daemon 128 | stats socket /var/lib/haproxy/stats 129 | ssl-default-bind-ciphers PROFILE=SYSTEM 130 | ssl-default-server-ciphers PROFILE=SYSTEM 131 | 132 | defaults 133 | mode http 134 | log global 135 | option httplog 136 | option dontlognull 137 | option http-server-close 138 | option forwardfor except 127.0.0.0/8 139 | option redispatch 140 | retries 3 141 | (...) 142 | #### BEGIN CLUSTER0 143 | 144 | frontend ocp4-kubernetes-api-server 145 | mode tcp 146 | option tcplog 147 | bind api.ocp.rhbr-labs.com:6443 148 | default_backend ocp4-kubernetes-api-server 149 | 150 | frontend ocp4-kubernetes-api-int-server 151 | mode tcp 152 | option tcplog 153 | bind api-int.ocp.rhbr-labs.com:6443 154 | default_backend ocp4-kubernetes-api-server 155 | (...) 156 | 157 | backend ocp4-kubernetes-api-server 158 | mode tcp 159 | balance source 160 | server boostrap-0-0 bootstrap-0.ocp.rhbr-labs.com:6443 check 161 | server master-0-0 master-0.ocp.rhbr-labs.com:6443 check 162 | server master-1-0 master-1.ocp.rhbr-labs.com:6443 check 163 | server master-2-0 master-2.ocp.rhbr-labs.com:6443 check 164 | 165 | backend ocp4-machine-config-server 166 | mode tcp 167 | balance source 168 | server bootstrap-0-0 bootstrap-0.ocp.rhbr-labs.com:22623 check 169 | server master-0-0 master-0.ocp.rhbr-labs.com:22623 check 170 | server master-1-0 master-1.ocp.rhbr-labs.com:22623 check 171 | server master-2-0 master-2.ocp.rhbr-labs.com:22623 check 172 | ---- 173 | 174 | *_Apache Webserver:_* 175 | ---- 176 | [root@infra-services ~]# cat /etc/httpd/conf/httpd.conf 177 | # 178 | # This is the main Apache HTTP server configuration file. It contains the 179 | # configuration directives that give the server its instructions. 180 | # See for detailed information. 181 | # In particular, see 182 | 183 | (...) 184 | #Listen 12.34.56.78:80 185 | Listen 8080 186 | ---- 187 | 188 | [NOTE] 189 | ==== 190 | Apache webserver was set to use port 8080 to avoid conflicts with haproxy that is running also in this same server. In a production environment you will use a dedicated Load Balancer so it is not required to change the Apache port. 191 | ==== 192 | 193 | 194 | 195 | ### Pre-requisites validation 196 | 197 | Check if the A, PTR and SRV records of the DNS are correctly set: 198 | 199 | *Checking A records:* 200 | ---- 201 | export GUID="" 202 | 203 | [user0@infra-services ~]$ dig bootstrap-0.ocp$GUID.rhbr-labs.com +short 204 | 10.0.0.100 205 | [user0@infra-services ~]$ dig master-0.ocp$GUID.rhbr-labs.com +short 206 | 10.0.0.101 207 | [user0@infra-services ~]$ dig etcd-0.ocp$GUID.rhbr-labs.com +short 208 | 10.0.0.101 209 | [user0@infra-services ~]$ dig worker-0.ocp$GUID.rhbr-labs.com +short 210 | 10.0.0.102 211 | ---- 212 | 213 | *Checking PTR records:* 214 | ---- 215 | [user0@infra-services ~]$ dig -x 10.0.0.100 +short 216 | bootstrap-0.ocp.rhbr-labs.com. 217 | [user0@infra-services ~]$ dig -x 10.0.0.101 +short 218 | master-0.ocp.rhbr-labs.com. 219 | [user0@infra-services ~]$ dig -x 10.0.0.102 +short 220 | worker-0.ocp.rhbr-labs.com. 221 | ---- 222 | 223 | *Checking API records:* 224 | ---- 225 | [user0@infra-services ~]$ dig api.ocp$GUID.rhbr-labs.com +short 226 | 10.0.0.5 227 | [user0@infra-services ~]$ dig api-int.ocp$GUID.rhbr-labs.com +short 228 | 10.0.0.5 229 | ---- 230 | 231 | *Checking APPs wildcard record:* 232 | ---- 233 | [user0@infra-services ~]$ dig *.apps.ocp$GUID.rhbr-labs.com +short 234 | 10.0.0.5 235 | ---- 236 | 237 | *Checking SRV records:* 238 | ---- 239 | [user0@infra-services ~]$ dig _etcd-server-ssl._tcp.ocp$GUID.rhbr-labs.com SRV +short 240 | 0 10 2380 etcd-0.ocp.rhbr-labs.com. 241 | ---- 242 | 243 | -------------------------------------------------------------------------------- /troubleshooting.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Troubleshooting tips 4 | 5 | To monitor the install progress: 6 | 7 | ---- 8 | openshift-install --dir=/root/ocp wait-for bootstrap-complete --log-level debug 9 | openshift-install --dir=/root/ocp wait-for install-complete --log-level debug 10 | ---- 11 | 12 | Example: 13 | 14 | ---- 15 | [root@bastion ~]# openshift-install --dir=/root/ocp wait-for bootstrap-complete --log-level debug 16 | DEBUG OpenShift Installer v4.2.1 17 | DEBUG Built from commit e349157f325dba2d06666987603da39965be5319 18 | INFO Waiting up to 30m0s for the Kubernetes API at https://api.ocp41.rhbr-labs.com:6443... 19 | INFO API v1.14.6+868bc38 up 20 | INFO Waiting up to 30m0s for bootstrapping to complete... 21 | DEBUG Bootstrap status: complete 22 | INFO It is now safe to remove the bootstrap resources 23 | [root@bastion ~]# openshift-install --dir=/root/ocp wait-for install-complete --log-level debug 24 | DEBUG OpenShift Installer v4.2.1 25 | DEBUG Built from commit e349157f325dba2d06666987603da39965be5319 26 | INFO Waiting up to 30m0s for the cluster at https://api.ocp41.rhbr-labs.com:6443 to initialize... 27 | DEBUG Still waiting for the cluster to initialize: Working towards 4.2.2: 96% complete, waiting on authentication, console, image-registry, ingress, marketplace, monitoring, openshift-samples 28 | DEBUG Still waiting for the cluster to initialize: Working towards 4.2.2: 97% complete, waiting on authentication, console, image-registry, monitoring, openshift-samples 29 | ---- 30 | 31 | 32 | To access the bootstrap machine: 33 | ---- 34 | ssh -i core@ 35 | ---- 36 | 37 | Example: 38 | 39 | ---- 40 | ssh -i ~/.ssh/id_rsa core@bootstrap-0 41 | ---- 42 | 43 | 44 | To access the bootstrap logs (on bootstrap machine): 45 | 46 | ---- 47 | journalctl -b -f -u bootkube.service 48 | ---- 49 | 50 | Example: 51 | 52 | ---- 53 | #### When the bootstrap takes too much time, it is useful the check the bootstrap logs in order to see if 54 | #### your etcd was installed successfully. Below is an example: 55 | 56 | [core@bootstrap-0 ~]$ journalctl -b -f -u bootkube.service 57 | -- Logs begin at Mon 2019-11-04 19:32:54 UTC. -- 58 | Nov 04 20:15:50 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-1.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: dial tcp 10.0.0.21:2379: connect: no route to host 59 | Nov 04 20:15:50 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-2.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: dial tcp 10.0.0.22:2379: connect: no route to host 60 | Nov 04 20:15:50 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-0.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: context deadline exceeded 61 | Nov 04 20:15:50 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: Error: unhealthy cluster 62 | Nov 04 20:15:50 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: etcdctl failed. Retrying in 5 seconds... 63 | Nov 04 20:25:56 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-0.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: context deadline exceeded 64 | Nov 04 20:25:56 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-1.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: dial tcp 10.0.0.21:2379: connect: no route to host 65 | Nov 04 20:25:56 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-2.ocp41.rhbr-labs.com:2379 is unhealthy: failed to connect: dial tcp 10.0.0.22:2379: connect: connection refused 66 | Nov 04 20:25:56 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: Error: unhealthy cluster 67 | Nov 04 20:25:57 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: etcdctl failed. Retrying in 5 seconds... 68 | Nov 04 20:32:24 bootstrap-0.ocp41.rhbr-labs.com bootkube.sh[1391]: https://etcd-0.ocp41.rhbr-labs.com:2379 is healthy: successfully committed proposal: took = 59.215123ms 69 | ---- 70 | 71 | 72 | To access a master server: 73 | 74 | ---- 75 | ssh -i ~/.ssh/id_rsa core@master-0 76 | ---- 77 | 78 | [NOTE] 79 | ==== 80 | SSH to the servers is not recommended for IPI servers - your server will be annotated and it can become tainted. 81 | ==== 82 | 83 | To access logs in the master server: 84 | ---- 85 | ssh -i ~/.ssh/id_rsa core@master-0 86 | sudo -i 87 | crictl ps # take note of the container id 88 | crictl logs 89 | ---- 90 | 91 | Example: 92 | ---- 93 | [root@bastion ~]# ssh core@master-0.ocp41.rhbr-labs.com 94 | Red Hat Enterprise Linux CoreOS 42.80.20191022.0 95 | WARNING: Direct SSH access to machines is not recommended. 96 | 97 | --- 98 | Last login: Thu Nov 7 13:10:53 2019 from 10.0.0.10 99 | [core@master-0 ~]$ sudo -i 100 | [root@master-0 ~]# crictl ps 101 | CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT POD ID 102 | (...) 103 | 16a88077135bc 15d11895e67779a458c2d2ab981865b9fc413653dd00a200bab2e74f77c1dc6a 32 minutes ago Running etcd-member 2 7ee9bcc8865a6 104 | a55b7fd0a4a16 e6adfb4b3938560cb0216a1616aec6269ffb2ae7e77b745a9961e0cb1120ba48 32 minutes ago Running kube-controller-manager-cert-syncer-11 2 27cc075a403c7 105 | cac672bbc457d cc480f7d86b3c53906ce61203bf118801fd78f945553f2b10c4d70ed7e1219c3 32 minutes ago Running kube-controller-manager-11 108 27cc075a403c7 106 | 107 | [root@master-0 ~]# crictl logs 16a88077135bc 108 | 2019-11-07 13:16:38.558131 W | etcdserver: read-only range request "key:\"/kubernetes.io/config.openshift.io/infrastructures\" range_end:\"/kubernetes.io/config.openshift.io/infrastructuret\" count_only:true " with result "range_response_count:0 size:8" took too long (5.441458563s) to execute 109 | 2019-11-07 13:16:38.558198 W | etcdserver: read-only range request "key:\"/kubernetes.io/deployments\" range_end:\"/kubernetes.io/deploymentt\" count_only:true " with result "range_response_count:0 size:8" took too long (1.759998564s) to execute 110 | 2019-11-07 13:16:38.558363 W | etcdserver: read-only range request "key:\"/kubernetes.io/priorityclasses\" range_end:\"/kubernetes.io/priorityclasset\" count_only:true " with result "range_response_count:0 size:8" took too long (5.564951088s) to execute 111 | 2019-11-07 13:16:38.558517 W | etcdserver: read-only range request "key:\"/kubernetes.io/leases\" range_end:\"/kubernetes.io/leaset\" count_only:true " with result "range_response_count:0 size:8" took too long (3.173732609s) to execute 112 | (...) 113 | ---- 114 | 115 | 116 | To check the API certificate (use this to check if the certificate is not expired): 117 | 118 | ---- 119 | echo | openssl s_client -connect api-int.ocp41.rhbr-labs.com:6443 | openssl x509 -noout -text 120 | ---- 121 | 122 | Example: 123 | 124 | ---- 125 | [root@bastion ~]# echo | openssl s_client -connect api-int.ocp41.rhbr-labs.com:6443 | openssl x509 -noout -text 126 | depth=1 OU = openshift, CN = kube-apiserver-lb-signer 127 | verify error:num=19:self signed certificate in certificate chain 128 | DONE 129 | Certificate: 130 | Data: 131 | Version: 3 (0x2) 132 | Serial Number: 7670043859767829723 (0x6a717840b920a4db) 133 | Signature Algorithm: sha256WithRSAEncryption 134 | Issuer: OU = openshift, CN = kube-apiserver-lb-signer 135 | Validity 136 | Not Before: Nov 7 13:24:42 2019 GMT 137 | Not After : Nov 8 13:24:48 2019 GMT 138 | Subject: O = kube-master, CN = system:kube-apiserver 139 | Subject Public Key Info: 140 | Public Key Algorithm: rsaEncryption 141 | RSA Public-Key: (2048 bit) 142 | Modulus: 143 | 00:c3:56:e2:a3:38:cb:d4:d3:1b:26:7a:b3:68:57: 144 | c2:75:73:10:0b:a3:b0:69:25:b0:74:a8:9c:88:e3: 145 | c1:23:3b:51:c2:90:8e:85:a7:ec:b9:79:eb:a0:1b: 146 | 04:03:d1:5d:3c:2d:a9:95:8e:7c:ef:6f:f9:39:85: 147 | 36:d2:43:09:8c:5a:c9:15:c4:4c:a8:70:64:98:9f: 148 | 23:b5:a0:ad:63:59:b0:52:69:2f:53:99:19:a4:22: 149 | 93:fb:ae:0c:e3:43:8d:9c:85:79:fe:6b:22:87:8f: 150 | 19:a0:47:07:59:a8:2c:bc:66:b2:17:d3:2f:d0:5f: 151 | 51:68:03:10:08:8e:f0:1b:d4:99:07:61:e9:05:40: 152 | fe:f8:69:d9:e8:88:c1:d0:e1:fe:16:9a:5e:2c:1b: 153 | eb:53:61:a9:80:cb:e4:f5:a7:0e:6c:19:90:45:b5: 154 | 00:5b:b8:1f:42:7f:cf:85:d1:f3:df:17:fb:01:c0: 155 | e5:de:4d:1d:0c:ae:65:a9:ef:b2:cd:2a:c5:a0:b3: 156 | f6:8a:83:e9:fb:3c:82:ef:67:c6:06:26:30:7f:ef: 157 | fc:b5:8e:98:e1:d3:c1:98:64:3c:e0:0b:84:24:34: 158 | 62:68:5a:5f:35:78:7e:1e:d1:22:3a:50:52:9b:a0: 159 | 33:29:a3:63:14:9f:f9:a2:44:d6:84:8f:b4:12:24: 160 | cb:97 161 | Exponent: 65537 (0x10001) 162 | X509v3 extensions: 163 | X509v3 Key Usage: critical 164 | Digital Signature, Key Encipherment 165 | X509v3 Extended Key Usage: 166 | TLS Web Server Authentication 167 | X509v3 Basic Constraints: critical 168 | CA:FALSE 169 | X509v3 Subject Key Identifier: 170 | F4:32:4B:D4:D5:EA:81:1C:D2:49:66:E2:A2:9F:7E:6E:BF:35:A9:31 171 | X509v3 Authority Key Identifier: 172 | keyid:F4:32:4B:D4:D5:EA:81:1C:D2:49:66:E2:A2:9F:7E:6E:BF:35:A9:31 173 | 174 | X509v3 Subject Alternative Name: 175 | DNS:api-int.ocp41.rhbr-labs.com 176 | Signature Algorithm: sha256WithRSAEncryption 177 | 66:b1:f1:ac:3d:5d:93:ea:c2:89:5c:6e:c8:e3:d5:6c:0b:e3: 178 | 7f:b7:bb:27:80:af:9c:13:79:1f:24:7d:6e:73:1d:69:fa:f7: 179 | 00:d0:01:73:97:d5:7e:e3:43:e3:02:f1:64:af:b9:90:87:2e: 180 | 5c:51:b4:8c:74:9a:cc:9a:fe:39:0e:52:ef:b1:dc:67:1e:27: 181 | dd:ed:1a:3c:d7:7e:d8:73:6b:ec:5f:20:8f:4b:fb:fa:d2:2f: 182 | 34:83:42:72:a6:ca:fb:ad:c5:06:5b:24:4d:c1:04:9f:aa:b5: 183 | 96:ca:34:02:d2:1e:76:08:c7:7e:87:dc:e4:9d:85:bc:7a:a5: 184 | 3b:c4:2f:d2:bf:c8:bb:97:21:77:b0:94:fb:1a:cf:2b:88:1d: 185 | cb:01:6d:86:32:51:06:d0:eb:39:93:2d:a4:53:4c:9a:52:df: 186 | a9:7b:cc:e6:4f:34:bf:1d:4b:5c:b7:9f:0f:7a:0a:53:52:53: 187 | 3e:14:6c:cf:ef:82:dc:e7:7c:1a:ba:f5:8c:45:bb:9c:77:34: 188 | 09:6b:81:5c:42:ca:1f:aa:9b:ea:4f:2d:35:32:f6:95:25:89: 189 | 85:6c:98:73:3f:56:c3:dc:fa:d4:f9:7a:ed:9e:e2:28:4f:ae: 190 | f0:08:92:98:36:86:23:b8:50:38:c7:67:da:df:8a:26:7f:f0: 191 | e1:80:6e:f7 192 | 193 | ---- 194 | 195 | [NOTE] 196 | ==== 197 | *Never reuse the openshift install dir!* In case a reinstallation is needed, delete the folder before generating ignition files again. The certificates that are generated by the installer are saved in hidden files inside this folder - if you only delete the ignition files and run the openshift-install again, it will use the same old certificates, that are already expired and you will have troubles!!! Also, you should keep your system up and running until the certificates are rotated and it can take up to 24 hours, so don't stop your environment until there. The command above helps you to check the certificate expiration and see if the certificate has been rotated already or not. 198 | ==== 199 | -------------------------------------------------------------------------------- /vsphere-import-ova.adoc: -------------------------------------------------------------------------------- 1 | # Workshop Hands-on - Deploy OCP 4.2 on vSphere 2 | 3 | ### Import OVA on vSphere 4 | 5 | Access the vCenter web ui: 6 | ---- 7 | https://vcsa.rhbr-labs.com 8 | User: administrator@vsphere.local 9 | Password: 10 | ---- 11 | 12 | #### Import OVA Template 13 | 14 | [NOTE] 15 | ==== 16 | Note 1: To save time, this OVA is has been already imported. The steps below is for reference only, use the template that is in "ocp-template" folder and skip the steps below. 17 | 18 | Note 2: *NEVER, NEVER, NEVER start up the template*. The ignition process runs on first boot, so booting the template would mean any ignition files provided after that wouldn't be evaluated. 19 | ==== 20 | 21 | Navigate to “VMs and Templates” (it’s the icon that looks like a piece of paper). From here right click on your datacenter and select New Folder → New VM and Template Folder. Name this new folder the name of your cluster id: ocp 22 | 23 | image::images/01-vcenter-create-folder.png[Creating Folder] 24 | 25 | Next, import the OVA by right clicking the folder and select “Deploy OVF Template”. 26 | 27 | image::images/02-vcenter-deploy-ova.png[Deploying OVA] 28 | 29 | Add the url to RHCOS OVA (https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.2/latest/[see here]) and click on NEXT button: 30 | 31 | image::images/03-vcenter-ova-url.png[Deploying OVA] 32 | 33 | Select the folder you created before and click on NEXT button: 34 | 35 | image::images/04-vcenter-ova-folder.png[Deploying OVA] 36 | 37 | Select the compute resource and click on NEXT button: 38 | 39 | image::images/05-vcenter-ova-compute.png[Deploying OVA] 40 | 41 | Select the datastore you filled in the installation conf file: 42 | 43 | image::images/06-vcenter-ova-storage.png[Deploying OVA] 44 | 45 | Select the network and click on NEXT button: 46 | 47 | image::images/07-vcenter-ova-network.png[Deploying OVA] 48 | 49 | Don't fill anything yet (these parameters will be filled further). Click NEXT. 50 | 51 | image::images/08-vcenter-ova-template.png[Deploying OVA] 52 | 53 | Click em Finish in the next screen 54 | 55 | image::images/09-vcenter-ova-finish.png[Deploying OVA] 56 | 57 | --------------------------------------------------------------------------------