├── .gitignore ├── LICENSE ├── README.rst ├── heat ├── README.rst ├── nvpnathan │ ├── 1vm-1lnet-1floatingip.yaml │ ├── LBaaS-DB-APP-Affinity.yaml │ ├── LBaaS-Three-Tier.yaml │ └── LBaaS-Web-Anti-Affinity.yaml └── poc │ └── validation_tests │ ├── 1-tier_photon_central-routing_nat.yaml │ ├── 2-tier_exclusive-routing-lb_nat.yaml │ ├── 3-tier_central-routing_nat.yaml │ ├── 3-tier_central-routing_nonat.yaml │ ├── 3-tier_distributed-routing_nat.yaml │ └── 3-tier_distributed-routing_nonat.yaml ├── tests ├── README.rst ├── buildwebapi │ ├── README.rst │ ├── buildwebapi │ │ ├── __init__.py │ │ └── api.py │ ├── requirements.txt │ ├── setup.py │ └── tests │ │ ├── buildapi │ │ └── test_api.py │ │ └── resources │ │ └── buildapi │ │ ├── build.json │ │ ├── buildlist.json │ │ ├── buildmetrics.json │ │ └── deliverable.json ├── install.sh ├── omsclient │ ├── README.rst │ ├── omsclient │ │ ├── __init__.py │ │ ├── oms_controller.py │ │ ├── restclient.py │ │ └── utils.py │ ├── requirements.txt │ └── setup.py ├── panda │ ├── README.rst │ ├── bin │ │ └── panda │ ├── panda │ │ ├── __init__.py │ │ ├── build_utils.py │ │ ├── cluster_utils.py │ │ ├── data │ │ │ ├── dvs-excluded-tests.txt │ │ │ ├── dvs-included-neutron.txt │ │ │ ├── logging.json │ │ │ ├── nsxv-excluded-tests.txt │ │ │ ├── tempest.conf.template │ │ │ └── vmware-excluded-tests.txt │ │ ├── end_to_end.py │ │ ├── exceptions.py │ │ ├── logging_utils.py │ │ ├── oms_utils.py │ │ ├── os_utils.py │ │ ├── setup.py │ │ ├── subunit2html.py │ │ ├── task_utils.py │ │ ├── tempest_utils.py │ │ └── test.py │ ├── requirements.txt │ └── setup.py ├── pyVmomiwrapper │ ├── README.rst │ ├── pyVmomiwrapper │ │ ├── __init__.py │ │ ├── task.py │ │ └── vmwareapi.py │ ├── requirements.txt │ └── setup.py ├── shellutil │ ├── README.rst │ ├── setup.py │ ├── shellutil │ │ ├── __init__.py │ │ └── shell.py │ └── tests │ │ ├── __init__.py │ │ └── test_shell.py └── sshutil │ ├── README.rst │ ├── bin │ ├── ssh_exec │ └── ssh_scp │ ├── requirements.txt │ ├── setup.py │ ├── sshutil │ ├── __init__.py │ └── remote.py │ └── tests │ ├── __init__.py │ └── test_remote.py └── util ├── README.rst ├── openstack-cli-windows ├── README.md └── openstack-cli-env.ps1 └── openstack-client ├── README.md ├── Vagrantfile └── src └── base-prep.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The content in this repository is distributed under the Apache v2.0 License. 2 | 3 | Apache License 4 | Version 2.0, January 2004 5 | http://www.apache.org/licenses/ 6 | 7 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 8 | 9 | 1. Definitions. 10 | 11 | "License" shall mean the terms and conditions for use, reproduction, 12 | and distribution as defined by Sections 1 through 9 of this document. 13 | 14 | "Licensor" shall mean the copyright owner or entity authorized by 15 | the copyright owner that is granting the License. 16 | 17 | "Legal Entity" shall mean the union of the acting entity and all 18 | other entities that control, are controlled by, or are under common 19 | control with that entity. For the purposes of this definition, 20 | "control" means (i) the power, direct or indirect, to cause the 21 | direction or management of such entity, whether by contract or 22 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 23 | outstanding shares, or (iii) beneficial ownership of such entity. 24 | 25 | "You" (or "Your") shall mean an individual or Legal Entity 26 | exercising permissions granted by this License. 27 | 28 | "Source" form shall mean the preferred form for making modifications, 29 | including but not limited to software source code, documentation 30 | source, and configuration files. 31 | 32 | "Object" form shall mean any form resulting from mechanical 33 | transformation or translation of a Source form, including but 34 | not limited to compiled object code, generated documentation, 35 | and conversions to other media types. 36 | 37 | "Work" shall mean the work of authorship, whether in Source or 38 | Object form, made available under the License, as indicated by a 39 | copyright notice that is included in or attached to the work 40 | (an example is provided in the Appendix below). 41 | 42 | "Derivative Works" shall mean any work, whether in Source or Object 43 | form, that is based on (or derived from) the Work and for which the 44 | editorial revisions, annotations, elaborations, or other modifications 45 | represent, as a whole, an original work of authorship. For the purposes 46 | of this License, Derivative Works shall not include works that remain 47 | separable from, or merely link (or bind by name) to the interfaces of, 48 | the Work and Derivative Works thereof. 49 | 50 | "Contribution" shall mean any work of authorship, including 51 | the original version of the Work and any modifications or additions 52 | to that Work or Derivative Works thereof, that is intentionally 53 | submitted to Licensor for inclusion in the Work by the copyright owner 54 | or by an individual or Legal Entity authorized to submit on behalf of 55 | the copyright owner. For the purposes of this definition, "submitted" 56 | means any form of electronic, verbal, or written communication sent 57 | to the Licensor or its representatives, including but not limited to 58 | communication on electronic mailing lists, source code control systems, 59 | and issue tracking systems that are managed by, or on behalf of, the 60 | Licensor for the purpose of discussing and improving the Work, but 61 | excluding communication that is conspicuously marked or otherwise 62 | designated in writing by the copyright owner as "Not a Contribution." 63 | 64 | "Contributor" shall mean Licensor and any individual or Legal Entity 65 | on behalf of whom a Contribution has been received by Licensor and 66 | subsequently incorporated within the Work. 67 | 68 | 2. Grant of Copyright License. Subject to the terms and conditions of 69 | this License, each Contributor hereby grants to You a perpetual, 70 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 71 | copyright license to reproduce, prepare Derivative Works of, 72 | publicly display, publicly perform, sublicense, and distribute the 73 | Work and such Derivative Works in Source or Object form. 74 | 75 | 3. Grant of Patent License. Subject to the terms and conditions of 76 | this License, each Contributor hereby grants to You a perpetual, 77 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 78 | (except as stated in this section) patent license to make, have made, 79 | use, offer to sell, sell, import, and otherwise transfer the Work, 80 | where such license applies only to those patent claims licensable 81 | by such Contributor that are necessarily infringed by their 82 | Contribution(s) alone or by combination of their Contribution(s) 83 | with the Work to which such Contribution(s) was submitted. If You 84 | institute patent litigation against any entity (including a 85 | cross-claim or counterclaim in a lawsuit) alleging that the Work 86 | or a Contribution incorporated within the Work constitutes direct 87 | or contributory patent infringement, then any patent licenses 88 | granted to You under this License for that Work shall terminate 89 | as of the date such litigation is filed. 90 | 91 | 4. Redistribution. You may reproduce and distribute copies of the 92 | Work or Derivative Works thereof in any medium, with or without 93 | modifications, and in Source or Object form, provided that You 94 | meet the following conditions: 95 | 96 | (a) You must give any other recipients of the Work or 97 | Derivative Works a copy of this License; and 98 | 99 | (b) You must cause any modified files to carry prominent notices 100 | stating that You changed the files; and 101 | 102 | (c) You must retain, in the Source form of any Derivative Works 103 | that You distribute, all copyright, patent, trademark, and 104 | attribution notices from the Source form of the Work, 105 | excluding those notices that do not pertain to any part of 106 | the Derivative Works; and 107 | 108 | (d) If the Work includes a "NOTICE" text file as part of its 109 | distribution, then any Derivative Works that You distribute must 110 | include a readable copy of the attribution notices contained 111 | within such NOTICE file, excluding those notices that do not 112 | pertain to any part of the Derivative Works, in at least one 113 | of the following places: within a NOTICE text file distributed 114 | as part of the Derivative Works; within the Source form or 115 | documentation, if provided along with the Derivative Works; or, 116 | within a display generated by the Derivative Works, if and 117 | wherever such third-party notices normally appear. The contents 118 | of the NOTICE file are for informational purposes only and 119 | do not modify the License. You may add Your own attribution 120 | notices within Derivative Works that You distribute, alongside 121 | or as an addendum to the NOTICE text from the Work, provided 122 | that such additional attribution notices cannot be construed 123 | as modifying the License. 124 | 125 | You may add Your own copyright statement to Your modifications and 126 | may provide additional or different license terms and conditions 127 | for use, reproduction, or distribution of Your modifications, or 128 | for any such Derivative Works as a whole, provided Your use, 129 | reproduction, and distribution of the Work otherwise complies with 130 | the conditions stated in this License. 131 | 132 | 5. Submission of Contributions. Unless You explicitly state otherwise, 133 | any Contribution intentionally submitted for inclusion in the Work 134 | by You to the Licensor shall be under the terms and conditions of 135 | this License, without any additional terms or conditions. 136 | Notwithstanding the above, nothing herein shall supersede or modify 137 | the terms of any separate license agreement you may have executed 138 | with Licensor regarding such Contributions. 139 | 140 | 6. Trademarks. This License does not grant permission to use the trade 141 | names, trademarks, service marks, or product names of the Licensor, 142 | except as required for reasonable and customary use in describing the 143 | origin of the Work and reproducing the content of the NOTICE file. 144 | 145 | 7. Disclaimer of Warranty. Unless required by applicable law or 146 | agreed to in writing, Licensor provides the Work (and each 147 | Contributor provides its Contributions) on an "AS IS" BASIS, 148 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 149 | implied, including, without limitation, any warranties or conditions 150 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 151 | PARTICULAR PURPOSE. You are solely responsible for determining the 152 | appropriateness of using or redistributing the Work and assume any 153 | risks associated with Your exercise of permissions under this License. 154 | 155 | 8. Limitation of Liability. In no event and under no legal theory, 156 | whether in tort (including negligence), contract, or otherwise, 157 | unless required by applicable law (such as deliberate and grossly 158 | negligent acts) or agreed to in writing, shall any Contributor be 159 | liable to You for damages, including any direct, indirect, special, 160 | incidental, or consequential damages of any character arising as a 161 | result of this License or out of the use or inability to use the 162 | Work (including but not limited to damages for loss of goodwill, 163 | work stoppage, computer failure or malfunction, or any and all 164 | other commercial damages or losses), even if such Contributor 165 | has been advised of the possibility of such damages. 166 | 167 | 9. Accepting Warranty or Additional Liability. While redistributing 168 | the Work or Derivative Works thereof, You may choose to offer, 169 | and charge a fee for, acceptance of support, warranty, indemnity, 170 | or other liability obligations and/or rights consistent with this 171 | License. However, in accepting such obligations, You may act only 172 | on Your own behalf and on Your sole responsibility, not on behalf 173 | of any other Contributor, and only if You agree to indemnify, 174 | defend, and hold each Contributor harmless for any liability 175 | incurred by, or claims asserted against, such Contributor by reason 176 | of your accepting any such warranty or additional liability. 177 | 178 | END OF TERMS AND CONDITIONS 179 | 180 | APPENDIX: How to apply the Apache License to your work. 181 | 182 | To apply the Apache License to your work, attach the following 183 | boilerplate notice, with the fields enclosed by brackets "{}" 184 | replaced with your own identifying information. (Don't include 185 | the brackets!) The text should be enclosed in the appropriate 186 | comment syntax for the file format. We also recommend that a 187 | file or class name and description of purpose be included on the 188 | same "printed page" as the copyright notice for easier 189 | identification within third-party archives. 190 | 191 | Copyright {yyyy} {name of copyright owner} 192 | 193 | Licensed under the Apache License, Version 2.0 (the "License"); 194 | you may not use this file except in compliance with the License. 195 | You may obtain a copy of the License at 196 | 197 | http://www.apache.org/licenses/LICENSE-2.0 198 | 199 | Unless required by applicable law or agreed to in writing, software 200 | distributed under the License is distributed on an "AS IS" BASIS, 201 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 202 | See the License for the specific language governing permissions and 203 | limitations under the License. 204 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | VMware has ended active development of this project, this repository will no longer be updated. 2 | VMware Integrated Openstack README 3 | ================================== 4 | 5 | This is a collection of scripts, heat templates, and other useful 6 | tools for working with VMware Integrated Openstack (VIO) 7 | 8 | 9 | -- End of broadcast 10 | -------------------------------------------------------------------------------- /heat/README.rst: -------------------------------------------------------------------------------- 1 | Sample Heat Templates for VIO 2 | ============================= 3 | -------------------------------------------------------------------------------- /heat/nvpnathan/1vm-1lnet-1floatingip.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2013-05-23 2 | 3 | description: > 4 | A simple Heat template that creates a VM, hooks it up to a logical network 5 | and associates a floating ip from an external network to the VM. 6 | 7 | parameters: 8 | 9 | external_network_id: 10 | type: string 11 | label: ext-net 12 | description: UUID of a Neutron external network 13 | default: fdb92e61-75e4-47ef-9b74-557af5b6147d 14 | 15 | flavor_name: 16 | type: string 17 | label: flavor name 18 | description: Name of the flavor to be used for the instances. 19 | default: m1.small 20 | 21 | image_name: 22 | type: string 23 | label: image name 24 | description: Name of the image from which to create the instances. 25 | default: ubuntu-14.04-server-amd64 26 | 27 | resources: 28 | # Create the logical network 29 | lnet_1: 30 | type: OS::Neutron::Net 31 | properties: 32 | admin_state_up: true 33 | name: lnet_1 34 | 35 | subnet_1: 36 | type: OS::Neutron::Subnet 37 | properties: 38 | name: subnet_1 39 | cidr: 10.10.10.0/24 40 | enable_dhcp: true 41 | gateway_ip: 10.10.10.1 42 | network_id: { get_resource: lnet_1 } 43 | 44 | # Create the logical router 45 | router_1: 46 | type: OS::Neutron::Router 47 | properties: 48 | admin_state_up: true 49 | name: router_1 50 | router_1_gw: 51 | type: OS::Neutron::RouterGateway 52 | properties: 53 | network_id: { get_param: external_network_id } 54 | router_id: { get_resource: router_1 } 55 | router1_int1: 56 | type: OS::Neutron::RouterInterface 57 | properties: 58 | router_id: { get_resource: router_1 } 59 | subnet_id: { get_resource: subnet_1 } 60 | 61 | instance1_port1: 62 | type: OS::Neutron::Port 63 | properties: 64 | admin_state_up: true 65 | network_id: { get_resource: lnet_1 } 66 | security_groups: 67 | - default 68 | 69 | # Create the instance 70 | instance1: 71 | type: OS::Nova::Server 72 | properties: 73 | name: instance1 74 | image: { get_param: image_name } 75 | flavor: { get_param: flavor_name } 76 | networks: 77 | - port: { get_resource: instance1_port1 } 78 | 79 | # Assign the floating IP to the instance 80 | instance1_floating_ip: 81 | type: OS::Neutron::FloatingIP 82 | properties: 83 | floating_network_id: { get_param: external_network_id } 84 | port_id: { get_resource: instance1_port1 } 85 | 86 | outputs: 87 | instance1_private_ip: 88 | description: Fixed IP address of instance1 on the private network. 89 | value: { get_attr: [ instance1, first_address] } 90 | instance1_floating_ip: 91 | description: Floating IP address of instance1 on external network. 92 | value: { get_attr: [ instance1_floating_ip, floating_ip_address] } 93 | 94 | -------------------------------------------------------------------------------- /heat/nvpnathan/LBaaS-DB-APP-Affinity.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2013-05-23 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Load Balancer, 2 Web Servers, 1 App Server, 5 | and 1 DB Server on logical networks and attaches them to security groups (HOT template in YAML). 6 | 7 | parameters: 8 | external_network_id: 9 | type: string 10 | label: ext-net 11 | description: UUID of a Neutron external network 12 | default: fdb92e61-75e4-47ef-9b74-557af5b6147d 13 | 14 | web_image: 15 | type: string 16 | description: Name of image to use for servers 17 | default: ubuntu-14.04-server-amd64 18 | 19 | app_image: 20 | type: string 21 | description: Name of image to use for servers 22 | default: ubuntu-14.04-server-amd64 23 | 24 | db_image: 25 | type: string 26 | description: Name of image to use for servers 27 | default: ubuntu-14.04-server-amd64 28 | 29 | 30 | resources: 31 | # Create the web logical switch and configure DHCP. 32 | web_network_01: 33 | type: OS::Neutron::Net 34 | properties: 35 | admin_state_up: true 36 | name: web-network-01 37 | web_subnet_01: 38 | type: OS::Neutron::Subnet 39 | properties: 40 | name: web-subnet-01 41 | cidr: 172.16.10.0/24 42 | dns_nameservers: [192.168.110.10] 43 | enable_dhcp: true 44 | gateway_ip: 172.16.10.1 45 | network_id: { get_resource: web_network_01 } 46 | 47 | # Create the app logical switch and configure DHCP. 48 | app_network_01: 49 | type: OS::Neutron::Net 50 | properties: 51 | admin_state_up: true 52 | name: app-network-01 53 | app_subnet_01: 54 | type: OS::Neutron::Subnet 55 | properties: 56 | name: app-subnet-01 57 | cidr: 172.16.20.0/24 58 | dns_nameservers: [192.168.110.10] 59 | enable_dhcp: true 60 | gateway_ip: 172.16.20.1 61 | network_id: { get_resource: app_network_01 } 62 | 63 | # Create the db logical switch and configure DHCP. 64 | db_network_01: 65 | type: OS::Neutron::Net 66 | properties: 67 | admin_state_up: true 68 | name: db-network-01 69 | db_subnet_01: 70 | type: OS::Neutron::Subnet 71 | properties: 72 | name: db-subnet-01 73 | cidr: 172.16.30.0/24 74 | dns_nameservers: [192.168.110.10] 75 | enable_dhcp: true 76 | gateway_ip: 172.16.30.1 77 | network_id: { get_resource: db_network_01 } 78 | 79 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 80 | heat_router_01: 81 | type: OS::Neutron::Router 82 | properties: 83 | admin_state_up: true 84 | name: heat-router-01 85 | value_specs: {router_type: exclusive} 86 | heat_router_01_gw: 87 | type: OS::Neutron::RouterGateway 88 | properties: 89 | network_id: {get_param: external_network_id} 90 | router_id: { get_resource: heat_router_01 } 91 | heat_router_int0: 92 | type: OS::Neutron::RouterInterface 93 | properties: 94 | router_id: { get_resource: heat_router_01 } 95 | subnet_id: { get_resource: web_subnet_01 } 96 | heat_router_int1: 97 | type: OS::Neutron::RouterInterface 98 | properties: 99 | router_id: { get_resource: heat_router_01 } 100 | subnet_id: { get_resource: app_subnet_01 } 101 | heat_router_int2: 102 | type: OS::Neutron::RouterInterface 103 | properties: 104 | router_id: { get_resource: heat_router_01 } 105 | subnet_id: { get_resource: db_subnet_01 } 106 | 107 | # Create security groups for the tiers. 108 | db_security_group: 109 | type: OS::Neutron::SecurityGroup 110 | properties: 111 | description: Allow sql traffic from web tier 112 | name: db-security-group 113 | rules: 114 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 115 | protocol: tcp 116 | port_range_min: 3306 117 | port_range_max: 3306 118 | 119 | web_security_group: 120 | type: OS::Neutron::SecurityGroup 121 | properties: 122 | description: Allow web traffic from anywhere 123 | name: web-security-group 124 | rules: 125 | - remote_ip_prefix: 0.0.0.0/0 126 | protocol: tcp 127 | port_range_min: 80 128 | port_range_max: 80 129 | - remote_ip_prefix: 0.0.0.0/0 130 | protocol: tcp 131 | port_range_min: 22 132 | port_range_max: 22 133 | 134 | app_security_group: 135 | type: OS::Neutron::SecurityGroup 136 | properties: 137 | description: Allow app traffic from web tier 138 | name: app-security-group 139 | rules: 140 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 141 | protocol: tcp 142 | port_range_min: 8443 143 | port_range_max: 8443 144 | 145 | # Create ports on web switch. 146 | web-svr-01_port0: 147 | type: OS::Neutron::Port 148 | properties: 149 | admin_state_up: true 150 | network_id: { get_resource: web_network_01 } 151 | security_groups: 152 | - { get_resource: web_security_group } 153 | web-svr-01_floating_ip: 154 | type: OS::Neutron::FloatingIP 155 | properties: 156 | floating_network_id: {get_param: external_network_id} 157 | port_id: { get_resource: web-svr-01_port0 } 158 | 159 | web-svr-02_port0: 160 | type: OS::Neutron::Port 161 | properties: 162 | admin_state_up: true 163 | network_id: { get_resource: web_network_01 } 164 | security_groups: 165 | - { get_resource: web_security_group } 166 | web-svr-02_floating_ip: 167 | type: OS::Neutron::FloatingIP 168 | properties: 169 | floating_network_id: {get_param: external_network_id} 170 | port_id: { get_resource: web-svr-02_port0 } 171 | 172 | # Create port(s) on app switch. 173 | app-svr-01_port0: 174 | type: OS::Neutron::Port 175 | properties: 176 | admin_state_up: true 177 | network_id: { get_resource: app_network_01 } 178 | security_groups: 179 | - { get_resource: app_security_group } 180 | 181 | # Create port(s) on db switch. 182 | db-svr-01_port0: 183 | type: OS::Neutron::Port 184 | properties: 185 | admin_state_up: true 186 | network_id: { get_resource: db_network_01 } 187 | security_groups: 188 | - { get_resource: db_security_group } 189 | 190 | # Create the Affinity Group 191 | policy_group: 192 | type: OS::Nova::ServerGroup 193 | properties: 194 | name: db-app-affinity 195 | policies: [affinity] 196 | 197 | # Provision instances. 198 | db-svr-01: 199 | type: OS::Nova::Server 200 | properties: 201 | name: db-instance-01 202 | image: { get_param: db_image } 203 | flavor: m1.small 204 | scheduler_hints: {group: {get_resource: policy_group}} 205 | networks: 206 | - port: { get_resource: db-svr-01_port0 } 207 | 208 | app-svr-01: 209 | type: OS::Nova::Server 210 | depends_on: db-svr-01 211 | properties: 212 | name: app-instance-01 213 | image: { get_param: app_image } 214 | flavor: m1.small 215 | scheduler_hints: {group: {get_resource: policy_group}} 216 | networks: 217 | - port: { get_resource: app-svr-01_port0 } 218 | user_data: 219 | str_replace: 220 | params: 221 | __db_ipaddr__: { get_attr: [db-svr-01, first_address] } 222 | template: | 223 | #!/bin/bash 224 | sed -i 's/##HOT_DB_SERVER##/__db_ipaddr__/' /var/www/cgi-bin/nsx-webapp.cgi 225 | 226 | web-svr-01: 227 | type: OS::Nova::Server 228 | depends_on: app-svr-01 229 | properties: 230 | name: web-instance-01 231 | image: { get_param: web_image } 232 | flavor: m1.small 233 | networks: 234 | - port: { get_resource: web-svr-01_port0 } 235 | user_data: 236 | str_replace: 237 | params: 238 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 239 | template: | 240 | #!/bin/bash 241 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 242 | service apache2 restart 243 | 244 | web-svr-02: 245 | type: OS::Nova::Server 246 | depends_on: app-svr-01 247 | properties: 248 | name: web-instance-02 249 | image: { get_param: web_image } 250 | flavor: m1.small 251 | networks: 252 | - port: { get_resource: web-svr-02_port0 } 253 | user_data: 254 | str_replace: 255 | params: 256 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 257 | template: | 258 | #!/bin/bash 259 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 260 | service apache2 restart 261 | 262 | monitor: 263 | type: OS::Neutron::HealthMonitor 264 | properties: 265 | type: TCP 266 | delay: 10 267 | max_retries: 5 268 | timeout: 10 269 | 270 | pool: 271 | type: OS::Neutron::Pool 272 | depends_on: heat_router_int0 273 | properties: 274 | name: mypool 275 | protocol: HTTP 276 | monitors: [{get_resource: monitor}] 277 | subnet_id: {get_resource: web_subnet_01} 278 | lb_method: ROUND_ROBIN 279 | vip: 280 | protocol_port: 80 281 | 282 | lb: 283 | type: OS::Neutron::LoadBalancer 284 | properties: 285 | members: [ { get_resource: web-svr-01 }, { get_resource: web-svr-02 } ] 286 | protocol_port: 80 287 | pool_id: { get_resource: pool } 288 | 289 | lb_floating: 290 | type: OS::Neutron::FloatingIP 291 | properties: 292 | floating_network_id: { get_param: external_network_id } 293 | port_id: { get_attr: [ pool, vip, port_id ] } 294 | 295 | outputs: 296 | lb_vip_ip: 297 | description: LB VIP 298 | value: { get_attr: [ lb_floating, floating_ip_address ] } 299 | web-svr-01_public_ip: 300 | description: Floating IP address of web-svr-01 in public network 301 | value: { get_attr: [ web-svr-01_floating_ip, floating_ip_address ] } 302 | web-svr-02_public_ip: 303 | description: Floating IP address of web-svr-01 in public network 304 | value: { get_attr: [ web-svr-02_floating_ip, floating_ip_address ] } 305 | -------------------------------------------------------------------------------- /heat/nvpnathan/LBaaS-Three-Tier.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2013-05-23 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Web Server, 1 App Server, 5 | and 1 DB Server on logical networks and attaches them to security groups (HOT template in YAML). 6 | 7 | parameters: 8 | external_network_id: 9 | type: string 10 | label: ext-net 11 | description: UUID of a Neutron external network 12 | default: 7092826e-d36b-46af-953b-fe587444a6c8 13 | 14 | web_image: 15 | type: string 16 | description: Name of image to use for servers 17 | default: ubuntu-14.04-server-amd64 18 | 19 | app_image: 20 | type: string 21 | description: Name of image to use for servers 22 | default: ubuntu-14.04-server-amd64 23 | 24 | db_image: 25 | type: string 26 | description: Name of image to use for servers 27 | default: ubuntu-14.04-server-amd64 28 | 29 | 30 | resources: 31 | # Create the web logical switch and configure DHCP. 32 | web_network_01: 33 | type: OS::Neutron::Net 34 | properties: 35 | admin_state_up: true 36 | name: web-network-01 37 | web_subnet_01: 38 | type: OS::Neutron::Subnet 39 | properties: 40 | name: web-subnet-01 41 | cidr: 172.16.10.0/24 42 | dns_nameservers: [192.168.110.10] 43 | enable_dhcp: true 44 | gateway_ip: 172.16.10.1 45 | network_id: { get_resource: web_network_01 } 46 | 47 | # Create the app logical switch and configure DHCP. 48 | app_network_01: 49 | type: OS::Neutron::Net 50 | properties: 51 | admin_state_up: true 52 | name: app-network-01 53 | app_subnet_01: 54 | type: OS::Neutron::Subnet 55 | properties: 56 | name: app-subnet-01 57 | cidr: 172.16.20.0/24 58 | dns_nameservers: [192.168.110.10] 59 | enable_dhcp: true 60 | gateway_ip: 172.16.20.1 61 | network_id: { get_resource: app_network_01 } 62 | 63 | # Create the db logical switch and configure DHCP. 64 | db_network_01: 65 | type: OS::Neutron::Net 66 | properties: 67 | admin_state_up: true 68 | name: db-network-01 69 | db_subnet_01: 70 | type: OS::Neutron::Subnet 71 | properties: 72 | name: db-subnet-01 73 | cidr: 172.16.30.0/24 74 | dns_nameservers: [192.168.110.10] 75 | enable_dhcp: true 76 | gateway_ip: 172.16.30.1 77 | network_id: { get_resource: db_network_01 } 78 | 79 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 80 | heat_router_01: 81 | type: OS::Neutron::Router 82 | properties: 83 | admin_state_up: true 84 | name: heat-router-01 85 | value_specs: {router_type: exclusive} 86 | heat_router_01_gw: 87 | type: OS::Neutron::RouterGateway 88 | properties: 89 | network_id: {get_param: external_network_id} 90 | router_id: { get_resource: heat_router_01 } 91 | heat_router_int0: 92 | type: OS::Neutron::RouterInterface 93 | properties: 94 | router_id: { get_resource: heat_router_01 } 95 | subnet_id: { get_resource: web_subnet_01 } 96 | heat_router_int1: 97 | type: OS::Neutron::RouterInterface 98 | properties: 99 | router_id: { get_resource: heat_router_01 } 100 | subnet_id: { get_resource: app_subnet_01 } 101 | heat_router_int2: 102 | type: OS::Neutron::RouterInterface 103 | properties: 104 | router_id: { get_resource: heat_router_01 } 105 | subnet_id: { get_resource: db_subnet_01 } 106 | 107 | # Create security groups for the tiers. 108 | db_security_group: 109 | type: OS::Neutron::SecurityGroup 110 | properties: 111 | description: Allow sql traffic from web tier 112 | name: db-security-group 113 | rules: 114 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 115 | protocol: tcp 116 | port_range_min: 3306 117 | port_range_max: 3306 118 | 119 | web_security_group: 120 | type: OS::Neutron::SecurityGroup 121 | properties: 122 | description: Allow web traffic from anywhere 123 | name: web-security-group 124 | rules: 125 | - remote_ip_prefix: 0.0.0.0/0 126 | protocol: tcp 127 | port_range_min: 80 128 | port_range_max: 80 129 | - remote_ip_prefix: 0.0.0.0/0 130 | protocol: tcp 131 | port_range_min: 22 132 | port_range_max: 22 133 | 134 | app_security_group: 135 | type: OS::Neutron::SecurityGroup 136 | properties: 137 | description: Allow app traffic from web tier 138 | name: app-security-group 139 | rules: 140 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 141 | protocol: tcp 142 | port_range_min: 8443 143 | port_range_max: 8443 144 | 145 | # Create ports on web switch. 146 | web-svr-01_port0: 147 | type: OS::Neutron::Port 148 | properties: 149 | admin_state_up: true 150 | network_id: { get_resource: web_network_01 } 151 | security_groups: 152 | - { get_resource: web_security_group } 153 | web-svr-01_floating_ip: 154 | type: OS::Neutron::FloatingIP 155 | properties: 156 | floating_network_id: {get_param: external_network_id} 157 | port_id: { get_resource: web-svr-01_port0 } 158 | 159 | web-svr-02_port0: 160 | type: OS::Neutron::Port 161 | properties: 162 | admin_state_up: true 163 | network_id: { get_resource: web_network_01 } 164 | security_groups: 165 | - { get_resource: web_security_group } 166 | web-svr-02_floating_ip: 167 | type: OS::Neutron::FloatingIP 168 | properties: 169 | floating_network_id: {get_param: external_network_id} 170 | port_id: { get_resource: web-svr-02_port0 } 171 | 172 | # Create port(s) on app switch. 173 | app-svr-01_port0: 174 | type: OS::Neutron::Port 175 | properties: 176 | admin_state_up: true 177 | network_id: { get_resource: app_network_01 } 178 | security_groups: 179 | - { get_resource: app_security_group } 180 | 181 | # Create port(s) on db switch. 182 | db-svr-01_port0: 183 | type: OS::Neutron::Port 184 | properties: 185 | admin_state_up: true 186 | network_id: { get_resource: db_network_01 } 187 | security_groups: 188 | - { get_resource: db_security_group } 189 | 190 | # Provision instances. 191 | db-svr-01: 192 | type: OS::Nova::Server 193 | properties: 194 | name: db-instance-01 195 | image: { get_param: db_image } 196 | flavor: m1.small 197 | networks: 198 | - port: { get_resource: db-svr-01_port0 } 199 | 200 | app-svr-01: 201 | type: OS::Nova::Server 202 | depends_on: db-svr-01 203 | properties: 204 | name: app-instance-01 205 | image: { get_param: app_image } 206 | flavor: m1.small 207 | networks: 208 | - port: { get_resource: app-svr-01_port0 } 209 | user_data: 210 | str_replace: 211 | params: 212 | __db_ipaddr__: { get_attr: [db-svr-01, first_address] } 213 | template: | 214 | #!/bin/bash 215 | sed -i 's/##HOT_DB_SERVER##/__db_ipaddr__/' /var/www/cgi-bin/nsx-webapp.cgi 216 | 217 | web-svr-01: 218 | type: OS::Nova::Server 219 | depends_on: app-svr-01 220 | properties: 221 | name: web-instance-01 222 | image: { get_param: web_image } 223 | flavor: m1.small 224 | networks: 225 | - port: { get_resource: web-svr-01_port0 } 226 | user_data: 227 | str_replace: 228 | params: 229 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 230 | template: | 231 | #!/bin/bash 232 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 233 | service apache2 restart 234 | 235 | web-svr-02: 236 | type: OS::Nova::Server 237 | depends_on: app-svr-01 238 | properties: 239 | name: web-instance-02 240 | image: { get_param: web_image } 241 | flavor: m1.small 242 | networks: 243 | - port: { get_resource: web-svr-02_port0 } 244 | user_data: 245 | str_replace: 246 | params: 247 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 248 | template: | 249 | #!/bin/bash 250 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 251 | service apache2 restart 252 | 253 | monitor: 254 | type: OS::Neutron::HealthMonitor 255 | properties: 256 | type: TCP 257 | delay: 10 258 | max_retries: 5 259 | timeout: 10 260 | 261 | pool: 262 | type: OS::Neutron::Pool 263 | depends_on: heat_router_int0 264 | properties: 265 | name: mypool 266 | protocol: HTTP 267 | monitors: [{get_resource: monitor}] 268 | subnet_id: {get_resource: web_subnet_01} 269 | lb_method: ROUND_ROBIN 270 | vip: 271 | protocol_port: 80 272 | 273 | lb: 274 | type: OS::Neutron::LoadBalancer 275 | properties: 276 | members: [ { get_resource: web-svr-01 }, { get_resource: web-svr-02 } ] 277 | protocol_port: 80 278 | pool_id: { get_resource: pool } 279 | 280 | lb_floating: 281 | type: OS::Neutron::FloatingIP 282 | properties: 283 | floating_network_id: { get_param: external_network_id } 284 | port_id: { get_attr: [ pool, vip, port_id ] } 285 | 286 | outputs: 287 | lb_vip_ip: 288 | description: LB VIP 289 | value: { get_attr: [ lb_floating, floating_ip_address ] } 290 | web-svr-01_public_ip: 291 | description: Floating IP address of web-svr-01 in public network 292 | value: { get_attr: [ web-svr-01_floating_ip, floating_ip_address ] } 293 | web-svr-02_public_ip: 294 | description: Floating IP address of web-svr-01 in public network 295 | value: { get_attr: [ web-svr-02_floating_ip, floating_ip_address ] } 296 | -------------------------------------------------------------------------------- /heat/nvpnathan/LBaaS-Web-Anti-Affinity.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2013-05-23 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Load Balancer, 2 Web Servers, 1 App Server, 5 | and 1 DB Server on logical networks and attaches them to security groups (HOT template in YAML). 6 | 7 | parameters: 8 | external_network_id: 9 | type: string 10 | label: ext-net 11 | description: UUID of a Neutron external network 12 | default: fdb92e61-75e4-47ef-9b74-557af5b6147d 13 | 14 | web_image: 15 | type: string 16 | description: Name of image to use for servers 17 | default: ubuntu-14.04-server-amd64 18 | 19 | app_image: 20 | type: string 21 | description: Name of image to use for servers 22 | default: ubuntu-14.04-server-amd64 23 | 24 | db_image: 25 | type: string 26 | description: Name of image to use for servers 27 | default: ubuntu-14.04-server-amd64 28 | 29 | 30 | resources: 31 | # Create the web logical switch and configure DHCP. 32 | web_network_01: 33 | type: OS::Neutron::Net 34 | properties: 35 | admin_state_up: true 36 | name: web-network-01 37 | web_subnet_01: 38 | type: OS::Neutron::Subnet 39 | properties: 40 | name: web-subnet-01 41 | cidr: 172.16.10.0/24 42 | dns_nameservers: [192.168.110.10] 43 | enable_dhcp: true 44 | gateway_ip: 172.16.10.1 45 | network_id: { get_resource: web_network_01 } 46 | 47 | # Create the app logical switch and configure DHCP. 48 | app_network_01: 49 | type: OS::Neutron::Net 50 | properties: 51 | admin_state_up: true 52 | name: app-network-01 53 | app_subnet_01: 54 | type: OS::Neutron::Subnet 55 | properties: 56 | name: app-subnet-01 57 | cidr: 172.16.20.0/24 58 | dns_nameservers: [192.168.110.10] 59 | enable_dhcp: true 60 | gateway_ip: 172.16.20.1 61 | network_id: { get_resource: app_network_01 } 62 | 63 | # Create the db logical switch and configure DHCP. 64 | db_network_01: 65 | type: OS::Neutron::Net 66 | properties: 67 | admin_state_up: true 68 | name: db-network-01 69 | db_subnet_01: 70 | type: OS::Neutron::Subnet 71 | properties: 72 | name: db-subnet-01 73 | cidr: 172.16.30.0/24 74 | dns_nameservers: [192.168.110.10] 75 | enable_dhcp: true 76 | gateway_ip: 172.16.30.1 77 | network_id: { get_resource: db_network_01 } 78 | 79 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 80 | heat_router_01: 81 | type: OS::Neutron::Router 82 | properties: 83 | admin_state_up: true 84 | name: heat-router-01 85 | value_specs: {router_type: exclusive} 86 | heat_router_01_gw: 87 | type: OS::Neutron::RouterGateway 88 | properties: 89 | network_id: {get_param: external_network_id} 90 | router_id: { get_resource: heat_router_01 } 91 | heat_router_int0: 92 | type: OS::Neutron::RouterInterface 93 | properties: 94 | router_id: { get_resource: heat_router_01 } 95 | subnet_id: { get_resource: web_subnet_01 } 96 | heat_router_int1: 97 | type: OS::Neutron::RouterInterface 98 | properties: 99 | router_id: { get_resource: heat_router_01 } 100 | subnet_id: { get_resource: app_subnet_01 } 101 | heat_router_int2: 102 | type: OS::Neutron::RouterInterface 103 | properties: 104 | router_id: { get_resource: heat_router_01 } 105 | subnet_id: { get_resource: db_subnet_01 } 106 | 107 | # Create security groups for the tiers. 108 | db_security_group: 109 | type: OS::Neutron::SecurityGroup 110 | properties: 111 | description: Allow sql traffic from web tier 112 | name: db-security-group 113 | rules: 114 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 115 | protocol: tcp 116 | port_range_min: 3306 117 | port_range_max: 3306 118 | 119 | web_security_group: 120 | type: OS::Neutron::SecurityGroup 121 | properties: 122 | description: Allow web traffic from anywhere 123 | name: web-security-group 124 | rules: 125 | - remote_ip_prefix: 0.0.0.0/0 126 | protocol: tcp 127 | port_range_min: 80 128 | port_range_max: 80 129 | - remote_ip_prefix: 0.0.0.0/0 130 | protocol: tcp 131 | port_range_min: 22 132 | port_range_max: 22 133 | 134 | app_security_group: 135 | type: OS::Neutron::SecurityGroup 136 | properties: 137 | description: Allow app traffic from web tier 138 | name: app-security-group 139 | rules: 140 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 141 | protocol: tcp 142 | port_range_min: 8443 143 | port_range_max: 8443 144 | 145 | # Create ports on web switch. 146 | web-svr-01_port0: 147 | type: OS::Neutron::Port 148 | properties: 149 | admin_state_up: true 150 | network_id: { get_resource: web_network_01 } 151 | security_groups: 152 | - { get_resource: web_security_group } 153 | web-svr-01_floating_ip: 154 | type: OS::Neutron::FloatingIP 155 | properties: 156 | floating_network_id: {get_param: external_network_id} 157 | port_id: { get_resource: web-svr-01_port0 } 158 | 159 | web-svr-02_port0: 160 | type: OS::Neutron::Port 161 | properties: 162 | admin_state_up: true 163 | network_id: { get_resource: web_network_01 } 164 | security_groups: 165 | - { get_resource: web_security_group } 166 | web-svr-02_floating_ip: 167 | type: OS::Neutron::FloatingIP 168 | properties: 169 | floating_network_id: {get_param: external_network_id} 170 | port_id: { get_resource: web-svr-02_port0 } 171 | 172 | # Create port(s) on app switch. 173 | app-svr-01_port0: 174 | type: OS::Neutron::Port 175 | properties: 176 | admin_state_up: true 177 | network_id: { get_resource: app_network_01 } 178 | security_groups: 179 | - { get_resource: app_security_group } 180 | 181 | # Create port(s) on db switch. 182 | db-svr-01_port0: 183 | type: OS::Neutron::Port 184 | properties: 185 | admin_state_up: true 186 | network_id: { get_resource: db_network_01 } 187 | security_groups: 188 | - { get_resource: db_security_group } 189 | 190 | # Create the Anti-affinity Group 191 | policy_group: 192 | type: OS::Nova::ServerGroup 193 | properties: 194 | name: web-anti-affinity 195 | policies: [anti-affinity] 196 | 197 | # Provision instances. 198 | db-svr-01: 199 | type: OS::Nova::Server 200 | properties: 201 | name: db-instance-01 202 | image: { get_param: db_image } 203 | flavor: m1.small 204 | networks: 205 | - port: { get_resource: db-svr-01_port0 } 206 | 207 | app-svr-01: 208 | type: OS::Nova::Server 209 | depends_on: db-svr-01 210 | properties: 211 | name: app-instance-01 212 | image: { get_param: app_image } 213 | flavor: m1.small 214 | networks: 215 | - port: { get_resource: app-svr-01_port0 } 216 | user_data: 217 | str_replace: 218 | params: 219 | __db_ipaddr__: { get_attr: [db-svr-01, first_address] } 220 | template: | 221 | #!/bin/bash 222 | sed -i 's/##HOT_DB_SERVER##/__db_ipaddr__/' /var/www/cgi-bin/nsx-webapp.cgi 223 | 224 | web-svr-01: 225 | type: OS::Nova::Server 226 | depends_on: app-svr-01 227 | properties: 228 | name: web-instance-01 229 | image: { get_param: web_image } 230 | flavor: m1.small 231 | scheduler_hints: {group: {get_resource: policy_group}} 232 | networks: 233 | - port: { get_resource: web-svr-01_port0 } 234 | user_data: 235 | str_replace: 236 | params: 237 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 238 | template: | 239 | #!/bin/bash 240 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 241 | service apache2 restart 242 | 243 | web-svr-02: 244 | type: OS::Nova::Server 245 | depends_on: app-svr-01 246 | properties: 247 | name: web-instance-02 248 | image: { get_param: web_image } 249 | flavor: m1.small 250 | scheduler_hints: {group: {get_resource: policy_group}} 251 | networks: 252 | - port: { get_resource: web-svr-02_port0 } 253 | user_data: 254 | str_replace: 255 | params: 256 | __app_ipaddr__: { get_attr: [ app-svr-01, first_address ] } 257 | template: | 258 | #!/bin/bash 259 | sed -i 's/##HOT_APP_SERVER##/__app_ipaddr__/g' /etc/apache2/sites-enabled/000-default.conf 260 | service apache2 restart 261 | 262 | monitor: 263 | type: OS::Neutron::HealthMonitor 264 | properties: 265 | type: TCP 266 | delay: 10 267 | max_retries: 5 268 | timeout: 10 269 | 270 | pool: 271 | type: OS::Neutron::Pool 272 | depends_on: heat_router_int0 273 | properties: 274 | name: mypool 275 | protocol: HTTP 276 | monitors: [{get_resource: monitor}] 277 | subnet_id: {get_resource: web_subnet_01} 278 | lb_method: ROUND_ROBIN 279 | vip: 280 | protocol_port: 80 281 | 282 | lb: 283 | type: OS::Neutron::LoadBalancer 284 | properties: 285 | members: [ { get_resource: web-svr-01 }, { get_resource: web-svr-02 } ] 286 | protocol_port: 80 287 | pool_id: { get_resource: pool } 288 | 289 | lb_floating: 290 | type: OS::Neutron::FloatingIP 291 | properties: 292 | floating_network_id: { get_param: external_network_id } 293 | port_id: { get_attr: [ pool, vip, port_id ] } 294 | 295 | outputs: 296 | lb_vip_ip: 297 | description: LB VIP 298 | value: { get_attr: [ lb_floating, floating_ip_address ] } 299 | web-svr-01_public_ip: 300 | description: Floating IP address of web-svr-01 in public network 301 | value: { get_attr: [ web-svr-01_floating_ip, floating_ip_address ] } 302 | web-svr-02_public_ip: 303 | description: Floating IP address of web-svr-01 in public network 304 | value: { get_attr: [ web-svr-02_floating_ip, floating_ip_address ] } 305 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/1-tier_photon_central-routing_nat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | Sample heat template to deploy container into Photon instance. This template requires external internet access 5 | to pull the container for the template. Please ensure DNS settings are correct. 6 | 7 | parameters: 8 | 9 | external_network_id: 10 | type: string 11 | label: ext-net 12 | description: UUID of a Neutron external network 13 | default: fdb92e61-75e4-47ef-9b74-557af5b6147d 14 | 15 | container_image: 16 | type: string 17 | description: Name of image to use for servers 18 | default: photon 19 | 20 | container_name: 21 | type: string 22 | description: Name of Docker Hub container to use 23 | default: vmwarecna/nginx 24 | 25 | flavor: 26 | type: string 27 | label: Flavor 28 | description: Type of instance (flavor) to be used on the compute instance. 29 | default: m1.small 30 | 31 | dns: 32 | type: string 33 | label: DNS nameserver 34 | description: Comma separated list of DNS nameservers for the private network 35 | default: "8.8.8.8" 36 | 37 | resources: 38 | # Create the web logical switch and configure DHCP. 39 | container_net0: 40 | type: OS::Neutron::Net 41 | properties: 42 | admin_state_up: true 43 | name: container-net0 44 | container_subnet_0: 45 | type: OS::Neutron::Subnet 46 | properties: 47 | name: container-subnet0 48 | cidr: 192.168.41.0/24 49 | dns_nameservers: [ { get_param: dns } ] 50 | enable_dhcp: true 51 | gateway_ip: 192.168.41.1 52 | network_id: { get_resource: container_net0 } 53 | # Create the router, add internal container interface, and add router uplink 54 | app_router_01: 55 | type: OS::Neutron::Router 56 | properties: 57 | admin_state_up: true 58 | name: app-router-01 59 | external_gateway_info: { "network": { get_param: external_network_id }} 60 | app_router_int0: 61 | type: OS::Neutron::RouterInterface 62 | properties: 63 | router_id: { get_resource: app_router_01 } 64 | subnet_id: { get_resource: container_subnet_0 } 65 | # Create the containers security group 66 | container_security_group: 67 | type: OS::Neutron::SecurityGroup 68 | properties: 69 | description: Allow web and SSH traffic from anywhere 70 | name: container-security-group 71 | rules: 72 | - remote_ip_prefix: 0.0.0.0/0 73 | protocol: tcp 74 | port_range_min: 80 75 | port_range_max: 80 76 | - remote_ip_prefix: 0.0.0.0/0 77 | protocol: tcp 78 | port_range_min: 22 79 | port_range_max: 22 80 | - remote_ip_prefix: 0.0.0.0/0 81 | protocol: icmp 82 | 83 | # Create port(s) on container logical network. 84 | container0_port0: 85 | type: OS::Neutron::Port 86 | properties: 87 | admin_state_up: true 88 | network_id: { get_resource: container_net0 } 89 | security_groups: 90 | - { get_resource: container_security_group } 91 | # Provision instance 92 | container0: 93 | type: OS::Nova::Server 94 | properties: 95 | name: Photon-Container0 96 | image: { get_param: container_image } 97 | flavor: { get_param: flavor } 98 | networks: 99 | - port: { get_resource: container0_port0 } 100 | user_data_format: RAW 101 | user_data: 102 | str_replace: 103 | params: 104 | __container__: { get_param: container_name } 105 | template: | 106 | #!/bin/bash 107 | set -ux 108 | 109 | docker pull __container__ 110 | docker run -d -p 80:80 __container__ 111 | # Attach floating IP 112 | container0_floating_ip: 113 | type: OS::Neutron::FloatingIP 114 | properties: 115 | floating_network_id: { get_param: external_network_id } 116 | port_id: { get_resource: container0_port0 } 117 | 118 | outputs: 119 | WebsiteURL: 120 | description: URL for the Container 121 | value: 122 | str_replace: 123 | template: http://host/ 124 | params: 125 | host: { get_attr: [ container0_floating_ip, floating_ip_address ] } 126 | 127 | container0_public_ip: 128 | description: Floating IP address of Container0 on the public network 129 | value: { get_attr: [ container0_floating_ip, floating_ip_address ] } 130 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/2-tier_exclusive-routing-lb_nat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | Sample heat template that spins up a 2 Tier App with 2 Web Servers and 1 DB server on private networks and attaches them to security 5 | groups and a Loadbalancer. Template is meant to be run on ubuntu for user-date to work properly. 6 | 7 | parameters: 8 | 9 | external_network_id: 10 | type: string 11 | label: ext-net 12 | description: UUID of a Neutron external network 13 | default: fdb92e61-75e4-47ef-9b74-557af5b6147d 14 | 15 | db_image: 16 | type: string 17 | description: Name of image to use for servers 18 | default: ubuntu-14.04-server-amd64 19 | 20 | web_image: 21 | type: string 22 | description: Name of image to use for servers 23 | default: ubuntu-14.04-server-amd64 24 | 25 | flavor: 26 | type: string 27 | label: Flavor 28 | description: Type of instance (flavor) to be used on the compute instance. 29 | default: m1.small 30 | 31 | dns: 32 | type: string 33 | label: DNS nameserver 34 | description: Comma separated list of DNS nameservers for the private network 35 | default: "8.8.8.8" 36 | 37 | resources: 38 | # Create the db logical switch and configure DHCP. 39 | db_net: 40 | type: OS::Neutron::Net 41 | properties: 42 | admin_state_up: true 43 | name: db-net 44 | db_subnet: 45 | type: OS::Neutron::Subnet 46 | properties: 47 | name: db-subnet 48 | cidr: 172.16.30.0/24 49 | dns_nameservers: [ { get_param: dns } ] 50 | enable_dhcp: true 51 | gateway_ip: 172.16.30.1 52 | network_id: { get_resource: db_net } 53 | 54 | # Create the web logical switch and configure DHCP. 55 | web_net: 56 | type: OS::Neutron::Net 57 | properties: 58 | admin_state_up: true 59 | name: web-net 60 | web_subnet: 61 | type: OS::Neutron::Subnet 62 | properties: 63 | name: web-subnet 64 | cidr: 172.16.10.0/24 65 | dns_nameservers: [ { get_param: dns } ] 66 | enable_dhcp: true 67 | gateway_ip: 172.16.10.1 68 | network_id: { get_resource: web_net } 69 | # Create the router, add internal db interface, and add router uplink 70 | app_router_01: 71 | type: OS::Neutron::Router 72 | properties: 73 | admin_state_up: true 74 | name: app-router-01 75 | value_specs: {router_type: exclusive} 76 | external_gateway_info: { "network": { get_param: external_network_id }} 77 | app_router_int0: 78 | type: OS::Neutron::RouterInterface 79 | properties: 80 | router_id: { get_resource: app_router_01 } 81 | subnet_id: { get_resource: db_subnet } 82 | app_router_int1: 83 | type: OS::Neutron::RouterInterface 84 | properties: 85 | router_id: { get_resource: app_router_01 } 86 | subnet_id: { get_resource: web_subnet } 87 | 88 | # Create Security Groups 89 | db_security_group: 90 | type: OS::Neutron::SecurityGroup 91 | properties: 92 | description: Allow sql traffic from web tier 93 | name: db-security-group 94 | rules: 95 | - remote_group_id: { get_resource: web_security_group } 96 | remote_mode: remote_group_id 97 | protocol: tcp 98 | port_range_min: 3306 99 | port_range_max: 3306 100 | - remote_ip_prefix: 0.0.0.0/0 101 | protocol: tcp 102 | port_range_min: 22 103 | port_range_max: 22 104 | - remote_ip_prefix: 0.0.0.0/0 105 | protocol: icmp 106 | 107 | web_security_group: 108 | type: OS::Neutron::SecurityGroup 109 | properties: 110 | description: Allow web traffic from anywhere 111 | name: web-security-group 112 | rules: 113 | - remote_ip_prefix: 0.0.0.0/0 114 | protocol: tcp 115 | port_range_min: 80 116 | port_range_max: 80 117 | - remote_ip_prefix: 0.0.0.0/0 118 | protocol: tcp 119 | port_range_min: 22 120 | port_range_max: 22 121 | - remote_ip_prefix: 0.0.0.0/0 122 | protocol: icmp 123 | 124 | # Create port(s) on db logical network. 125 | db1_port0: 126 | type: OS::Neutron::Port 127 | properties: 128 | admin_state_up: true 129 | network_id: { get_resource: db_net } 130 | security_groups: 131 | - { get_resource: db_security_group } 132 | 133 | # Create port(s) on web logical network. 134 | web1_port0: 135 | type: OS::Neutron::Port 136 | properties: 137 | admin_state_up: true 138 | network_id: { get_resource: web_net } 139 | security_groups: 140 | - { get_resource: web_security_group } 141 | 142 | web2_port1: 143 | type: OS::Neutron::Port 144 | properties: 145 | admin_state_up: true 146 | network_id: { get_resource: web_net } 147 | security_groups: 148 | - { get_resource: web_security_group } 149 | 150 | #Provision instances 151 | web1: 152 | type: OS::Nova::Server 153 | properties: 154 | name: web1 155 | image: { get_param: web_image } 156 | flavor: { get_param: flavor } 157 | networks: 158 | - port: { get_resource: web1_port0 } 159 | user_data_format: RAW 160 | user_data: | 161 | #!/bin/bash 162 | echo This is Web Server-01 >> file.txt 163 | python -m SimpleHTTPServer 80 164 | 165 | web2: 166 | type: OS::Nova::Server 167 | properties: 168 | name: web2 169 | image: { get_param: web_image } 170 | flavor: { get_param: flavor } 171 | networks: 172 | - port: { get_resource: web2_port1 } 173 | user_data_format: RAW 174 | user_data: | 175 | #!/bin/bash 176 | echo This is Web Server-02 >> file.txt 177 | python -m SimpleHTTPServer 80 178 | 179 | db1: 180 | type: OS::Nova::Server 181 | properties: 182 | name: db1 183 | image: { get_param: db_image } 184 | flavor: { get_param: flavor } 185 | networks: 186 | - port: { get_resource: db1_port0 } 187 | 188 | # Attach floating IP 189 | db1_floating_ip: 190 | type: OS::Neutron::FloatingIP 191 | properties: 192 | floating_network_id: { get_param: external_network_id } 193 | port_id: { get_resource: db1_port0 } 194 | web1_floating_ip: 195 | type: OS::Neutron::FloatingIP 196 | properties: 197 | floating_network_id: { get_param: external_network_id } 198 | port_id: { get_resource: web1_port0 } 199 | web2_floating_ip: 200 | type: OS::Neutron::FloatingIP 201 | properties: 202 | floating_network_id: { get_param: external_network_id } 203 | port_id: { get_resource: web2_port1 } 204 | 205 | # Configure the LB'er 206 | monitor: 207 | type: OS::Neutron::HealthMonitor 208 | properties: 209 | type: TCP 210 | delay: 10 211 | max_retries: 3 212 | timeout: 10 213 | 214 | pool: 215 | type: OS::Neutron::Pool 216 | depends_on: app_router_int1 217 | properties: 218 | name: mypool 219 | protocol: HTTP 220 | monitors: [{get_resource: monitor}] 221 | subnet_id: { get_resource: web_subnet } 222 | lb_method: ROUND_ROBIN 223 | vip: 224 | protocol_port: 80 225 | 226 | lb: 227 | type: OS::Neutron::LoadBalancer 228 | properties: 229 | members: [ { get_resource: web1 }, { get_resource: web2 } ] 230 | protocol_port: 80 231 | pool_id: { get_resource: pool } 232 | 233 | lb_floating: 234 | type: OS::Neutron::FloatingIP 235 | properties: 236 | floating_network_id: { get_param: external_network_id } 237 | port_id: { get_attr: [ pool, vip, port_id ] } 238 | 239 | outputs: 240 | WebsiteURL: 241 | description: URL for WebServerTier 242 | value: 243 | str_replace: 244 | template: http://host/file.txt 245 | params: 246 | host: { get_attr: [ lb_floating, floating_ip_address ] } 247 | 248 | db1_public_ip: 249 | description: Floating IP address of DB1 on the public network 250 | value: { get_attr: [ db1_floating_ip, floating_ip_address ] } 251 | web1_public_ip: 252 | description: Floating IP address of Web1 on the public network 253 | value: { get_attr: [ web1_floating_ip, floating_ip_address ] } 254 | web2_public_ip: 255 | description: Floating IP address of Web1 on the public network 256 | value: { get_attr: [ web2_floating_ip, floating_ip_address ] } 257 | 258 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/3-tier_central-routing_nat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Web Server, 1 App Server, 5 | and 1 DB Server on private networks and attaches them to security groups (HOT template in YAML). 6 | 7 | parameters: 8 | external_network_id: 9 | type: string 10 | label: ext-net 11 | description: UUID of a Neutron external network 12 | default: 995cf315-8d49-427e-aa16-16051bab37aa 13 | 14 | web_image: 15 | type: string 16 | description: Name of image to use for servers 17 | default: ubuntu-14.04-server-amd64 18 | 19 | app_image: 20 | type: string 21 | description: Name of image to use for servers 22 | default: ubuntu-14.04-server-amd64 23 | 24 | db_image: 25 | type: string 26 | description: Name of image to use for servers 27 | default: ubuntu-14.04-server-amd64 28 | 29 | flavor: 30 | type: string 31 | label: Flavor name 32 | description: Name of the flavor to be used for the instances. 33 | default: m1.small 34 | 35 | resources: 36 | # Create the web logical switch and configure DHCP. 37 | web_network_01: 38 | type: OS::Neutron::Net 39 | properties: 40 | admin_state_up: true 41 | name: web-network-01 42 | web_subnet_01: 43 | type: OS::Neutron::Subnet 44 | properties: 45 | name: web-subnet-01 46 | cidr: 172.16.10.0/24 47 | dns_nameservers: [8.8.8.8] 48 | enable_dhcp: true 49 | gateway_ip: 172.16.10.1 50 | network_id: { get_resource: web_network_01 } 51 | 52 | # Create the app logical switch and configure DHCP. 53 | app_network_01: 54 | type: OS::Neutron::Net 55 | properties: 56 | admin_state_up: true 57 | name: app-network-01 58 | app_subnet_01: 59 | type: OS::Neutron::Subnet 60 | properties: 61 | name: app-subnet-01 62 | cidr: 172.16.20.0/24 63 | dns_nameservers: [8.8.8.8] 64 | enable_dhcp: true 65 | gateway_ip: 172.16.20.1 66 | network_id: { get_resource: app_network_01 } 67 | 68 | # Create the db logical switch and configure DHCP. 69 | db_network_01: 70 | type: OS::Neutron::Net 71 | properties: 72 | admin_state_up: true 73 | name: db-network-01 74 | db_subnet_01: 75 | type: OS::Neutron::Subnet 76 | properties: 77 | name: db-subnet-01 78 | cidr: 172.16.30.0/24 79 | dns_nameservers: [8.8.8.8] 80 | enable_dhcp: true 81 | gateway_ip: 172.16.30.1 82 | network_id: { get_resource: db_network_01 } 83 | 84 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 85 | heat_router_01: 86 | type: OS::Neutron::Router 87 | properties: 88 | name: heat-router-01 89 | admin_state_up: true 90 | external_gateway_info: { "network": { get_param: external_network_id }} 91 | heat_router_int0: 92 | type: OS::Neutron::RouterInterface 93 | properties: 94 | router_id: { get_resource: heat_router_01 } 95 | subnet_id: { get_resource: web_subnet_01 } 96 | heat_router_int1: 97 | type: OS::Neutron::RouterInterface 98 | properties: 99 | router_id: { get_resource: heat_router_01 } 100 | subnet_id: { get_resource: app_subnet_01 } 101 | heat_router_int2: 102 | type: OS::Neutron::RouterInterface 103 | properties: 104 | router_id: { get_resource: heat_router_01 } 105 | subnet_id: { get_resource: db_subnet_01 } 106 | 107 | # Create security groups for the tiers. 108 | db_security_group: 109 | type: OS::Neutron::SecurityGroup 110 | properties: 111 | description: Allow sql traffic from web tier 112 | name: db-security-group 113 | rules: 114 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 115 | protocol: tcp 116 | port_range_min: 3306 117 | port_range_max: 3306 118 | 119 | web_security_group: 120 | type: OS::Neutron::SecurityGroup 121 | properties: 122 | description: Allow web traffic from anywhere 123 | name: web-security-group 124 | rules: 125 | - remote_ip_prefix: 0.0.0.0/0 126 | protocol: tcp 127 | port_range_min: 80 128 | port_range_max: 80 129 | - remote_ip_prefix: 0.0.0.0/0 130 | protocol: tcp 131 | port_range_min: 22 132 | port_range_max: 22 133 | 134 | app_security_group: 135 | type: OS::Neutron::SecurityGroup 136 | properties: 137 | description: Allow app traffic from web tier 138 | name: app-security-group 139 | rules: 140 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 141 | protocol: tcp 142 | port_range_min: 8443 143 | port_range_max: 8443 144 | 145 | # Create ports on web switch. 146 | web-svr-01_port0: 147 | type: OS::Neutron::Port 148 | properties: 149 | admin_state_up: true 150 | network_id: { get_resource: web_network_01 } 151 | security_groups: 152 | - { get_resource: web_security_group } 153 | web-svr-01_floating_ip: 154 | type: OS::Neutron::FloatingIP 155 | properties: 156 | floating_network_id: {get_param: external_network_id} 157 | port_id: { get_resource: web-svr-01_port0 } 158 | 159 | # Create port(s) on app switch. 160 | app-svr-01_port0: 161 | type: OS::Neutron::Port 162 | properties: 163 | admin_state_up: true 164 | network_id: { get_resource: app_network_01 } 165 | security_groups: 166 | - { get_resource: app_security_group } 167 | 168 | # Create port(s) on db switch. 169 | db-svr-01_port0: 170 | type: OS::Neutron::Port 171 | properties: 172 | admin_state_up: true 173 | network_id: { get_resource: db_network_01 } 174 | security_groups: 175 | - { get_resource: db_security_group } 176 | 177 | # Provision instances. 178 | db-svr-01: 179 | type: OS::Nova::Server 180 | properties: 181 | name: db-instance-01 182 | image: { get_param: db_image } 183 | flavor: { get_param: flavor } 184 | networks: 185 | - port: { get_resource: db-svr-01_port0 } 186 | 187 | app-svr-01: 188 | type: OS::Nova::Server 189 | depends_on: db-svr-01 190 | properties: 191 | name: app-instance-01 192 | image: { get_param: app_image } 193 | flavor: { get_param: flavor } 194 | networks: 195 | - port: { get_resource: app-svr-01_port0 } 196 | 197 | web-svr-01: 198 | type: OS::Nova::Server 199 | depends_on: app-svr-01 200 | properties: 201 | name: web-instance-01 202 | image: { get_param: web_image } 203 | flavor: { get_param: flavor } 204 | networks: 205 | - port: { get_resource: web-svr-01_port0 } 206 | 207 | outputs: 208 | web-svr-01_private_ip: 209 | description: IP address of web-svr-01 in private network 210 | value: { get_attr: [ web-svr-01, first_address ] } 211 | web-svr-01_public_ip: 212 | description: Floating IP address of web-svr-01 in public network 213 | value: { get_attr: [ web-svr-01_floating_ip, floating_ip_address ] } 214 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/3-tier_central-routing_nonat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Web Server, 1 App Server, 5 | and 1 DB Server on private networks and attaches them to security groups (HOT template in YAML). 6 | Since this is a NO-NAT template this MUST be done by an OpenStack Admin. 7 | 8 | parameters: 9 | external_network_id: 10 | type: string 11 | label: ext-net 12 | description: UUID of a Neutron external network 13 | default: 995cf315-8d49-427e-aa16-16051bab37aa 14 | 15 | web_image: 16 | type: string 17 | description: Name of image to use for servers 18 | default: ubuntu-14.04-server-amd64 19 | 20 | app_image: 21 | type: string 22 | description: Name of image to use for servers 23 | default: ubuntu-14.04-server-amd64 24 | 25 | db_image: 26 | type: string 27 | description: Name of image to use for servers 28 | default: ubuntu-14.04-server-amd64 29 | 30 | flavor: 31 | type: string 32 | label: Flavor name 33 | description: Name of the flavor to be used for the instances. 34 | default: m1.small 35 | 36 | dns: 37 | type: string 38 | label: DNS nameserver 39 | description: Comma separated list of DNS nameservers for the private network 40 | default: "8.8.8.8" 41 | 42 | resources: 43 | # Create the web logical switch and configure DHCP. 44 | web_network_01: 45 | type: OS::Neutron::Net 46 | properties: 47 | admin_state_up: true 48 | name: web-network-01 49 | web_subnet_01: 50 | type: OS::Neutron::Subnet 51 | properties: 52 | name: web-subnet-01 53 | cidr: 172.16.10.0/24 54 | enable_dhcp: true 55 | dns_nameservers: [ { get_param: dns } ] 56 | gateway_ip: 172.16.10.1 57 | network_id: { get_resource: web_network_01 } 58 | 59 | # Create the app logical switch and configure DHCP. 60 | app_network_01: 61 | type: OS::Neutron::Net 62 | properties: 63 | admin_state_up: true 64 | name: app-network-01 65 | app_subnet_01: 66 | type: OS::Neutron::Subnet 67 | properties: 68 | name: app-subnet-01 69 | cidr: 172.16.20.0/24 70 | dns_nameservers: [ { get_param: dns } ] 71 | enable_dhcp: true 72 | gateway_ip: 172.16.20.1 73 | network_id: { get_resource: app_network_01 } 74 | 75 | # Create the db logical switch and configure DHCP. 76 | db_network_01: 77 | type: OS::Neutron::Net 78 | properties: 79 | admin_state_up: true 80 | name: db-network-01 81 | db_subnet_01: 82 | type: OS::Neutron::Subnet 83 | properties: 84 | name: db-subnet-01 85 | cidr: 172.16.30.0/24 86 | dns_nameservers: [ { get_param: dns } ] 87 | enable_dhcp: true 88 | gateway_ip: 172.16.30.1 89 | network_id: { get_resource: db_network_01 } 90 | 91 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 92 | heat_router_01: 93 | type: OS::Neutron::Router 94 | properties: 95 | name: heat-router-01 96 | value_specs: {router_type: exclusive} 97 | admin_state_up: true 98 | external_gateway_info: { "enable_snat": false, "network": { get_param: external_network_id }} 99 | heat_router_int0: 100 | type: OS::Neutron::RouterInterface 101 | properties: 102 | router_id: { get_resource: heat_router_01 } 103 | subnet_id: { get_resource: web_subnet_01 } 104 | heat_router_int1: 105 | type: OS::Neutron::RouterInterface 106 | properties: 107 | router_id: { get_resource: heat_router_01 } 108 | subnet_id: { get_resource: app_subnet_01 } 109 | heat_router_int2: 110 | type: OS::Neutron::RouterInterface 111 | properties: 112 | router_id: { get_resource: heat_router_01 } 113 | subnet_id: { get_resource: db_subnet_01 } 114 | 115 | # Create security groups for the tiers. 116 | db_security_group: 117 | type: OS::Neutron::SecurityGroup 118 | properties: 119 | description: Allow sql traffic from web tier 120 | name: db-security-group 121 | rules: 122 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 123 | protocol: tcp 124 | port_range_min: 3306 125 | port_range_max: 3306 126 | - remote_ip_prefix: 0.0.0.0/0 127 | protocol: icmp 128 | 129 | web_security_group: 130 | type: OS::Neutron::SecurityGroup 131 | properties: 132 | description: Allow web traffic from anywhere 133 | name: web-security-group 134 | rules: 135 | - remote_ip_prefix: 0.0.0.0/0 136 | protocol: tcp 137 | port_range_min: 80 138 | port_range_max: 80 139 | - remote_ip_prefix: 0.0.0.0/0 140 | protocol: tcp 141 | port_range_min: 22 142 | port_range_max: 22 143 | - remote_ip_prefix: 0.0.0.0/0 144 | protocol: icmp 145 | 146 | app_security_group: 147 | type: OS::Neutron::SecurityGroup 148 | properties: 149 | description: Allow app traffic from web tier 150 | name: app-security-group 151 | rules: 152 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 153 | protocol: tcp 154 | port_range_min: 8443 155 | port_range_max: 8443 156 | - remote_ip_prefix: 0.0.0.0/0 157 | protocol: icmp 158 | 159 | # Create ports on web switch. 160 | web-svr-01_port0: 161 | type: OS::Neutron::Port 162 | properties: 163 | admin_state_up: true 164 | network_id: { get_resource: web_network_01 } 165 | security_groups: 166 | - { get_resource: web_security_group } 167 | 168 | # Create port(s) on app switch. 169 | app-svr-01_port0: 170 | type: OS::Neutron::Port 171 | properties: 172 | admin_state_up: true 173 | network_id: { get_resource: app_network_01 } 174 | security_groups: 175 | - { get_resource: app_security_group } 176 | 177 | # Create port(s) on db switch. 178 | db-svr-01_port0: 179 | type: OS::Neutron::Port 180 | properties: 181 | admin_state_up: true 182 | network_id: { get_resource: db_network_01 } 183 | security_groups: 184 | - { get_resource: db_security_group } 185 | 186 | # Provision instances. 187 | db-svr-01: 188 | type: OS::Nova::Server 189 | properties: 190 | name: db-instance-01 191 | image: { get_param: db_image } 192 | flavor: { get_param: flavor } 193 | networks: 194 | - port: { get_resource: db-svr-01_port0 } 195 | 196 | app-svr-01: 197 | type: OS::Nova::Server 198 | depends_on: db-svr-01 199 | properties: 200 | name: app-instance-01 201 | image: { get_param: app_image } 202 | flavor: { get_param: flavor } 203 | networks: 204 | - port: { get_resource: app-svr-01_port0 } 205 | 206 | web-svr-01: 207 | type: OS::Nova::Server 208 | depends_on: app-svr-01 209 | properties: 210 | name: web-instance-01 211 | image: { get_param: web_image } 212 | flavor: { get_param: flavor } 213 | networks: 214 | - port: { get_resource: web-svr-01_port0 } 215 | 216 | outputs: 217 | web-svr-01_private_ip: 218 | description: IP address of web-svr-01 in private network 219 | value: { get_attr: [ web-svr-01, first_address ] } 220 | 221 | app-svr-01_private_ip: 222 | description: IP address of app-svr-01 in private network 223 | value: { get_attr: [ app-svr-01, first_address ] } 224 | 225 | db-svr-01_private_ip: 226 | description: IP address of db-svr-01 in private network 227 | value: { get_attr: [ db-svr-01, first_address ] } 228 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/3-tier_distributed-routing_nat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Web Server, 1 App Server, 5 | and 1 DB Server on private networks and attaches them to security groups (HOT template in YAML). 6 | 7 | parameters: 8 | external_network_id: 9 | type: string 10 | label: ext-net 11 | description: UUID of a Neutron external network 12 | default: 995cf315-8d49-427e-aa16-16051bab37aa 13 | 14 | web_image: 15 | type: string 16 | description: Name of image to use for servers 17 | default: ubuntu-14.04-server-amd64 18 | 19 | app_image: 20 | type: string 21 | description: Name of image to use for servers 22 | default: ubuntu-14.04-server-amd64 23 | 24 | db_image: 25 | type: string 26 | description: Name of image to use for servers 27 | default: ubuntu-14.04-server-amd64 28 | 29 | flavor: 30 | type: string 31 | label: Flavor name 32 | description: Name of the flavor to be used for the instances. 33 | default: m1.small 34 | 35 | dns: 36 | type: string 37 | label: DNS nameserver 38 | description: Comma separated list of DNS nameservers for the private network 39 | default: "8.8.8.8" 40 | 41 | resources: 42 | # Create the web logical switch and configure DHCP. 43 | web_network_01: 44 | type: OS::Neutron::Net 45 | properties: 46 | admin_state_up: true 47 | name: web-network-01 48 | web_subnet_01: 49 | type: OS::Neutron::Subnet 50 | properties: 51 | name: web-subnet-01 52 | cidr: 172.16.10.0/24 53 | dns_nameservers: [ { get_param: dns } ] 54 | enable_dhcp: true 55 | gateway_ip: 172.16.10.1 56 | network_id: { get_resource: web_network_01 } 57 | 58 | # Create the app logical switch and configure DHCP. 59 | app_network_01: 60 | type: OS::Neutron::Net 61 | properties: 62 | admin_state_up: true 63 | name: app-network-01 64 | app_subnet_01: 65 | type: OS::Neutron::Subnet 66 | properties: 67 | name: app-subnet-01 68 | cidr: 172.16.20.0/24 69 | dns_nameservers: [ { get_param: dns } ] 70 | enable_dhcp: true 71 | gateway_ip: 172.16.20.1 72 | network_id: { get_resource: app_network_01 } 73 | 74 | # Create the db logical switch and configure DHCP. 75 | db_network_01: 76 | type: OS::Neutron::Net 77 | properties: 78 | admin_state_up: true 79 | name: db-network-01 80 | db_subnet_01: 81 | type: OS::Neutron::Subnet 82 | properties: 83 | name: db-subnet-01 84 | cidr: 172.16.30.0/24 85 | dns_nameservers: [ { get_param: dns } ] 86 | enable_dhcp: true 87 | gateway_ip: 172.16.30.1 88 | network_id: { get_resource: db_network_01 } 89 | 90 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 91 | heat_router_01: 92 | type: OS::Neutron::Router 93 | properties: 94 | name: heat-router-01 95 | admin_state_up: true 96 | distributed: true 97 | external_gateway_info: { "network": { get_param: external_network_id }} 98 | heat_router_int0: 99 | type: OS::Neutron::RouterInterface 100 | properties: 101 | router_id: { get_resource: heat_router_01 } 102 | subnet_id: { get_resource: web_subnet_01 } 103 | heat_router_int1: 104 | type: OS::Neutron::RouterInterface 105 | properties: 106 | router_id: { get_resource: heat_router_01 } 107 | subnet_id: { get_resource: app_subnet_01 } 108 | heat_router_int2: 109 | type: OS::Neutron::RouterInterface 110 | properties: 111 | router_id: { get_resource: heat_router_01 } 112 | subnet_id: { get_resource: db_subnet_01 } 113 | 114 | # Create security groups for the tiers. 115 | db_security_group: 116 | type: OS::Neutron::SecurityGroup 117 | properties: 118 | description: Allow sql traffic from web tier 119 | name: db-security-group 120 | rules: 121 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 122 | protocol: tcp 123 | port_range_min: 3306 124 | port_range_max: 3306 125 | - remote_ip_prefix: 0.0.0.0/0 126 | protocol: icmp 127 | 128 | web_security_group: 129 | type: OS::Neutron::SecurityGroup 130 | properties: 131 | description: Allow web traffic from anywhere 132 | name: web-security-group 133 | rules: 134 | - remote_ip_prefix: 0.0.0.0/0 135 | protocol: tcp 136 | port_range_min: 80 137 | port_range_max: 80 138 | - remote_ip_prefix: 0.0.0.0/0 139 | protocol: tcp 140 | port_range_min: 22 141 | port_range_max: 22 142 | - remote_ip_prefix: 0.0.0.0/0 143 | protocol: icmp 144 | 145 | app_security_group: 146 | type: OS::Neutron::SecurityGroup 147 | properties: 148 | description: Allow app traffic from web tier 149 | name: app-security-group 150 | rules: 151 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 152 | protocol: tcp 153 | port_range_min: 8443 154 | port_range_max: 8443 155 | - remote_ip_prefix: 0.0.0.0/0 156 | protocol: icmp 157 | 158 | # Create ports on web switch. 159 | web-svr-01_port0: 160 | type: OS::Neutron::Port 161 | properties: 162 | admin_state_up: true 163 | network_id: { get_resource: web_network_01 } 164 | security_groups: 165 | - { get_resource: web_security_group } 166 | web-svr-01_floating_ip: 167 | type: OS::Neutron::FloatingIP 168 | properties: 169 | floating_network_id: {get_param: external_network_id} 170 | port_id: { get_resource: web-svr-01_port0 } 171 | 172 | # Create port(s) on app switch. 173 | app-svr-01_port0: 174 | type: OS::Neutron::Port 175 | properties: 176 | admin_state_up: true 177 | network_id: { get_resource: app_network_01 } 178 | security_groups: 179 | - { get_resource: app_security_group } 180 | 181 | # Create port(s) on db switch. 182 | db-svr-01_port0: 183 | type: OS::Neutron::Port 184 | properties: 185 | admin_state_up: true 186 | network_id: { get_resource: db_network_01 } 187 | security_groups: 188 | - { get_resource: db_security_group } 189 | 190 | # Provision instances. 191 | db-svr-01: 192 | type: OS::Nova::Server 193 | properties: 194 | name: db-instance-01 195 | image: { get_param: db_image } 196 | flavor: { get_param: flavor } 197 | networks: 198 | - port: { get_resource: db-svr-01_port0 } 199 | 200 | app-svr-01: 201 | type: OS::Nova::Server 202 | depends_on: db-svr-01 203 | properties: 204 | name: app-instance-01 205 | image: { get_param: app_image } 206 | flavor: { get_param: flavor } 207 | networks: 208 | - port: { get_resource: app-svr-01_port0 } 209 | 210 | web-svr-01: 211 | type: OS::Nova::Server 212 | depends_on: app-svr-01 213 | properties: 214 | name: web-instance-01 215 | image: { get_param: web_image } 216 | flavor: { get_param: flavor } 217 | networks: 218 | - port: { get_resource: web-svr-01_port0 } 219 | 220 | outputs: 221 | web-svr-01_private_ip: 222 | description: IP address of web-svr-01 in private network 223 | value: { get_attr: [ web-svr-01, first_address ] } 224 | web-svr-01_public_ip: 225 | description: Floating IP address of web-svr-01 in public network 226 | value: { get_attr: [ web-svr-01_floating_ip, floating_ip_address ] } 227 | app-svr-01_private_ip: 228 | description: IP address of app-svr-01 in private network 229 | value: { get_attr: [ app-svr-01, first_address ] } 230 | db-svr-01_private_ip: 231 | description: IP address of db-svr-01 in private network 232 | value: { get_attr: [ db-svr-01, first_address ] } 233 | -------------------------------------------------------------------------------- /heat/poc/validation_tests/3-tier_distributed-routing_nonat.yaml: -------------------------------------------------------------------------------- 1 | heat_template_version: 2015-04-30 2 | 3 | description: | 4 | A simple Heat template that spins up a 3 Tier App with 1 Web Server, 1 App Server, 5 | and 1 DB Server on private networks and attaches them to security groups (HOT template in YAML). 6 | Since this is a NO-NAT template this MUST be done by an OpenStack Admin. 7 | 8 | parameters: 9 | external_network_id: 10 | type: string 11 | label: ext-net 12 | description: UUID of a Neutron external network 13 | default: 995cf315-8d49-427e-aa16-16051bab37aa 14 | 15 | web_image: 16 | type: string 17 | description: Name of image to use for servers 18 | default: ubuntu-14.04-server-amd64 19 | 20 | app_image: 21 | type: string 22 | description: Name of image to use for servers 23 | default: ubuntu-14.04-server-amd64 24 | 25 | db_image: 26 | type: string 27 | description: Name of image to use for servers 28 | default: ubuntu-14.04-server-amd64 29 | 30 | flavor: 31 | type: string 32 | label: Flavor name 33 | description: Name of the flavor to be used for the instances. 34 | default: m1.small 35 | 36 | resources: 37 | # Create the web logical switch and configure DHCP. 38 | web_network_01: 39 | type: OS::Neutron::Net 40 | properties: 41 | admin_state_up: true 42 | name: web-network-01 43 | web_subnet_01: 44 | type: OS::Neutron::Subnet 45 | properties: 46 | name: web-subnet-01 47 | cidr: 172.16.10.0/24 48 | dns_nameservers: [8.8.8.8] 49 | enable_dhcp: true 50 | gateway_ip: 172.16.10.1 51 | network_id: { get_resource: web_network_01 } 52 | 53 | # Create the app logical switch and configure DHCP. 54 | app_network_01: 55 | type: OS::Neutron::Net 56 | properties: 57 | admin_state_up: true 58 | name: app-network-01 59 | app_subnet_01: 60 | type: OS::Neutron::Subnet 61 | properties: 62 | name: app-subnet-01 63 | cidr: 172.16.20.0/24 64 | dns_nameservers: [8.8.8.8] 65 | enable_dhcp: true 66 | gateway_ip: 172.16.20.1 67 | network_id: { get_resource: app_network_01 } 68 | 69 | # Create the db logical switch and configure DHCP. 70 | db_network_01: 71 | type: OS::Neutron::Net 72 | properties: 73 | admin_state_up: true 74 | name: db-network-01 75 | db_subnet_01: 76 | type: OS::Neutron::Subnet 77 | properties: 78 | name: db-subnet-01 79 | cidr: 172.16.30.0/24 80 | dns_nameservers: [8.8.8.8] 81 | enable_dhcp: true 82 | gateway_ip: 172.16.30.1 83 | network_id: { get_resource: db_network_01 } 84 | 85 | # Create router, add internal interfaces for 3 tiers, and also an uplink. 86 | heat_router_01: 87 | type: OS::Neutron::Router 88 | properties: 89 | name: heat-router-01 90 | value_specs: {router_type: exclusive} 91 | distributed: true 92 | admin_state_up: true 93 | external_gateway_info: { "enable_snat": false, "network": { get_param: external_network_id }} 94 | heat_router_int0: 95 | type: OS::Neutron::RouterInterface 96 | properties: 97 | router_id: { get_resource: heat_router_01 } 98 | subnet_id: { get_resource: web_subnet_01 } 99 | heat_router_int1: 100 | type: OS::Neutron::RouterInterface 101 | properties: 102 | router_id: { get_resource: heat_router_01 } 103 | subnet_id: { get_resource: app_subnet_01 } 104 | heat_router_int2: 105 | type: OS::Neutron::RouterInterface 106 | properties: 107 | router_id: { get_resource: heat_router_01 } 108 | subnet_id: { get_resource: db_subnet_01 } 109 | 110 | # Create security groups for the tiers. 111 | db_security_group: 112 | type: OS::Neutron::SecurityGroup 113 | properties: 114 | description: Allow sql traffic from web tier 115 | name: db-security-group 116 | rules: 117 | - remote_ip_prefix: { get_attr: [ app_subnet_01, cidr ] } 118 | protocol: tcp 119 | port_range_min: 3306 120 | port_range_max: 3306 121 | 122 | web_security_group: 123 | type: OS::Neutron::SecurityGroup 124 | properties: 125 | description: Allow web traffic from anywhere 126 | name: web-security-group 127 | rules: 128 | - remote_ip_prefix: 0.0.0.0/0 129 | protocol: tcp 130 | port_range_min: 80 131 | port_range_max: 80 132 | - remote_ip_prefix: 0.0.0.0/0 133 | protocol: tcp 134 | port_range_min: 22 135 | port_range_max: 22 136 | 137 | app_security_group: 138 | type: OS::Neutron::SecurityGroup 139 | properties: 140 | description: Allow app traffic from web tier 141 | name: app-security-group 142 | rules: 143 | - remote_ip_prefix: { get_attr: [ web_subnet_01, cidr ] } 144 | protocol: tcp 145 | port_range_min: 8443 146 | port_range_max: 8443 147 | 148 | # Create ports on web switch. 149 | web-svr-01_port0: 150 | type: OS::Neutron::Port 151 | properties: 152 | admin_state_up: true 153 | network_id: { get_resource: web_network_01 } 154 | security_groups: 155 | - { get_resource: web_security_group } 156 | 157 | # Create port(s) on app switch. 158 | app-svr-01_port0: 159 | type: OS::Neutron::Port 160 | properties: 161 | admin_state_up: true 162 | network_id: { get_resource: app_network_01 } 163 | security_groups: 164 | - { get_resource: app_security_group } 165 | 166 | # Create port(s) on db switch. 167 | db-svr-01_port0: 168 | type: OS::Neutron::Port 169 | properties: 170 | admin_state_up: true 171 | network_id: { get_resource: db_network_01 } 172 | security_groups: 173 | - { get_resource: db_security_group } 174 | 175 | # Provision instances. 176 | db-svr-01: 177 | type: OS::Nova::Server 178 | properties: 179 | name: db-instance-01 180 | image: { get_param: db_image } 181 | flavor: { get_param: flavor } 182 | networks: 183 | - port: { get_resource: db-svr-01_port0 } 184 | 185 | app-svr-01: 186 | type: OS::Nova::Server 187 | depends_on: db-svr-01 188 | properties: 189 | name: app-instance-01 190 | image: { get_param: app_image } 191 | flavor: { get_param: flavor } 192 | networks: 193 | - port: { get_resource: app-svr-01_port0 } 194 | 195 | web-svr-01: 196 | type: OS::Nova::Server 197 | depends_on: app-svr-01 198 | properties: 199 | name: web-instance-01 200 | image: { get_param: web_image } 201 | flavor: { get_param: flavor } 202 | networks: 203 | - port: { get_resource: web-svr-01_port0 } 204 | 205 | outputs: 206 | web-svr-01_private_ip: 207 | description: IP address of web-svr-01 in private network 208 | value: { get_attr: [ web-svr-01, first_address ] } 209 | 210 | app-svr-01_private_ip: 211 | description: IP address of app-svr-01 in private network 212 | value: { get_attr: [ app-svr-01, first_address ] } 213 | 214 | db-svr-01_private_ip: 215 | description: IP address of db-svr-01 in private network 216 | value: { get_attr: [ db-svr-01, first_address ] } 217 | -------------------------------------------------------------------------------- /tests/README.rst: -------------------------------------------------------------------------------- 1 | Tests that help you ensure your Openstack Env is healthy 2 | ========================================================= 3 | Tempest is a common tool to verify your Openstack. Follow this Guide you can 4 | run tempest tests to check your existed VIO setup. 5 | 6 | 7 | System dependencies installation 8 | ================================= 9 | Set up an Ubuntu 14.04(or above) VM, make sure it has internet connectivity. 10 | 11 | * Install required libs: python-dev libffi-dev libssl-dev git swig libxml2-dev libxslt-dev. 12 | * Install pip: See https://pip.pypa.io/en/stable/installing/ 13 | * Install virtualenv with pip. 14 | 15 | .. code:: shell 16 | 17 | apt-get -y install python-dev libffi-dev libssl-dev git swig libxml2-dev libxslt-dev 18 | wget https://bootstrap.pypa.io/get-pip.py 19 | python get-pip.py 20 | pip install virtualenv 21 | 22 | 23 | Installation 24 | ============= 25 | Clone this repository from github, then go to ``vio/tests`` folder. 26 | Run ``./install.sh`` to install these test projects. Run 27 | ``source .venv/bin/activate`` to activate them. 28 | 29 | Type command ``panda tempest install -h`` to check if ``panda`` is installed. 30 | 31 | 32 | Configuration 33 | ============== 34 | Move this Ubuntu VM to your VIO network, make sure it has a IP address in the 35 | same L2 subnet with VIO private VIP. The VM should be able to reach private VIP 36 | without routing. Add another NIC to the VM and make sure it has connectivity to 37 | NSX Edge external network. 38 | 39 | If the MTU of the networks are larger than 1500, set them to a small value with 40 | below commands: 41 | 42 | .. code:: shell 43 | 44 | ip li set mtu 1200 dev eth0 45 | ip li set mtu 1200 dev eth1 46 | 47 | 48 | Run command ``panda tempest config -h`` to get the help information about all 49 | parameters to configure tempest in your setup. 50 | 51 | Below are sample commands to configure tempest according to VIO neutron and 52 | keystone backend. 53 | 54 | .. code:: shell 55 | 56 | # Configure tempest against a NSXv neutron/SQL keystone backend VIO 57 | panda tempest config '192.168.111.160' 'admin' 'vmware' 'nsxv' --ext-cidr '192.168.112.0/24' --ext-start-ip '192.168.112.170' --ext-end-ip '192.168.112.200' --ext-gateway '192.168.112.1' --nsx-manager '192.168.111.15' --nsx-user 'admin' --nsx-password 'default' 58 | # Configure tempest against a NSXv neutron/LDAP keystone backend VIO. 59 | panda tempest config '192.168.111.160' 'vioadmin@vio.com' 'VMware1!' 'nsxv' --credentials-provider 'pre-provisioned' --user1 'xiaoy@vio.com' --user1-password 'VMware1!' --user2 'sren@vio.com' --user2-password 'VMware1!' --ext-cidr '192.168.112.0/24' --ext-start-ip '192.168.112.170' --ext-end-ip '192.168.112.200' --ext-gateway '192.168.112.1' --nsx-manager '192.168.111.15' --nsx-user 'admin' --nsx-password 'default' 60 | # Configure tempest against a DVS neutron/SQL keystone backend VIO. 61 | panda tempest config '192.168.111.153' 'admin' 'vmware' 'dvs' 62 | 63 | 64 | Note: 65 | 66 | * Make sure you run them in the directory ``tests/`` 67 | * For LDAP backend, you should find available LDAP users like ``xiaoy@vio.com`` and ``sren@vio.com`` as the parameters. Replace ``vioadmin@vio.com`` to the admin user of your VIO setup. 68 | * '192.168.112.0/24', '192.168.112.170' etc. are the Edge external network configurations in your VIO setup. Tempest tests will ssh through floating IP and verify instances. 69 | * '192.168.111.160' 'admin' 'vmware' are the Private VIP and authentication of your VIO setup, change them to yours. 70 | 71 | Check generated configurations in ``tempest/etc/tempest.conf`` afterward. 72 | 73 | 74 | Run test suites 75 | ================ 76 | Make a report directory and kick off tests with below commands: 77 | 78 | .. code:: shell 79 | 80 | mkdir reports 81 | panda tempest run 'keystone,glance,nova,cinder,neutron,heat,scenario' --report-dir reports/ 82 | 83 | Note: HTML reports will be generated to specified report dir and logs of tempest 84 | are written to tempest/tempest.log. Any error during 'panda' command please refer 85 | to panda.log. Only scenario is recommended for health verification. 86 | 87 | 88 | -------------------------------------------------------------------------------- /tests/buildwebapi/README.rst: -------------------------------------------------------------------------------- 1 | buildwebapi 2 | ============= 3 | 4 | A python library for querying buildweb resources using buildapi. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com buildwebapi 13 | 14 | Usage 15 | ====== 16 | 17 | Example 1: Get latest succeeded build id (MetricResource) 18 | 19 | .. code:: python 20 | 21 | from buildwebapi import api as buildapi 22 | build_metrics = buildapi.MetricResource.by_name( 23 | 'build', product='vmw_openstack', buildstate='succeeded') 24 | 25 | 26 | Example 2: Get a list of all builds (ListResource) 27 | 28 | .. code:: python 29 | 30 | from buildwebapi import api as buildapi 31 | builds = buildapi.ListResource.by_name( 32 | 'build', product='vmw_openstack') 33 | 34 | 35 | Example 3: Get build details by its Id (ItemResource) 36 | 37 | .. code:: python 38 | 39 | from buildwebapi import api as buildapi 40 | build = buildapi.ItemResource.by_id('build', 1924554) -------------------------------------------------------------------------------- /tests/buildwebapi/buildwebapi/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/buildwebapi/buildwebapi/__init__.py -------------------------------------------------------------------------------- /tests/buildwebapi/buildwebapi/api.py: -------------------------------------------------------------------------------- 1 | import itertools 2 | import json 3 | import logging 4 | import requests 5 | 6 | BUILDAPI_URL = 'http://buildapi.eng.vmware.com' 7 | BUILDAPI_LIST_RESOURCE_URL = BUILDAPI_URL + '/%s/%s' 8 | BUILDAPI_METRICS_RESOURCE_URL = BUILDAPI_URL + '/%s/%s_metrics' 9 | BUILDAPI_ITEM_RESOURCE_URL = BUILDAPI_URL + '/%s/%s/%d' 10 | 11 | DEFAULT_PARAMS = {'_format': 'json'} 12 | 13 | LOG = logging.getLogger(__name__) 14 | 15 | 16 | def _make_params(**kwargs): 17 | return dict(itertools.chain(DEFAULT_PARAMS.items(), kwargs.items())) 18 | 19 | 20 | def _get(*args, **kwargs): 21 | LOG.debug('url: %s' % args[0]) 22 | resp = requests.get(*args, **kwargs) 23 | LOG.debug('response: \n %s' % resp.text) 24 | return json.loads(resp.text) 25 | 26 | 27 | class _Resource(object): 28 | def __init__(self, data): 29 | self._data = data 30 | 31 | def __getattr__(self, name): 32 | return self._data[name] 33 | 34 | 35 | class ListResource(_Resource): 36 | 37 | @classmethod 38 | def by_url(cls, url): 39 | if not url.startswith(BUILDAPI_URL): 40 | url = BUILDAPI_URL + url 41 | return cls(_get(url, params=_make_params())) 42 | 43 | @classmethod 44 | def by_name(cls, name, build_system='ob', **filters): 45 | url = BUILDAPI_LIST_RESOURCE_URL % (build_system, name) 46 | return cls(_get(url, params=_make_params(**filters))) 47 | 48 | def __init__(self, data): 49 | assert '_total_count' in data and '_list' in data 50 | super(ListResource, self).__init__(data) 51 | self.items = self._parse_items() 52 | 53 | def __iter__(self): 54 | return iter(self.items) 55 | 56 | def _parse_items(self): 57 | return [ItemResource(data) for data in self._data['_list']] 58 | 59 | 60 | class MetricResource(_Resource): 61 | 62 | @classmethod 63 | def by_name(cls, name, build_system='ob', **filters): 64 | url = BUILDAPI_METRICS_RESOURCE_URL % (build_system, name) 65 | return cls(_get(url, params=_make_params(**filters))) 66 | 67 | def __init__(self, data): 68 | assert '_total_count' in data and data['_total_count'] == 1 69 | assert '_list' in data 70 | assert 'max_id' in data['_list'][0] 71 | assert 'min_id' in data['_list'][0] 72 | super(MetricResource, self).__init__(data) 73 | 74 | def get_max_id(self): 75 | return self._data['_list'][0]['max_id'] 76 | 77 | def get_min_id(self): 78 | return self._data['_list'][0]['min_id'] 79 | 80 | 81 | class ItemResource(_Resource): 82 | 83 | @classmethod 84 | def by_id(cls, name, res_id, build_system='ob'): 85 | url = BUILDAPI_ITEM_RESOURCE_URL % (build_system, name, res_id) 86 | return cls(_get(url, params=_make_params())) 87 | 88 | def __init__(self, data): 89 | assert '_this_resource' in data 90 | super(ItemResource, self).__init__(data) 91 | 92 | def matches(self, **filters): 93 | for k, v in filters.items(): 94 | if not hasattr(self, k): 95 | return False 96 | selfv = getattr(self, k) 97 | is_substring = (isinstance(v, basestring) and (v in selfv)) 98 | is_equals = (v == selfv) 99 | if not (is_substring or is_equals): 100 | return False 101 | else: 102 | return True 103 | -------------------------------------------------------------------------------- /tests/buildwebapi/requirements.txt: -------------------------------------------------------------------------------- 1 | requests -------------------------------------------------------------------------------- /tests/buildwebapi/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='buildwebapi', 5 | version='0.0.2', 6 | scripts=[], 7 | packages=['buildwebapi'], 8 | include_package_data=True, 9 | install_requires=[ 10 | 'requests' 11 | ] 12 | ) 13 | -------------------------------------------------------------------------------- /tests/buildwebapi/tests/buildapi/test_api.py: -------------------------------------------------------------------------------- 1 | from buildwebapi import api as buildapi 2 | 3 | import mock 4 | import json 5 | import os 6 | import unittest 7 | import urlparse 8 | 9 | RESOURCES_PATH = os.path.join( 10 | os.path.dirname(__file__), '..', 'resources/buildapi') 11 | 12 | 13 | def _get(*args, **kwargs): 14 | payload = None 15 | p = urlparse.urlparse(args[0]) 16 | if p.path == '/ob/build': 17 | payload = os.path.join(RESOURCES_PATH, 'buildlist.json') 18 | elif p.path == '/ob/deliverable': 19 | payload = os.path.join(RESOURCES_PATH, 'deliverable.json') 20 | elif p.path == '/ob/build_metrics': 21 | payload = os.path.join(RESOURCES_PATH, 'buildmetrics.json') 22 | elif p.path == '/ob/build/1929854': 23 | payload = os.path.join(RESOURCES_PATH, 'build.json') 24 | 25 | if payload: 26 | with open(payload) as f: 27 | return json.load(f) 28 | 29 | 30 | class ApiTest(unittest.TestCase): 31 | 32 | @classmethod 33 | def setUpClass(cls): 34 | cls.patcher = mock.patch('buildwebapi.api._get', _get) 35 | cls.patcher.start() 36 | 37 | @classmethod 38 | def tearDownClass(cls): 39 | cls.patcher.stop() 40 | 41 | 42 | class TestListResource(ApiTest): 43 | 44 | def testByName(self): 45 | builds = buildapi.ListResource.by_name( 46 | 'build', product='vmw_openstack') 47 | for build in builds: 48 | self.assertEqual('build', build._this_resource) 49 | 50 | def testByUrl(self): 51 | url = 'http://buildapi.eng.vmware.com/ob/deliverable?build=1935022' 52 | deliverables = buildapi.ListResource.by_url(url) 53 | for deliverable in deliverables: 54 | self.assertEqual('deliverable', deliverable._this_resource) 55 | 56 | 57 | class TestMetricResource(ApiTest): 58 | 59 | def testByName(self): 60 | build_metrics = buildapi.MetricResource.by_name( 61 | 'build', product='vmw_openstack', buildstate='succeeded') 62 | self.assertEqual(1935022, build_metrics.get_max_id()) 63 | self.assertEqual(1924554, build_metrics.get_min_id()) 64 | 65 | 66 | class TestItemResource(ApiTest): 67 | 68 | def testById(self): 69 | build_id = 1929854 70 | build = buildapi.ItemResource.by_id('build', build_id) 71 | self.assertEqual(build_id, build.id) 72 | 73 | 74 | if __name__ == '__main__': 75 | unittest.main() 76 | -------------------------------------------------------------------------------- /tests/buildwebapi/tests/resources/buildapi/build.json: -------------------------------------------------------------------------------- 1 | { 2 | "changeset":"5fbfc6c7885c422b11700214e3b9a0cb8ffe8c54", 3 | "_deliverables_url":"/ob/deliverable/?build=1929854", 4 | "_buildtype_url":"/ob/buildtype/beta/", 5 | "_incremental_builds_url":null, 6 | "exit_code":0, 7 | "buildsystem":"ob", 8 | "_requestor_url":"/ob/requestor/?build=1929854", 9 | "_buildmachines_url":"/ob/buildmachine/?build=1929854", 10 | "buildtree_size_in_mb":5231, 11 | "_buildstate_url":"/ob/buildstate/succeeded/", 12 | "_buildtags_url":"/ob/buildtags/?build=1929854", 13 | "saved":false, 14 | "id":1929854, 15 | "_buildprogressnotes_url":"/ob/buildprogressnote/?build=1929854", 16 | "_buildnotes_url":"/ob/buildnote/?build=1929854", 17 | "scmserver":"ssh://git@git.eng.vmware.com", 18 | "_buildstateprogress_url":"/ob/buildstateprogress/?build=1929854", 19 | "reuse_trees":false, 20 | "ondisk":true, 21 | "reuse_trees_tag":null, 22 | "keep_until":"2014-07-17 11:23:47.305671", 23 | "_product_url":"/ob/product/vmw-openstack/", 24 | "buildstate":"succeeded", 25 | "_buildtree_url":"http://build-squid.eng.vmware.com/build/mts/release/bora-1929854/", 26 | "_storage_queue_url":"/ob/storagequeue/?build=1929854", 27 | "version":"0.5.0", 28 | "_localtree_url":"/ob/localtree/?build=1929854", 29 | "branch":"master", 30 | "progress":null, 31 | "_releasetag_url":"/ob/releasetag/?build=1929854", 32 | "_qatestresults_url":"/ob/qatestresult/?build=1929854", 33 | "buildtype":"beta", 34 | "product":"vmw-openstack", 35 | "_sb_parent_builds_url":"/sb/buildcomponent/?component_buildid=1929854", 36 | "_incremental_next_builds_url":null, 37 | "zerocopy":false, 38 | "prodbuildnum":4, 39 | "_this_resource":"build", 40 | "buildtree":"/build/storage60/release/bora-1929854", 41 | "_parent_builds_url":"/ob/buildcomponent/?component_buildid=1929854", 42 | "reusable_id":null, 43 | "endtime":"2014-06-25 12:00:25.495995", 44 | "expired":false, 45 | "_ob_parent_builds_url":"/ob/buildcomponent/?component_buildid=1929854", 46 | "locales":"en", 47 | "_incremental":false, 48 | "_this_url":"/ob/build/1929854/", 49 | "_branch_url":"/ob/branch/master/", 50 | "_buildflags_url":"/ob/buildflag/?build=1929854", 51 | "releasetype":"beta", 52 | "_elapsed_sec":2199.1111139999998, 53 | "_currenttime":"2014-06-27 16:38:35.281554", 54 | "_releasetype_url":"/ob/releasetype/beta/", 55 | "backedup":false, 56 | "starttime":"2014-06-25 11:23:46.384881", 57 | "_component_builds_url":"/ob/buildcomponent/?build=1929854", 58 | "bugid":null, 59 | "_incremental_previous_builds_url":null 60 | } -------------------------------------------------------------------------------- /tests/buildwebapi/tests/resources/buildapi/buildmetrics.json: -------------------------------------------------------------------------------- 1 | { 2 | "_total_count":1, 3 | "_next_url":null, 4 | "_page_count":1, 5 | "_list":[ 6 | { 7 | "min_elapsed_sec":1867.0623270000001, 8 | "count":7, 9 | "avg_elapsed_sec":2359.3521082857101, 10 | "max_elapsed_sec":3730.1441089999998, 11 | "min_id":1924554, 12 | "max_id":1935022 13 | } 14 | ], 15 | "_previous_url":null 16 | } 17 | -------------------------------------------------------------------------------- /tests/buildwebapi/tests/resources/buildapi/deliverable.json: -------------------------------------------------------------------------------- 1 | { 2 | "_total_count":14, 3 | "_next_url":null, 4 | "_page_count":14, 5 | "_list":[ 6 | { 7 | "_this_url":"/ob/deliverable/182969046/", 8 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/app_monitor_dependency.tar.gz", 9 | "_currenttime":"2014-06-27 14:48:03.720880", 10 | "_build_url":"/ob/build/1935022/", 11 | "_this_resource":"deliverable", 12 | "size_in_mb":9, 13 | "path":"publish/app_monitor_dependency.tar.gz", 14 | "id":182969046, 15 | "deliverable_class":"external" 16 | }, 17 | { 18 | "_this_url":"/ob/deliverable/182969047/", 19 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/VMware-oms-ha-0.5.0.8-1935022.x86_64.rpm", 20 | "_currenttime":"2014-06-27 14:48:03.720880", 21 | "_build_url":"/ob/build/1935022/", 22 | "_this_resource":"deliverable", 23 | "size_in_mb":1, 24 | "path":"publish/VMware-oms-ha-0.5.0.8-1935022.x86_64.rpm", 25 | "id":182969047, 26 | "deliverable_class":"external" 27 | }, 28 | { 29 | "_this_url":"/ob/deliverable/182969048/", 30 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/VMware-oms-sysctl-0.5.0.8-1935022.x86_64.rpm", 31 | "_currenttime":"2014-06-27 14:48:03.720880", 32 | "_build_url":"/ob/build/1935022/", 33 | "_this_resource":"deliverable", 34 | "size_in_mb":1, 35 | "path":"publish/VMware-oms-sysctl-0.5.0.8-1935022.x86_64.rpm", 36 | "id":182969048, 37 | "deliverable_class":"external" 38 | }, 39 | { 40 | "_this_url":"/ob/deliverable/182969049/", 41 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/management-server-0.5.0.0-1935022.oss.tgz", 42 | "_currenttime":"2014-06-27 14:48:03.720880", 43 | "_build_url":"/ob/build/1935022/", 44 | "_this_resource":"deliverable", 45 | "size_in_mb":574, 46 | "path":"publish/management-server-0.5.0.0-1935022.oss.tgz", 47 | "id":182969049, 48 | "deliverable_class":"external" 49 | }, 50 | { 51 | "_this_url":"/ob/deliverable/182969050/", 52 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/VMware-openstack-core-0.5.0.8-1935022.noarch.rpm", 53 | "_currenttime":"2014-06-27 14:48:03.720880", 54 | "_build_url":"/ob/build/1935022/", 55 | "_this_resource":"deliverable", 56 | "size_in_mb":32, 57 | "path":"publish/VMware-openstack-core-0.5.0.8-1935022.noarch.rpm", 58 | "id":182969050, 59 | "deliverable_class":"external" 60 | }, 61 | { 62 | "_this_url":"/ob/deliverable/182969051/", 63 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ova/VMware-OpenStack-0.5.0.0-1935022_OVF10.ova", 64 | "_currenttime":"2014-06-27 14:48:03.720880", 65 | "_build_url":"/ob/build/1935022/", 66 | "_this_resource":"deliverable", 67 | "size_in_mb":1647, 68 | "path":"publish/openstack_vapp/exports/ova/VMware-OpenStack-0.5.0.0-1935022_OVF10.ova", 69 | "id":182969051, 70 | "deliverable_class":"external" 71 | }, 72 | { 73 | "_this_url":"/ob/deliverable/182969101/", 74 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/openstack-template_VMwareOSTemplate-1.1.4.0-1934096-system.vmdk", 75 | "_currenttime":"2014-06-27 14:48:03.720880", 76 | "_build_url":"/ob/build/1935022/", 77 | "_this_resource":"deliverable", 78 | "size_in_mb":756, 79 | "path":"publish/openstack_vapp/exports/ovf/openstack-template_VMwareOSTemplate-1.1.4.0-1934096-system.vmdk", 80 | "id":182969101, 81 | "deliverable_class":"external" 82 | }, 83 | { 84 | "_this_url":"/ob/deliverable/182969102/", 85 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/management-server_management-server-0.5.0.0-1935022-cloud-components.vmdk", 86 | "_currenttime":"2014-06-27 14:48:03.720880", 87 | "_build_url":"/ob/build/1935022/", 88 | "_this_resource":"deliverable", 89 | "size_in_mb":15, 90 | "path":"publish/openstack_vapp/exports/ovf/management-server_management-server-0.5.0.0-1935022-cloud-components.vmdk", 91 | "id":182969102, 92 | "deliverable_class":"external" 93 | }, 94 | { 95 | "_this_url":"/ob/deliverable/182969103/", 96 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.mf", 97 | "_currenttime":"2014-06-27 14:48:03.720880", 98 | "_build_url":"/ob/build/1935022/", 99 | "_this_resource":"deliverable", 100 | "size_in_mb":1, 101 | "path":"publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.mf", 102 | "id":182969103, 103 | "deliverable_class":"external" 104 | }, 105 | { 106 | "_this_url":"/ob/deliverable/182969104/", 107 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.cert", 108 | "_currenttime":"2014-06-27 14:48:03.720880", 109 | "_build_url":"/ob/build/1935022/", 110 | "_this_resource":"deliverable", 111 | "size_in_mb":1, 112 | "path":"publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.cert", 113 | "id":182969104, 114 | "deliverable_class":"external" 115 | }, 116 | { 117 | "_this_url":"/ob/deliverable/182969105/", 118 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.ovf", 119 | "_currenttime":"2014-06-27 14:48:03.720880", 120 | "_build_url":"/ob/build/1935022/", 121 | "_this_resource":"deliverable", 122 | "size_in_mb":1, 123 | "path":"publish/openstack_vapp/exports/ovf/VMware-OpenStack-0.5.0.0-1935022_OVF10.ovf", 124 | "id":182969105, 125 | "deliverable_class":"external" 126 | }, 127 | { 128 | "_this_url":"/ob/deliverable/182969106/", 129 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/openstack_vapp/exports/ovf/management-server_management-server-0.5.0.0-1935022-system.vmdk", 130 | "_currenttime":"2014-06-27 14:48:03.720880", 131 | "_build_url":"/ob/build/1935022/", 132 | "_this_resource":"deliverable", 133 | "size_in_mb":877, 134 | "path":"publish/openstack_vapp/exports/ovf/management-server_management-server-0.5.0.0-1935022-system.vmdk", 135 | "id":182969106, 136 | "deliverable_class":"external" 137 | }, 138 | { 139 | "_this_url":"/ob/deliverable/182969107/", 140 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/MD5SUM.txt", 141 | "_currenttime":"2014-06-27 14:48:03.720880", 142 | "_build_url":"/ob/build/1935022/", 143 | "_this_resource":"deliverable", 144 | "size_in_mb":0, 145 | "path":"publish/MD5SUM.txt", 146 | "id":182969107, 147 | "deliverable_class":"external" 148 | }, 149 | { 150 | "_this_url":"/ob/deliverable/182969108/", 151 | "_download_url":"http://buildweb.eng.vmware.com/ob/api/1935022/deliverable/?file=publish/SHA1SUM.txt", 152 | "_currenttime":"2014-06-27 14:48:03.720880", 153 | "_build_url":"/ob/build/1935022/", 154 | "_this_resource":"deliverable", 155 | "size_in_mb":0, 156 | "path":"publish/SHA1SUM.txt", 157 | "id":182969108, 158 | "deliverable_class":"external" 159 | } 160 | ], 161 | "_previous_url":null 162 | } -------------------------------------------------------------------------------- /tests/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | install_projects() { 5 | pushd $1 6 | if [ -e requirements.txt ]; then 7 | pip install -r requirements.txt 8 | fi 9 | pip install -e . 10 | popd 11 | } 12 | 13 | export SCRIPT_DIR=$(cd $(dirname "$0") && pwd) 14 | virtualenv .venv 15 | source .venv/bin/activate 16 | pushd "$SCRIPT_DIR" 17 | install_projects "shellutil" 18 | install_projects "sshutil" 19 | install_projects "buildwebapi" 20 | install_projects "omsclient" 21 | install_projects "pyVmomiwrapper" 22 | install_projects "panda" 23 | popd 24 | 25 | panda tempest install 26 | echo "---------------------------------------------" 27 | echo "" 28 | echo "Test projects are successfully installed." 29 | echo "Please run command: source .venv/bin/activate" 30 | -------------------------------------------------------------------------------- /tests/omsclient/README.rst: -------------------------------------------------------------------------------- 1 | oms-client 2 | ============= 3 | 4 | A python library wraps oms rest api client. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com oms-client 13 | 14 | 15 | Usage 16 | ====== 17 | .. code:: python 18 | 19 | oms_ctl = OmsController('192.168.111.151', 'root', 'vmware') 20 | resp = oms_ctl.create_deployment_plan(spec_str) 21 | 22 | 23 | -------------------------------------------------------------------------------- /tests/omsclient/omsclient/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/omsclient/omsclient/__init__.py -------------------------------------------------------------------------------- /tests/omsclient/omsclient/restclient.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import requests 3 | 4 | LOG = logging.getLogger(__name__) 5 | requests.packages.urllib3.disable_warnings() 6 | 7 | 8 | class RestClient(object): 9 | """OMS RestClient 10 | 11 | This is the client implementation based on "requests". 12 | """ 13 | _URL_TEMPLATE_PREFIX = "https://%s:8443/oms/%s" 14 | 15 | def __init__(self, server, username, password): 16 | """Create a connection to the remote OMS server 17 | 18 | :param server: IP or hostname of the OMS server 19 | :param username: User name 20 | :param password: Password 21 | :return: None 22 | """ 23 | self._server = server 24 | self._username = username 25 | self._password = password.replace('+', '%2B') 26 | 27 | # TODO Do we need to have logout logic? 28 | self._session = self._login() 29 | 30 | def _api_url(self, path): 31 | api_url_template = "api/%s" 32 | api_path = api_url_template % path 33 | return self._URL_TEMPLATE_PREFIX % (self._server, api_path) 34 | 35 | def _login_url(self): 36 | login_url_template = \ 37 | "j_spring_security_check?j_username=%s&j_password=%s" 38 | login_url = login_url_template % (self._username, self._password) 39 | return self._URL_TEMPLATE_PREFIX % (self._server, login_url) 40 | 41 | def _login(self): 42 | session = requests.Session() 43 | 44 | LOG.debug("Request login...") 45 | response = session.post(self._login_url(), verify=False) 46 | LOG.debug(response) 47 | 48 | return session 49 | 50 | def login(self): 51 | self._session = self._login() 52 | 53 | def do_get(self, path): 54 | url = self._api_url(path) 55 | 56 | LOG.debug("Request GET: %s" % url) 57 | response = self._session.get(url, verify=False) 58 | LOG.debug(response) 59 | 60 | return response 61 | 62 | def do_delete(self, path, object_id): 63 | url = self._api_url(path) + "/" + object_id 64 | 65 | LOG.debug("Request DELETE: %s" % url) 66 | response = self._session.delete(url, verify=False) 67 | LOG.debug(response) 68 | return response 69 | 70 | def do_post(self, path, data): 71 | url = self._api_url(path) 72 | headers = {'Content-type': 'application/json'} 73 | 74 | LOG.debug("Request POST: %s" % url) 75 | response = self._session.post(url, data, headers=headers, verify=False) 76 | LOG.debug(response) 77 | return response 78 | 79 | def do_put(self, path, data): 80 | url = self._api_url(path) 81 | headers = {'Content-type': 'application/json'} 82 | LOG.debug("Request PUT: %s" % url) 83 | 84 | response = self._session.put(url, data, headers=headers, verify=False) 85 | LOG.debug(response) 86 | return response 87 | -------------------------------------------------------------------------------- /tests/omsclient/omsclient/utils.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import logging 3 | import re 4 | import time 5 | 6 | 7 | LOG = logging.getLogger(__name__) 8 | 9 | 10 | class TimeoutError(Exception): 11 | """Time out exceptions""" 12 | 13 | 14 | class NotFoundError(Exception): 15 | """Not found exceptions""" 16 | 17 | 18 | class TaskError(Exception): 19 | """Task status exceptions""" 20 | 21 | 22 | def get_task_id(url): 23 | LOG.debug('Grep task id from url: %s', url) 24 | pattern = re.compile(r'/task/(\d+)') 25 | result = pattern.search(url) 26 | if result: 27 | task_id = result.group(1) 28 | LOG.debug('Task id: %s' % task_id) 29 | return task_id 30 | else: 31 | raise NotFoundError('Can not get task id from url %s', url) 32 | 33 | 34 | def wait_for_task_completed(oms_ctl, task_id, delay=30, timeout=1200): 35 | begin_poll = datetime.datetime.now() 36 | status_list = ['COMPLETED', 'STOPPING', "STOPPED", 'FAILED'] 37 | while (datetime.datetime.now() - begin_poll).seconds < timeout: 38 | task = oms_ctl.get_task(task_id) 39 | if task['status'] in status_list: 40 | LOG.debug('Task %s status: %s', task_id, task['status']) 41 | return task['status'], task['errorMessage'] 42 | time.sleep(delay) 43 | timeout -= delay 44 | raise TimeoutError('Waited %s seconds for task %s' % (timeout, task_id)) 45 | 46 | 47 | def validate_task_succeeded(oms_ctl, task_name, resp, delay=30, timeout=1200): 48 | LOG.debug('Responce header: %s', resp.headers) 49 | LOG.debug('Responce body: %s', resp.text) 50 | if resp.status_code != 202: 51 | raise TaskError('%s failed: %s' % (task_name, resp.text)) 52 | url = resp.headers['Location'] 53 | LOG.debug('Retrieve %s from response of %s', url, task_name) 54 | task_id = get_task_id(url) 55 | status, msg = wait_for_task_completed(oms_ctl, task_id, delay, timeout) 56 | if status != 'COMPLETED': 57 | raise TaskError('%s failed: %s' % (task_name, msg)) 58 | -------------------------------------------------------------------------------- /tests/omsclient/requirements.txt: -------------------------------------------------------------------------------- 1 | requests -------------------------------------------------------------------------------- /tests/omsclient/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='oms-client', 5 | version='0.0.1', 6 | scripts=[], 7 | packages=['omsclient'], 8 | include_package_data=True, 9 | install_requires=[ 10 | 'requests' 11 | ] 12 | ) 13 | -------------------------------------------------------------------------------- /tests/panda/README.rst: -------------------------------------------------------------------------------- 1 | panda 2 | ============= 3 | 4 | A python library for deploying and testing VIO. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com panda 13 | 14 | 15 | Usage 16 | ====== 17 | System libs are required by tempest: 18 | python-dev, libffi-dev, libssl-dev, git, swig, libxml2-dev, libxslt-dev. Install them by apt-get install. 19 | 20 | python virtualenv is required by 'panda tempest' command, install them beforehand. 21 | 22 | Get help: 23 | panda [command] [sub command] -h 24 | 25 | Deploy VIO vApp: 26 | panda oms deploy '192.168.111.130' 'Administrator@vsphere.local' 'Admin!23' 'vio-datacenter' 'mgmt_cluster' 'vdnetSharedStorage' 'VM Network' 'http://build-squid.eng.vmware.com/build/mts/release/bora-3302254/publish/openstack_vapp/exports/ova/VMware-OpenStack-2.0.0.0-3302254_OVF10.ova' --ip '192.168.111.151' --netmask '255.255.255.0' --dns '192.168.111.1' --password 'vmware' --gateway '192.168.111.1' 27 | 28 | Config omjs.properties: 29 | panda oms config-omjs '192.168.111.151' 'vmware' 'Administrator@vsphere.local' 'Admin!23' 'oms.use_linked_clone=true,oms.skip_cluster_vmotion_check=true,oms.disable_datastores_anti_affinity=true' 30 | 31 | Remove VIO vApp: 32 | panda oms remove '192.168.111.130' 'Administrator@vsphere.local' 'Admin!23' '^VMware-OpenStack.*\d$' 33 | 34 | Create Openstack cluster: 35 | panda cluster create '192.168.111.151' 'Administrator@vsphere.local' 'Admin!23' cluster_spec.json 36 | 37 | Delete Openstack cluster: 38 | panda cluster delete "192.168.111.151" 'Administrator@vsphere.local' 'Admin!23' 39 | 40 | Enable LDAP backend admin user on multiple backend setup. 41 | panda openstack enable_ldap_admin '192.168.111.160' 'admin' 'vmware' 'vioadmin@vio.com' 42 | 43 | Run tempest: 44 | Step 1 45 | Install tempest: 46 | panda tempest install 47 | 48 | Step 2 49 | Configure tempest against a SQL keystone/NSXv neutron backend VIO: 50 | panda tempest config '192.168.111.160' 'admin' 'vmware' 'nsxv' --ext-cidr '192.168.112.0/24' --ext-start-ip '192.168.112.170' --ext-end-ip '192.168.112.200' --ext-gateway '192.168.112.1' --compute-nodes 2 --nsx-manager '192.168.111.15' --nsx-user 'admin' --nsx-password 'default' 51 | Configure tempest against a SQL keystone/DVS neutron backend VIO: 52 | panda tempest config '192.168.111.153' 'admin' 'vmware' 'dvs' --compute-nodes 2 53 | Configure tempest against a LDAP keystone backend VIO: 54 | panda tempest config '192.168.111.160' 'vioadmin@vio.com' 'VMware1!' 'nsxv' --credentials-provider 'pre-provisioned' --user1 'xiaoy@vio.com' --user1-password 'VMware1!' --user2 'sren@vio.com' --user2-password 'VMware1!' --ext-cidr '192.168.112.0/24' --ext-start-ip '192.168.112.170' --ext-end-ip '192.168.112.200' --ext-gateway '192.168.112.1' --nsx-manager '192.168.111.15' --nsx-user 'admin' --nsx-password 'default' 55 | 56 | Step 3 57 | Run tempest tests: 58 | panda tempest run 'keystone,glance,nova,cinder,neutron,heat,scenario,nsxv,nsxt' 59 | 60 | Run VMware tempest: 61 | Step 1 62 | Install VMware tempest: 63 | panda vmware_tempest install 64 | 65 | Step 2 66 | Configure VMware tempest 67 | panda vmware_tempest config '192.168.111.130' 'Administrator@vsphere.local' 'Admin!23' 68 | 69 | Step 3 70 | Run VMware tempest tests: 71 | panda vmware_tempest run 72 | 73 | All in one command: 74 | panda go oms_spec.json cluster_spec.json --tests 'keystone,glance,nova,cinder,neutron,heat,scenario,vmware' 75 | -------------------------------------------------------------------------------- /tests/panda/panda/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/panda/panda/__init__.py -------------------------------------------------------------------------------- /tests/panda/panda/build_utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import logging 3 | 4 | from buildwebapi import api as buildapi 5 | from task_utils import safe_run 6 | 7 | 8 | LOG = logging.getLogger(__name__) 9 | 10 | 11 | def get_build_type(build_id): 12 | build = get_build(build_id) 13 | LOG.debug('%s is %s build', build_id, build.buildtype) 14 | return build.buildtype 15 | 16 | 17 | def get_build_id_and_system(build_id): 18 | build_system = 'ob' 19 | if '-' in str(build_id): 20 | temp = build_id.split('-') 21 | build_id = temp[1] 22 | build_system = temp[0] 23 | return build_id, build_system 24 | 25 | 26 | def get_ova_url(build_id): 27 | return get_url(build_id, '_OVF10.ova') 28 | 29 | 30 | def get_patch_url(build_id): 31 | return get_url(build_id, '_all.deb') 32 | 33 | 34 | def get_upgrade_url(build_id): 35 | return get_url(build_id, '-upgrade-') 36 | 37 | 38 | def get_url(build_id, deliverable_name): 39 | build = get_build(build_id) 40 | deliverables = buildapi.ListResource.by_url(build._deliverables_url) 41 | deliverable = [d for d in deliverables 42 | if d.matches(path=deliverable_name)][0] 43 | LOG.debug('Download URL of %s is %s', build_id, deliverable._download_url) 44 | return deliverable._download_url 45 | 46 | 47 | def get_product(build_id): 48 | build = get_build(build_id) 49 | LOG.debug('Product of %s is %s.', build_id, build.product) 50 | return build.product 51 | 52 | 53 | def download_ova(build_id, path=None): 54 | ova_url = get_ova_url(build_id) 55 | return download_file(ova_url, path) 56 | 57 | 58 | def download_patch(build_id, path=None): 59 | deb_url = get_patch_url(build_id) 60 | return download_file(deb_url, path) 61 | 62 | 63 | def download_upgrade(build_id, path=None): 64 | deb_url = get_upgrade_url(build_id) 65 | return download_file(deb_url, path) 66 | 67 | 68 | def download_file(url, path=None): 69 | file_name = os.path.basename(url) 70 | if path: 71 | abs_path = os.path.join(path, file_name) 72 | else: 73 | abs_path = os.path.join(os.getcwd(), file_name) 74 | if not os.path.exists(abs_path): 75 | cmd = "wget --no-verbose -O %s %s" % (abs_path, url) 76 | safe_run(cmd, 'download %s' % url) 77 | LOG.info('Downloaded %s to %s', url, abs_path) 78 | else: 79 | LOG.info('%s already exists, skip downloading it.', abs_path) 80 | return abs_path 81 | 82 | 83 | def get_latest_build_url(branch, build_type, product='vmw-openstack'): 84 | build_id = get_latest_build_id(branch, build_type, product) 85 | return get_ova_url(build_id) 86 | 87 | 88 | def get_latest_build_id(branch, build_type, product='vmw-openstack'): 89 | return buildapi.MetricResource.by_name('build', 90 | product=product, 91 | buildstate='succeeded', 92 | buildtype=build_type, 93 | branch=branch).get_max_id() 94 | 95 | 96 | def get_build(build_id): 97 | build_id, build_system = get_build_id_and_system(build_id) 98 | return buildapi.ItemResource.by_id('build', int(build_id), build_system) 99 | 100 | 101 | def get_build_version(build_id): 102 | build = get_build(build_id) 103 | LOG.debug('Version of %s is %s.', build_id, build.version) 104 | return build.version 105 | -------------------------------------------------------------------------------- /tests/panda/panda/data/dvs-included-neutron.txt: -------------------------------------------------------------------------------- 1 | tempest.api.network.admin.test_quotas.QuotasTest.test_quotas[id-2390f766-836d-40ef-9aeb-e810d78207fb] 2 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_create_port_on_non_existent_network[id-13d3b106-47e6-4b9b-8d53-dae947f092fe,negative] 3 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_delete_non_existent_network[id-03795047-4a94-4120-a0a1-bd376e36fd4e,negative] 4 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_delete_non_existent_port[id-49ec2bbd-ac2e-46fd-8054-798e679ff894,negative] 5 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_delete_non_existent_subnet[id-a176c859-99fb-42ec-a208-8a85b552a239,negative] 6 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_show_non_existent_network[id-9293e937-824d-42d2-8d5b-e985ea67002a,negative] 7 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_show_non_existent_port[id-a954861d-cbfd-44e8-b0a9-7fab111f235d,negative] 8 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_show_non_existent_subnet[id-d746b40c-5e09-4043-99f7-cba1be8b70df,negative] 9 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_update_non_existent_network[id-98bfe4e3-574e-4012-8b17-b2647063de87,negative] 10 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_update_non_existent_port[id-cf8eef21-4351-4f53-adcd-cc5cb1e76b92,negative] 11 | tempest.api.network.test_networks_negative.NetworksNegativeTestJSON.test_update_non_existent_subnet[id-1cc47884-ac52-4415-a31c-e7ce5474a868,negative] 12 | tempest.api.network.admin.test_networks_admin_actions.AdminNetworksTestJSON.test_create_update_delete_network_subnet[id-15d3d53c-3328-401f-b8f5-3a29aee2ea3a,smoke] 13 | tempest.api.network.admin.test_networks_admin_actions.AdminNetworksTestJSON.test_list_networks[id-b86d50ef-39a7-4136-8c89-e5e534fe92aa,smoke] 14 | tempest.api.network.admin.test_networks_admin_actions.AdminNetworksTestJSON.test_show_network[id-838aee5f-92f2-47b9-86c6-629a04aa6269,smoke] 15 | tempest.api.network.admin.test_networks_admin_actions.AdminNetworksTestJSON.test_show_subnet[id-ee3f8b79-da3f-4394-9bea-012488202257,smoke] 16 | tempest.api.network.admin.test_ports_admin_actions.AdminPortsTestJSON.test_create_update_delete_port[id-c3f751d4-e358-44b9-bfd2-3d563c4a2d04,smoke] 17 | tempest.api.network.admin.test_ports_admin_actions.AdminPortsTestJSON.test_list_ports[id-c5f74042-c512-4569-b9b9-bc2bf46e77e1,smoke] 18 | tempest.api.network.admin.test_ports_admin_actions.AdminPortsTestJSON.test_list_ports_fields[id-2775f96c-a09b-49e1-a5a4-adb83a3e91c7,smoke] 19 | tempest.api.network.admin.test_ports_admin_actions.AdminPortsTestJSON.test_show_port[id-d3dcd23b-7d5a-4720-8d88-473fb154d609,smoke] -------------------------------------------------------------------------------- /tests/panda/panda/data/logging.json: -------------------------------------------------------------------------------- 1 | { 2 | "version": 1, 3 | "disable_existing_loggers": false, 4 | "formatters": { 5 | "simple": { 6 | "format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s" 7 | } 8 | }, 9 | "handlers": { 10 | "console": { 11 | "class": "logging.StreamHandler", 12 | "level": "INFO", 13 | "formatter": "simple", 14 | "stream": "ext://sys.stdout" 15 | }, 16 | 17 | "file_handler": { 18 | "class": "logging.handlers.RotatingFileHandler", 19 | "level": "DEBUG", 20 | "formatter": "simple", 21 | "filename": "panda.log", 22 | "maxBytes": 10485760, 23 | "backupCount": 20, 24 | "encoding": "utf8" 25 | } 26 | }, 27 | "loggers": { 28 | "panda": { 29 | "level": "DEBUG", 30 | "handlers": ["console"], 31 | "propagate": "1" 32 | }, 33 | "buildwebapi": { 34 | "level": "DEBUG", 35 | "handlers": ["console"], 36 | "propagate": "1" 37 | }, 38 | "omsclient": { 39 | "level": "DEBUG", 40 | "handlers": ["console"], 41 | "propagate": "1" 42 | }, 43 | "pyVmomiwrapper": { 44 | "level": "DEBUG", 45 | "handlers": ["console"], 46 | "propagate": "1" 47 | }, 48 | "shellutil": { 49 | "level": "DEBUG", 50 | "handlers": ["console"], 51 | "propagate": "1" 52 | }, 53 | "sshutil": { 54 | "level": "DEBUG", 55 | "handlers": ["console"], 56 | "propagate": "1" 57 | }, 58 | "paramiko": { 59 | "level": "INFO", 60 | "handlers": ["file_handler"], 61 | "propagate": "0" 62 | } 63 | }, 64 | "root": { 65 | "level": "DEBUG", 66 | "handlers": ["file_handler"] 67 | } 68 | } -------------------------------------------------------------------------------- /tests/panda/panda/data/tempest.conf.template: -------------------------------------------------------------------------------- 1 | [DEFAULT] 2 | debug = True 3 | log_file = tempest.log 4 | use_stderr = False 5 | 6 | [auth] 7 | tempest_roles = TOBEFILLED 8 | use_dynamic_credentials = TOBEFILLED 9 | create_isolated_networks = TOBEFILLED 10 | admin_domain_name = TOBEFILLED 11 | admin_tenant_name = TOBEFILLED 12 | admin_password = TOBEFILLED 13 | admin_username = TOBEFILLED 14 | 15 | [compute] 16 | image_ref = TOBEFILLED 17 | image_ref_alt = TOBEFILLED 18 | flavor_ref = TOBEFILLED 19 | flavor_ref_alt = TOBEFILLED 20 | build_timeout = 1800 21 | fixed_network_name = TOBEFILLED 22 | endpoint_type = internalURL 23 | volume_device_name = sdb 24 | 25 | [compute-feature-enabled] 26 | pause = false 27 | block_migrate_cinder_iscsi = true 28 | block_migration_for_live_migration = true 29 | resize = true 30 | console_output = false 31 | personality = false 32 | scheduler_available_filters = RetryFilter,AvailabilityZoneFilter,RamFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter 33 | 34 | [dashboard] 35 | login_url = TOBEFILLED 36 | dashboard_url = TOBEFILLED 37 | 38 | [identity] 39 | disable_ssl_certificate_validation = true 40 | auth_version = TOBEFILLED 41 | uri_v3 = TOBEFILLED 42 | uri = TOBEFILLED 43 | region = nova 44 | 45 | [image] 46 | build_timeout = 600 47 | endpoint_type = internalURL 48 | disk_formats = vmdk,raw,qcow2,vdi,iso,vhd 49 | container_formats = bare,ova 50 | 51 | [input-scenario] 52 | ssh_user_regex = [["^.*[Uu]buntu.*$", "ubuntu"]] 53 | 54 | [network] 55 | build_timeout = 500 56 | endpoint_type = internalURL 57 | 58 | [network-feature-enabled] 59 | api_extensions = binding, dist-router, multi-provider, provider, quotas, external-net, extraroute, router, security-group 60 | ipv6 = false 61 | 62 | [object-storage] 63 | endpoint_type = internalURL 64 | operator_role = storage-ops-tempest 65 | 66 | [orchestration] 67 | endpoint_type = internalURL 68 | build_timeout = 2400 69 | instance_type = TOBEFILLED 70 | 71 | [oslo_concurrency] 72 | lock_path = /tmp/stack/data/tempest 73 | 74 | [scenario] 75 | dhcp_client = dhclient 76 | 77 | [service_available] 78 | neutron = True 79 | heat = True 80 | swift = False 81 | cinder = True 82 | nova = True 83 | glance = True 84 | ceilometer = false 85 | 86 | [validation] 87 | ssh_timeout = 900 88 | image_ssh_user = ubuntu 89 | image_ssh_password = vmware 90 | 91 | [volume] 92 | build_timeout = 900 93 | endpoint_type = internalURL 94 | storage_protocol = vmdk 95 | vendor_name = VMware 96 | disk_format = vmdk 97 | volume_size = 5 98 | 99 | [volume-feature-enabled] 100 | backup = False 101 | -------------------------------------------------------------------------------- /tests/panda/panda/data/vmware-excluded-tests.txt: -------------------------------------------------------------------------------- 1 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_bronze_eager 2 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_bronze_thick 3 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_bronze_thin 4 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_gold_eager 5 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_gold_thick 6 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_gold_thin 7 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_silver_eager 8 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_silver_thick 9 | vmware_tempest.cinder.tests.profile_type_mix_test.profile_type_mix.test_mixprofile_silver_thin 10 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_clonevol_for_bronze 11 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_clonevol_for_gold 12 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_clonevol_for_silver 13 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_snapshotvol_for_bronze 14 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_snapshotvol_for_gold 15 | vmware_tempest.cinder.tests.profile_type_ops_test.profile_type_ops.test_profile_snapshotvol_for_silver 16 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_invalid_profile 17 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_bronze_poweroff 18 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_bronze_poweron 19 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_gold_poweroff 20 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_gold_poweron 21 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_silver_poweroff 22 | vmware_tempest.cinder.tests.profile_type_test.profile_type_tests.test_profile_silver_poweron 23 | vmware_tempest.cinder.tests.clone_type_test.clone_type.test_create_clone_type_full 24 | vmware_tempest.cinder.tests.clone_type_test.clone_type.test_create_clone_type_linked 25 | vmware_tempest.cinder.tests.disk_type_snapshot_test.disk_type_snapshot.test_create_vol_type_thick_snapshot 26 | 27 | #Add new filter list due to class name change 28 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_bronze_eager 29 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_bronze_thick 30 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_bronze_thin 31 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_gold_eager 32 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_gold_thick 33 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_gold_thin 34 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_silver_eager 35 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_silver_thick 36 | vmware_tempest.cinder.tests.profile_type_mix_test.ProfileTypeMixTests.test_mixprofile_silver_thin 37 | vmware_tempest.cinder.tests.clone_type_test.CloneTypeTests.test_create_clone_type_full 38 | vmware_tempest.cinder.tests.clone_type_test.CloneTypeTests.test_create_clone_type_linked 39 | vmware_tempest.cinder.tests.disk_type_snapshot_test.DiskTypeSnapshotTests.test_create_vol_type_thick_snapshot 40 | 41 | #Exclude backup/restore cases 42 | vmware_tempest.cinder.tests.test_backup_restore.backup_restore.test_create_backup 43 | vmware_tempest.cinder.tests.test_backup_restore.backup_restore.test_create_backup_with_backing 44 | vmware_tempest.cinder.tests.test_backup_restore.backup_restore.test_restore_to_existing_volume 45 | vmware_tempest.cinder.tests.test_backup_restore.backup_restore.test_restore_to_new_volume 46 | 47 | #Exclude non-p0 profile cases 48 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image_profile 49 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_create_vol_profiletype 50 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_profiletype_attach 51 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_profiletype_detach 52 | vmware_tempest.cinder.tests.test_retype.retype.test_retype_mix_type 53 | vmware_tempest.cinder.tests.test_retype.retype.test_retype_mix_type_with_snapshot 54 | vmware_tempest.cinder.tests.test_volume_backing_resilience.VolumeBackingResiliencyTests.test_disk_device_removal 55 | 56 | #TBD fix this case in case of extending failure 57 | vmware_tempest.cinder.tests.disk_type_extend_test.disk_type_extend.test_extend_vol_morethan_datastore_size 58 | 59 | # Pending on bug #1666232 60 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_glance_image 61 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image 62 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image_attach_vol 63 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image_delete_vol 64 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image_detach_vol 65 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_create_vol_from_image_extend_vol 66 | vmware_tempest.cinder.tests.expire_session_img_vol_test.session_exp_vol_img.test_upload_vol_to_image 67 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_clone_vol_withbacking 68 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_clone_vol_without_backing 69 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_create_vol 70 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_create_vol_clonetype 71 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_create_vol_disktype 72 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_create_vol_invalid_profile 73 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_delete_clone_vol_withbacking 74 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_snapshot_del_withBacking 75 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_snapshot_delete 76 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_snapshot_vol 77 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_snapshot_vol_withBacking 78 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_attach 79 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_delete 80 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_delete_withbacking 81 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_detach 82 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_from_snapshot 83 | vmware_tempest.cinder.tests.expire_session_test.expire_session.test_vol_from_snapshot_withBacking 84 | 85 | # Pending on bug #1695079 86 | vmware_tempest.scenario.test_iso_scenario.IsoScenarioTests.test_boot_from_volume 87 | vmware_tempest.scenario.test_iso_scenario.IsoScenarioTests.test_boot_from_volume_snapshot -------------------------------------------------------------------------------- /tests/panda/panda/end_to_end.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import time 3 | 4 | import oms_utils 5 | from setup import VIO 6 | from test import Test 7 | from test import PASS 8 | 9 | 10 | LOG = logging.getLogger(__name__) 11 | 12 | 13 | def vio_orchestration(oms_spec, log_dir, cluster_spec=None, tests=None): 14 | """ VIO end to end CI orchestration layer. 15 | Deploy vApp, create OpenStack cluster and run various tests. 16 | 17 | :param oms_spec: oms spec dict, see below sample. 18 | { 19 | "build": "3037963", 20 | "ova_path": "", 21 | "username": "viouser", 22 | "password": "vmware", 23 | "host_ip": "192.168.111.151", 24 | "gateway": "192.168.111.1", 25 | "netmask": "255.255.255.0", 26 | "dns": "192.168.111.1", 27 | "ntp_server": "", 28 | "omjs_properties": {"oms.use_linked_clone": "true", 29 | "oms.skip_cluster_vmotion_check": "true", 30 | "oms.disable_datastores_anti_affinity": "true"}, 31 | "patches": ["vio-patch-201_2.0.1.3309787_all.deb"], 32 | "vc_host": "192.168.111.130", 33 | "vc_user": "Administrator@vsphere.local", 34 | "vc_password": "Admin!23", 35 | "datacenter": "vio-datacenter", 36 | "cluster": "mgmt_cluster", 37 | "datastore": "vdnetSharedStorage", 38 | "network": "VM Network", 39 | "openstack_creds_provider": "dynamic", 40 | "ext_net_cidr": "192.168.112.0/24", 41 | "ext_net_start_ip": "192.168.112.170", 42 | "ext_net_end_ip": "192.168.112.200", 43 | "ext_net_gateway": "192.168.112.1", 44 | "public_vip_range": "192.168.112.201-192.168.112.203", 45 | "private_vip_range": "192.168.111.201-192.168.111.203" 46 | } 47 | :param log_dir: directory deployment and test logs. 48 | :param cluster_spec: dict spec for creating OpenStack cluster from oms api. 49 | :param tests: string test name separated by comma. 50 | """ 51 | LOG.debug('OMS spec: %s' % oms_spec) 52 | LOG.debug('Log path: %s' % log_dir) 53 | result = PASS 54 | vio_setup = VIO(oms_spec, cluster_spec, log_dir) 55 | vio_setup.deploy_vapp() 56 | if 'omjs_properties' in oms_spec: 57 | vio_setup.config_omjs(oms_spec['omjs_properties']) 58 | if 'version' not in oms_spec: 59 | oms_spec['version'] = vio_setup.get_version() 60 | if cluster_spec: 61 | LOG.debug('Cluster spec: %s' % cluster_spec) 62 | vio_setup.deploy_openstack() 63 | if 'compute_clusters' in oms_spec \ 64 | and len(oms_spec['compute_clusters']) > 1: 65 | day2_compute_clusters = oms_spec['compute_clusters'][1:] 66 | for cluster in day2_compute_clusters: 67 | vio_setup.add_compute_cluster(cluster) 68 | if 'patches' in oms_spec: 69 | public_vip_range = oms_utils.get_ip_range(oms_spec, 'public_vip_range') 70 | private_vip_range = oms_utils.get_ip_range(oms_spec, 71 | 'private_vip_range') 72 | for patch in oms_spec['patches']: 73 | # Applying patches continuously is easy to fail. In real world, 74 | # user won't apply them like this. So sleep 3 minutes beforehand. 75 | LOG.debug('Sleep 3 minutes before patching.') 76 | time.sleep(60 * 3) 77 | vio_setup.apply_patch(patch) 78 | if '-upgrade-' in patch: 79 | public_vip = public_vip_range.next().format() 80 | private_vip = private_vip_range.next().format() 81 | cluster_spec = vio_setup.upgrade(public_vip, private_vip) 82 | if tests: 83 | LOG.debug('Tests: %s' % tests) 84 | result = Test.run_tests(tests, log_dir, oms_spec, cluster_spec) 85 | vio_setup.get_support_bundle() 86 | return result 87 | -------------------------------------------------------------------------------- /tests/panda/panda/exceptions.py: -------------------------------------------------------------------------------- 1 | class ProvisionError(Exception): 2 | """Provisioning exceptions""" 3 | 4 | 5 | class NotSupportedError(Exception): 6 | """Not supported exceptions""" 7 | 8 | 9 | class NotCompletedError(Exception): 10 | """Not completed exceptions""" 11 | 12 | 13 | class TimeoutError(Exception): 14 | """Time out exceptions""" 15 | 16 | 17 | class NotFoundError(Exception): 18 | """Not Found exceptions""" 19 | -------------------------------------------------------------------------------- /tests/panda/panda/logging_utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import logging.config 4 | 5 | 6 | def setup_logging(log_cfg='logging.json', env_key='LOG_CFG', log_file=None): 7 | value = os.getenv(env_key, None) 8 | if value: 9 | path = value 10 | elif os.path.exists(log_cfg): 11 | path = log_cfg 12 | else: 13 | path = "%s/data/logging.json" % \ 14 | os.path.dirname(os.path.abspath(__file__)) 15 | with open(path) as fh: 16 | dict_conf = json.load(fh) 17 | if log_file: 18 | dict_conf['handlers']['file_handler']['filename'] = log_file 19 | logging.config.dictConfig(dict_conf) 20 | -------------------------------------------------------------------------------- /tests/panda/panda/oms_utils.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | import sys 4 | 5 | from netaddr import iter_iprange 6 | 7 | from omsclient.oms_controller import OmsController 8 | from sshutil.remote import RemoteClient 9 | from pyVmomiwrapper import vmwareapi 10 | from exceptions import NotSupportedError, NotCompletedError 11 | import task_utils 12 | 13 | 14 | LOG = logging.getLogger(__name__) 15 | DEFAULT_LOCAL_OVF_TOOL_PATH = '/usr/bin/ovftool' 16 | OMJS_PATH = '/opt/vmware/vio/etc/omjs.properties' 17 | 18 | 19 | def get_ovf_tool_path(): 20 | platform = sys.platform 21 | os.environ['TCROOT'] = "/build/toolchain/" 22 | 23 | if platform.startswith('linux'): 24 | path = 'lin64' 25 | elif platform.startswith('win'): 26 | path = 'win64' 27 | elif platform.startswith('darwin'): 28 | path = 'mac32' 29 | else: 30 | LOG.debug("unsupported platform %s" % platform) 31 | return None 32 | ovf_path = os.environ['OVF_TOOL'] = "%s/%s/ovftool-4.1.0/ovftool" \ 33 | % (os.environ['TCROOT'], path) 34 | if os.path.isfile(ovf_path): 35 | # check if file exists 36 | LOG.debug("ovf tool exists at the following location: %s" % ovf_path) 37 | else: 38 | LOG.debug("couldn't not find ovftool in toolchain %s " % ovf_path) 39 | ovf_path = None 40 | return ovf_path 41 | 42 | 43 | def wait_for_mgmt_service(oms_ip, vc_user, vc_password): 44 | LOG.info('Waiting for management service') 45 | task_utils.wait_for(func=OmsController, timeout=500, delay=10, oms=oms_ip, 46 | sso_user=vc_user, sso_pwd=vc_password) 47 | LOG.info('Management service is running.') 48 | 49 | 50 | def deploy_vapp(vc_host, vc_user, vc_password, dc, cluster, ds, network, 51 | ova_path, ntp_server=None, viouser_pwd='vmware', log_path=None, 52 | ip=None, netmask=None, gateway=None, dns=None, 53 | ovf_tool_path=None): 54 | if not ovf_tool_path: 55 | ovf_tool_path = get_ovf_tool_path() 56 | if not ovf_tool_path: 57 | ovf_tool_path = DEFAULT_LOCAL_OVF_TOOL_PATH 58 | if not os.path.isfile(ovf_tool_path): 59 | LOG.error('ovftool not found.') 60 | raise NotSupportedError('ovftool not found') 61 | if not log_path: 62 | log_path = os.getcwd() 63 | 64 | ntp_config = '--prop:ntpServer=%s ' % ntp_server if ntp_server else '' 65 | dns_config = '--prop:vami.DNS.management-server=%s ' % dns if dns else '' 66 | # deploy ova and poweron vm 67 | # TODO: implement deploy with dhcp 68 | cmd = ('"%s" --X:"logFile"="%s/deploy_oms.log" ' 69 | '--vService:"installation"=' 70 | '"com.vmware.vim.vsm:extension_vservice" ' 71 | '--acceptAllEulas --noSSLVerify --powerOn ' 72 | '--datastore="%s" ' 73 | '-dm=thin ' 74 | '--net:"VIO Management Server Network"="%s" ' 75 | '--prop:vami.ip0.management-server=%s ' 76 | '--prop:vami.netmask0.management-server=%s ' 77 | '--prop:vami.gateway.management-server=%s ' 78 | '%s ' 79 | '--prop:viouser_passwd="%s" ' 80 | '%s "%s" "vi://%s:%s@%s/%s/host/%s"' 81 | '' % (ovf_tool_path, log_path, ds, network, ip, netmask, 82 | gateway, dns_config, viouser_pwd, ntp_config, ova_path, 83 | vc_user, vc_password, vc_host, dc, cluster)) 84 | LOG.info('Start to deploy management server.') 85 | task_utils.safe_run(cmd, 'deploy VIO vApp') 86 | wait_for_mgmt_service(ip, vc_user, vc_password) 87 | LOG.info('Successfully deployed management server.') 88 | 89 | 90 | def check_vapp_exists(vc_host, vc_user, vc_password, 91 | name_regex=r'^VMware-OpenStack.*\d$'): 92 | with vmwareapi.VirtualCenter(vc_host, vc_user, vc_password) as vc: 93 | vapp = vc.get_entity_by_regex(vmwareapi.Vapp, name_regex) 94 | return True if vapp else False 95 | 96 | 97 | def set_omjs_value(ssh_client, key, value, path=OMJS_PATH): 98 | ssh_client.run('sed -i "s|%s.*|%s = %s|g" %s' % 99 | (key, key, value, path), sudo=True, raise_error=True) 100 | 101 | 102 | def config_omjs(ip, vc_user, vc_password, properties, user='viouser', 103 | password='vmware'): 104 | LOG.info('Update omjs.properties: %s', properties) 105 | ssh_client = RemoteClient(ip, user, password) 106 | for key in properties: 107 | set_omjs_value(ssh_client, key, properties[key]) 108 | ssh_client.run('restart oms', sudo=True, raise_error=True) 109 | wait_for_mgmt_service(ip, vc_user, vc_password) 110 | 111 | 112 | def config_omjs_for_release_build(ip, vc_user, vc_password): 113 | params = {'oms.use_linked_clone': 'true', 114 | 'oms.skip_cluster_vmotion_check': 'true', 115 | 'oms.disable_datastores_anti_affinity': 'true', 116 | 'oms.disable_hosts_anti_affinity': 'true'} 117 | config_omjs(ip, vc_user, vc_password, params) 118 | 119 | 120 | def remove_vapp(vc_host, vc_user, vc_password, name_regex): 121 | with vmwareapi.VirtualCenter(vc_host, vc_user, vc_password) as vc: 122 | vapp = vc.get_entity_by_regex(vmwareapi.Vapp, name_regex) 123 | if vapp: 124 | LOG.info("Start to remove %s" % vapp.name) 125 | state = vapp.get_state() 126 | if state == 'started': 127 | vapp.poweroff() 128 | vapp.destroy() 129 | else: 130 | LOG.info("%s not found" % name_regex) 131 | 132 | 133 | def get_vapp_version(vc_host, vc_user, vc_password, name_regex): 134 | with vmwareapi.VirtualCenter(vc_host, vc_user, vc_password) as vc: 135 | vapp = vc.get_entity_by_regex(vmwareapi.Vapp, name_regex) 136 | return vapp.version if vapp else None 137 | 138 | 139 | def get_patch_info(ssh_client, patch_version): 140 | output = ssh_client.run('viopatch list', sudo=True, raise_error=True) 141 | lines = output.split('\n') 142 | if len(lines) > 2: 143 | lines = lines[2:] 144 | for line in lines: 145 | if line.strip(): 146 | items = line.split() 147 | if patch_version == items[1]: 148 | LOG.debug('Find patch info: %s' % line) 149 | return {'Name': items[0], 150 | 'Version': items[1], 151 | 'Type': items[2], 152 | 'Installed': items[-1]} 153 | 154 | 155 | def apply_patch(ip, file_path, user='viouser', password='vmware'): 156 | file_name = os.path.basename(file_path) 157 | patch_name, patch_version = file_name.split('_')[0:2] 158 | # Check if patch has already been installed 159 | ssh_client = RemoteClient(ip, user, password) 160 | patch_info = get_patch_info(ssh_client, patch_version) 161 | if patch_info and patch_info['Installed'] == 'Yes': 162 | LOG.info('Patch %s has already been installed, Skip installing it.', 163 | file_name) 164 | return patch_info 165 | if not patch_info: 166 | LOG.info('Adding patch %s' % file_name) 167 | remote_path = os.path.join('/tmp', file_name) 168 | ssh_client.scp(file_name, '/tmp') 169 | ssh_client.run('viopatch add -l %s' % remote_path, sudo=True, 170 | raise_error=True) 171 | # Clean up patch file in case oms disk becomes full 172 | ssh_client.run('rm -f %s' % remote_path) 173 | patch_info = get_patch_info(ssh_client, patch_version) 174 | if not patch_info: 175 | raise NotSupportedError('Failed to add Patch %s' % file_name) 176 | LOG.info('Start to install patch %s' % file_name) 177 | ssh_client.run('viopatch install --patch %s --version %s --as-infra' % 178 | (patch_name, patch_version), sudo=True, raise_error=True, 179 | feed_input='Y') 180 | patch_info = get_patch_info(ssh_client, patch_version) 181 | if patch_info['Installed'] != 'Yes': 182 | LOG.error('Failed to install patch %s' % patch_info) 183 | raise NotCompletedError('Failed to install patch %s.' % file_name) 184 | LOG.info('Successfully Installed patch %s' % file_name) 185 | return patch_info 186 | 187 | 188 | def get_ip_range(oms_spec, key): 189 | if key in oms_spec: 190 | start, end = oms_spec[key].split('-') 191 | return iter_iprange(start, end) 192 | -------------------------------------------------------------------------------- /tests/panda/panda/os_utils.py: -------------------------------------------------------------------------------- 1 | import logging 2 | 3 | from keystoneauth1.identity import v3 4 | from keystoneauth1 import session 5 | from keystoneclient.v3 import client as keystone_client 6 | 7 | 8 | LOG = logging.getLogger(__name__) 9 | LOCAL_DOMAIN_ID = 'local' 10 | DEFAULT_DOMAIN_ID = 'default' 11 | ADMIN_PROJECT_NAME = 'admin' 12 | ADMIN_ROLE_NAME = 'admin' 13 | 14 | 15 | def create_if_not_exist(func, kind, name, **kwargs): 16 | entity = get_entity(func, kind, name) 17 | if entity: 18 | return entity 19 | LOG.info("Create %s %s" % (kind, name)) 20 | return func.create(name, **kwargs) 21 | 22 | 23 | def get_entity(func, kind, name, **kwargs): 24 | for entity in func.list(**kwargs): 25 | if entity.name.lower() == name.lower(): 26 | LOG.info("Found %s %s" % (kind, name)) 27 | return entity 28 | return None 29 | 30 | 31 | def grant_role_on_project(keystone, project, user, role): 32 | for existed_role in keystone.roles.list(user=user, project=project): 33 | if existed_role.name == role.name: 34 | LOG.info('Role %s is already granted to user %s on project %s' % 35 | (role.name, user.name, project.name)) 36 | return 37 | LOG.info('Grant role %s to user %s on project %s', role.name, user.name, 38 | project.name) 39 | keystone.roles.grant(role, user=user, project=project) 40 | 41 | 42 | def grant_role_on_domain(keystone, domain, user, role): 43 | for existed_role in keystone.roles.list(user=user, domain=domain): 44 | if existed_role.name == role.name: 45 | LOG.info('Role %s is already granted to user %s on domain %s' % 46 | (role.name, user.name, domain.name)) 47 | return 48 | LOG.info('Grant role %s to user %s on domain %s', role.name, user.name, 49 | domain.name) 50 | keystone.roles.grant(role, user=user, domain=domain) 51 | 52 | 53 | def enable_ldap_admin(private_vip, local_user_name, local_user_pwd, 54 | ldap_user_name): 55 | keystone = get_keystone_client(private_vip=private_vip, 56 | username=local_user_name, 57 | password=local_user_pwd, 58 | project_name=ADMIN_PROJECT_NAME, 59 | domain_name=LOCAL_DOMAIN_ID) 60 | ldap_domain = keystone.domains.get(DEFAULT_DOMAIN_ID) 61 | admin_role = get_entity(keystone.roles, 'role', ADMIN_ROLE_NAME) 62 | ldap_user = get_entity(keystone.users, 'user', ldap_user_name, 63 | domain=DEFAULT_DOMAIN_ID) 64 | grant_role_on_domain(keystone, 65 | domain=ldap_domain, 66 | user=ldap_user, 67 | role=admin_role) 68 | ldap_project = get_entity(keystone.projects, 'project', 69 | ADMIN_PROJECT_NAME, domain=DEFAULT_DOMAIN_ID) 70 | if not ldap_project: 71 | LOG.info('Create project %s', ADMIN_PROJECT_NAME) 72 | keystone.projects.create(name=ADMIN_PROJECT_NAME, 73 | domain=DEFAULT_DOMAIN_ID) 74 | grant_role_on_project(keystone, 75 | project=ldap_project, 76 | user=ldap_user, 77 | role=admin_role) 78 | 79 | 80 | def get_keystone_client(private_vip, username, password, project_name, 81 | domain_name): 82 | auth_url = get_auth_url(private_vip, 'v3', port='35357') 83 | auth = v3.Password(auth_url=auth_url, 84 | username=username, 85 | password=password, 86 | project_name=project_name, 87 | user_domain_name=domain_name, 88 | project_domain_name=domain_name) 89 | sess = session.Session(auth=auth) 90 | return keystone_client.Client(session=sess) 91 | 92 | 93 | def get_auth_url(private_vip, version='v2.0', port='5000'): 94 | return 'http://%s:%s/%s/' % (private_vip, port, version) 95 | -------------------------------------------------------------------------------- /tests/panda/panda/setup.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import json 3 | import os 4 | 5 | import oms_utils 6 | import cluster_utils 7 | import build_utils 8 | from omsclient.oms_controller import OmsController 9 | from sshutil.remote import RemoteClient 10 | from os_utils import enable_ldap_admin 11 | from shellutil import shell 12 | 13 | 14 | LOG = logging.getLogger(__name__) 15 | 16 | 17 | class Setup(object): 18 | def __init__(self, **kwargs): 19 | pass 20 | 21 | 22 | class Openstack(Setup): 23 | def __init__(self, vc_host, vc_user=None, vc_pwd=None, datacenter=None, 24 | cluster=None, datastore=None, build_id=None, version=None): 25 | self.vc_host = vc_host 26 | self.vc_user = vc_user 27 | self.vc_pwd = vc_pwd 28 | self.datacenter = datacenter 29 | self.cluster = cluster 30 | self.datastore = datastore 31 | self.build_id = build_id 32 | self.version = version 33 | 34 | def deploy_openstack(self, config_spec): 35 | raise NotImplementedError 36 | 37 | 38 | class VIO(Openstack): 39 | def __init__(self, oms_spec, cluster_spec, log_dir): 40 | super(VIO, self).__init__(oms_spec['vc_host'], 41 | vc_user=oms_spec['vc_user'], 42 | vc_pwd=oms_spec['vc_password'], 43 | datacenter=oms_spec['datacenter'], 44 | cluster=oms_spec['cluster'], 45 | datastore=oms_spec['datastore'], 46 | build_id=oms_spec['build'], 47 | version=oms_spec.get('version', None)) 48 | self.oms_ip = oms_spec['host_ip'] 49 | self.oms_netmask = oms_spec['netmask'] 50 | self.oms_gateway = oms_spec['gateway'] 51 | self.oms_dns = oms_spec.get('dns', None) 52 | self.oms_ntp = oms_spec.get('ntp_server', None) 53 | self.oms_ctl = None 54 | self.oms_network = oms_spec['network'] 55 | self.oms_user = oms_spec['username'] 56 | self.oms_pwd = oms_spec['password'] 57 | self.log_dir = log_dir if os.path.isabs(log_dir) else \ 58 | os.path.abspath(log_dir) 59 | ova_path = oms_spec.get('ova_path', '').strip() 60 | if ova_path: 61 | self.remove_ova = False 62 | self.ova_path = ova_path 63 | else: 64 | self.remove_ova = True 65 | self.ova_path = build_utils.download_ova(self.build_id) 66 | self.vapp_name = os.path.basename(self.ova_path).replace('.ova', '') 67 | self.cluster_spec = cluster_spec 68 | self.omjs_properties = oms_spec.get('omjs_properties', {}) 69 | self.upgrade_index = 1 70 | self.cluster_name = cluster_spec['name'] if cluster_spec else 'VIO' 71 | if 'compute_vc_host' in oms_spec: 72 | self.compute_vc_host = oms_spec['compute_vc_host'] 73 | self.compute_vc_user = oms_spec['compute_vc_user'] 74 | self.compute_vc_pwd = oms_spec['compute_vc_password'] 75 | self.compute_datacenter = oms_spec['compute_datacenter'] 76 | else: 77 | self.compute_vc_host = self.vc_host 78 | self.compute_vc_user = self.vc_user 79 | self.compute_vc_pwd = self.vc_pwd 80 | self.compute_datacenter = self.datacenter 81 | 82 | def deploy_vapp(self): 83 | if not oms_utils.check_vapp_exists(self.vc_host, self.vc_user, 84 | self.vc_pwd, self.vapp_name): 85 | oms_utils.deploy_vapp(vc_host=self.vc_host, 86 | vc_user=self.vc_user, 87 | vc_password=self.vc_pwd, 88 | dc=self.datacenter, 89 | cluster=self.cluster, 90 | ds=self.datastore, 91 | network=self.oms_network, 92 | ova_path=self.ova_path, 93 | ntp_server=self.oms_ntp, 94 | viouser_pwd=self.oms_pwd, 95 | log_path=self.log_dir, 96 | ip=self.oms_ip, 97 | netmask=self.oms_netmask, 98 | gateway=self.oms_gateway, 99 | dns=self.oms_dns) 100 | else: 101 | LOG.info('VIO vApp already exists. Skip deploying vApp.') 102 | # Remove downloaded ova 103 | if self.remove_ova: 104 | shell.local('rm -f %s' % self.ova_path) 105 | self.oms_ctl = OmsController(self.oms_ip, self.vc_user, self.vc_pwd) 106 | 107 | def upgrade(self, public_vip, private_vip=None): 108 | blue_name = self.cluster_name 109 | self.cluster_name = 'UPGRADE%s' % self.upgrade_index 110 | if self.is_deployed(self.cluster_name): 111 | LOG.info('Cluster %s exists, skip upgrading.', self.cluster_name) 112 | else: 113 | oms_utils.wait_for_mgmt_service(self.oms_ip, self.vc_host, 114 | self.vc_pwd) 115 | # Write back the same omjs properties since b2b patch overwrite 116 | # them. 117 | self.config_omjs(self.omjs_properties) 118 | if int(self.get_version()[0]) >= 3: 119 | attributes = cluster_utils.get_controller_attrs( 120 | self.cluster_spec) 121 | admin_user = attributes['admin_user'] 122 | admin_password = attributes['admin_password'] 123 | spec = {'clusterName': self.cluster_name, 124 | 'public_vip': public_vip, 125 | 'admin_user': admin_user, 126 | 'admin_password': admin_password} 127 | else: 128 | spec = {'clusterName': self.cluster_name, 129 | 'publicVIP': public_vip, 130 | 'internalVIP': private_vip} 131 | try: 132 | cluster_utils.upgrade(self.oms_ctl, blue_name, 133 | self.cluster_name, spec) 134 | except Exception: 135 | self.get_support_bundle() 136 | raise 137 | self.upgrade_index += 1 138 | return cluster_utils.get_cluster(self.oms_ctl, self.cluster_name) 139 | 140 | def get_version(self): 141 | if not self.version: 142 | self.version = oms_utils.get_vapp_version(self.vc_host, 143 | self.vc_user, 144 | self.vc_pwd, 145 | self.vapp_name)[0:5] 146 | return self.version 147 | 148 | def is_deployed(self, cluster_name): 149 | try: 150 | return cluster_utils.check_cluster_status(self.oms_ctl, 151 | cluster_name, 152 | ['RUNNING', 'STOPPED']) 153 | except Exception, error: 154 | LOG.debug('Failed to retrieve VIO cluster status: %s' % error) 155 | return False 156 | 157 | def config_omjs(self, properties): 158 | oms_utils.config_omjs(ip=self.oms_ip, 159 | vc_user=self.vc_user, 160 | vc_password=self.vc_pwd, 161 | properties=properties, 162 | user=self.oms_user, 163 | password=self.oms_pwd) 164 | self.oms_ctl = OmsController(self.oms_ip, self.vc_user, self.vc_pwd) 165 | 166 | def deploy_openstack(self): 167 | if not self.is_deployed(self.cluster_name): 168 | # TODO(xiaoy): Remove provision failed cluster 169 | # Add compute VC if multiple VC 170 | attrs = cluster_utils.get_controller_attrs(self.cluster_spec) 171 | ssh_client = RemoteClient(self.oms_ip, self.oms_user, self.oms_pwd) 172 | if attrs['vcenter_ip'] != self.vc_host: 173 | LOG.debug('Managment VC: %s, Compute VC: %s. This is multi-vc', 174 | self.vc_host, attrs['vcenter_ip']) 175 | cluster_utils.add_compute_vc(self.oms_ctl, 176 | ssh_client, 177 | attrs.get('vcenter_insecure', ''), 178 | attrs['vcenter_ip'], 179 | attrs['vcenter_user'], 180 | attrs['vcenter_password']) 181 | try: 182 | cluster_utils.set_vc_fqdn(self.cluster_spec, ssh_client) 183 | # Create plan when it is empty 184 | if not self.cluster_spec['attributes']['plan']: 185 | self.cluster_spec = cluster_utils.create_deployment_plan( 186 | self.oms_ctl, self.cluster_spec) 187 | cluster_utils.create_openstack_cluster(self.oms_ctl, 188 | self.cluster_spec) 189 | if attrs['keystone_backend'] == cluster_utils.LDAP_BACKEND \ 190 | and int(self.get_version()[0]) >= 3: 191 | private_vip = cluster_utils.get_private_vip( 192 | self.oms_ctl, self.cluster_name) 193 | enable_ldap_admin(private_vip=private_vip, 194 | local_user_name=attrs['admin_user'], 195 | local_user_pwd=attrs['admin_password'], 196 | ldap_user_name=attrs['ldap_user']) 197 | except Exception: 198 | self.get_support_bundle() 199 | raise 200 | else: 201 | LOG.info('Cluster %s already exists. Skip creating it.', 202 | self.cluster_name) 203 | LOG.debug('Current VIO Version: %s', self.version) 204 | 205 | def get_support_bundle(self): 206 | spec = {"deployment_name": self.cluster_name} 207 | json_str = json.dumps(spec) 208 | try: 209 | # Rest client session always time out after tests 210 | self.oms_ctl.login() 211 | file_name = self.oms_ctl.get_support_bundle(json_str, self.log_dir) 212 | LOG.info('Downloaded support bundle to %s/%s.' % 213 | (self.log_dir, file_name)) 214 | except Exception as error: 215 | LOG.exception('Failed to get support bundle: %s' % error) 216 | 217 | def apply_patch(self, patch_file): 218 | try: 219 | patch_info = oms_utils.apply_patch(self.oms_ip, patch_file, 220 | self.oms_user, self.oms_pwd) 221 | self.version = patch_info['Version'] 222 | except Exception: 223 | self.get_support_bundle() 224 | raise 225 | LOG.debug('Current VIO Version: %s', self.version) 226 | 227 | def add_compute_cluster(self, name): 228 | LOG.info('Add compute cluster: %s', name) 229 | cluster_moid = cluster_utils.get_cluster_moid(self.compute_vc_host, 230 | self.compute_vc_user, 231 | self.compute_vc_pwd, 232 | self.compute_datacenter, 233 | name) 234 | cluster_spec = cluster_utils.get_cluster(self.oms_ctl, 235 | self.cluster_name) 236 | moids = cluster_utils.get_compute_cluster_moids(cluster_spec) 237 | if cluster_moid in moids: 238 | LOG.info('Cluster %s is already a compute cluster, skip adding ' 239 | 'it.', name) 240 | return 241 | spec = [{ 242 | "cluster_name": name, 243 | "datastore_regex": self.datastore, 244 | "cluster_moid": cluster_moid 245 | }] 246 | try: 247 | self.oms_ctl.add_nova_node(self.cluster_name, "ComputeDriver", 248 | json.dumps(spec)) 249 | except Exception: 250 | self.get_support_bundle() 251 | raise 252 | LOG.info('Cluster %s is added as a compute cluster.', name) 253 | 254 | 255 | class Devstack(Openstack): 256 | def __init__(self): 257 | raise NotImplementedError 258 | 259 | def deploy_openstack(self, config_spec): 260 | raise NotImplementedError 261 | -------------------------------------------------------------------------------- /tests/panda/panda/task_utils.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import logging 3 | import time 4 | 5 | from exceptions import TimeoutError 6 | from shellutil import shell 7 | 8 | 9 | LOG = logging.getLogger(__name__) 10 | 11 | 12 | def wait_for(func, timeout, delay, *args, **kargs): 13 | """Decorator for waiting for until a function finished running.""" 14 | 15 | poll_timeout = timeout 16 | poll_sleep_retry = delay 17 | 18 | begin_poll = datetime.datetime.now() 19 | while True: 20 | try: 21 | return func(*args, **kargs) 22 | break 23 | except Exception as e: 24 | if (datetime.datetime.now() - begin_poll).seconds > poll_timeout: 25 | LOG.exception('Time out after %s seconds.' % poll_timeout) 26 | raise TimeoutError('Timed out after %s seconds. Reason: ' 27 | '%s' % (poll_timeout, e)) 28 | else: 29 | LOG.debug('Sleeping %s seconds before retrying' 30 | '' % poll_sleep_retry) 31 | time.sleep(poll_sleep_retry) 32 | 33 | 34 | def safe_run(cmd, msg, sleep_time=180): 35 | exit_code = shell.local(cmd)[0] 36 | if exit_code: 37 | LOG.warning('Failed to %s. Retry it after %s seconds' % 38 | (msg, sleep_time)) 39 | time.sleep(sleep_time) 40 | shell.local(cmd, raise_error=True) 41 | -------------------------------------------------------------------------------- /tests/panda/panda/test.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | import json 4 | 5 | import cluster_utils 6 | from panda.exceptions import NotSupportedError 7 | import tempest_utils 8 | from cluster_utils import NSXT_BACKEND 9 | from cluster_utils import NSXV_BACKEND 10 | from cluster_utils import LDAP_BACKEND 11 | from tempest_utils import LEGACY_PROVIDER 12 | from tempest_utils import PRE_PROVISIONED_PROVIDER 13 | from shellutil import shell 14 | from omsclient.oms_controller import OmsController 15 | 16 | 17 | LOG = logging.getLogger(__name__) 18 | PASS = 0 19 | FAIL = 1 20 | VIO_LOGS_DIR = 'vio' 21 | V1_DVS_BRANCH = 'dvs-1.0.0' 22 | V2_DVS_BRANCH = 'dvs-2.0' 23 | 24 | 25 | class Test(object): 26 | def __init__(self, test_name, log_dir, oms_spec, cluster_spec): 27 | self.test_name = test_name 28 | self.oms_spec = oms_spec 29 | self.cluster_spec = cluster_spec 30 | self.log_dir = log_dir if os.path.isabs(log_dir) else \ 31 | os.path.abspath(log_dir) 32 | 33 | @staticmethod 34 | def run_tests(tests, log_dir, oms_spec, cluster_spec): 35 | results = PASS 36 | for test in tests.split(','): 37 | test = test.strip() 38 | if test in CLS_MAP.keys(): 39 | cls = CLS_MAP.get(test) 40 | result = cls.run_test(test, log_dir, oms_spec, cluster_spec) 41 | if result != PASS: 42 | results = FAIL 43 | else: 44 | raise Exception("Test %s not supported!" % test) 45 | return results 46 | 47 | @classmethod 48 | def run_test(cls, test, log_dir, oms_spec, cluster_spec): 49 | instance = cls(test, log_dir, oms_spec, cluster_spec) 50 | instance.set_up() 51 | instance.run() 52 | instance.clean_up() 53 | return instance.check_results() 54 | 55 | 56 | class OMSAPI(Test): 57 | def __init__(self, test_name, log_dir, oms_spec, cluster_spec): 58 | super(OMSAPI, self).__init__(test_name, log_dir, oms_spec, 59 | cluster_spec) 60 | self.project_path = None 61 | 62 | def set_up(self): 63 | config = { 64 | "vc_ip": self.oms_spec['vc_host'], 65 | "vc_user": self.oms_spec['vc_user'], 66 | "vc_password": self.oms_spec['vc_password'], 67 | "vapp_name": '', 68 | "oms_ip": self.oms_spec['host_ip'], 69 | "oms_gateway": self.oms_spec['gateway'], 70 | "oms_netmask": self.oms_spec['netmask'], 71 | "oms_dns": self.oms_spec['dns'], 72 | "oms_network": '', 73 | "oms_dc": '', 74 | "oms_cluster": '', 75 | "oms_datastore": '', 76 | "nsxv_manager": self.oms_spec['nsxv_ip'], 77 | "nsxv_username": self.oms_spec['nsxv_user'], 78 | "nsxv_password": self.oms_spec['nsxv_password'] 79 | } 80 | branch = self.oms_spec['version'][0:3] 81 | project_path = os.path.join(os.getcwd(), 'vio-api-test') 82 | if not os.path.exists(project_path): 83 | shell.local('git clone -b %s http://p3-review.eng.vmware.com/' 84 | 'vio-api-test' % branch) 85 | LOG.info('Install VIO OMS api test project in %s', project_path) 86 | shell.local('sudo pip install -r vio-api-test/requirements.txt') 87 | shell.local('sudo pip install -e vio-api-test/') 88 | config_path = "%s/vio/data/vio.vapp.json" % project_path 89 | LOG.info("Generate VIO OMS API test configuration to %s" % config_path) 90 | LOG.debug("vio.vapp.json: %s" % config) 91 | with open(config_path, 'w+') as fh: 92 | json.dump(config, fh, indent=2, separators=(',', ': ')) 93 | self.project_path = project_path 94 | 95 | def _run(self, neutron_backend): 96 | report = '%s-nosetests.xml' % neutron_backend 97 | cmd = 'cd %s; python run_test.py -t %s --deployment_type %s ' \ 98 | '--report %s' % (self.project_path, neutron_backend, 99 | self.oms_spec['api_test_deployment_type'], 100 | os.path.join(self.log_dir, report)) 101 | LOG.info('[local] run: %s' % cmd) 102 | os.system(cmd) 103 | 104 | def run(self): 105 | LOG.info("oms-api tests begin.") 106 | self._run('dvs') 107 | self._run('nsxv') 108 | LOG.info("oms-api tests end.") 109 | # return result 110 | 111 | def check_results(self): 112 | # TODO (xiaoy): check if test fails in nosetests.xml 113 | return PASS 114 | 115 | def clean_up(self): 116 | pass 117 | 118 | 119 | class Tempest(Test): 120 | def set_up(self): 121 | if not os.path.exists('tempest/included-tests.txt'): 122 | controller = cluster_utils.get_nodegroup_by_role(self.cluster_spec, 123 | 'Controller') 124 | admin_user = controller['attributes']['admin_user'] 125 | admin_pwd = controller['attributes']['admin_password'] 126 | neutron_backend = controller['attributes']['neutron_backend'] 127 | keystone_backend = controller['attributes']['keystone_backend'] 128 | admin_tenant = controller['attributes']['admin_tenant_name'] 129 | ext_net_cidr = None 130 | ext_net_start_ip = None 131 | ext_net_end_ip = None 132 | ext_net_gateway = None 133 | nsx_manager = None 134 | nsx_user = None 135 | nsx_pwd = None 136 | if LDAP_BACKEND == keystone_backend: 137 | admin_user = self.oms_spec['openstack_admin'] 138 | admin_pwd = self.oms_spec['openstack_admin_pwd'] 139 | if neutron_backend == NSXV_BACKEND: 140 | nsx_manager = controller['attributes']['nsxv_manager'] 141 | nsx_user = controller['attributes']['nsxv_username'] 142 | nsx_pwd = controller['attributes']['nsxv_password'] 143 | elif neutron_backend == NSXT_BACKEND: 144 | # TODO: get nsxt configure from spec 145 | pass 146 | if neutron_backend in [NSXV_BACKEND, NSXT_BACKEND]: 147 | ext_net_cidr = self.oms_spec['ext_net_cidr'] 148 | ext_net_start_ip = self.oms_spec['ext_net_start_ip'] 149 | ext_net_end_ip = self.oms_spec['ext_net_end_ip'] 150 | ext_net_gateway = self.oms_spec['ext_net_gateway'] 151 | creds_provider = self.oms_spec['openstack_creds_provider'].strip() 152 | if creds_provider in [LEGACY_PROVIDER, PRE_PROVISIONED_PROVIDER]: 153 | user1 = self.oms_spec['openstack_user1'] 154 | user1_pwd = self.oms_spec['openstack_user1_pwd'] 155 | user2 = self.oms_spec['openstack_user2'] 156 | user2_pwd = self.oms_spec['openstack_user2_pwd'] 157 | else: 158 | user1 = None 159 | user1_pwd = None 160 | user2 = None 161 | user2_pwd = None 162 | if 'compute_clusters' in self.oms_spec: 163 | min_compute_nodes = len(self.oms_spec['compute_clusters']) 164 | else: 165 | min_compute_nodes = 1 166 | tempest_utils.install_tempest() 167 | tempest_log_file = '%s/tempest.log' % self.log_dir 168 | oms_ctl = OmsController(self.oms_spec['host_ip'], 169 | self.oms_spec['vc_user'], 170 | self.oms_spec['vc_password']) 171 | private_vip = cluster_utils.get_private_vip( 172 | oms_ctl, self.cluster_spec['name']) 173 | if not private_vip: 174 | raise NotSupportedError('Can not get private VIP.') 175 | tempest_utils.config_tempest(private_vip=private_vip, 176 | admin_user=admin_user, 177 | admin_pwd=admin_pwd, 178 | neutron_backend=neutron_backend, 179 | creds_provider=creds_provider, 180 | default_user=user1, 181 | default_pwd=user1_pwd, 182 | alter_user=user2, 183 | alter_pwd=user2_pwd, 184 | ext_net_cidr=ext_net_cidr, 185 | ext_net_start_ip=ext_net_start_ip, 186 | ext_net_end_ip=ext_net_end_ip, 187 | ext_net_gateway=ext_net_gateway, 188 | tempest_log_file=tempest_log_file, 189 | admin_tenant=admin_tenant, 190 | min_compute_nodes=min_compute_nodes, 191 | nsx_manager=nsx_manager, 192 | nsx_user=nsx_user, 193 | nsx_pwd=nsx_pwd) 194 | shell.local('cp %s/etc/tempest.conf %s/' % ( 195 | tempest_utils.TEMPEST_DIR, self.log_dir)) 196 | tempest_utils.generate_run_list(neutron_backend) 197 | else: 198 | LOG.info('Tempest already exists. Skip setting up it.') 199 | 200 | def run(self): 201 | tempest_utils.run_test(self.test_name, self.log_dir) 202 | 203 | def check_results(self): 204 | # TODO (xiaoy): pass, rerun pass or fail 205 | return PASS 206 | 207 | def clean_up(self): 208 | pass 209 | 210 | 211 | class VMwareTempest(Tempest): 212 | def set_up(self): 213 | if not os.path.exists('vmware_tempest/run-tests.txt'): 214 | super(VMwareTempest, self).set_up() 215 | tempest_utils.install_vmware_tempest() 216 | if 'compute_vc_host' in self.oms_spec: 217 | # Use compute VC in multi-VC setup. 218 | vc_host = self.oms_spec['compute_vc_host'] 219 | vc_user = self.oms_spec['compute_vc_user'] 220 | vc_password = self.oms_spec['compute_vc_password'] 221 | else: 222 | vc_host = self.oms_spec['vc_host'] 223 | vc_user = self.oms_spec['vc_user'] 224 | vc_password = self.oms_spec['vc_password'] 225 | tempest_utils.config_vmware_tempest(vc_host=vc_host, 226 | vc_user=vc_user, 227 | vc_password=vc_password) 228 | tempest_utils.generate_vmware_run_list() 229 | else: 230 | LOG.info('VMware tempest already exists. Skip setting up it.') 231 | 232 | def run(self): 233 | tempest_utils.run_vmware_test(self.log_dir) 234 | 235 | def check_results(self): 236 | return PASS 237 | 238 | def clean_up(self): 239 | pass 240 | 241 | CLS_MAP = {'oms-api': OMSAPI, 242 | 'vmware': VMwareTempest, 243 | 'nova': Tempest, 244 | 'cinder': Tempest, 245 | 'neutron': Tempest, 246 | 'heat': Tempest, 247 | 'keystone': Tempest, 248 | 'glance': Tempest, 249 | 'scenario': Tempest, 250 | 'nsxv': Tempest, 251 | 'nsxt': Tempest} 252 | -------------------------------------------------------------------------------- /tests/panda/requirements.txt: -------------------------------------------------------------------------------- 1 | buildwebapi>=0.0.2 2 | ssh-util 3 | shell-util 4 | oms-client 5 | pyVmomiwrapper>=0.0.2 6 | python-keystoneclient>=2.0.0 7 | python-neutronclient 8 | python-novaclient 9 | python-subunit 10 | junitxml 11 | testtools 12 | m2crypto 13 | os-testr -------------------------------------------------------------------------------- /tests/panda/setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from setuptools import setup 4 | 5 | 6 | def package_files(directory): 7 | paths = [] 8 | for (path, directories, filenames) in os.walk(directory): 9 | for filename in filenames: 10 | paths.append('/'.join(os.path.join(path, filename).split('/')[1:])) 11 | return paths 12 | 13 | 14 | extra_files = package_files('panda/data') 15 | 16 | setup( 17 | name='panda', 18 | version='0.0.1', 19 | scripts=['bin/panda'], 20 | packages=['panda'], 21 | package_data={ 22 | 'panda': extra_files 23 | }, 24 | install_requires=[ 25 | 'buildwebapi', 26 | 'ssh-util', 27 | 'shell-util', 28 | 'oms-client', 29 | 'pyVmomiwrapper', 30 | 'python-keystoneclient', 31 | 'python-neutronclient', 32 | 'python-novaclient', 33 | 'python-subunit', 34 | 'junitxml', 35 | 'testtools' 36 | ] 37 | ) 38 | -------------------------------------------------------------------------------- /tests/pyVmomiwrapper/README.rst: -------------------------------------------------------------------------------- 1 | pyVmomiwrapper 2 | ============= 3 | 4 | This module wraps pyVmomi. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com pyVmomiwrapper 13 | 14 | 15 | Usage 16 | ====== 17 | .. code:: python 18 | 19 | import ssl 20 | 21 | from pyVmomi import vim 22 | 23 | from vmwareapi import VirtualCenter 24 | from vmwareapi import VM 25 | from vmwareapi import Host 26 | from vmwareapi import DataStore 27 | from vmwareapi import Network 28 | from vmwareapi import Cluster 29 | from vmwareapi import Vapp 30 | from vmwareapi import DistributedVirtualSwitch 31 | from vmwareapi import DistributedVirtualPortgroup 32 | 33 | 34 | with VirtualCenter('192.168.111.1', 'root', 'vmware') as vc: 35 | # Create data center 36 | dc = vc.create_datacenter('dc-01') 37 | print "dc: %s(%s)" % (dc.name, dc.moid) 38 | # Get data center 39 | dc = vc.get_datacenter('openstack-dc-01') 40 | print "dc: %s(%s)" % (dc.name, dc.moid) 41 | # Get managed object attributes 42 | print "dc.datastoreFolder: %s" % dc.datastoreFolder 43 | # Search entities under vc by name 44 | vm = vc.get_entities_by_name(VM, 'VIO-DB-1')[0] 45 | print "vm: %s(%s)" % (vm.name, vm.moid) 46 | host = vc.get_entities_by_name(Host, 'sin2-openstack-006.eng.vmware.com')[0] 47 | print "host: %s(%s)" % (host.name, host.moid) 48 | ds = vc.get_entities_by_name(DataStore, 'vsanDatastore')[0] 49 | print "ds: %s(%s)" % (ds.name, ds.moid) 50 | net = vc.get_entities_by_name(Network, 'VM Network')[0] 51 | print "net: %s(%s)" % (net.name, net.moid) 52 | cluster = vc.get_entities_by_name(Cluster, 'nova-cluster')[0] 53 | print "cluster: %s(%s)" % (cluster.name, cluster.moid) 54 | # Search entities under vc by regex 55 | vapp = vc.get_entities_by_regex(Vapp, r'^VMware-OpenStack.*\d$')[0] 56 | print "vapp: %s(%s)" % (vapp.name, vapp.moid) 57 | # Create dvs 58 | dc = vc.get_datacenter('openstack-dc-01') 59 | dvs_spec = vim.DistributedVirtualSwitch.CreateSpec() 60 | dvs_config = vim.dvs.VmwareDistributedVirtualSwitch.ConfigSpec() 61 | dvs_config.name = 'test-dvs' 62 | dvs_config.host = [] 63 | dvs_spec.configSpec = dvs_config 64 | dvs = dc.create_dvs(dvs_spec) 65 | print "dvs: %s(%s)" % (dvs.name, dvs.moid) 66 | # Search entities under data center by name 67 | vm = dc.get_entities_by_name(VM, 'VIO-DB-1')[0] 68 | print "vm: %s(%s)" % (vm.name, vm.moid) 69 | host = dc.get_entities_by_name(Host, 'sin2-openstack-006.eng.vmware.com')[0] 70 | print "host: %s(%s)" % (host.name, host.moid) 71 | ds = dc.get_entities_by_name(DataStore, 'vsanDatastore')[0] 72 | print "ds: %s(%s)" % (ds.name, ds.moid) 73 | net = dc.get_entities_by_name(Network, 'VM Network')[0] 74 | print "net: %s(%s)" % (net.name, net.moid) 75 | cluster = dc.get_entities_by_name(Cluster, 'nova-cluster')[0] 76 | print "cluster: %s(%s)" % (cluster.name, cluster.moid) 77 | dvs = dc.get_entities_by_name(DistributedVirtualSwitch, 'VIO-Data')[0] 78 | print "dvs: %s(%s)" % (dvs.name, dvs.moid) 79 | # Search entities under data center by regex 80 | vapp = dc.get_entities_by_regex(Vapp, r'^VMware-OpenStack.*\d$')[0] 81 | print "vapp: %s(%s)" % (vapp.name, vapp.moid) 82 | # Add a nic 83 | dvpg = dc.get_entities_by_name(DistributedVirtualPortgroup, 'dvp-vio-external')[0] 84 | vm.add_nic(dvpg) 85 | # Retrieve and update vpx settings 86 | option_value = vc.query_vpx_settings('vpxd.httpClientIdleTimeout') 87 | print option_value[0].key, option_value[0].value 88 | option_value = vc.query_vpx_settings('config.vmacore.http.readTimeoutMs') 89 | print option_value[0].key, option_value[0].value 90 | 91 | option_values = {'config.vmacore.http.readTimeoutMs': '600000', 92 | 'vpxd.httpClientIdleTimeout': 900} 93 | vc.update_vpx_settings(option_values) 94 | print '------update settings------' 95 | option_value = vc.query_vpx_settings('vpxd.httpClientIdleTimeout') 96 | print option_value[0].key, option_value[0].value 97 | option_value = vc.query_vpx_settings('config.vmacore.http.readTimeoutMs') 98 | print option_value[0].key, option_value[0].value 99 | 100 | 101 | 102 | -------------------------------------------------------------------------------- /tests/pyVmomiwrapper/pyVmomiwrapper/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/pyVmomiwrapper/pyVmomiwrapper/__init__.py -------------------------------------------------------------------------------- /tests/pyVmomiwrapper/pyVmomiwrapper/task.py: -------------------------------------------------------------------------------- 1 | # @file task.py 2 | # @brief Task functions 3 | # 4 | # This module provies synchronization of client/server operations 5 | # since many vIM operations return 'tasks' which can have 6 | # varying completion times. 7 | 8 | """ 9 | Task functions 10 | 11 | This module provies synchronization of client/server operations since 12 | many vIM operations return 'tasks' which can have varying completion 13 | times. 14 | """ 15 | from pyVmomi import vmodl, vim 16 | 17 | 18 | __author__ = 'VMware, Inc' 19 | 20 | 21 | # 22 | # @brief Exception class to represent when task is blocked (e.g.: 23 | # waiting for an answer to a question. 24 | # 25 | class TaskBlocked(Exception): 26 | """ 27 | Exception class to represent when task is blocked (e.g.: waiting 28 | for an answer to a question. 29 | """ 30 | pass 31 | 32 | 33 | # 34 | # TaskUpdates 35 | # verbose information about task progress 36 | # 37 | def TaskUpdatesVerbose(task, progress): 38 | if isinstance(task.info.progress, int): 39 | info = task.info 40 | if not isinstance(progress, str): 41 | progress = '%d%% (%s)' % (info.progress, info.state) 42 | print 'Task %s (key:%s, desc:%s) - %s' % ( 43 | info.name.info.name, info.key, info.description, progress) 44 | 45 | 46 | globalTaskUpdate = None 47 | 48 | 49 | def SetTasksVerbose(verbose=True): 50 | global globalTaskUpdate 51 | if verbose: 52 | globalTaskUpdate = TaskUpdatesVerbose 53 | else: 54 | globalTaskUpdate = None 55 | 56 | 57 | # 58 | # @param raiseOnError [in] Any exception thrown is thrown up to the caller if 59 | # raiseOnError is set to true 60 | # @param si [in] ServiceInstance to use. If set to None, use the default one. 61 | # @param pc [in] property collector to use else retrieve one from cache 62 | # @param onProgressUpdate [in] callable to call with task progress updates. 63 | # For example: 64 | # 65 | # def OnTaskProgressUpdate(task, percentDone): 66 | # sys.stderr.write('# Task %s: %d%% complete ...\n' % (task, percentDone)) 67 | # 68 | # Given a task object and a service instance, wait for the task completion 69 | # 70 | # @return state as either "success" or "error". To look at any errors, the 71 | # user should reexamine the task object. 72 | # 73 | # NOTE: This is a blocking call. 74 | # 75 | def WaitForTask(task, 76 | si, 77 | raiseOnError=True, 78 | pc=None, 79 | onProgressUpdate=None): 80 | """ 81 | Wait for task to complete. 82 | @type si : ManagedObjectReference to a ServiceInstance. 83 | @param si : ServiceInstance to use. 84 | @type raiseOnError : bool 85 | @param raiseOnError : Any exception thrown is thrown up to the caller 86 | if raiseOnError is set to true. 87 | @type pc : ManagedObjectReference to a PropertyCollector. 88 | @param pc : Property collector to use. If None, get it from 89 | the ServiceInstance. 90 | @type onProgressUpdate : callable 91 | @param onProgressUpdate : Callable to call with task progress updates. 92 | 93 | For example:: 94 | 95 | def OnTaskProgressUpdate(task, percentDone): 96 | print 'Task %s is %d%% complete.' % (task, percentDone) 97 | """ 98 | if pc is None: 99 | pc = si.content.propertyCollector 100 | 101 | progressUpdater = ProgressUpdater(task, onProgressUpdate) 102 | progressUpdater.Update('created') 103 | 104 | filter = CreateFilter(pc, task) 105 | 106 | version, state = None, None 107 | # Loop looking for updates till the state moves to a completed state. 108 | while state not in (vim.TaskInfo.State.success, vim.TaskInfo.State.error): 109 | version, state = GetTaskStatus(task, version, pc) 110 | progressUpdater.UpdateIfNeeded() 111 | 112 | filter.Destroy() 113 | 114 | if state == "error": 115 | progressUpdater.Update('error: %s' % str(task.info.error)) 116 | if raiseOnError: 117 | raise task.info.error 118 | else: 119 | print "Task reported error: " + str(task.info.error) 120 | else: 121 | progressUpdater.Update('completed') 122 | return state 123 | 124 | 125 | # Wait for multiple tasks to complete 126 | # See WaitForTask for detail 127 | # 128 | # Difference: WaitForTasks won't return the state of tasks. User can check 129 | # tasks state directly with task.info.state 130 | # 131 | # TODO: Did not check for question pending 132 | def WaitForTasks(tasks, 133 | si=None, 134 | raiseOnError=True, 135 | pc=None, 136 | onProgressUpdate=None): 137 | """ 138 | Wait for mulitiple tasks to complete. Much faster than calling WaitForTask 139 | N times 140 | """ 141 | if not tasks: 142 | return 143 | 144 | if pc is None: 145 | pc = si.content.propertyCollector 146 | 147 | progressUpdaters = {} 148 | for task in tasks: 149 | progressUpdater = ProgressUpdater(task, onProgressUpdate) 150 | progressUpdater.Update('created') 151 | progressUpdaters[str(task)] = progressUpdater 152 | 153 | filter = CreateTasksFilter(pc, tasks) 154 | 155 | try: 156 | version, state = None, None 157 | 158 | # Loop looking for updates till the state moves to a completed state. 159 | while len(progressUpdaters): 160 | update = pc.WaitForUpdates(version) 161 | for filterSet in update.filterSet: 162 | for objSet in filterSet.objectSet: 163 | task = objSet.obj 164 | taskId = str(task) 165 | for change in objSet.changeSet: 166 | if change.name == 'info': 167 | state = change.val.state 168 | elif change.name == 'info.state': 169 | state = change.val 170 | else: 171 | continue 172 | 173 | progressUpdater = progressUpdaters.get(taskId) 174 | if not progressUpdater: 175 | continue 176 | 177 | if state == vim.TaskInfo.State.success: 178 | progressUpdater.Update('completed') 179 | progressUpdaters.pop(taskId) 180 | elif state == vim.TaskInfo.State.error: 181 | err = task.info.error 182 | progressUpdater.Update('error: %s' % str(err)) 183 | if raiseOnError: 184 | raise err 185 | else: 186 | print "Task %s reported error: %s" % (taskId, 187 | str(err)) 188 | progressUpdaters.pop(taskId) 189 | else: 190 | if onProgressUpdate: 191 | progressUpdater.UpdateIfNeeded() 192 | # Move to next version 193 | version = update.version 194 | finally: 195 | if filter: 196 | filter.Destroy() 197 | return 198 | 199 | 200 | def GetTaskStatus(task, version, pc): 201 | update = pc.WaitForUpdates(version) 202 | state = task.info.state 203 | 204 | if state == 'running' and \ 205 | task.info.name is not None and \ 206 | task.info.name.info.name != "Destroy": 207 | CheckForQuestionPending(task) 208 | 209 | return update.version, state 210 | 211 | 212 | def CreateFilter(pc, task): 213 | """ Create property collector filter for task """ 214 | return CreateTasksFilter(pc, [task]) 215 | 216 | 217 | def CreateTasksFilter(pc, tasks): 218 | """ Create property collector filter for tasks """ 219 | if not tasks: 220 | return None 221 | 222 | # First create the object specification as the task object. 223 | objspecs = [ 224 | vmodl.query.PropertyCollector.ObjectSpec(obj=task) for task in tasks] 225 | 226 | # Next, create the property specification as the state. 227 | propspec = vmodl.query.PropertyCollector.PropertySpec( 228 | type=vim.Task, pathSet=[], all=True) 229 | 230 | # Create a filter spec with the specified object and property spec. 231 | filterspec = vmodl.query.PropertyCollector.FilterSpec() 232 | filterspec.objectSet = objspecs 233 | filterspec.propSet = [propspec] 234 | 235 | # Create the filter 236 | return pc.CreateFilter(filterspec, True) 237 | 238 | 239 | def CheckForQuestionPending(task): 240 | """ 241 | Check to see if VM needs to ask a question, throw exception 242 | """ 243 | 244 | vm = task.info.entity 245 | if vm is not None and isinstance(vm, vim.VirtualMachine): 246 | qst = vm.runtime.question 247 | if qst is not None: 248 | raise TaskBlocked("Task blocked, User Intervention required") 249 | 250 | 251 | # 252 | # @brief Class that keeps track of task percentage complete and calls 253 | # a provided callback when it changes. 254 | # 255 | class ProgressUpdater(object): 256 | """ 257 | Class that keeps track of task percentage complete and calls a 258 | provided callback when it changes. 259 | """ 260 | 261 | def __init__(self, task, onProgressUpdate): 262 | self.task = task 263 | self.onProgressUpdate = onProgressUpdate 264 | self.prevProgress = 0 265 | self.progress = 0 266 | 267 | def Update(self, state): 268 | global globalTaskUpdate 269 | taskUpdate = globalTaskUpdate 270 | if self.onProgressUpdate: 271 | taskUpdate = self.onProgressUpdate 272 | if taskUpdate: 273 | taskUpdate(self.task, state) 274 | 275 | def UpdateIfNeeded(self): 276 | self.progress = self.task.info.progress 277 | 278 | if self.progress != self.prevProgress: 279 | self.Update(self.progress) 280 | 281 | self.prevProgress = self.progress 282 | -------------------------------------------------------------------------------- /tests/pyVmomiwrapper/requirements.txt: -------------------------------------------------------------------------------- 1 | pyVmomi -------------------------------------------------------------------------------- /tests/pyVmomiwrapper/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='pyVmomiwrapper', 5 | version='0.0.3', 6 | scripts=[], 7 | packages=['pyVmomiwrapper'], 8 | include_package_data=True, 9 | install_requires=[ 10 | 'pyvmomi>=6.0.0' 11 | ] 12 | ) 13 | -------------------------------------------------------------------------------- /tests/shellutil/README.rst: -------------------------------------------------------------------------------- 1 | shellutil 2 | ============= 3 | 4 | A utility module to execute shell commands on linux machine. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com shell-util 13 | 14 | 15 | Usage 16 | ====== 17 | 18 | See tests/test_shell.py 19 | -------------------------------------------------------------------------------- /tests/shellutil/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='shell-util', 5 | version='0.0.1', 6 | scripts=[], 7 | packages=['shellutil'], 8 | include_package_data=True, 9 | install_requires=[] 10 | ) 11 | -------------------------------------------------------------------------------- /tests/shellutil/shellutil/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/shellutil/shellutil/__init__.py -------------------------------------------------------------------------------- /tests/shellutil/shellutil/shell.py: -------------------------------------------------------------------------------- 1 | import contextlib 2 | import logging 3 | import os 4 | import subprocess 5 | 6 | 7 | LOG = logging.getLogger(__name__) 8 | 9 | GLOBAL_PWD = ['.'] 10 | 11 | 12 | class CommandError(Exception): 13 | """Shell command exceptions""" 14 | 15 | 16 | @contextlib.contextmanager 17 | def cd(directory): 18 | """A context manager for switching the current working directory when 19 | 20 | using the local() function. Not thread safe. 21 | """ 22 | 23 | global GLOBAL_PWD 24 | GLOBAL_PWD.append(directory) 25 | yield 26 | if len(GLOBAL_PWD) > 1: 27 | GLOBAL_PWD.pop() 28 | 29 | 30 | def local(cmd, capture=True, pipefail=False, log_method='debug', env=None, 31 | raise_error=False): 32 | """Run a command locally. return code, Output as return value.""" 33 | 34 | log_method = getattr(LOG, log_method) 35 | 36 | if len(GLOBAL_PWD) > 1: 37 | cmd = 'cd %s && %s' % (GLOBAL_PWD[-1], cmd) 38 | 39 | if pipefail: 40 | cmd = 'set -o pipefail && ' + cmd 41 | 42 | env_vars = dict(os.environ.items() + env.items()) if env else os.environ 43 | 44 | log_method('[local] run: %s' % cmd) 45 | process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 46 | stderr=subprocess.STDOUT, env=env_vars, 47 | executable='/bin/bash') 48 | buffer_ = [] 49 | 50 | while True: 51 | line = process.stdout.readline() 52 | if capture: 53 | buffer_.append(line) 54 | if isinstance(line, str): 55 | log_method('[local] out: %s' % 56 | unicode(line.rstrip(), errors='ignore')) 57 | if line == '' and process.poll() is not None: 58 | break 59 | if 'KILL STACK.SH' in line: 60 | process.terminate() 61 | return 0, ''.join(buffer_) 62 | if raise_error and process.returncode: 63 | raise CommandError('Failed to execute: %s' % cmd) 64 | return process.returncode, ''.join(buffer_) 65 | -------------------------------------------------------------------------------- /tests/shellutil/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/shellutil/tests/__init__.py -------------------------------------------------------------------------------- /tests/shellutil/tests/test_shell.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | from shellutil import shell 4 | 5 | 6 | class ShellTest(unittest.TestCase): 7 | def test_cmd_under_dir(self): 8 | shell.local("mkdir -p /tmp/test") 9 | with shell.cd("/tmp"): 10 | with shell.cd("/tmp/test"): 11 | code, output = shell.local("pwd") 12 | self.assertEqual(output, '/tmp/test\n') 13 | code, output = shell.local("pwd") 14 | self.assertEqual(output, '/tmp\n') 15 | 16 | 17 | if __name__ == '__main__': 18 | unittest.main() 19 | -------------------------------------------------------------------------------- /tests/sshutil/README.rst: -------------------------------------------------------------------------------- 1 | sshutil 2 | ============= 3 | 4 | A utility module to connect to and execute commands on remote linux machine. Also supports SCP to/from remote machine. 5 | 6 | 7 | Installation 8 | ============= 9 | 10 | python setup.py install 11 | 12 | pip install --index-url http://p3-pypi.eng.vmware.com:3141/slave/dev/+simple --trusted-host p3-pypi.eng.vmware.com ssh-util 13 | 14 | 15 | Usage 16 | ====== 17 | 18 | See tests/test_remote.py 19 | 20 | Example to use the cli: 21 | ``ssh_exec '10.111.160.16' 'viouser' 'vmware' 'restart oms' --sudo`` 22 | -------------------------------------------------------------------------------- /tests/sshutil/bin/ssh_exec: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import logging 3 | import sys 4 | 5 | import argparse 6 | 7 | from sshutil.remote import RemoteClient 8 | 9 | LOG = logging.getLogger('sshutil') 10 | 11 | 12 | def configure_logging(): 13 | LOG.setLevel(logging.DEBUG) 14 | formatter = logging.Formatter('%(message)s') 15 | 16 | console_handler = logging.StreamHandler(sys.stdout) 17 | console_handler.setLevel(logging.DEBUG) 18 | console_handler.setFormatter(formatter) 19 | LOG.addHandler(console_handler) 20 | 21 | 22 | def main(): 23 | configure_logging() 24 | parser = argparse.ArgumentParser( 25 | prog='ssh_exec', 26 | description='ssh to remote and execute commands.') 27 | parser.add_argument('host', 28 | help='IP address or hostname.') 29 | parser.add_argument('user', 30 | help='User name.') 31 | parser.add_argument('password', 32 | help='Password of the user.') 33 | parser.add_argument('command', 34 | help='Command to run.') 35 | parser.add_argument('--sudo', 36 | action="store_true", 37 | default=False, 38 | help='Execute with sudo.') 39 | args = parser.parse_args() 40 | rc = RemoteClient(args.host, args.user, args.password) 41 | rc.run(args.command, sudo=args.sudo) 42 | return rc.last_exit_status 43 | 44 | 45 | if __name__ == '__main__': 46 | sys.exit(main()) 47 | 48 | -------------------------------------------------------------------------------- /tests/sshutil/bin/ssh_scp: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import logging 3 | import sys 4 | 5 | import argparse 6 | 7 | from sshutil.remote import RemoteClient 8 | 9 | LOG = logging.getLogger('sshutil') 10 | 11 | 12 | def configure_logging(): 13 | LOG.setLevel(logging.DEBUG) 14 | formatter = logging.Formatter('%(message)s') 15 | 16 | console_handler = logging.StreamHandler(sys.stdout) 17 | console_handler.setLevel(logging.DEBUG) 18 | console_handler.setFormatter(formatter) 19 | LOG.addHandler(console_handler) 20 | 21 | 22 | def main(): 23 | configure_logging() 24 | parser = argparse.ArgumentParser( 25 | prog='ssh_scp', 26 | description='scp tool for transfer files between two hosts.') 27 | parser.add_argument('host', 28 | help='IP address or hostname.') 29 | parser.add_argument('user', 30 | help='User name.') 31 | parser.add_argument('password', 32 | help='Password of the user.') 33 | parser.add_argument('command', 34 | choices = ('get','put'), 35 | help='get:Transfers a remote file to the local host.' 36 | 37 | 'put:Transfers a local file to the remote host.') 38 | parser.add_argument('src', 39 | help='Source path is a relative or absolute path to a file.') 40 | parser.add_argument('dest', 41 | help='Destination path is an absolute path on the destination host. ') 42 | args = parser.parse_args() 43 | command = args.command 44 | rc = RemoteClient(args.host, args.user, args.password) 45 | if command == 'get': 46 | rc.get(args.src,args.dest) 47 | else: 48 | rc.scp(args.src,args.dest) 49 | return rc.last_exit_status 50 | 51 | if __name__ == '__main__': 52 | sys.exit(main()) -------------------------------------------------------------------------------- /tests/sshutil/requirements.txt: -------------------------------------------------------------------------------- 1 | paramiko>=1.16.0,<2.0.0 -------------------------------------------------------------------------------- /tests/sshutil/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='ssh-util', 5 | version='0.0.1', 6 | scripts=['bin/ssh_exec', 'bin/ssh_scp'], 7 | packages=['sshutil'], 8 | include_package_data=True, 9 | 10 | install_requires=[ 11 | 'paramiko' 12 | ] 13 | ) 14 | -------------------------------------------------------------------------------- /tests/sshutil/sshutil/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/sshutil/sshutil/__init__.py -------------------------------------------------------------------------------- /tests/sshutil/sshutil/remote.py: -------------------------------------------------------------------------------- 1 | """This module contains class that represents a host reachable by SSH.""" 2 | 3 | import logging 4 | import paramiko 5 | import re 6 | import select 7 | 8 | 9 | LOG = logging.getLogger(__name__) 10 | 11 | 12 | class RemoteError(Exception): 13 | """Remote command exceptions""" 14 | 15 | 16 | class RemoteClient(object): 17 | """Representation of a host reachable by SSH""" 18 | 19 | def __init__(self, host_ip, user, password=None): 20 | """ 21 | :param host_ip: IP address of host 22 | :param user: Username of user to login with 23 | :param password: Password of user to login with 24 | :param alt_logger: Alternative logger to use instead of this module's 25 | logger. Used only in run() method. 26 | :param alt_log_method: Alternative logger method to use when 27 | alt_logger is enabled. 28 | """ 29 | self.host_ip = host_ip 30 | self.user = user 31 | self.password = password 32 | self.client = None 33 | self.last_exit_status = None 34 | 35 | def run(self, cmd, capture=True, sudo=False, env_vars=None, 36 | raise_error=False, log_method='debug', feed_input=None): 37 | """Run a command on the host and return its output 38 | :param cmd: Command to run on host 39 | :param capture: Whether to save and return output from command run 40 | :param sudo: add sudo before command 41 | :param env_vars: dict that contains environment variable to be set 42 | :param raise_error: raise exception if exit status is not zero. 43 | :param log_method: log method of logger. 44 | """ 45 | log_method = getattr(LOG, log_method) 46 | if not self.client: 47 | self._set_client() 48 | 49 | transport = self.client.get_transport() 50 | channel = transport.open_session() 51 | channel.set_combine_stderr(True) 52 | env_cmd = "" 53 | if env_vars: 54 | for var, value in env_vars.iteritems(): 55 | env_cmd += 'export %s=%s;' % (var, value) 56 | cmd = '%s %s' % (env_cmd, cmd) 57 | 58 | feed_password = False 59 | if sudo and self.user != "root": 60 | cmd = "sudo -S -p '' %s" % cmd 61 | feed_password = self.password is not None and ( 62 | len(self.password) > 0) 63 | 64 | log_method('[%s] run: %s' % (self.host_ip, cmd)) 65 | channel.exec_command(cmd) 66 | 67 | buffer_ = [] # This should probably be a limited-length buffer 68 | line = [] 69 | seen_cr = False 70 | 71 | def flush(text): 72 | if isinstance(text, str): 73 | log_method('[%s] out: %s' % (self.host_ip, 74 | unicode(text.rstrip(), errors='ignore'))) 75 | 76 | def has_newline(bytelist): 77 | return '\r' in bytelist or '\n' in bytelist 78 | 79 | if feed_password: 80 | channel.sendall(self.password + '\n') 81 | 82 | if feed_input: 83 | channel.sendall(feed_input + '\n') 84 | 85 | while not channel.exit_status_ready(): 86 | rlist, wlist, xlist = select.select([channel], [], [], 10.0) 87 | # wait until ready for reading 88 | if len(rlist) > 0: 89 | bytelist = channel.recv(512) 90 | if capture: 91 | buffer_.append(bytelist) 92 | 93 | # empty byte signifies EOS 94 | if bytelist == '': 95 | if line: 96 | flush(''.join(line)) 97 | break 98 | 99 | if bytelist[-1] == '\r': 100 | seen_cr = True 101 | if bytelist[0] == '\n' and seen_cr: 102 | bytelist = bytelist[1:] 103 | seen_cr = False 104 | 105 | while has_newline(bytelist) and bytelist != '': 106 | # at most 1 split ! 107 | cr = re.search('(\r\n|\r|\n)', bytelist) 108 | if cr is None: 109 | break 110 | end_of_line = bytelist[:cr.start(0)] 111 | bytelist = bytelist[cr.end(0):] 112 | 113 | if has_newline(end_of_line): 114 | end_of_line = '' 115 | 116 | flush(''.join(line) + end_of_line + '\n') 117 | line = [] 118 | 119 | line += [bytelist] 120 | self.last_exit_status = channel.recv_exit_status() 121 | if raise_error and self.last_exit_status: 122 | raise RemoteError('In host %s failed to execute: %s' % 123 | (self.host_ip, cmd)) 124 | return ''.join(buffer_) 125 | 126 | def scp(self, src, dest, log_method='debug'): 127 | """Transfers a local file to the remote host. src is a relative or 128 | absolute path to a file. dest is the absolute path on the destination 129 | host.""" 130 | log_method = getattr(LOG, log_method) 131 | if not self.client: 132 | self._set_client() 133 | 134 | sftp = paramiko.SFTPClient.from_transport(self.client.get_transport()) 135 | file_ = src.rpartition('/')[-1] 136 | sftp.put(src, '/'.join([dest.rstrip('/'), file_])) 137 | log_method('[%s] scp: %s to %s' % (self.host_ip, src, dest)) 138 | sftp.close() 139 | 140 | def get(self, src, dest, log_method='debug'): 141 | """Transfers a remote file to the local host. src is a relative or 142 | absolute path to a file. dest is the absolute path on the destination 143 | host.""" 144 | log_method = getattr(LOG, log_method) 145 | if not self.client: 146 | self._set_client() 147 | 148 | sftp = paramiko.SFTPClient.from_transport(self.client.get_transport()) 149 | file_ = src.rpartition('/')[-1] 150 | sftp.get(src, '/'.join([dest.rstrip('/'), file_])) 151 | log_method('[%s] get: %s to %s' % (self.host_ip, src, dest)) 152 | sftp.close() 153 | 154 | def reload_client(self): 155 | if self.client: 156 | self.client.close() 157 | 158 | self._set_client() 159 | 160 | def check_connection(self): 161 | self._set_client() 162 | 163 | def _set_client(self): 164 | self.client = paramiko.SSHClient() 165 | self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) 166 | self.client.connect(self.host_ip, username=self.user, 167 | password=self.password) 168 | -------------------------------------------------------------------------------- /tests/sshutil/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vmware-archive/vio/f2d5511bf1bac57f9670d5384debfb89750f9873/tests/sshutil/tests/__init__.py -------------------------------------------------------------------------------- /tests/sshutil/tests/test_remote.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | from sshutil import remote 4 | 5 | 6 | class RemoteTest(unittest.TestCase): 7 | def test_run_with_capture(self): 8 | rc = remote.RemoteClient('nimbus-gateway.eng.vmware.com', 9 | 'vio-autouser', '!ya6u4uWY2u@egYvU') 10 | self.assertEqual(rc.run('pwd'), '/mts/home1/vio-autouser\n') 11 | 12 | def test_run_without_capture(self): 13 | rc = remote.RemoteClient('nimbus-gateway.eng.vmware.com', 14 | 'vio-autouser', '!ya6u4uWY2u@egYvU') 15 | self.assertEqual(rc.run('pwd', capture=False), '') 16 | 17 | 18 | if __name__ == '__main__': 19 | unittest.main() 20 | -------------------------------------------------------------------------------- /util/README.rst: -------------------------------------------------------------------------------- 1 | Helpful utilities 2 | ================= 3 | 4 | 1. openstack-client - A simple Vagrantfile to deploy Ubuntu 15.04 with the OpenStack CLI binaries deployed for you. 5 | 2. openstack-cli-windows - PowerShell script for setting your OpenStack environment variables and instructions for setting up the OpenStack CLI utilities. -------------------------------------------------------------------------------- /util/openstack-cli-windows/README.md: -------------------------------------------------------------------------------- 1 | # openstack-cli-on-windows 2 | Helpful hints on how to use the OpenStack clients on your Windows desktop 3 | 4 | ## Prerequisites: 5 | 1. Install the [GitHub Windows client](https://desktop.github.com/) 6 | 1. Install the latest release of [Python 2](https://www.python.org/downloads/) on your Windows desktop. 7 | 2. Upgrade the Python package manager (pip): ```python -m pip install --upgrade pip``` 8 | 3. Install the OpenStack CLI packages: 9 | ``` 10 | pip install python-novaclient python-glanceclient python-neutronclient python-cinderclient python-swiftclient python-heatclient 11 | ``` 12 | 13 | ## Usage: 14 | 1. Clone the repo: ```git clone https://github.com/vmware/vio.git``` 15 | 2. Download your openrc file from your own tenant on your group's OpenStack cloud. 16 | 3. Change to the **vio/util/openstack-cli-windows** directory and edit the **openstack-cli-env.ps1** file with the values from the openrc file. 17 | 4. Run the **openstack-cli-env.ps1** in a PowerShell window and execute your OpenStack commands as you normally would 18 | **NOTE** Using multiple lines for your OpenStack CLI commands will require a backtick (`) instead of a backslash (\\) 19 | ``` 20 | glance image-create --property vmware_disktype=preallocated ` 21 | --property vmware_adaptertype=lsiLogicsas --name windows-2012-r2-test ` 22 | --property vmware_ostype=windows8Server64Guest --container-format bare ` 23 | --disk-format vmdk --min-disk 40 --min-ram 512 --progress ` 24 | --file z:\windows2012-blank-flat-control.vmdk 25 | ``` 26 | -------------------------------------------------------------------------------- /util/openstack-cli-windows/openstack-cli-env.ps1: -------------------------------------------------------------------------------- 1 | $env:OS_AUTH_URL="https://your-server.domain.com:5000/v2.0" 2 | $env:OS_TENANT_NAME="your-project" 3 | $env:OS_USERNAME="your-user" 4 | #CACERT is optional, and only use it if you don't have a CA-signed certificate for your cloud 5 | #$env:OS_CACERT="C:\your\path\vio.pem" 6 | $Password = Read-Host -Prompt "OpenStack User Password?" -AsSecureString 7 | $env:OS_PASSWORD = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($Password)) -------------------------------------------------------------------------------- /util/openstack-client/README.md: -------------------------------------------------------------------------------- 1 | # openstack-client 2 | Vagrant-automated Ubuntu 15.04 deployment that contains essential OpenStack CLI packages 3 | 4 | ## Prerequisites: 5 | 1. [Vagrant](www.vagrantup.com) 6 | 2. Desktop Virtualization solution ([VMware App Catalyst](https://www.vmware.com/cloudnative/appcatalyst-download), [VMware Workstation](https://www.vmware.com/products/workstation), [VMware Fusion](https://www.vmware.com/products/fusion), [Virtualbox](https://www.virtualbox.org/wiki/Downloads)) 7 | 8 | **NOTE:** Vagrant supports Virtualbox by default. Vagrant requires plugins to support VMware solutions: 9 | - [App Catalyst Vagrant Plugin](https://github.com/vmware/vagrant-vmware-appcatalyst) 10 | - [Workstation and Fusion Vagrant Plugins](https://www.vagrantup.com/vmware/) 11 | 12 | ## Usage: 13 | 1. Clone the repo: ```git clone https://github.com/vmware/vio.git``` 14 | 2. Change to the **vio/util/openstack-client** directory and Vagrant up! ```vagrant up``` 15 | 3. Wait for the provisioning shell script to complete (You will eventually see some **apt-get** output at it completes). 16 | 4. Connect to the VM ```vagrant ssh``` 17 | 5. Download your openrc file from your OpenStack cloud. Here is a sample 18 | ``` 19 | #!/bin/bash 20 | # Optional environment variable to suppress cosmetic HTTPS warnings from the updated OpenStack CLI packages 21 | export PYTHONWARNINGS="ignore:Unverified HTTPS request" 22 | export OS_AUTH_URL=https://your-openstack-deployment.vmware.com:5000/v2.0 23 | export OS_TENANT_NAME="demo-project" 24 | export OS_USERNAME="demo-user" 25 | # Optional environment variable if you do not yet have a signed certificate. 26 | # In VIO, the PEM file is located /etc/ssl/vio.pem on the Load Balancer VMs 27 | export OS_CACERT=/your/path/vio.pem 28 | echo "Please enter your OpenStack Password: " 29 | read -sr OS_PASSWORD_INPUT 30 | export OS_PASSWORD=$OS_PASSWORD_INPUT 31 | ``` -------------------------------------------------------------------------------- /util/openstack-client/Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | Vagrant.configure(2) do |config| 5 | config.vm.box = "boxcutter/ubuntu1504" 6 | config.vm.hostname = "openstack-client" 7 | config.vm.provision "shell", path: "src/base-prep.sh" 8 | end 9 | -------------------------------------------------------------------------------- /util/openstack-client/src/base-prep.sh: -------------------------------------------------------------------------------- 1 | # Update package repo sources 2 | sudo apt-get -qq update 3 | 4 | # Install pre-req apt packages 5 | sudo apt-get install -qqy vim python-pip python-dev build-essential libxslt1-dev libxml2-dev 6 | 7 | # Install pre-req pip package 8 | sudo pip install -q netifaces 9 | 10 | # Install OpenStack clients 11 | sudo pip install -q python-novaclient python-glanceclient python-cinderclient python-neutronclient python-heatclient python-openstackclient 12 | --------------------------------------------------------------------------------