├── .github └── ISSUE_TEMPLATE.md ├── .gitignore ├── CONTRIBUTING.md ├── DEFCORE.md ├── Dockerfile ├── LICENSE ├── README.md ├── TESTING.md ├── Vagrantfile ├── ansible-role-master-requirements.yml ├── ansible-role-newton-requirements.yml ├── ansible-role-ocata-requirements.yml ├── ansible-role-pike-requirements.yml ├── ansible-role-queens-requirements.yml ├── ansible-role-rocky-requirements.yml ├── etc └── openstack_deploy │ ├── env.d │ └── fleet.yml │ ├── group_vars │ ├── all │ │ ├── osa.yml │ │ ├── release.yml │ │ └── rpc-o.yml │ └── log_hosts.yml │ ├── user_rpco_maas_variables.yml.example │ └── user_rpco_secrets.yml.example ├── gating ├── capabilities │ └── aio_config ├── check │ ├── collect_logs.yml │ ├── post │ ├── post_deploy.sh │ ├── post_send_junit_to_qtest.sh │ ├── pre │ ├── pre_deploy_mnaio.sh │ ├── run │ ├── run_deploy_mnaio.sh │ ├── run_elk_tests.sh │ └── run_system_tests.sh ├── gate ├── gating_prerequisites.sh ├── gating_vars.sh ├── generate_release_notes │ ├── generate_commit_diff_notes.sh │ ├── generate_release_notes.sh │ ├── generate_reno_report.sh │ └── run ├── mnaio_vars.sh ├── periodic ├── release ├── thaw │ ├── fix_endpoints.yml │ ├── haproxycheck.yml │ ├── run │ └── thaw.yml └── update_dependencies │ ├── pre │ ├── release-update.py │ ├── role-requirements-update.py │ └── run ├── logo_files ├── favicon.ico ├── logo-splash.png ├── logo-splash.svg ├── logo.png └── logo.svg ├── playbooks ├── configure-release.yml ├── deployment-elk.yml ├── deployment-osquery.yml ├── files │ └── ironic_all.yml ├── get-maas.yml ├── openstack-ansible-install.yml ├── openstack-ansible-ops-get.yml ├── openstack-flavor-setup.yml ├── openstack-image-setup.yml ├── ops-cpu-govenor-setup.yml ├── run_tempest.yml ├── site-logging.yml ├── site-openstack.yml ├── site-ops.yml ├── site-release.yml ├── site.yml └── vars │ ├── openstack-service-config.yml │ └── rpc-release.yml ├── releasenotes ├── config.yaml ├── notes │ ├── 13.0.0-8158f0be466af558.yaml │ ├── LVMISCSIDriver-b25cdd70ef0c5a4d.yaml │ ├── add-magnum-f5-93cc4c09fdb60d23.yaml │ ├── add-more-galera-checks-032c765829e922a2.yaml │ ├── add-option-to-disable-artifacts-23125367d1c1e834.yaml │ ├── add-post-upgrade-role-3821a510f5be4aae.yaml │ ├── add-tempest-execution-playbook-efe64a286f255c34.yaml │ ├── ceph-upgrade-434a53379d0db8f5.yaml │ ├── disable_octavia_image_download-0d64f2dc019f7d0d.yaml │ ├── empty-yaml-bcbc0c731bed02e7.yaml │ ├── enable-security-hardening-2d9f88309ad45ac0.yaml │ ├── fix-get-rpc_release-rpc_product_release.yaml │ ├── fix-rpc-maas-tool-check-validation-a5190a72be5830f6.yaml │ ├── handle-merging-new-secrets-0e1fd06d86f0abbd.yaml │ ├── holland-venv-9eb62140a2c8ff56.yaml │ ├── horizon-site-name-62c749ec2ab8b0b3.yaml │ ├── image-metadata-8d1562d924ffd7e5.yaml │ ├── maas-metric-limit-increase-fc17cbec85a7952c.yaml │ ├── maas-plugins-venv-4a195e0b0271bf29.yaml │ ├── maas-version-var-2ea08457b0ad00b4.yaml │ ├── master-27a94947a8611804.yaml │ ├── master-86edb3c41de33c79.yaml │ ├── master-96b4f6d34b87fe3d.yaml │ ├── master-b7edb49a0bbe6c2c.yaml │ ├── multi-layered-group-vars-cb60fcf454ca3bbe.yaml │ ├── nova-13.0-0c2497128e677cb2.yaml │ ├── nova-allocation-ratios-869b6d3cabbf914d.yaml │ ├── override-l2pop-default-d7426c07d8b72f2e.yaml │ ├── rabbitmq-ha-configuration-c5062133932bd82b.yaml │ ├── releasenotes-a800a042ad4e67cf.yaml │ ├── remove-legacy-elk-f603293888afce53.yaml │ ├── rename-ansible-ssh-host-7332cc551f494495.yaml │ ├── rm-horizon-extensions-from-repo-server-3d441a4d0d1c6ad0.yaml │ ├── rpc-pre-upgrade-93d419ae17c5e495.yaml │ ├── rpc_package_index-a03d4d131c16d710.yaml │ ├── security-hardening-upgrade-69d0bca94fdc7df9.yaml │ ├── setcheckfirewallvariabletofalse-de49a72bbad0e7db.yaml │ ├── turn-off-l3ha-18274d19d54a6315.yaml │ └── xtrabackup-localhost-8d1562d924ffd6e5.yaml └── source │ ├── _deconst.json │ ├── _static │ └── .placeholder │ ├── _templates │ └── .placeholder │ ├── conf.py │ ├── deconst-requirements.txt │ ├── index.rst │ └── master.rst ├── scripts ├── deploy-rpco.sh ├── deploy.sh ├── functions.sh ├── get-rpc_release.py ├── install.sh └── user_aio_variables.yml ├── setup.cfg ├── setup.py ├── test-requirements.txt ├── tests ├── .gitkeep └── test.yml ├── tox.ini └── upper-constraints.txt /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | > **THIS IS A PUBLIC FACING ISSUE.** 2 | > 3 | > Do not include private or confidential information in this issue. 4 | 5 | ## Is this a feature or a bug? 6 | 7 | - [ ] Feature _(request for enhancement, new functionality)_ 8 | - [ ] Bug _(something is broken)_ 9 | 10 | ## Which version of rpc-openstack does this issue relate to? 11 | - [ ] master 12 | - [ ] newton-14.0 13 | - [ ] mitaka-13.1 14 | - [ ] liberty-12.2 15 | 16 | Run `git log --pretty=oneline | head -n 1` and provide the output below between 17 | the backtick lines: 18 | 19 | ``` 20 | 21 | ``` 22 | 23 | ## What happened? 24 | 25 | > Example: I can't snapshot a cinder volume on an AIO RPC-O deployment. 26 | 27 | ## What were you doing when it happened? 28 | 29 | > Example: I have an existing cinder volume and I get this error when I try 30 | > to snapshot it using openstackclient: 31 | > 32 | > $ openstack --os-cloud=mycloud volume snapshot list 33 | > publicURL endpoint for volumev2 service in RegionOne region not found 34 | 35 | ## What did you expect to happen instead of what actually happened? 36 | 37 | > Example: I expected a cinder volume snapshot to be successfully created. 38 | 39 | ## Do you think the documentation team wants to know about this? 40 | 41 | - [ ] Yes, and I submitted an issue to [Docs issues](https://github.com/rackerlabs/docs-rpc/issues "Docs issues") 42 | - [ ] Yes, and I contributed a documentation patch to [docs-rpc](https://github.com/rackerlabs/docs-rpc "docs-rpc") 43 | - [ ] No documentation required for this change 44 | 45 | ## Does your change require a release note? 46 | 47 | - [ ] Yes, it's a pretty big deal like I am and I've made a reno note! 48 | - [ ] Nope, this change has no major impact. 49 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled source # 2 | ################### 3 | *.com 4 | *.class 5 | *.dll 6 | *.exe 7 | *.o 8 | *.so 9 | *.pyc 10 | build/ 11 | dist/ 12 | doc/build/ 13 | 14 | # Packages # 15 | ############ 16 | # it's better to unpack these files and commit the raw source 17 | # git has its own built in compression methods 18 | *.7z 19 | *.dmg 20 | *.gz 21 | *.iso 22 | *.jar 23 | *.rar 24 | *.tar 25 | *.zip 26 | 27 | # Logs and databases # 28 | ###################### 29 | *.log 30 | *.sql 31 | *.sqlite 32 | logs/* 33 | 34 | # OS generated files # 35 | ###################### 36 | .DS_Store 37 | .DS_Store? 38 | ._* 39 | .Spotlight-V100 40 | .Trashes 41 | .idea 42 | .tox 43 | *.sublime* 44 | *.egg-info 45 | Icon? 46 | ehthumbs.db 47 | Thumbs.db 48 | .eggs 49 | 50 | # Sphinx documentation 51 | docs/_build/ 52 | 53 | # User driven backup files # 54 | ############################ 55 | *.bak 56 | *.swp 57 | 58 | # Generated by pbr while building docs 59 | ###################################### 60 | AUTHORS 61 | ChangeLog 62 | 63 | # Files created by releasenotes build 64 | releasenotes/build 65 | 66 | # Test temp files 67 | tests/plugins 68 | tests/playbooks 69 | tests/common 70 | tests/*.retry 71 | 72 | # Vagrant artifacts 73 | .vagrant 74 | 75 | # Eclipse artifacts 76 | .project 77 | .pydevproject 78 | -------------------------------------------------------------------------------- /DEFCORE.md: -------------------------------------------------------------------------------- 1 | # Running RefStack against RPC-OpenStack 2 | 3 | The [RefStack](https://refstack.openstack.org/#/) tool runs a specific set of 4 | Tempest tests against an OpenStack deployment to ensure it meets the 5 | interoperability requirements for OpenStack clouds. This is often called 6 | "DefCore". 7 | 8 | You will need a fully deployed RPC environment to run the RefStack tests. AIO 9 | environments are fine, so long as they have spare RAM and disk available for 10 | some temporary virtual machines. The RPC environment must have a utility 11 | container and hardware virtualization (such as Intel VMX) must be enabled. 12 | 13 | ### Installing `refstack-client` 14 | 15 | Within the utility container, run the following: 16 | 17 | ``` shell 18 | git clone https://github.com/openstack/refstack-client /root/refstack-client 19 | rm -rf ~/.pip 20 | cd /root/refstack-client 21 | ./setup_env 22 | ``` 23 | 24 | You should now have `refstack-client` installed into a virtual environment 25 | along with tempest in `/root/refstack-client/.venv`. 26 | 27 | ### Get the RefStack test list 28 | 29 | The list of RefStack tests is revised regularly (every 6-8 months) and you will 30 | need the latest list from the 31 | [RefStack site](https://refstack.openstack.org/#/). 32 | Go to the [DefCore Guidelines](https://refstack.openstack.org/#/guidelines) tab 33 | and ensure the following items are selected: 34 | 35 | * **Version:** *choose the latest available version* 36 | * **Target Program:** OpenStack Powered Platform 37 | * **Capability Status:** Only *Required* is checked 38 | 39 | Be sure that the *Corresponding OpenStack Releases* contains the branch of 40 | OpenStack that you are attempting to test. 41 | 42 | Click **Test List** on the right side of the page and note the `wget` line 43 | provided in the pop-up window. Run that `wget` line within `/root/refstack- 44 | client/` directory in the utility container. 45 | 46 | ### Install tempest 47 | 48 | From the deployment host, install tempest into the utility container: 49 | 50 | ``` shell 51 | cd /opt/rpc-openstack/openstack-ansible/playbooks 52 | openstack-ansible os-tempest-install.yml 53 | ``` 54 | 55 | ### Configure tempest 56 | 57 | Refstack now requires pre-provisioned credentials and the corresponding 58 | accounts file to funtion. Create an accounts file in `~/.tempest/etc/` called 59 | `accounts.yaml` with the following content: 60 | 61 | ``` yaml 62 | - username: 'admin' 63 | tenant_name: 'admin' 64 | password: 'yourpasswordhere' 65 | resources: 66 | network: 'private' 67 | router: 'router' 68 | ``` 69 | 70 | Next, change add the following line to the `[auth]` section in 71 | `~/.tempest/etc/tempest.conf`: 72 | 73 | ``` conf 74 | test_accounts_file = /root/.tempest/etc/accounts.yaml 75 | ``` 76 | 77 | For more information please see: https://docs.openstack.org/developer/tempest/configuration.html#pre-provisioned-credentials 78 | 79 | ### Run RefStack 80 | 81 | Go back to the utility container and run `refstack-client`: 82 | 83 | ``` shell 84 | cd /root/refstack-client 85 | source .venv/bin/activate 86 | ./refstack-client test \ 87 | -c ~/.tempest/etc/tempest.conf \ 88 | --upload \ 89 | --test-list 2017.01-test-list.txt \ 90 | -v | tee -a reflog.txt 91 | ``` 92 | 93 | It will take 20-40 minutes to run, depending on the speed of your server(s). 94 | When the run is complete, you will see a URL printed at the end of the output. 95 | This URL contains the detailed results for your RefStack run and the results 96 | must show 100% compliance for required tests. 97 | 98 | As an example, 99 | [this is our Newton (14.0) report](https://refstack.openstack.org/#/results/6a4a6cb4-13ba-42d2-8789-f456060370ca) 100 | [and this is our Pike (16.0) report](https://refstack.openstack.org/#/results/2d2e8eaa-4cf5-4494-a418-97dbb1fee74a) 101 | 102 | ## Troubleshooting 103 | 104 | If a test failed and you're not sure why, you can run that individual test 105 | again. Use the `--regex` option provided by `ostestr` (from tempest) to specify 106 | a regex for tests that you want to run. 107 | 108 | Here's an example: 109 | 110 | ``` shell 111 | cd /root/refstack-client 112 | source .venv/bin/activate 113 | ./refstack-client test \ 114 | -c /opt/tempest_untagged/etc/tempest.conf \ 115 | -v \ 116 | -- --regex 'test_list_servers_filtered_by_ip_regex' 117 | ``` 118 | 119 | Only the test(s) with `test_list_servers_filtered_by_ip_regex` in their name 120 | will run again. You can troubleshoot failing tests by: 121 | 122 | * Watching `/var/log/utility/tempest.log` in the utility container shows the 123 | API requests that tempest is making 124 | * Reviewing individual service logs in other containers (go look at 125 | neutron-server logs if a Neutron test is failing) 126 | * Trying to reproduce the issue yourself via API or via Horizon 127 | 128 | ## Notify the OpenStack Foundation 129 | 130 | Start by sending a formal request to the 131 | [Interop list](mailto:interop@openstack.org). This creates a ticket with the 132 | OpenStack Foundation to have the DefCore results updated. You can also ping 133 | `hogepoge` in `#refstack` on Freenode to see if the process has changed. 134 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | RUN apt-get update && apt-get install -y python2.7 python-pip python-dev python3 python3-pip python3-dev build-essential libssl-dev libffi-dev sudo git 3 | RUN pip install virtualenv 'tox!=2.4.0,>=2.3' jenkinsapi 4 | RUN useradd jenkins --shell /bin/bash --create-home --uid 500 5 | RUN echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers 6 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # This project is EOL 3 | 4 | This project is retired with Rocky and releases forward are using native openstack-ansible. 5 | Do not file any issues in this project 6 | 7 | **This project is EOL, so we will not migrate it to the new docs.rackspace.com publishing infrastructure.** 8 | 9 | # Rackspace Private Cloud - OpenStack 10 | 11 | The RPC-OpenStack repository contains additional scripts, variables, and 12 | options for deploying an OpenStack cloud. It is a thin wrapper around the 13 | [OpenStack-Ansible](https://github.com/openstack/openstack-ansible) 14 | deployment framework that is part of the OpenStack namespace. 15 | 16 | ## Deployment options 17 | 18 | There are two different types of RPC-OpenStack deployments available: 19 | 20 | * **All-In-One (AIO) Deployment.** An AIO is a quick way to test a 21 | RPC-OpenStack deployment. All of the cloud's internal services are deployed 22 | on the same server, which could be a physical server or a virtual machine. 23 | 24 | * **Production Deployment.** Production deployments should be done on more 25 | than one server with at least three nodes available to run the internal 26 | cloud services. 27 | 28 | * **Upgrading the RPC-OpenStack Product.** Upgrading the RPC-OpenStack Product 29 | using intra-series releases. 30 | 31 | ### All-In-One (AIO) Deployment Quickstart 32 | 33 | Clone the RPC-OpenStack repository: 34 | 35 | ``` shell 36 | git clone https://github.com/rcbops/rpc-openstack /opt/rpc-openstack 37 | ``` 38 | 39 | Start a screen or tmux session (to ensure that the deployment continues even 40 | if the ssh connection is broken) and run `deploy.sh`: 41 | 42 | Run the ``deploy.sh`` script within a tmux or screen session: 43 | 44 | ``` shell 45 | tmux 46 | cd /opt/rpc-openstack 47 | export DEPLOY_AIO=true 48 | export RPC_PRODUCT_RELEASE="master" # This is optional, if unset the current stable product will be used 49 | ./scripts/deploy.sh 50 | ``` 51 | 52 | The `deploy.sh` script will run all of the necessary playbooks to deploy an 53 | AIO cloud and it normally completes in 90 to 120 minutes. 54 | 55 | ### Production Deployment Guide 56 | 57 | Clone the RPC-OpenStack repository: 58 | 59 | ``` shell 60 | git clone https://github.com/rcbops/rpc-openstack /opt/rpc-openstack 61 | ``` 62 | 63 | #### Run the basic system installation 64 | 65 | Start a screen or tmux session (to ensure that the deployment continues even 66 | if the ssh connection is broken) and run `deploy.sh`: 67 | 68 | Run the ``deploy.sh`` script within a tmux or screen session: 69 | 70 | ``` shell 71 | cd /opt/rpc-openstack 72 | export RPC_PRODUCT_RELEASE="master" # This is optional, if unset the current stable product will be used 73 | ./scripts/deploy.sh 74 | ``` 75 | 76 | #### Configure and deploy the cloud 77 | 78 | To configure the installation please refer to the upstream OpenStack-Ansible 79 | documentation regarding basic [system setup](https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/configure.html). 80 | 81 | ##### OpenStack-Ansible Installation 82 | 83 | OpenStack-Ansible will need to be installed. While you can simply run the 84 | `bootstrap-ansible.sh` script provided by the OpenStack-Ansible community 85 | you may also run the `openstack-ansible-install.yml` playbook which was 86 | created for convenience and will maintain impotency. 87 | 88 | ``` shell 89 | cd /opt/rpc-openstack 90 | export RPC_PRODUCT_RELEASE="master" # This is optional, if unset the current stable product will be used 91 | /opt/rpc-ansible/bin/ansible-playbook -i 'localhost,' playbooks/openstack-ansible-install.yml 92 | ``` 93 | 94 | ###### Optional | Setting the OpenStack-Ansible release 95 | 96 | It is possible to set the OSA release outside of the predefined "stable" release 97 | curated by the RPC-OpenStack product. To set the release define the Ansible 98 | variable `osa_release` to a SHA, Branch, or Tag and run the `site-release.yml` 99 | and `openstack-ansible-install.yml` playbooks to install the correct version. 100 | 101 | ``` shell 102 | openstack-ansible site-release.yml openstack-ansible-install.yml -e 'osa_release=master' 103 | ``` 104 | 105 | ##### Running the playbooks 106 | 107 | Once the deploy configuration has been completed please refer to the 108 | OpenStack-Ansible documentation regarding [running the playbooks](https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/run-playbooks.html). 109 | 110 | ---- 111 | 112 | #### Deploy the Rackspace Value Added Services 113 | 114 | Upon completion of the deployment run `scripts/deploy-rpco.sh` script to 115 | apply the RPC-OpenStack value added services; you may also run the playbooks 116 | `site-logging.yml` to accomplish much of the same things. 117 | 118 | ``` shell 119 | cd /opt/rpc-openstack 120 | openstack-ansible site-logging.yml 121 | ``` 122 | 123 | Post deployment run the **optional** `site-openstack.yml` playbooks to setup 124 | default flavors and images. 125 | 126 | ``` shell 127 | cd /opt/rpc-openstack 128 | openstack-ansible site-openstack.yml 129 | ``` 130 | 131 | ---- 132 | 133 | ### Perform an Intra-Series Product Upgrade 134 | 135 | To run a basic system upgrade set the `${RPC_PRODUCT_RELEASE}` option, re-run 136 | `deploy.sh`. 137 | 138 | ``` shell 139 | tmux 140 | cd /opt/rpc-openstack 141 | export RPC_PRODUCT_RELEASE="master" # This is optional, if unset the current stable product will be used 142 | ./scripts/deploy.sh 143 | openstack-ansible openstack-ansible-install.yml 144 | ``` 145 | 146 | Once basic system configuration has completed, [run through the upgrade process](https://docs.openstack.org/openstack-ansible/latest/user/minor-upgrade.html) 147 | for the specified product release. 148 | 149 | ### Perform a Major Product Upgrade (BETA) 150 | 151 | To run a major upgrade set the `${RPC_PRODUCT_RELEASE}` option, re-run 152 | `deploy.sh`. 153 | 154 | ``` shell 155 | tmux 156 | cd /opt/rpc-openstack 157 | export RPC_PRODUCT_RELEASE="master" # This needs to be set to the new product 158 | ./scripts/deploy.sh 159 | openstack-ansible openstack-ansible-install.yml 160 | ``` 161 | 162 | Once the deployment is ready either [run the major upgrade script](https://docs.openstack.org/openstack-ansible/latest/user/script-upgrade.html) 163 | or [run the manual upgrade](https://docs.openstack.org/openstack-ansible/latest/user/manual-upgrade.html) 164 | process. 165 | 166 | 167 | ### Remove legacy ELK from an RPC-OpenStack deployment and deploy ELK 6x. 168 | 169 | The following procedure will deactivate the legacy ELK tooling within an 170 | environment and redeploy the new tools. 171 | 172 | If the **optional** step to destroy the containers is not executed, the old 173 | containers will remain online but will no longer receive any more data. 174 | 175 | * Retrieve the ops tooling from openstack-ansible. 176 | 177 | ``` shell 178 | git clone https://github.com/openstack/openstack-ansible-ops /opt/openstack-ansible-ops 179 | ``` 180 | 181 | * **OPTIONAL** | Destroy existing ELK related containers. 182 | 183 | ``` shell 184 | cd /opt/openstack-ansible/playbooks || cd /opt/rpc-openstack/openstack-ansible/playbooks 185 | ansible all -m service -a 'name=filebeat state=stopped' 186 | openstack-ansible lxc-containers-destroy.yml --limit 'elasticsearch_all:kibana_all:logstash_all' 187 | ``` 188 | 189 | * Move old config files out of the way. 190 | 191 | ``` shell 192 | rm /etc/openstack_deploy/env.d/{elasticsearch,kibana,logstash}.yml 193 | ``` 194 | 195 | * Ensure the legacy implementation of `filebeat` is stopped and removed. 196 | 197 | ``` shell 198 | ansible -m apt -a 'name=filebeat state=absent' all 199 | ``` 200 | 201 | * Remove old containers from openstack-ansible inventory. 202 | 203 | ``` shell 204 | for i in $(../scripts/inventory-manage.py -l | grep -e elastic -e kibana -e logstash | awk '{print $2}'); do 205 | echo "Removing $i" 206 | ../scripts/inventory-manage.py -r "${i}" 207 | done 208 | ``` 209 | 210 | * Deploy the new ELK infrastructure 211 | 212 | ``` shell 213 | cd /opt/rpc-openstack/playbooks 214 | openstack-ansible deployment-elk.yml 215 | ``` 216 | 217 | ### Testing and Gating 218 | 219 | Please see the documentation in [rpc-gating/README.md](https://github.com/rcbops/rpc-gating/blob/master/README.md) 220 | -------------------------------------------------------------------------------- /TESTING.md: -------------------------------------------------------------------------------- 1 | # Testing for Rackspace Private Cloud Powered by OpenStack 2 | 3 | ## Overview 4 | RPCO is an integration of OpenStack-Ansible, and Monitoring (MaaS). RPCO testing 5 | is not designed to test the individual services, it is designed to test the 6 | functionality of the whole platform. 7 | 8 | The reason for this is that each service is tested by the appropriate upstream, 9 | for example the deployment of Nova is tested in OSA, and the nova service itself 10 | is tested in OpenStack. 11 | 12 | ### Build Types 13 | * AIO: Single cloud instance build of containerised RPC deployment 14 | * [MNAIO](https://github.com/openstack/openstack-ansible-ops/blob/master/multi-node-aio/README.rst): 15 | Single OnMetal instance that hosts VMs simulating multiple RPC nodes. Each vm 16 | contains multiple containers as it would on a physical deploy. 17 | 18 | #### Actions 19 | * Deploy: Deploy, Test 20 | * Upgrade: Deploy, Test, Upgrade Test 21 | * Leapfrog Upgrade: Deploy, Test, Leapfrog Upgrade, Test 22 | 23 | #### Test Types 24 | * [Tempest](https://github.com/openstack/tempest): Run a subset of OpenStack tempest tests. 25 | 26 | ## Notes 27 | * Jenkins has a useful [categorised view](https://rpc.jenkins.cit.rackspace.net/view/AIO/), 28 | this breaks down the 90+ jobs into categories making finding relevant jobs 29 | much easier. 30 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | # Copyright 2017, Rackspace US, Inc. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # Verify whether required plugins are installed. 16 | required_plugins = [ "vagrant-disksize" ] 17 | required_plugins.each do |plugin| 18 | if not Vagrant.has_plugin?(plugin) 19 | raise "The vagrant plugin #{plugin} is required. Please run `vagrant plugin install #{plugin}`" 20 | end 21 | end 22 | 23 | Vagrant.configure("2") do |config| 24 | config.vm.provider "virtualbox" do |v| 25 | v.memory = 12288 26 | v.cpus = 4 27 | end 28 | 29 | # Configure the disk size. 30 | disk_size = "100GB" 31 | 32 | config.vm.define "master" do |master| 33 | master.vm.box = "ubuntu/xenial64" 34 | master.disksize.size = disk_size 35 | master.vm.hostname = "rpco-master" 36 | config.vm.provision "shell", 37 | privileged: true, 38 | inline: <<-SHELL 39 | sudo su - 40 | apt update 41 | apt-get -y install git build-essential gcc libssl-dev libffi-dev python-dev 42 | git clone -b master --recursive https://github.com/rcbops/rpc-openstack.git /opt/rpc-openstack 43 | cd /opt/rpc-openstack/ 44 | export DEPLOY_AIO=yes 45 | ./scripts/deploy.sh 46 | SHELL 47 | end 48 | 49 | config.vm.define "ocata" do |ocata| 50 | ocata.vm.box = "ubuntu/xenial64" 51 | ocata.disksize.size = disk_size 52 | ocata.vm.hostname = "rpco-ocata" 53 | config.vm.provision "shell", 54 | privileged: true, 55 | inline: <<-SHELL 56 | sudo su - 57 | apt update 58 | apt-get -y install git build-essential gcc libssl-dev libffi-dev python-dev 59 | git clone -b ocata --recursive https://github.com/rcbops/rpc-openstack.git /opt/rpc-openstack 60 | cd /opt/rpc-openstack/ 61 | export DEPLOY_AIO=yes 62 | ./scripts/deploy.sh 63 | SHELL 64 | end 65 | 66 | config.vm.define "newton" do |newton| 67 | newton.vm.box = "ubuntu/xenial64" 68 | newton.disksize.size = disk_size 69 | newton.vm.hostname = "rpco-newton" 70 | config.vm.provision "shell", 71 | privileged: true, 72 | inline: <<-SHELL 73 | sudo su - 74 | apt update 75 | apt-get -y install git build-essential gcc libssl-dev libffi-dev python-dev 76 | git clone -b newton --recursive https://github.com/rcbops/rpc-openstack.git /opt/rpc-openstack 77 | cd /opt/rpc-openstack/ 78 | export DEPLOY_AIO=yes 79 | ./scripts/deploy.sh 80 | SHELL 81 | end 82 | end 83 | -------------------------------------------------------------------------------- /ansible-role-master-requirements.yml: -------------------------------------------------------------------------------- 1 | - name: rcbops_openstack-ops 2 | scm: git 3 | src: https://github.com/rsoprivatecloud/openstack-ops 4 | version: 1.3.2 5 | -------------------------------------------------------------------------------- /ansible-role-newton-requirements.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # RPC-Support specific role 17 | - name: rcbops_openstack-ops 18 | scm: git 19 | src: https://github.com/rsoprivatecloud/openstack-ops 20 | version: dfc2180e9bf28c8f59baca2a233db2a726b97bcb 21 | -------------------------------------------------------------------------------- /ansible-role-ocata-requirements.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # RPC-Support specific role 17 | - name: rcbops_openstack-ops 18 | scm: git 19 | src: https://github.com/rsoprivatecloud/openstack-ops 20 | version: dfc2180e9bf28c8f59baca2a233db2a726b97bcb 21 | -------------------------------------------------------------------------------- /ansible-role-pike-requirements.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # RPC-Support specific role 17 | - name: rcbops_openstack-ops 18 | scm: git 19 | src: https://github.com/rsoprivatecloud/openstack-ops 20 | version: 86c8e3c6327b5958b5eb3124455e3c12cd280e75 21 | -------------------------------------------------------------------------------- /ansible-role-queens-requirements.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # RPC-Support specific role 17 | - name: rcbops_openstack-ops 18 | scm: git 19 | src: https://github.com/rsoprivatecloud/openstack-ops 20 | version: 86c8e3c6327b5958b5eb3124455e3c12cd280e75 21 | -------------------------------------------------------------------------------- /ansible-role-rocky-requirements.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # RPC-Support specific role 17 | - name: rcbops_openstack-ops 18 | scm: git 19 | src: https://github.com/rsoprivatecloud/openstack-ops 20 | version: 1.3.2 21 | -------------------------------------------------------------------------------- /etc/openstack_deploy/env.d/fleet.yml: -------------------------------------------------------------------------------- 1 | --- 2 | component_skel: 3 | kolide-fleet: 4 | belongs_to: 5 | - fleet_all 6 | - kolide-fleet_all 7 | 8 | mariadb: 9 | belongs_to: 10 | - fleet_all 11 | - mariadb_all 12 | 13 | container_skel: 14 | kolide-fleet_container: 15 | belongs_to: 16 | - kolide_containers 17 | contains: 18 | - kolide-fleet 19 | - mariadb 20 | 21 | physical_skel: 22 | kolide_containers: 23 | belongs_to: 24 | - all_containers 25 | 26 | kolide_hosts: 27 | belongs_to: 28 | - hosts 29 | -------------------------------------------------------------------------------- /etc/openstack_deploy/group_vars/all/osa.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # Nova config overrides 17 | nova_console_type: novnc 18 | 19 | # Set RabbitMQ message replication count to 2 20 | # The desired setting would be {{ (groups.rabbitmq|length /2 +1)|round(0,'floor') |int }} 21 | # but can't be used due to https://github.com/ansible/ansible/issues/9362 22 | rabbitmq_policies: 23 | - name: "HA" 24 | pattern: '^(?!amq\.).*' 25 | tags: 26 | ha-mode: exactly 27 | ha-params: 2 28 | ha-sync-mode: automatic 29 | 30 | # Memcached overrides 31 | # https://github.com/rcbops/rpc-openstack/issues/1048 32 | # memcached_memory is calculated via https://github.com/openstack/openstack-ansible-memcached_server/blob/master/defaults/main.yml#L33 33 | # based on container memory but this determination it not necessarily correct, 34 | # since memory utilization is primarily driven by the objects placed in memcached. 35 | memcached_connections: 16384 36 | 37 | # Galera overrides 38 | galera_cluster_name: rpc_galera_cluster 39 | 40 | ## Enable Neutron l2_population 41 | # We are overriding the default value for neutron_l2_population. Please see 42 | # https://github.com/rcbops/rpc-openstack/issues/973 for further details. 43 | neutron_l2_population: True 44 | 45 | # Based on https://github.com/rcbops/rpc-openstack/issues/1149 46 | # L3HA has to be disabled until all major issues are fixed. 47 | neutron_neutron_conf_overrides: 48 | DEFAULT: 49 | l3_ha: False 50 | 51 | #Set the default for Neutron-HA-tool to true 52 | neutron_legacy_ha_tool_enabled: true 53 | 54 | # Increase interval between checks after container start. 55 | # Reduces the "Wait for container ssh" error. 56 | ssh_delay: 60 57 | 58 | # Use the correct secrets file 59 | osa_secrets_file_name: "user_osa_secrets.yml" 60 | 61 | # Disable glance image cache, default is keystone+cachemanagement 62 | glance_flavor: keystone 63 | 64 | # Octavia tuning 65 | # Keep this while there is no Barbican or Vault 66 | octavia_tls_listener_enabled: False 67 | 68 | # Octavia HA settings 69 | octavia_loadbalancer_topology: ACTIVE_STANDBY 70 | octavia_spare_amphora_pool_size: 0 71 | octavia_enable_anti_affinity: "{{ lookup('env', 'DEPLOY_AIO') != 'yes' }}" 72 | # RPC disables connection logging, so setting this back to 2GB 73 | octavia_amp_disk: 2 74 | # RPC doesn't use images from an artefact storage - so disable download 75 | octavia_download_artefact: False 76 | 77 | octavia_octavia_conf_overrides: 78 | # Production clouds should be able to boot VMs in ten minutes 79 | haproxy_amphora: 80 | connection_max_retries: 120 81 | build_active_retries: 120 82 | # RPC is disabling connection logging in the amphroa as they are not 83 | # accessible anyway. 84 | connection_logging: False 85 | keepalived_vrrp: 86 | vrrp_check_interval: 2 87 | # Wait for up to half an hour for nova to actually delete an instance and 88 | # release the port 89 | networking: 90 | port_detach_timeout: 1800 91 | task_flow: 92 | engine: parallel 93 | 94 | # Elasticsearch global variables 95 | elasticsearch_http_port: 9200 96 | elasticsearch_tcp_port: 9300 97 | 98 | # Adding elastic search and kibana to haproxy_extra_services 99 | haproxy_extra_services: 100 | - service: 101 | haproxy_service_name: elasticsearch 102 | haproxy_backend_nodes: "{{ groups['elasticsearch'] | default([]) }}" 103 | haproxy_ssl: True 104 | haproxy_port: 9201 105 | haproxy_backend_port: 9200 106 | haproxy_balance_type: http 107 | haproxy_backend_options: 108 | - "httpchk" 109 | haproxy_enabled: "{{ groups['elasticsearch'] is defined and groups['elasticsearch'] | length > 0 }}" 110 | - service: 111 | haproxy_service_name: kibana_ssl 112 | haproxy_backend_nodes: "{{ groups['kibana'] | default([]) }}" 113 | haproxy_ssl: True 114 | haproxy_port: 8443 115 | haproxy_backend_port: 81 116 | haproxy_balance_type: http 117 | haproxy_backend_options: 118 | - "httpchk" 119 | haproxy_enabled: "{{ groups['kibana'] is defined and groups['kibana'] | length > 0 }}" 120 | - service: 121 | haproxy_service_name: kolide-fleet 122 | haproxy_ssl: False 123 | haproxy_backend_nodes: "{{ groups['fleet_all'] | default([]) }}" 124 | haproxy_port: 6443 125 | haproxy_check_port: 443 126 | haproxy_backend_port: 443 127 | haproxy_balance_type: tcp 128 | haproxy_enabled: "{{ groups['fleet_all'] is defined and groups['fleet_all'] | length > 0 }}" 129 | # Force TLS-1.2 for haproxy 130 | haproxy_ssl_bind_options: "force-tlsv12" 131 | 132 | # Define the distro version globally 133 | repo_build_os_distro_version: "{{ (ansible_distribution | lower) | replace(' ', '_') }}-{{ ansible_distribution_version.split('.')[:2] | join('.') }}-{{ ansible_architecture | lower }}" 134 | 135 | # NOTE(cloudnull): Ensure that the ceph client is always present and not "latest". 136 | # This is being done to ensure that the VMs created using RBD 137 | # are not terminated on client upgrades. 138 | ceph_client_package_state: "present" 139 | -------------------------------------------------------------------------------- /etc/openstack_deploy/group_vars/all/release.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2016, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # The release tag to use 17 | # Note that this is set here so that the openstack_release override in 18 | # "osa_defaults" is able to use it. 19 | rpc_release: "undefined" 20 | 21 | # TODO(evrardjp): Move this to group_vars/all/release.yml when possible 22 | # The release tag to use for the repo and venvs 23 | # This can't be overriden because OSA is using group_vars. 24 | openstack_release: "{{ rpc_release }}" 25 | -------------------------------------------------------------------------------- /etc/openstack_deploy/group_vars/all/rpc-o.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # List of files to override using the OSA horizon role 17 | rackspace_static_files_folder: "/opt/rpc-openstack/logo_files" 18 | horizon_custom_uploads: 19 | logo-splash-old: 20 | src: "{{ rackspace_static_files_folder }}/logo-splash.png" 21 | dest: img/logo-splash.png 22 | logo-old: 23 | src: "{{ rackspace_static_files_folder }}/logo.png" 24 | dest: img/logo.png 25 | logo-splash: 26 | src: "{{ rackspace_static_files_folder }}/logo-splash.svg" 27 | dest: img/logo-splash.svg 28 | logo: 29 | src: "{{ rackspace_static_files_folder }}/logo.svg" 30 | dest: img/logo.svg 31 | favicon: 32 | src: "{{ rackspace_static_files_folder }}/favicon.ico" 33 | dest: img/favicon.ico 34 | -------------------------------------------------------------------------------- /etc/openstack_deploy/group_vars/log_hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | openstack_user_kernel_options: 17 | - key: 'vm.max_map_count' 18 | value: 262144 19 | - key: 'vm.mmap_min_addr' 20 | value: 65536 21 | - key: 'kernel.kptr_restrict' 22 | value: 1 23 | - key: 'net.ipv4.tcp_syncookies' 24 | value: 1 25 | - key: 'kernel.sysrq' 26 | value: 176 27 | - key: 'fs.protected_hardlinks' 28 | value: 1 29 | - key: 'fs.protected_symlinks' 30 | value: 1 31 | - key: 'kernel.yama.ptrace_scope' 32 | value: 1 33 | - key: 'kernel.printk' 34 | value: '4 4 1 7' 35 | -------------------------------------------------------------------------------- /etc/openstack_deploy/user_rpco_maas_variables.yml.example: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # influx_telegraf_targets: ["http://${INFLUX_IP}:${INFLUX_PORT}"] 17 | 18 | maas_release: master 19 | 20 | maas_job_reference: "${BUILD_TAG}" 21 | 22 | # Authentication 23 | maas_auth_method: password 24 | maas_tenant_id: "{{ rackspace_cloud_tenant_id | default('') }}" 25 | maas_username: "{{ rackspace_cloud_username | default('') }}" 26 | maas_api_key: "{{ rackspace_cloud_api_key | default('') }}" 27 | maas_auth_token: "{{ rackspace_cloud_auth_token | default('') }}" 28 | 29 | # Notification 30 | maas_notification_plan: npManaged 31 | 32 | # Set the maas_scheme 33 | # Note: (alextricity25) This has been changed to https because 34 | # SSL termination is done by default since Newton per https://review.openstack.org/#/c/277199/ 35 | maas_scheme: https 36 | 37 | # Set whether checks should be enabled or disabled 38 | maas_ssl_check: false 39 | maas_host_check: false 40 | maas_remote_check: true 41 | 42 | # By default we will create an agent token for each entity, however if you'd 43 | # prefer to use the same agent token for all entities then specify it here 44 | #maas_agent_token: some_token 45 | maas_target_alias: public0_v4 46 | 47 | # Enable/Disable the cinder-backup monitor 48 | maas_monitor_cinder_backup: "{{ cinder_service_backup_program_enabled | default(false) }}" 49 | 50 | # Allow all of the host machiens running maas agents to restart once at the end of a maas playbook. 51 | maas_restart_independent: false 52 | -------------------------------------------------------------------------------- /etc/openstack_deploy/user_rpco_secrets.yml.example: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2016, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | rpc_support_holland_password: 17 | kibana_password: 18 | kibana_enable_basic_auth: true 19 | 20 | fsid_uuid: 21 | maas_swift_accesscheck_password: 22 | maas_rabbitmq_password: 23 | maas_keystone_password: 24 | elasticsearch_basic_auth_password: 25 | -------------------------------------------------------------------------------- /gating/capabilities/aio_config: -------------------------------------------------------------------------------- 1 | This file exists as a marker to let Jenkins/rpc-gating know that adding config overrides for gating and 2 | scenarios will be handled in repo and doesn't need to be touched by gating scripts. 3 | 4 | 5 | Once this has been backported to all branches, the functionality will be removed from rpc-gating, then these 6 | files won't be needed. 7 | -------------------------------------------------------------------------------- /gating/check/collect_logs.yml: -------------------------------------------------------------------------------- 1 | - hosts: "{{ target_hosts }}" 2 | gather_facts: false 3 | tasks: 4 | - name: Ensure artefacts directories exists 5 | file: 6 | path: "{{ item }}" 7 | state: directory 8 | with_items: 9 | - "{{ artifacts_dir }}/{{ inventory_hostname }}/host" 10 | - "{{ artifacts_dir }}/{{ inventory_hostname }}/containers" 11 | delegate_to: "localhost" 12 | 13 | - name: Generate ARA reports 14 | shell: | 15 | ARA_CMD="/opt/ansible-runtime/bin/ara" 16 | if [[ -e ${ARA_CMD} ]]; then 17 | ARA_REPORT_PATH="/openstack/log/ara-report" 18 | mkdir -p ${ARA_REPORT_PATH} 19 | ${ARA_CMD} generate html ${ARA_REPORT_PATH} 20 | ${ARA_CMD} generate junit ${ARA_REPORT_PATH}/deployment_results.xml 21 | cp "${HOME}/.ara/ansible.sqlite" ${ARA_REPORT_PATH}/ 22 | fi 23 | args: 24 | executable: /bin/bash 25 | # We never want a failure of this task to cause a 26 | # failure in the playbook, so we ignore errors to 27 | # ensure that the task shows failure, but does not 28 | # fail the playbook. 29 | ignore_errors: yes 30 | # This needs to run wherever the ansible runtime is. 31 | # For AIO deployments, the deploy host is localhost. 32 | # For MNAIO deployments, the deploy host is infra1. 33 | when: 34 | - (inventory_hostname == 'localhost') or 35 | (inventory_hostname == 'infra1') 36 | 37 | - name: Collect journald logs 38 | command: > 39 | journalctl 40 | --output json 41 | --no-pager 42 | register: host_journald_logs 43 | ignore_errors: true 44 | tags: 45 | - skip_ansible_lint 46 | 47 | - name: Write journald logs to a file 48 | copy: 49 | content: "{{ host_journald_logs.stdout }}" 50 | dest: /var/log/{{ inventory_hostname }}-journald.log.json 51 | ignore_errors: true 52 | tags: 53 | - skip_ansible_lint 54 | 55 | - name: Grab host data 56 | command: > 57 | rsync 58 | --archive 59 | --compress 60 | --verbose 61 | --relative 62 | --rsh 'ssh -o StrictHostKeyChecking=no' 63 | --ignore-missing-args 64 | --safe-links 65 | --no-perms 66 | --no-owner 67 | --no-group 68 | {{ inventory_hostname }}:{{ item }} 69 | {{ artifacts_dir }}/{{ inventory_hostname }}/host 70 | with_items: 71 | - "/openstack/log" 72 | - "/etc" 73 | - "/var/./log" 74 | delegate_to: "localhost" 75 | ignore_errors: true 76 | tags: 77 | - skip_ansible_lint 78 | 79 | - name: List containers 80 | command: "lxc-ls -1" 81 | failed_when: 82 | - containers.rc != 0 83 | - containers.msg != '[Errno 2] No such file or directory' 84 | changed_when: false 85 | register: containers 86 | 87 | - name: Get container PIDs 88 | command: "lxc-info --name {{ item }} --no-humanize --pid" 89 | with_items: 90 | - "{{ containers.stdout_lines | default([]) }}" 91 | register: container_pids 92 | 93 | - name: Grab container data 94 | command: > 95 | rsync 96 | --archive 97 | --compress 98 | --verbose 99 | --rsh 'ssh -o StrictHostKeyChecking=no' 100 | --ignore-missing-args 101 | --safe-links 102 | --no-perms 103 | --no-owner 104 | --no-group 105 | {{ inventory_hostname }}:/proc/{{ item[0].stdout }}/root/{{ item[1] }} 106 | {{ artifacts_dir }}/{{ inventory_hostname }}/containers/{{ item[0].item }} 107 | when: 108 | - containers.rc == 0 109 | - item[0].stdout != "" 110 | with_nested: 111 | - "{{ container_pids.results }}" 112 | - 113 | - "etc" 114 | - "var/log" 115 | delegate_to: "localhost" 116 | tags: 117 | - skip_ansible_lint 118 | vars: 119 | artifacts_dir: "/tmp/artifacts" 120 | target_hosts: "localhost" 121 | 122 | - hosts: localhost 123 | gather_facts: false 124 | tasks: 125 | - name: Ensure result directory exists 126 | file: 127 | path: "{{ result_dir }}" 128 | state: directory 129 | 130 | - name: Find tempest results file 131 | find: 132 | paths: "{{ artifacts_dir }}" 133 | recurse: yes 134 | patterns: "deployment_results.xml,tempest_results.xml,test_list.txt" 135 | register: results_files 136 | 137 | - name: Copy tempest results to RE_HOOK_RESULT_DIR 138 | copy: 139 | src: "{{ item.path }}" 140 | dest: "{{ result_dir }}/" 141 | with_items: "{{ results_files.files }}" 142 | when: results_files.matched > 0 143 | vars: 144 | result_dir: "/tmp/result" 145 | -------------------------------------------------------------------------------- /gating/check/post: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ## Shell Opts ---------------------------------------------------------------- 4 | 5 | set -eux 6 | set -o pipefail 7 | 8 | ## Vars ---------------------------------------------------------------------- 9 | 10 | source "$(readlink -f $(dirname ${0}))/../gating_vars.sh" 11 | 12 | ## Main ---------------------------------------------------------------------- 13 | 14 | if [ ${RE_JOB_ACTION} != "tox-test" ]; then 15 | bash -c "$(readlink -f $(dirname ${0})/post_deploy.sh)" 16 | fi 17 | 18 | if [[ ${RE_JOB_ACTION} == system* ]]; then 19 | bash -c "$(readlink -f $(dirname ${0})/post_send_junit_to_qtest.sh)" 20 | fi 21 | -------------------------------------------------------------------------------- /gating/check/post_deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ## Shell Opts ---------------------------------------------------------------- 4 | 5 | set -eux 6 | set -o pipefail 7 | 8 | ## Vars ---------------------------------------------------------------------- 9 | 10 | # These vars are set by the CI environment, but are given defaults 11 | # here to cater for situations where someone is executing the test 12 | # outside of the CI environment. 13 | export RE_HOOK_ARTIFACT_DIR="${RE_HOOK_ARTIFACT_DIR:-/tmp/artifacts}" 14 | export RE_HOOK_RESULT_DIR="${RE_HOOK_RESULT_DIR:-/tmp/results}" 15 | 16 | ## Functions ----------------------------------------------------------------- 17 | source $(dirname ${0})/../../scripts/functions.sh 18 | 19 | ## Main ---------------------------------------------------------------------- 20 | 21 | echo "Killing dstat process to ensure the data file is flushed for log collection." 22 | if [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]]; then 23 | ansible -m shell -a 'kill $(pgrep -f dstat) || true' pxe_servers || true 24 | else 25 | kill $(pgrep -f dstat) || true 26 | fi 27 | 28 | echo "#### BEGIN LOG COLLECTION ###" 29 | 30 | collect_logs_cmd="ansible-playbook" 31 | collect_logs_cmd+=" --ssh-common-args='-o StrictHostKeyChecking=no'" 32 | collect_logs_cmd+=" --extra-vars='artifacts_dir=${RE_HOOK_ARTIFACT_DIR}'" 33 | collect_logs_cmd+=" --extra-vars='result_dir=${RE_HOOK_RESULT_DIR}'" 34 | 35 | if [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]]; then 36 | collect_logs_cmd+=" --extra-vars='target_hosts=pxe_servers:localhost'" 37 | collect_logs_cmd+=" --inventory='/opt/openstack-ansible-ops/multi-node-aio/playbooks/inventory/hosts'" 38 | else 39 | collect_logs_cmd+=" --extra-vars='target_hosts=localhost'" 40 | fi 41 | 42 | collect_logs_cmd+=" $(dirname ${0})/collect_logs.yml" 43 | 44 | eval $collect_logs_cmd || true 45 | 46 | echo "#### END LOG COLLECTION ###" 47 | 48 | echo "#### BEGIN DSTAT CHART GENERATION ###" 49 | # RE-1501 50 | # Generate the dstat charts. 51 | # Unfortunately using the OSA function does not work, and 52 | # it spits out a bunch of junk we don't want either. 53 | # We therefore replicate the content of it here instead. 54 | dstat_files=$(find ${RE_HOOK_ARTIFACT_DIR} -name 'dstat.csv') 55 | if [ -n "${dstat_files}" ]; then 56 | if [[ ! -d /opt/dstat_graph ]]; then 57 | git clone https://github.com/Dabz/dstat_graph /opt/dstat_graph 58 | fi 59 | pushd /opt/dstat_graph 60 | for dstat_file in ${dstat_files}; do 61 | /usr/bin/env bash -e ./generate_page.sh ${dstat_file} > ${dstat_file%.csv}.html 62 | done 63 | popd 64 | else 65 | echo "No dstat.csv source files found. Skipping report generation." 66 | fi 67 | echo "#### END DSTAT CHART GENERATION ###" 68 | 69 | # Only enable snapshot when triggered by a commit push. 70 | # This is to enable image updates whenever a PR is merged, but not before 71 | if [[ "${RE_JOB_TRIGGER:-USER}" == "PUSH" ]] && [[ ${RE_JOB_STATUS:-SUCCESS} == "SUCCESS" ]]; then 72 | echo "### BEGIN SNAPSHOT PREP ###" 73 | mkdir -p /gating/thaw 74 | 75 | # /opt/rpc-openstack may be a symlink to $WORKSPACE/rpc-openstack 76 | # However $WORKSPACE will be wiped prior to the snapshot being taken 77 | # so will not be present in instances created from the snapshot. 78 | # In order to ensure /opt/rpc-openstack is present in the snapshot 79 | # We check if its a link if it is, unlink it and create a full copy. 80 | pushd /opt 81 | target="$(readlink -f rpc-openstack)" 82 | if [[ -n "$target" ]]; then 83 | echo "/opt/rpc-openstack is linked into $WORKSPACE/rpc-openstack which" 84 | echo "will be wiped prior to creating a snapshot." 85 | echo "Removing symlink and copying $WORKSPACE/rpc-openstack to /opt." 86 | rm rpc-openstack 87 | cp -ar $target rpc-openstack 88 | fi 89 | 90 | # /root/.ssh is cleared on thaw. We need to store the keys that were there. 91 | mkdir -p /opt/root_ssh_backup 92 | cp -vr /root/.ssh /opt/root_ssh_backup 93 | 94 | ln -s /opt/rpc-openstack/gating/thaw/run /gating/thaw/run 95 | 96 | echo "rpc-${RPC_RELEASE}-${RE_JOB_IMAGE}-${RE_JOB_SCENARIO}" > /gating/thaw/image_name 97 | echo "### END SNAPSHOT PREP ###" 98 | fi 99 | 100 | if [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]] && [[ ${RE_JOB_ACTION} == "deploy" ]] && [[ "${RE_JOB_STATUS:-SUCCESS}" == "SUCCESS" ]]; then 101 | echo "Preparing machine image artifacts." 102 | pushd /opt/openstack-ansible-ops/multi-node-aio 103 | ansible-playbook -vv -i ${MNAIO_INVENTORY:-"playbooks/inventory"} playbooks/save-vms.yml 104 | popd 105 | 106 | echo "Deleting unnecessary image files to prevent artifacting them." 107 | find /data/images -name \*.img -not -name \*-base.img -delete 108 | 109 | echo "Moving files to named folder for artifact upload." 110 | mv /data/images /data/${RPC_RELEASE}-${RE_JOB_IMAGE}-${RE_JOB_SCENARIO} 111 | 112 | echo "Preparing machine image artifact upload configuration." 113 | cat > component_metadata.yml < 26 | virtualenv --no-wheel "${VENV_PATH}" 27 | 28 | # Activate virtualenv 29 | source "${VENV_PATH}/bin/activate" 30 | 31 | VENV_PIP="${VENV_PATH}/bin/pip" 32 | 33 | # Install zigzag from PyPI 34 | ${VENV_PIP} install rpc-zigzag 35 | 36 | # Search for xml files in RE_HOOK_RESULT_DIR 37 | xml_files=() 38 | while IFS= read -r -d $'\0' FILE; do 39 | xml_files+=("${FILE}") 40 | done < <(find ${RE_HOOK_RESULT_DIR} -type f -name 'molecule_test_results.xml' -print0) 41 | 42 | # Upload the files 43 | echo "Attempting upload of ${#xml_files[@]} XML files" 44 | printf '%s\n' "${xml_files[@]}" 45 | for i in "${xml_files[@]}"; do 46 | # Use to process and upload to qtest 47 | if zigzag $i ${PROJECT_ID}; then 48 | echo "Upload Success: $i" 49 | else 50 | echo "Upload Failure: $i" 51 | fi 52 | done 53 | 54 | # exit is hardcoded since xml from tempest is known to to fail 55 | exit 0 56 | -------------------------------------------------------------------------------- /gating/check/pre: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ## Shell Opts ---------------------------------------------------------------- 4 | 5 | set -eux 6 | set -o pipefail 7 | 8 | ## Vars ---------------------------------------------------------------------- 9 | 10 | source "$(readlink -f $(dirname ${0}))/../gating_vars.sh" 11 | 12 | ## Main ---------------------------------------------------------------------- 13 | 14 | # If the current folder's basename is rpc-openstack then we assume 15 | # that it is the root of the git clone. If the git clone is not in 16 | # /opt then we symlink the current folder there so that all the 17 | # rpc-openstack scripts work as expected. 18 | if [[ "$(basename ${PWD})" == "rpc-openstack" ]]; then 19 | if [[ "${PWD}" != "/opt/rpc-openstack" ]]; then 20 | ln -sfn ${PWD} /opt/rpc-openstack 21 | fi 22 | fi 23 | 24 | # We need to ensure that we use the rackspace mirrors, as they are 25 | # most reliable. We also need to ensure that python and the python 26 | # yaml library are present for ansible to work. 27 | source "$(readlink -f $(dirname ${0}))/../gating_prerequisites.sh" 28 | 29 | # Install pre-requisites for the different types of tests 30 | distro_pkgs_to_install="" 31 | py_pkgs_to_install="" 32 | 33 | if [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]]; then 34 | dpkg-query --list | grep curl &>/dev/null || distro_pkgs_to_install+="curl " 35 | fi 36 | 37 | if [[ ${RE_JOB_ACTION} == "tox-test" ]]; then 38 | dpkg-query --list | grep wget &>/dev/null || distro_pkgs_to_install+="wget " 39 | pip freeze | grep tox &>/dev/null || py_pkgs_to_install+="tox " 40 | fi 41 | 42 | if [ "${distro_pkgs_to_install}" != "" ]; then 43 | apt-get install -y ${distro_pkgs_to_install} 44 | fi 45 | 46 | if [ "${py_pkgs_to_install}" != "" ]; then 47 | pip install --disable-pip-version-check --upgrade --upgrade-strategy only-if-needed ${py_pkgs_to_install} -c upper-constraints.txt 48 | fi 49 | 50 | if [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]]; then 51 | source "$(readlink -f $(dirname ${0})/pre_deploy_mnaio.sh)" 52 | fi 53 | -------------------------------------------------------------------------------- /gating/check/pre_deploy_mnaio.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright 2017, Rackspace US, Inc. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | ## Shell Opts ---------------------------------------------------------------- 18 | 19 | set -exu 20 | 21 | echo "Preparing a Multi Node AIO (MNAIO)" 22 | 23 | ## Vars and Functions -------------------------------------------------------- 24 | 25 | source "$(readlink -f $(dirname ${0}))/../gating_vars.sh" 26 | 27 | source /opt/rpc-openstack/scripts/functions.sh 28 | 29 | source "$(readlink -f $(dirname ${0}))/../mnaio_vars.sh" 30 | 31 | ## Main ---------------------------------------------------------------------- 32 | 33 | # checkout openstack-ansible-ops 34 | if [ ! -d "/opt/openstack-ansible-ops" ]; then 35 | git clone --recursive https://github.com/openstack/openstack-ansible-ops /opt/openstack-ansible-ops 36 | else 37 | pushd /opt/openstack-ansible-ops 38 | git fetch --all 39 | popd 40 | fi 41 | 42 | # build the multi node aio 43 | pushd /opt/openstack-ansible-ops/multi-node-aio 44 | # Prepare the multi node aio host 45 | # Note (odyssey4me): 46 | # We cannot use build.sh here because the playbook imports 47 | # result in running all plays with all tasks having a conditional 48 | # set on them, and because the VM's aren't running at this stage 49 | # the plays fail. This is a workaround until there are upstream 50 | # improvements to counteract that issue. 51 | source bootstrap.sh 52 | source ansible-env.rc 53 | [[ "${SETUP_HOST}" == "true" ]] && run_mnaio_playbook playbooks/setup-host.yml 54 | [[ "${SETUP_PXEBOOT}" == "true" ]] && run_mnaio_playbook playbooks/deploy-acng.yml 55 | [[ "${SETUP_PXEBOOT}" == "true" ]] && run_mnaio_playbook playbooks/deploy-pxe.yml 56 | [[ "${SETUP_DHCPD}" == "true" ]] && run_mnaio_playbook playbooks/deploy-dhcp.yml 57 | [[ "${DEPLOY_VMS}" == "true" ]] && run_mnaio_playbook playbooks/deploy-vms.yml 58 | 59 | # If there are images available, and this is not the 'deploy' action, 60 | # then we need to download the images, then create the VM's. The conditional 61 | # was already evaluated in mnaio_vars, so we key off DEPLOY_VMS here. 62 | if [[ "${DEPLOY_VMS}" == "false" ]]; then 63 | run_mnaio_playbook playbooks/download-vms.yml -e manifest_url=${RPCO_IMAGE_MANIFEST_URL} -e aria2c_log_path=${RE_HOOK_ARTIFACT_DIR} 64 | run_mnaio_playbook playbooks/deploy-vms.yml 65 | fi 66 | popd 67 | 68 | echo "Multi Node AIO preparation completed..." 69 | -------------------------------------------------------------------------------- /gating/check/run: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ## Shell Opts ---------------------------------------------------------------- 4 | 5 | set -eux 6 | set -o pipefail 7 | 8 | ## Display environment 9 | echo "+-------------------- ENV VARS --------------------+" 10 | env 11 | echo "+-------------------- ENV VARS --------------------+" 12 | 13 | ## Vars ---------------------------------------------------------------------- 14 | 15 | export DEPLOY_AIO="true" 16 | 17 | source "$(readlink -f $(dirname ${0}))/../gating_vars.sh" 18 | 19 | ## Main ---------------------------------------------------------------------- 20 | 21 | echo "rpc-openstack at SHA $(git rev-parse HEAD)" 22 | 23 | if [[ ${RE_JOB_ACTION} == "tox-test" ]]; then 24 | 25 | tmp_tox_dir=$(mktemp -d) 26 | tox -e ${RE_JOB_SCENARIO} --workdir $tmp_tox_dir 27 | 28 | elif [[ ${RE_JOB_IMAGE} =~ .*mnaio.* ]]; then 29 | bash -c "$(readlink -f $(dirname ${0})/run_deploy_mnaio.sh)" 30 | else 31 | bash -c "$(readlink -f $(dirname ${0})/../../scripts/deploy.sh)" 32 | 33 | # Enable the debug stdout callback to ensure that the console 34 | # output is easily legible 35 | export ANSIBLE_STDOUT_CALLBACK=debug 36 | # Verify that there are no haproxy backends down 37 | cd /opt/openstack-ansible/playbooks/ 38 | openstack-ansible -v /opt/rpc-openstack/gating/thaw/haproxycheck.yml 39 | fi 40 | 41 | if [[ ${RE_JOB_ACTION} == system* ]]; then 42 | bash -c "$(readlink -f $(dirname ${0})/run_system_tests.sh)" 43 | fi 44 | 45 | if [[ ${RE_JOB_ACTION} == "sdqc" ]]; then 46 | export RE_JOB_BRANCH="openstack-ops-only" 47 | bash -c "$(readlink -f $(dirname ${0})/run_system_tests.sh)" 48 | fi 49 | 50 | if [[ ${RE_JOB_ACTION} == "elk" ]]; then 51 | bash -c "$(readlink -f $(dirname ${0})/run_elk_tests.sh)" 52 | fi 53 | -------------------------------------------------------------------------------- /gating/check/run_deploy_mnaio.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright 2017, Rackspace US, Inc. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | ## Shell Opts ---------------------------------------------------------------- 18 | 19 | set -exu 20 | 21 | echo "Installing RPC-O on a Multi Node AIO (MNAIO)" 22 | 23 | ## Vars and Functions -------------------------------------------------------- 24 | 25 | source "$(readlink -f $(dirname ${0}))/../gating_vars.sh" 26 | 27 | source /opt/rpc-openstack/scripts/functions.sh 28 | 29 | source "$(readlink -f $(dirname ${0}))/../mnaio_vars.sh" 30 | 31 | # If there is no set of images available yet, or this is the deploy 32 | # action, then we need to build from scratch. 33 | # We need to run the OSA deploy playbook for some pre-configuration 34 | # before doing the RPC-O bits. The conditional was already evaluated in 35 | # mnaio_vars, so we key off DEPLOY_VMS here. 36 | if [[ "${DEPLOY_VMS}" == "true" ]]; then 37 | # apply various modifications for mnaio 38 | pushd /opt/openstack-ansible-ops/multi-node-aio 39 | # By default the MNAIO deploys metering services, so we override 40 | # osa_enable_meter to prevent those services from being deployed. 41 | # TODO(odyssey4me): 42 | # Remove this once https://review.openstack.org/604034 has merged. 43 | sed -i 's/osa_enable_meter: true/osa_enable_meter: false/' playbooks/group_vars/all.yml 44 | 45 | export DEPLOY_OSA="true" 46 | export PRE_CONFIG_OSA="true" 47 | 48 | # Run the initial deployment configuration 49 | run_mnaio_playbook playbooks/deploy-osa.yml 50 | popd 51 | else 52 | # If this is not a deploy test, and the images were used in 53 | # the 'pre' stage to setup the VM's, then we do not need to 54 | # execute any more of this script. 55 | # We implement a simple 5 minute wait to give time for the 56 | # containers to start in the VM's. 57 | echo "MNAIO RPC-O deploy completed using images... waiting 5 mins for system startup." 58 | sleep 300 59 | exit 0 60 | fi 61 | 62 | ## Main -------------------------------------------------------------------- 63 | 64 | echo "Multi Node AIO setup completed..." 65 | 66 | # capture all RE_ variables for push to infra1 67 | > /opt/rpc-openstack/RE_ENV 68 | env | grep RE_ | while read -r match; do 69 | varName=$(echo ${match} | cut -d= -f1) 70 | echo "export ${varName}='${!varName}'" >> /opt/rpc-openstack/RE_ENV 71 | done 72 | 73 | # generate infra1 deploy script 74 | cat > /opt/rpc-openstack/deploy-infra1.sh </etc/apt/sources.list 24 | deb ${DISTRO_MIRROR} ${DISTRO_RELEASE} ${DISTRO_COMPONENTS//,/ } 25 | deb ${DISTRO_MIRROR} ${DISTRO_RELEASE}-updates ${DISTRO_COMPONENTS//,/ } 26 | deb ${DISTRO_MIRROR} ${DISTRO_RELEASE}-backports ${DISTRO_COMPONENTS//,/ } 27 | deb ${DISTRO_MIRROR} ${DISTRO_RELEASE}-security ${DISTRO_COMPONENTS//,/ } 28 | EOF 29 | 30 | # Add apt debug configuration 31 | echo 'Debug::Acquire::http "true";' > /etc/apt/apt.conf.d/99debug 32 | 33 | # Ensure package installs are in headless mode 34 | export DEBIAN_FRONTEND=noninteractive 35 | 36 | # Update the apt cache 37 | apt-get update 38 | 39 | # Install pre-requisites 40 | pkgs_to_install="" 41 | dpkg-query --list | grep python-minimal &>/dev/null || pkgs_to_install+="python-minimal " 42 | dpkg-query --list | grep python-yaml &>/dev/null || pkgs_to_install+="python-yaml " 43 | if [ "${pkgs_to_install}" != "" ]; then 44 | apt-get install -y ${pkgs_to_install} 45 | fi 46 | -------------------------------------------------------------------------------- /gating/gating_vars.sh: -------------------------------------------------------------------------------- 1 | # These vars are set by the CI environment, but are given defaults 2 | # here to cater for situations where someone is executing the test 3 | # outside of the CI environment. 4 | export RE_JOB_IMAGE="${RE_JOB_IMAGE:-xenial_no_artifacts}" 5 | export RE_JOB_SCENARIO="${RE_JOB_SCENARIO:-swift}" 6 | export RE_JOB_ACTION="${RE_JOB_ACTION:-deploy}" 7 | export RE_JOB_FLAVOR="${RE_JOB_FLAVOR:-}" 8 | export RE_JOB_TRIGGER="${RE_JOB_TRIGGER:-USER}" 9 | export RE_HOOK_ARTIFACT_DIR="${RE_HOOK_ARTIFACT_DIR:-/tmp/artifacts}" 10 | export RE_HOOK_RESULT_DIR="${RE_HOOK_RESULT_DIR:-/tmp/results}" 11 | export RE_JOB_NAME="${RE_JOB_NAME:-${RE_JOB_TRIGGER}_rpc-openstack-master-${RE_JOB_IMAGE}_no_artifacts-${RE_JOB_SCENARIO}-${RE_JOB_ACTION}}" 12 | export RE_JOB_PROJECT_NAME="${RE_JOB_PROJECT_NAME:-}" 13 | 14 | # OSA Tests SHA 15 | # These variables pin the SHA for the OSA Testing repository 16 | export OSA_TEST_RELEASE=${OSA_TEST_RELEASE:-d90acf00b639496cd0669153534fe5588875f3ee} 17 | export OSA_UPPER_CONSTRAINTS=${OSA_UPPER_CONSTRAINTS:-377fde64ac16dc94da2e29e16a4102adcc081a6e} 18 | -------------------------------------------------------------------------------- /gating/generate_release_notes/generate_commit_diff_notes.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -xe 2 | 3 | # Set the base RPC-O directory which the functions use 4 | export BASE_DIR="${PWD}" 5 | 6 | # Source the functions 7 | source "${BASE_DIR}/scripts/functions.sh" 8 | 9 | 10 | if echo ${PREVIOUS_TAG} | egrep '^r[0-9]+\.'; then 11 | PREVIOUS_RPC_PRODUCT_RELEASE="$(component release --component-name rpc-openstack get --version ${PREVIOUS_TAG} | grep 'series:' | awk '{print $2}')" 12 | else 13 | # Assume PREVIOUS_TAG is one of the release branches 14 | PREVIOUS_RPC_PRODUCT_RELEASE="${PREVIOUS_TAG}" 15 | fi 16 | 17 | rpc-differ \ 18 | --debug \ 19 | -rpoc ${PREVIOUS_RPC_PRODUCT_RELEASE} \ 20 | -rroc "ansible-role-${PREVIOUS_RPC_PRODUCT_RELEASE}-requirements.yml" \ 21 | -rp ${RPC_PRODUCT_RELEASE} \ 22 | -rr "ansible-role-${RPC_PRODUCT_RELEASE}-requirements.yml" \ 23 | -r "$REPO_URL" \ 24 | --update \ 25 | "$PREVIOUS_TAG" \ 26 | "$NEW_TAG" \ 27 | --file diff_notes.rst 28 | pandoc --from rst --to markdown_github < diff_notes.rst > diff_notes.md 29 | -------------------------------------------------------------------------------- /gating/generate_release_notes/generate_release_notes.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -xe 2 | 3 | gating/generate_release_notes/generate_commit_diff_notes.sh 4 | gating/generate_release_notes/generate_reno_report.sh $NEW_TAG reno_report.md 5 | cat reno_report.md diff_notes.md > all_notes.md 6 | -------------------------------------------------------------------------------- /gating/generate_release_notes/generate_reno_report.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | release=${1} 4 | out_file=${2} 5 | err_file=${out_file}.err 6 | rst_file=${out_file}.rst 7 | 8 | reno report --branch ${release} --version ${release} --no-show-source --output ${rst_file} &> ${err_file} 9 | return_code=$? 10 | cat $err_file 11 | 12 | if [[ ${return_code} != 0 ]]; then 13 | if grep -q "KeyError: '${release}'" ${err_file}; then 14 | cat > ${out_file} << EOF 15 | Release Notes 16 | ============= 17 | 18 | ${release} 19 | ------------- 20 | 21 | ### No release notes 22 | 23 | EOF 24 | return_code=0 25 | echo "Warning: No new Reno release notes found, this can indicate an issue with the tag." 26 | else 27 | echo "Failure: Reno failed to generate the report." 28 | fi 29 | else 30 | pandoc --from rst --to markdown_github < ${rst_file} > ${out_file} 31 | echo "Success: New Reno release notes found." 32 | fi 33 | 34 | exit ${return_code} 35 | -------------------------------------------------------------------------------- /gating/generate_release_notes/run: -------------------------------------------------------------------------------- 1 | #!/bin/bash -xe 2 | 3 | # This script is called by Jenkins to generate release notes. 4 | 5 | # The resulting notes must be written to $WORKSPACE/artifacts/release_notes 6 | 7 | export PREVIOUS_TAG=${RE_HOOK_PREVIOUS_VERSION} 8 | export NEW_TAG=${RE_HOOK_VERSION} 9 | export REPO_URL=${RE_HOOK_REPO_HTTP_URL} 10 | 11 | apt-get install -y pandoc python3-dev python-dev 12 | 13 | pip install dulwich -c upper-constraints.txt 14 | pip install -r test-requirements.txt -c upper-constraints.txt 15 | pip3 install 'rpc_component==0.0.2' 16 | 17 | gating/generate_release_notes/generate_release_notes.sh 18 | 19 | cp all_notes.md "${RE_HOOK_RELEASE_NOTES}" 20 | -------------------------------------------------------------------------------- /gating/mnaio_vars.sh: -------------------------------------------------------------------------------- 1 | # 2 | # Variable to use for preparation 3 | # 4 | 5 | # the base URL to download image artifacts from 6 | export RPCO_ARTIFACT_URL="https://a5ce27333a8948d82738-b28e2b85e22a27f072118ea786afca3a.ssl.cf5.rackcdn.com/" 7 | export RPCO_IMAGE_MANIFEST_URL="${RPCO_ARTIFACT_URL}/${RPC_RELEASE}-${RE_JOB_IMAGE}-${RE_JOB_SCENARIO}/manifest.json" 8 | 9 | # Check whether there is a manifest file available 10 | if curl --fail --silent --show-error --retry 5 --output /dev/null ${RPCO_IMAGE_MANIFEST_URL}; then 11 | export RPCO_IMAGES_AVAILABLE="true" 12 | else 13 | export RPCO_IMAGES_AVAILABLE="false" 14 | fi 15 | 16 | # 17 | # Environment settings 18 | # 19 | if [[ ${RE_JOB_IMAGE} =~ .*xenial.* ]]; then 20 | export DEFAULT_IMAGE="ubuntu-16.04-amd64" 21 | elif [[ ${RE_JOB_IMAGE} =~ .*bionic.* ]]; then 22 | export DEFAULT_IMAGE="ubuntu-18.04-amd64" 23 | fi 24 | export DEFAULT_MIRROR_HOSTNAME=mirror.rackspace.com 25 | export DEFAULT_MIRROR_DIR=/ubuntu 26 | export INFRA_VM_SERVER_RAM=16384 27 | export MNAIO_ANSIBLE_PARAMETERS="-e default_vm_disk_mode=file" 28 | export OSA_BRANCH="${OSA_RELEASE:-master}" 29 | 30 | # TODO(odyssey4me): 31 | # This can be removed once https://review.openstack.org/604059 32 | # merges. 33 | export DNS_NAMESERVER="8.8.8.8" 34 | 35 | # 36 | # Steps to execute when running build.sh 37 | # 38 | export SETUP_HOST="true" 39 | export CONFIG_PREROUTING="false" 40 | export SETUP_PXEBOOT="true" 41 | export SETUP_DHCPD="true" 42 | export DEPLOY_VMS="${DEPLOY_VMS:-false}" 43 | export PRE_CONFIG_OSA="false" 44 | export DEPLOY_OSA="false" 45 | export RUN_OSA="false" 46 | export CONFIGURE_OPENSTACK="false" 47 | 48 | # If there is no set of images available yet, or this 49 | # is the deploy action, then we need to build from scratch. 50 | if [[ "${RPCO_IMAGES_AVAILABLE}" == "false" ]] || [[ "${RE_JOB_ACTION}" == "deploy" ]]; then 51 | export DEPLOY_VMS="true" 52 | fi 53 | 54 | # 55 | # Non-MNAIO RPC Specific settings 56 | # 57 | export RPC_BRANCH="${RE_JOB_BRANCH}" 58 | export DEPLOY_MAAS=false 59 | 60 | # ssh command used to execute tests on infra1 61 | export MNAIO_SSH="ssh -ttt -oStrictHostKeyChecking=no root@infra1" 62 | 63 | # Function to execute playbooks with all the right vars 64 | function run_mnaio_playbook() { 65 | ansible-playbook \ 66 | -i ${MNAIO_INVENTORY:-"playbooks/inventory"} \ 67 | -e setup_host=${SETUP_HOST:-"true"} \ 68 | -e setup_pxeboot=${SETUP_PXEBOOT:-"true"} \ 69 | -e setup_dhcpd=${SETUP_DHCPD:-"true"} \ 70 | -e deploy_vms=${DEPLOY_VMS:-"true"} \ 71 | -e deploy_osa=${DEPLOY_OSA:-"true"} \ 72 | -e deploy_elk=${DEPLOY_ELK:-"false"} \ 73 | -e osa_repo=${OSA_REPO:-"https://git.openstack.org/openstack/openstack-ansible"} \ 74 | -e os_ops_repo=${OS_OPS_REPO:-"https://git.openstack.org/openstack/openstack-ansible-ops"} \ 75 | -e osa_branch=${OSA_BRANCH:-"master"} \ 76 | -e os_ops_branch=${OS_OPS_BRANCH:-"master"} \ 77 | -e default_network=${DEFAULT_NETWORK:-"eth0"} \ 78 | -e default_image=${DEFAULT_IMAGE:-"ubuntu-16.04-amd64"} \ 79 | -e vm_disk_size=${VM_DISK_SIZE:-92160} \ 80 | -e http_proxy=${http_proxy:-''} \ 81 | -e run_osa=${RUN_OSA:-"true"} \ 82 | -e run_elk=${RUN_ELK:-"false"} \ 83 | -e pre_config_osa=${PRE_CONFIG_OSA:-"true"} \ 84 | -e configure_openstack=${CONFIGURE_OPENSTACK:-"true"} \ 85 | -e config_prerouting=${CONFIG_PREROUTING:-"false"} \ 86 | -e default_ubuntu_kernel=${DEFAULT_KERNEL:-"linux-image-generic"} \ 87 | -e default_ubuntu_mirror_hostname=${DEFAULT_MIRROR_HOSTNAME:-"archive.ubuntu.com"} \ 88 | -e default_ubuntu_mirror_directory=${DEFAULT_MIRROR_DIR:-"/ubuntu"} \ 89 | -e cinder_vm_server_ram=${CINDER_VM_SERVER_RAM:-"2048"} \ 90 | -e compute_vm_server_ram=${COMPUTE_VM_SERVER_RAM:-"8196"} \ 91 | -e infra_vm_server_ram=${INFRA_VM_SERVER_RAM:-"8196"} \ 92 | -e loadbalancer_vm_server_ram=${LOADBALANCER_VM_SERVER_RAM:-"2048"} \ 93 | -e logging_vm_server_ram=${LOGGING_VM_SERVER_RAM:-"16384"} \ 94 | -e swift_vm_server_ram=${SWIFT_VM_SERVER_RAM:-"2048"} \ 95 | -e container_tech=${CONTAINER_TECH:-"lxc"} \ 96 | -e ipxe_kernel_base_url=${IPXE_KERNEL_BASE_URL:-"http://boot.ipxe.org"} \ 97 | -e ipxe_path_url=${IPXE_PATH_URL:-""} ${MNAIO_ANSIBLE_PARAMETERS} \ 98 | --force-handlers \ 99 | --flush-cache \ 100 | $@ 101 | } 102 | -------------------------------------------------------------------------------- /gating/periodic: -------------------------------------------------------------------------------- 1 | check -------------------------------------------------------------------------------- /gating/release: -------------------------------------------------------------------------------- 1 | check -------------------------------------------------------------------------------- /gating/thaw/fix_endpoints.yml: -------------------------------------------------------------------------------- 1 | - hosts: utility_all[0] 2 | user: root 3 | tasks: 4 | - name: Get a list of public endpoints 5 | shell: | 6 | . {{ ansible_env.HOME }}/openrc 7 | openstack endpoint list --interface public --column ID --column URL -f json 8 | args: 9 | executable: /bin/bash 10 | register: public_endpoints 11 | - name: Update all public endpoints w/ new public IP 12 | shell: | 13 | . {{ ansible_env.HOME }}/openrc 14 | openstack endpoint set --url '{{ item.URL | replace(orig_ip, hostvars['localhost']['ansible_default_ipv4']['address']) }}' {{ item.ID }} 15 | args: 16 | executable: /bin/bash 17 | with_items: "{{ public_endpoints.stdout | from_json }}" 18 | -------------------------------------------------------------------------------- /gating/thaw/haproxycheck.yml: -------------------------------------------------------------------------------- 1 | - name: Check HAProxy status after reconfiguration 2 | hosts: localhost 3 | tasks: 4 | - name: Check state of haproxy backends 5 | shell: | 6 | set -eu 7 | 8 | # There are two flavors of netcat on Ubuntu, but 9 | # only this one supports the -U parameter. 10 | if ! dpkg-query --status netcat-openbsd &>/dev/null; then 11 | echo "netcat not installed - installing it now" 12 | apt-get update 13 | apt-get install netcat-openbsd 14 | fi 15 | 16 | # Check haproxy backends excluding elasticsearch 17 | # Haproxy prints the stats iteration so if the line is "1" its ignored 18 | # Lines where the first field is hash are comments and are ignored 19 | # Field 6 is srv_op_state, and 2 is fully up. 20 | 21 | # http://cbonte.github.io/haproxy-dconv/1.7/management.html#9.3-show%20servers%20state 22 | 23 | echo "Checking HAProxy status" 24 | set -o pipefail 25 | 26 | for i in {1..30} 27 | do 28 | echo "show servers state" |nc -U /var/run/haproxy.stat > haproxy.stat 29 | awk '$2 !~ "elastic" && $0 != "1" && $1 != "#" && $6 != "2" {print $0}' \ 30 | < haproxy.stat \ 31 | |tee haproxy.down 32 | if [[ $(wc -l < haproxy.down) == 1 ]] 33 | then 34 | echo "HAProxy healthchecks passing." 35 | exit 0 36 | else 37 | echo "At least one HAProxy backend is down, sleeping for 60s before" 38 | echo "rechecking." 39 | sleep 60 40 | fi 41 | done 42 | print "HAProxy backends failed to come up in time" 43 | exit 1 44 | args: 45 | executable: /bin/bash 46 | -------------------------------------------------------------------------------- /gating/thaw/run: -------------------------------------------------------------------------------- 1 | #!/bin/bash -xeu 2 | 3 | # Thaw Script 4 | 5 | # This script is executed when an instance is created from a snapshot of 6 | # an RPCO deployment. It should fix things that are broken by the switch 7 | # to a difference instance (eg IPs, hostname) 8 | 9 | ORIG_IP=$( 10 | cat /etc/openstack_deploy/openstack_user_config.yml | \ 11 | awk '/external_lb_vip_address/{print $2}' | sed -e 's/[ "]//g' 12 | ) 13 | 14 | # Need to ensure SSH config is ok before running ansible 15 | mkdir -p /root/.ssh 16 | cat /opt/root_ssh_backup/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys 17 | cat /opt/root_ssh_backup/.ssh/authorized_keys >> /root/.ssh/authorized_keys 18 | cp /opt/root_ssh_backup/.ssh/id_rsa /root/.ssh 19 | cp /opt/root_ssh_backup/.ssh/id_rsa.pub /root/.ssh 20 | cp /opt/root_ssh_backup/.ssh/known_hosts /root/.ssh ||: 21 | ssh-keyscan localhost >> /root/.ssh/known_hosts 22 | 23 | # Re-populate /etc/hosts file with container data 24 | /usr/local/bin/openstack-host-hostfile-setup.sh 25 | 26 | cd /opt/openstack-ansible/playbooks/ 27 | 28 | # Enable the debug stdout callback to ensure that the console 29 | # output is easily legible 30 | export ANSIBLE_STDOUT_CALLBACK=debug 31 | 32 | # Use implicit fact gathering to ensure the fact cache is ignored, 33 | # and facts are always gathered. This important we rely on the public 34 | # IP fact being up to date. 35 | export ANSIBLE_GATHERING=implicit 36 | openstack-ansible -v /opt/rpc-openstack/gating/thaw/thaw.yml 37 | openstack-ansible -t haproxy_server-config haproxy-install.yml 38 | 39 | lxc-autostart --all 40 | 41 | openstack-ansible -v /opt/rpc-openstack/gating/thaw/haproxycheck.yml 42 | openstack-ansible -v /opt/rpc-openstack/gating/thaw/fix_endpoints.yml -e orig_ip=${ORIG_IP} 43 | 44 | # Remove the /gating directory to prevent any further snapshots from being 45 | # taken. 46 | if [[ -d "/gating" ]]; then 47 | rm -rf /gating 48 | fi 49 | -------------------------------------------------------------------------------- /gating/thaw/thaw.yml: -------------------------------------------------------------------------------- 1 | - name: Initialise network interfaces 2 | hosts: localhost 3 | tasks: 4 | - name: Add source line to interfaces file 5 | lineinfile: 6 | dest: /etc/network/interfaces 7 | regexp: '' 8 | insertafter: EOF 9 | line: 'source /etc/network/interfaces.d/*.cfg' 10 | 11 | - name: Enable additional interfaces 12 | shell: | 13 | awk '/iface/{print $2}' /etc/network/interfaces.d/*\ 14 | |while read iface; do ifup $iface; done 15 | 16 | # Split into a new play so that fact gathering runs after all the interfaces 17 | # are up. 18 | - name: Post Network Thaw Tasks 19 | hosts: localhost 20 | tasks: 21 | - name: Check for cinder loop image 22 | stat: 23 | path: /openstack/cinder.img 24 | get_md5: false 25 | get_checksum: false 26 | register: cinder_img 27 | 28 | - name: Setup Cinder LVM loop 29 | shell: | 30 | set -xeu 31 | losetup -f /openstack/cinder.img 32 | args: 33 | executable: /bin/bash 34 | when: cinder_img.stat.exists|bool 35 | 36 | - name: Show new public IP 37 | debug: 38 | var: ansible_default_ipv4.address 39 | 40 | - name: Fix up Public IP 41 | replace: 42 | dest: /etc/openstack_deploy/openstack_user_config.yml 43 | regexp: "external_lb_vip_address:.*" 44 | replace: "external_lb_vip_address: {{ ansible_default_ipv4.address }}" 45 | backup: yes 46 | 47 | - name: Fix up keystone_service 48 | replace: 49 | dest: /etc/haproxy/conf.d/keystone_service 50 | regexp: "bind.*" 51 | replace: "bind {{ ansible_default_ipv4.address }}:5000" 52 | -------------------------------------------------------------------------------- /gating/update_dependencies/pre: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ## Shell Opts ---------------------------------------------------------------- 4 | 5 | set -eux 6 | set -o pipefail 7 | 8 | ## Vars ---------------------------------------------------------------------- 9 | 10 | ## Main ---------------------------------------------------------------------- 11 | 12 | # We need to ensure that we use the rackspace mirrors, as they are 13 | # most reliable. We also need to ensure that python and the python 14 | # yaml library are present for ansible to work. 15 | source "$(readlink -f $(dirname ${0}))/../gating_prerequisites.sh" 16 | -------------------------------------------------------------------------------- /gating/update_dependencies/release-update.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | import re 17 | import os 18 | import semver 19 | import yaml 20 | 21 | # Read the release file path from an environment variable 22 | release_file = os.environ['RELEASE_FILE'] 23 | 24 | # Read the RPC release series name from an environment variable 25 | release_series = os.environ['RPC_PRODUCT_RELEASE'] 26 | 27 | # Read the maas_release to set from an an environment variable 28 | maas_release = os.environ['MAAS_TAG'] 29 | 30 | # Read the osa_release to set from an environment variable 31 | osa_release = os.environ['OSA_SHA'] 32 | 33 | # Read the rpc_rc_release version from an environment variable 34 | rpc_rc_release = os.environ['RC_BRANCH_VERSION'] 35 | 36 | # Read the file contents 37 | with open(release_file) as f: 38 | release_file_content = yaml.safe_load(f.read()) 39 | 40 | # Read the series-specific data 41 | release_data = release_file_content['rpc_product_releases'][release_series] 42 | 43 | # Get the current rpc_release version 44 | rpc_release = release_data['rpc_release'] 45 | 46 | # Validate that the version complies with the RPC-O 47 | # version naming standards. 48 | # ref: https://rpc-openstack.atlassian.net/browse/RE-1199 49 | # 50 | # This is important for all PR's to ensure that we're 51 | # changing the version to something matching the right 52 | # standard. 53 | 54 | release_naming_standard = re.compile( 55 | "^r[0-9]+.[0-9]+.[0-9]+(-(alpha|beta|rc).[0-9]+)?$|^master$") 56 | 57 | assert release_naming_standard.match(rpc_release), ( 58 | "The rpc_release value of %s does not comply with the release naming" 59 | " standard. Please correct it!" % rpc_release) 60 | # Extract the SemVer-compliant portion of the version (no 'r' prefix) 61 | rpc_release_semver = re.sub('^r', '', rpc_release) 62 | 63 | # If the rpc_rc_release and rpc_release versions match, 64 | # then we need to increment the value of rpc_release. 65 | if rpc_rc_release == rpc_release: 66 | 67 | # If the current version is a prerelease version, 68 | # then increment the prerelease. 69 | 70 | if rpc_release != "master": 71 | rpc_release_parts = semver.parse(rpc_release_semver) 72 | if rpc_release_parts['prerelease'] is not None: 73 | rpc_release_semver_new = semver.bump_prerelease(rpc_release_semver) 74 | # Otherwise, this is a standard release and we 75 | # just need to do a patch version increment. 76 | else: 77 | rpc_release_semver_new = semver.bump_patch(rpc_release_semver) 78 | 79 | # Now add the 'r' prefix back on for the final version 80 | rpc_release = "r" + rpc_release_semver_new 81 | 82 | if rpc_release == 'master': 83 | release_data['maas_release'] = 'master' 84 | release_data['osa_release'] = 'master' 85 | release_data['rpc_release'] = 'master' 86 | else: 87 | # Adjust the maas release 88 | release_data['maas_release'] = maas_release 89 | 90 | # Adjust the OSA SHA 91 | release_data['osa_release'] = osa_release 92 | 93 | # Adjust the RPC release version 94 | release_data['rpc_release'] = rpc_release 95 | 96 | # Write the file 97 | with open(release_file, 'w') as f: 98 | f.write( 99 | yaml.safe_dump( 100 | release_file_content, default_flow_style=False, width=1000)) 101 | 102 | # Output the details for debugging purposes 103 | print('Product release set: [%s]' % release_data) 104 | -------------------------------------------------------------------------------- /gating/update_dependencies/role-requirements-update.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | import os 17 | import subprocess 18 | import shutil 19 | import tempfile 20 | 21 | import yaml 22 | 23 | 24 | __doc__ = """Script usage. 25 | Export the environment variable "ROLE_REQUIREMENTS_FILE" with the path to a 26 | known ansible role requirements file. The items within the file will be read 27 | and anything that is using the "git" "scm" setting it will clone extract the 28 | last "SHA" from a given branch. By default this script will assume the repo 29 | within the role requirements file is tracking the "master" branch however if 30 | a different branch is desired set the "branch" option in the role 31 | requirements file to whatever branch the system should track. 32 | 33 | Example entry: 34 | - name: role-name 35 | scm: git 36 | src: https://github.com/rcbops/role-name.git 37 | version: XXXX 38 | 39 | Example entry setting the tracking branch: 40 | - name: role-name 41 | scm: git 42 | src: https://github.com/rcbops/role-name.git 43 | version: XXXX 44 | branch: stable 45 | """ 46 | 47 | 48 | class TempDirMake(object): 49 | def __init__(self): 50 | """Create a temp workspace and cleanup on exit. 51 | 52 | This class creates a context manager which makes a temp workspace and 53 | cleans up when the context manager exits. 54 | 55 | Entering the context manager returns the temp workspace path as a 56 | string. 57 | """ 58 | self.temp_dir = None 59 | self.cwd = None 60 | 61 | def __enter__(self): 62 | os.getcwd() 63 | self.temp_dir = tempfile.mkdtemp() 64 | self.cwd = os.getcwd() 65 | os.chdir(self.temp_dir) 66 | return self.temp_dir 67 | 68 | def __exit__(self, exc_type, exc_val, exc_tb): 69 | shutil.rmtree(self.temp_dir) 70 | os.chdir(self.cwd) 71 | 72 | 73 | # Read the file contents 74 | requirements_file = os.environ['ROLE_REQUIREMENTS_FILE'] 75 | 76 | 77 | with open(requirements_file) as f: 78 | release_file_content = yaml.safe_load(f.read()) 79 | 80 | 81 | with TempDirMake() as mkd: 82 | for item in release_file_content: 83 | if item['scm'] != 'git': 84 | pass 85 | 86 | branch = item.get('branch', 'master') 87 | src = item['src'] 88 | 89 | subprocess.call( 90 | ["git", "clone", src], 91 | stdin=subprocess.PIPE, 92 | stdout=subprocess.PIPE 93 | ) 94 | os.chdir(os.path.basename(src.split('.git')[0])) 95 | subprocess.call( 96 | ["git", "checkout", branch], 97 | stdin=subprocess.PIPE, 98 | stdout=subprocess.PIPE 99 | ) 100 | if item['version'] != 'master': 101 | p = subprocess.check_output( 102 | ['git', 'log', '-n', '1', '--format=%H']) 103 | item['version'] = p.strip() 104 | else: 105 | item['version'] = 'master' 106 | 107 | with open(requirements_file, 'w') as f: 108 | f.write( 109 | yaml.safe_dump( 110 | release_file_content, default_flow_style=False, width=1000)) 111 | -------------------------------------------------------------------------------- /gating/update_dependencies/run: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Set options to ensure easier debugging 4 | # and failure on any problems encountered 5 | set -euxo pipefail 6 | 7 | # Find the absolute path to the directory this script 8 | # is executed from. 9 | export GATING_PATH="$(readlink -f $(dirname ${0}))" 10 | 11 | # Set the base RPC-O directory which the functions use 12 | export BASE_DIR="$(readlink -f ${GATING_PATH}/../..)" 13 | 14 | # Source the functions 15 | source "${BASE_DIR}/scripts/functions.sh" 16 | 17 | # We need pip on the host in order to install the required 18 | # SemVer library, so make sure it is installed and available. 19 | if ! pip --version &>/dev/null; then 20 | echo "Pip not available. Installing it." 21 | apt-get install -y python-pip 22 | fi 23 | 24 | # Install the semver library to manipulate the 25 | # rpc_release version strings 26 | if [[ "$(pip --disable-pip-version-check freeze | grep semver)" != *"semver"* ]]; then 27 | echo "Python SemVer library not available. Installing it." 28 | pip install semver -c upper-constraints.txt 29 | fi 30 | 31 | # Install tox to run the update constraints job 32 | if [[ "$(pip --disable-pip-version-check freeze | grep tox)" != *"tox"* ]]; then 33 | echo "tox not installed, Installing it." 34 | pip install tox -c upper-constraints.txt 35 | fi 36 | 37 | # update constraints 38 | tox -e requirements 39 | 40 | ## Update OSA SHA to head of stable/XXX 41 | # These var must be set per branch of RPC-Openstack 42 | rpco_branch="${BRANCH:-master}" # BRANCH injected by Jenkins. 43 | rc_branch="${rpco_branch}-rc" 44 | 45 | # Env vars injected by Jenkins: 46 | WORKSPACE="${WORKSPACE:-/opt}" 47 | 48 | # Note for local testing: this script expects to be executed from the root 49 | # of an rpco clone, checkout at master. 50 | 51 | ## Get current head of osa 52 | # We clean up any existing directory to make 53 | # testing simpler (re-executing this script) 54 | # and to ensure we have a fresh clone. 55 | osa_dir="${WORKSPACE}/openstack-ansible" 56 | if [[ -e ${osa_dir} ]]; then 57 | rm -rf ${osa_dir} 58 | fi 59 | git clone "https://github.com/openstack/openstack-ansible" "${osa_dir}" 60 | pushd "${osa_dir}" 61 | git checkout "${OSA_RELEASE_BRANCH}" 62 | export OSA_SHA="$(git log -n 1 --format=%H)" 63 | popd 64 | 65 | ## Update rpc-maas to latest tag 66 | # We clean up any existing directory to make 67 | # testing simpler (re-executing this script) 68 | # and to ensure we have a fresh clone. 69 | rpc_maas_dir="${WORKSPACE}/rpc-maas" 70 | if [[ -e ${rpc_maas_dir} ]]; then 71 | rm -rf ${rpc_maas_dir} 72 | fi 73 | git clone https://github.com/rcbops/rpc-maas "${rpc_maas_dir}" 74 | pushd "${rpc_maas_dir}" 75 | # the maas repo includes old tags eg v9.x.x and 9.x.x when the version 76 | # at the time of writing is 1.x.x. All tags that include a character 77 | # or that start with 9 or 10 are filtered out. 78 | export MAAS_TAG="$(git tag -l |grep -v '[a-zA-Z]\|^\(9\.\|10\.\)' |sort -n |tail -n 1)" 79 | popd 80 | 81 | ## Check what version is set in the RC branch 82 | 83 | file_to_fetch="origin/${rc_branch}:playbooks/vars/rpc-release.yml" 84 | release_data_file="${WORKSPACE}/rc-release-data.yml" 85 | 86 | # if there is an RC branch, then use the version information from it 87 | if git show origin/${rc_branch} &>/dev/null; then 88 | git show ${file_to_fetch} > ${release_data_file} 89 | export RC_BRANCH_VERSION=$(${BASE_DIR}/scripts/get-rpc_release.py -f ${release_data_file}) 90 | 91 | # if there is no RC branch, then use the last tag from the branch 92 | else 93 | export RC_BRANCH_VERSION="$(git describe --abbrev=0 --tags)" 94 | fi 95 | 96 | 97 | # Execute role requirements file update 98 | export ROLE_REQUIREMENTS_FILE="${BASE_DIR}/ansible-role-${RPC_PRODUCT_RELEASE}-requirements.yml" 99 | ${GATING_PATH}/role-requirements-update.py 100 | 101 | 102 | # Execute the update of all the data 103 | export RELEASE_FILE="${BASE_DIR}/playbooks/vars/rpc-release.yml" 104 | ${GATING_PATH}/release-update.py 105 | -------------------------------------------------------------------------------- /logo_files/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/logo_files/favicon.ico -------------------------------------------------------------------------------- /logo_files/logo-splash.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/logo_files/logo-splash.png -------------------------------------------------------------------------------- /logo_files/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/logo_files/logo.png -------------------------------------------------------------------------------- /playbooks/configure-release.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Check for RPC-OpenStack product_release variable 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | user: root 21 | gather_facts: true 22 | pre_tasks: 23 | - name: Refresh local facts 24 | setup: 25 | filter: ansible_local 26 | gather_subset: "!all" 27 | tags: 28 | - always 29 | 30 | - name: Ensure local facts directory exists 31 | file: 32 | dest: "/etc/ansible/facts.d" 33 | state: directory 34 | group: "root" 35 | owner: "root" 36 | mode: "0755" 37 | recurse: no 38 | 39 | - name: initialize local facts 40 | ini_file: 41 | dest: "/etc/ansible/facts.d/rpc_openstack.fact" 42 | section: "rpc_product" 43 | option: initialized 44 | value: true 45 | 46 | - name: Refresh local facts 47 | setup: 48 | filter: ansible_local 49 | gather_subset: "!all" 50 | tags: 51 | - always 52 | 53 | - name: Set the rpc-openstack variables 54 | set_fact: 55 | rpc_openstack: "{{ ansible_local['rpc_openstack']['rpc_product'] }}" 56 | 57 | - name: Set the rpc-release variables 58 | set_fact: 59 | rpc_product_release: "{{ rpc_openstack['rpc_product_release'] }}" 60 | when: 61 | - rpc_openstack['rpc_product_release'] is defined 62 | - rpc_product_release is undefined or 63 | rpc_product_release == 'undefined' 64 | 65 | tasks: 66 | - name: Ensure root has a .ssh directory 67 | file: 68 | path: /root/.ssh 69 | state: directory 70 | owner: root 71 | group: root 72 | mode: "0700" 73 | 74 | - name: Create ssh key pair for root 75 | user: 76 | name: root 77 | generate_ssh_key: yes 78 | ssh_key_bits: 2048 79 | ssh_key_file: /root/.ssh/id_rsa 80 | 81 | - name: Store id_rsa.pub 82 | slurp: 83 | src: "/root/.ssh/id_rsa.pub" 84 | register: _root_id_rsa_pub 85 | 86 | - name: Ensure root can ssh to localhost 87 | authorized_key: 88 | user: "root" 89 | key: "{{ _root_id_rsa_pub.content | b64decode }}" 90 | 91 | - name: Check for product_release variable 92 | fail: 93 | msg: >- 94 | RPC product [{{ rpc_product_release }}] is unknown. Available options 95 | are {{ rpc_product_releases.keys() }}. Set the environment variable 96 | RPC_PRODUCT_RELEASE to the product release required or use the 97 | ansible override [rpc_product_release] when running this play. 98 | when: 99 | - rpc_product_release is undefined or 100 | rpc_product_release == 'undefined' or 101 | not rpc_product_releases[rpc_product_release] is defined 102 | 103 | - name: Set OpenStack-Ansible release option 104 | set_fact: 105 | osa_release: "{{ rpc_product_releases[rpc_product_release]['osa_release'] }}" 106 | osa_force_clone: true 107 | when: 108 | - osa_release is undefined 109 | 110 | - name: Set MaaS release option 111 | set_fact: 112 | maas_release: "{{ rpc_product_releases[rpc_product_release]['maas_release'] }}" 113 | when: 114 | - maas_release is undefined 115 | 116 | - name: Set the rpc-product bootstrapped variables 117 | set_fact: 118 | rpc_product_bootstrapped: "{{ rpc_openstack['rpc_product_bootstrapped'] | default('undefined') }}" 119 | rpc_product_new_bootstrap: "rpc-{{ rpc_product_release }}-{{ osa_release }}" 120 | 121 | - name: Clone / Checkout OpenStack-Ansible 122 | git: 123 | repo: "https://git.openstack.org/openstack/openstack-ansible" 124 | dest: "/opt/openstack-ansible" 125 | version: "{{ osa_release }}" 126 | force: "{{ osa_force_clone | default(false) }}" 127 | 128 | - name: Copy basic files into place 129 | shell: | 130 | rsync -av \ 131 | --include='*.yml' --include='*/' --exclude='*' \ 132 | "/opt/openstack-ansible/etc/openstack_deploy/" \ 133 | /etc/openstack_deploy/ 134 | args: 135 | executable: /bin/bash 136 | creates: /etc/openstack_deploy 137 | tags: 138 | - skip_ansible_lint 139 | 140 | - name: Sync configuration for RPC-OpenStack files 141 | shell: | 142 | rsync -av \ 143 | --include='*.yml' --include='*/' --exclude='*' \ 144 | "{{ playbook_dir }}/../etc/openstack_deploy/" \ 145 | /etc/openstack_deploy/ 146 | args: 147 | executable: /bin/bash 148 | when: 149 | - rpc_product_bootstrapped != rpc_product_new_bootstrap 150 | tags: 151 | - skip_ansible_lint 152 | 153 | - name: Set the product release 154 | lineinfile: 155 | dest: /etc/openstack_deploy/group_vars/all/release.yml 156 | state: present 157 | regexp: "^{{ item.key }}:" 158 | line: '{{ item.key }}: "{{ item.value }}"' 159 | with_dict: "{{ rpc_product_releases[rpc_product_release] }}" 160 | 161 | - name: Remove any existing roles 162 | file: 163 | path: "{{ item.path | default(role_path_default) }}/{{ item.name | default(item.src | basename) }}" 164 | state: absent 165 | when: 166 | - item.scm == "git" or item.scm is undefined 167 | - rpc_product_bootstrapped != rpc_product_new_bootstrap 168 | with_items: 169 | - "{{ rpco_required_roles }}" 170 | - "{{ osa_required_roles }}" 171 | 172 | - name: Ensure the default roles directory exists 173 | file: 174 | path: "{{ role_path_default }}" 175 | state: directory 176 | 177 | - name: Clone OSA and RPC-O roles from git 178 | git: 179 | repo: "{{ item.src }}" 180 | dest: "{{ item.path | default(role_path_default) }}/{{ item.name | default(item.src | basename) }}" 181 | version: "{{ item.version | default('master') }}" 182 | refspec: "{{ item.refspec | default(omit) }}" 183 | depth: "{{ item.depth | default('10') }}" 184 | update: true 185 | force: true 186 | when: 187 | - item.scm == "git" or item.scm is undefined 188 | - rpc_product_bootstrapped != rpc_product_new_bootstrap 189 | with_items: 190 | - "{{ osa_required_roles }}" 191 | - "{{ rpco_required_roles }}" 192 | register: git_clone 193 | until: git_clone is succeeded 194 | retries: "{{ git_clone_retries }}" 195 | delay: "{{ git_clone_retry_delay }}" 196 | tags: 197 | - skip_ansible_lint 198 | 199 | post_tasks: 200 | - name: Remove legacy user variable files 201 | file: 202 | dest: "/etc/openstack_deploy/{{ item }}" 203 | state: absent 204 | with_items: 205 | - "user_extras_variables.yml" 206 | - "user_liberty_vars.yml" 207 | - "user_master_vars.yml" 208 | - "user_newton_vars.yml" 209 | - "user_osa_variables_defaults.yml" 210 | - "user_osa_varibles_defaults.yml" 211 | - "user_rpc_maas_variables.yml" 212 | - "user_rpcm_default_variables.yml" 213 | - "user_rpcm_variables_defaults.yml" 214 | - "user_rpco_user_variables_defaults.yml" 215 | - "user_rpco_variables_defaults.yml" 216 | 217 | - name: Set product release local fact 218 | ini_file: 219 | dest: "/etc/ansible/facts.d/rpc_openstack.fact" 220 | section: "rpc_product" 221 | option: "{{ item.option }}" 222 | value: "{{ item.value }}" 223 | with_items: 224 | - { option: "rpc_product_release", value: "{{ rpc_product_release }}" } 225 | - { option: "rpc_product_bootstrapped", value: "{{ rpc_product_new_bootstrap }}" } 226 | 227 | - name: Set the product release local facts 228 | ini_file: 229 | dest: "/etc/ansible/facts.d/rpc_openstack.fact" 230 | section: "rpc_product" 231 | option: "{{ item.key }}" 232 | value: "{{ item.value }}" 233 | with_dict: "{{ rpc_product_releases[rpc_product_release] }}" 234 | 235 | vars: 236 | ansible_python_interpreter: "/usr/bin/python" 237 | rpc_product_release: "{{ lookup('env', 'RPC_PRODUCT_RELEASE') | default('undefined', true) }}" 238 | osa_required_roles: "{{ lookup('file', osa_role_file) | from_yaml }}" 239 | rpco_required_roles: "{{ lookup('file', rpco_role_file) | from_yaml }}" 240 | osa_role_file: "/opt/openstack-ansible/ansible-role-requirements.yml" 241 | rpco_role_file: "{{ playbook_dir }}/../ansible-role-{{ rpc_product_release }}-requirements.yml" 242 | role_path_default: '/etc/ansible/roles' 243 | git_clone_retries: 2 244 | git_clone_retry_delay: 5 245 | 246 | vars_files: 247 | - vars/rpc-release.yml 248 | 249 | tags: 250 | - rpc 251 | -------------------------------------------------------------------------------- /playbooks/deployment-elk.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2018, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Bootstrap embedded ansible 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | gather_facts: true 21 | tasks: 22 | - name: Run bootstrap process 23 | command: "/opt/openstack-ansible-ops/elk_metrics_6x/bootstrap-embedded-ansible.sh" 24 | changed_when: false 25 | 26 | - name: Ensure that the required user space folders are present 27 | file: 28 | path: "{{ item }}" 29 | state: directory 30 | with_items: 31 | - "/etc/openstack_deploy/conf.d" 32 | - "/etc/openstack_deploy/env.d" 33 | 34 | - name: Add the inventory environment information 35 | copy: 36 | src: "/opt/openstack-ansible-ops/elk_metrics_6x/env.d/elk.yml" 37 | dest: "/etc/openstack_deploy/env.d/elk.yml" 38 | remote_src: yes 39 | 40 | - name: Configure ELK groups in the inventory 41 | copy: 42 | content: | 43 | --- 44 | # Kibana hosts 45 | kibana_hosts: 46 | {% for item in groups['log_hosts'] %} 47 | {{ item }}: 48 | ip: {{ hostvars[item]['ansible_host'] }} 49 | # Elastic hosts 50 | elastic-logstash_hosts: 51 | {{ item }}: 52 | ip: {{ hostvars[item]['ansible_host'] }} 53 | {% endfor %} 54 | # APM hosts 55 | apm-server_hosts: {} 56 | dest: "/etc/openstack_deploy/conf.d/elk.yml" 57 | 58 | - name: Reload inventory 59 | command: "ansible -m ping localhost" 60 | changed_when: false 61 | args: 62 | chdir: "/opt/openstack-ansible/playbooks" 63 | tags: 64 | - elk 65 | - elk-bootstrap 66 | 67 | 68 | - name: Run elk deployment 69 | hosts: localhost 70 | environment: "{{ deployment_environment_variables | default({}) }}" 71 | connection: local 72 | gather_facts: true 73 | tasks: 74 | - name: Create elk container(s) 75 | become: yes 76 | become_user: root 77 | command: >- 78 | openstack-ansible containers-nspawn-create.yml containers-lxc-create.yml --limit lxc_hosts:elk_all 79 | args: 80 | chdir: "/opt/openstack-ansible/playbooks" 81 | tags: 82 | - skip_ansible_lint 83 | environment: 84 | ANSIBLE_LOG_PATH: "/var/log/ansible-elk-beats-deployment.log" 85 | 86 | - name: Find secrets files 87 | find: 88 | paths: "/etc/openstack_deploy" 89 | patterns: 'user_.*(secret|elk|aio).*.(yml|yaml)$' 90 | use_regex: yes 91 | register: secrets_files 92 | 93 | - name: Run elk+beat(s) deployment 94 | become: yes 95 | become_user: root 96 | command: >- 97 | {{ ansible_env.HOME }}/ansible_venv/bin/ansible-playbook 98 | {{ secrets_files.files | map(attribute='path') | list | map('regex_replace', '(.*)' ,'-e @' ~ '\1') | list | join(' ') }} 99 | -e elk_package_state=latest 100 | {{ item }} 101 | with_items: 102 | - site-elka.yml 103 | - installFilebeat.yml 104 | - installJournalbeat.yml 105 | - installHeartbeat.yml 106 | - installAuditbeat.yml 107 | tags: 108 | - skip_ansible_lint 109 | environment: 110 | ANSIBLE_LOG_PATH: "/var/log/ansible-elk-beats-deployment.log" 111 | ANSIBLE_INVENTORY: "{{ ansible_env.HOME }}/ansible_venv/inventory/openstack_inventory.sh" 112 | ANSIBLE_HOST_KEY_CHECKING: "False" 113 | ANSIBLE_ROLES_PATH: "{{ ansible_env.HOME }}/ansible_venv/repositories/roles" 114 | ANSIBLE_ACTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/ansible-config_template/action" 115 | ANSIBLE_CONNECTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/openstack-ansible-plugins/connection/" 116 | args: 117 | chdir: "/opt/openstack-ansible-ops/elk_metrics_6x" 118 | tags: 119 | - elk 120 | - elk-deployment 121 | -------------------------------------------------------------------------------- /playbooks/deployment-osquery.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2018, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Bootstrap embedded ansible 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | gather_facts: true 21 | tasks: 22 | - name: Run bootstrap process 23 | command: "/opt/openstack-ansible-ops/elk_metrics_6x/bootstrap-embedded-ansible.sh" 24 | changed_when: false 25 | 26 | - name: Create kolide-fleet groups 27 | copy: 28 | content: | 29 | --- 30 | # Fleet hosts 31 | kolide_hosts: 32 | {% for item in groups['log_hosts'] %} 33 | {{ item }}: 34 | ip: {{ hostvars[item]['ansible_host'] }} 35 | {% endfor %} 36 | dest: "/etc/openstack_deploy/conf.d/fleet.yml" 37 | 38 | - name: Reload inventory 39 | command: "ansible -m ping localhost" 40 | changed_when: false 41 | args: 42 | chdir: "/opt/openstack-ansible/playbooks" 43 | tags: 44 | - fleet 45 | - fleet-bootstrap 46 | 47 | 48 | - name: Run kolide-fleet deployment 49 | hosts: localhost 50 | environment: "{{ deployment_environment_variables | default({}) }}" 51 | connection: local 52 | gather_facts: true 53 | tasks: 54 | - name: Create kolide-fleet container(s) 55 | become: yes 56 | become_user: root 57 | command: >- 58 | openstack-ansible containers-nspawn-create.yml containers-lxc-create.yml --limit lxc_hosts:fleet_all 59 | args: 60 | chdir: "/opt/openstack-ansible/playbooks" 61 | tags: 62 | - skip_ansible_lint 63 | environment: 64 | ANSIBLE_LOG_PATH: "/var/log/ansible-kolide-fleet-deployment.log" 65 | 66 | - name: Set kolide-fleet secrets 67 | lineinfile: 68 | dest: /etc/openstack_deploy/user_secrets.yml 69 | state: present 70 | regexp: "^{{ item.key }}" 71 | line: '{{ item.key }}: "{{ item.value }}"' 72 | no_log: True 73 | with_items: 74 | - key: kolide_fleet_db_password 75 | value: "{{ lookup('password', '/dev/null length=24 chars=ascii_letters') }}" 76 | - key: kolide_fleet_jwt_key 77 | value: "{{ lookup('password', '/dev/null length=16 chars=ascii_letters') }}" 78 | - key: kolide_fleet_admin_password 79 | value: "{{ lookup('password', '/dev/null length=8 chars=ascii_letters') }}" 80 | 81 | - name: Get osquery roles 82 | become: yes 83 | become_user: root 84 | command: >- 85 | {{ ansible_env.HOME }}/ansible_venv/bin/ansible-galaxy install -r ansible-role-requirements.yml 86 | --roles-path={{ ansible_env.HOME }}/ansible_venv/repositories/roles 87 | --ignore-errors 88 | tags: 89 | - skip_ansible_lint 90 | environment: 91 | ANSIBLE_LOG_PATH: "/var/log/ansible-kolide-fleet-deployment.log" 92 | ANSIBLE_INVENTORY: "{{ ansible_env.HOME }}/ansible_venv/inventory/openstack_inventory.sh" 93 | ANSIBLE_HOST_KEY_CHECKING: "False" 94 | ANSIBLE_ROLES_PATH: "{{ ansible_env.HOME }}/ansible_venv/repositories/roles" 95 | ANSIBLE_ACTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/ansible-config_template/action" 96 | ANSIBLE_CONNECTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/openstack-ansible-plugins/connection/" 97 | args: 98 | chdir: "/opt/openstack-ansible-ops/osquery" 99 | 100 | - name: Find secrets files 101 | find: 102 | paths: "/etc/openstack_deploy" 103 | patterns: 'user_.*(secret|kolide|fleet|aio).*.(yml|yaml)$' 104 | use_regex: yes 105 | register: secrets_files 106 | 107 | - name: Run osquery deployment 108 | become: yes 109 | become_user: root 110 | command: >- 111 | {{ ansible_env.HOME }}/ansible_venv/bin/ansible-playbook 112 | {{ secrets_files.files | map(attribute='path') | list | map('regex_replace', '(.*)' ,'-e @' ~ '\1') | list | join(' ') }} 113 | {{ item }} 114 | with_items: 115 | - site-fleet.yml 116 | - site-osquery.yml 117 | tags: 118 | - skip_ansible_lint 119 | environment: 120 | ANSIBLE_LOG_PATH: "/var/log/ansible-kolide-fleet-deployment.log" 121 | ANSIBLE_INVENTORY: "{{ ansible_env.HOME }}/ansible_venv/inventory/openstack_inventory.sh" 122 | ANSIBLE_HOST_KEY_CHECKING: "False" 123 | ANSIBLE_ROLES_PATH: "{{ ansible_env.HOME }}/ansible_venv/repositories/roles" 124 | ANSIBLE_ACTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/ansible-config_template/action" 125 | ANSIBLE_CONNECTION_PLUGINS: "{{ ansible_env.HOME }}/ansible_venv/repositories/openstack-ansible-plugins/connection/" 126 | args: 127 | chdir: "/opt/openstack-ansible-ops/osquery" 128 | tags: 129 | - fleet 130 | - fleet-deployment 131 | -------------------------------------------------------------------------------- /playbooks/files/ironic_all.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | extra_lb_vip_addresses: 17 | - "https://{{ external_lb_vip_address }}" 18 | 19 | ironic_openstack_api_url: "https://{{ external_lb_vip_address }}:{{ ironic_service_port }}" 20 | ironic_swift_endpoint: "https://{{ external_lb_vip_address }}:8080" 21 | 22 | nova_scheduler_use_baremetal_filters: False 23 | -------------------------------------------------------------------------------- /playbooks/get-maas.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2014, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Clone MaaS 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | gather_facts: false 21 | tasks: 22 | - name: Refresh local facts 23 | setup: 24 | filter: ansible_local 25 | gather_subset: "!all" 26 | tags: 27 | - always 28 | 29 | - name: Clone RPC-MaaS 30 | git: 31 | repo: "https://github.com/rcbops/rpc-maas" 32 | dest: "/opt/rpc-maas" 33 | version: "{{ ansible_local['rpc_openstack']['maas_release'] }}" 34 | 35 | - name: Stat /etc/openstack_deploy 36 | stat: 37 | path: "/etc/openstack_deploy" 38 | register: stat_openstack_deploy 39 | 40 | - name: Fail when /etc/openstack_deploy doesn't exist 41 | fail: 42 | msg: "/etc/openstack_deploy is required but doesn't exist" 43 | when: 44 | - not stat_openstack_deploy.stat.exists 45 | 46 | - name: Copy over base maas vars 47 | copy: 48 | src: "/opt/rpc-maas/tests/user_master_vars.yml" 49 | dest: "/etc/openstack_deploy/user_rpcm_variables_defaults.yml" 50 | 51 | tags: 52 | - maas 53 | - rpc 54 | -------------------------------------------------------------------------------- /playbooks/openstack-ansible-install.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Install OpenStack-Ansible 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | user: root 21 | gather_facts: true 22 | pre_tasks: 23 | - name: Refresh local facts 24 | setup: 25 | filter: ansible_local 26 | gather_subset: "!all" 27 | tags: 28 | - always 29 | 30 | - name: Ensure local facts directory exists 31 | file: 32 | dest: "/etc/ansible/facts.d" 33 | state: directory 34 | group: "root" 35 | owner: "root" 36 | mode: "0755" 37 | recurse: no 38 | 39 | - name: initialize local facts 40 | ini_file: 41 | dest: "/etc/ansible/facts.d/rpc_openstack.fact" 42 | section: "rpc_product" 43 | option: initialized 44 | value: true 45 | 46 | - name: Set the rpc-openstack variables 47 | set_fact: 48 | rpc_openstack: "{{ ansible_local['rpc_openstack']['rpc_product'] }}" 49 | 50 | - name: Set the rpc-release variables 51 | set_fact: 52 | rpc_product_release: "{{ rpc_openstack['rpc_product_release'] }}" 53 | when: 54 | - rpc_openstack['rpc_product_release'] is defined 55 | - rpc_product_release is undefined or 56 | rpc_product_release == 'undefined' 57 | 58 | - name: Set OpenStack-Ansible release option 59 | set_fact: 60 | osa_release: "{{ rpc_product_releases[rpc_product_release]['osa_release'] }}" 61 | when: 62 | - osa_release is undefined 63 | 64 | - name: Set the rpc-product bootstrapped variables 65 | set_fact: 66 | osa_bootstrapped: "{{ rpc_openstack['osa_bootstrapped'] | default('undefined') }}" 67 | osa_new_bootstrap: "osa-{{ rpc_product_release }}-{{ osa_release }}" 68 | 69 | tasks: 70 | - name: Install the OpenStack-Ansible 71 | shell: | 72 | ./scripts/bootstrap-ansible.sh 73 | args: 74 | chdir: "/opt/openstack-ansible" 75 | when: 76 | - osa_bootstrapped != osa_new_bootstrap 77 | tags: 78 | - skip_ansible_lint 79 | 80 | post_tasks: 81 | - name: Set product release local fact 82 | ini_file: 83 | dest: "/etc/ansible/facts.d/rpc_openstack.fact" 84 | section: "rpc_product" 85 | option: "{{ item.option }}" 86 | value: "{{ item.value }}" 87 | with_items: 88 | - { option: "osa_bootstrapped", value: "{{ osa_new_bootstrap }}" } 89 | 90 | vars: 91 | rpc_product_release: "{{ lookup('env', 'RPC_PRODUCT_RELEASE') | default('undefined', true) }}" 92 | 93 | vars_files: 94 | - vars/rpc-release.yml 95 | 96 | tags: 97 | - rpc 98 | -------------------------------------------------------------------------------- /playbooks/openstack-ansible-ops-get.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2018, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Clone openstack-ansible-ops 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | connection: local 20 | gather_facts: false 21 | tasks: 22 | - name: Refresh local facts 23 | setup: 24 | filter: ansible_local 25 | gather_subset: "!all" 26 | tags: 27 | - always 28 | 29 | - name: Clone OpenStack-Ansible-OPS 30 | git: 31 | repo: "https://github.com/openstack/openstack-ansible-ops" 32 | dest: "/opt/openstack-ansible-ops" 33 | version: "{{ ansible_local['rpc_openstack']['rpc_product']['openstack_ansible_ops'] }}" 34 | tags: 35 | - ops 36 | - ops-get 37 | -------------------------------------------------------------------------------- /playbooks/openstack-flavor-setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: OpenStack flavor setup 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | user: root 20 | tasks: 21 | - name: Create OpenStack client configuration 22 | include_role: 23 | name: "openstack_openrc" 24 | 25 | - name: Create flavors of nova VMs 26 | os_nova_flavor: 27 | endpoint_type: internal 28 | cloud: default 29 | state: present 30 | name: "{{ item.name }}" 31 | ram: "{{ item.ram }}" 32 | vcpus: "{{ item.vcpus }}" 33 | disk: "{{ item.disk }}" 34 | swap: "{{ item.swap }}" 35 | ephemeral: "{{ item.ephemeral }}" 36 | with_items: "{{ openstack_vm_flavors }}" 37 | 38 | vars_files: 39 | - vars/openstack-service-config.yml 40 | 41 | tags: 42 | - rpc 43 | -------------------------------------------------------------------------------- /playbooks/openstack-image-setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: OpenStack image setup 17 | hosts: localhost 18 | environment: "{{ deployment_environment_variables | default({}) }}" 19 | user: root 20 | tasks: 21 | - name: Create OpenStack client configuration 22 | include_role: 23 | name: "openstack_openrc" 24 | 25 | - name: Download system image file 26 | get_url: 27 | url: "{{ item.url }}" 28 | dest: "/var/tmp/os_image_{{ item.name }}" 29 | timeout: 1200 30 | with_items: "{{ openstack_images }}" 31 | 32 | - name: Install system image 33 | os_image: 34 | endpoint_type: internal 35 | cloud: default 36 | state: present 37 | is_public: true 38 | name: "{{ item.name }}" 39 | filename: "/var/tmp/os_image_{{ item.name }}" 40 | disk_format: "{{ item.format }}" 41 | properties: "{{ item.properties | default(omit) }}" 42 | with_items: "{{ openstack_images }}" 43 | 44 | - name: Clean up temp file 45 | file: 46 | dest: "/var/tmp/os_image_{{ item.name }}" 47 | state: absent 48 | with_items: "{{ openstack_images }}" 49 | 50 | vars_files: 51 | - vars/openstack-service-config.yml 52 | 53 | tags: 54 | - rpc 55 | -------------------------------------------------------------------------------- /playbooks/ops-cpu-govenor-setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2018-Present, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Configure Linux CPU governor 17 | hosts: hosts:mons:osds 18 | vars: 19 | governor: "{{ cpu_governor | default('performance') }}" 20 | handlers: 21 | - name: restart sysfs 22 | systemd: 23 | name: sysfsutils 24 | enabled: yes 25 | state: restarted 26 | tasks: 27 | - name: Check for cpuidle 28 | stat: 29 | path: /sys/devices/system/cpu/cpu0/cpuidle 30 | register: cpuidle_check 31 | 32 | - name: Check for governor 33 | stat: 34 | path: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 35 | register: scaling_governor_check 36 | 37 | - name: setup cpu_scaling_governor 38 | block: 39 | - name: Disable ondemand service 40 | systemd: 41 | name: ondemand 42 | state: stopped 43 | masked: yes 44 | enabled: no 45 | failed_when: false 46 | 47 | - name: Disable cpufrequtils service 48 | systemd: 49 | name: cpufrequtils 50 | state: stopped 51 | enabled: no 52 | masked: yes 53 | failed_when: false 54 | 55 | - name: Find cpus scaling governor 56 | shell: ls -1 /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_governor 57 | changed_when: false 58 | register: cpus_scaling_governor 59 | 60 | - name: Persist cpus scaling governor 61 | lineinfile: 62 | dest: "/etc/sysfs.conf" 63 | line: "{{ item.lstrip('/sys/') }} = {{ governor }}" 64 | regexp: "^{{ item.lstrip('/sys/') }}.*" 65 | with_items: "{{ cpus_scaling_governor.stdout_lines }}" 66 | notify: 67 | - restart sysfs 68 | when: 69 | - scaling_governor_check.stat.exists 70 | 71 | - name: setup cpuidle 72 | block: 73 | - name: Find cpus cpuidle 74 | shell: ls -1 /sys/devices/system/cpu/cpu*/cpuidle/state[2-4]/disable 75 | changed_when: false 76 | register: cpus_cpuidle 77 | 78 | - name: Persist cpus scaling governor 79 | lineinfile: 80 | dest: "/etc/sysfs.conf" 81 | line: "{{ item.lstrip('/sys/') }} = 1" 82 | regexp: "^{{ item.lstrip('/sys/') }}.*" 83 | with_items: "{{ cpus_cpuidle.stdout_lines }}" 84 | notify: 85 | - restart sysfs 86 | when: 87 | - cpuidle_check.stat.exists 88 | -------------------------------------------------------------------------------- /playbooks/run_tempest.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - name: Return legacy notice 17 | hosts: localhost 18 | connection: localhost 19 | tasks: 20 | - name: Notice 21 | debug: 22 | msg: >- 23 | This playbook and task are only here for legacy purposes. As soon as 24 | we can fix gating so that it's not calling scripts repeatedly this 25 | should be removed. 26 | -------------------------------------------------------------------------------- /playbooks/site-logging.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2015, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: openstack-ansible-ops-get.yml 17 | - include: deployment-osquery.yml 18 | when: groups['log_hosts'] is defined and (groups['log_hosts'] | length>0) 19 | - include: deployment-elk.yml 20 | when: groups['log_hosts'] is defined and (groups['log_hosts'] | length>0) 21 | -------------------------------------------------------------------------------- /playbooks/site-openstack.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: openstack-image-setup.yml 17 | - include: openstack-flavor-setup.yml 18 | -------------------------------------------------------------------------------- /playbooks/site-ops.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2015, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: ops-cpu-govenor-setup.yml 17 | -------------------------------------------------------------------------------- /playbooks/site-release.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2015, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: configure-release.yml 17 | -------------------------------------------------------------------------------- /playbooks/site.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2015, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: site-release.yml 17 | - include: site-openstack.yml 18 | - include: site-ops.yml 19 | -------------------------------------------------------------------------------- /playbooks/vars/openstack-service-config.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | openstack_vm_flavors: 17 | - name: m1.micro 18 | ram: 256 19 | vcpus: 1 20 | disk: 1 21 | swap: 0 22 | ephemeral: 0 23 | - name: m1.tiny 24 | ram: 512 25 | vcpus: 1 26 | disk: 1 27 | swap: 0 28 | ephemeral: 0 29 | - name: m1.mini 30 | ram: 1024 31 | vcpus: 2 32 | disk: 3 33 | swap: 0 34 | ephemeral: 0 35 | - name: m1.small 36 | ram: 2048 37 | vcpus: 3 38 | disk: 12 39 | swap: 4 40 | ephemeral: 4 41 | - name: m1.medium 42 | ram: 4096 43 | vcpus: 6 44 | disk: 60 45 | swap: 4 46 | ephemeral: 20 47 | - name: m1.large 48 | ram: 8192 49 | vcpus: 12 50 | disk: 300 51 | swap: 4 52 | ephemeral: 150 53 | - name: m1.xlarge 54 | ram: 16384 55 | vcpus: 24 56 | disk: 600 57 | swap: 4 58 | ephemeral: 256 59 | - name: m1.heavy 60 | ram: 32768 61 | vcpus: 48 62 | disk: 1200 63 | swap: 4 64 | ephemeral: 256 65 | 66 | openstack_images: 67 | - name: CentOS 7 68 | format: qcow2 69 | url: http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 70 | properties: 71 | hw_scsi_model: "virtio-scsi" 72 | hw_disk_bus: "scsi" 73 | hw_vif_multiqueue_enabled: "true" 74 | hw_qemu_guest_agent: "yes" 75 | hypervisor_type: "kvm" 76 | os_require_quiesce: "yes" 77 | img_config_drive: "optional" 78 | - name: Cirros-0.3.5 (kvm) 79 | format: qcow2 80 | url: http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img 81 | properties: 82 | hypervisor_type: "kvm" 83 | - name: Cirros-0.3.5 84 | format: qcow2 85 | url: http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img 86 | properties: 87 | hypervisor_type: "qemu" 88 | - name: Debian 9 89 | format: qcow2 90 | url: http://cdimage.debian.org/cdimage/openstack/current-9/debian-9-openstack-amd64.qcow2 91 | properties: 92 | hw_scsi_model: "virtio-scsi" 93 | hw_disk_bus: "scsi" 94 | hw_vif_multiqueue_enabled: "true" 95 | hw_qemu_guest_agent: "yes" 96 | hypervisor_type: "kvm" 97 | os_require_quiesce: "yes" 98 | img_config_drive: "optional" 99 | - name: OpenSuse Leap 42.3 100 | format: qcow2 101 | url: http://download.opensuse.org/repositories/Cloud:/Images:/Leap_42.3/images/openSUSE-Leap-42.3-OpenStack.x86_64.qcow2 102 | properties: 103 | hw_scsi_model: "virtio-scsi" 104 | hw_disk_bus: "scsi" 105 | hw_vif_multiqueue_enabled: "true" 106 | hw_qemu_guest_agent: "yes" 107 | hypervisor_type: "kvm" 108 | os_require_quiesce: "yes" 109 | img_config_drive: "optional" 110 | - name: Ubuntu 14.04 LTS 111 | format: qcow2 112 | url: http://uec-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img 113 | properties: 114 | hw_scsi_model: "virtio-scsi" 115 | hw_disk_bus: "scsi" 116 | hw_vif_multiqueue_enabled: "true" 117 | hw_qemu_guest_agent: "yes" 118 | hypervisor_type: "kvm" 119 | os_require_quiesce: "yes" 120 | img_config_drive: "optional" 121 | - name: Ubuntu 16.04 122 | format: qcow2 123 | url: http://uec-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img 124 | properties: 125 | hw_scsi_model: "virtio-scsi" 126 | hw_disk_bus: "scsi" 127 | hw_vif_multiqueue_enabled: "true" 128 | hw_qemu_guest_agent: "yes" 129 | hypervisor_type: "kvm" 130 | os_require_quiesce: "yes" 131 | img_config_drive: "optional" 132 | - name: Ubuntu 18.04 133 | format: qcow2 134 | url: http://uec-images.ubuntu.com/releases/18.04/release/ubuntu-18.04-server-cloudimg-amd64.img 135 | properties: 136 | hw_scsi_model: "virtio-scsi" 137 | hw_disk_bus: "scsi" 138 | hw_vif_multiqueue_enabled: "true" 139 | hw_qemu_guest_agent: "yes" 140 | hypervisor_type: "kvm" 141 | os_require_quiesce: "yes" 142 | img_config_drive: "optional" 143 | -------------------------------------------------------------------------------- /playbooks/vars/rpc-release.yml: -------------------------------------------------------------------------------- 1 | rpc_product_releases: 2 | master: 3 | maas_release: master 4 | openstack_ansible_ops: master 5 | osa_release: master 6 | rpc_release: master 7 | newton: 8 | maas_release: 1.7.0 9 | openstack_ansible_ops: master 10 | osa_release: 3553c048188093c7a8d912cc32c30730536d70d3 11 | rpc_release: r14.8.0 12 | ocata: 13 | maas_release: 1.7.0 14 | openstack_ansible_ops: master 15 | osa_release: 5047124f1fe181306674c60beeccd189252a9d62 16 | rpc_release: r15.0.0 17 | pike: 18 | maas_release: 1.7.0 19 | openstack_ansible_ops: master 20 | osa_release: 5c341a7bada78edab5f3d132d55adb00eaf2413f 21 | rpc_release: r16.2.0 22 | queens: 23 | maas_release: 1.7.4 24 | openstack_ansible_ops: master 25 | osa_release: d38e190e43dfb737e6684096084b9f98f89e0637 26 | rpc_release: r17.0.2 27 | rocky: 28 | maas_release: 1.7.5 29 | openstack_ansible_ops: master 30 | osa_release: 9e72443b0d0a2e72f85419ae3aba94dee22f1436 31 | rpc_release: r18.0.0 32 | -------------------------------------------------------------------------------- /releasenotes/config.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | ## TODO(odyssey4me): 3 | ## Whenever master is branched, this must be enabled 4 | ## and modified to include the first tag on the branch. 5 | #earliest_version: rx.y.z 6 | release_tag_re: '^r\d+\.\d+\.\d+(rc\d+)?' 7 | pre_release_tag_re: '(?Prc\d+$)' 8 | -------------------------------------------------------------------------------- /releasenotes/notes/13.0.0-8158f0be466af558.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - OSA Variable ssh_delay is now overriden to 60 from the osa default of 5. 4 | fixes: 5 | - OSA Variable ssh_delay is now overriden to 60 from the osa default of 5, 6 | This is to reduce failures when ansible is attempting to access a container 7 | while it is (re)starting. 8 | -------------------------------------------------------------------------------- /releasenotes/notes/LVMISCSIDriver-b25cdd70ef0c5a4d.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - The cinder driver LVMISCSIDriver has been removed, any configurations should be 4 | updated to use LVMVolumeDriver instead prior to deploying Mitaka. 5 | -------------------------------------------------------------------------------- /releasenotes/notes/add-magnum-f5-93cc4c09fdb60d23.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - | 4 | Added a Public endpoint for Magnum API 5 | fixes: 6 | - | 7 | Magnum endpoint now uses a Magnum-specific monitor 8 | -------------------------------------------------------------------------------- /releasenotes/notes/add-more-galera-checks-032c765829e922a2.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - | 4 | The galera_check.py plugin now provides the following 5 | metrics. 6 | * num_of_open_files 7 | * open_files_limit 8 | * innodb_row_lock_time_avg 9 | * innodb_deadlocks 10 | * access_denied_errors 11 | * aborted_clients 12 | * aborted_connects 13 | - | 14 | | New alarms have been created with the following criteria. 15 | | open_file_size_limit_reached: 16 | | metric["num_of_open_files"] > metric["open_files_limit"] 17 | | innodb_row_lock_time_avg: 18 | | metric["innodb_row_lock_time_avg"] > {{ innodb_row_lock_time_avg_critical_threshold }} 19 | | innodb_deadlocks: 20 | | metric["innodb_deadlocks"] != 0 21 | | access_denied_errors: 22 | | rate(metric["access_denied_errors"]) > {{ mysql_access_denied_errors_rate_warning_threshold }} 23 | | aborted_clients: 24 | | rate(metric["aborted_clients"]) > 1 25 | | aborted_connects: 26 | | rate(metric["aborted_connects"]) > 1 27 | -------------------------------------------------------------------------------- /releasenotes/notes/add-option-to-disable-artifacts-23125367d1c1e834.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - The option to enable or disable artifacts has been added giving the 4 | deployer the ability to chose how artifacts will be consumed. Three new 5 | variables are now available to users, `apt_artifact_enabled` 6 | `container_artifact_enabled` `py_artifact_enabled` which are all 7 | **Boolean**. If a user defines any of variables it will used as the 8 | ultimate source of truth, even if artifacts are "found" for the given 9 | release. 10 | - Artifacted builds will save facts as an ansible local fact. If a deployer 11 | needs to expunge facts from our cached local facts the file 12 | `/etc/ansible/facts.d/rpc_openstack.fact` can be modified or removed as 13 | needed. All artifacted facts will be saved under the "rpc_artifacts" 14 | section. 15 | -------------------------------------------------------------------------------- /releasenotes/notes/add-post-upgrade-role-3821a510f5be4aae.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - A new playbook has been created, ``post-upgrade.yml``, to 4 | assist operators with capturing the state of an environment 5 | after an upgrade. 6 | upgrade: 7 | - A new playbook has been created, ``post-upgrade.yml``, to 8 | assist operators with capturing the state of an environment 9 | after an upgrade. 10 | -------------------------------------------------------------------------------- /releasenotes/notes/add-tempest-execution-playbook-efe64a286f255c34.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - | 4 | A new playbook has been introduced in the ``scripts/`` directory called 5 | ``run_tempest.yml``. This playbook will install tempest using the upstream 6 | OSA role, then execute the tempest tests defined in the sets enumerated 7 | in the ``tempest_test_sets`` variable. 8 | - | 9 | A new variable, ``tempest_test_sets`` has been added to 10 | group_vars/all/rpc-o.yml. This variable contains a space-delimited string 11 | of tempest test sets to execute when running the ``scripts/run_tempest.yml`` 12 | playbook. 13 | -------------------------------------------------------------------------------- /releasenotes/notes/ceph-upgrade-434a53379d0db8f5.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - The ceph monitors and osds needs their configuration 4 | file to be updated for the Mitaka release, to adapt 5 | with the hostnames changes done in Mitaka. Please 6 | backup, destroy, and re-create the monitors, and 7 | make them rejoin the cluster in a serial way. The 8 | osds also need to have their configuration file 9 | updated. 10 | - When re-created during the upgrade procedure, the 11 | ceph monitors will now have bind mounts to the hosts. 12 | This will make the backup of the mons easier in the 13 | future. 14 | -------------------------------------------------------------------------------- /releasenotes/notes/disable_octavia_image_download-0d64f2dc019f7d0d.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - | 4 | OS-octavia was downlaoding its amphora image from the upstream 5 | daily master build - thus delaying the deploy for the duration 6 | of the downlaod and potentially using untested octavia versions. 7 | This patch deactivates the automatic download which might result 8 | in the deployer ghaving to uplaod the octavia amphora image to 9 | glance themselves. 10 | -------------------------------------------------------------------------------- /releasenotes/notes/empty-yaml-bcbc0c731bed02e7.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - The script update-yaml.py will now handle empty yaml 4 | files 5 | -------------------------------------------------------------------------------- /releasenotes/notes/enable-security-hardening-2d9f88309ad45ac0.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | security: 3 | - | 4 | The `openstack-ansible-security role `_ 5 | is now applied to all environments by default. It provides enhanced host 6 | security hardening without disrupting an existing OpenStack environment. 7 | 8 | To opt-out of host security hardening, set 'apply_security_hardening: false' in /etc/openstack_deploy/user_osa_variables_overrides.yml. 9 | 10 | The security role documentation explains which 11 | `security configurations are applied `_ 12 | and `how to apply custom configurations `_. 13 | -------------------------------------------------------------------------------- /releasenotes/notes/fix-get-rpc_release-rpc_product_release.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - | 4 | `scripts/get-rpc_release.py` was ignoring the RPC_PRODUCT_RELEASE 5 | environment variable to allow for branchless deployments. This fix 6 | will cause allow RPC_PRODUCT_RELEASE to overfide the branch 7 | default. 8 | 9 | 10 | -------------------------------------------------------------------------------- /releasenotes/notes/fix-rpc-maas-tool-check-validation-a5190a72be5830f6.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - | 4 | `rpc-maas-tool.py` is modified so that validating the status of checks 5 | correctly reports when there are failures. This tool is used by the 6 | playbook `verify-maas.yml`. 7 | -------------------------------------------------------------------------------- /releasenotes/notes/handle-merging-new-secrets-0e1fd06d86f0abbd.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - The ``update-yaml.py`` script now contains an optional 4 | argument, ``--output-file``, for specifying an output 5 | file. 6 | upgrade: 7 | - Any new secret variables that may be introduced in 8 | later releases are now handled by the ``update-secrets.sh`` 9 | script. ``test-upgrade.sh`` and ``deploy.sh`` have been updated 10 | to use ``update-secrets.sh`` to handle any new variables 11 | that may be introduced in newer releases. If the deployer 12 | is not using these scripts for upgrades, then 13 | ``update-secrets.sh`` needs to be invoked manually. Please 14 | see official RPCO upgrade docs for usage. 15 | fixes: 16 | - Secret variables defined in rpco specific variable 17 | files are now properly merged with any new secret 18 | variables that might be introduced in a subsequent 19 | release. 20 | -------------------------------------------------------------------------------- /releasenotes/notes/holland-venv-9eb62140a2c8ff56.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - Holland now runs inside a python virtual environment 4 | by default. By default, the virtual environment will 5 | be placed in /openstack/venvs/holland-. 6 | upgrade: 7 | - On upgrades, the holland pip packages will linger in 8 | the global environment if not cleaned up. 9 | -------------------------------------------------------------------------------- /releasenotes/notes/horizon-site-name-62c749ec2ab8b0b3.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - Added a new optional argument to the MaaS plugin 4 | horizon_check.py that takes in the name of the 5 | horizon dashboard that is used to find the login 6 | page. The default value is set to 7 | 'openstack dashboard'. 8 | -------------------------------------------------------------------------------- /releasenotes/notes/image-metadata-8d1562d924ffd7e5.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - Image metadata has been added to boost base image performance for the Images 4 | that we may provide to a customer deployment. 5 | - The metadata option `hw_scsi_model` is being passed into the images we 6 | provide by default and is set to **virtio_scsi**. This option will 7 | improve IO performance on our default images. While not required, it is 8 | recommended that the **virtio_scsi** kernel module be loaded on the host for 9 | this change to have a functional benifit. 10 | - The metadata option `hw_vif_multiqueue_enabled` is being passed into the 11 | images we provide by default and is set to **true**. This option will 12 | improve network performance on our default images when the guest OS has 13 | "multi-queuing" enabled. If the guest OS can not automatically understand 14 | "multi-queuing" enablment the user can run the following command to enable 15 | it within the guest `ethtool -L ${NIC} combined ${CPU_CORES}`. 16 | - The metadata option `hw_qemu_guest_agent` is being passsed into the images 17 | we provide by default and is set to "yes". This option will attempt to 18 | enable qemu guest agent which provides additional options to nova when 19 | managing the instance. While not required, it is recommended that the 20 | **virtio_net** kernel module be loaded on the host for this change to have a 21 | functional benifit. 22 | -------------------------------------------------------------------------------- /releasenotes/notes/maas-metric-limit-increase-fc17cbec85a7952c.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - The number of metrics that maas plugins can emit has 4 | been increased from 30 to 50, in line with new 5 | limits in the maas API. 6 | -------------------------------------------------------------------------------- /releasenotes/notes/maas-plugins-venv-4a195e0b0271bf29.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - Deployers need to be aware of upgrading from a 4 | system that isn't running the maas plugins inside 5 | of a virtualenv. They need to decide if they want 6 | the maas plugins to continue to run on the global 7 | environment or a virtual environment. 8 | fixes: 9 | - MaaS Plugins now run inside their own virtual 10 | environment 11 | -------------------------------------------------------------------------------- /releasenotes/notes/maas-version-var-2ea08457b0ad00b4.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - RPC-MaaS has been seperated out into it's own repo. This 4 | change allows maas to be developed outside of rpc and 5 | maintained as an OpenStack version agnostic deployment 6 | for all things monitoring. 7 | - A variable has been added to allow RPC-MaaS to locked 8 | down to a specific version of the repo as needed. This 9 | variable is ``maas_version`` which has a default of 10 | "master". 11 | -------------------------------------------------------------------------------- /releasenotes/notes/master-27a94947a8611804.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - f5-config.py now works with both LEM and PROD to 4 | allow for better (unified) usage. We have added 5 | the preliminary support for python3. SSL endpoints 6 | have been added as well. 7 | -------------------------------------------------------------------------------- /releasenotes/notes/master-86edb3c41de33c79.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - The new config file layout requires a manual migration 4 | of overrides from the old to the new location. Not 5 | performing this migration could cause some overrides 6 | to not be set and unknown behavior. 7 | 8 | The steps to do this are to run the migrate-yaml.py 9 | script and verify that the output is as you wish it to 10 | be to build your overrides files. You are encouraged 11 | to put rpc specific overrides in 12 | `/etc/openstack-deploy/user_rpco_variables_overrides.yml` 13 | and the openstack-ansible overrides in 14 | `/etc/openstack-deploy/user_osa_variables_overrides.yml` 15 | 16 | The secrets files will need to be combined and 17 | migrated manually to 18 | `/etc/openstack_deploy/user_rpco_secrets.yml` 19 | -------------------------------------------------------------------------------- /releasenotes/notes/master-96b4f6d34b87fe3d.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - Variables and secrets are now managed in a manner that 4 | allows downstream deployers to not have to worry about 5 | their settings being overwritten. They are provided 6 | with overrides files that they alone manage after we 7 | lay them down. We have also taken this time to rename 8 | our user_* files to separate out what specifically the 9 | variables override, whether it be upstream splitting 10 | the OSA relevant changed of the local RPC-O changes or 11 | local RPCO changes. 12 | deprecations: 13 | - We are deprecating the use of user_variables and 14 | user_secrets files other than the ones found below. 15 | 16 | user_osa_secrets.yml 17 | user_osa_variables_defaults.yml 18 | user_osa_variables_overrides.yml 19 | user_rpco_secrets.yml 20 | user_rpco_variables_defaults.yml 21 | user_rpco_variables_overrides.yml 22 | -------------------------------------------------------------------------------- /releasenotes/notes/master-b7edb49a0bbe6c2c.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - LBaaSv2 can now be configured via Horizon. 4 | - MTUs are now calculated correctly, overrides have 5 | been removed. 6 | upgrade: 7 | - The default value for the arp-responder has changed 8 | when using ML2 and the Linux Bridge agent. The 9 | logical network will now utilize traditional flood 10 | and learn through the overlay. When upgrading, 11 | existing vxlan devices will retain their old setup 12 | and be unimpacted by changes to this flag. To apply 13 | this to older devices created with the Liberty agent, 14 | the vxlan device must be removed and then the Mitaka 15 | agent restarted. The agent will recreate the vxlan 16 | devices with the current settings upon restart. 17 | - The default_subnet_pools option is now deprecated and 18 | will be removed in the Newton release. The same 19 | functionality is now provided by setting is_default 20 | attribute on subnetpools to True using the API or 21 | client. 22 | other: 23 | - Configuring bridges and IPs for overlay networks is now optional; if unset the neutron_local_ip value will default to the ansible_ssh_host value. 24 | -------------------------------------------------------------------------------- /releasenotes/notes/multi-layered-group-vars-cb60fcf454ca3bbe.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - | 4 | RPC-O previously relied on user-space configuration files 5 | to set variables used in playbooks across both 6 | OpenStack-Ansible and RPC-O playbooks. These user-space 7 | configuration files provided global overrides only and 8 | could not be scoped to specific host groups. These defaults 9 | have now been set in group_vars in the code tree, reducing 10 | the need to overwrite user-space files, making them easier 11 | to scope to host groups, and reducing their precedence level 12 | to improve future development flexibility. 13 | -------------------------------------------------------------------------------- /releasenotes/notes/nova-13.0-0c2497128e677cb2.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | security: 3 | - Live migration of nova is now ssh encrypted, using 4 | qemu+ssh instead of qemu+tcp. This behavior can be 5 | overriden by setting another uri in the variable 6 | ``live_migration_uri`` 7 | other: 8 | - Due to the introduction of the new nova api db, the 9 | variables ``nova_api_db_max_overflow``, 10 | ``nova_api_db_max_pool_size`` and 11 | ``nova_api_db_pool_timeout`` were introduced. 12 | The tweaking of these variables can be done by 13 | overriding them directly (for an api db change only), 14 | or by adapting ``db_max_overflow``, 15 | ``db_max_pool_size`` and ``db_pool_timeout`` (for a 16 | global change) 17 | -------------------------------------------------------------------------------- /releasenotes/notes/nova-allocation-ratios-869b6d3cabbf914d.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - Previously, the nova_cloud_stats maas plugin was 4 | incorrectly reporting total cpu and total memory 5 | amounts available across all hypervisors, as it was 6 | not taking into account the allocation_ratios that 7 | are set at the hypervisor level. Now, it attempts 8 | to correctly scale those values by passing in a 9 | multiplier based on the allocation ratios set in 10 | the config 11 | -------------------------------------------------------------------------------- /releasenotes/notes/override-l2pop-default-d7426c07d8b72f2e.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | issues: 3 | - The default value for `neutron_l2_population` has been 4 | overridden, which will require deployers to recreate 5 | any VXLAN networks created with `neutron_l2_population` 6 | set `False`. Not recreating these networks will lead to 7 | connectivity issues for virtual machines. 8 | upgrade: 9 | - Based on feedback from Support, the default value for 10 | `neutron_l2_population` has been overridden to `True`. 11 | -------------------------------------------------------------------------------- /releasenotes/notes/rabbitmq-ha-configuration-c5062133932bd82b.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - The HA policy for RabbitMQ is changed to replicate to two nodes, 4 | instead of three nodes, in order to increase performance and 5 | scalability of the RabbitMQ service. The override `rabbitmq_policies` 6 | set the new default value via the `group_vars/all/osa.yml` 7 | configuration. 8 | The replication count ideally resembles the number of RabbitMQ 9 | nodes necessary to maintain quorum (2 for 3 nodes, 3 for 5 nodes). 10 | -------------------------------------------------------------------------------- /releasenotes/notes/releasenotes-a800a042ad4e67cf.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | prelude: > 3 | Reno release notes will now be provided as of r14.1.0! 4 | features: 5 | - | 6 | Reno release notes will now be provided as of r14.1.0! 7 | -------------------------------------------------------------------------------- /releasenotes/notes/remove-legacy-elk-f603293888afce53.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | deprecations: 3 | - Legacy RPC-OpenStack Elasticsearch components have all been removed. These 4 | tools were no longer being maintained. All new ELK+ deployments are starting 5 | from the new upstream `elk_metrics_6x` tooling. 6 | -------------------------------------------------------------------------------- /releasenotes/notes/rename-ansible-ssh-host-7332cc551f494495.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | deprecations: 3 | - | 4 | ``ansible_ssh_host`` has been removed from Ansible 2.2 and renamed to 5 | ``ansible_host``. All references of ``ansible_ssh_host`` in RPCO have 6 | been renamed to ``ansible_host``. 7 | -------------------------------------------------------------------------------- /releasenotes/notes/rm-horizon-extensions-from-repo-server-3d441a4d0d1c6ad0.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - As the horizon-extensions repo changed in v13.0.0, operators 4 | should run ``ansible -m file -a 5 | "path=/var/www/repo/openstackgit/horizon-extensions state=absent" 6 | repo_all`` prior to upgrading to this release. 7 | -------------------------------------------------------------------------------- /releasenotes/notes/rpc-pre-upgrade-93d419ae17c5e495.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | features: 3 | - A new playbook has been created, ``pre-upgrade.yml`` to 4 | assist operators with capturing the state of an 5 | environment prior to an upgrade. 6 | upgrade: 7 | - The ``rpc_pre_upgrade.yml`` playbook performs the pre-upgrade 8 | steps as outlined in the RPC-O Maintenance Template. Output 9 | from the various steps are stored in 10 | ``$HOME/rpc13-upgrade-`` on the deployment server. 11 | These files should be closely examined prior to proceeding 12 | with the upgrade. 13 | -------------------------------------------------------------------------------- /releasenotes/notes/rpc_package_index-a03d4d131c16d710.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - A new user variable (list type) named 4 | ``repo_build_pip_extra_indexes`` was introduced 5 | to fetch pip packages from rpc repository. 6 | -------------------------------------------------------------------------------- /releasenotes/notes/security-hardening-upgrade-69d0bca94fdc7df9.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - > 4 | By default security hardening is applied on upgrade. If you wish to prevent its application you must ensure the openstack-ansible playbook security-hardening.yml 5 | skips its tasks during an upgrade by adding 'apply_security_hardening: false' to /etc/openstack_deploy/user_osa_variables_overrides.yml. 6 | -------------------------------------------------------------------------------- /releasenotes/notes/setcheckfirewallvariabletofalse-de49a72bbad0e7db.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fixes: 3 | - | 4 | The ``check_firewall`` variable is now set to false because the 5 | ``check firewall`` tasks are incompatible with Ansible 2.1.5. This 6 | fix allows for our ceph playbooks to run successfully. 7 | -------------------------------------------------------------------------------- /releasenotes/notes/turn-off-l3ha-18274d19d54a6315.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | upgrade: 3 | - | 4 | If upgrading from an environment that has L3HA on, 5 | please keep the following in mind: 6 | 7 | * HA routers, networks, and ports will linger. 8 | Manually deleting these resources could cause 9 | the router namespaces to disappear, which would 10 | in turn cause network downtime. 11 | 12 | For more information, see: 13 | `bug 1149 `_ 14 | or the known issues section in the v13.0 RPCO documentation. 15 | other: 16 | - Neutron L3HA as a default has been disabled. This has been done 17 | due to the number of bugs that still exist in that feature. 18 | The default now is to use the neutron_ha_tool for backing up 19 | l3 routers. 20 | -------------------------------------------------------------------------------- /releasenotes/notes/xtrabackup-localhost-8d1562d924ffd6e5.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | other: 3 | - The holland configuration for xtrabackup (the tool for performing mariadb database backups) is now set to perform backups against localhost in each container, as opposed to the load balanced IP address. This ensures that each of the containers is actually backing up it's own copy of the databases, which can be helpful if there are inconsistencies discovered between cluster members. 4 | -------------------------------------------------------------------------------- /releasenotes/source/_deconst.json: -------------------------------------------------------------------------------- 1 | { 2 | "contentIDBase": "https://github.com/rcbops/rpc-openstack/master/", 3 | "githubUrl": "https://github.com/rcbops/rpc-openstack/", 4 | "githubBranch": "master", 5 | "meta": { 6 | "preferGithubIssues": true 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /releasenotes/source/_static/.placeholder: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/releasenotes/source/_static/.placeholder -------------------------------------------------------------------------------- /releasenotes/source/_templates/.placeholder: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/releasenotes/source/_templates/.placeholder -------------------------------------------------------------------------------- /releasenotes/source/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Licensed under the Apache License, Version 2.0 (the "License"); 3 | # you may not use this file except in compliance with the License. 4 | # You may obtain a copy of the License at 5 | # 6 | # http://www.apache.org/licenses/LICENSE-2.0 7 | # 8 | # Unless required by applicable law or agreed to in writing, software 9 | # distributed under the License is distributed on an "AS IS" BASIS, 10 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 11 | # implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # Glance Release Notes documentation build configuration file, created by 16 | # sphinx-quickstart on Tue Nov 3 17:40:50 2015. 17 | # 18 | # This file is execfile()d with the current directory set to its 19 | # containing dir. 20 | # 21 | # Note that not all possible configuration values are present in this 22 | # autogenerated file. 23 | # 24 | # All configuration values have a default; values that are commented out 25 | # serve to show the default. 26 | 27 | from pygments.lexers.web import PhpLexer 28 | from sphinx.highlighting import lexers 29 | 30 | # If extensions (or modules to document with autodoc) are in another directory, 31 | # add these directories to sys.path here. If the directory is relative to the 32 | # documentation root, use os.path.abspath to make it absolute, like shown here. 33 | # sys.path.insert(0, os.path.abspath('.')) 34 | 35 | # -- General configuration ------------------------------------------------ 36 | 37 | # If your documentation needs a minimal Sphinx version, state it here. 38 | # needs_sphinx = '1.0' 39 | 40 | # Add any Sphinx extension module names here, as strings. They can be 41 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 42 | # ones. 43 | extensions = [ 44 | 'oslosphinx', 45 | 'reno.sphinxext', 46 | ] 47 | 48 | # Add any paths that contain templates here, relative to this directory. 49 | templates_path = ['_templates'] 50 | 51 | # The suffix of source filenames. 52 | source_suffix = '.rst' 53 | 54 | # The encoding of source files. 55 | # source_encoding = 'utf-8-sig' 56 | 57 | # The master toctree document. 58 | master_doc = 'index' 59 | 60 | # The builder to use when running via the deconst preparer 61 | # builder = 'deconst-serial' 62 | builder = 'deconst-single' 63 | 64 | # Exclude content from elastic search index 65 | deconst_default_unsearchable = True 66 | 67 | # General information about the project. 68 | project = u'Rackspace Private Cloud powered by OpenStack Release Notes' 69 | copyright = u'2016, Rackspace' 70 | 71 | # The version info for the project you're documenting, acts as replacement for 72 | # |version| and |release|, also used in various other places throughout the 73 | # built documents. 74 | # 75 | # The short X.Y version. 76 | import pbr.version 77 | version_info = pbr.version.VersionInfo('RPCO') 78 | # The full version, including alpha/beta/rc tags. 79 | release = version_info.version_string_with_vcs() 80 | # The short X.Y version. 81 | version = version_info.canonical_version_string() 82 | 83 | # The language for content autogenerated by Sphinx. Refer to documentation 84 | # for a list of supported languages. 85 | # language = None 86 | 87 | # There are two options for replacing |today|: either, you set today to some 88 | # non-false value, then it is used: 89 | # today = '' 90 | # Else, today_fmt is used as the format for a strftime call. 91 | # today_fmt = '%B %d, %Y' 92 | 93 | # List of patterns, relative to source directory, that match files and 94 | # directories to ignore when looking for source files. 95 | exclude_patterns = ['_static', '_templates'] 96 | 97 | # The reST default role (used for this markup: `text`) to use for all 98 | # documents. 99 | # default_role = None 100 | 101 | # If true, '()' will be appended to :func: etc. cross-reference text. 102 | # add_function_parentheses = True 103 | 104 | # If true, the current module name will be prepended to all description 105 | # unit titles (such as .. function::). 106 | # add_module_names = True 107 | 108 | # If true, sectionauthor and moduleauthor directives will be shown in the 109 | # output. They are ignored by default. 110 | # show_authors = False 111 | 112 | # The name of the Pygments (syntax highlighting) style to use. 113 | pygments_style = 'sphinx' 114 | 115 | # A list of ignored prefixes for module index sorting. 116 | # modindex_common_prefix = [] 117 | 118 | # If true, keep warnings as "system message" paragraphs in the built documents. 119 | # keep_warnings = False 120 | 121 | 122 | # -- Options for HTML output ---------------------------------------------- 123 | 124 | # The theme to use for HTML and HTML Help pages. See the documentation for 125 | # a list of builtin themes. 126 | # html_theme = 'sphinx_rtd_theme' 127 | 128 | # Theme options are theme-specific and customize the look and feel of a theme 129 | # further. For a list of options available for each theme, see the 130 | # documentation. 131 | # html_theme_options = {} 132 | 133 | # Add any paths that contain custom themes here, relative to this directory. 134 | # html_theme_path = [] 135 | 136 | # The name for this set of Sphinx documents. If None, it defaults to 137 | # " v documentation". 138 | # html_title = None 139 | 140 | # A shorter title for the navigation bar. Default is the same as html_title. 141 | # html_short_title = None 142 | 143 | # The name of an image file (relative to this directory) to place at the top 144 | # of the sidebar. 145 | # html_logo = None 146 | 147 | # The name of an image file (within the static path) to use as favicon of the 148 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 149 | # pixels large. 150 | # html_favicon = None 151 | 152 | # Add any paths that contain custom static files (such as style sheets) here, 153 | # relative to this directory. They are copied after the builtin static files, 154 | # so a file named "default.css" will overwrite the builtin "default.css". 155 | html_static_path = ['_static'] 156 | 157 | # Add any extra paths that contain custom files (such as robots.txt or 158 | # .htaccess) here, relative to this directory. These files are copied 159 | # directly to the root of the documentation. 160 | # html_extra_path = [] 161 | 162 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 163 | # using the given strftime format. 164 | # html_last_updated_fmt = '%b %d, %Y' 165 | 166 | # If true, SmartyPants will be used to convert quotes and dashes to 167 | # typographically correct entities. 168 | # html_use_smartypants = True 169 | 170 | # Custom sidebar templates, maps document names to template names. 171 | # html_sidebars = {} 172 | 173 | # Additional templates that should be rendered to pages, maps page names to 174 | # template names. 175 | # html_additional_pages = {} 176 | 177 | # If false, no module index is generated. 178 | # html_domain_indices = True 179 | 180 | # If false, no index is generated. 181 | # html_use_index = True 182 | 183 | # If true, the index is split into individual pages for each letter. 184 | # html_split_index = False 185 | 186 | # If true, links to the reST sources are added to the pages. 187 | # html_show_sourcelink = True 188 | 189 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 190 | # html_show_sphinx = True 191 | 192 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 193 | # html_show_copyright = True 194 | 195 | # If true, an OpenSearch description file will be output, and all pages will 196 | # contain a tag referring to it. The value of this option must be the 197 | # base URL from which the finished HTML is served. 198 | # html_use_opensearch = '' 199 | 200 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 201 | # html_file_suffix = None 202 | 203 | # Output file base name for HTML help builder. 204 | htmlhelp_basename = 'RPCO-ReleaseNotes' 205 | 206 | # this will change the 'paragraph' character to '#' 207 | html_add_permalinks = '#' 208 | 209 | # -- Options for LaTeX output --------------------------------------------- 210 | 211 | latex_elements = { 212 | # The paper size ('letterpaper' or 'a4paper'). 213 | # 'papersize': 'letterpaper', 214 | 215 | # The font size ('10pt', '11pt' or '12pt'). 216 | # 'pointsize': '10pt', 217 | 218 | # Additional stuff for the LaTeX preamble. 219 | # 'preamble': '', 220 | } 221 | 222 | # Grouping the document tree into LaTeX files. List of tuples 223 | # (source start file, target name, title, 224 | # author, documentclass [howto, manual, or own class]). 225 | latex_documents = [ 226 | ('index', 'RPCOReleaseNotes.tex', u'RPCO Release Notes Documentation', 227 | u'RPCO Developers', 'manual'), 228 | ] 229 | 230 | # The name of an image file (relative to this directory) to place at the top of 231 | # the title page. 232 | # latex_logo = None 233 | 234 | # For "manual" documents, if this is true, then toplevel headings are parts, 235 | # not chapters. 236 | # latex_use_parts = False 237 | 238 | # If true, show page references after internal links. 239 | # latex_show_pagerefs = False 240 | 241 | # If true, show URL addresses after external links. 242 | # latex_show_urls = False 243 | 244 | # Documents to append as an appendix to all manuals. 245 | # latex_appendices = [] 246 | 247 | # If false, no module index is generated. 248 | # latex_domain_indices = True 249 | 250 | 251 | # -- Options for manual page output --------------------------------------- 252 | 253 | # One entry per manual page. List of tuples 254 | # (source start file, name, description, authors, manual section). 255 | man_pages = [ 256 | ('index', 'RPCOreleasenotes', u'RPCO Release Notes Documentation', 257 | [u'RPCO Developers'], 1) 258 | ] 259 | 260 | # If true, show URL addresses after external links. 261 | # man_show_urls = False 262 | 263 | 264 | # -- Options for Texinfo output ------------------------------------------- 265 | 266 | # Grouping the document tree into Texinfo files. List of tuples 267 | # (source start file, target name, title, author, 268 | # dir menu entry, description, category) 269 | texinfo_documents = [ 270 | ('index', 'RPCOReleaseNotes', u'RPCO Release Notes Documentation', 271 | u'RPCO Developers', 'RPCOReleaseNotes', 272 | 'One line description of project.', 273 | 'Miscellaneous'), 274 | ] 275 | 276 | # Documents to append as an appendix to all manuals. 277 | # texinfo_appendices = [] 278 | 279 | # If false, no module index is generated. 280 | # texinfo_domain_indices = True 281 | 282 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 283 | # texinfo_show_urls = 'footnote' 284 | 285 | # If true, do not generate a @detailmenu in the "Top" node's menu. 286 | # texinfo_no_detailmenu = False 287 | 288 | # -- Custom options for PHP output ---------------------------------------- 289 | lexers['php'] = PhpLexer(startinline=True) 290 | -------------------------------------------------------------------------------- /releasenotes/source/deconst-requirements.txt: -------------------------------------------------------------------------------- 1 | reno 2 | oslosphinx>=2.5.0 3 | -------------------------------------------------------------------------------- /releasenotes/source/index.rst: -------------------------------------------------------------------------------- 1 | ============================================================ 2 | Rackspace Private Cloud Powered By OpenStack - Release Notes 3 | ============================================================ 4 | 5 | Rackspace Private Cloud Powered By OpenStack uses OpenStack Ansible (OSA) 6 | playbooks to deploy an OpenStack private cloud. For details 7 | about the OSA release, see `OpenStack Ansible 8 | Series Release Notes 9 | `_. 10 | 11 | .. toctree:: 12 | :maxdepth: 1 13 | 14 | master 15 | -------------------------------------------------------------------------------- /releasenotes/source/master.rst: -------------------------------------------------------------------------------- 1 | ============================ 2 | Current Series Release Notes 3 | ============================ 4 | 5 | .. release-notes:: Release Notes 6 | -------------------------------------------------------------------------------- /scripts/deploy-rpco.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # Copyright 2014-2017, Rackspace US, Inc. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | ## Shell Opts ---------------------------------------------------------------- 18 | set -eux 19 | set -o pipefail 20 | 21 | ## Vars ---------------------------------------------------------------------- 22 | 23 | export SCRIPT_PATH="$(readlink -f $(dirname ${0}))" 24 | 25 | ## Functions ----------------------------------------------------------------- 26 | source "${SCRIPT_PATH}/functions.sh" 27 | 28 | ## Main ---------------------------------------------------------------------- 29 | 30 | # Generate the secrets required for the deployment. 31 | if [[ ! -f "/etc/openstack_deploy/user_rpco_secrets.yml" ]]; then 32 | cp ${SCRIPT_PATH}/../etc/openstack_deploy/user_rpco_secrets.yml.example /etc/openstack_deploy/user_rpco_secrets.yml 33 | fi 34 | 35 | for file_name in user_secrets.yml user_rpco_secrets.yml; do 36 | if [[ -f "/etc/openstack_deploy/${file_name}" ]]; then 37 | python /opt/openstack-ansible/scripts/pw-token-gen.py --file "/etc/openstack_deploy/${file_name}" 38 | fi 39 | done 40 | 41 | if [ "${DEPLOY_AIO:-false}" != false ]; then 42 | cp "${SCRIPT_PATH}/user_aio_variables.yml" /etc/openstack_deploy/user_aio_variables.yml 43 | fi 44 | 45 | # Begin the RPC installation by uploading images and creating flavors. 46 | pushd "${SCRIPT_PATH}/../playbooks" 47 | # Create default VM images and flavors 48 | openstack-ansible site-openstack.yml 49 | 50 | # Deploy RPC operational modifications 51 | openstack-ansible site-ops.yml 52 | 53 | # Deploy logging tools 54 | openstack-ansible site-logging.yml 55 | popd 56 | 57 | if [ "${DEPLOY_MAAS}" != false ]; then 58 | pushd /opt/rpc-maas/playbooks 59 | # Deploy and configure RAX MaaS 60 | if [ "${DEPLOY_TELEGRAF}" != false ]; then 61 | # Set the rpc_maas vars. 62 | if [[ ! -f "/etc/openstack_deploy/user_rpco_maas_variables.yml" ]]; then 63 | envsubst < \ 64 | ${SCRIPT_PATH}/../etc/openstack_deploy/user_rpco_maas_variables.yml.example > \ 65 | /etc/openstack_deploy/user_rpco_maas_variables.yml 66 | fi 67 | 68 | # If influx port and IP are set enable the variable 69 | sed -i 's|^# influx_telegraf_targets|influx_telegraf_targets|g' /etc/openstack_deploy/user_rpco_maas_variables.yml 70 | fi 71 | # Run the rpc-maas setup process 72 | openstack-ansible site.yml 73 | popd 74 | fi 75 | -------------------------------------------------------------------------------- /scripts/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | ## Shell Opts ---------------------------------------------------------------- 17 | set -eux 18 | set -o pipefail 19 | 20 | ## Vars ---------------------------------------------------------------------- 21 | 22 | export SCRIPT_PATH="$(readlink -f $(dirname ${0}))" 23 | 24 | ## Functions ----------------------------------------------------------------- 25 | function exit_notice { 26 | cat "${SCRIPT_PATH}/../README.md" 27 | } 28 | 29 | function basic_install { 30 | # Run the RPC-O installation process 31 | bash -c "${SCRIPT_PATH}/install.sh" 32 | } 33 | 34 | ## Main ---------------------------------------------------------------------- 35 | 36 | # Setup OpenStack 37 | if [ "${DEPLOY_AIO:-false}" != false ]; then 38 | ## Run the basic installation script 39 | basic_install 40 | 41 | # Install OpenStack-Ansible 42 | /opt/rpc-ansible/bin/ansible-playbook \ 43 | -i 'localhost,' \ 44 | "${SCRIPT_PATH}/../playbooks/openstack-ansible-install.yml" 45 | 46 | # RO-4211 47 | # Implement debug output for apt so that we can see more information 48 | # about whether the 'Acquire-by-hash' feature is being used, and what 49 | # might be causing it to fall back to the old style. 50 | # This config file should be copied into containers by the lxc_hosts 51 | # role. 52 | echo 'Debug::Acquire::http "true";' > /etc/apt/apt.conf.d/99debug 53 | 54 | # NOTE(odyssey4me): 55 | # The test execution nodes do not have these packages installed, so 56 | # we need to do that until an upstream patch is merged and used by 57 | # RPC-O which does not require it to be installed, or installs the 58 | # required packages, before running gate-check-commit. 59 | apt-get install -y iptables util-linux 60 | 61 | ## Create the AIO 62 | pushd /opt/openstack-ansible 63 | bash -c "ANSIBLE_ROLE_FILE='/tmp/does-not-exist' scripts/gate-check-commit.sh" 64 | popd 65 | 66 | # Deploy RPC-OpenStack. 67 | bash -c "DEPLOY_AIO=true ${SCRIPT_PATH}/deploy-rpco.sh" 68 | else 69 | basic_install 70 | exit_notice 71 | fi 72 | -------------------------------------------------------------------------------- /scripts/functions.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Copyright 2014-2017 , Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | ## Vars ---------------------------------------------------------------------- 17 | # Set the DEPLOY_ variables to true to enable these services 18 | export DEPLOY_AIO=${DEPLOY_AIO:-false} 19 | export DEPLOY_MAAS=${DEPLOY_MAAS:-false} 20 | export DEPLOY_TELEGRAF=${DEPLOY_TELEGRAF:-false} 21 | export DEPLOY_INFLUX=${DEPLOY_INFLUX:-false} 22 | 23 | if [ ${DEPLOY_AIO} == true ]; then 24 | export DEPLOY_MAAS=false 25 | fi 26 | # To send data to the influxdb server, we need to deploy and configure 27 | # telegraf. By default, telegraf will use log_hosts (rsyslog hosts) to 28 | # define its influxdb servers. These playbooks need maas-get to have run 29 | # previously. 30 | # Set the following variables when when deploying maas with influx to log 31 | # to our upstream influx server. 32 | export INFLUX_IP="${INFLUX_IP:-127.0.0.1}" 33 | export INFLUX_PORT="${INFLUX_PORT:-8086}" 34 | 35 | # Set the build tag to create a unique ID within influxdb 36 | export BUILD_TAG="${BUILD_TAG:-testing}" 37 | 38 | # RPC-OpenStack product release, this variable is used in the config playbooks. 39 | export RPC_PRODUCT_RELEASE="${RPC_PRODUCT_RELEASE:-master}" 40 | 41 | # OSA release branch 42 | if [ -z ${OSA_RELEASE_BRANCH+x} ]; then 43 | if [[ "${RPC_PRODUCT_RELEASE}" != "master" ]]; then 44 | export OSA_RELEASE_BRANCH="stable/${RPC_PRODUCT_RELEASE}" 45 | else 46 | export OSA_RELEASE_BRANCH="master" 47 | fi 48 | fi 49 | 50 | # Read the OS information 51 | for rc_file in openstack-release os-release lsb-release redhat-release; do 52 | if [[ -f "/etc/${rc_file}" ]]; then 53 | source "/etc/${rc_file}" 54 | fi 55 | done 56 | 57 | # Other 58 | export BASE_DIR=${BASE_DIR:-"/opt/rpc-openstack"} 59 | export RPC_RELEASE="$(${BASE_DIR}/scripts/get-rpc_release.py -f ${BASE_DIR}/playbooks/vars/rpc-release.yml)" 60 | export OSA_RELEASE="$(${BASE_DIR}/scripts/get-rpc_release.py -f ${BASE_DIR}/playbooks/vars/rpc-release.yml -c osa)" 61 | export MAAS_RELEASE="$(${BASE_DIR}/scripts/get-rpc_release.py -f ${BASE_DIR}/playbooks/vars/rpc-release.yml -c maas)" 62 | export RPC_OS="${ID}-${VERSION_ID}-x86_64" 63 | export RPC_ANSIBLE_VERSION="2.5.5" 64 | 65 | # Validate that RPC_RELEASE is set and has a value 66 | # before continuing. If it is not, then something has 67 | # gone wrong. 68 | if [ "${RPC_RELEASE}" == "" ]; then 69 | echo "Something has gone wrong: RPC_RELEASE has no value." 70 | exit 1 71 | fi 72 | -------------------------------------------------------------------------------- /scripts/get-rpc_release.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | import argparse 17 | import os 18 | import sys 19 | import yaml 20 | 21 | 22 | # Sourced from: https://stackoverflow.com/a/10551190 23 | class EnvDefault(argparse.Action): 24 | def __init__(self, envvar, required=True, default=None, **kwargs): 25 | if envvar in os.environ: 26 | default = os.environ[envvar] 27 | if required and default: 28 | required = False 29 | super(EnvDefault, self).__init__(default=default, required=required, 30 | **kwargs) 31 | 32 | def __call__(self, parser, namespace, values, option_string=None): 33 | setattr(namespace, self.dest, values) 34 | 35 | # Setup argument parsing 36 | parser = argparse.ArgumentParser( 37 | description='Utility to retrieve the rpc_release version' 38 | ' based on the series given.', 39 | epilog='Licensed "Apache 2.0"') 40 | 41 | parser.add_argument( 42 | '-f', 43 | '--release_file', 44 | action=EnvDefault, 45 | default='/opt/rpc-openstack/playbooks/vars/rpc-release.yml', 46 | envvar='RELEASE_FILE', 47 | help='Release file path, optionally set using the RELEASE_FILE' 48 | ' environment variable.', 49 | required=False 50 | ) 51 | 52 | parser.add_argument( 53 | '-r', 54 | '--release_series', 55 | action=EnvDefault, 56 | default='master', 57 | envvar='RPC_PRODUCT_RELEASE', 58 | help='Release series, optionally set using the RPC_PRODUCT_RELEASE' 59 | 'environment variable.', 60 | required=False 61 | ) 62 | 63 | parser.add_argument( 64 | '-c', 65 | '--release_component', 66 | action=EnvDefault, 67 | default='rpc', 68 | envvar='RPC_PRODUCT_COMPONENT', 69 | help='Release component, optionally set using the RPC_PRODUCT_COMPONENT' 70 | 'environment variable.', 71 | required=False 72 | ) 73 | 74 | # Parse arguments 75 | args = parser.parse_args() 76 | 77 | # Read the file contents 78 | try: 79 | with open(args.release_file) as f: 80 | release_file_content = yaml.safe_load(f.read()) 81 | except IOError as e: 82 | print >> sys.stderr, "Unable to open file '%s'." % args.release_file 83 | sys.exit(1) 84 | 85 | # Read the series-specific data 86 | try: 87 | release_data = ( 88 | release_file_content['rpc_product_releases'][args.release_series]) 89 | except KeyError as e: 90 | print >> sys.stderr, "Unable to find release '%s'." % args.release_series 91 | sys.exit(1) 92 | 93 | # Get the current component release version 94 | component_release = release_data['%s_release' % args.release_component] 95 | 96 | # Print out the component_release value 97 | print(component_release) 98 | -------------------------------------------------------------------------------- /scripts/install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Copyright 2014-2017, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | ## Shell Opts ---------------------------------------------------------------- 17 | set -eux 18 | set -o pipefail 19 | 20 | ## Run first ------------------------------------------------------------------ 21 | # NOTE(cloudnull): Install the minimum packages required before anything can be 22 | # executed. While these should have been included in the base 23 | # kick, if they were not, they will need to be installed prior 24 | # to doing anything else. 25 | apt-get update 26 | DEBIAN_FRONTEND=noninteractive apt-get install -y \ 27 | python \ 28 | python-yaml \ 29 | python-virtualenv 30 | 31 | ## Vars ---------------------------------------------------------------------- 32 | export SCRIPT_PATH="$(readlink -f $(dirname ${0}))" 33 | 34 | ## Functions ----------------------------------------------------------------- 35 | source "${SCRIPT_PATH}/functions.sh" 36 | 37 | function install_ansible_source { 38 | DEBIAN_FRONTEND=noninteractive apt-get -y install \ 39 | gcc libssl-dev libffi-dev \ 40 | python-apt python3-apt \ 41 | python-dev python3-dev \ 42 | python-minimal python-virtualenv 43 | 44 | /opt/rpc-ansible/bin/pip install "ansible==${RPC_ANSIBLE_VERSION}" 45 | } 46 | 47 | ## Main ---------------------------------------------------------------------- 48 | # NOTE(cloudnull): Create a virtualenv for RPC-Ansible which is used for 49 | # initial bootstrap purposes. While playbooks can be run using 50 | # this ansible release in the general sense, the OSA ansible 51 | # version will superseed this globally. 52 | virtualenv /opt/rpc-ansible 53 | 54 | # Install a Ansible for RPC-OpenStack. 55 | install_ansible_source 56 | 57 | # Setup the basic release 58 | pushd "${SCRIPT_PATH}/../playbooks" 59 | /opt/rpc-ansible/bin/ansible-playbook -i 'localhost,' site-release.yml 60 | popd 61 | -------------------------------------------------------------------------------- /scripts/user_aio_variables.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | # NOTE(cloudnull): Test configs used to minimize the impact of a 4 | # multi-node install with limited resources. 5 | openstack_images: [] 6 | q_storage: 1 7 | q_mem: 512 8 | h_mem: 512 9 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = rpc-openstack 3 | summary = Role for setting up RPC OpenStack 4 | description-file = 5 | README.md 6 | author = OpenStack 7 | author-email = openstack-dev@lists.openstack.org 8 | home-page = http://docs.rackspace.com 9 | classifier = 10 | Intended Audience :: Developers 11 | Intended Audience :: System Administrators 12 | License :: OSI Approved :: Apache Software License 13 | Operating System :: POSIX :: Linux 14 | 15 | [build_sphinx] 16 | all_files = 1 17 | build-dir = doc/build 18 | source-dir = doc/source 19 | 20 | [pbr] 21 | warnerrors = True 22 | 23 | [wheel] 24 | universal = 1 25 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 12 | # implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT 17 | import setuptools 18 | 19 | # In python < 2.7.4, a lazy loading of package `pbr` will break 20 | # setuptools if some other modules registered functions in `atexit`. 21 | # solution from: http://bugs.python.org/issue15881#msg170215 22 | try: 23 | import multiprocessing # noqa 24 | except ImportError: 25 | pass 26 | 27 | setuptools.setup( 28 | setup_requires=['pbr>=2.0.0'], 29 | pbr=True) 30 | 31 | -------------------------------------------------------------------------------- /test-requirements.txt: -------------------------------------------------------------------------------- 1 | # The order of packages is significant, because pip processes them in the order 2 | # of appearance. Changing the order has an impact on the overall integration 3 | # process, which may cause wedges in the gate later. 4 | bashate>=0.2 # Apache-2.0 5 | flake8<2.6.0,>=2.5.4 # MIT 6 | pyasn1>=0.2.0,!=0.2.3 # BSD 7 | pyOpenSSL>=0.14 # Apache-2.0 8 | requests>=2.14.2 # Apache-2.0 9 | ndg-httpsclient>=0.4.2;python_version<'3.0' # BSD 10 | bandit>=1.4.0 # Apache-2.0 11 | tox>=1.3.0 # MIT 12 | openstack-requirements>=1.2.0 # Apache-2.0 13 | 14 | # this is required for the docs build jobs 15 | sphinx>=1.5.1 # BSD 16 | oslosphinx>=4.7.0 # Apache-2.0 17 | openstackdocstheme>=1.5.0 # Apache-2.0 18 | doc8>=0.8.0 # Apache-2.0 19 | reno>=2.5.1 # Apache-2.0 20 | sphinxmark>=0.1.14 # Apache-2.0 21 | osa_differ>=0.3.10 # Apache-2.0 22 | rpc_differ>=0.3.12 # Apache-2.0 23 | semver>=0.7.9 # BSD 24 | -------------------------------------------------------------------------------- /tests/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rcbops/rpc-openstack/f8fdb2ee2a7c66b3490c55513312e4fa3ff241b4/tests/.gitkeep -------------------------------------------------------------------------------- /tests/test.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2015, Rackspace US, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | - include: ../playbooks/site.yml 17 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | minversion = 2.0 3 | skipsdist = True 4 | envlist = bindep,docs,linters 5 | sitepackages = True 6 | 7 | 8 | [testenv] 9 | install_command = 10 | pip install -c{env:UPPER_CONSTRAINTS_FILE} {opts} {packages} --isolated 11 | deps = 12 | -r{toxinidir}/test-requirements.txt 13 | commands = 14 | /usr/bin/find . -type f -name "*.pyc" -delete 15 | passenv = * 16 | whitelist_externals = 17 | bash 18 | setenv = 19 | BINDEP_FILE={toxinidir}/bindep.txt 20 | PYTHONUNBUFFERED=1 21 | TEST_IDEMPOTENCE=false 22 | VIRTUAL_ENV={envdir} 23 | WORKING_DIR={toxinidir} 24 | # bug#1682108 25 | PYTHONPATH={envsitepackagesdir} 26 | # RPC product release name 27 | RPC_PRODUCT_RELEASE={env:RPC_PRODUCT_RELEASE:master} 28 | ### OSA specific call back files 29 | # Set the checkout to any supported tag, branch, or sha. 30 | # NOTE(cloudnull): This should be set to "master" as soon the gate is capable of 31 | # setting this option. 32 | OSA_RELEASE_BRANCH={env:OSA_RELEASE_BRANCH:master} 33 | OSA_TEST_RELEASE={env:OSA_TEST_RELEASE:master} 34 | OSA_UPPER_CONSTRAINTS={env:OSA_UPPER_CONSTRAINTS:master} 35 | UPPER_CONSTRAINTS_FILE=https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h={env:OSA_UPPER_CONSTRAINTS:master} 36 | OSA_TEST_DEPS=https://git.openstack.org/cgit/openstack/openstack-ansible-tests/plain/test-ansible-deps.txt?h={env:OSA_TEST_RELEASE:master} 37 | OSA_ROLE_REQUIREMENTS=https://git.openstack.org/cgit/openstack/openstack-ansible/plain/ansible-role-requirements.yml?h={env:OSA_RELEASE_BRANCH:master} 38 | basepython = python2.7 39 | 40 | [testenv:bindep] 41 | # Deps is empty on purpose 42 | deps = 43 | commands = 44 | bash -c "{toxinidir}/run-bindep.sh" 45 | 46 | [testenv:bandit] 47 | deps = -rtest-requirements.txt 48 | commands = 49 | # Ignore B404 about importing subprocess. We're knowingly and intentionally 50 | # using subprocess and don't need a warning about mere usage alone. 51 | # Specific warnings will still come through as they're found. 52 | # 53 | # Until bandit has the ability to log issues at lower severities than it's 54 | # configured to pass/fail based on, we need to run bandit twice. Run it 55 | # once with no severity or confidence level set to get everything, but ignore 56 | # that exit status. Then run it again on the highest severity and confidence 57 | # levels and get our pass/fail status from that. 58 | # This is reported to bandit at https://github.com/PyCQA/bandit/issues/341 59 | - bandit -s B404 -i -l -r playbooks/ scripts/ gating/ 60 | bandit -s B404 -iii -lll -r playbooks/ scripts/ gating/ 61 | 62 | [testenv:docs] 63 | commands = 64 | bash -c "rm -rf doc/build" 65 | doc8 doc 66 | python setup.py build_sphinx 67 | 68 | 69 | [doc8] 70 | # Settings for doc8: 71 | extensions = .rst 72 | 73 | 74 | [testenv:releasenotes] 75 | install_command = 76 | pip install -c {toxinidir}/upper-constraints.txt {opts} {packages} --isolated 77 | commands = 78 | sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html 79 | 80 | 81 | # environment used by the -infra templated docs job 82 | [testenv:venv] 83 | commands = 84 | {posargs} 85 | 86 | 87 | [testenv:tests_clone] 88 | commands = 89 | bash -c "if [ ! -d "{toxinidir}/tests/common" ]; then \ 90 | git clone https://git.openstack.org/openstack/openstack-ansible-tests {toxinidir}/tests/common; \ 91 | pushd {toxinidir}/tests/common; \ 92 | git checkout {env:OSA_TEST_RELEASE:master}; \ 93 | popd; \ 94 | else \ 95 | pushd {toxinidir}/tests/common; \ 96 | git fetch origin; \ 97 | git checkout {env:OSA_TEST_RELEASE:master}; \ 98 | popd; \ 99 | fi" 100 | 101 | 102 | [testenv:pep8] 103 | commands = 104 | {[testenv:tests_clone]commands} 105 | bash -c "{toxinidir}/tests/common/test-pep8.sh" 106 | 107 | 108 | [flake8] 109 | # Ignores the following rules due to how ansible modules work in general 110 | # F403 'from ansible.module_utils.basic import *' used; 111 | # unable to detect undefined names 112 | ignore=F403,E731 113 | 114 | 115 | [testenv:bashate] 116 | # PURPOSE: 117 | # This script executes bashate against all the files it find that match 118 | # the search pattern. The search pattern is meant to find any shell 119 | # scripts present in the role. 120 | # 121 | # The test ignores the following rules: 122 | # 123 | # E003: Indent not multiple of 4 (we prefer to use multiples of 2) 124 | # 125 | # E004: Skip adding a newline at the bottom of bash scripts 126 | # 127 | # E006: Line longer than 79 columns (as many scripts use jinja 128 | # templating, this is very difficult) 129 | # 130 | # E040: Syntax error determined using `bash -n` (as many scripts 131 | # use jinja templating, this will often fail and the syntax 132 | # error will be discovered in execution anyway) 133 | commands = 134 | bash -c "grep --recursive --binary-files=without-match \ 135 | --files-with-match '^.!.*\(ba\)\?sh$' \ 136 | --exclude-dir .tox \ 137 | --exclude-dir .git \ 138 | --exclude-dir tests \ 139 | "{toxinidir}" | xargs bashate --error . --verbose --ignore=E003,E004,E006,E040" 140 | 141 | 142 | [testenv:ansible] 143 | deps = 144 | {[testenv]deps} 145 | -r{env:OSA_TEST_DEPS} 146 | commands = 147 | bash -c "wget {env:OSA_ROLE_REQUIREMENTS} -O {toxinidir}/tests/common/osa-ansible-role-requirements.yml" 148 | ansible-galaxy install --role-file={toxinidir}/tests/common/osa-ansible-role-requirements.yml \ 149 | --roles-path={homedir}/.ansible/roles \ 150 | --force 151 | ansible-galaxy install --role-file={toxinidir}/ansible-role-{env:RPC_PRODUCT_RELEASE:master}-requirements.yml \ 152 | --roles-path={homedir}/.ansible/roles \ 153 | --force 154 | 155 | 156 | [testenv:ansible-syntax] 157 | deps = 158 | {[testenv:ansible]deps} 159 | commands = 160 | {[testenv:tests_clone]commands} 161 | {[testenv:ansible]commands} 162 | bash -c "{toxinidir}/tests/common/test-ansible-syntax.sh" 163 | 164 | 165 | [testenv:ansible-lint] 166 | deps = 167 | {[testenv:ansible]deps} 168 | commands = 169 | {[testenv:tests_clone]commands} 170 | {[testenv:ansible]commands} 171 | bash -c "{toxinidir}/tests/common/test-ansible-lint.sh" 172 | 173 | 174 | [testenv:linters] 175 | deps = 176 | {[testenv:ansible]deps} 177 | commands = 178 | {[testenv:tests_clone]commands} 179 | {[testenv:pep8]commands} 180 | {[testenv:bashate]commands} 181 | {[testenv:ansible-lint]commands} 182 | {[testenv:ansible-syntax]commands} 183 | {[testenv:docs]commands} 184 | 185 | 186 | [testenv:requirements] 187 | deps = openstack-requirements 188 | setenv = UPPER_CONSTRAINTS_FILE={toxinidir}/upper-constraints.txt 189 | commands = bash -c "generate-constraints -r {toxinidir}/test-requirements.txt -p {envdir}/bin/{basepython} > {toxinidir}/upper-constraints.txt" 190 | -------------------------------------------------------------------------------- /upper-constraints.txt: -------------------------------------------------------------------------------- 1 | chardet===3.0.4 2 | restructuredtext-lint===1.2.2 3 | typing===3.6.6 4 | bandit===1.5.1 5 | certifi===2018.11.29 6 | alabaster===0.7.12 7 | pbr===5.1.2 8 | fixtures===3.0.0 9 | sphinxcontrib-websupport===1.1.0 10 | ipaddress===1.0.22 11 | bashate===0.6.0 12 | Pillow===5.4.1 13 | python-mimeparse===1.6.0 14 | pyOpenSSL===19.0.0 15 | asn1crypto===0.24.0 16 | MarkupSafe===1.1.0 17 | doc8===0.8.0 18 | traceback2===1.4.0 19 | extras===1.0.0 20 | reno===2.11.2 21 | imagesize===1.1.0 22 | rpc-differ===0.3.12 23 | urllib3===1.24.1 24 | osa-differ===0.3.10 25 | pycparser===2.19 26 | PyYAML===3.13 27 | cryptography===2.5 28 | oslosphinx===4.18.0 29 | unittest2===1.1.0 30 | Pygments===2.3.1 31 | requests===2.21.0 32 | snowballstemmer===1.2.1 33 | Jinja2===2.10 34 | pluggy===0.8.1 35 | toml===0.10.0 36 | docutils===0.14 37 | tox===3.7.0 38 | mccabe===0.4.0 39 | enum34===1.1.6 40 | flake8===2.5.5 41 | packaging===19.0 42 | py===1.7.0 43 | testtools===2.3.0 44 | Parsley===1.3 45 | linecache2===1.0.0 46 | idna===2.8 47 | sphinxmark===0.1.19 48 | openstack-requirements===1.2.0 49 | Sphinx===1.8.4 50 | openstackdocstheme===1.29.1 51 | bottle===0.12.16 52 | pyparsing===2.3.1 53 | semver===2.8.1 54 | stevedore===1.30.0 55 | pyasn1===0.4.5 56 | pyflakes===1.0.0 57 | six===1.12.0 58 | dulwich===0.19.11 59 | pep8===1.7.1 60 | GitPython===2.1.11 61 | ndg-httpsclient===0.5.1 62 | cffi===1.11.5 63 | Babel===2.6.0 64 | smmap2===2.0.5 65 | filelock===3.0.10 66 | gitdb2===2.0.5 67 | virtualenv===16.3.0 68 | pytz===2018.9 69 | --------------------------------------------------------------------------------