├── .github └── workflows │ └── main.yml ├── LICENSE.md ├── README.md ├── ansible ├── .gitignore ├── README.md ├── Vagrantfile ├── all-initial.yml ├── all-upgrade.yml ├── ansible.cfg ├── build.sh ├── ci-deployment.yml ├── ci-docker-dev-hosts.yml ├── ci-docker-prod-hosts.yml ├── ci-jenkins-linux.yml ├── ci-playbooks │ ├── ci-initial-setup.yml │ ├── ci-setup.yml │ ├── os-ci-create-machine.yml │ ├── roles │ │ ├── ci-admin-auth │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── unix.yml │ │ ├── ci-admin-user │ │ │ ├── README.md │ │ │ ├── files │ │ │ │ └── 90-ome-admin-users │ │ │ └── tasks │ │ │ │ ├── darwin.yml │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ ├── unix.yml │ │ │ │ └── windows.yml │ │ ├── ci-base-setup │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ └── windows.yml │ │ ├── ci-data-repo │ │ │ ├── README.md │ │ │ ├── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ ├── unix.yml │ │ │ │ └── windows.yml │ │ │ └── templates │ │ │ │ ├── debian-credentials.j2 │ │ │ │ └── freebsd-nsmb.j2 │ │ ├── ci-jenkins-service │ │ │ ├── README.md │ │ │ ├── meta │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── ci-jenkins-user │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ ├── unix.yml │ │ │ │ └── windows.yml │ │ ├── ci-jenkins-workspace │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ └── main.yml │ │ ├── ci-scratch-storage │ │ │ ├── README.md │ │ │ ├── files │ │ │ │ └── win-scratch-diskpart │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ └── windows.yml │ │ ├── cygwin │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-cmake │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-cpp-libs │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ └── main.yml │ │ ├── pkg-cpp-tools │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── redhat.yml │ │ ├── pkg-devtoolset-4 │ │ │ └── tasks │ │ │ │ ├── centos.yml │ │ │ │ └── main.yml │ │ ├── pkg-doxygen │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── redhat.yml │ │ ├── pkg-git │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ ├── files │ │ │ │ └── git-installer-options.txt │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ └── windows.yml │ │ ├── pkg-java-tools │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ └── main.yml │ │ ├── pkg-jdk8 │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ └── windows.yml │ │ ├── pkg-jenkins-tools │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── redhat.yml │ │ ├── pkg-ninja │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── linux.yml │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-nssm │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-python27 │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-sphinx │ │ │ ├── README.md │ │ │ ├── meta │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-vs2015 │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-vs2017 │ │ │ ├── README.md │ │ │ ├── defaults │ │ │ │ └── main.yml │ │ │ └── tasks │ │ │ │ ├── main.yml │ │ │ │ └── windows.yml │ │ ├── pkg-zip │ │ │ ├── README.md │ │ │ └── tasks │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ └── redhat.yml │ │ ├── system-base-update │ │ │ ├── README.md │ │ │ ├── handlers │ │ │ │ ├── main.yml │ │ │ │ └── unix.yml │ │ │ └── tasks │ │ │ │ ├── darwin.yml │ │ │ │ ├── debian.yml │ │ │ │ ├── freebsd.yml │ │ │ │ ├── main.yml │ │ │ │ ├── redhat.yml │ │ │ │ └── windows.yml │ │ └── system-package-update │ │ │ ├── README.md │ │ │ └── tasks │ │ │ ├── apt.yml │ │ │ ├── homebrew.yml │ │ │ ├── main.yml │ │ │ ├── pkgng.yml │ │ │ └── yum.yml │ └── system-update.yml ├── ci-provision.yml ├── devspace.yml ├── example-hosts ├── group_vars │ ├── .gitignore │ └── all ├── hardware-dell │ └── dell-update-fc630.yml ├── host_vars │ ├── .gitignore │ └── all ├── idr-openstack-post.yml ├── idr-openstack-pre.yml ├── idr-playbooks │ └── README.md ├── idrsystems-playbooks │ ├── ansible.cfg │ ├── idr-analysis.yml │ ├── idr-docker.yml │ ├── idr-gpfs-client.yml │ ├── idr-initial.yml │ ├── idr-nfs-hosts.yml │ ├── idr-samba-hosts.yml │ ├── idr-upgrade.yml │ ├── idrsystems-deployment.yml │ ├── idrsystems-provision.yml │ └── requirements.yml ├── inventory │ ├── README.md │ ├── openstack.py │ └── variables ├── ome-analysis │ ├── ansible.cfg │ ├── initial.yml │ ├── ome-jupyter.yml │ └── requirements.yml ├── os-devspace.yml ├── os-uod-docker.yml ├── os-uod-slave.yml ├── os-uod-web.yml ├── plugins │ └── callback │ │ └── error_if_no_hosts_match.py ├── qa-web-deployment.yml ├── requirements-compat.yml ├── requirements-internal.yml ├── requirements.yml ├── roles │ ├── README.md │ ├── active-directory-join │ │ ├── README.md │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ ├── krb5-conf.j2 │ │ │ ├── security-access-conf.j2 │ │ │ └── sssd-sssd-conf.j2 │ ├── active-directory-samba-share │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ └── samba-smb-conf.j2 │ ├── ansible │ │ ├── README.tmd │ │ └── tasks │ │ │ └── main.yml │ ├── debug-dumpallvars │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ └── dumpall.j2 │ ├── dell-update │ │ ├── README.md │ │ └── tasks │ │ │ └── main.yml │ ├── devtoolset-4 │ │ ├── README.md │ │ └── tasks │ │ │ └── main.yml │ ├── docker-dns-client │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ └── systemd-docker-registrator.j2 │ ├── docker-dns-server │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ ├── systemd-docker-etcd.j2 │ │ │ └── systemd-docker-skydns.j2 │ ├── gpfs │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── files │ │ │ ├── .gitignore │ │ │ └── mmfs-bin-mmglobfuncs.Linux.patch │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ ├── gpfs-build.yml │ │ │ ├── gpfs-configure.yml │ │ │ ├── gpfs-install.yml │ │ │ ├── gpfs-repoinstall.yml │ │ │ └── main.yml │ │ └── templates │ │ │ └── gpfs-repo.j2 │ ├── jenkinsslave │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── meta │ │ │ └── main.yml │ │ ├── tasks │ │ │ ├── main.yml │ │ │ └── setup_virtualenv.yml │ │ └── templates │ │ │ └── gitconfig.j2 │ ├── network │ │ ├── README.md │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ ├── etc-sysconfig-network-scripts-ifcfg-disabled.j2 │ │ │ └── etc-sysconfig-network-scripts-ifcfg.j2 │ ├── nfs-ganesha-share │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ └── ganesha-conf.j2 │ ├── omero-build-cpp │ │ ├── README.md │ │ ├── meta │ │ │ └── main.yml │ │ └── tasks │ │ │ └── main.yml │ ├── omero-build │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ ├── meta │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── main.yml │ │ └── templates │ │ │ └── cmake-path.sh.j2 │ ├── omero-runtime │ │ ├── README.md │ │ └── meta │ │ │ └── main.yml │ ├── omero-scripts-dependencies │ │ ├── README.md │ │ └── tasks │ │ │ ├── debian.yml │ │ │ ├── main.yml │ │ │ └── redhat.yml │ ├── omero-web-apps-build-dependencies │ │ ├── README.md │ │ └── tasks │ │ │ ├── main.yml │ │ │ └── redhat.yml │ ├── openldap-devel │ │ └── tasks │ │ │ └── main.yml │ ├── openstack-prepare │ │ ├── README.md │ │ ├── handlers │ │ │ └── main.yml │ │ └── tasks │ │ │ └── main.yml │ ├── python-devel │ │ └── tasks │ │ │ └── main.yml │ ├── server-swap │ │ ├── README.md │ │ ├── defaults │ │ │ └── main.yml │ │ └── tasks │ │ │ └── main.yml │ ├── sphinx-build │ │ ├── README.md │ │ ├── meta │ │ │ └── main.yml │ │ └── tasks │ │ │ └── main.yml │ └── yum-excludes │ │ ├── README.md │ │ └── tasks │ │ └── main.yml ├── scripts │ ├── README.md │ ├── os-idr-download.sh │ ├── os-idr-mkimg.sh │ ├── os-idr-mkvol.sh │ ├── os-idr-snapshot.sh │ ├── os-idr-upload.sh │ └── settings.env ├── server-state-playbooks │ ├── .gitignore │ └── README.md ├── test.yml ├── tests │ ├── haproxy.yml │ ├── java.yml │ ├── nginx-mainline.yml │ ├── nginx.yml │ ├── omero-server-ice35.yml │ └── omero-server-ice36.yml ├── uod-docker.yml ├── uod-slave.yml └── vendor │ └── .gitignore ├── docs ├── README.md ├── ansible │ ├── ansible.md │ ├── contributing.md │ ├── example_workflows.md │ ├── galaxy_openmicroscopy.png │ └── installation.md ├── openstack │ ├── centos-image.md │ ├── ci-setup.md │ ├── freebsd-image.md │ ├── idr-openstack-setup-tenant.sh │ ├── idr-openstack-using.md │ ├── idr-openstack.md │ ├── local-setup.md │ ├── ubuntu-image.md │ └── windows-image.md ├── storage │ └── gpfs.md └── vmware │ └── mac-image.md └── requirements.txt /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | # Builds using ansible 2 | --- 3 | name: Build 4 | 5 | on: 6 | push: 7 | pull_request: 8 | schedule: 9 | - cron: '0 0 * * 0' 10 | 11 | jobs: 12 | build: 13 | strategy: 14 | matrix: 15 | python: [3.6, 3.7, 3.8, 3.9] 16 | runs-on: ubuntu-latest 17 | steps: 18 | - uses: actions/checkout@v2 19 | - name: Set up Python ${{ matrix.python }} 20 | uses: actions/setup-python@v2 21 | with: 22 | python-version: ${{ matrix.python }} 23 | - name: Install requirements 24 | run: pip install -r requirements.txt 25 | - name: Build 26 | run: | 27 | cd ansible 28 | bash build.sh -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014, Open Microscopy Environment 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | 1. Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 2. Redistributions in binary form must reproduce the above copyright notice, 10 | this list of conditions and the following disclaimer in the documentation 11 | and/or other materials provided with the distribution. 12 | 13 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR 17 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 18 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 19 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 20 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 21 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 22 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # OME Infrastructure 2 | 3 | [![Actions Status](https://github.com/ome/infrastructure/workflows/Build/badge.svg)](https://github.com/ome/infrastructure/actions) 4 | 5 | Tools, scripts and other systems infrastructure used to support the work of OME (http://www.openmicroscopy.org/). 6 | 7 | If you are interested in this work or require more information, please 8 | [contact us](http://www.openmicroscopy.org/site/community). 9 | 10 | ## Ansible 11 | 12 | OME uses Ansible extensively for deploying production services. Installation and usage instructions are 13 | provided in the [ansible README](ansible/README.md). 14 | 15 | ## OpenStack 16 | 17 | - [Server setup](docs/openstack/idr-openstack.md) outlines the 18 | installation of OpenStack which is running at the University of 19 | Dundee. 20 | - [Basic usage](docs/openstack/idr-openstack-using.md) outlines possible 21 | first steps. 22 | - [Local setup](docs/openstack/local-setup.md) details the steps to 23 | access OpenStack from a client system 24 | - [CentOS image creation](docs/openstack/centos-image.md) 25 | - [FreeBSD image creation](docs/openstack/freebsd-image.md) 26 | - [Ubuntu image creation](docs/openstack/ubuntu-image.md) 27 | - [Windows image creation](docs/openstack/windows-image.md) 28 | - [MacOS X image creation](docs/vmware/mac-image.md) 29 | - [CI setup](docs/openstack/ci-setup.md) details the steps to 30 | set up a CI node 31 | 32 | ## GPFS 33 | 34 | [GPFS.md](docs/storage/gpfs.md) provides details on the configuration of GPFS that is used 35 | at the University of Dundee. 36 | 37 | --------- 38 | 39 | For further information, you may want to read the 40 | [Contributing to OME](https://www.openmicroscopy.org/site/support/contributing/) page. 41 | -------------------------------------------------------------------------------- /ansible/.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | *.retry 3 | *.un~ 4 | -------------------------------------------------------------------------------- /ansible/Vagrantfile: -------------------------------------------------------------------------------- 1 | # This guide is optimized for Vagrant 1.7 and above. 2 | # Although versions 1.6.x should behave very similarly, it is recommended 3 | # to upgrade instead of disabling the requirement below. 4 | # 5 | # See: http://docs.ansible.com/ansible/guide_vagrant.html 6 | Vagrant.require_version ">= 1.7.0" 7 | 8 | Vagrant.configure(2) do |config| 9 | 10 | config.vm.provider "virtualbox" do |vb| 11 | vb.customize ["modifyvm", :id, "--memory", "2048"] 12 | end 13 | 14 | config.vm.provider :openstack do |os, override| 15 | override.ssh.username = "centos" 16 | # Workaround sudo restriction 17 | override.ssh.pty = true 18 | 19 | os.openstack_auth_url = "#{ENV['OS_AUTH_URL']}" 20 | os.username = "#{ENV['OS_USERNAME']}" 21 | os.password = "#{ENV['OS_PASSWORD']}" 22 | os.tenant_name = "#{ENV['OS_TENANT_NAME']}" 23 | os.region = "#{ENV['OS_REGION_NAME']}" 24 | 25 | os.flavor = 'm1.small' 26 | os.image = 'CentOS 7 1607' 27 | os.floating_ip_pool = 'external_network' 28 | end 29 | 30 | [ 31 | "cliutils", 32 | "docker", 33 | "docs", 34 | "jupyter", 35 | "postgres", 36 | "redis", 37 | "samba", 38 | "selinux", 39 | "omerowebappsbuilddependencies" 40 | ].each do |server| 41 | config.vm.define "#{server}" do |node| 42 | node.vm.box = "centos/7" 43 | node.vm.provision "ansible" do |ansible| 44 | ansible.playbook = "test.yml" 45 | end 46 | end 47 | end 48 | 49 | [ 50 | "omeroscriptsdependencies-debian" 51 | ].each do |server| 52 | config.vm.define "#{server}" do |node| 53 | node.vm.box = "ubuntu/trusty64" 54 | node.vm.provision "ansible" do |ansible| 55 | ansible.playbook = "test.yml" 56 | end 57 | end 58 | end 59 | 60 | [ 61 | "haproxy", 62 | "java", 63 | "nginx", 64 | "nginx-mainline", 65 | "omero-server-ice35", 66 | "omero-server-ice36", 67 | ].each do |server| 68 | config.vm.define "#{server}" do |node| 69 | node.vm.box = "centos/7" 70 | node.vm.provision "ansible" do |ansible| 71 | ansible.playbook = "tests/#{server}.yml" 72 | end 73 | end 74 | end 75 | 76 | # Disable the new default behavior introduced in Vagrant 1.7, to 77 | # ensure that all Vagrant machines will use the same SSH key pair. 78 | # See https://github.com/mitchellh/vagrant/issues/5005 79 | config.ssh.insert_key = false 80 | 81 | end 82 | -------------------------------------------------------------------------------- /ansible/all-initial.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Initial configuration of CI nodes 3 | # Note you should reboot after changing the network configuration, and 4 | # manually verify that the configuration is correct 5 | 6 | - hosts: all 7 | roles: 8 | - role: upgrade-distpackages 9 | - role: network 10 | -------------------------------------------------------------------------------- /ansible/all-upgrade.yml: -------------------------------------------------------------------------------- 1 | # Generic playbook for updating hosts which don't require any special tasks 2 | # For instance, this should not update excluded packages 3 | 4 | - hosts: all 5 | roles: 6 | - role: upgrade-distpackages 7 | -------------------------------------------------------------------------------- /ansible/ansible.cfg: -------------------------------------------------------------------------------- 1 | # Ansible configuration file 2 | # If ansible is executed from this directory it will automatically load 3 | # this file 4 | # See https://github.com/ansible/ansible/blob/devel/examples/ansible.cfg 5 | # for all options 6 | # 7 | # Ensure any changes to this file will work on any system (it is best 8 | # to be conservative) 9 | # Set `ANSIBLE_CONFIG` for local changes 10 | 11 | [defaults] 12 | # Try more servers at once, smartly. 13 | forks = 20 14 | gathering = smart 15 | 16 | # Assume all actions take place under infrastructure/ansible 17 | # The first path is the destination for `ansible-galaxy install` 18 | roles_path = vendor/:roles/:roles-dev/ 19 | inventory = ../../ansible/inventory/ 20 | 21 | # These tend to be annoying. 22 | retry_files_enabled = False 23 | 24 | # Deals with the files/ directory used by IDR 25 | inventory_ignore_extensions = ~, .orig, .bak, .ini, .retry, .pyc, .pyo, .html, .omero, .gif, .png, .js, .md 26 | 27 | # Helps determine what's running slowly 28 | #callback_whitelist = profile_tasks 29 | 30 | # Additional plugins 31 | callback_whitelist = ara 32 | callback_plugins = ./plugins/callback:$VIRTUAL_ENV/lib/python2.7/site-packages/ara/plugins/callbacks 33 | 34 | 35 | # Performance options 36 | [ssh_connection] 37 | #pipelining = True 38 | -------------------------------------------------------------------------------- /ansible/build.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/bash 2 | 3 | set -eux 4 | shopt -s extglob 5 | 6 | # ansible-galaxy returns 0 even if an error occurs so parse output 7 | ansible-galaxy install -r requirements.yml 2>&1 | tee galaxy-1.log 8 | grep ERROR galaxy-1.log && exit 2 9 | ansible-galaxy install -r requirements-compat.yml 2>&1 | tee galaxy-2.log 10 | grep ERROR galaxy-2.log && exit 2 11 | ansible-galaxy install -r requirements-internal.yml 2>&1 | tee galaxy-3.log 12 | grep ERROR galaxy-3.log && exit 2 13 | 14 | for f in !(requirements*).yml; do 15 | # ansible-lint $f 16 | ansible-playbook -i example-hosts --syntax-check $f 17 | done 18 | 19 | pushd idrsystems-playbooks 20 | ansible-galaxy install -r requirements.yml 2>&1 | tee galaxy-4.log 21 | grep ERROR galaxy-4.log && exit 2 22 | for f in !(requirements*).yml; do 23 | # ansible-lint $f 24 | ansible-playbook -i example-hosts --syntax-check $f 25 | done 26 | -------------------------------------------------------------------------------- /ansible/ci-deployment.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for deploying Jenkins CI nodes 3 | 4 | - hosts: ci-jenkins-linux 5 | roles: 6 | - role: jenkinsslave 7 | - role: openmicroscopy.sudoers 8 | - role: openmicroscopy.versioncontrol-utils 9 | - role: openmicroscopy.system-monitor-agent 10 | 11 | # Deploy OMERO build and runtime prerequisites 12 | - hosts: ci-omero 13 | roles: 14 | - role: omero-build 15 | - role: omero-runtime 16 | # - role: omero-build-cpp 17 | - role: openmicroscopy.postgresql 18 | postgresql_server_listen: "'*'" 19 | postgresql_server_auth: 20 | - database: all 21 | user: omero 22 | address: 0.0.0.0/0 23 | postgresql_users_databases: 24 | - user: omero 25 | password: omero 26 | databases: [omero] 27 | postgresql_server_chown_datadir: True 28 | 29 | # Deploy prerequisites to build web apps 30 | - hosts: ci-omero-web-apps-build 31 | roles: 32 | - role: omero-web-apps-build-dependencies 33 | 34 | # Deploy OMERO build and runtime prerequisites 35 | - hosts: ci-omero-web 36 | roles: 37 | - role: omero-runtime 38 | - role: openmicroscopy.omero-web-runtime 39 | - role: openmicroscopy.nginx 40 | 41 | # Deploy C++ build dependencies 42 | - hosts: cowfish.openmicroscopy.org 43 | roles: 44 | - role: devtoolset-4 45 | 46 | # Deploy Documentation prerequisites 47 | - hosts: ci-docs 48 | roles: 49 | - role: sphinx-build 50 | tags: sphinx 51 | - role: ansible 52 | -------------------------------------------------------------------------------- /ansible/ci-docker-dev-hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning Docker development nodes 3 | 4 | - hosts: ci-docker-dev-hosts 5 | roles: 6 | - role: lvm-partition 7 | lvm_lvname: root 8 | lvm_lvmount: / 9 | lvm_lvsize: "{{ rootsize }}" 10 | lvm_lvfilesystem: "{{ root_filesystem }}" 11 | - role: lvm-partition 12 | lvm_lvname: scratch 13 | lvm_lvmount: /scratch 14 | lvm_lvsize: "{{ scratch_size }}" 15 | lvm_lvfilesystem: "{{ scratch_filesystem }}" 16 | - role: basedeps 17 | # - role: system-monitor-agent 18 | # - role: versioncontrol-utils 19 | - role: docker 20 | - role: docker-dns-server 21 | - role: docker-dns-client 22 | 23 | - hosts: ci-docker-dev-hosts 24 | tasks: 25 | - file: 26 | path: /scratch 27 | mode: "1777" 28 | -------------------------------------------------------------------------------- /ansible/ci-docker-prod-hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning Docker Production nodes 3 | 4 | - hosts: ci-docker-prod-hosts 5 | roles: 6 | - role: lvm-partition 7 | lvm_lvname: root 8 | lvm_lvmount: / 9 | lvm_lvsize: "{{ rootsize }}" 10 | lvm_lvfilesystem: "{{ root_filesystem }}" 11 | - role: lvm-partition 12 | lvm_lvname: scratch 13 | lvm_lvmount: /scratch 14 | lvm_lvsize: "{{ scratch_size }}" 15 | lvm_lvfilesystem: "{{ scratch_filesystem }}" 16 | - role: basedeps 17 | - role: system-monitor-agent 18 | - role: versioncontrol-utils 19 | - role: docker 20 | 21 | - hosts: ci-docker-prod-hosts 22 | tasks: 23 | - file: 24 | path: /scratch 25 | mode: "1777" 26 | -------------------------------------------------------------------------------- /ansible/ci-jenkins-linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning Jenkins CI nodes 3 | 4 | - hosts: ci-jenkins-linux 5 | roles: 6 | # Root LV Size 7 | - role: lvm-partition 8 | lvm_lvname: root 9 | lvm_lvmount: / 10 | lvm_lvsize: "{{ provision_rootsize }}" 11 | lvm_lvfilesystem: "{{ provision_root_filesystem }}" 12 | # Jenkins work dir 13 | - role: lvm-partition 14 | lvm_lvname: jenkins-workdir 15 | lvm_lvmount: "{{ jenkinsworkdir }}" 16 | lvm_lvsize: "{{ jenkinssize }}" 17 | - role: sudoers 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/ci-initial-setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Initial setup to enable administration with ansible. 3 | 4 | - hosts: ci-jenkins-linux 5 | gather_facts: False 6 | 7 | tasks: 8 | # Python might not yet be installed (needed for ansible). 9 | - name: ci-initial-setup | Install python 10 | raw: "if test -x /usr/bin/apt; then test -e /usr/bin/python || (apt -y update && apt install -y python-minimal); fi" 11 | become: yes 12 | - name: ci-initial-setup (debian) | Install pip 13 | raw: "if test -x /usr/bin/apt; then test -e /usr/bin/pip || (apt -y update && apt install -y python-pip); fi" 14 | become: yes 15 | - name: ci-initial-setup (redhat) | Install pip 16 | raw: "if test -f /etc/redhat-release; then test -e /usr/bin/pip || (yum -y install epel-release && yum -y install python-pip); fi" 17 | become: yes 18 | # aptitude might not yet be installed (needed for package tasks). 19 | - name: ci-initial-setup | Install aptitude 20 | raw: "if test -x /usr/bin/apt; then test -e /usr/bin/aptitude || (apt -y update && apt install -y aptitude); fi" 21 | become: yes 22 | 23 | - hosts: all 24 | roles: 25 | - role: system-base-update 26 | - role: system-package-update 27 | - role: ci-base-setup 28 | - role: ci-admin-user 29 | - role: ci-admin-auth 30 | - role: ci-scratch-storage 31 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/ci-setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Set up host as a CI worker 3 | 4 | - hosts: all 5 | roles: 6 | - role: ci-base-setup 7 | - role: ci-admin-auth 8 | 9 | - hosts: role-ome-data 10 | roles: 11 | - role: ci-data-repo 12 | 13 | - hosts: role-jenkins-slave:role-java 14 | roles: 15 | - role: pkg-jdk8 16 | 17 | - hosts: role-jenkins-slave 18 | roles: 19 | - role: ci-jenkins-user 20 | - role: ci-jenkins-workspace 21 | - role: pkg-jenkins-tools 22 | - role: ci-jenkins-service 23 | 24 | - hosts: all 25 | roles: 26 | - role: pkg-git 27 | - role: pkg-zip 28 | 29 | - hosts: role-cpp 30 | roles: 31 | - role: pkg-cmake 32 | - role: pkg-ninja 33 | - role: pkg-cpp-tools 34 | 35 | - hosts: role-cpp-full 36 | roles: 37 | # Due to the big dependency collection required for doxygen, we 38 | # don't want it for minimal systems since the libraries it pulls 39 | # in may conflict with our builds. 40 | - role: pkg-doxygen 41 | - role: pkg-cpp-libs 42 | 43 | - hosts: role-java 44 | roles: 45 | - role: pkg-java-tools 46 | 47 | - hosts: role-python2 48 | roles: 49 | - role: pkg-python27 50 | 51 | - hosts: role-docs 52 | roles: 53 | - role: pkg-sphinx 54 | 55 | - hosts: role-devtoolset-4 56 | roles: 57 | - role: pkg-devtoolset-4 58 | 59 | - hosts: role-vs2015 60 | roles: 61 | - role: pkg-vs2015 62 | 63 | - hosts: role-vs2017 64 | roles: 65 | - role: pkg-vs2017 66 | 67 | - hosts: role-cygwin 68 | roles: 69 | - role: cygwin 70 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-auth/README.md: -------------------------------------------------------------------------------- 1 | CI admin auth 2 | ============= 3 | 4 | Add SSH public keys for the ci-admin user to 5 | ~ci-admin/authorized_keys. 6 | 7 | Requirements 8 | ------------ 9 | 10 | None. 11 | 12 | Author Information 13 | ------------------ 14 | 15 | ome-devel@lists.openmicroscopy.org.uk 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-auth/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: unix.yml 3 | when: ansible_os_family != 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-auth/tasks/unix.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-auth (unix) | Create .ssh 3 | file: 4 | path: ~ci-admin/.ssh 5 | state: directory 6 | owner: ci-admin 7 | group: ci-admin 8 | mode: 0755 9 | become: yes 10 | 11 | - name: ci-admin-auth (unix) | Update .ssh/authorized_keys 12 | lineinfile: 13 | path: ~ci-admin/.ssh/authorized_keys 14 | create: yes 15 | owner: ci-admin 16 | group: ci-admin 17 | mode: 0600 18 | state: "{{ item.state }}" 19 | line: "{{ item.sshpubkey }}" 20 | with_items: "{{ ci_admin_authorized_keys }}" 21 | become: yes 22 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/README.md: -------------------------------------------------------------------------------- 1 | CI admin user 2 | ============= 3 | 4 | Create admin user for ansible and other use. 5 | 6 | - FreeBSD (create, add to wheel group) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/files/90-ome-admin-users: -------------------------------------------------------------------------------- 1 | # User rules for ci-admin 2 | ci-admin ALL=(ALL) NOPASSWD:ALL 3 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/darwin.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (linux) | Add ci-admin to sudoers 3 | copy: 4 | src: 90-ome-admin-users 5 | dest: /etc/sudoers.d/90-ome-admin-users 6 | owner: root 7 | group: wheel 8 | mode: 0440 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (debian) | Add ci-admin to admin groups 3 | user: 4 | name: ci-admin 5 | groups: [adm,sudo] 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (freebsd) | Add ci-admin to wheel group 3 | user: 4 | name: ci-admin 5 | groups: [wheel] 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (linux) | Add ci-admin to sudoers 3 | copy: 4 | src: 90-ome-admin-users 5 | dest: /etc/sudoers.d/90-ome-admin-users 6 | owner: root 7 | group: root 8 | mode: 0440 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: unix.yml 3 | when: ansible_os_family != 'Windows' 4 | 5 | - include_tasks: darwin.yml 6 | when: ansible_os_family == 'Darwin' 7 | 8 | - include_tasks: linux.yml 9 | when: ansible_system == 'Linux' 10 | 11 | - include_tasks: debian.yml 12 | when: ansible_os_family == 'Debian' 13 | 14 | - include_tasks: freebsd.yml 15 | when: ansible_os_family == 'FreeBSD' 16 | 17 | - include_tasks: windows.yml 18 | when: ansible_os_family == 'Windows' 19 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/unix.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (unix) | Create ci-admin group 3 | group: 4 | name: ci-admin 5 | state: present 6 | become: yes 7 | 8 | - name: ci-admin-user (unix) | Create ci-admin user 9 | user: 10 | name: ci-admin 11 | comment: "CI System Administrator for Ansible use" 12 | group: ci-admin 13 | state: present 14 | become: yes 15 | 16 | - name: ci-admin-user (unix) | Set ci-admin user password 17 | user: 18 | name: ci-admin 19 | password: "{{ ci_admin_password_hash }}" 20 | become: yes 21 | when: ansible_os_family != 'Darwin' 22 | 23 | - name: ci-admin-user (darwin) | Set ci-admin user password 24 | user: 25 | name: ci-admin 26 | password: "{{ ci_admin_password }}" 27 | become: yes 28 | when: ansible_os_family == 'Darwin' 29 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-admin-user/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-admin-user (windows) | Create ci-admins group 3 | win_group: 4 | name: ci-admins 5 | description: "CI Administrator group" 6 | state: present 7 | 8 | - name: ci-admin-user (windows) | Create ci-admin user 9 | win_user: 10 | name: ci-admin 11 | fullname: "CI Administrator" 12 | description: "CI System Administrator for Ansible use" 13 | groups: 14 | - ci-admins 15 | - Administrators 16 | password: "{{ ci_admin_password }}" 17 | state: present 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/README.md: -------------------------------------------------------------------------------- 1 | CI base setup 2 | ============= 3 | 4 | Platform-specific basic setup tasks to bring the host to a working 5 | state to be used with ansible. 6 | 7 | - FreeBSD (install basic python2 package) 8 | 9 | Requirements 10 | ------------ 11 | 12 | None. 13 | 14 | Author Information 15 | ------------------ 16 | 17 | ome-devel@lists.openmicroscopy.org.uk 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-base-setup (debian) | Set hostname 3 | hostname: 4 | name: "{{ inventory_hostname }}" 5 | become: yes 6 | 7 | - name: ci-base-setup (debian) | Install Python build dependencies 8 | apt: 9 | name: "{{ item }}" 10 | state: present 11 | with_items: 12 | - libssl-dev 13 | - python-dev 14 | - libffi-dev 15 | become: yes 16 | when: ansible_distribution_release == 'trusty' 17 | 18 | - name: ci-base-setup (debian) | Install Python packages for SNI 19 | pip: 20 | name: "{{ item }}" 21 | with_items: 22 | - pyOpenSSL 23 | - ndg-httpsclient 24 | - pyasn1 25 | become: yes 26 | when: ansible_distribution_release == 'trusty' 27 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-base-setup (freebsd) | Set hostname 3 | lineinfile: 4 | dest: /etc/rc.conf 5 | regexp: '^hostname=' 6 | line: 'hostname="{{ inventory_hostname }}"' 7 | become: yes 8 | register: rc_hostname 9 | 10 | - name: ci-scratch-storage (freebsd) | Reload hostname 11 | service: 12 | name: hostname 13 | state: started 14 | become: yes 15 | when: rc_hostname.changed 16 | 17 | # cloudinit installs python2.7, but not the base python and python2 links. 18 | - name: ci-base-setup (freebsd) | Install python 19 | pkgng: 20 | name: python 21 | state: present 22 | become: yes 23 | 24 | - name: ci-base-setup (freebsd) | Mount fdescfs 25 | mount: 26 | path: /dev/fd 27 | src: fdesc 28 | fstype: fdescfs 29 | opts: rw 30 | state: mounted 31 | become: yes 32 | 33 | - name: ci-base-setup (freebsd) | Mount procfs 34 | mount: 35 | path: /proc 36 | src: proc 37 | fstype: procfs 38 | opts: rw 39 | state: mounted 40 | become: yes 41 | 42 | # Default CI user environment 43 | - name: ci-base-setup (freebsd) | Set CI login class 44 | blockinfile: 45 | path: /etc/login.conf 46 | marker: "# {mark} ANSIBLE MANAGED BLOCK (CI)" 47 | content: | 48 | ci:\ 49 | :umask=002:\ 50 | :path=~/bin /usr/local/bin /usr/local/sbin /bin /sbin /usr/bin /usr/sbin:\ 51 | :setenv=CMAKE_PREFIX_PATH=/usr/local:\ 52 | :charset=UTF-8:\ 53 | :lang=en_US.UTF-8:\ 54 | :tc=default: 55 | become: yes 56 | register: ci_login 57 | 58 | - name: ci-base-setup (freebsd) | Update db 59 | command: cap_mkdb /etc/login.conf 60 | become: yes 61 | when: ci_login.changed 62 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-base-setup (linux) | Create tools directory 3 | file: 4 | path: "{{ tools_location }}/bin" 5 | state: directory 6 | become: yes 7 | 8 | - name: ci-base-setup (linux) | Add tools/bin to global PATH 9 | lineinfile: 10 | dest: /etc/environment 11 | state: present 12 | backrefs: yes 13 | regexp: 'PATH=(["]*)((?!.*?/opt/tools/bin).*?)(["]*)$' 14 | line: 'PATH=\1/opt/tools/bin:\2\3' 15 | become: yes 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: linux.yml 9 | when: ansible_system == 'Linux' 10 | 11 | - include_tasks: freebsd.yml 12 | when: ansible_os_family == 'FreeBSD' 13 | 14 | - include_tasks: windows.yml 15 | when: ansible_os_family == 'Windows' 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-base-setup (redhat) | Set hostname 3 | hostname: 4 | name: "{{ inventory_hostname }}" 5 | become: yes 6 | 7 | - name: ci-base-setup (redhat) | Add PATH to /etc/environment 8 | lineinfile: 9 | dest: /etc/environment 10 | state: present 11 | create: yes 12 | regexp: '^PATH=' 13 | line: 'PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin' 14 | become: yes 15 | 16 | - name: ci-base-setup (redhat) | Install Python packages for SNI 17 | yum: 18 | name: "{{ item }}" 19 | state: latest 20 | with_items: 21 | - pyOpenSSL 22 | - python2-ndg_httpsclient 23 | - python-pyasn1 24 | become: yes 25 | when: ansible_distribution_major_version == '6' 26 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-base-setup/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-base-setup (windows) | Get domain 3 | win_shell: "net config workstation | findstr /C:\"Full Computer name\" | %{$_ -replace \"^Full Computer name *\",\"\"}" 4 | register: win_domain 5 | 6 | - name: ci-base-setup (windows) | Set domain 7 | win_shell: "netdom computername $env:computername /add:{{ inventory_hostname }}" 8 | when: (inventory_hostname != win_domain.stdout.rstrip()) 9 | 10 | - name: ci-base-setup (windows) | Set primary domain 11 | win_shell: "netdom computername $env:computername /makeprimary:{{ inventory_hostname }}" 12 | when: (inventory_hostname != win_domain.stdout.rstrip()) 13 | 14 | - name: ci-base-setup (windows) | Remove old domain 15 | win_shell: "netdom computername $env:computername /remove:{{ win_domain.stdout.rstrip() }}" 16 | when: (inventory_hostname != win_domain.stdout.rstrip()) 17 | 18 | - name: ci-base-setup (windows) | Get hostname 19 | win_command: hostname 20 | register: win_hostname 21 | 22 | - name: ci-base-setup (windows) | Set hostname 23 | win_shell: "netdom renamecomputer $env:computername /newname:{{ inventory_hostname.split('.')[0] }} /force" 24 | when: (inventory_hostname.split('.')[0] != win_hostname.stdout.rstrip()) 25 | 26 | - name: ci-base-setup (windows) | Export security policy 27 | win_command: "secedit /export /cfg {{ ansible_user_dir }}\\secpol.cfg" 28 | 29 | - name: ci-base-setup (windows) | Update security policy to relax password rules and match Unix hosts 30 | win_lineinfile: 31 | dest: "{{ ansible_user_dir }}\\secpol.cfg" 32 | regexp: '^PasswordComplexity =.*' 33 | line: 'PasswordComplexity = 0' 34 | register: secpol 35 | 36 | - name: ci-base-setup (windows) | Import security policy 37 | win_command: "secedit /configure /db c:\\windows\\security\\local.sdb /cfg {{ ansible_user_dir }}\\secpol.cfg /areas SECURITYPOLICY" 38 | when: secpol.changed 39 | 40 | - name: ci-base-setup (windows) | Delete exported security policy 41 | win_file: 42 | path: "{{ ansible_user_dir }}\\secpol.cfg" 43 | state: absent 44 | 45 | - name: ci-base-setup (windows) | Reboot 46 | win_reboot: 47 | when: (inventory_hostname != win_domain.stdout.rstrip() or inventory_hostname.split('.')[0] != win_hostname.stdout.rstrip()) 48 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/README.md: -------------------------------------------------------------------------------- 1 | CI data repository 2 | ================== 3 | 4 | Create ci-data user to own the data on the local mount. Mount 5 | ome-data-repo using the read-only hslave-x account. 6 | 7 | Requirements 8 | ------------ 9 | 10 | None. 11 | 12 | Author Information 13 | ------------------ 14 | 15 | ome-devel@lists.openmicroscopy.org.uk 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-repo (debian) | Install CIFS support 3 | apt: 4 | name: cifs-utils 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-repo (freebsd) | Create nsmb credentials 3 | become: yes 4 | template: 5 | src: freebsd-nsmb.j2 6 | dest: "/etc/nsmb.conf" 7 | force: yes 8 | mode: 0600 9 | 10 | - name: ci-data-repo (freebsd) | Mount data repo 11 | mount: 12 | path: /data/ome-data-repo 13 | src: //hslave-x@FS0/ome-data-repo 14 | fstype: smbfs 15 | opts: ro,late,-N,-uci-data,-gci-data 16 | state: mounted 17 | become: yes 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-repo (linux) | Create CIFS credentials 3 | become: yes 4 | template: 5 | src: debian-credentials.j2 6 | dest: "/etc/fstab-fs0-hslave-x.cred" 7 | force: yes 8 | mode: 0600 9 | 10 | - name: ci-data-repo (linux) | Mount data repo 11 | mount: 12 | path: /data/ome-data-repo 13 | src: //fs0.lifesci.dundee.ac.uk/ome-data-repo 14 | fstype: cifs 15 | opts: ro,uid=ci-data,gid=ci-data,credentials=/etc/fstab-fs0-hslave-x.cred 16 | state: mounted 17 | become: yes 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: unix.yml 3 | when: ansible_os_family != 'Windows' 4 | 5 | - include_tasks: debian.yml 6 | when: ansible_os_family == 'Debian' 7 | 8 | - include_tasks: redhat.yml 9 | when: ansible_os_family == 'RedHat' 10 | 11 | - include_tasks: linux.yml 12 | when: ansible_system == 'Linux' 13 | 14 | - include_tasks: freebsd.yml 15 | when: ansible_os_family == 'FreeBSD' 16 | 17 | - include_tasks: windows.yml 18 | when: ansible_os_family == 'Windows' 19 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-repo (redhat) | Install CIFS support 3 | yum: 4 | name: cifs-utils 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/unix.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-user | Create ci-data group 3 | group: 4 | name: ci-data 5 | state: present 6 | become: yes 7 | 8 | - name: ci-data-user | Create ci-data user 9 | user: 10 | name: ci-data 11 | comment: "CI Jenkins worker" 12 | group: ci-data 13 | password: "{{ ci_jenkins_password_hash }}" 14 | state: present 15 | become: yes 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-data-repo (windows) | Mount data repo 3 | win_shell: | 4 | $User = "LIFESCI-AD\hslave-x" 5 | $PWord = ConvertTo-SecureString -String "{{ ci_hslave_x_password }}" -AsPlainText -Force 6 | $Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $User, $PWord 7 | New-PSDrive -Name O -PSProvider FileSystem -Root \\fs0.lifesci.dundee.ac.uk\ome-data-repo -Credential $Credential -Persist 8 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/templates/debian-credentials.j2: -------------------------------------------------------------------------------- 1 | username=hslave-x 2 | password={{ ci_hslave_x_password }} 3 | domain=LIFESCI-AD 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-data-repo/templates/freebsd-nsmb.j2: -------------------------------------------------------------------------------- 1 | # smbfs configuration 2 | 3 | [default] 4 | workgroup=LIFESCI-AD 5 | 6 | [FS0] 7 | addr=fs0.lifesci.dundee.ac.uk 8 | 9 | [FS0:HSLAVE-X] 10 | password={{ ci_hslave_x_password }} 11 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-service/README.md: -------------------------------------------------------------------------------- 1 | CI jenkins service 2 | ================== 3 | 4 | Create jenkins service for Jenkins slave use. 5 | 6 | - Windows (create service) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-service/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: pkg-nssm } -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-service/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: windows.yml 3 | when: ansible_os_family == 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-service/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-jenkins-service (windows) | Remove Jenkins service 3 | win_nssm: 4 | name: jenkins 5 | state: absent 6 | environment: 7 | PATH: "{{ tools_location }}\\sbin;{{ ansible_env.Path }}" 8 | 9 | # Download after the service is stopped, or the file will be in use. 10 | - name: ci-jenkins-service (windows) | Download the Jenkins slave jar 11 | win_get_url: 12 | url: "http://{{ jenkins_master }}:8080/jnlpJars/slave.jar" 13 | dest: "{{ jenkins_location }}\\slave.jar" 14 | 15 | - name: ci-jenkins-service (windows) | Change owner of Jenkins slave jar 16 | win_owner: 17 | path: "{{ jenkins_location }}\\slave.jar" 18 | user: "{{ ci_jenkins_user }}" 19 | 20 | # win_nssm had quoting issues, so run by hand. 21 | - name: ci-jenkins-service (windows) | Add Jenkins service 22 | win_command: "{{ tools_location }}\\sbin\\nssm.exe install jenkins \\\"C:\\Program Files\\ojdkbuild\\java-1.8.0-openjdk-1.8.0.121-2\\bin\\java.exe\\\" -jar \"{{ jenkins_location }}\\slave.jar\" -jnlpUrl http://{{ jenkins_master }}:8080/computer/{{ jenkins_slave.name }}/slave-agent.jnlp -secret {{ jenkins_slave.authtoken }}" 23 | 24 | - name: ci-jenkins-service (windows) | Set Jenkins service startup directory 25 | win_command: "{{ tools_location }}\\sbin\\nssm.exe set jenkins AppDirectory \"{{ jenkins_location }}\"" 26 | 27 | - name: ci-jenkins-service (windows) | Set Jenkins service user 28 | win_command: "{{ tools_location }}\\sbin\\nssm.exe set jenkins ObjectName .\\{{ ci_jenkins_user }} {{ ci_jenkins_password }}" 29 | 30 | - name: ci-jenkins-service (windows) | Set Jenkins environment 31 | win_command: "{{ tools_location }}\\sbin\\nssm.exe set jenkins AppEnvironmentExtra \\\"PATH=C:\\Program Files\\Git\\bin;C:\\Program Files\\CMake\\bin;{{ tools_location }}\\bin;{{ ansible_env.Path }}\\\"" 32 | 33 | - name: ci-jenkins-service (windows) | Start Jenkins service 34 | win_service: 35 | name: jenkins 36 | start_mode: auto 37 | state: started 38 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-user/README.md: -------------------------------------------------------------------------------- 1 | CI jenkins user 2 | =============== 3 | 4 | Create jenkins user for Jenkins slave use. 5 | 6 | - FreeBSD (create user) 7 | - Windows (create user) 8 | 9 | Requirements 10 | ------------ 11 | 12 | None. 13 | 14 | Author Information 15 | ------------------ 16 | 17 | ome-devel@lists.openmicroscopy.org.uk 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-user/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: unix.yml 3 | when: ansible_os_family != 'Windows' 4 | 5 | - include_tasks: windows.yml 6 | when: ansible_os_family == 'Windows' 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-user/tasks/unix.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-jenkins-user (unix) | Create ci-jenkins group 3 | group: 4 | name: "{{ ci_jenkins_group }}" 5 | state: present 6 | become: yes 7 | 8 | - name: ci-jenkins-user (unix) | Create ci-jenkins user 9 | user: 10 | name: "{{ ci_jenkins_user }}" 11 | comment: "CI Jenkins worker" 12 | group: "{{ ci_jenkins_group }}" 13 | login_class: ci 14 | password: "{{ ci_jenkins_password_hash }}" 15 | state: present 16 | become: yes 17 | 18 | - name: ci-jenkins-user (unix) | Add SSH key 19 | authorized_key: 20 | user: "{{ ci_jenkins_user }}" 21 | state: present 22 | key: "{{ ci_jenkins_ssh_key }}" 23 | become: yes 24 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-user/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-jenkins-user (windows) | Add Jenkins user 3 | win_user: 4 | name: "{{ ci_jenkins_user }}" 5 | password: "{{ ci_jenkins_password }}" 6 | description: Jenkins build slave 7 | state: present 8 | groups: 9 | - Users 10 | 11 | - name: ci-jenkins-user (windows) | Create jenkins directory 12 | win_file: 13 | path: "{{ jenkins_location }}" 14 | state: directory 15 | 16 | # Could recurse, but it's currently broken 17 | # https://github.com/ansible/ansible-modules-extras/issues/2932 18 | - name: Change owner of Jenkins directory 19 | win_owner: 20 | path: "{{ jenkins_location }}" 21 | user: "{{ ci_jenkins_user }}" 22 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-workspace/README.md: -------------------------------------------------------------------------------- 1 | CI jenkins workspace 2 | ==================== 3 | 4 | Create workspace for Jenkins slave use. 5 | 6 | - FreeBSD (create ZFS datasets) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-workspace/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-jenkins-workspace (freebsd) | Add jenkins dataset 3 | zfs: 4 | name: scratch/jenkins 5 | state: present 6 | compression: lz4 7 | devices: off 8 | setuid: off 9 | atime: off 10 | sync: disabled 11 | become: yes 12 | 13 | - name: ci-jenkins-workspace (freebsd) | Add workspace dataset 14 | zfs: 15 | name: scratch/jenkins/workspace 16 | state: present 17 | compression: lz4 18 | devices: off 19 | setuid: off 20 | atime: off 21 | sync: disabled 22 | become: yes 23 | 24 | - name: ci-jenkins-workspace (freebsd) | Set jenkins dataset ownership 25 | file: 26 | path: /scratch/jenkins 27 | owner: "{{ ci_jenkins_user }}" 28 | group: "{{ ci_jenkins_group }}" 29 | mode: 0775 30 | become: yes 31 | 32 | - name: ci-jenkins-workspace (freebsd) | Set workspace dataset ownership 33 | file: 34 | path: /scratch/jenkins/workspace 35 | owner: "{{ ci_jenkins_user }}" 36 | group: "{{ ci_jenkins_group }}" 37 | mode: 0775 38 | become: yes 39 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-workspace/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-jenkins-workspace (linux) | Add jenkins logical volume 3 | lvol: 4 | vg: scratch 5 | lv: scratch 6 | size: 64512 7 | become: yes 8 | 9 | - name: ci-jenkins-workspace (linux) | Create scratch filesystem 10 | filesystem: 11 | # Because it's a scratch filesytem with no important data on it, 12 | # disable journalling for performance. 13 | fstype: ext4 14 | dev: /dev/scratch/scratch 15 | opts: -O ^has_journal 16 | become: yes 17 | 18 | - name: ci-data-repo (linux) | Mount data repo 19 | mount: 20 | path: /scratch 21 | src: /dev/scratch/scratch 22 | fstype: ext4 23 | opts: rw,noatime 24 | state: mounted 25 | become: yes 26 | 27 | - name: ci-jenkins-workspace (linux) | Create /scratch/jenkins/workspace 28 | file: 29 | path: /scratch/jenkins/workspace 30 | state: directory 31 | mode: 0755 32 | become: yes 33 | 34 | - name: ci-jenkins-workspace (linux) | Set scratch directory ownership 35 | file: 36 | path: /scratch 37 | owner: root 38 | group: root 39 | mode: 0755 40 | become: yes 41 | 42 | - name: ci-jenkins-workspace (linux) | Set jenkins directory ownership 43 | file: 44 | path: /scratch/jenkins 45 | owner: "{{ ci_jenkins_user }}" 46 | group: "{{ ci_jenkins_group }}" 47 | mode: 0775 48 | become: yes 49 | 50 | - name: ci-jenkins-workspace (linux) | Set workspace directory ownership 51 | file: 52 | path: /scratch/jenkins/workspace 53 | owner: "{{ ci_jenkins_user }}" 54 | group: "{{ ci_jenkins_group }}" 55 | mode: 0775 56 | become: yes 57 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-jenkins-workspace/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: linux.yml 3 | when: ansible_system == 'Linux' 4 | 5 | - include_tasks: freebsd.yml 6 | when: ansible_os_family == 'FreeBSD' 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/README.md: -------------------------------------------------------------------------------- 1 | CI scratch storage 2 | ================== 3 | 4 | Create scratch storage volume. 5 | 6 | - FreeBSD (create ZFS pool) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/files/win-scratch-diskpart: -------------------------------------------------------------------------------- 1 | select disk 1 2 | online disk 3 | attributes disk clear readonly 4 | clean 5 | convert gpt 6 | create partition primary 7 | format quick fs=ntfs label="Scratch space" 8 | assign letter="S:" 9 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-scratch-storage (debian) | Install LVM 3 | apt: 4 | name: lvm2 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-scratch-storage (freebsd) | Check vtbd1 disklabel 3 | command: gpart list vtbd1 4 | ignore_errors: yes 5 | register: vtbd1_exists 6 | 7 | # Create disklabel if it doesn't exist 8 | - name: ci-scratch-storage (freebsd) | Create vtbd1 disklabel 9 | command: gpart create -s GPT vtbd1 10 | become: yes 11 | when: vtbd1_exists.rc == 1 12 | 13 | - name: ci-scratch-storage (freebsd) | Check vtbd1p1 partition 14 | command: test -c /dev/vtbd1p1 15 | ignore_errors: yes 16 | register: vtbd1p1_exists 17 | 18 | # Create partition if it doesn't exist 19 | - name: ci-scratch-storage (freebsd) | Create vtbd1p1 partition 20 | command: gpart add -a 4k -t freebsd-zfs -l scratch0 vtbd1 21 | become: yes 22 | when: vtbd1p1_exists.rc == 1 23 | 24 | - name: ci-scratch-storage (freebsd) | Set up ZFS at boot 25 | lineinfile: 26 | dest: /etc/rc.conf 27 | regexp: '^zfs_enable' 28 | line: 'zfs_enable="YES"' 29 | become: yes 30 | register: rc_zfs 31 | 32 | - name: ci-scratch-storage (freebsd) | Reload zfs 33 | service: 34 | name: zfs 35 | state: reloaded 36 | become: yes 37 | when: rc_zfs.changed 38 | 39 | - name: ci-scratch-storage (freebsd) | Set up ZFS 4k ashift 40 | lineinfile: 41 | dest: /etc/sysctl.conf 42 | regexp: '^vfs.zfs.min_auto_ashift=' 43 | line: 'vfs.zfs.min_auto_ashift=12' 44 | become: yes 45 | register: sysctl 46 | 47 | - name: ci-scratch-storage (freebsd) | Reload sysctls 48 | service: 49 | name: sysctl 50 | state: reloaded 51 | become: yes 52 | when: sysctl.changed 53 | 54 | - name: ci-scratch-storage (freebsd) | Check scratch pool exists 55 | command: zpool list scratch 56 | ignore_errors: yes 57 | register: scratch_pool_exists 58 | 59 | # Create pool if it doesn't exist 60 | - name: ci-scratch-storage (freebsd) | Create scratch pool 61 | command: zpool create -m /scratch scratch /dev/vtbd1p1 62 | become: yes 63 | when: scratch_pool_exists.rc == 1 64 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-scratch-storage (linux) | Create volume group 3 | lvg: 4 | pvs: /dev/vdb 5 | state: present 6 | vg: scratch 7 | become: yes 8 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: linux.yml 9 | when: ansible_system == 'Linux' 10 | 11 | - include_tasks: freebsd.yml 12 | when: ansible_os_family == 'FreeBSD' 13 | 14 | - include_tasks: windows.yml 15 | when: ansible_os_family == 'Windows' 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-scratch-storage (redhat) | Install LVM 3 | yum: 4 | name: lvm2 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/ci-scratch-storage/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ci-scratch-storage (windows) | Check that the drive S exists 3 | win_stat: path='S:/' 4 | register: stat_drive 5 | 6 | - name: ci-scratch-storage (windows) | Create diskpart script 7 | win_copy: 8 | src: win-scratch-diskpart 9 | dest: "{{ ansible_user_dir }}\\scratch-diskpart" 10 | when: stat_drive.stat.exists == False 11 | 12 | - name: ci-scratch-storage (windows) | Create S drive 13 | win_command: "diskpart /s {{ ansible_user_dir }}\\scratch-diskpart" 14 | when: stat_drive.stat.exists == False 15 | 16 | - name: ci-scratch-storage (windows) | Delete diskpart script 17 | win_file: 18 | path: "{{ ansible_user_dir }}\\scratch-diskpart" 19 | state: absent 20 | when: stat_drive.stat.exists == False 21 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/cygwin/README.md: -------------------------------------------------------------------------------- 1 | Package: cygwin 2 | =============== 3 | 4 | Install cygwin (Windows only) 5 | 6 | Requirements 7 | ------------ 8 | 9 | None. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/cygwin/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | cmake: 3 | shortversion: "3.8" 4 | version: "3.8.1" 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/cygwin/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: windows.yml 3 | when: ansible_os_family == 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/cygwin/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: cygwin (windows) | Download Cygwin installer 3 | win_get_url: 4 | url: 'https://cygwin.org/setup-x86_64.exe' 5 | dest: "{{ ansible_user_dir }}\\Downloads\\cygwin-setup.exe" 6 | 7 | - name: cygwin (windows) | Install and/or update Cygwin 8 | win_command: "{{ ansible_user_dir }}\\Downloads\\cygwin-setup.exe -s https://www.mirrorservice.org/sites/sourceware.org/pub/cygwin -q -R C:\\cygwin -l {{ ansible_user_dir }}\\Downloads\\cygwin-packages -N -d -g -o -f -P bash,binutils,bzip2,cmake,coreutils,curl,cygwin-devel,dash,diffutils,findutils,file,gawk,gcc-core,gcc-g++,grep,groff,gzip,hostname,less,make,man-db,openssh,openssl,patch,pkg-config,rsync,sed,tar,unzip,util-linux,w32api-headers,w32api-runtime,xz,zip,libcurl-devel,libicu-devel" 9 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/README.md: -------------------------------------------------------------------------------- 1 | Package: cmake 2 | ============= 3 | 4 | Install cmake 5 | 6 | - FreeBSD (install cmake) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | cmake: 3 | shortversion: "3.8" 4 | version: "3.8.1" 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: cmake (debian) | Install CMake 3 | apt: 4 | name: cmake 5 | state: absent 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: cmake (freebsd) | Install CMake 3 | pkgng: 4 | name: cmake 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: cmake (linux) | Check CMake installation 3 | stat: 4 | path: "/opt/cmake-{{ cmake.version }}/bin/cmake" 5 | register: cmake_installed 6 | 7 | - name: cmake (linux) | Download CMake 8 | get_url: 9 | url: "https://cmake.org/files/v{{ cmake.shortversion }}/cmake-{{ cmake.version }}-Linux-x86_64.tar.gz" 10 | dest: /tmp 11 | register: cmake_download 12 | when: cmake_installed.stat.exists == false 13 | 14 | - name: ci-jenkins-workspace (debian) | Create /scratch/jenkins/workspace 15 | file: 16 | path: "/opt/cmake-{{ cmake.version }}" 17 | state: directory 18 | owner: root 19 | group: root 20 | mode: 0755 21 | become: yes 22 | 23 | - name: cmake (linux) | Extract CMake 24 | unarchive: 25 | src: "{{ cmake_download.dest }}" 26 | dest: "/opt/cmake-{{ cmake.version }}" 27 | remote_src: yes 28 | extra_opts: ['--strip-components=1'] 29 | when: cmake_installed.stat.exists == false 30 | become: yes 31 | 32 | - name: cmake (linux) | Link CMake 33 | file: 34 | src: "/opt/cmake-{{ cmake.version }}" 35 | dest: /opt/cmake 36 | state: link 37 | become: yes 38 | 39 | - name: cmake (linux) | Add to tools 40 | file: 41 | src: "/opt/cmake/bin/{{ item }}" 42 | dest: "{{ tools_location }}/bin/{{ item }}" 43 | state: link 44 | with_items: 45 | - ccmake 46 | - cmake 47 | - cmake-gui 48 | - cpack 49 | - ctest 50 | become: yes 51 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: linux.yml 3 | when: ansible_system == 'Linux' 4 | 5 | - include_tasks: freebsd.yml 6 | when: ansible_os_family == 'FreeBSD' 7 | 8 | - include_tasks: windows.yml 9 | when: ansible_os_family == 'Windows' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cmake/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: cmake (windows) | Check if CMake is installed 3 | win_stat: path='C:/Program Files/CMake/bin/cmake.exe' 4 | register: cmake_installed 5 | 6 | - name: cmake (windows) | Download CMake installer 7 | win_get_url: 8 | url: 'https://cmake.org/files/v{{ cmake.shortversion }}/cmake-{{ cmake.version }}-win64-x64.msi' 9 | dest: "{{ ansible_user_dir }}\\Downloads\\cmake-{{ cmake.version }}-win64-x64.msi" 10 | when: not cmake_installed.stat.exists 11 | 12 | - name: cmake (windows) | Install CMake 13 | win_msi: 14 | creates: 'C:\Program Files\CMake\bin\cmake' 15 | extra_args: "ADD_CMAKE_TO_PATH=None ALL_USERS=1" 16 | path: "{{ ansible_user_dir }}\\Downloads\\cmake-{{ cmake.version }}-win64-x64.msi" 17 | state: present 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-libs/README.md: -------------------------------------------------------------------------------- 1 | Package: cpp-libs 2 | ================= 3 | 4 | Install C++ libraries needed as build dependencies. 5 | 6 | - FreeBSD (install various libraries) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-libs/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-cpp-libs (debian) | Install C++ core libraries 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - libboost-all-dev 8 | - libgmp-dev 9 | - libhdf5-dev 10 | - libicu-dev 11 | - libxalan-c-dev 12 | - libxerces-c-dev 13 | - zlib1g-dev 14 | - libgtest-dev 15 | become: yes 16 | 17 | - name: pkg-cpp-libs (debian) | Install C++ image libraries 18 | apt: 19 | name: "{{ item }}" 20 | state: present 21 | with_items: 22 | - libgif-dev 23 | - jasper 24 | - libjbig-dev 25 | - libjpeg-turbo8-dev 26 | - libmng-dev 27 | - libpng-dev 28 | - libtiff5-dev 29 | become: yes 30 | 31 | - name: pkg-cpp-libs (debian) | Install C++ image libraries (xenial) 32 | apt: 33 | name: "{{ item }}" 34 | state: present 35 | with_items: 36 | - libopenjp2-7-dev 37 | become: yes 38 | when: ansible_distribution_release != 'trusty' 39 | 40 | - name: pkg-cpp-libs (debian) | Install C++ serialisation libraries 41 | apt: 42 | name: "{{ item }}" 43 | state: present 44 | with_items: 45 | - libsqlite3-dev 46 | - libyaml-cpp-dev 47 | become: yes 48 | 49 | - name: pkg-cpp-libs (debian) | Install C++ Qt and OpenGL libraries 50 | apt: 51 | name: "{{ item }}" 52 | state: present 53 | with_items: 54 | - qt5-default 55 | - libqt5opengl5-dev 56 | - libqt5svg5-dev 57 | - libglm-dev 58 | become: yes 59 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-libs/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-cpp-libs (freebsd) | Install C++ core libraries 3 | pkgng: 4 | name: boost-all,gmp,hdf5,icu,xalan-c,xerces-c3,googletest 5 | state: present 6 | become: yes 7 | 8 | - name: pkg-cpp-libs (freebsd) | Install C++ image libraries 9 | pkgng: 10 | name: giflib,jasper,jbigkit,jpeg-turbo,libmng,openjpeg,png,tiff 11 | state: present 12 | become: yes 13 | 14 | - name: pkg-cpp-libs (freebsd) | Install C++ serialisation libraries 15 | pkgng: 16 | name: sqlite3,yaml-cpp 17 | state: present 18 | become: yes 19 | 20 | - name: pkg-cpp-libs (freebsd) | Install C++ Qt and OpenGL libraries 21 | pkgng: 22 | name: qt5,glm 23 | state: present 24 | become: yes 25 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-libs/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: freebsd.yml 6 | when: ansible_os_family == 'FreeBSD' 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-tools/README.md: -------------------------------------------------------------------------------- 1 | Package: cpp-tools 2 | ================== 3 | 4 | Install C++ tools, including 5 | 6 | - GNU make 7 | - GNU binutils 8 | - GNU autotools 9 | 10 | - FreeBSD (install tools) 11 | 12 | Requirements 13 | ------------ 14 | 15 | None. 16 | 17 | Author Information 18 | ------------------ 19 | 20 | ome-devel@lists.openmicroscopy.org.uk 21 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-tools/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-cpp-tools (debian) | Install C++ build tools 3 | apt: 4 | name: build-essential 5 | state: present 6 | become: yes 7 | 8 | - name: pkg-cpp-tools (debian) | Install GNU Autotools 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: 13 | - autoconf 14 | - automake 15 | - libtool 16 | become: yes 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-tools/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-cpp-tools (freebsd) | Install C++ build tools 3 | pkgng: 4 | name: binutils,gmake 5 | state: present 6 | become: yes 7 | 8 | - name: pkg-cpp-tools (freebsd) | Install GNU Autotools 9 | pkgng: 10 | name: autoconf,automake,libtool,m4 11 | state: present 12 | become: yes 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-tools/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-cpp-tools/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-cpp-tools (redhat) | Install C++ build tools 3 | yum: 4 | name: '@Development tools' 5 | state: latest 6 | become: yes 7 | 8 | - name: pkg-cpp-tools (redhat) | Install GNU Autotools 9 | yum: 10 | name: "{{ item }}" 11 | state: latest 12 | with_items: 13 | - autoconf 14 | - automake 15 | - libtool 16 | become: yes 17 | 18 | - name: pkg-cpp-tools (redhat) | Enable SCL repository 19 | yum: 20 | name: centos-release-scl-rh 21 | state: latest 22 | become: yes 23 | 24 | - name: pkg-cpp-tools (redhat) | Install Devtoolset-4 25 | yum: 26 | name: devtoolset-4 27 | state: latest 28 | become: yes 29 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-devtoolset-4/tasks/centos.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: devtoolset-4 (redhat) | Install devtoolset-4 3 | yum: 4 | name: "{{ item }}" 5 | state: latest 6 | with_items: 7 | - centos-release-scl 8 | - devtoolset-4 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-devtoolset-4/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: centos.yml 3 | when: ansible_distribution == 'CentOS' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-doxygen/README.md: -------------------------------------------------------------------------------- 1 | Package: doxygen 2 | ============= 3 | 4 | Install doxygen 5 | 6 | - FreeBSD (install doxygen) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-doxygen/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: doxygen (debian) | Install Doxygen 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - doxygen 8 | - graphviz 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-doxygen/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: doxygen (freebsd) | Install Doxygen 3 | pkgng: 4 | name: doxygen,graphviz 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-doxygen/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-doxygen/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: doxygen (redhat) | Install Doxygen 3 | yum: 4 | name: "{{ item }}" 5 | state: latest 6 | with_items: 7 | - doxygen 8 | - graphviz 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/README.md: -------------------------------------------------------------------------------- 1 | Package: git 2 | ============= 3 | 4 | Install git 5 | 6 | - FreeBSD (install git) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | git: 3 | windows: 4 | version: "2.12.2" 5 | build: "1" 6 | installer_options: "{{ ansible_user_dir }}\\Downloads\\git-installer-options.txt" 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/files/git-installer-options.txt: -------------------------------------------------------------------------------- 1 | [Setup] 2 | Lang=default 3 | Dir=C:\Program Files\Git 4 | Group=Git 5 | NoIcons=0 6 | SetupType=default 7 | Components=ext,ext\shellhere,ext\guihere,assoc,assoc_sh 8 | Tasks= 9 | PathOption=BashOnly 10 | SSHOption=OpenSSH 11 | CURLOption=OpenSSL 12 | CRLFOption=LFOnly 13 | BashTerminalOption=MinTTY 14 | PerformanceTweaksFSCache=Enabled 15 | UseCredentialManager=Enabled 16 | EnableSymlinks=Disabled 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: git (debian) | Install Git 3 | apt: 4 | name: git 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: git (freebsd) | Install Git 3 | pkgng: 4 | name: git 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | 11 | - include_tasks: windows.yml 12 | when: ansible_os_family == 'Windows' 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: git (redhat) | Install Git 3 | yum: 4 | name: git 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-git/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-git (windows) | Check if git is installed 3 | win_stat: path='C:/Program Files/Git/bin/git.exe' 4 | register: git_installed 5 | 6 | - name: pkg-git (windows) | Download git installer 7 | win_get_url: 8 | url: 'https://github.com/git-for-windows/git/releases/download/v{{ git.windows.version }}.windows.{{ git.windows.build }}/Git-{{ git.windows.version }}-64-bit.exe' 9 | dest: "{{ ansible_user_dir }}\\Downloads\\Git-{{ git.windows.version }}-64-bit.exe" 10 | when: not git_installed.stat.exists 11 | 12 | - name: pkg-git (windows) | Copy git installer configuration 13 | win_copy: 14 | src: git-installer-options.txt 15 | dest: "{{ git.windows.installer_options }}" 16 | when: not git_installed.stat.exists 17 | 18 | - name: pkg-git (windows) | Install git 19 | raw: "{{ ansible_user_dir }}\\Downloads\\Git-{{ git.windows.version }}-64-bit.exe /silent /suppressmsgboxes /loadinf={{ git.windows.installer_options }}" 20 | when: not git_installed.stat.exists 21 | 22 | - name: pkg-git (windows) | Delete git installer configuration 23 | win_file: 24 | path: "{{ git.windows.installer_options }}" 25 | state: absent 26 | when: not git_installed.stat.exists 27 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/README.md: -------------------------------------------------------------------------------- 1 | Package: java-tools 2 | ================== 3 | 4 | Install Java tools, including 5 | 6 | - Ant 7 | - Maven 8 | - JUnit 9 | - FindBugs 10 | 11 | - FreeBSD (install tools) 12 | 13 | Requirements 14 | ------------ 15 | 16 | None. 17 | 18 | Author Information 19 | ------------------ 20 | 21 | ome-devel@lists.openmicroscopy.org.uk 22 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ant: 3 | version: "1.10.1" 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-java-tools (debian) | Install Java build tools 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - ant 8 | - maven 9 | become: yes 10 | 11 | - name: pkg-java-tools (debian) | Install Java testing tools 12 | apt: 13 | name: "{{ item }}" 14 | state: present 15 | with_items: 16 | - findbugs 17 | - junit4 18 | become: yes 19 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-java-tools (freebsd) | Install Java build tools 3 | pkgng: 4 | name: apache-ant,maven 5 | state: present 6 | become: yes 7 | 8 | - name: pkg-java-tools (freebsd) | Install Java testing tools 9 | pkgng: 10 | name: java-findbugs,junit 11 | state: present 12 | become: yes 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ant (linux) | Check Ant installation 3 | stat: 4 | path: "/opt/ant-{{ ant.version }}/bin/ant" 5 | register: ant_installed 6 | 7 | - name: ant (linux) | Download Ant 8 | get_url: 9 | url: "http://mirror.ox.ac.uk/sites/rsync.apache.org/ant/binaries/apache-ant-{{ ant.version}}-bin.tar.xz" 10 | checksum: "sha256:51dd6b4ec740013dc5ad71812ce5d727a9956aa3a56de7164c76cbd70d015d79" 11 | dest: /tmp 12 | register: ant_download 13 | when: ant_installed.stat.exists == false 14 | 15 | - name: ant (linux) | Create Ant directory 16 | file: 17 | path: "/opt/ant-{{ ant.version }}" 18 | state: directory 19 | owner: root 20 | group: root 21 | mode: 0755 22 | become: yes 23 | 24 | - name: ant (linux) | Extract Ant 25 | unarchive: 26 | src: "{{ ant_download.dest }}" 27 | dest: "/opt/ant-{{ ant.version }}" 28 | remote_src: yes 29 | extra_opts: ['--strip-components=1'] 30 | when: ant_installed.stat.exists == false 31 | become: yes 32 | 33 | - name: ant (linux) | Link Ant 34 | file: 35 | src: "/opt/ant-{{ ant.version }}" 36 | dest: /opt/ant 37 | state: link 38 | become: yes 39 | 40 | - name: ant (linux) | Add to tools 41 | file: 42 | src: "/opt/ant/bin/ant" 43 | dest: "{{ tools_location }}/bin/ant" 44 | state: link 45 | become: yes 46 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-java-tools/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: linux.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/README.md: -------------------------------------------------------------------------------- 1 | Package: jdk8 2 | ============= 3 | 4 | Install jdk8 5 | 6 | - FreeBSD (install openjdk8) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | jdk8: 3 | windows: 4 | version: "1.8.0" 5 | fullversion: "1.8.0.121-2" 6 | build: "13" 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - apt_key: 3 | keyserver: keyserver.ubuntu.com 4 | id: DA1A4A13543B466853BAF164EB9B1D8886F44E2A 5 | state: present 6 | keyring: /etc/apt/trusted.gpg.d/openjdk-r-ppa.gpg 7 | when: ansible_distribution_release == 'trusty' 8 | become: yes 9 | 10 | - name: pkg-jdk8 (debian) | Add JDK8 PPA 11 | apt_repository: 12 | repo: 'deb http://ppa.launchpad.net/openjdk-r/ppa/ubuntu trusty main' 13 | state: present 14 | when: ansible_distribution_release == 'trusty' 15 | become: yes 16 | 17 | - name: pkg-jdk8 (debian) | Install JDK8 18 | apt: 19 | name: "{{ item }}" 20 | state: present 21 | with_items: 22 | - java-common 23 | - openjdk-8-jdk 24 | - openjdk-8-jre 25 | become: yes 26 | 27 | - name: pkg-jdk8 (debian) | Set default JDK 28 | command: 'update-java-alternatives -s java-1.8.0-openjdk-amd64' 29 | become: Yes 30 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jdk8 (freebsd) | Install JDK8 3 | pkgng: 4 | name: openjdk8 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | 11 | - include_tasks: windows.yml 12 | when: ansible_os_family == 'Windows' 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jdk8 (redhat) | Install JDK8 3 | yum: 4 | name: java-1.8.0-openjdk-devel 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jdk8/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jdk8 (windows) | Check to see if java is installed 3 | win_stat: path="C:/Program Files/ojdkbuild/java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}/bin/java.exe" 4 | register: java_installed 5 | 6 | - name: pkg-jdk8 (windows) | Download the Java 8 installer 7 | win_get_url: 8 | # URL from https://github.com/ojdkbuild/ojdkbuild 9 | url: "https://github.com/ojdkbuild/ojdkbuild/releases/download/{{ jdk8.windows.fullversion }}/java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}.b{{ jdk8.windows.build }}.ojdkbuild.windows.x86_64.msi" 10 | dest: "{{ ansible_user_dir }}\\Downloads\\java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}.b{{ jdk8.windows.build }}.ojdkbuild.windows.x86_64.msi" 11 | when: not java_installed.stat.exists 12 | 13 | - name: pkg-jdk8 (windows) | Install Java 8 14 | win_msi: 15 | creates: "C:\Program Files\\ojdkbuild\\java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}\\bin\\java.exe" 16 | path: "{{ ansible_user_dir }}\\Downloads\\java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}.b{{ jdk8.windows.build }}.ojdkbuild.windows.x86_64.msi" 17 | state: present 18 | when: not java_installed.stat.exists 19 | 20 | - name: pkg-jdk8 (windows) | Set JAVA_HOME 21 | win_environment: 22 | state: present 23 | name: JAVA_HOME 24 | value: "C:\\Program Files\\ojdkbuild\\java-{{ jdk8.windows.version }}-openjdk-{{ jdk8.windows.fullversion }}" 25 | level: machine 26 | register: java_path 27 | 28 | - name: ci-base-setup (windows) | Reboot 29 | win_reboot: 30 | when: java_path.changed 31 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jenkins-tools/README.md: -------------------------------------------------------------------------------- 1 | Package: jenkins-tools 2 | ================== 3 | 4 | Install tools needed by Jenkins, including 5 | 6 | - Shells (bash) 7 | - Archival tools (tar, gzip, bzip2, lzma, xz) 8 | - Download and upload tools (curl, wget, rsync) 9 | - Miscellaneous tools (fakeroot) 10 | 11 | - FreeBSD (install tools) 12 | 13 | Requirements 14 | ------------ 15 | 16 | None. 17 | 18 | Author Information 19 | ------------------ 20 | 21 | ome-devel@lists.openmicroscopy.org.uk 22 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jenkins-tools/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jenkins-tools (debian) | Install download and upload tools 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - curl 8 | - wget 9 | - rsync 10 | become: yes 11 | 12 | - name: pkg-jenkins-tools (debian) | Install miscellaneous tools 13 | apt: 14 | name: fakeroot 15 | state: present 16 | become: yes 17 | 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jenkins-tools/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jenkins-tools (freebsd) | Install shells 3 | pkgng: 4 | name: bash 5 | state: present 6 | become: yes 7 | 8 | - name: pkg-jenkins-tools (freebsd) | Install archival tools 9 | pkgng: 10 | name: gtar 11 | state: present 12 | become: yes 13 | 14 | - name: pkg-jenkins-tools (freebsd) | Install download and upload tools 15 | pkgng: 16 | name: curl,wget,rsync 17 | state: present 18 | become: yes 19 | 20 | - name: pkg-jenkins-tools (freebsd) | Install miscellaneous tools 21 | pkgng: 22 | name: fakeroot 23 | state: present 24 | become: yes 25 | 26 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jenkins-tools/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-jenkins-tools/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: pkg-jenkins-tools (redhat) | Install download and upload tools 3 | yum: 4 | name: "{{ item }}" 5 | state: latest 6 | with_items: 7 | - curl 8 | - wget 9 | - rsync 10 | become: yes 11 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/README.md: -------------------------------------------------------------------------------- 1 | Package: ninja 2 | ============== 3 | 4 | Install ninja 5 | 6 | - FreeBSD (install ninja) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ninja: 3 | windows: 4 | version: "1.7.2" 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ninja (debian) | Install Ninja 3 | apt: 4 | name: ninja-build 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ninja (freebsd) | Install Ninja 3 | pkgng: 4 | name: ninja 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/tasks/linux.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ninja (linux) | Check if Ninja is installed 3 | stat: path="{{ tools_location }}/bin/ninja" 4 | register: ninja_installed 5 | 6 | - name: ninja (linux) | Download Ninja 7 | get_url: 8 | url: 'https://github.com/ninja-build/ninja/releases/download/v{{ ninja.windows.version }}/ninja-linux.zip' 9 | checksum: 'sha256:38fa8cfb9c1632a5cdf7a32fe1a7c5aa89e96c1d492c28624f4cc018e68458b9' 10 | dest: /tmp 11 | register: ninja_download 12 | when: ninja_installed.stat.exists == false 13 | 14 | - name: ninja (linux) | Extract Ninja 15 | unarchive: 16 | src: "{{ ninja_download.dest }}" 17 | dest: "{{ tools_location }}/bin" 18 | remote_src: yes 19 | when: ninja_installed.stat.exists == false 20 | become: yes 21 | 22 | - name: ninja (linux) | Set Ninja permissions 23 | file: 24 | path: "{{ tools_location }}/bin/ninja" 25 | mode: 0755 26 | when: ninja_installed.stat.exists == false 27 | become: yes 28 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: linux.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | 11 | - include_tasks: windows.yml 12 | when: ansible_os_family == 'Windows' 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-ninja/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ninja (windows) | Check if Ninja is installed 3 | win_stat: path="{{ tools_location_norm }}/bin/ninja.exe" 4 | register: ninja_installed 5 | 6 | - name: ninja (windows) | Create tools directory 7 | win_file: 8 | path: "{{ tools_location }}\\bin" 9 | state: directory 10 | 11 | - name: ninja (windows) | Download Ninja 12 | win_get_url: 13 | url: 'https://github.com/ninja-build/ninja/releases/download/v{{ ninja.windows.version }}/ninja-win.zip' 14 | dest: "{{ ansible_user_dir }}\\Downloads\\ninja-win.zip" 15 | when: not ninja_installed.stat.exists 16 | 17 | - name: ninja (windows) | Extract Ninja 18 | win_unzip: 19 | src: "{{ ansible_user_dir }}\\Downloads\\ninja-win.zip" 20 | dest: "{{ tools_location }}\\bin" 21 | creates: "{{ tools_location }}\\bin\\ninja.exe" 22 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-nssm/README.md: -------------------------------------------------------------------------------- 1 | Package: nssm 2 | ============== 3 | 4 | Install nssm (service manager for Windows) 5 | 6 | - windows (install nssm) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-nssm/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: windows.yml 3 | when: ansible_os_family == 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-nssm/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: nssm (windows) | Check if nssm is installed 3 | win_stat: path="{{ tools_location }}/sbin/nssm.exe" 4 | register: nssm_installed 5 | 6 | - name: nssm (windows) | Create sbin directory 7 | win_file: 8 | path: "{{ tools_location }}\\sbin" 9 | state: directory 10 | 11 | - name: nssm (windows) | Download nssm 12 | win_get_url: 13 | url: 'https://nssm.cc/release/nssm-2.24.zip' 14 | dest: "{{ ansible_user_dir }}\\Downloads\\nssm-2.24.zip" 15 | when: not nssm_installed.stat.exists 16 | 17 | - name: nssm (windows) | Extract nssm 18 | win_unzip: 19 | src: "{{ ansible_user_dir }}\\Downloads\\nssm-2.24.zip" 20 | dest: "{{ ansible_user_dir }}\\Downloads" 21 | creates: "{{ ansible_user_dir }}\\Downloads\\nssm-2.24\\win64\\nssm.exe" 22 | when: not nssm_installed.stat.exists 23 | 24 | - name: nssm (windows) | Copy nssm 25 | win_copy: 26 | src: "{{ ansible_user_dir }}\\Downloads\\nssm-2.24\\win64\\nssm.exe" 27 | dest: "{{ tools_location }}\\sbin\\nssm.exe" 28 | remote_src: yes 29 | when: not nssm_installed.stat.exists 30 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/README.md: -------------------------------------------------------------------------------- 1 | Package: python27 2 | ================= 3 | 4 | Install Python 2.7 and virtualenv 5 | 6 | - FreeBSD (install python27, py27-virtualenv) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | python27: 3 | windows: 4 | version: "2.7.13" 5 | installer_options: "ALLUSERS=1 ADDLOCAL=DefaultFeature,SharedCRT,Extensions,TclTk,Documentation,Tools,pip_feature,Testsuite" 6 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: python27 (debian) | Install Python 2.7 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - python2.7 8 | - python-virtualenv 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: python27 (freebsd) | Install Python 2.7 3 | pkgng: 4 | name: python27,py27-virtualenv 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: freebsd.yml 6 | when: ansible_os_family == 'FreeBSD' 7 | 8 | - include_tasks: windows.yml 9 | when: ansible_os_family == 'Windows' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-python27/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: python27 (windows) | Check if Python 2.7 is installed 3 | win_stat: path='C:/Python27/python.exe' 4 | register: python_installed 5 | 6 | - name: python27 (windows) | Download the Python 2.7 installer 7 | win_get_url: 8 | url: 'https://www.python.org/ftp/python/{{ python27.windows.version }}/python-{{ python27.windows.version }}.amd64.msi' 9 | dest: "{{ ansible_user_dir }}\\Downloads\\python-{{ python27.windows.version }}.amd64.msi" 10 | when: not python_installed.stat.exists 11 | 12 | - name: python27 (windows) | Install MSI 13 | win_msi: 14 | creates: 'C:\Python27\python.exe' 15 | extra_args: "{{ python27.windows.installer_options }}" 16 | path: "{{ ansible_user_dir }}\\Downloads\\python-{{ python27.windows.version }}.amd64.msi" 17 | state: present 18 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/README.md: -------------------------------------------------------------------------------- 1 | Package: sphinx 2 | =============== 3 | 4 | Install the Sphinx documentation generator 5 | 6 | - FreeBSD (install sphinx) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: pkg-python27 } -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: sphinx (debian) | Install Sphinx 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | python-sphinx 8 | become: yes 9 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: sphinx (freebsd) | Install Sphinx 3 | pkgng: 4 | name: py27-sphinx 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: freebsd.yml 6 | when: ansible_os_family == 'FreeBSD' 7 | 8 | - include_tasks: windows.yml 9 | when: ansible_os_family == 'Windows' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-sphinx/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: sphinx (windows) | Check if Python 2.7 is installed 3 | win_stat: path='C:/Python27/scripts/pip2.exe' 4 | register: pip2_installed 5 | 6 | - name: sphinx (windows) Install Sphinx 7 | win_shell: 'C:\Python27\scripts\pip2.exe sphinx' 8 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2015/README.md: -------------------------------------------------------------------------------- 1 | Package: vs2015 2 | =============== 3 | 4 | Install Visual Studio 2015 Build Tools (Windows only) 5 | 6 | Requirements 7 | ------------ 8 | 9 | None. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2015/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ninja: 3 | windows: 4 | version: "1.7.2" 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2015/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: windows.yml 3 | when: ansible_os_family == 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2015/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: vs2015 (windows) | Check if Visual Studio 2015 Build Tools are installed 3 | win_stat: path='C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/vcvarsall.bat' 4 | register: vs2015_installed 5 | 6 | - name: vs2015 (windows) | Download the Visual Studio 2015 Build Tools installer 7 | win_get_url: 8 | url: 'http://go.microsoft.com/fwlink/?LinkId=691126&__hstc=268264337.1d17b8336c63a9a2e9dd4ba1dbfb08f5.1495812655625.1495812655625.1495812655625.1&__hssc=268264337.3.1495812655626&__hsfp=3622320289&fixForIE=.exe' 9 | dest: "{{ ansible_user_dir }}\\Downloads\\vs_buildtools-2015.exe" 10 | when: not vs2015_installed.stat.exists 11 | 12 | - name: vs2015 (windows) | Install Visual Studio 2015 Build Tools 13 | raw: "{{ ansible_user_dir }}\\Downloads\\vs_buildtools-2015.exe /silent /norestart /SuppressRefreshPrompt" 14 | when: not vs2015_installed.stat.exists 15 | 16 | - name: ci-base-setup (windows) | Reboot 17 | win_reboot: 18 | when: not vs2015_installed.stat.exists 19 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2017/README.md: -------------------------------------------------------------------------------- 1 | Package: vs2017 2 | =============== 3 | 4 | Install Visual Studio 2017 Build Tools (Windows only) 5 | 6 | Requirements 7 | ------------ 8 | 9 | None. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2017/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ninja: 3 | windows: 4 | version: "1.7.2" 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2017/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: windows.yml 3 | when: ansible_os_family == 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-vs2017/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: vs2017 (windows) | Check if Visual Studio 2017 Build Tools are installed 3 | win_stat: path='C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Auxiliary/Build/vcvarsall.bat' 4 | register: vs2017_installed 5 | 6 | - name: vs2017 (windows) | Download the Visual Studio 2017 Build Tools installer 7 | win_get_url: 8 | url: 'https://aka.ms/vs/15/release/vs_buildtools.exe' 9 | dest: "{{ ansible_user_dir }}\\Downloads\\vs_buildtools-2017.exe" 10 | when: not vs2017_installed.stat.exists 11 | 12 | - name: vs2017 (windows) | Install Visual Studio 2017 Build Tools 13 | raw: "{{ ansible_user_dir }}\\Downloads\\vs_buildtools-2017.exe --quiet --wait --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended" 14 | when: not vs2017_installed.stat.exists 15 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-zip/README.md: -------------------------------------------------------------------------------- 1 | Package: zip 2 | ============ 3 | 4 | Install zip and unzip 5 | 6 | - FreeBSD (install zip) 7 | 8 | Requirements 9 | ------------ 10 | 11 | None. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-zip/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: zip (debian) | Install zip and unzip 3 | apt: 4 | name: "{{ item }}" 5 | state: present 6 | with_items: 7 | - zip 8 | - unzip 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-zip/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: zip (freebsd) | Install zip and unzip 3 | pkgng: 4 | name: zip 5 | state: present 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-zip/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: debian.yml 3 | when: ansible_os_family == 'Debian' 4 | 5 | - include_tasks: redhat.yml 6 | when: ansible_os_family == 'RedHat' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/pkg-zip/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: zip (redhat) | Install zip and unzip 3 | yum: 4 | name: "{{ item }}" 5 | state: latest 6 | with_items: 7 | - zip 8 | - unzip 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/README.md: -------------------------------------------------------------------------------- 1 | System base update 2 | ================== 3 | 4 | Update the base operating system components. Supports the native 5 | update mechanisms for: 6 | 7 | - FreeBSD (freebsd-update) 8 | - Linux/RedHat (yum) 9 | - Linux/Debian (apt) 10 | - MacOS X (softwareupdate) 11 | - Windows 12 | 13 | Note that on systems with a package manager, this may update packages 14 | in addition to the base system. 15 | 16 | Requirements 17 | ------------ 18 | 19 | None. 20 | 21 | Author Information 22 | ------------------ 23 | 24 | ome-devel@lists.openmicroscopy.org.uk 25 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: unix.yml 3 | when: ansible_os_family != 'Windows' 4 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/handlers/unix.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: reboot 3 | command: shutdown -r now "ansible triggered restart" 4 | become: yes 5 | 6 | - name: wait for restart 7 | local_action: 8 | module: wait_for 9 | host: "{{ inventory_hostname }}" 10 | port: 22 11 | delay: 30 12 | timeout: 320 13 | state: started 14 | 15 | - name: wait for ssh 16 | local_action: 17 | module: pause 18 | seconds: 5 19 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/darwin.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-base-update (darwin) | Update base system 3 | command: softwareupdate -i -a 4 | become: yes 5 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-base-update (debian) | Update packages 3 | apt: 4 | update_cache: yes 5 | become: yes 6 | 7 | - name: system-base-update (debian) | Upgrade packages 8 | apt: 9 | upgrade: dist 10 | become: yes 11 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/freebsd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-base-update (freebsd) | Fetch new FreeBSD updates 3 | command: freebsd-update fetch --not-running-from-cron 4 | register: result_update 5 | changed_when: "'No updates needed' not in result_update.stdout" 6 | become: yes 7 | 8 | - name: system-base-update (freebsd) | Install FreeBSD updates 9 | command: freebsd-update install 10 | when: ansible_distribution == 'FreeBSD' and result_update.changed 11 | register: result_update_install 12 | become: yes 13 | notify: 14 | - reboot 15 | - wait for restart 16 | - wait for ssh 17 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: darwin.yml 3 | when: ansible_os_family == 'Darwin' 4 | 5 | - include_tasks: debian.yml 6 | when: ansible_os_family == 'Debian' 7 | 8 | - include_tasks: freebsd.yml 9 | when: ansible_os_family == 'FreeBSD' 10 | 11 | - include_tasks: redhat.yml 12 | when: ansible_os_family == 'RedHat' 13 | 14 | - include_tasks: windows.yml 15 | when: ansible_os_family == 'Windows' 16 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-base-update (redhat) | Upgrade all packages 3 | yum: 4 | name: '*' 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-base-update/tasks/windows.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Install all security, critical, and rollup updates 3 | - name: system-base-update (windows) | Install Windows updates 4 | win_updates: 5 | category_names: 6 | - SecurityUpdates 7 | - CriticalUpdates 8 | - UpdateRollups 9 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/README.md: -------------------------------------------------------------------------------- 1 | System package update 2 | ===================== 3 | 4 | Update packages with the system package manager. Supports the 5 | following systems: 6 | 7 | - FreeBSD (pkgng) 8 | - Linux/RedHat (yum) 9 | - Linux/Debian (apt) 10 | - MacOS X (homebrew) 11 | 12 | Requirements 13 | ------------ 14 | 15 | None. 16 | 17 | Author Information 18 | ------------------ 19 | 20 | ome-devel@lists.openmicroscopy.org.uk 21 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/tasks/apt.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-package-update (apt) | Update package list 3 | apt: 4 | update_cache: yes 5 | become: yes 6 | 7 | - name: system-package-update (apt) | Upgrade packages 8 | apt: 9 | upgrade: safe 10 | become: yes 11 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/tasks/homebrew.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-package-update (homebrew) | Check homebrew installation 3 | stat: 4 | path: "/usr/local/bin/brew" 5 | register: brew_installed 6 | 7 | - name: system-package-update (homebrew) | Install homebrew 8 | shell: '/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"' 9 | become: true 10 | become_user: "{{homebrew_user}}" 11 | when: brew_installed.stat.exists == false 12 | 13 | - name: system-package-update (homebrew) | Update and upgrade packages 14 | homebrew: 15 | update_homebrew: yes 16 | upgrade_all: yes 17 | become: true 18 | become_user: "{{homebrew_user}}" 19 | 20 | - name: system-package-update (homebrew) | Clean up 21 | command: brew cleanup 22 | become: true 23 | become_user: "{{homebrew_user}}" 24 | environment: 25 | PATH: "/usr/local/bin:/usr/local/sbin:{{ ansible_env.PATH }}" 26 | 27 | - name: system-package-update (homebrew) | Prune stale links 28 | command: brew prune 29 | become: true 30 | become_user: "{{homebrew_user}}" 31 | environment: 32 | PATH: "/usr/local/bin:/usr/local/sbin:{{ ansible_env.PATH }}" 33 | 34 | - name: system-package-update (homebrew) | Check for problems 35 | command: brew doctor 36 | become: true 37 | become_user: "{{homebrew_user}}" 38 | environment: 39 | PATH: "/usr/local/bin:/usr/local/sbin:{{ ansible_env.PATH }}" 40 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: homebrew.yml 3 | when: (ansible_pkg_mgr is defined and ansible_pkg_mgr == 'homebrew') or ansible_os_family == 'Darwin' 4 | 5 | - include_tasks: apt.yml 6 | when: ansible_pkg_mgr is defined and ansible_pkg_mgr == 'apt' 7 | 8 | - include_tasks: pkgng.yml 9 | when: ansible_pkg_mgr is defined and ansible_pkg_mgr == 'pkgng' 10 | 11 | - include_tasks: yum.yml 12 | when: ansible_pkg_mgr is defined and ansible_pkg_mgr == 'yum' 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/tasks/pkgng.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-package-update (freebsd) | Update package list 3 | command: pkg update -q 4 | become: yes 5 | 6 | - name: system-package-update (freebsd) | Upgrade packages 7 | command: pkg upgrade -y 8 | become: yes 9 | 10 | - name: system-package-update (freebsd) | Clean package cache 11 | command: pkg clean -qy 12 | become: yes 13 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/roles/system-package-update/tasks/yum.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: system-package-update (yum) | Upgrade all packages 3 | yum: 4 | name: '*' 5 | state: latest 6 | become: yes 7 | -------------------------------------------------------------------------------- /ansible/ci-playbooks/system-update.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Apply operating system and package updates 3 | 4 | - hosts: all 5 | roles: 6 | - role: system-base-update 7 | - role: system-package-update 8 | -------------------------------------------------------------------------------- /ansible/ci-provision.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for running all CI provisioning playbooks 3 | 4 | # Initial networking and basic system configuration 5 | # A reboot may be required after this playbook 6 | - include: all-initial.yml 7 | 8 | # Jenkins CI servers 9 | - include: ci-jenkins-linux.yml 10 | 11 | # Docker CI hosts 12 | - include: ci-docker-dev-hosts.yml 13 | 14 | # Docker production hosts 15 | - include: ci-docker-prod-hosts.yml 16 | -------------------------------------------------------------------------------- /ansible/devspace.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning Devspace 3 | 4 | - hosts: devspace 5 | 6 | vars: 7 | # Overrides the default value in the docker role 8 | docker_use_ipv4_nic_mtu: True 9 | 10 | vars_prompt: 11 | - name: "devspace_jenkins_password" 12 | prompt: "Secure Jenkins with admin password (username: devspace)" 13 | private: yes 14 | 15 | pre_tasks: 16 | 17 | - fail: 18 | msg: "Jenkins admin password is required" 19 | when: devspace_jenkins_password is undefined or not devspace_jenkins_password 20 | 21 | roles: 22 | - role: openmicroscopy.devspace 23 | -------------------------------------------------------------------------------- /ansible/example-hosts: -------------------------------------------------------------------------------- 1 | # Hosts can be part of multiple hostgroups, but note that vars will be 2 | # read for all groups that the host is a member of 3 | 4 | # Jenkins CI slave nodes 5 | [ci-jenkins-linux] 6 | localhost 7 | -------------------------------------------------------------------------------- /ansible/group_vars/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | -------------------------------------------------------------------------------- /ansible/group_vars/all: -------------------------------------------------------------------------------- 1 | # Default / placeholder for variables 2 | -------------------------------------------------------------------------------- /ansible/hardware-dell/dell-update-fc630.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Update the iDRAC and BIOS on Dell FC630 servers 3 | # The BIOS update requires a reboot so put it last 4 | # The dell-update role expect to find the required files in the `files/` 5 | # sub-directory of the role. Set dell_update_package_dir to override it. 6 | 7 | - hosts: all 8 | roles: 9 | - role: dell-update 10 | dell_update_filename: iDRAC-with-Lifecycle-Controller_Firmware_4950Y_LN_2.41.40.40_A00.bin 11 | - role: dell-update 12 | dell_update_filename: BIOS_DH7R1_LN_2.4.2.bin 13 | 14 | # vars: 15 | # - dell_update_package_dir: ~/ansible-data/dell-fc630/ 16 | -------------------------------------------------------------------------------- /ansible/host_vars/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | -------------------------------------------------------------------------------- /ansible/host_vars/all: -------------------------------------------------------------------------------- 1 | # Default / placeholder for variables 2 | -------------------------------------------------------------------------------- /ansible/idr-openstack-post.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for preparing IDR openstack nodes 3 | 4 | # idr-initial.yml must have already been run 5 | 6 | - hosts: idr-openstack 7 | 8 | roles: 9 | - role: network 10 | - role: openmicroscopy.logrotate 11 | - role: openmicroscopy.system-monitor-agent 12 | - role: openmicroscopy.mysql-backup 13 | when: "{{ openstack_master_node | default(False) }}" 14 | 15 | vars: 16 | network_ifaces: "{{ post_network_ifaces }}" 17 | 18 | tasks: 19 | - name: add extra packages 20 | become: yes 21 | yum: 22 | name: jq 23 | state: present 24 | -------------------------------------------------------------------------------- /ansible/idr-openstack-pre.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for initialising and preparing IDR openstack nodes. 3 | # You may need to reboot after this 4 | 5 | - hosts: idr-openstack 6 | 7 | roles: 8 | - role: upgrade-distpackages 9 | - role: server-swap 10 | - role: network 11 | - role: lvm-partition 12 | lvm_lvname: var_lib 13 | lvm_lvmount: /var/lib 14 | lvm_lvsize: 200G 15 | lvm_vgname: VGopenstack 16 | lvm_lvfilesystem: ext4 17 | - role: openstack-prepare 18 | 19 | vars: 20 | network_ifaces: "{{ pre_network_ifaces }}" 21 | 22 | # You may wish to run idr-gpfs-client.yml 23 | -------------------------------------------------------------------------------- /ansible/idr-playbooks/README.md: -------------------------------------------------------------------------------- 1 | # IDR Playbooks 2 | 3 | The [IDR](https://idr-demo.openmicroscopy.org/) deployment playbooks are in their own repository: https://github.com/IDR/deployment/ 4 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/ansible.cfg: -------------------------------------------------------------------------------- 1 | # Ansible configuration file 2 | 3 | [defaults] 4 | # Try more servers at once, smartly. 5 | forks = 20 6 | gathering = smart 7 | 8 | # Assume all actions take place under infrastructure/ansible/idrsystems-playbooks 9 | roles_path = vendor/:../roles/ 10 | inventory = ../../../ansible/inventory/ 11 | 12 | # These tend to be annoying. 13 | retry_files_enabled = False 14 | 15 | # Deals with the files/ directory used by IDR 16 | inventory_ignore_extensions = ~, .orig, .bak, .ini, .retry, .pyc, .pyo, .html, .omero, .gif, .png, .js, .md 17 | 18 | # Helps determine what's running slowly 19 | #callback_whitelist = profile_tasks 20 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-analysis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for maintaining IDR analysis nodes 3 | 4 | - hosts: idr-analysis 5 | roles: 6 | 7 | - role: openmicroscopy.lvm-partition 8 | lvm_lvname: root 9 | lvm_lvmount: / 10 | lvm_lvsize: "{{ rootsize }}" 11 | lvm_lvfilesystem: "{{ root_filesystem }}" 12 | - role: openmicroscopy.lvm-partition 13 | lvm_lvname: var_log 14 | lvm_lvmount: /var/log 15 | lvm_lvsize: "{{ varlog_size }}" 16 | lvm_lvfilesystem: "{{ root_filesystem }}" 17 | - role: openmicroscopy.lvm-partition 18 | lvm_lvname: var_lib_docker 19 | lvm_lvmount: /var/lib/docker 20 | lvm_lvsize: "{{ docker_volumesize }}" 21 | lvm_lvfilesystem: "{{ docker_filesystem }}" 22 | - role: openmicroscopy.lvm-partition 23 | lvm_lvname: scratch 24 | lvm_lvmount: /scratch 25 | lvm_lvsize: "{{ scratch_size }}" 26 | lvm_lvfilesystem: "{{ root_filesystem }}" 27 | 28 | - role: openmicroscopy.docker 29 | - role: openmicroscopy.logrotate 30 | - role: openmicroscopy.sudoers 31 | 32 | - role: openmicroscopy.analysis-tools 33 | - role: openmicroscopy.nfs-mount 34 | - role: openmicroscopy.versioncontrol-utils 35 | 36 | - role: openmicroscopy.celery-docker 37 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning IDR Docker nodes with local docker storage 3 | 4 | - hosts: idr-docker 5 | roles: 6 | - role: openmicroscopy.lvm-partition 7 | lvm_lvname: root 8 | lvm_lvmount: / 9 | lvm_lvsize: "{{ rootsize }}" 10 | lvm_lvfilesystem: "{{ root_filesystem }}" 11 | - role: openmicroscopy.lvm-partition 12 | lvm_lvname: var_log 13 | lvm_lvmount: /var/log 14 | lvm_lvsize: "{{ varlog_size }}" 15 | lvm_lvfilesystem: "{{ root_filesystem }}" 16 | - role: openmicroscopy.lvm-partition 17 | lvm_lvname: scratch 18 | lvm_lvmount: /scratch 19 | lvm_lvsize: "{{ scratch_size }}" 20 | lvm_lvfilesystem: "{{ root_filesystem }}" 21 | - role: openmicroscopy.basedeps 22 | - role: openmicroscopy.logrotate 23 | - role: openmicroscopy.docker 24 | - role: openmicroscopy.sudoers 25 | - role: openmicroscopy.versioncontrol-utils 26 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-gpfs-client.yml: -------------------------------------------------------------------------------- 1 | # Install RPMs from a yum repo, compile and install kernel module 2 | # See roles/gpfs/README.md if you only have the original IBM Spectrum packages 3 | 4 | - hosts: idr-gpfs-client 5 | roles: 6 | - role: yum-excludes 7 | - role: gpfs 8 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-initial.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Initial configuration of IDR nodes 3 | # Note you should reboot after changing the network configuration, and 4 | # manually verify that the configuration is correct 5 | 6 | - hosts: idr 7 | roles: 8 | - role: openmicroscopy.upgrade-distpackages 9 | - role: server-swap 10 | - role: network 11 | # Fails due to yum excludes, install it manually if you want it: 12 | # yum --disableexcludes=all install check-mk-agent-1.2.6p16 13 | #- role: system-monitor-agent 14 | 15 | post_tasks: 16 | - debug: 17 | msg: "You may need to reboot your system" 18 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-nfs-hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for maintaining IDR NFS nodes 3 | 4 | - hosts: idr-nfs-hosts 5 | roles: 6 | - role: nfs-ganesha-share 7 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-samba-hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for maintaining IDR Samba nodes 3 | 4 | - hosts: idr-samba-hosts 5 | roles: 6 | - role: active-directory-samba-share 7 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idr-upgrade.yml: -------------------------------------------------------------------------------- 1 | # Update system packages 2 | 3 | - hosts: idr 4 | roles: 5 | - role: openmicroscopy.upgrade-distpackages 6 | upgrade_distpackages_reboot_kernel: True 7 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idrsystems-deployment.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for running all IDR deployment playbooks 3 | 4 | - include: idr-analysis.yml 5 | - include: idr-docker.yml 6 | - include: idr-gpfs-client.yml 7 | - include: idr-nfs-hosts.yml 8 | - include: idr-samba-hosts.yml 9 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/idrsystems-provision.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for running all IDR playbooks 3 | 4 | # Initial networking and basic system configuration 5 | # A reboot may be required after this playbook 6 | - include: idr-initial.yml 7 | 8 | # Upgrade all system packages 9 | # A reboot may be required after this playbook 10 | - include: idr-upgrade.yml 11 | 12 | # Install everything else 13 | - include: idrsystems-deployment.yml 14 | -------------------------------------------------------------------------------- /ansible/idrsystems-playbooks/requirements.yml: -------------------------------------------------------------------------------- 1 | # External Ansible roles required by this repository 2 | 3 | - src: openmicroscopy.analysis-tools 4 | version: 1.0.0 5 | 6 | - src: openmicroscopy.basedeps 7 | version: 1.0.0 8 | 9 | - src: openmicroscopy.cli-utils 10 | version: 1.0.0 11 | 12 | - src: openmicroscopy.celery-docker 13 | version: 0.1.0 14 | 15 | - src: openmicroscopy.docker 16 | version: 2.0.0 17 | 18 | - src: openmicroscopy.docker-tools 19 | version: 1.0.0 20 | 21 | - src: openmicroscopy.local-accounts 22 | version: 1.0.0 23 | 24 | - src: openmicroscopy.logrotate 25 | version: 1.0.0 26 | 27 | - src: openmicroscopy.lvm-partition 28 | version: 1.0.0 29 | 30 | - src: openmicroscopy.nfs-mount 31 | version: 1.0.0 32 | 33 | - src: openmicroscopy.nfs-share 34 | version: 1.0.0 35 | 36 | - src: openmicroscopy.nginx 37 | version: 1.0.0 38 | 39 | - src: openmicroscopy.nginx-proxy 40 | version: 1.0.0 41 | 42 | - src: openmicroscopy.python-pydata 43 | version: 1.0.0 44 | 45 | - src: openmicroscopy.redis 46 | version: 1.0.0 47 | 48 | - src: openmicroscopy.samba-client 49 | version: 1.0.0 50 | 51 | - src: openmicroscopy.selinux-utils 52 | version: 1.0.1 53 | 54 | - src: openmicroscopy.sudoers 55 | version: 1.0.0 56 | 57 | - src: openmicroscopy.upgrade-distpackages 58 | version: 1.0.0 59 | 60 | - src: openmicroscopy.versioncontrol-utils 61 | version: 1.0.0 62 | -------------------------------------------------------------------------------- /ansible/inventory/README.md: -------------------------------------------------------------------------------- 1 | This directory is useful for testing against openstack, but 2 | likely you will want to have a copy of these files in your 3 | own inventory directory with group_vars and host_vars as 4 | needed. 5 | -------------------------------------------------------------------------------- /ansible/inventory/variables: -------------------------------------------------------------------------------- 1 | [os-image-centos] 2 | 3 | [os-image-centos:vars] 4 | -------------------------------------------------------------------------------- /ansible/ome-analysis/ansible.cfg: -------------------------------------------------------------------------------- 1 | # Ansible configuration file 2 | 3 | [defaults] 4 | 5 | # Assume all actions take place under infrastructure/ansible/ome-analysis 6 | roles_path = vendor/:../roles/ 7 | inventory = ../../../ansible/inventory/ 8 | 9 | retry_files_enabled = False 10 | -------------------------------------------------------------------------------- /ansible/ome-analysis/initial.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Initial configuration of IDR nodes 3 | # Note you should reboot after changing the network configuration, and 4 | # manually verify that the configuration is correct 5 | 6 | - hosts: ome-jupyter 7 | roles: 8 | - role: openmicroscopy.upgrade-distpackages 9 | 10 | # Fails due to yum excludes, install it manually if you want it: 11 | # yum --disableexcludes=all install check-mk-agent-1.2.6p16 12 | #- role: system-monitor-agent 13 | 14 | post_tasks: 15 | - debug: 16 | msg: "You may need to reboot your system" 17 | -------------------------------------------------------------------------------- /ansible/ome-analysis/requirements.yml: -------------------------------------------------------------------------------- 1 | # External Ansible roles required by this repository 2 | 3 | - src: openmicroscopy.basedeps 4 | version: 1.0.0 5 | 6 | - src: openmicroscopy.cli-utils 7 | version: 1.0.0 8 | 9 | - src: openmicroscopy.cadvisor 10 | version: 0.1.0 11 | 12 | - src: openmicroscopy.docker 13 | version: 2.0.0 14 | 15 | - src: openmicroscopy.docker-tools 16 | version: 1.0.0 17 | 18 | - src: openmicroscopy.logrotate 19 | version: 1.0.0 20 | 21 | - src: openmicroscopy.lvm-partition 22 | version: 1.0.0 23 | 24 | - src: openmicroscopy.nfs-mount 25 | version: 1.0.0 26 | 27 | - src: openmicroscopy.nginx 28 | version: 1.0.0 29 | 30 | - src: openmicroscopy.nginx-proxy 31 | version: 1.0.0 32 | 33 | - src: openmicroscopy.selinux-utils 34 | version: 1.0.1 35 | 36 | - src: openmicroscopy.sudoers 37 | version: 1.0.0 38 | 39 | - src: openmicroscopy.upgrade-distpackages 40 | version: 1.0.0 41 | 42 | - src: openmicroscopy.versioncontrol-utils 43 | version: 1.0.0 44 | 45 | - name: IDR.idr-jupyter 46 | src: https://github.com/IDR/ansible-role-idr-jupyter.git 47 | version: 2.0.0 48 | -------------------------------------------------------------------------------- /ansible/os-devspace.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for starting a docker instance in the 3 | # openstack cloud. 4 | 5 | ## See os-create.yml 6 | 7 | ## Run: 8 | # $ source openrc.sh 9 | # $ ansible-playbook os-devspace.yml -e vm_name=test-devspace -e vm_key_name=yourprofile 10 | 11 | - hosts: localhost 12 | connection: local 13 | #gather_facts: false 14 | 15 | vars: 16 | 17 | vm_image: "CentOS 7" 18 | vm_flavour_ram: 32000 19 | vm_size: 50 20 | vm_groups: "ansible-managed,os-image-centos,docker-hosts,devspace" 21 | 22 | pre_tasks: 23 | 24 | - fail: 25 | msg: "vm_key_name is required" 26 | when: vm_key_name is undefined or not vm_key_name 27 | 28 | - fail: 29 | msg: "vm_name is required" 30 | when: vm_name is undefined or not vm_name 31 | 32 | 33 | tasks: 34 | 35 | - name: Docker external access security group 36 | os_security_group: 37 | description: External access to Docker servers (managed by Ansible) 38 | name: docker-devspace-external 39 | state: present 40 | 41 | - name: Docker external access security group rules 42 | os_security_group_rule: 43 | direction: ingress 44 | port_range_max: "{{ item }}" 45 | port_range_min: "{{ item }}" 46 | protocol: tcp 47 | remote_ip_prefix: 0.0.0.0/0 48 | security_group: docker-devspace-external 49 | state: present 50 | with_items: 51 | - 22 52 | - 80 53 | - 443 54 | - 8443 55 | - 4444 56 | - 4063 57 | - 4064 58 | - 14063 59 | - 14064 60 | 61 | - name: Docker VM 62 | os_server: 63 | name: "{{ vm_name }}" 64 | state: present 65 | image: "{{ vm_image }}" 66 | boot_from_volume: True 67 | volume_size: "{{ vm_size }}" 68 | terminate_volume: true 69 | key_name: "{{ vm_key_name }}" 70 | flavor_ram: "{{ vm_flavour_ram }}" 71 | auto_ip: yes 72 | meta: 73 | hostname: "{{ vm_name }}" 74 | groups: "{{ vm_groups }}" 75 | security_groups: [default, docker-devspace-external] 76 | register: vmdocker 77 | 78 | - debug: 79 | msg: "IPs (Docker) private:{{ vmdocker.openstack.private_v4 }} floating:{{ vmdocker.openstack.public_v4 | default('') }}" 80 | -------------------------------------------------------------------------------- /ansible/os-uod-slave.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for starting a Jenkins slave in the UoD 3 | # openstack cloud. 4 | # 5 | # Usage: 6 | # $ source openrc.sh 7 | # $ ansible-playbook -i {inventory_dir} os-uod-slave.yml -e vm_name=web-server 8 | # $ ansible-playbook -i {inventory_dir} uod-slave.yml -l web-server 9 | 10 | # Re-using the docker playbook so that we have the option 11 | # of starting docker services as a part of a Jenkins job. 12 | - include: os-uod-docker.yml 13 | vars: 14 | vm_groups: "ansible-managed,os-image-centos,docker-hosts,ci-jenkins-linux,ci-omero,ci-omero-web" 15 | vm_flavour: "m1.medium" 16 | 17 | - hosts: localhost 18 | connection: local 19 | gather_facts: false 20 | roles: 21 | - role: openstack-volume-storage 22 | openstack_volume_size: 100 23 | openstack_volume_vmname: "{{ vm_name }}" 24 | openstack_volume_name: opthudson 25 | openstack_volume_device: /dev/vdb 26 | -------------------------------------------------------------------------------- /ansible/os-uod-web.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for starting a web instance in the 3 | # openstack cloud. 4 | 5 | ## See os-create.yml 6 | 7 | - hosts: localhost 8 | connection: local 9 | #gather_facts: false 10 | 11 | vars: 12 | 13 | vm_image: "CentOS 7" 14 | vm_flavour: m1.medium 15 | vm_groups: "ansible-managed,os-image-centos,web-linux" 16 | ignore_internal_known_hosts: True 17 | 18 | pre_tasks: 19 | 20 | - fail: 21 | msg: "vm_key_name is required" 22 | when: vm_key_name is undefined or not vm_key_name 23 | 24 | - fail: 25 | msg: "vm_name is required" 26 | when: vm_name is undefined or not vm_name 27 | 28 | 29 | tasks: 30 | 31 | - name: Web external access security group 32 | os_security_group: 33 | description: External access to Web servers (managed by Ansible) 34 | name: web-external 35 | state: present 36 | 37 | - name: Web external access security group rules 38 | os_security_group_rule: 39 | direction: ingress 40 | port_range_max: "{{ item }}" 41 | port_range_min: "{{ item }}" 42 | protocol: tcp 43 | remote_ip_prefix: 0.0.0.0/0 44 | security_group: web-external 45 | state: present 46 | with_items: 47 | - 22 # could be restricted to fewer IPs 48 | - 80 49 | - 443 50 | 51 | - name: Web VM 52 | os_server: 53 | name: "{{ vm_name }}" 54 | state: present 55 | image: "{{ vm_image }}" 56 | key_name: "{{ vm_key_name }}" 57 | flavor: "{{ vm_flavour }}" 58 | auto_ip: yes 59 | meta: 60 | hostname: "{{ vm_name }}" 61 | groups: "{{ vm_groups }}" 62 | security_groups: [default, web-external] 63 | register: vmweb 64 | 65 | # To prevent duplicates with the dynamic inventory only add_host if 66 | # the VM was created in this run 67 | 68 | - name: Web VM hosts files 69 | add_host: 70 | name: "{{ vm_name }}" 71 | groups: "{{ vm_groups }}" 72 | ansible_host: "{{ vmweb.server.public_v4 }}" 73 | ansible_user: "centos" 74 | ansible_become: true 75 | 76 | - debug: 77 | msg: "IPs (Web) private:{{ vmweb.openstack.private_v4 }} floating:{{ vmweb.openstack.public_v4 | default('') }}" 78 | 79 | - hosts: localhost 80 | connection: local 81 | #gather_facts: false 82 | roles: 83 | - role: openstack-volume-storage 84 | openstack_volume_size: 100 85 | openstack_volume_vmname: "{{ vm_name }}" 86 | openstack_volume_name: var 87 | openstack_volume_device: /dev/vdb 88 | -------------------------------------------------------------------------------- /ansible/plugins/callback/error_if_no_hosts_match.py: -------------------------------------------------------------------------------- 1 | # https://gist.github.com/jjshoe/ace3070906e5bc5cc432 2 | # https://github.com/ansible/ansible/pull/14742 3 | # https://github.com/ansible/ansible/issues/14693 4 | 5 | # Make coding more python3-ish 6 | 7 | from __future__ import (absolute_import, division, print_function) 8 | __metaclass__ = type 9 | 10 | import os 11 | import time 12 | import sys 13 | 14 | from ansible.plugins.callback import CallbackBase 15 | 16 | 17 | class CallbackModule(CallbackBase): 18 | """ 19 | This callback module exits with non-zero if no hosts match 20 | """ 21 | CALLBACK_VERSION = 2.0 22 | CALLBACK_TYPE = 'aggregate' 23 | CALLBACK_NAME = 'no_hosts_match_exit_non_zero' 24 | CALLBACK_NEEDS_WHITELIST = False 25 | 26 | def __init__(self): 27 | super(CallbackModule, self).__init__() 28 | 29 | def playbook_on_stats(self, stats): 30 | found_stats = False 31 | 32 | for key in ['ok', 'failures', 'dark', 'changed', 'skipped']: 33 | if len(getattr(stats, key)) > 0: 34 | found_stats = True 35 | break 36 | 37 | if found_stats == False: 38 | print('ERROR: No hosts matched') 39 | sys.exit(10) 40 | -------------------------------------------------------------------------------- /ansible/qa-web-deployment.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for provisioning QA Web nodes 3 | 4 | - hosts: qa-web-linux 5 | roles: 6 | - role: postgresql 7 | - role: nginx 8 | - role: python-devel 9 | - role: openldap-devel 10 | - role: redis -------------------------------------------------------------------------------- /ansible/requirements-internal.yml: -------------------------------------------------------------------------------- 1 | # External Ansible roles that will only work internally 2 | # For example they may rely on an internal restricted resource which can 3 | # be easily replicated outside. 4 | # Roles which could be used outside the OME should go in requirements.yml 5 | 6 | - name: system-monitor-agent 7 | src: https://github.com/openmicroscopy/ansible-role-system-monitor-agent 8 | version: 0.1.0 9 | -------------------------------------------------------------------------------- /ansible/roles/README.md: -------------------------------------------------------------------------------- 1 | Roles 2 | ----- 3 | 4 | Core roles are now maintained and versioned in separate 5 | [GitHub repositories](https://github.com/search?q=topic%3Aansible+org%3Aopenmicroscopy&type=Repositories) and available through [Ansible Galaxy](http://galaxy.ansible.com/openmicroscopy/). 6 | 7 | All remaining roles in this directory will either be decoupled out or migrated 8 | into dedicated playbook directories. 9 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-join/README.md: -------------------------------------------------------------------------------- 1 | Active Directory Join 2 | ===================== 3 | 4 | Join a Linux server to an existing Active Directory domain. 5 | 6 | 7 | Role Variables 8 | -------------- 9 | 10 | `active_directory_realm`: The main AD domain 11 | `active_directory_workgroup`: The samba workgroup 12 | `active_directory_server`: A list of the AD servers 13 | `active_directory_realmd_tags`: A list of AD realm tags 14 | `active_directory_kerberos_realms`: A dict of the form `{REALM: [domain, ...]}` where keys are the realms and values are a list of domains associated with that realm 15 | 16 | `active_directory_join_ou`: The domain/unit the server should be joined to 17 | `active_directory_join_user`: The `username%password` of the AD administrative account used to join the node 18 | `active_directory_join_access`: A list of user and/or group names that should be allowed to access this node 19 | `active_directory_sssd_conf`: A variable containing the contents of `/etc/sssd/sssd.conf`, so that the contents of the file can be kept in a separate private repository 20 | `active_directory_ssh_passwords`: `yes|no`, whether ssh access using passwords should be allowed, if omitted do not change the current configuration 21 | `active_directory_user_homes`: The parent directory for user homes, required if they should be auto-created on login 22 | 23 | 24 | Author Information 25 | ------------------ 26 | 27 | ome-devel@lists.openmicroscopy.org.uk 28 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-join/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handlers for active-directory-join 3 | 4 | - name: restart sshd 5 | become: yes 6 | service: 7 | name: sshd 8 | state: restarted 9 | 10 | - name: restart sssd 11 | become: yes 12 | service: 13 | name: sssd 14 | state: restarted 15 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-join/templates/krb5-conf.j2: -------------------------------------------------------------------------------- 1 | [logging] 2 | default = FILE:/var/log/krb5libs.log 3 | kdc = FILE:/var/log/krb5kdc.log 4 | admin_server = FILE:/var/log/kadmind.log 5 | 6 | [libdefaults] 7 | default_realm = {{ active_directory_realm }} 8 | dns_lookup_realm = true 9 | dns_lookup_kdc = true 10 | ticket_lifetime = 24h 11 | renew_lifetime = 7d 12 | forwardable = yes 13 | rdns = false 14 | 15 | [realms] 16 | {% for item in active_directory_kerberos_realms %} 17 | {{ item }} = { 18 | } 19 | {% endfor %} 20 | 21 | [domain_realm] 22 | {% for item in active_directory_kerberos_realms %} 23 | {% for value in active_directory_kerberos_realms[item] %} 24 | {{ value }} = {{ item }} 25 | {% endfor %} 26 | {% endfor %} 27 | 28 | [appdefaults] 29 | pam = { 30 | debug = false 31 | ticket_lifetime = 36000 32 | renew_lifetime = 36000 33 | forwardable = true 34 | krb4_convert = false 35 | } 36 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-join/templates/security-access-conf.j2: -------------------------------------------------------------------------------- 1 | {% for item in active_directory_join_access %} 2 | +:{{ item }}:ALL 3 | {% endfor %} 4 | -:ALL EXCEPT root:ALL 5 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-join/templates/sssd-sssd-conf.j2: -------------------------------------------------------------------------------- 1 | [sssd] 2 | domains = {{ active_directory_realm | lower }} 3 | config_file_version = 2 4 | services = nss, pam, pac 5 | 6 | [domain/{{ active_directory_realm | lower }}] 7 | id_provider = ad 8 | ad_domain = {{ active_directory_realm | lower }} 9 | ad_server = {{ active_directory_server }} 10 | auth_provider = ad 11 | chpass_provider = ad 12 | access_provider = ad 13 | 14 | krb5_realm = {{ active_directory_realm }} 15 | realmd_tags = {{ active_directory_realmd_tags }} 16 | cache_credentials = True 17 | krb5_store_password_if_offline = True 18 | default_shell = /bin/bash 19 | 20 | ldap_schema = ad 21 | ldap_id_mapping = False 22 | 23 | use_fully_qualified_names = False 24 | fallback_homedir = /home/%u 25 | 26 | cache_credentials = true 27 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-samba-share/README.md: -------------------------------------------------------------------------------- 1 | Active Directory Samba Share 2 | ============================ 3 | 4 | Manage samba file shares with authentication provided by active directory. 5 | If the `active-directory-join` role is also present on this server it should be run before this role, and common variables must match. 6 | 7 | If you are running SELinux see http://selinuxproject.org/page/SambaRecipes 8 | 9 | 10 | Role Variables 11 | -------------- 12 | 13 | `active_directory_realm`: The main AD domain 14 | `active_directory_workgroup`: The samba workgroup 15 | `active_directory_shares`: A dictionary of dictionaries of shares in the form `{share-name: { path: , comment: (optional), readonly: (default True), users: [list of users/groups] }}` 16 | 17 | 18 | Example Playbook 19 | ---------------- 20 | 21 | - hosts: localhost 22 | roles: 23 | - role: active-directory-samba-share 24 | active_directory_realm: AD.EXAMPLE.ORG 25 | active_directory_workgroup: workgroup 26 | active_directory_shares: 27 | share-ro: 28 | path: /srv/samba/ro 29 | comment: Read-only share 30 | users: [root] 31 | 32 | share-rw: 33 | path: /srv/samba/rw 34 | comment: Read-write share 35 | readonly: False 36 | users: [root] 37 | 38 | 39 | Author Information 40 | ------------------ 41 | 42 | ome-devel@lists.openmicroscopy.org.uk 43 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-samba-share/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/active-direcgtory-samba 3 | 4 | # A dictionary of dictionaries of shares, see README.md 5 | active_directory_shares: {} 6 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-samba-share/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handlers for active-directory-samba-share 3 | 4 | - name: restart smb 5 | become: yes 6 | service: 7 | name: smb 8 | state: restarted 9 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-samba-share/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/active-directory-samba-share 3 | 4 | - name: system packages | samba server 5 | become: yes 6 | yum: 7 | name: "{{ item }}" 8 | state: present 9 | with_items: 10 | - samba 11 | - samba-client 12 | 13 | - name: samba server | configure shares 14 | become: yes 15 | template: 16 | backup: yes 17 | dest: /etc/samba/smb.conf 18 | src: samba-smb-conf.j2 19 | validate: "testparm -s %s" 20 | notify: 21 | - restart smb 22 | 23 | - name: active directory | enable smb 24 | become: yes 25 | service: 26 | enabled: yes 27 | name: smb 28 | -------------------------------------------------------------------------------- /ansible/roles/active-directory-samba-share/templates/samba-smb-conf.j2: -------------------------------------------------------------------------------- 1 | [global] 2 | workgroup = {{ active_directory_workgroup }} 3 | realm = {{ active_directory_realm }} 4 | security = ads 5 | #idmap config * : range = 16777216-33554431 6 | kerberos method = secrets and keytab 7 | template shell = /bin/bash 8 | server string = Samba Server Version %v 9 | # logs split per machine 10 | log file = /var/log/samba/log.%m 11 | # max 50KB per log file, then rotate 12 | max log size = 50 13 | passdb backend = tdbsam 14 | load printers = no 15 | cups options = raw 16 | printcap name = /dev/null 17 | 18 | {% for key, value in active_directory_shares.iteritems() %} 19 | [{{ key }}] 20 | path = {{ value.path }} 21 | comment = {{ value.comment | default(key) }} 22 | read only = {{ (value.readonly | default(True)) | ternary("yes", "no") }} 23 | valid users = {{ value.users | join(", ") }} 24 | {% endfor %} 25 | -------------------------------------------------------------------------------- /ansible/roles/ansible/README.tmd: -------------------------------------------------------------------------------- 1 | Ansible 2 | ======= 3 | 4 | Install ansible. 5 | 6 | TODO: Add configuration options. 7 | 8 | Author Information 9 | ------------------ 10 | 11 | ome-devel@lists.openmicroscopy.org.uk 12 | -------------------------------------------------------------------------------- /ansible/roles/ansible/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/redis 3 | 4 | - name: install ansible 5 | become: yes 6 | yum: 7 | name: ansible 8 | state: present 9 | -------------------------------------------------------------------------------- /ansible/roles/debug-dumpallvars/README.md: -------------------------------------------------------------------------------- 1 | Debug Dump All Vars 2 | =================== 3 | 4 | Dump all variables to a file on the remote host. 5 | 6 | This may be useful when debugging Ansible roles. Most Ansible variables will be written to a file on the remote host. 7 | 8 | 9 | Role Variables 10 | -------------- 11 | 12 | Optional variables: 13 | 14 | - `debug_dumpallvars_file`: The file to write the variables to, default `/tmp/ansible-debug-dumpallvars.txt`. 15 | 16 | 17 | Example Playbook 18 | ---------------- 19 | 20 | - hosts: localhost 21 | roles: 22 | - role: debug-dumpallvars 23 | debug_dumpallvars_file: /tmp/ansible-dumpvars.txt 24 | 25 | 26 | Author Information 27 | ------------------ 28 | 29 | ome-devel@lists.openmicroscopy.org.uk 30 | -------------------------------------------------------------------------------- /ansible/roles/debug-dumpallvars/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/debug-dumpallvars 3 | 4 | # Dump file 5 | debug_dumpallvars_file: /tmp/ansible-debug-dumpallvars.txt 6 | -------------------------------------------------------------------------------- /ansible/roles/debug-dumpallvars/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Dump all variables for debugging 3 | # https://coderwall.com/p/13lh6w/dump-all-variables 4 | 5 | - name: Dump all vars 6 | template: 7 | src: dumpall.j2 8 | dest: "{{ debug_dumpallvars_file }}" 9 | -------------------------------------------------------------------------------- /ansible/roles/debug-dumpallvars/templates/dumpall.j2: -------------------------------------------------------------------------------- 1 | Module Variables ("vars"): 2 | -------------------------------- 3 | {{ vars | default('') | to_nice_json }} 4 | 5 | Environment Variables ("environment"): 6 | -------------------------------- 7 | {{ environment | default('') | to_nice_json }} 8 | 9 | Group Name Variables ("group_names"): 10 | -------------------------------- 11 | {{ group_names | default('') | to_nice_json }} 12 | 13 | Group Variables ("groups"): 14 | -------------------------------- 15 | {{ groups | default('') | to_nice_json }} 16 | 17 | Host Variables ("hostvars"): 18 | -------------------------------- 19 | {{ hostvars | default('') | to_nice_json }} 20 | -------------------------------------------------------------------------------- /ansible/roles/dell-update/README.md: -------------------------------------------------------------------------------- 1 | Dell Update 2 | =========== 3 | 4 | Run Dell Linux driver updates such as those for the BIOS and iDRAC. 5 | 6 | Assumes the update package takes the following options (`--help`): 7 | 8 | Command-line options for the Update Package 9 | 10 | Usage: [options...] 11 | 12 | Options: 13 | 14 | -h,--help : Display command-line usage help 15 | -c : Determine if the update can be applied to the system (1) 16 | -f : Force a downgrade to an older version. (1)(2) 17 | -q : Execute the update package silently without user intervention 18 | -n : Execute the update package without security verification 19 | -r : Reboot if necessary after the update (2) 20 | -v,--version : Display version information 21 | --list : Display contents of package (3) 22 | --extract : Extract files to specified path (3)(4) 23 | 24 | The system will be automatically rebooted if necessary, however this role will not wait for it to come back up so you will see an error. 25 | 26 | TODO: Wait for host to come back up after a reboot 27 | 28 | Note: Update packages are run in non-interactive mode with automatic reboot if necessary (`-q -r`). 29 | It is possible the package will reboot the server before installing the update, for example BIOS updates, in contrast to interactive mode where the command line remains connected until after the BIOS is updated. 30 | Do not be unduly worried (unless the server remains inaccessible after several minutes in which case check the console). 31 | 32 | 33 | Requirements 34 | ------------ 35 | 36 | Ensure the Dell update package is in the directory specified by `dell_update_package_dir`. 37 | 38 | 39 | Role Variables 40 | -------------- 41 | 42 | - `dell_update_package_dir`: The directory containing the update package, default `files` 43 | - `dell_update_filename`: The filename of the update packaged, required 44 | 45 | 46 | Example Playbook 47 | ---------------- 48 | 49 | # Update the iDRAC and BIOS, the BIOS update requires a reboot so put it last 50 | - hosts: dell-servers 51 | roles: 52 | - role: dell-update 53 | dell_update_filename: iDRAC-with-Lifecycle-Controller_Firmware_5GCHC_LN_2.30.30.30_A00.BIN 54 | - role: dell-update 55 | dell_update_filename: BIOS_DC9XJ_LN_2.1.7.BIN 56 | 57 | 58 | 59 | Author Information 60 | ------------------ 61 | 62 | ome-devel@lists.openmicroscopy.org.uk 63 | -------------------------------------------------------------------------------- /ansible/roles/dell-update/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Run a Dell driver update 3 | 4 | - name: dell update | create directory 5 | become: yes 6 | file: 7 | path: dell-updates 8 | state: directory 9 | 10 | - name: dell update | copy package 11 | become: yes 12 | copy: 13 | dest: "dell-updates/{{ dell_update_filename }}" 14 | src: "{{ dell_update_package_dir | default('files') }}/{{ dell_update_filename }}" 15 | mode: 0555 16 | 17 | - name: dell update | check update can be applied 18 | become: yes 19 | command: dell-updates/{{ dell_update_filename }} -c -q 20 | register: check 21 | failed_when: check.rc not in [0, 3] 22 | # Return codes 23 | # 0: success 24 | # 3: current version is equal to or more recent than this one 25 | # Others: See http://mirrors.vbi.vt.edu/mirrors/vbi/hardware/dell/tools/suu/om_5.5.2/docs/dup/dup261_Lin_UG_HTML/3cliref.htm 26 | 27 | - name: dell update | not required 28 | debug: 29 | msg: "Current version is equal to or newer than {{ dell_update_filename }}" 30 | when: check.rc == 3 31 | 32 | - name: dell update | run update 33 | become: yes 34 | command: dell-updates/{{ dell_update_filename }} -q -r 35 | when: check.rc == 0 36 | -------------------------------------------------------------------------------- /ansible/roles/devtoolset-4/README.md: -------------------------------------------------------------------------------- 1 | devtoolset-4 2 | ============ 3 | 4 | RHSCL devtoolset-4 toolchain. 5 | 6 | Requirements 7 | ------------ 8 | 9 | None. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/roles/devtoolset-4/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: devtoolset-4 | Install devtoolset-4 3 | yum: 4 | name: "{{ item }}" 5 | state: latest 6 | with_items: 7 | - centos-release-scl 8 | - devtoolset-4 9 | become: yes 10 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-client/README.md: -------------------------------------------------------------------------------- 1 | Docker DNS Client 2 | ================= 3 | 4 | Set up the registrator service for updating the Docker DNS server with the currently running containers. 5 | TODO: Currently this only works when run on the same host as the server. 6 | 7 | Role Variables 8 | -------------- 9 | 10 | The defaults will work unless the name of the `etcd` container has been changed: 11 | 12 | - `docker_dns.etcd`: Runtime name of the `etcd` container, default: `etcd1` 13 | - `docker_dns.domain`: Domain for the DNS service, see role `docker-dns-server` 14 | 15 | Author Information 16 | ------------------ 17 | 18 | ome-devel@lists.openmicroscopy.org.uk 19 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-client/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/docker-dns-client 3 | 4 | docker_dns: 5 | domain: docker.internal 6 | etcd: etcd1 7 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-client/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Setup production Docker DNS clients to run automatically 3 | 4 | - name: docker production | registrator 5 | become: yes 6 | template: 7 | src: systemd-docker-registrator.j2 8 | dest: /etc/systemd/system/docker-registrator.service 9 | register: systemdregistrator 10 | 11 | # Can't use a notifier because the reload must happen before the next step 12 | 13 | - name: reload systemd 14 | become: yes 15 | command: systemctl daemon-reload 16 | when: systemdregistrator.changed 17 | 18 | - name: docker production | enable registrator 19 | become: yes 20 | service: 21 | enabled: yes 22 | name: docker-registrator.service 23 | state: started 24 | 25 | # service module doesn't seem to work for a new systemd.service 26 | 27 | - name: docker production | enable registrator 28 | become: yes 29 | command: systemctl restart docker-registrator.service 30 | when: systemdregistrator.changed 31 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-client/templates/systemd-docker-registrator.j2: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker registrator 3 | Requires=docker.service 4 | # TODO: Should work with a remote etcd daemon 5 | Requires=docker-{{ docker_dns.etcd }}.service 6 | After=docker-{{ docker_dns.etcd }}.service 7 | 8 | [Service] 9 | Restart=on-failure 10 | RestartSec=10 11 | ExecStartPre=-/usr/bin/docker kill registrator 12 | ExecStartPre=-/usr/bin/docker rm registrator 13 | ExecStart=/usr/bin/docker run --name registrator \ 14 | --link {{ docker_dns.etcd }}:etcd \ 15 | -v /var/run/docker.sock:/tmp/docker.sock \ 16 | --entrypoint=sh \ 17 | manics/registrator -c \ 18 | "sleep 5; \ 19 | registrator \ 20 | -iponly \ 21 | -ttl 60 \ 22 | -ttl-refresh=30 \ 23 | skydns2://etcd:2379/{{ docker_dns.domain }}" 24 | ExecStop=/usr/bin/docker stop registrator 25 | 26 | [Install] 27 | WantedBy=multi-user.target 28 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-server/README.md: -------------------------------------------------------------------------------- 1 | Docker DNS Server 2 | ================= 3 | 4 | Setup a dynamic DNS service for Docker containers. This can optionally be defined as the authority for a subdomain in an upstread DNS server. 5 | 6 | Role Variables 7 | -------------- 8 | 9 | Although defaults are provided it is highly recommended that you customise the configuration. 10 | 11 | - `docker_dns.forwarders`: Forward queries for other domains to these servers, default: `8.8.8.8:53,8.8.4.4:53` 12 | - `docker_dns.domain`: Domain for the DNS service, default: `docker.internal` so the DNS will respond to queries for `*.docker.internal` 13 | - `docker_dns.etcd: Runtime name of the `etcd` container, default: `etcd1` 14 | 15 | Author Information 16 | ------------------ 17 | 18 | ome-devel@lists.openmicroscopy.org.uk 19 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-server/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/docker-dns-server 3 | 4 | docker_dns: 5 | forwarders: 8.8.8.8:53,8.8.4.4:53 6 | domain: docker.internal 7 | etcd: etcd1 8 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-server/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Setup production Docker services to run automatically 3 | 4 | - name: docker production | etcd 5 | become: yes 6 | template: 7 | src: systemd-docker-etcd.j2 8 | dest: /etc/systemd/system/docker-{{ docker_dns.etcd }}.service 9 | register: systemdetcd 10 | 11 | - name: docker production | skydns 12 | become: yes 13 | template: 14 | src: systemd-docker-skydns.j2 15 | dest: /etc/systemd/system/docker-skydns.service 16 | register: systemdskydns 17 | 18 | # Can't use a notifier because the reload must happen before the next step 19 | 20 | - name: docker production | reload systemd 21 | become: yes 22 | command: systemctl daemon-reload 23 | when: systemdetcd.changed or systemdskydns.changed 24 | 25 | - name: docker production | enable etcd 26 | become: yes 27 | service: 28 | enabled: yes 29 | name: docker-{{ docker_dns.etcd }}.service 30 | state: started 31 | 32 | - name: docker production | enable skydns 33 | become: yes 34 | service: 35 | enabled: yes 36 | name: docker-skydns.service 37 | state: started 38 | 39 | # service module doesn't seem to work for a new systemd.service so force a 40 | # restart 41 | # TODO: check if it works in the latest version 42 | 43 | - name: docker production | restart etcd 44 | become: yes 45 | command: systemctl restart docker-{{ docker_dns.etcd }}.service 46 | when: systemdetcd.changed 47 | 48 | - name: docker production | restart skydns 49 | become: yes 50 | command: systemctl restart docker-skydns.service 51 | when: systemdskydns.changed 52 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-server/templates/systemd-docker-etcd.j2: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker etcd 3 | Requires=docker.service 4 | After=docker.service 5 | 6 | [Service] 7 | Restart=on-failure 8 | RestartSec=10 9 | ExecStartPre=-/usr/bin/docker kill {{ docker_dns.etcd }} 10 | ExecStartPre=-/usr/bin/docker rm {{ docker_dns.etcd }} 11 | #ExecStart=/usr/bin/docker run --name {{ docker_dns.etcd }} \ 12 | # quay.io/coreos/etcd \ 13 | # --listen-client-urls http://0.0.0.0:2379 \ 14 | # --advertise-client-urls http://0.0.0.0:2379 15 | ExecStart=/usr/bin/docker run --name {{ docker_dns.etcd }} \ 16 | quay.io/coreos/etcd:v0.4.8 \ 17 | -addr localhost:2379 18 | ExecStop=/usr/bin/docker stop {{ docker_dns.etcd }} 19 | 20 | [Install] 21 | WantedBy=multi-user.target 22 | -------------------------------------------------------------------------------- /ansible/roles/docker-dns-server/templates/systemd-docker-skydns.j2: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Docker skydns 3 | Requires=docker.service 4 | After=docker-{{ docker_dns.etcd }}.service 5 | 6 | [Service] 7 | Restart=on-failure 8 | RestartSec=10 9 | ExecStartPre=-/usr/bin/docker kill skydns 10 | ExecStartPre=-/usr/bin/docker rm skydns 11 | ExecStart=/usr/bin/docker run --name skydns \ 12 | -p 53:53 -p 53:53/udp \ 13 | --link {{ docker_dns.etcd }}:etcd \ 14 | skynetservices/skydns \ 15 | -addr=0.0.0.0:53 \ 16 | -nameservers={{ docker_dns.forwarders }} \ 17 | -domain {{ docker_dns.domain }} \ 18 | -machines http://etcd:2379/ \ 19 | -verbose 20 | ExecStop=/usr/bin/docker stop skydns 21 | 22 | [Install] 23 | WantedBy=multi-user.target 24 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/files/.gitignore: -------------------------------------------------------------------------------- 1 | # The original GPFS installer files: 2 | Spectrum_Scale_Standard-4.1.1.4-x86_64-Linux-update 3 | Spectrum_Scale_install-4.1.1.0_x86_64_standard_protocols 4 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/files/mmfs-bin-mmglobfuncs.Linux.patch: -------------------------------------------------------------------------------- 1 | --- /usr/lpp/mmfs/bin/mmglobfuncs.Linux.orig 2016-02-15 13:13:32.218701682 +0000 2 | +++ /usr/lpp/mmfs/bin/mmglobfuncs.Linux 2016-02-16 10:14:48.657256622 +0000 3 | @@ -269,7 +269,7 @@ 4 | 5 | elif [[ -f /etc/redhat-release ]] 6 | then 7 | - DISTRO=$($awk '/Enterprise Linux/ { print "RHEL"; exit }' /etc/redhat-release) 8 | + DISTRO=$($awk '/(Enterprise|CentOS) Linux/ { print "RHEL"; exit }' /etc/redhat-release) 9 | DISTRO_VERSION=$($awk '/release/ && $NF != "release" { \ 10 | for(i = 1 ; i < NF ; i++) { \ 11 | if( $i == "release" ) { print int($(i+1)) ; exit } \ 12 | @@ -277,7 +277,7 @@ 13 | }' /etc/redhat-release) 14 | DISTRO_MINOR_VERSION=$($awk '/release/ && $NF != "release" { \ 15 | for(i = 1 ; i < NF ; i++) { \ 16 | - if( $i == "release" ) { print substr($(i+1), index($(i+1), ".")+1) ; exit } \ 17 | + if( $i == "release" ) { split($(i+1), a, ".") ; print a[2] ; exit } \ 18 | } \ 19 | }' /etc/redhat-release) 20 | [[ -z $DISTRO_VERSION ]] && DISTRO_VERSION=0 21 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handlers for gpfs 3 | 4 | - name: restart sshd 5 | become: yes 6 | service: 7 | name: sshd 8 | state: restarted 9 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/tasks/gpfs-configure.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Local system configuration for GPFS, including ssh keys and mount options 3 | 4 | - name: gpfs | nsd access ssh root 5 | become: yes 6 | lineinfile: 7 | backup: yes 8 | create: no 9 | dest: /etc/ssh/sshd_config 10 | line: "PermitRootLogin yes" 11 | regexp: '[\s#]*PermitRootLogin\s.*' 12 | state: present 13 | notify: 14 | - restart sshd 15 | 16 | - name: gpfs | nsd access ssh keys 17 | become: yes 18 | authorized_key: 19 | key: "{{ item }}" 20 | manage_dir: yes 21 | state: present 22 | user: root 23 | with_items: "{{ gpfs_public_keys | default([]) }}" 24 | 25 | # This regexp checks for the absence of gpfs_binaries_path in PATH 26 | # It won't work if there are multiple PATH statements in .bash_profile 27 | # https://coderwall.com/p/ynvi0q/updating-path-with-ansible-system-wide 28 | - name: gpfs | root path 29 | become: yes 30 | lineinfile: 31 | backrefs: yes 32 | backup: yes 33 | create: no 34 | dest: /root/.bash_profile 35 | line: 'PATH=\1\2:{{ gpfs_binaries_path }}\3' 36 | regexp: 'PATH=(["]*)((?!.*?{{ gpfs_binaries_path }}).*?)(["]*)$' 37 | state: present 38 | 39 | # Node-specific GPFS configuration 40 | # https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/File%20System%20Management 41 | 42 | - name: gpfs | set mount options 43 | become: yes 44 | copy: 45 | content: "{{ item.value }}" 46 | dest: /var/mmfs/etc/localMountOptions.{{ item.key }} 47 | when: item.value 48 | with_dict: "{{ gpfs_node_specific_mount_options | default ({}) }}" 49 | 50 | - name: gpfs | clear mount options 51 | become: yes 52 | file: 53 | path: /var/mmfs/etc/localMountOptions.{{ item.key }} 54 | state: absent 55 | when: not item.value 56 | with_dict: "{{ gpfs_node_specific_mount_options | default ({}) }}" 57 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/tasks/gpfs-install.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Install GPFS packages extracted and compiled by the gpfs-build tasks 3 | 4 | - name: gpfs | current kernel version 5 | fail: 6 | msg: "Current kernel {{ ansible_kernel }} must match specified GPFS kernel {{ gpfs_kernel_version }}" 7 | when: (gpfs_install_check_kernel_version == True) and (ansible_kernel != gpfs_kernel_version) 8 | 9 | - name: gpfs | download directory 10 | become: yes 11 | file: 12 | path: /opt/gpfs-installer 13 | state: directory 14 | 15 | # Note this is a relative user directory to avoid problems with running rsync as root 16 | - name: gpfs | copy rpms 17 | synchronize: 18 | dest: gpfs-rpms/ 19 | src: "{{ gpfs_local_rpm_dir }}/" 20 | 21 | - name: gpfs | remove packages 22 | become: yes 23 | yum: 24 | name: "{{ gpfs_remove_rpms }}" 25 | state: absent 26 | 27 | - name: gpfs | install installer packages 28 | become: yes 29 | yum: 30 | name: gpfs-rpms/1/{{ item }} 31 | state: present 32 | with_items: "{{ gpfs_installer_rpms }}" 33 | 34 | - name: gpfs | install patch packages 35 | become: yes 36 | yum: 37 | name: gpfs-rpms/2/{{ item }} 38 | state: present 39 | with_items: "{{ gpfs_patch_rpms }}" 40 | 41 | - name: gpfs | install kernel module 42 | become: yes 43 | yum: 44 | name: gpfs-rpms/3/{{ gpfs_kernel_rpm_name }} 45 | state: present 46 | 47 | - name: gpfs | install patch 48 | become: yes 49 | yum: 50 | name: patch 51 | state: present 52 | when: gpfs_local_patches | default(False) 53 | 54 | - name: gpfs | apply local patches 55 | become: yes 56 | patch: 57 | dest: "{{ item.dest }}" 58 | src: "{{ item.src }}" 59 | with_items: gpfs_local_patches | default([]) 60 | when: gpfs_local_patches 61 | 62 | # Systemd-219 workaround (see readme) 63 | 64 | - name: gpfs | get initd file status 65 | become: yes 66 | stat: 67 | path: /etc/init.d/gpfs 68 | register: initd_gpfs 69 | when: gpfs_enable_systemd_workaround | default(False) 70 | 71 | - name: gpfs | get initd file status 72 | become: yes 73 | file: 74 | path: /etc/init.d/gpfs 75 | state: absent 76 | when: (gpfs_enable_systemd_workaround | default(False)) and (initd_gpfs.stat.islnk) 77 | 78 | - name: gpfs | apply systemd workaround 79 | become: yes 80 | file: 81 | path: /etc/init.d/gpfs 82 | src: /usr/lpp/mmfs/bin/gpfsrunlevel 83 | state: hard 84 | when: gpfs_enable_systemd_workaround | default(False) 85 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/tasks/gpfs-repoinstall.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Install GPFS from an internal repo, compile and install kernel module 3 | 4 | - name: gpfs | setup repository 5 | become: yes 6 | template: 7 | backup: no 8 | dest: /etc/yum.repos.d/gpfs.repo 9 | src: gpfs-repo.j2 10 | when: "{{ (gpfs_repo_url_base | length > 0) or (gpfs_repo_url_updates | length > 0) }}" 11 | 12 | - name: gpfs | install kernel module build requirements 13 | become: yes 14 | yum: 15 | name: "{{ item }}" 16 | state: present 17 | with_items: 18 | - kernel-devel 19 | - cpp 20 | - gcc 21 | - gcc-c++ 22 | - rpm-build 23 | - kernel-headers 24 | 25 | - name: gpfs | install rpms 26 | become: yes 27 | yum: 28 | name: "{{ item }}" 29 | state: present 30 | disable_gpg_check: yes 31 | with_items: 32 | - gpfs.base 33 | - gpfs.docs 34 | - gpfs.ext 35 | - gpfs.gpl 36 | - gpfs.gskit 37 | - gpfs.msg.en_US 38 | 39 | # Autodetect the GPFS kernel and patch version 40 | 41 | - name: gpfs | current kernel version 42 | command: uname -r 43 | register: running_kernel_version 44 | always_run: True 45 | changed_when: False 46 | 47 | - name: gpfs | latest installed gpfs.gpl 48 | command: rpm -q gpfs.gpl --qf '%{VERSION}-%{RELEASE}' 49 | register: gpfs_gpl_version 50 | always_run: True 51 | changed_when: False 52 | 53 | - name: gpfs | expected kernel module rpm 54 | set_fact: 55 | gpfs_kernel_rpm: /root/rpmbuild/RPMS/x86_64/gpfs.gplbin-{{ running_kernel_version.stdout }}-{{ gpfs_gpl_version.stdout }}.x86_64.rpm 56 | 57 | - name: gpfs | build kernel module 58 | become: yes 59 | command: /usr/lpp/mmfs/bin/mmbuildgpl --buildrpm 60 | args: 61 | creates: "{{ gpfs_kernel_rpm }}" 62 | environment: 63 | LINUX_DISTRIBUTION: REDHAT_AS_LINUX 64 | 65 | - name: gpfs | install kernel module 66 | become: yes 67 | yum: 68 | name: " {{ gpfs_kernel_rpm }}" 69 | state: present 70 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Setup a GPFS client node 3 | 4 | 5 | - name: gpfs | check gpfs_local_rpm_dir defined 6 | assert: 7 | that: gpfs_local_rpm_dir is defined and gpfs_local_rpm_dir 8 | when: gpfs_build or gpfs_install 9 | 10 | - include: gpfs-build.yml 11 | when: gpfs_build 12 | 13 | - include: gpfs-install.yml 14 | when: gpfs_install 15 | 16 | - include: gpfs-repoinstall.yml 17 | when: gpfs_repoinstall 18 | 19 | - include: gpfs-configure.yml 20 | when: gpfs_configure 21 | -------------------------------------------------------------------------------- /ansible/roles/gpfs/templates/gpfs-repo.j2: -------------------------------------------------------------------------------- 1 | # GPFS yum repository 2 | # Disabled by default since GPFS requires a compiled kernel module, upgraded 3 | # with care. 4 | 5 | {% if gpfs_repo_url_base | length > 0 %} 6 | [gpfs-base] 7 | name=GPFS Repository 8 | baseurl={{ gpfs_repo_url_base }} 9 | enabled=1 10 | gpgcheck=0 11 | {% endif %} 12 | 13 | {% if gpfs_repo_url_updates | length > 0 %} 14 | [gpfs-updates] 15 | name=GPFS Repository 16 | baseurl={{ gpfs_repo_url_updates }} 17 | enabled=1 18 | gpgcheck=0 19 | {% endif %} 20 | -------------------------------------------------------------------------------- /ansible/roles/jenkinsslave/README.md: -------------------------------------------------------------------------------- 1 | Jenkins Slave 2 | ============= 3 | 4 | Install the pre-requisites for a node to be setup as a Jenkins slave. 5 | 6 | Role Variables 7 | -------------- 8 | 9 | Defaults: `defaults/main.yml` 10 | 11 | - `jenkinsuser`: The operating system user for the Jenkins slave 12 | - `jenkinsworkdir`: The Jenkins work directory 13 | - `authorized_key`: The Jenkins public ssh key 14 | - `gitconfig.user`: Git email address 15 | - `gitconfig.name`: Git full name 16 | - `gitconfig.githubuser`: GitHub user 17 | - `gitconfig.githubtoken`: GitHub token 18 | 19 | - `spacewalk`: For internal OME use: Set to `True` if node is also maintained in spacewalk (this disables some operations), default False. 20 | 21 | Dependencies 22 | ------------ 23 | 24 | Depends on the `java` role. 25 | 26 | Author Information 27 | ------------------ 28 | 29 | ome-devel@lists.openmicroscopy.org.uk 30 | -------------------------------------------------------------------------------- /ansible/roles/jenkinsslave/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/jenkinsslave 3 | 4 | jenkinsuser: hudson 5 | jenkinsworkdir: /opt/hudson 6 | authorized_key: "ssh-rsa KEY" 7 | snoopy_dir_path: "" 8 | gitconfig: 9 | user: email@example.org 10 | name: Firstname Lastname 11 | githubuser: example-user 12 | githubtoken: example-token 13 | 14 | spacewalk: False -------------------------------------------------------------------------------- /ansible/roles/jenkinsslave/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: openmicroscopy.java } 4 | - { role: openmicroscopy.basedeps } 5 | -------------------------------------------------------------------------------- /ansible/roles/jenkinsslave/tasks/setup_virtualenv.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: virtualenv | install python tools 4 | become: yes 5 | yum: 6 | name: "{{ item }}" 7 | state: present 8 | with_items: 9 | - python-pip 10 | - python-virtualenv 11 | 12 | - name: install python packages 13 | become: yes 14 | become_user: "{{ jenkinsuser }}" 15 | pip: 16 | name: "{{ item }}" 17 | virtualenv: "/home/{{ jenkinsuser }}/virtualenv" 18 | virtualenv_command: "virtualenv" 19 | virtualenv_site_packages: "yes" 20 | state: present 21 | with_items: 22 | - scc 23 | - omego 24 | - genshi yaclifw pytest PyYAML 25 | - Sphinx==1.2.3 epydoc Pygments>=1.2 26 | - "django>=1.8,<1.9" 27 | 28 | -------------------------------------------------------------------------------- /ansible/roles/jenkinsslave/templates/gitconfig.j2: -------------------------------------------------------------------------------- 1 | [user] 2 | email = {{ gitconfig.user }} 3 | name = {{ gitconfig.name }} 4 | [github] 5 | user = {{ gitconfig.githubuser }} 6 | token = {{ gitconfig.githubtoken }} 7 | -------------------------------------------------------------------------------- /ansible/roles/network/README.md: -------------------------------------------------------------------------------- 1 | Network 2 | ======= 3 | 4 | Set up custom network interface configurations for a server. 5 | 6 | 7 | Role Variables 8 | -------------- 9 | 10 | - `network_ifaces`: A list of dictionaries, one per network device, of network parameters which will be substituted into `templates/etc-sysconfig-network-scripts-ifcfg.j2`. 11 | - `network_ifaces[].device`: The device name. All other fields are optional, see the template for details. 12 | - `network_ifaces[].bondmaster`: If specified this NIC will be part of a bonded interface. If the `device` name matches `bondmaster` it will be set as the master, otherwise it will be a slave of `bondmaster`. 13 | - `network_disable_ifaces`: A list of network device names to be explicitly disabled, use this if you want to be sure the interface is disabled (as opposed to being auto-configured by the system). 14 | - `network_delete_ifaces`: A regular expression describing the network device name(s) to be removed (note this means the system may auto-configure them), use this for cleaning up spare configuration files. 15 | 16 | 17 | Example Playbook 18 | ---------------- 19 | 20 | # Simple network 21 | - hosts: localhost 22 | roles: 23 | - role: network 24 | network_ifaces: 25 | - device: eth0 26 | ip: 192.168.1.1 27 | netmask: 255.255.255.0 28 | type: ethernet 29 | gateway: 192.168.1.254 30 | dns1: 8.8.4.4 31 | dns2: 8.8.8.8 32 | 33 | # Bonded network combining eth0 and eth1 34 | - hosts: localhost 35 | roles: 36 | - role: network 37 | network_ifaces: 38 | - device: bond0 39 | ip: 192.168.1.1 40 | prefix: 24 41 | gateway: 192.168.1.254 42 | dns1: 8.8.4.4 43 | dns2: 8.8.8.8 44 | bondmaster: bond0 45 | - device: eth0 46 | bondmaster: bond0 47 | - device: eth1 48 | bondmaster: bond0 49 | 50 | 51 | Notes 52 | ----- 53 | 54 | - If you change the network settings it may be restarted, which means your connection from Ansible may be broken. 55 | - In some cases restarting the network is insufficient, you may need to reboot. 56 | - If you are using this role to set a network IP after a system has been PXEed you may need to temporarily set `ansible_host` in your host inventory if DNS isn't already setup for the host. 57 | 58 | 59 | Author Information 60 | ------------------ 61 | 62 | ome-devel@lists.openmicroscopy.org.uk 63 | -------------------------------------------------------------------------------- /ansible/roles/network/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handler for network configuration 3 | 4 | - name: restart network 5 | become: yes 6 | service: name=network state=restarted 7 | -------------------------------------------------------------------------------- /ansible/roles/network/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Setup network interfaces 3 | 4 | - name: network | find unwanted nics 5 | become: yes 6 | find: 7 | paths: /etc/sysconfig/network-scripts/ 8 | patterns: ifcfg-{{ network_delete_ifaces }} 9 | use_regex: True 10 | when: network_delete_ifaces is defined and network_delete_ifaces 11 | register: unwanted 12 | 13 | - name: network | remove nics 14 | become: yes 15 | file: 16 | path: "{{ item.path }}" 17 | state: absent 18 | with_items: "{{ unwanted.files | default([]) }}" 19 | notify: 20 | - restart network 21 | 22 | - name: network | setup nics 23 | become: yes 24 | template: 25 | backup: yes 26 | src: etc-sysconfig-network-scripts-ifcfg.j2 27 | dest: /etc/sysconfig/network-scripts/ifcfg-{{ item.device }} 28 | with_items: "{{ network_ifaces | default([]) }}" 29 | notify: 30 | - restart network 31 | 32 | - name: network | disable nics 33 | become: yes 34 | template: 35 | backup: yes 36 | src: etc-sysconfig-network-scripts-ifcfg-disabled.j2 37 | dest: /etc/sysconfig/network-scripts/ifcfg-{{ item }} 38 | with_items: "{{ network_disable_ifaces | default([]) }}" 39 | notify: 40 | - restart network 41 | 42 | - name: network | restart network if necessary before checking bonding 43 | meta: flush_handlers 44 | 45 | - name: network | check bonding 46 | stat: 47 | path: /proc/net/bonding/{{ item.bondmaster }} 48 | with_items: "{{ network_ifaces | default([]) }}" 49 | when: "'bondmaster' in item" 50 | register: checkbonds 51 | 52 | #- debug: 53 | # msg: "checkbonds.results.item: {{ item }}" 54 | # with_items: 55 | # - "{{ checkbonds.results }}" 56 | 57 | - name: network | check bonding active 58 | assert: 59 | that: 60 | - "item.stat.exists and not item.stat.isdir" 61 | # 'skipped' field may be missing if it wasn't skipped 62 | when: not item.skipped | default (False) 63 | with_items: "{{ checkbonds.results }}" 64 | -------------------------------------------------------------------------------- /ansible/roles/network/templates/etc-sysconfig-network-scripts-ifcfg-disabled.j2: -------------------------------------------------------------------------------- 1 | DEVICE={{ item }} 2 | BOOTPROTO=none 3 | ONBOOT=no 4 | NM_CONTROLLED=no 5 | -------------------------------------------------------------------------------- /ansible/roles/network/templates/etc-sysconfig-network-scripts-ifcfg.j2: -------------------------------------------------------------------------------- 1 | DEVICE={{ item.device }} 2 | {% if 'bootproto' in item %} 3 | BOOTPROTO={{ item.bootproto }} 4 | {% else %} 5 | BOOTPROTO=none 6 | {% endif %} 7 | {% if 'ipv6init' in item %} 8 | IPV6INIT={{ item.ipv6init }} 9 | {% else %} 10 | IPV6INIT=yes 11 | {% endif %} 12 | {% if 'mtu' in item and item['mtu'] %} 13 | MTU={{ item.mtu }} 14 | {% endif %} 15 | ONBOOT=yes 16 | {% if 'networkmanager' in item %} 17 | NM_CONTROLLED={{ item.networkmanager }} 18 | {% endif %} 19 | {% if 'ip' in item %} 20 | IPADDR={{ item.ip }} 21 | {% endif %} 22 | {% if 'netmask' in item %} 23 | NETMASK={{ item.netmask }} 24 | {% endif %} 25 | {% if 'prefix' in item %} 26 | PREFIX={{ item.prefix }} 27 | {% endif %} 28 | {% if 'devicetype' in item %} 29 | DEVICETYPE={{ item.devicetype }} 30 | {% endif %} 31 | {% if 'type' in item %} 32 | TYPE={{ item.type }} 33 | {% endif %} 34 | {% if 'hwaddr' in item %} 35 | HWADDR={{ item.hwaddr }} 36 | {% endif %} 37 | {% if 'gateway' in item %} 38 | GATEWAY={{ item.gateway }} 39 | {% endif %} 40 | {% if 'dns1' in item %} 41 | DNS1={{ item.dns1 }} 42 | {% endif %} 43 | {% if 'dns2' in item %} 44 | DNS2={{ item.dns2 }} 45 | {% endif %} 46 | {% if 'bridge' in item %} 47 | BRIDGE={{ item.bridge }} 48 | {% endif %} 49 | USERCTL=no 50 | {% if 'bondmaster' in item %} 51 | {% if item.bondmaster == item.device %} 52 | {% if 'type' not in item %} 53 | TYPE=Bond 54 | {% endif %} 55 | BONDING_MASTER=yes 56 | BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=active-backup primary=eno1 lacp_rate=0 arp_interval=0 ad_select=0" 57 | {% else %} 58 | MASTER={{ item.bondmaster }} 59 | SLAVE=yes 60 | {% endif %} 61 | {% endif %} 62 | {% if 'ovsbridge' in item %} 63 | OVS_BRIDGE={{ item.ovsbridge }} 64 | {% endif %} 65 | -------------------------------------------------------------------------------- /ansible/roles/nfs-ganesha-share/README.md: -------------------------------------------------------------------------------- 1 | NFS Ganesha Share 2 | ================= 3 | 4 | Manage NFS user mode (Ganesha) file shares (no authentication). 5 | 6 | Note if SELinux is enabled you may need modify the configure of the the shared directories (not handled by this role). 7 | 8 | 9 | Conflicts 10 | --------- 11 | 12 | The NFS Ganesha server conflicts with the standard kernel NFS server, you can only run one. 13 | 14 | 15 | Role Variables 16 | -------------- 17 | 18 | All variables are optional, though if `nfs_ganesha_shares` is unset the role is rather useless: 19 | - `nfs_ganesha_shares`: A list of dictionaries exports, hosts and options `{ path: /exported/directory, pseudopath: /pseudo/path (optional), fs: ganesha fs type (optional), clients: [{ host: host-cidr, access: RO|RW (optional), squash: None|Root|All (optional) }, ...] }` 20 | - `nfs_ganesha_default_log_level`: Change the default logging level 21 | 22 | The default options are `access: RO`, `squash: All`. 23 | Currently only the default of `fs: VFS` is supported by this role. 24 | For full details of configuration options see https://github.com/nfs-ganesha/nfs-ganesha/blob/master/src/config_samples/export.txt 25 | 26 | Note: during testing nfs-ganesha sometimes behaved inconsistently when processing the access rules in `/etc/ganesha/ganesha.conf`. 27 | I have no idea why. 28 | If you are trying to debug access problems note that at present (2016-07-21) the output of `showmount` may be incorrect. 29 | 30 | 31 | Example Playbook 32 | ---------------- 33 | 34 | - hosts: localhost 35 | roles: 36 | - role: nfs-ganesha-share 37 | nfs_ganesha_shares: 38 | - path: /srv/share1 39 | pseudopath: /share1 40 | clients: 41 | - host: "192.168.1.0/25" 42 | - path: /srv/share2 43 | pseudopath: /share2 44 | clients: 45 | - host: "192.168.1.0/25, 172.16.0.0/20" 46 | access: RW 47 | squash: Root 48 | 49 | 50 | Author Information 51 | ------------------ 52 | 53 | ome-devel@lists.openmicroscopy.org.uk 54 | -------------------------------------------------------------------------------- /ansible/roles/nfs-ganesha-share/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/nfs-gaensha-share 3 | 4 | # A list of shares, see README.md 5 | nfs_ganesha_shares: [] 6 | 7 | # Logging level, leave unset for default 8 | nfs_ganesha_default_log_level: 9 | -------------------------------------------------------------------------------- /ansible/roles/nfs-ganesha-share/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handlers for nfs-ganesha-share 3 | 4 | - name: restart nfs-ganesha 5 | become: yes 6 | service: 7 | name: nfs-ganesha 8 | state: restarted 9 | -------------------------------------------------------------------------------- /ansible/roles/nfs-ganesha-share/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/nfs-ganesha-share 3 | 4 | - name: system packages | install epel repo 5 | become: yes 6 | yum: 7 | name: epel-release 8 | state: present 9 | 10 | - name: nfs ganesha | install ganesha packages 11 | become: yes 12 | yum: 13 | name: "{{ item }}" 14 | state: present 15 | with_items: 16 | - nfs-ganesha 17 | - nfs-ganesha-vfs 18 | 19 | - name: nfs ganesha | configure shares 20 | become: yes 21 | template: 22 | backup: yes 23 | dest: /etc/ganesha/ganesha.conf 24 | src: ganesha-conf.j2 25 | notify: 26 | - restart nfs-ganesha 27 | 28 | - name: nfs ganesha | enable nfs-ganesha 29 | become: yes 30 | service: 31 | enabled: yes 32 | name: nfs-ganesha 33 | state: started 34 | -------------------------------------------------------------------------------- /ansible/roles/nfs-ganesha-share/templates/ganesha-conf.j2: -------------------------------------------------------------------------------- 1 | # Managed by Ansible 2 | 3 | {% if nfs_ganesha_default_log_level %} 4 | LOG 5 | { 6 | default_log_level = {{ nfs_ganesha_default_log_level }}; 7 | } 8 | {% endif %} 9 | 10 | {% for item in nfs_ganesha_shares %} 11 | EXPORT 12 | { 13 | # Export Id (mandatory, each EXPORT must have a unique Export_Id) 14 | Export_Id = {{ loop.index }}; 15 | 16 | # Exported path (mandatory) 17 | Path = {{ item.path }}; 18 | 19 | # Pseudo Path (required for NFS v4) 20 | Pseudo = {{ item.pseudopath | default(item.path) }}; 21 | 22 | # Required for access (default is None) 23 | {% for client in item.clients %} 24 | CLIENT 25 | { 26 | Clients = {{ client.host }}; 27 | Access_Type = {{ client.access | default('RO') }}; 28 | Squash = {{ client.squash | default('All') }}; 29 | } 30 | {% endfor %} 31 | 32 | # Exporting FSAL 33 | FSAL { 34 | Name = {{ item.fs | default('VFS') }}; 35 | } 36 | } 37 | {% endfor %} 38 | -------------------------------------------------------------------------------- /ansible/roles/omero-build-cpp/README.md: -------------------------------------------------------------------------------- 1 | OMERO Build C++ 2 | =============== 3 | 4 | C++ build requirements. 5 | 6 | Dependencies 7 | ------------ 8 | 9 | Depends on omero-build. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/roles/omero-build-cpp/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: omero-build } 4 | -------------------------------------------------------------------------------- /ansible/roles/omero-build-cpp/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-build-cpp 3 | 4 | - name: system packages | install C++ build tools 5 | become: yes 6 | yum: 7 | name: "@Development Tools" 8 | state: present 9 | -------------------------------------------------------------------------------- /ansible/roles/omero-build/README.md: -------------------------------------------------------------------------------- 1 | OMERO Build 2 | =========== 3 | 4 | Most build dependencies (except for C++). 5 | 6 | Requirements 7 | ------------ 8 | 9 | Depends on basedeps, java, ice. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/roles/omero-build/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/omero-build 3 | 4 | # A license key (default null) 5 | exe4j_license_key: 6 | 7 | spacewalk: False 8 | -------------------------------------------------------------------------------- /ansible/roles/omero-build/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: openmicroscopy.basedeps } 4 | - { role: openmicroscopy.java } 5 | - { role: openmicroscopy.ice } 6 | -------------------------------------------------------------------------------- /ansible/roles/omero-build/templates/cmake-path.sh.j2: -------------------------------------------------------------------------------- 1 | export PATH=/opt/cmake/bin:$PATH 2 | -------------------------------------------------------------------------------- /ansible/roles/omero-runtime/README.md: -------------------------------------------------------------------------------- 1 | OMERO Runtime 2 | ============= 3 | 4 | OMERO runtime dependencies. 5 | 6 | Mostly Python modules. 7 | 8 | Dependencies 9 | ------------ 10 | 11 | Depends on basedeps, java, ice. 12 | 13 | Author Information 14 | ------------------ 15 | 16 | ome-devel@lists.openmicroscopy.org.uk 17 | -------------------------------------------------------------------------------- /ansible/roles/omero-runtime/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: openmicroscopy.basedeps } 4 | - { role: openmicroscopy.java } 5 | - { role: openmicroscopy.omero-python-deps } 6 | - { role: openmicroscopy.ice } 7 | -------------------------------------------------------------------------------- /ansible/roles/omero-scripts-dependencies/README.md: -------------------------------------------------------------------------------- 1 | OMERO Scripts Dependencies 2 | ========================== 3 | 4 | Install the dependencies required by the OMERO scripts 5 | 6 | Note that the role is incomplete for CentOS 7. 7 | mencoder is not installed so the make_movie script 8 | cannot be used. 9 | 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk -------------------------------------------------------------------------------- /ansible/roles/omero-scripts-dependencies/tasks/debian.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-scripts-dependencies 3 | # if the os is ubuntu 4 | 5 | # dependencies for pdf export (figure) 6 | - name: system packages | install dependencies for pdf export 7 | become: yes 8 | apt: 9 | pkg: "{{ item }}" 10 | state: present 11 | with_items: 12 | - python-reportlab 13 | - python-markdown 14 | 15 | # dependency for make movie 16 | - name: system packages | install dependency for make movie 17 | become: yes 18 | apt: 19 | pkg: "{{ item }}" 20 | state: present 21 | with_items: 22 | - mencoder -------------------------------------------------------------------------------- /ansible/roles/omero-scripts-dependencies/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-scripts-dependencies 3 | 4 | - include: redhat.yml 5 | when: ansible_os_family == 'RedHat' 6 | 7 | - include: debian.yml 8 | when: ansible_os_family == 'Debian' -------------------------------------------------------------------------------- /ansible/roles/omero-scripts-dependencies/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-scripts-dependencies 3 | # if the os is centOS 4 | 5 | - name: system packages | install epel repo 6 | become: yes 7 | yum: 8 | name: epel-release 9 | state: present 10 | 11 | # dependencies for pdf export (figure) 12 | - name: system packages | install dependencies for pdf export 13 | become: yes 14 | yum: 15 | name: "{{ item }}" 16 | state: present 17 | with_items: 18 | - python-reportlab 19 | - python-markdown 20 | -------------------------------------------------------------------------------- /ansible/roles/omero-web-apps-build-dependencies/README.md: -------------------------------------------------------------------------------- 1 | OMERO Web Applications Build Dependencies 2 | ========================================== 3 | 4 | Install the dependencies required to build some 5 | of the Web applications e.g. OMERO.figure and OMERO.iviewer 6 | 7 | Author Information 8 | ------------------ 9 | 10 | ome-devel@lists.openmicroscopy.org.uk 11 | -------------------------------------------------------------------------------- /ansible/roles/omero-web-apps-build-dependencies/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-web-apps-build-dependencies 3 | 4 | - include: redhat.yml 5 | when: ansible_os_family == 'RedHat' 6 | -------------------------------------------------------------------------------- /ansible/roles/omero-web-apps-build-dependencies/tasks/redhat.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/omero-web-apps-build-dependencies 3 | # if the os is centOS 4 | 5 | - name: system packages | install epel repo 6 | become: yes 7 | yum: 8 | name: epel-release 9 | state: present 10 | 11 | - name: system packages | install build tools 12 | become: yes 13 | yum: 14 | name: "{{ item }}" 15 | state: present 16 | with_items: 17 | - ant 18 | - ant-contrib 19 | 20 | - name: Install nodejs 21 | become: yes 22 | yum: 23 | name: https://rpm.nodesource.com/pub_6.x/el/7/x86_64/nodesource-release-el7-1.noarch.rpm 24 | state: present 25 | 26 | - name: Install npm 27 | become: yes 28 | yum: 29 | name: npm 30 | state: present 31 | 32 | - name: Install grunt command line tool 33 | become: yes 34 | npm: 35 | name: grunt-cli 36 | global: yes 37 | -------------------------------------------------------------------------------- /ansible/roles/openldap-devel/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/openldap-devel 3 | 4 | - name: install openldap-devel 5 | become: yes 6 | yum: 7 | name: "{{ item }}" 8 | state: present 9 | with_items: 10 | - openldap-devel 11 | -------------------------------------------------------------------------------- /ansible/roles/openstack-prepare/README.md: -------------------------------------------------------------------------------- 1 | OpenStack Prepare 2 | ================= 3 | 4 | Basic system setup before installing OpenStack with Packstack. 5 | 6 | 7 | Author Information 8 | ------------------ 9 | 10 | ome-devel@lists.openmicroscopy.org.uk 11 | -------------------------------------------------------------------------------- /ansible/roles/openstack-prepare/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Handlers for openstack-prepare 3 | 4 | - name: restart ssh 5 | become: yes 6 | service: 7 | name: sshd 8 | state: restarted 9 | -------------------------------------------------------------------------------- /ansible/roles/openstack-prepare/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Setup system and install Packstack 3 | 4 | - name: openstack | Setup RDO repository 5 | become: yes 6 | yum: 7 | # pkg: https://www.rdoproject.org/repos/rdo-release.rpm 8 | pkg: https://repos.fedorapeople.org/repos/openstack/openstack-mitaka/rdo-release-mitaka-6.noarch.rpm 9 | state: present 10 | 11 | - name: openstack | Install packstack 12 | become: yes 13 | yum: 14 | pkg: openstack-packstack 15 | state: present 16 | when: openstack_master_node | default(False) 17 | 18 | - name: openstack | Enable root ssh 19 | become: yes 20 | lineinfile: 21 | backup: yes 22 | create: no 23 | dest: /etc/ssh/sshd_config 24 | line: "PermitRootLogin yes" 25 | regexp: '[\s#]*PermitRootLogin\s.*' 26 | state: present 27 | notify: 28 | - restart ssh 29 | 30 | - name: openstack | Enable sudoers.d 31 | become: yes 32 | lineinfile: 33 | backup: no 34 | create: no 35 | dest: /etc/sudoers 36 | insertafter: EOF 37 | line: "#includedir /etc/sudoers.d" 38 | state: present 39 | validate: "visudo -cf %s" 40 | 41 | - name: openstack | Disable Network Manager 42 | become: yes 43 | service: 44 | name: NetworkManager 45 | enabled: no 46 | state: stopped 47 | -------------------------------------------------------------------------------- /ansible/roles/python-devel/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/python-devel 3 | 4 | - name: system packages | install epel repo 5 | become: yes 6 | yum: 7 | name: epel-release 8 | state: present 9 | 10 | - name: install python-devel 11 | become: yes 12 | yum: 13 | name: "{{ item }}" 14 | state: present 15 | with_items: 16 | - python-devel 17 | 18 | - name: virtualenv | install python tools 19 | become: yes 20 | yum: 21 | name: "{{ item }}" 22 | state: present 23 | with_items: 24 | - python-pip 25 | - python-setuptools 26 | - python-virtualenv 27 | -------------------------------------------------------------------------------- /ansible/roles/server-swap/README.md: -------------------------------------------------------------------------------- 1 | Server Swap 2 | =========== 3 | 4 | Configure swap for a server. 5 | 6 | TODO: This is a work in progress. 7 | 8 | 9 | Role Variables 10 | -------------- 11 | 12 | `sysctl_vm_swappiness`: vm.swappiness 13 | 14 | 15 | Author Information 16 | ------------------ 17 | 18 | ome-devel@lists.openmicroscopy.org.uk 19 | -------------------------------------------------------------------------------- /ansible/roles/server-swap/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # defaults file for roles/server-swap 3 | 4 | sysctl_vm_swappiness: 1 5 | -------------------------------------------------------------------------------- /ansible/roles/server-swap/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/server-swap 3 | 4 | - name: sysctl | set vm swappiness 5 | become: yes 6 | sysctl: 7 | ignoreerrors: no 8 | reload: yes 9 | state: present 10 | name: vm.swappiness 11 | value: "{{ sysctl_vm_swappiness }}" 12 | -------------------------------------------------------------------------------- /ansible/roles/sphinx-build/README.md: -------------------------------------------------------------------------------- 1 | Sphinx documentation build 2 | ========================== 3 | 4 | Documentation build dependencies. 5 | 6 | Requirements 7 | ------------ 8 | 9 | Depends on python. 10 | 11 | Author Information 12 | ------------------ 13 | 14 | ome-devel@lists.openmicroscopy.org.uk 15 | -------------------------------------------------------------------------------- /ansible/roles/sphinx-build/meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | dependencies: 3 | - { role: python-devel } -------------------------------------------------------------------------------- /ansible/roles/sphinx-build/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/sphinx-build 3 | 4 | 5 | - name: Sphinx documentation packages | pip install packages 6 | become: yes 7 | pip: 8 | name: "{{ item.name }}" 9 | state: present 10 | version: "{{ item.version }}" 11 | with_items: 12 | - name: sphinx 13 | version: 1.6.3 14 | -------------------------------------------------------------------------------- /ansible/roles/yum-excludes/README.md: -------------------------------------------------------------------------------- 1 | Yum Excludes 2 | ============ 3 | 4 | Manage a list of packages the are excluded from updates. 5 | This role assumes it has full control of the `excludes=` line in `/etc/yum.conf`, manual changes will be lost. 6 | 7 | 8 | Role Variables 9 | -------------- 10 | 11 | Optional variables: 12 | 13 | - `yum_excludes_packages`: A of packages to exclude, for example `[kernel-*, systemd-*]`. 14 | 15 | 16 | Author Information 17 | ------------------ 18 | 19 | ome-devel@lists.openmicroscopy.org.uk 20 | -------------------------------------------------------------------------------- /ansible/roles/yum-excludes/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # tasks file for roles/yum-excludes 3 | 4 | - name: system packages | exclude 5 | become: yes 6 | ini_file: 7 | backup: yes 8 | dest: /etc/yum.conf 9 | option: exclude 10 | section: main 11 | state: present 12 | value: "{{ (yum_excludes_packages | default([])) | join(',') }}" 13 | -------------------------------------------------------------------------------- /ansible/scripts/README.md: -------------------------------------------------------------------------------- 1 | Scripts for snapshotting IDR OpenStack volumes 2 | ============================================== 3 | 4 | The process of snapshotting multiple volumes and 5 | migrating them between clouds is non-trivial. 6 | 7 | Scripts for doing so are provided here. 8 | 9 | For steps 2-4 you should `source settings.env`, optionally setting the `PREFIX` and `DATE` environment variables as required. 10 | 11 | 1. os-idr-snapshot.sh - Snapshot the instances. 12 | Requires `vm_prefix` as the first argument. 13 | You must wait for the statuses of the snapshots to change from `creating` to `available` before continuing 14 | 2. os-idr-mkvol.sh - Convert the snapshots into volumes. 15 | Wait for the status to change to `available`. 16 | 3. os-idr-mkimg.sh - Convert the volumes into images. 17 | 4. os-idr-download.sh - Download the images locally. 18 | 5. os-idr-upload.sh - Upload the images to a new cloud. 19 | Be sure to source a second openrc file. 20 | 21 | Todos 22 | ----- 23 | 24 | These scripts will attempt to wait for the background openstack processes to finish. 25 | However they do not distinguish between operations performed by this script and those performed by others, so for instance if someone else is creating volumes or images these scripts will wait for all tasks to complete, not just your own. 26 | -------------------------------------------------------------------------------- /ansible/scripts/os-idr-download.sh: -------------------------------------------------------------------------------- 1 | set -e 2 | set -u 3 | #source settings.env 4 | 5 | openstack image list -f json | jq -r ".[] | select(.Name | \ 6 | match(\"^$PREFIX.+(database-db|omero-data).+$DATE\")) | \ 7 | [.ID, .Name] | join(\",\")" | \ 8 | while IFS='' read -r line; do 9 | IFS=, read -a arr <<< "$line" 10 | echo "Downloading image ${arr[1]}" 11 | time openstack image save --file "${arr[1]}.raw" "${arr[0]}" 12 | done 13 | -------------------------------------------------------------------------------- /ansible/scripts/os-idr-mkimg.sh: -------------------------------------------------------------------------------- 1 | set -e 2 | set -u 3 | #source settings.env 4 | 5 | openstack volume list -f json | jq -r ".[] | select(.[\"Display Name\"] | \ 6 | match(\"^$PREFIX.+(database-db|omero-data).+$DATE\")) | \ 7 | [.ID, .[\"Display Name\"]] | join(\",\")" | \ 8 | while IFS='' read -r line; do 9 | IFS=, read -a arr <<< "$line" 10 | echo "Creating image ${arr[1]}" 11 | # TODO: open a bug 12 | # openstack image create behaves incorrectly in a script when the shell is non-interactive 13 | # https://github.com/openstack/python-glanceclient/blob/12e92558e50d0aa200d2ca0f98a7110e0adce922/glanceclient/common/utils.py#L376 14 | #openstack image create --volume "${arr[0]}" "${arr[1]}" 15 | time cinder upload-to-image "${arr[0]}" "${arr[1]}" 16 | done 17 | 18 | while [ $(openstack image list -f json | jq -c ".[] | select(.Status==\"saving\")" | wc -l) -ne 0 ]; do 19 | echo -n . 20 | sleep 1 21 | done 22 | echo Done 23 | -------------------------------------------------------------------------------- /ansible/scripts/os-idr-mkvol.sh: -------------------------------------------------------------------------------- 1 | set -e 2 | set -u 3 | #source settings.env 4 | 5 | openstack snapshot list -f json | jq -r ".[] | select(.Name | \ 6 | match(\"^$PREFIX.+(database-db|omero-data).+$DATE\")) | \ 7 | [.ID, .Name, .Size|tostring] | join(\",\")" | \ 8 | while IFS='' read -r line; do 9 | IFS=, read -a arr <<< "$line" 10 | description="Created $TODAY from Snapshot ${arr[0]} ${arr[1]}" 11 | echo "Creating volume ${arr[1]}" 12 | time openstack volume create --snapshot "${arr[0]}" "${arr[1]}" \ 13 | --size "${arr[2]}" --description "$description" 14 | done 15 | 16 | while [ $(openstack volume list -f json | jq -c ".[] | select(.Status==\"creating\")" | wc -l) -ne 0 ]; do 17 | echo -n . 18 | sleep 1 19 | done 20 | echo Done 21 | -------------------------------------------------------------------------------- /ansible/scripts/os-idr-snapshot.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Snapshot IDR volumes and instances 3 | 4 | # Attempt to continue on error 5 | #set -e 6 | set -u 7 | 8 | if [ $# -ne 1 ]; then 9 | echo "USAGE: $(basename "$0") vm_prefix" 10 | exit 1 11 | fi 12 | 13 | vm_prefix="$1" 14 | today=$(date +%Y%m%d) 15 | errors=0 16 | 17 | for vm in \ 18 | database \ 19 | omero \ 20 | proxy \ 21 | a-dockermanager \ 22 | management \ 23 | ; do 24 | server="$vm_prefix-$vm" 25 | echo "Snapshotting server $server" 26 | openstack server image create --name "$server-$today" "$server" -f yaml 27 | [ $? -eq 0 ] || let errors++ 28 | echo 29 | done 30 | 31 | for vol in \ 32 | database-db \ 33 | omero-data \ 34 | proxy-nginxcache \ 35 | a-dockermanager-jupyter \ 36 | ; do 37 | volume="$vm_prefix-$vol" 38 | echo "Snapshotting volume $volume" 39 | openstack snapshot create --force --name "$volume-$today" "$volume" -f yaml 40 | [ $? -eq 0 ] || let errors++ 41 | echo 42 | done 43 | 44 | if [ $errors -ne 0 ]; then 45 | echo "ERROR: $errors snapshots failed" 46 | exit $errors 47 | fi 48 | -------------------------------------------------------------------------------- /ansible/scripts/os-idr-upload.sh: -------------------------------------------------------------------------------- 1 | set -e 2 | set -u 3 | 4 | if [ $# -ne 2 ]; then 5 | echo "USAGE: $(basename "$0") image-name file-path" 6 | echo "Uploads the given image to Openstack as a new image using glance" 7 | exit 1 8 | fi 9 | NAME=$1 10 | FILE=$2 11 | # Upload 12 | #exec openstack image create --file="$FILE" "$NAME" 13 | exec glance image-create --name "$NAME" --disk-format=qcow2 --container-format=bare --file="$FILE" --progress 14 | -------------------------------------------------------------------------------- /ansible/scripts/settings.env: -------------------------------------------------------------------------------- 1 | TODAY=$(date +%Y%m%d) 2 | DATE=${DATE:-$TODAY} 3 | #PREFIX=${PREFIX:-demo2} 4 | #PREFIX=${PREFIX:-reference} 5 | PREFIX=${PREFIX:-} 6 | export TODAY DATE PREFIX 7 | -------------------------------------------------------------------------------- /ansible/server-state-playbooks/.gitignore: -------------------------------------------------------------------------------- 1 | roles 2 | -------------------------------------------------------------------------------- /ansible/server-state-playbooks/README.md: -------------------------------------------------------------------------------- 1 | # Server-state Playbooks 2 | 3 | The playbooks for the production servers run by the OME team are in their own repository: https://github.com/openmicroscopy/prod-playbooks 4 | -------------------------------------------------------------------------------- /ansible/tests/haproxy.yml: -------------------------------------------------------------------------------- 1 | - hosts: haproxy 2 | 3 | roles: 4 | 5 | - role: selinux-utils 6 | - role: haproxy 7 | haproxy_frontends: 8 | - name: frontend-web 9 | address: '*:80' 10 | mode: http 11 | backend: backend-web 12 | - name: frontend-ssh 13 | address: '*:10022' 14 | mode: tcp 15 | backend: backend-ssh 16 | haproxy_backends: 17 | - name: backend-web 18 | mode: http 19 | balance_method: roundrobin 20 | servers: 21 | - name: backend-web-1 22 | address: www.openmicroscopy.org:80 23 | - name: backend-web-2 24 | address: idr-demo.openmicroscopy.org:80 25 | options: 26 | - forwardfor 27 | - httpchk GET /about/ 28 | cookie: "SERVERID insert indirect" 29 | - name: backend-ssh 30 | mode: tcp 31 | servers: 32 | - name: backend-ssh-1 33 | address: 10.0.0.1:22 34 | -------------------------------------------------------------------------------- /ansible/tests/java.yml: -------------------------------------------------------------------------------- 1 | # JRE only 2 | - hosts: java 3 | roles: 4 | - role: java 5 | 6 | tasks: 7 | - stat: 8 | path: /usr/bin/java 9 | register: st1_java 10 | 11 | - stat: 12 | path: /usr/bin/javac 13 | register: st1_javac 14 | 15 | # WARNING: This will only pass on the first run, since the next playbook 16 | # invalidates this assert 17 | - assert: 18 | that: 19 | - "st1_java.stat.exists" 20 | - "not st1_javac.stat.exists" 21 | 22 | 23 | # JRE and JDK 24 | - hosts: java 25 | roles: 26 | - role: java 27 | java_jdk_install: True 28 | 29 | tasks: 30 | - stat: 31 | path: /usr/bin/javac 32 | register: st2_javac 33 | 34 | - assert: 35 | that: 36 | - "st2_javac.stat.exists" 37 | -------------------------------------------------------------------------------- /ansible/tests/nginx-mainline.yml: -------------------------------------------------------------------------------- 1 | - hosts: nginx-mainline 2 | roles: 3 | - role: nginx 4 | nginx_stable_repo: no 5 | 6 | tasks: 7 | - command: /usr/sbin/nginx -v 8 | register: ng_version 9 | changed_when: False 10 | 11 | # This will need to be updated when the major version is incremented 12 | - assert: 13 | that: > 14 | "{{ ng_version.stderr | regex_replace('.+/(\d+\.\d+)\.\d+$', '\1') }}" 15 | == "1.11" 16 | -------------------------------------------------------------------------------- /ansible/tests/nginx.yml: -------------------------------------------------------------------------------- 1 | - hosts: nginx 2 | roles: 3 | - role: nginx 4 | 5 | tasks: 6 | - command: /usr/sbin/nginx -v 7 | register: ng_version 8 | changed_when: False 9 | 10 | # This will need to be updated when the major version is incremented 11 | - assert: 12 | that: > 13 | "{{ ng_version.stderr | regex_replace('.+/(\d+\.\d+)\.\d+$', '\1') }}" 14 | == "1.10" 15 | -------------------------------------------------------------------------------- /ansible/tests/omero-server-ice35.yml: -------------------------------------------------------------------------------- 1 | - hosts: omero-server-ice35 2 | 3 | roles: 4 | 5 | - role: omero-server 6 | postgresql_users_databases: 7 | - user: omero 8 | password: omero 9 | databases: [omero] 10 | omero_systemd_setup: True 11 | 12 | 13 | tasks: 14 | 15 | - name: Get icegridnode version seen by OMERO 16 | become: yes 17 | become_user: omero 18 | command: icegridnode --version 19 | register: ice_check_ver 20 | always_run: yes 21 | 22 | - name: Check ice version 23 | assert: 24 | that: 25 | - '{{ ice_check_ver.stderr | version_compare("3.5", ">=") }}' 26 | - '{{ ice_check_ver.stderr | version_compare("3.6", "<") }}' 27 | 28 | - name: Get omero version 29 | become: yes 30 | become_user: omero 31 | command: /home/omero/OMERO.server/bin/omero version 32 | register: omero_check_ver 33 | always_run: yes 34 | 35 | - name: Check OMERO version 36 | assert: 37 | that: 38 | - '"-ice35-" in "{{ omero_check_ver.stdout }}"' 39 | 40 | # If these commands return with exit code 0 they've succeeded 41 | # TODO: If these commands hang it means login failed and omero is 42 | # interactively prompting for a response. Fixing this requires a code 43 | # change to OMERO.cli 44 | 45 | - name: Test OMERO root login 46 | command: "/home/omero/OMERO.server/bin/omero login -C -s localhost -u root -w 'omero'" 47 | -------------------------------------------------------------------------------- /ansible/tests/omero-server-ice36.yml: -------------------------------------------------------------------------------- 1 | - hosts: omero-server-ice36 2 | 3 | roles: 4 | 5 | - role: omero-server 6 | ice_version: "3.6" 7 | postgresql_users_databases: 8 | - user: omero 9 | password: omero 10 | databases: [omero] 11 | omero_systemd_setup: True 12 | omero_release: OMERO-DEV-latest 13 | # TODO: web is currently disabled because omero-user requires a development 14 | # build of OMERO.server but this has different web requirements 15 | omero_web_install: False 16 | 17 | - role: omero-user 18 | omero_user_admin_pass: "non default password" 19 | omero_user_reset_root_password: "{{ omero_user_admin_pass }}" 20 | omero_group_create: 21 | - name: "Test Group 1" 22 | type: read-only 23 | omero_user_create: 24 | - login: test-user-1 25 | firstname: Test 26 | lastname: User 27 | password: "user password" 28 | groups: '--group-name "Test Group 1"' 29 | 30 | 31 | tasks: 32 | 33 | - name: Get icegridnode version seen by OMERO 34 | become: yes 35 | become_user: omero 36 | command: icegridnode --version 37 | register: ice_check_ver 38 | always_run: yes 39 | 40 | - name: Check ice version 41 | assert: 42 | that: 43 | - '{{ ice_check_ver.stderr | version_compare("3.6", ">=") }}' 44 | - '{{ ice_check_ver.stderr | version_compare("3.7", "<") }}' 45 | 46 | - name: Get omero version 47 | become: yes 48 | become_user: omero 49 | command: /home/omero/OMERO.server/bin/omero version 50 | register: omero_check_ver 51 | always_run: yes 52 | 53 | - name: Check OMERO version 54 | assert: 55 | that: 56 | - '"-ice36-" in "{{ omero_check_ver.stdout }}"' 57 | 58 | 59 | # If these commands return with exit code 0 they've succeeded 60 | # TODO: If these commands hang it means login failed and omero is 61 | # interactively prompting for a response. Fixing this requires a code 62 | # change to OMERO.cli 63 | 64 | - name: Test OMERO root login 65 | command: "/home/omero/OMERO.server/bin/omero login -C -s localhost -u root -w 'non default password'" 66 | 67 | - name: Test OMERO user login 68 | command: "/home/omero/OMERO.server/bin/omero login -C -s localhost -u test-user-1 -w 'user password'" 69 | -------------------------------------------------------------------------------- /ansible/uod-docker.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for managing a docker instance in the 3 | # openstack cloud. 4 | 5 | - hosts: docker-hosts 6 | vars: 7 | ansible_user: centos 8 | roles: 9 | - role: storage-volume-initialise 10 | storage_volume_initialise_device: /dev/vdb 11 | storage_volume_initialise_mount: /var/lib/docker 12 | - role: basedeps 13 | - role: system-monitor-agent 14 | - role: versioncontrol-utils 15 | - role: docker 16 | docker_groupmembers: [centos] 17 | - role: sudoers 18 | - role: local-accounts 19 | -------------------------------------------------------------------------------- /ansible/uod-slave.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Playbook for managing a Jenkins slave in the 3 | # openstack cloud. 4 | 5 | # Duplicate of part of uod-docker.yml for the /opt/hudson directory 6 | - hosts: ci-jenkins-linux 7 | vars: 8 | ansible_user: centos 9 | roles: 10 | - role: storage-volume-initialise 11 | storage_volume_initialise_device: /dev/vdc 12 | storage_volume_initialise_mount: /opt/hudson 13 | 14 | - include: uod-docker.yml 15 | 16 | - include: ci-deployment.yml 17 | -------------------------------------------------------------------------------- /ansible/vendor/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | OME Infrastructure Docs 2 | ======================= 3 | 4 | General documentation and workflows related to the OME's system infrastructure. 5 | Although these documents are tailored for the OME's systems they should be useful for others wishing to setup similar infrastructure. 6 | 7 | The infrastructure repository is centred around the use of Ansible, and a lot of information can be found in the README files of individual Ansible roles. 8 | This folder contains higher-level documentation, or information that doesn't fit elsewhere. 9 | 10 | Inevitably some private configuration information has been omitted. 11 | 12 | 13 | Ansible 14 | ------- 15 | 16 | - [Ansible](ansible/ansible.md): Overview of Ansible, the configuration management system. 17 | - [Example workflows](ansible/example_workflows.md): Examples of provisioning new hosts, and running Ansible playbooks. 18 | - [Contributing](ansible/contributing.md): Suggestions on submitting modifications and extensions to the OME ansible roles and playbooks. 19 | 20 | 21 | Storage 22 | ------- 23 | 24 | - [GPFS](storage/gpfs.md): useful commands for managing GPFS filesystems. 25 | -------------------------------------------------------------------------------- /docs/ansible/ansible.md: -------------------------------------------------------------------------------- 1 | # Infrastructure and Ansible 2 | 3 | This repository stores all non-private information required for setting up production servers with Ansible in the Open Microscopy environment. 4 | This includes productions services, and continuous infrastructure/deployment servers, but does not include short-term test servers or infrastructure which does not need to be maintained long term. 5 | 6 | Since it is a relatively new initiative and is still under heavy development it may be necessary to make breaking changes. 7 | 8 | Wherever possible configuration data is included in this public repository, but inevitably some private configuration data (host and group variables) are not included. 9 | In many cases it is possible to run a role using the default role arguments, the main exceptions being roles related to hardware configuration, or which interact with external services. 10 | 11 | 12 | ## Overview 13 | 14 | At present there are effectively two groups of hosts and playbooks, `idr-*` and `ci-*`. 15 | Both share the same set of roles. 16 | 17 | Some of the roles are directly related to managing dependencies for production OMERO servers- these should always default to the current recommended versions for the latest OMERO release. 18 | If necessary optional variables can be added to support alternatives. 19 | 20 | The inventory files, and group and host specific configuration files, are currently held in a private repository. 21 | 22 | `group_vars` contains variables which are common to most hosts in that group. 23 | In general most configuration that differs from the role defaults should be done here. 24 | 25 | `host_vars` contains host specific configuration e.g. network configuration, or overrides/additions to variables specified in `group_vars` 26 | 27 | There are several playbooks, each corresponding to a host group. 28 | Since host-groups correspond to servers with a similar purpose this provides an easy way to manage groups of services (note that a server can be in multiple groups). 29 | In addition there are high-level playbooks that include the individual group playbooks, to allow the entire infrastructure to be managed as a whole. 30 | 31 | For examples of running playbooks from this repository see [example_workflows.md]. 32 | -------------------------------------------------------------------------------- /docs/ansible/contributing.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | General guidelines for contributing to the Ansible-based 4 | infrastructure used by OME. 5 | 6 | * Roles 7 | - SemVer is used for developing roles. 8 | - Production roles should be distributed via Galaxy. 9 | - Prefix variables should be prefixed with role names. 10 | - Molecule should be used for testing complex roles. 11 | 12 | * Playbooks 13 | - Prefer the long-format 14 | -------------------------------------------------------------------------------- /docs/ansible/galaxy_openmicroscopy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ome/infrastructure/e37b70ae1e2098cf8651bf1d462d3fafd89706c1/docs/ansible/galaxy_openmicroscopy.png -------------------------------------------------------------------------------- /docs/ansible/installation.md: -------------------------------------------------------------------------------- 1 | ## OME Ansible installation 2 | 3 | This repository uses Ansible 2.0 features. See 4 | [the Ansible documentation](http://docs.ansible.com/ansible/intro_installation.html) 5 | for full installation instructions. 6 | 7 | ### Installation 8 | 9 | - Create a virtual environment and install the Ansible requirements (including 10 | `shade` for using with OpenStack): 11 | 12 | virtualenv ~/venvs/ansible 13 | ~/venvs/ansible/bin/pip install -r requirements.txt 14 | 15 | - Clone this repository: 16 | 17 | git clone https://github.com/openmicroscopy/infrastructure.git 18 | 19 | - Execute the following commands from the `ansible` subdirectory of the 20 | infrastructure repository. 21 | 22 | - Download the necessary Ansible roles: 23 | 24 | ansible-galaxy install -r requirements.yml 25 | 26 | - Clone the repository containing the inventory, host and group vars files. 27 | Ansible will automatically look for `host_vars` and `group_vars` 28 | directories in the parent directory of the inventory file. This should be 29 | located at `../../ansible/inventory` such that `-i ../../ansible/inventory` 30 | would be correct. 31 | 32 | ### Examples 33 | 34 | In the following examples replace example-hosts with the private host inventory file 35 | 36 | Dry-run `ci-provision.yml` for all hosts listed in `ci-provision.yml`: 37 | - `-u` Login as this user 38 | - `--ask-become-pass` prompt for sudo password 39 | - `-C` Dry-run mode 40 | - `-v` Verbose (repeat to increase verbosity) 41 | 42 | Note this may fail since some tasks are dependent on others being completed: 43 | 44 | ansible-playbook -u $USERNAME --ask-become-pass -C -v ci-provision.yml 45 | 46 | Run `provision.yml`: 47 | 48 | ansible-playbook -u $USERNAME --ask-become-pass ci-provision.yml 49 | 50 | Run `provision.yml` for all subset of the hosts or groups listed in `provision.yml`: 51 | 52 | ansible-playbook -u $USERNAME --ask-become-pass ci-provision.yml --limit $HOST_OR_GROUP_NAME 53 | 54 | List the hosts that would be targeted by a command, don't do anything else: 55 | 56 | ansible-playbook ci-provision.yml --list-hosts 57 | 58 | 59 | Playbooks which do not alter hardware can often be tested in Docker instead of a full VM, for example by using the [omero-ssh](https://github.com/manics/ome-docker/blob/omero-ssh/omero-ssh/Dockerfile) image: 60 | 61 | docker run -d omero-ssh 62 | # Optional: 63 | ssh-copy-id omero@172.17.1.1 64 | # Pass -K if sudo requires a password, and -k if ssh keys aren't setup 65 | ansible-playbook -i etc/test-hosts -u omero ci-deployment.yml -bv 66 | 67 | -------------------------------------------------------------------------------- /docs/openstack/centos-image.md: -------------------------------------------------------------------------------- 1 | # CentOS OpenStack image creation 2 | 3 | OpenStack images for CentOS are available from 4 | [cloud.centos.org](https://cloud.centos.org), including 5 | current images for: 6 | 7 | - [CentOS 6](https://cloud.centos.org/centos/6/images/CentOS-6-x86_64-GenericCloud.qcow2.xz) 8 | - [CentOS 7](https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2.xz) 9 | 10 | Follow the [local setup](local-setup.md) instructions before starting. 11 | 12 | ## Image creation 13 | 14 | Download image from the above URL, then run: 15 | 16 | ```sh 17 | openstack image create --file ~/Downloads/CentOS-6-x86_64-GenericCloud.qcow2 --disk-format qcow2 --container-format bare "CentOS 6" 18 | openstack image create --file ~/Downloads/CentOS-7-x86_64-GenericCloud.qcow2 --disk-format qcow2 --container-format bare "CentOS 7" 19 | ``` 20 | -------------------------------------------------------------------------------- /docs/openstack/idr-openstack-setup-tenant.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # Setup default networks and security groups for an OpenStack tenant 3 | 4 | set -eu 5 | 6 | TENANT=${1:-} 7 | SUBNET=${2:-} 8 | DNS1=${3:-} 9 | DNS2=${4:-} 10 | if [ -z "$TENANT" -o -z "$SUBNET" -o -z "$DNS1" ]; then 11 | echo "ERROR: Usage: `basename $0` NEW-TENANT SUBNET DNS1 [DNS2]" 12 | echo " E.g. `basename $0` new-tenant 192.168.1.0/24 8.8.4.4 8.8.8.8" 13 | exit 2 14 | fi 15 | 16 | DNS="--dns-nameserver $DNS1" 17 | if [ -n "$DNS2" ]; then 18 | DNS="$DNS --dns-nameserver $DNS2" 19 | fi 20 | 21 | # Create a private network 22 | 23 | neutron net-create ${TENANT}_network 24 | neutron subnet-create --name ${TENANT}_subnet ${TENANT}_network ${SUBNET} ${DNS} 25 | 26 | # Create a router to connect the external and private networks 27 | 28 | neutron router-create ${TENANT}_router 29 | neutron router-gateway-set ${TENANT}_router external_network 30 | neutron router-interface-add ${TENANT}_router ${TENANT}_subnet 31 | 32 | # Create some security groups 33 | 34 | nova secgroup-create ssh "SSH from anywhere" 35 | nova secgroup-add-rule ssh tcp 22 22 0.0.0.0/0 36 | 37 | nova secgroup-create all "TCP/UDP/ICMP from anywhere" 38 | nova secgroup-add-rule all tcp 1 65535 0.0.0.0/0 39 | nova secgroup-add-rule all udp 1 65535 0.0.0.0/0 40 | nova secgroup-add-rule all icmp -1 -1 0.0.0.0/0 41 | -------------------------------------------------------------------------------- /docs/openstack/local-setup.md: -------------------------------------------------------------------------------- 1 | # Local OpenStack setup 2 | 3 | The main task is to create a Python virtualenv with all the needed 4 | OpenStack modules and commands: 5 | 6 | ```sh 7 | if [ -f ~/venv/openstack/bin/activate ]; then 8 | source ~/venv/openstack/bin/activate 9 | else 10 | mkdir -p ~/venv 11 | virtualenv ~/venv/openstack 12 | source ~/venv/openstack/bin/activate 13 | 14 | pip install python-openstackclient 15 | fi 16 | ``` 17 | 18 | And ansible: 19 | 20 | ``` 21 | pip install ansible shade 22 | pip install 'pywinrm[credssp]' 23 | ``` 24 | 25 | Log into the OpenStack Horizon web interface and download the 26 | configuration as `~/.openstackrc` (Compute»Access & Security»API 27 | Access»Download OpenStack RC File v2.0). Note "v3" configuration 28 | files don't work at present for some reason. 29 | 30 | ```sh 31 | source ~/.openstackrc 32 | ``` 33 | 34 | Set additional details required below (used in the other examples): 35 | 36 | ```sh 37 | # SSH key name registered with OpenStack from "openstack keypair list" 38 | ssh_key="your_ssh_key" 39 | # Security group name from "openstack security group list" 40 | security_group="required_security_group" 41 | # SSH private key file for the above SSH key 42 | ssh_private_key_file="/path/to/private/key" 43 | ``` 44 | -------------------------------------------------------------------------------- /docs/openstack/ubuntu-image.md: -------------------------------------------------------------------------------- 1 | # Ubuntu OpenStack image creation 2 | 3 | OpenStack images for Ubuntu are available from 4 | [cloudbase.it](https://cloud-images.ubuntu.com/), including 5 | images for: 6 | 7 | - [Ubuntu 16.04](https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img) 8 | - [Ubuntu 18.04](https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img) 9 | 10 | Follow the [local setup](local-setup.md) instructions before starting. 11 | 12 | ## Image creation 13 | 14 | Download image from the above URL, then run: 15 | 16 | ```sh 17 | openstack image create --file xenial-server-cloudimg-amd64-disk1.img --disk-format qcow2 "Ubuntu 16.04" 18 | openstack image create --file bionic-server-cloudimg-amd64.img --disk-format qcow2 "Ubuntu 18.04" 19 | ``` 20 | -------------------------------------------------------------------------------- /docs/openstack/windows-image.md: -------------------------------------------------------------------------------- 1 | # Windows OpenStack image creation 2 | 3 | OpenStack images for Windows are available from 4 | [cloudbase.it](https://cloudbase.it/windows-cloud-images/), including 5 | images for [Windows Server 6 | 2012R2](https://cloudbase.it/openstack-windows-server-2012-r2-evalution-images/). 7 | 8 | Follow the [local setup](local-setup.md) instructions before starting. 9 | 10 | ## Image creation 11 | 12 | Download image from the above URL, then run: 13 | 14 | ```sh 15 | openstack image create --file ~/Downloads/windows_server_2012_r2_standard_eval_kvm_20170321.qcow2 --disk-format qcow2 --container-format bare "Windows Server 2012R2" 16 | ``` 17 | 18 | ## Test 19 | 20 | List available networks: 21 | 22 | ```sh 23 | openstack network list 24 | ``` 25 | 26 | Set `net_id` using `id` for required network (must permit outgoing 27 | external connections for downloading packages). 28 | 29 | Create a VM using our new image: 30 | 31 | ```sh 32 | openstack server create --image="Windows Server 2012R2" --security-group "$security_group" --nic "net-id=$net_id" --key-name "$ssh_key" --flavor "m1.small" "test7" 33 | ``` 34 | 35 | Set `test_vm_id` using `id` of created VM. 36 | 37 | The Windows image requires the Admin password to be obtained: 38 | 39 | ```sh 40 | nova get-password "$test_vm_id" | base64 -d | openssl rsautl -decrypt -inkey "$ssh_private_key_file" 41 | ``` 42 | 43 | It should now be possible to log on to the OpenStack console for this 44 | VM using the username `Admin` and this password. You can also use 45 | PowerShell remoting (RMI) to administer the system with the same 46 | credentials. 47 | -------------------------------------------------------------------------------- /docs/vmware/mac-image.md: -------------------------------------------------------------------------------- 1 | # MacOS X VMware image creation 2 | 3 | ## Base image install 4 | 5 | Obtain installation image: 6 | 7 | - Download MacOS X 10.13 installer `Install macOS High Sierra` from 8 | the Apple app store, or use a previously downloaded image. 9 | 10 | ## Base machine creation 11 | 12 | In VMware Fusion: 13 | 14 | - File→New then drag and drop the installer from the Finder to the 15 | "Install from disc or image" part of the "Select the Installation 16 | Method" window. 17 | 18 | - Select `macOS 10.12` under "Choose Operating System" (or the correct 19 | version with a newer version of VMware) 20 | 21 | - Choose `Customize Settings` under "Finish" 22 | 23 | - Save As `ansible-macos-10.13-base` (wait while disc image is created) 24 | 25 | - 2 processor cores 26 | - 2 GiB RAM 27 | - Network is Bridged Networking (`Autodetect`) 28 | - Disc is 200GB (sparse, not pre-allocated) 29 | - Disable features: 30 | - No 3D graphics 31 | - No CD 32 | - No sound 33 | - No USB 34 | - No Bluetooth 35 | - No drag+drop 36 | - No copy+paste 37 | 38 | ## Base machine installation 39 | 40 | In VMware Fusion: 41 | 42 | - Start the virtual machine (the installer will automatically run) 43 | - Language: `English` 44 | - `Install macOS` (follow through prompts until installer runs and 45 | completes) 46 | - After automatic restart: 47 | - Location: United Kingdom 48 | - Keyboard: British 49 | - Don't transfer information 50 | - Don't sign in with Apple ID 51 | - Create user account: 52 | - Full name: CI Admin 53 | - Account name: ci-admin 54 | - Password: [see ci/all.gpg in credentials.git] 55 | - Hint: See ci/all.gpg in credentials.git 56 | - Final configuration: 57 | - Don't enable location services 58 | - Time zone GMT/Edinburgh 59 | - Don't share analytics with Apple 60 | - Don't share crash data 61 | - Check for and install any software updates 62 | - Shut down machine 63 | - Snapshot for future reference 64 | 65 | # Create machine for deployment 66 | 67 | In VMware Fusion: 68 | 69 | - Clone the base machine (use full clone for better performance but 70 | increased disc usage) 71 | - Adjust CPU count, memory and disc as needed 72 | - Boot system 73 | 74 | Log in as the ci-admin user, and perform the following actions: 75 | 76 | - Set the hostname 77 | - Set static IP address, DNS, search domains 78 | - Enable SSH for all users 79 | 80 | Perform all remaining setup steps with ansible. 81 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | ansible==2.6.18 2 | shade==1.14.1 3 | --------------------------------------------------------------------------------