├── .gitignore ├── ansible ├── tmp │ └── .empty ├── files │ ├── ssl │ │ ├── dev │ │ │ └── .empty │ │ └── prod │ │ │ └── .empty │ ├── rpmmacros.j2 │ └── ssh │ │ ├── hostkeys │ │ ├── github.com.pub │ │ └── docs.ceph.com.pub │ │ └── keys │ │ └── jenkins_build.pub ├── private-docker-registry │ ├── group_vars │ │ └── all │ ├── site.yml │ ├── vagrant_variables.yml │ ├── roles │ │ ├── docker-registry │ │ │ ├── templates │ │ │ │ └── self-csr.json.j2 │ │ │ └── tasks │ │ │ │ └── main.yml │ │ └── test-client │ │ │ └── tasks │ │ │ └── main.yml │ └── Vagrantfile ├── roles │ ├── ansible-jenkins │ │ ├── .gitignore │ │ ├── tasks │ │ │ ├── main.yml │ │ │ ├── jenkins.yml │ │ │ ├── dependencies.yml │ │ │ ├── plugins.yml │ │ │ ├── repo.yml │ │ │ └── nginx.yml │ │ ├── files │ │ │ ├── jenkins.repo │ │ │ ├── ssh │ │ │ │ ├── jenkins_build.pub │ │ │ │ └── keys │ │ │ │ │ └── jenkins_build.pub │ │ │ └── jenkins.conf │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── templates │ │ │ ├── hudson.tasks.Mailer.xml.j2 │ │ │ ├── jenkins.conf │ │ │ └── nginx.conf │ │ └── vars │ │ │ └── main.yml │ ├── grafana │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── vars │ │ │ └── main.yml │ │ ├── tasks │ │ │ ├── nginx.yml │ │ │ ├── postgresql.yml │ │ │ └── main.yml │ │ └── templates │ │ │ └── nginx_site.conf │ ├── graphite │ │ ├── defaults │ │ │ └── main.yml │ │ ├── templates │ │ │ ├── systemd │ │ │ │ ├── 80-graphite.preset.j2 │ │ │ │ └── graphite.service.j2 │ │ │ ├── storage-schemas.conf.j2 │ │ │ ├── whitelist.conf.j2 │ │ │ └── rewrite-rules.conf.j2 │ │ ├── handlers │ │ │ └── main.yml │ │ ├── vars │ │ │ └── main.yml │ │ └── tasks │ │ │ ├── systemd.yml │ │ │ ├── main.yml │ │ │ ├── carbon.yml │ │ │ └── postgresql.yml │ ├── ceph-release │ │ ├── handlers │ │ │ └── main.yml │ │ └── tasks │ │ │ └── release │ │ │ ├── candidate.yml │ │ │ ├── development.yml │ │ │ └── stable.yml │ ├── nginx │ │ ├── handlers │ │ │ └── main.yml │ │ ├── defaults │ │ │ └── main.yml │ │ ├── templates │ │ │ ├── nginx_tmp_site.conf │ │ │ ├── nginx_site.conf │ │ │ └── nginx.conf │ │ └── tasks │ │ │ ├── ssl.yml │ │ │ ├── main.yml │ │ │ └── letsencrypt.yml │ ├── redis │ │ └── tasks │ │ │ └── main.yml │ └── ceph-deploy-release │ │ └── tasks │ │ ├── clear_version.yml │ │ ├── main.yml │ │ └── release.yml ├── host_vars │ ├── prado.ceph.com.yml │ └── all.yml ├── sensu-hosts ├── ansible.cfg ├── templates │ ├── yum-repos │ │ └── epel.repo │ └── systemd │ │ └── jenkins.service.j2 ├── requirements │ └── sensu-requirements.yml ├── slaves │ └── hosts ├── examples │ ├── deploy_graphite.yml │ ├── shaman_load_balancer.yml │ ├── hosts-static │ ├── deploy_grafana.yml │ ├── master.yml │ ├── sensu.yml │ └── init.yml ├── .gitignore ├── vars │ ├── load-balance-vars.yml │ └── sensu-vars.yml └── release.yml ├── calamari └── build │ ├── post │ ├── setup │ ├── build_deb │ └── build_rpm ├── tag_release.sh ├── ceph-deploy-pull-requests ├── build │ ├── build │ └── setup ├── setup │ └── playbooks │ │ ├── setup.yml │ │ └── tasks │ │ └── ubuntu.yml └── config │ └── definitions │ └── ceph-deploy-pull-requests.yml ├── chacra-pull-requests ├── setup │ └── playbooks │ │ ├── ansible.cfg │ │ ├── setup.yml │ │ └── tasks │ │ └── postgresql.yml ├── build │ └── build └── config │ └── definitions │ └── chacra-pull-requests.yml ├── shaman-pull-requests ├── setup │ └── playbooks │ │ ├── ansible.cfg │ │ ├── setup.yml │ │ └── tasks │ │ └── postgresql.yml ├── build │ └── build └── config │ └── definitions │ └── shaman-pull-requests.yml ├── deb_dists ├── deb_hosts ├── kernel └── build │ ├── validate_deb │ ├── validate_rpm │ ├── prepare_config │ └── setup ├── ceph-build └── build │ ├── validate_deb │ ├── validate_rpm │ └── failure ├── ceph-dev-build └── build │ ├── validate_deb │ ├── validate_rpm │ └── failure ├── diamond-build └── build │ ├── validate_deb │ ├── validate_rpm │ └── setup ├── ceph-dev-new-build └── build │ ├── validate_deb │ ├── validate_rpm │ └── failure ├── rpm_hosts ├── calamari-clients-build ├── build │ ├── validate_rpm │ └── setup └── config │ └── definitions │ └── calamari-clients-build.yml ├── ceph-installer-pull-requests ├── build │ └── build └── config │ └── definitions │ └── ceph-installer-pull-requests.yml ├── rpm-autosign.exp ├── ceph-ansible-galaxy ├── build │ └── build └── config │ └── definitions │ └── ceph-ansible-galaxy.yml ├── ceph-installer-docs-pull-requests ├── build │ └── build └── config │ └── definitions │ └── ceph-installer-docs-pull-requests.yml ├── teuthology-docs ├── setup │ └── setup ├── build │ └── build └── config │ └── definitions │ └── teuthology-docs.yml ├── teuthology-pull-requests ├── setup │ └── setup ├── build │ └── build └── config │ └── definitions │ └── teuthology-pull-requests.yml ├── ceph-pr-commits ├── build │ ├── build │ └── test_commits.py └── config │ └── definitions │ └── ceph-pr-commits.yml ├── merfi-pull-requests ├── build │ └── build └── config │ └── definitions │ └── merfi-pull-requests.yml ├── ceph-dev-trigger ├── build │ └── notify └── config │ └── definitions │ └── ceph-dev-trigger.yml ├── ceph-dev-new-trigger ├── build │ └── notify └── config │ └── definitions │ └── ceph-dev-new-trigger.yml ├── ceph-installer-docs ├── build │ └── build └── config │ └── definitions │ └── ceph-installer-docs.yml ├── ceph-setup ├── build │ └── failure └── config │ └── definitions │ └── ceph-setup.yml ├── ceph-deploy-docs ├── build │ └── build └── config │ └── definitions │ └── ceph-deploy-docs.yml ├── ceph-dev-setup ├── build │ └── failure └── config │ └── definitions │ └── ceph-dev-setup.yml ├── ceph-dev-new-setup ├── build │ └── failure └── config │ └── definitions │ └── ceph-dev-new-setup.yml ├── ceph-qa-suite-pull-requests ├── build │ └── build └── config │ └── definitions │ └── ceph-qa-suite-pull-requests.yml ├── mita-deploy ├── build │ └── build └── config │ └── definitions │ └── mita-deploy.yml ├── ceph-installer-tests ├── build │ └── build └── config │ └── definitions │ └── ceph-installer-tests.yml ├── ceph-ansible-prs └── build │ ├── teardown │ └── build ├── ceph-docker-nightly ├── build │ ├── teardown │ └── build └── config │ └── definitions │ └── ceph-docker-nightly.yml ├── ceph-docker-prs ├── build │ ├── teardown │ └── build └── config │ └── definitions │ └── ceph-docker-prs.yml ├── ceph-installer-coverage ├── build │ └── build └── config │ └── definitions │ └── ceph-installer-coverage.yml ├── update_all_pbuilders.sh ├── ceph-docs ├── build │ └── build └── config │ └── definitions │ └── ceph-docs.yml ├── ceph-deploy-tag ├── build │ └── build └── config │ └── definitions │ └── ceph-tag.yml ├── ceph-tag ├── build │ └── build └── config │ └── definitions │ └── ceph-tag.yml ├── ceph-deploy-build ├── build │ └── setup └── config │ └── definitions │ └── ceph-deploy-build.yml ├── ceph-pr-submodules ├── build │ └── build └── config │ └── definitions │ └── ceph-pr-commits.yml ├── gen_debian_version.sh ├── ceph-ansible-pr-syntax-check ├── build │ └── build └── config │ └── definitions │ └── ceph-ansible-pr-syntax-check.yml ├── get_rpm_dist.sh ├── gen_reprepro_conf.sh ├── radosgw-agent-pull-requests ├── build │ └── build └── config │ └── definitions │ └── radosgw-agent-pull-requests.yml ├── scripts ├── sync-pull └── sync-push ├── sign_and_index_rpm_repo.sh ├── kernel-trigger └── config │ └── definitions │ └── kernel-trigger.yml ├── sign_debs.sh ├── sign_rpms.sh ├── push_to_rpm_repo.sh ├── ceph-ansible-rpm ├── config │ └── definitions │ │ └── ceph-ansible-rpm.yml └── build │ └── build ├── ceph-installer-rpm ├── config │ └── definitions │ │ └── ceph-installer-rpm.yml └── build │ └── build ├── push_to_deb_repo.sh ├── samba └── build │ ├── setup │ └── build_deb ├── nfs-ganesha └── build │ └── setup ├── ceph-release-rpm └── config │ └── definitions │ └── ceph-release-rpm.yml ├── ceph-build-pull-requests ├── config │ └── definitions │ │ └── ceph-build-pull-requests.yml └── build │ └── build ├── calamari-clients-setup ├── build │ └── build └── config │ └── definitions │ └── calamari-clients-setup.yml ├── jenkins-job-builder ├── config │ └── definitions │ │ └── jjb.yml └── build │ └── build ├── ceph-pull-requests └── config │ └── definitions │ └── ceph-pull-requests.yml ├── do_rpm_release.sh ├── diamond-setup ├── config │ └── definitions │ │ └── diamond-setup.yml └── build │ └── build ├── update_pbuilder.sh ├── release_tarball.sh ├── radosgw-agent └── config │ └── definitions │ └── radosgw-agent.yml └── ceph-deploy └── config └── definitions └── ceph-deploy.yml /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | -------------------------------------------------------------------------------- /ansible/tmp/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ansible/files/ssl/dev/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ansible/files/ssl/prod/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /calamari/build/post: -------------------------------------------------------------------------------- 1 | sudo chown -R jenkins-build ./dist 2 | 3 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/group_vars/all: -------------------------------------------------------------------------------- 1 | --- 2 | dummy: 3 | -------------------------------------------------------------------------------- /ansible/files/rpmmacros.j2: -------------------------------------------------------------------------------- 1 | %dist .el{{ ansible_lsb.major_release }} 2 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | .vagrant 3 | Vagrantfile -------------------------------------------------------------------------------- /tag_release.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | git tag -s $1 -u 460F3994 -m $1 4 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | cd $WORKSPACE/ceph-deploy && $VENV/tox -rv 2 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include: jenkins.yml tags=jenkins 3 | -------------------------------------------------------------------------------- /chacra-pull-requests/setup/playbooks/ansible.cfg: -------------------------------------------------------------------------------- 1 | [ssh_connection] 2 | pipelining=True 3 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/ansible.cfg: -------------------------------------------------------------------------------- 1 | [ssh_connection] 2 | pipelining=True 3 | -------------------------------------------------------------------------------- /ansible/host_vars/prado.ceph.com.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sensu_client_subscription_names: 3 | - common 4 | -------------------------------------------------------------------------------- /ansible/host_vars/all.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sensu_client_subscription_names: 3 | - common 4 | - rabbitmq 5 | -------------------------------------------------------------------------------- /ansible/roles/grafana/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | app_name: "grafana" 4 | fqdn: "grafana.local" 5 | -------------------------------------------------------------------------------- /ansible/roles/graphite/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | app_name: "graphite" 4 | fqdn: "graphite.local" 5 | -------------------------------------------------------------------------------- /deb_dists: -------------------------------------------------------------------------------- 1 | sid 2 | wheezy 3 | squeeze 4 | jessie 5 | raring 6 | precise 7 | saucy 8 | trusty 9 | xenial 10 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: "update apt cache" 3 | action: apt update-cache=yes 4 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/systemd/80-graphite.preset.j2: -------------------------------------------------------------------------------- 1 | # graphite web service 2 | 3 | enable graphite.service 4 | -------------------------------------------------------------------------------- /deb_hosts: -------------------------------------------------------------------------------- 1 | ubuntu@gitbuilder-precise-amd64.front.sepia.ceph.com 2 | ubuntu@gitbuilder-precise-i386.front.sepia.ceph.com 3 | -------------------------------------------------------------------------------- /ansible/sensu-hosts: -------------------------------------------------------------------------------- 1 | [sensu-server] 2 | 158.69.67.168 3 | 4 | [sensu-clients] 5 | jenkins.ceph.com 6 | chacra.ceph.com 7 | prado.ceph.com 8 | -------------------------------------------------------------------------------- /ansible/ansible.cfg: -------------------------------------------------------------------------------- 1 | [defaults] 2 | callback_plugins = callbacks 3 | retry_files_enabled = False 4 | 5 | [ssh_connection] 6 | pipelining=True 7 | 8 | -------------------------------------------------------------------------------- /ansible/roles/nginx/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: restart nginx 4 | sudo: yes 5 | action: service name=nginx state=restarted enabled=yes 6 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/storage-schemas.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # 3 | [default] 4 | pattern = .* 5 | retentions = 5s:30d,1m:120d,10m:1y 6 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/jenkins.repo: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | [jenkins] 4 | name=Jenkins 5 | baseurl=http://pkg.jenkins-ci.org/redhat 6 | gpgcheck=1 7 | -------------------------------------------------------------------------------- /kernel/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /diamond-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /chacra-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | sudo: True 6 | 7 | tasks: 8 | - include: tasks/postgresql.yml 9 | -------------------------------------------------------------------------------- /rpm_hosts: -------------------------------------------------------------------------------- 1 | ubuntu@gitbuilder-centos6-amd64.front.sepia.ceph.com 2 | ubuntu@gitbuilder-sles-11sp2-amd64.front.sepia.ceph.com 3 | ubuntu@gitbuilder-opensuse12-amd64.front.sepia.ceph.com 4 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | sudo: True 6 | 7 | tasks: 8 | - include: tasks/postgresql.yml 9 | -------------------------------------------------------------------------------- /kernel/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /diamond-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /calamari-clients-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-installer-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | $VENV/tox -rv 8 | -------------------------------------------------------------------------------- /ansible/roles/nginx/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ssl_support_email: "adeza@redhat.com" 3 | ssl_webroot_base_path: "/var/www" 4 | 5 | ssl_cert: "files/ssl/dev/ssl/ssl.crt" 6 | ssl_key: "files/ssl/dev/ssl/ssl.key" 7 | -------------------------------------------------------------------------------- /ansible/templates/yum-repos/epel.repo: -------------------------------------------------------------------------------- 1 | [epel] 2 | baseurl = http://download.fedoraproject.org/pub/epel/$releasever/$basearch/ 3 | gpgkey = https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7 4 | name = EPEL YUM repo 5 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | placeholder: 'placeholder' 3 | port: 8080 4 | # plugins: 5 | # - 'ldap' 6 | # - 'github' 7 | # - 'translation' 8 | # - 'preSCMbuildstep' 9 | # - 'gravatar' 10 | -------------------------------------------------------------------------------- /rpm-autosign.exp: -------------------------------------------------------------------------------- 1 | #!/usr/bin/expect -f 2 | 3 | # First parameter should be the key: 4 | # --define "_gpg_name 03C3951A" 5 | 6 | spawn rpm --addsign {*}$argv 7 | expect -exact "Enter pass phrase: " 8 | send -- "\r" 9 | expect eof 10 | -------------------------------------------------------------------------------- /ansible/requirements/sensu-requirements.yml: -------------------------------------------------------------------------------- 1 | - src: https://github.com/andrewschoen/ansible-playbook-sensu 2 | scm: git 3 | name: Mayeu.sensu 4 | version: remotes/origin/plugin-gem-install 5 | 6 | - src: Mayeu.RabbitMQ 7 | version: 1.4.0 8 | -------------------------------------------------------------------------------- /ansible/slaves/hosts: -------------------------------------------------------------------------------- 1 | [rpm] 2 | jenkins-centos6-5.front.sepia.ceph.com 3 | jenkins-centos7.front.sepia.ceph.com 4 | jenkins-centos6-5b.front.sepia.ceph.com 5 | jenkins-centos7b.front.sepia.ceph.com 6 | 7 | [deb] 8 | mira062.front.sepia.ceph.com 9 | -------------------------------------------------------------------------------- /ceph-ansible-galaxy/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # propagates the change in the necessary Ansible Galaxy repos. 4 | # i.e. https://github.com/ceph/ansible-ceph-common 5 | bash "$WORKSPACE"/ceph-ansible/contrib/push-roles-to-ansible-galaxy.sh 6 | -------------------------------------------------------------------------------- /ceph-installer-docs-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # create the docs build with tox 8 | $VENV/tox -rv -e docs 9 | -------------------------------------------------------------------------------- /ansible/roles/nginx/templates/nginx_tmp_site.conf: -------------------------------------------------------------------------------- 1 | server { 2 | server_name {{ item.fqdn }}; 3 | location '/.well-known/acme-challenge' { 4 | default_type "text/plain"; 5 | root {{ ssl_webroot_base_path }}/{{ item.fqdn }}; 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /ansible/examples/deploy_graphite.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - graphite 7 | vars: 8 | fqdn: "graphite.ceph.com" 9 | app_name: "graphite" 10 | development_server: true 11 | graphite_api_key: "secret" 12 | -------------------------------------------------------------------------------- /ansible/.gitignore: -------------------------------------------------------------------------------- 1 | # these should never make it to version control 2 | files/ssl/dev 3 | files/ssl/prod 4 | tmp 5 | 6 | # Top level YAML files are ignored, so that users can simply copy 7 | # from examples/* to the top level dir and not worry about committing them 8 | /*.yml 9 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/candidate.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "New upstream release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/development.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "Development release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/stable.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "New upstream release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | 8 | -------------------------------------------------------------------------------- /teuthology-docs/setup/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | APT_DEPS="libmysqlclient-dev libevent-dev libffi-dev libssl-dev pkg-config libvirt-dev" 6 | # We don't have tty-less sudo on docs.ceph.com; these deps must be installed 7 | # manually. 8 | #sudo apt install $APT_DEPS 9 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/site.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Defines deployment design and assigns role to server groups 3 | 4 | - hosts: registry 5 | become: True 6 | roles: 7 | - docker-registry 8 | 9 | - hosts: testclient 10 | become: True 11 | roles: 12 | - test-client 13 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | sudo: True 6 | 7 | tasks: 8 | - include: tasks/ubuntu.yml 9 | when: ansible_distribution == "Ubuntu" 10 | 11 | # TODO: maybe add RPM or Debian handling? 12 | -------------------------------------------------------------------------------- /teuthology-pull-requests/setup/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # run the teuthology bootstrap to setup this jenkins slave for 6 | # teuthology pull requests testing. Passing 'install' here 7 | # installs any missing requirements found needed to bootstrap teuthology 8 | ./bootstrap install 9 | -------------------------------------------------------------------------------- /ceph-pr-commits/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "pytest" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | 8 | cd "$WORKSPACE" 9 | timeout 600 $VENV/py.test -vs --junitxml="$WORKSPACE/report.xml" "$WORKSPACE/ceph-build/ceph-pr-commits/build/test_commits.py" 10 | -------------------------------------------------------------------------------- /merfi-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | # run tox by recreating the environment and in verbose mode 10 | # by default this will run all environments defined 11 | $VENV/tox -rv 12 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/vagrant_variables.yml: -------------------------------------------------------------------------------- 1 | --- 2 | provision_test_client_vm: true 3 | memory: 1024 4 | vagrant_box: centos/atomic-host 5 | # The sync directory changes based on vagrant box 6 | # Set to /home/vagrant/sync for Centos/7, /home/{ user }/vagrant for openstack and defaults to /vagrant 7 | vagrant_sync_dir: /home/vagrant/sync 8 | -------------------------------------------------------------------------------- /teuthology-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | # run tox by recreating the environment and in verbose mode 10 | # by default this will run all environments defined 11 | $VENV/tox -rv 12 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/handlers/main.yml: -------------------------------------------------------------------------------- 1 | # Safe-restart Jenkins 2 | - name: restart jenkins 3 | sudo: yes 4 | command: java -jar {{ jenkins.cli_dest }} -s http://localhost:{{ port }} safe-restart 5 | 6 | - name: restart nginx 7 | sudo: yes 8 | action: service name=nginx state=restarted 9 | #command: service nginx restart 10 | -------------------------------------------------------------------------------- /ceph-dev-trigger/build/notify: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the triggered build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # build information 6 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 7 | SHA1=${GIT_COMMIT} 8 | 9 | create_build_status "queued" "ceph" 10 | -------------------------------------------------------------------------------- /ceph-dev-new-trigger/build/notify: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the triggered build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # build information 6 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 7 | SHA1=${GIT_COMMIT} 8 | 9 | create_build_status "queued" "ceph" 10 | 11 | -------------------------------------------------------------------------------- /ceph-installer-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | 8 | # create the docs build with tox 9 | $VENV/tox -rv -e docs 10 | 11 | # publish docs to http://docs.ceph.com/docs/ceph-installer 12 | rsync -auv --delete .tox/docs/tmp/html/* /var/ceph-installer/docs/ 13 | -------------------------------------------------------------------------------- /ceph-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /ceph-deploy-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | # create the docs build with tox 10 | $VENV/tox -rv -e docs 11 | 12 | # publish docs to http://docs.ceph.com/docs/ceph-deploy 13 | rsync -auv --delete .tox/docs/tmp/html/* /var/ceph-deploy/docs/ 14 | -------------------------------------------------------------------------------- /ceph-dev-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /teuthology-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | # create the docs build with tox 10 | $VENV/tox -rv -e docs 11 | 12 | # publish docs to http://docs.ceph.com/docs/teuthology 13 | rsync -auv --delete .tox/docs/tmp/html/* /var/teuthology/docs/ 14 | -------------------------------------------------------------------------------- /ansible/roles/redis/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: install Redis server 3 | apt: 4 | name: redis-server 5 | state: latest 6 | update_cache: yes 7 | 8 | - name: check if Redis is running 9 | service: 10 | name: redis-server 11 | state: started 12 | 13 | - name: enable redis-server to survive reboot 14 | service: 15 | name: redis-server 16 | enabled: yes 17 | -------------------------------------------------------------------------------- /ceph-dev-new-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /ceph-qa-suite-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | 10 | # run tox by recreating the environment and in verbose mode 11 | # by default this will run all environments defined, although currently 12 | # it is just flake8 13 | $VENV/tox -rv 14 | -------------------------------------------------------------------------------- /mita-deploy/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | cd "$WORKSPACE/deploy/playbooks/" 10 | $VENV/ansible-playbook -i "localhost," -c local local_deploy.yml --extra-vars="branch=$BRANCH jenkins_prado_token=$JENKINS_PRADO_TOKEN prado_token=$PRADO_TOKEN" 11 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/hudson.tasks.Mailer.xml.j2: -------------------------------------------------------------------------------- 1 | 2 | 3 | {{ email.default_email_suffix }} 4 | {{ email.smtp_host }} 5 | {{ email.smtp_ssl }} 6 | UTF-8 7 | 8 | -------------------------------------------------------------------------------- /ceph-installer-tests/build/build: -------------------------------------------------------------------------------- 1 | # the following two methods exist in scripts/build_utils.sh 2 | pkgs=( "tox" ) 3 | install_python_packages "pkgs[@]" 4 | 5 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 6 | 7 | cd $WORKSPACE/tests/functional 8 | 9 | INSTALLER_DEV_BRANCH=$INSTALLER_BRANCH CEPH_ANSIBLE_DEV_BRANCH=$CEPH_ANSIBLE_BRANCH $VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR -- --provider=libvirt 10 | -------------------------------------------------------------------------------- /ansible/roles/grafana/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: reload systemd 4 | sudo: yes 5 | command: systemctl daemon-reload 6 | 7 | - name: restart app 8 | sudo: true 9 | service: 10 | name: grafana-server 11 | state: restarted 12 | enabled: yes 13 | 14 | - name: restart nginx 15 | sudo: true 16 | service: 17 | name: nginx 18 | state: restarted 19 | enabled: yes 20 | -------------------------------------------------------------------------------- /ceph-ansible-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | cd $WORKSPACE/tests 6 | 7 | scenarios=$(find . | grep Vagrantfile | xargs dirname) 8 | 9 | for scenario in $scenarios; do 10 | cd $scenario 11 | vagrant destroy -f 12 | cd - 13 | done 14 | -------------------------------------------------------------------------------- /ansible/roles/grafana/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | system_packages: 4 | - grafana 5 | - git 6 | - g++ 7 | - gcc 8 | - libpq-dev 9 | - postgresql 10 | - postgresql-common 11 | - postgresql-contrib 12 | - python-psycopg2 13 | - nginx 14 | - vim 15 | # needed for the ansible apt_repository module 16 | - python-apt 17 | - python 18 | 19 | ssl_requirements: 20 | - openssl 21 | - libssl-dev 22 | -------------------------------------------------------------------------------- /ansible/roles/graphite/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: reload systemd 4 | sudo: yes 5 | command: systemctl daemon-reload 6 | 7 | - name: restart app 8 | sudo: true 9 | service: 10 | name: graphite 11 | state: restarted 12 | enabled: yes 13 | 14 | - name: restart carbon 15 | service: 16 | name: carbon-cache 17 | state: restarted 18 | enabled: yes 19 | sudo: yes 20 | 21 | -------------------------------------------------------------------------------- /ansible/examples/shaman_load_balancer.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - nginx 7 | vars_files: 8 | - vars/load-balance-vars.yml 9 | vars: 10 | app_name: "shaman" 11 | fqdn: "shaman.ceph.com" 12 | development_server: true 13 | # only needed when enabling Github Auth 14 | # github_client_id: "111aaa222" 15 | # github_client_secret: "qwerty1234" 16 | -------------------------------------------------------------------------------- /ansible/files/ssh/hostkeys/github.com.pub: -------------------------------------------------------------------------------- 1 | github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ== 2 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/nginx.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create nginx site config 3 | action: template src=../templates/nginx_site.conf dest=/etc/nginx/sites-available/{{ app_name }}.conf 4 | sudo: true 5 | notify: 6 | - restart nginx 7 | 8 | - name: link nginx config 9 | action: file src=/etc/nginx/sites-available/{{ app_name }}.conf dest=/etc/nginx/sites-enabled/{{ app_name }}.conf state=link 10 | sudo: true 11 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/whitelist.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # This file takes a single regular expression per line 3 | # If USE_WHITELIST is set to True in carbon.conf, only metrics received which 4 | # match one of these expressions will be persisted. If this file is empty or 5 | # missing, all metrics will pass through. 6 | # This file is reloaded automatically when changes are made 7 | ^{{ graphite_api_key }}.* 8 | -------------------------------------------------------------------------------- /ansible/files/ssh/keys/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD4l25l32QJxD9aWZxXcUaVPAClY53yuLwRvnn4V5ZzE3Wd5ddrbbsBhON6iPdb5Vv4d9Q3b1SfKhWjAIZgSeHwD1vcTPt/Kd2nIpwv2qQYjKAvnByS21NNdd5kf1Iz+NFieJlWi+x3vBaDR53OY9OpVnwZj7pZ4jQXHJ4bhVOEG9auxncpy4/HJi70IpnqRFg9GidqcaGsiFUu4+ji1euU2WKnX2hVmIxK/47vKTGSVpDEAPM8OqBueKXU78fh2xeFR76hVWkKpLwhhdr5OAY1dDzuADjFNUT/x76eceXZ8B1aPBsQuhQ9W2oPtQZRHqu8ElXFuhrwQ30pm5IAzimb jenkins-build@jenkins 2 | -------------------------------------------------------------------------------- /ceph-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ceph-dev-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ceph-docker-nightly/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | cd $WORKSPACE/ceph-ansible/tests/functional 6 | 7 | scenarios=$(find . | grep Vagrantfile | xargs dirname) 8 | 9 | for scenario in $scenarios; do 10 | cd $scenario 11 | vagrant destroy -f 12 | cd - 13 | done 14 | -------------------------------------------------------------------------------- /ceph-docker-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | cd $WORKSPACE/ceph-ansible/tests/functional 6 | 7 | scenarios=$(find . | grep Vagrantfile | xargs dirname) 8 | 9 | for scenario in $scenarios; do 10 | cd $scenario 11 | vagrant destroy -f 12 | cd - 13 | done 14 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/docker-registry/templates/self-csr.json.j2: -------------------------------------------------------------------------------- 1 | { 2 | "CN": "docker-registry", 3 | "hosts": [ 4 | "{{ ansible_default_ipv4.address }}", 5 | "127.0.0.1" 6 | ], 7 | "key": { 8 | "algo": "rsa", 9 | "size": 2048 10 | }, 11 | "names": [ 12 | { 13 | "C": "XX", 14 | "L": "Default City", 15 | "O": "Default Company Ltd", 16 | "ST": "." 17 | } 18 | ] 19 | } 20 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/ssh/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI0tHxJQ7n+uMiLpsoR6CKAVd0xatgVQuqp/gmnGpZU0kE54a29vPNnEt7/aLitbfyhc57rrbHOT09H3ov74GZKkoVBSbMJUSsK3drbN+58wcuk+HK0htRewmwCfcfi9AkrVbyw6pbPXW/pbjxnxLep52fKmpJJnImZ5eHRV5le9OSAcLA1LHYR4y9R3IOrTp7jgpE205UxZi5OopAx7gkyTsmfydvmq4MjaSwbVOJ7aW/Fdt5FVxNJP3Zl/OrvDoo/1WovoRIDbVQH8JFpLikMSnCqtBVIHDeW6imAKl6dpn9Gf4FxD94+OcurhXo2p0pvSzC4Strg4d2Sxqh4wph jenkins-build 2 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/ssh/keys/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI0tHxJQ7n+uMiLpsoR6CKAVd0xatgVQuqp/gmnGpZU0kE54a29vPNnEt7/aLitbfyhc57rrbHOT09H3ov74GZKkoVBSbMJUSsK3drbN+58wcuk+HK0htRewmwCfcfi9AkrVbyw6pbPXW/pbjxnxLep52fKmpJJnImZ5eHRV5le9OSAcLA1LHYR4y9R3IOrTp7jgpE205UxZi5OopAx7gkyTsmfydvmq4MjaSwbVOJ7aW/Fdt5FVxNJP3Zl/OrvDoo/1WovoRIDbVQH8JFpLikMSnCqtBVIHDeW6imAKl6dpn9Gf4FxD94+OcurhXo2p0pvSzC4Strg4d2Sxqh4wph jenkins-build 2 | -------------------------------------------------------------------------------- /ceph-ansible-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # XXX this might not be needed 8 | source $VENV/activate 9 | 10 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 11 | 12 | # the $SCENARIO var is injected by the job template. It maps 13 | # to an actual, defined, tox environment 14 | $VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR -- --provider=libvirt 15 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/systemd/graphite.service.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | [Unit] 3 | Description=graphite gunicorn service 4 | After=network.target 5 | 6 | [Service] 7 | Type=simple 8 | ExecStart=/usr/bin/gunicorn -b 127.0.0.1:8000 -w 10 -t 30 graphite_web:application 9 | User=_graphite 10 | WorkingDirectory=/usr/lib/python2.7/dist-packages/graphite 11 | StandardOutput=journal 12 | StandardError=journal 13 | 14 | [Install] 15 | WantedBy=multi-user.target 16 | -------------------------------------------------------------------------------- /ceph-installer-coverage/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "pytest" "pytest-coverage" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | 8 | # run py.test, output both junit from unit tests and coverage report (should 9 | # spit out a coverage.xml file) 10 | cd $WORKSPACE 11 | $VENV/python setup.py develop 12 | $VENV/py.test -v --junitxml junit.xml --cov-report xml --cov ceph_installer ceph_installer/tests 13 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | 8 | # run ansible to get this current host to meet our requirements, specifying 9 | # a local connection and 'localhost' as the host where to execute 10 | cd "$WORKSPACE/ceph-build/ceph-deploy-pull-requests/setup/playbooks" 11 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 12 | -------------------------------------------------------------------------------- /ansible/vars/load-balance-vars.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | # These values allow the nginx role to fully configure load balancing and 4 | # regular hosts. 5 | nginx_hosts: 6 | - fqdn: "grafana.ceph.com" 7 | app_name: "grafana" 8 | proxy_pass: "http://127.0.0.1:3000" 9 | - fqdn: "shaman.ceph.com" 10 | app_name: "shaman" 11 | upstreams: 12 | name: "shaman" 13 | strategy: "least_conn" 14 | servers: 15 | - "1.shaman.ceph.com" 16 | - "2.shaman.ceph.com" 17 | -------------------------------------------------------------------------------- /ansible/roles/graphite/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | system_packages: 4 | - grafana 5 | - graphite-web 6 | - graphite-api 7 | - graphite-carbon 8 | - git 9 | - g++ 10 | - gcc 11 | - libpq-dev 12 | - postgresql 13 | - postgresql-common 14 | - postgresql-contrib 15 | - python-psycopg2 16 | - nginx 17 | - vim 18 | # needed for the ansible apt_repository module 19 | - python-apt 20 | - python 21 | - gunicorn 22 | 23 | ssl_requirements: 24 | - openssl 25 | - libssl-dev 26 | -------------------------------------------------------------------------------- /chacra-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # run ansible to get this current host to meet our requirements, specifying 8 | # a local connection and 'localhost' as the host where to execute 9 | cd "$WORKSPACE/ceph-build/chacra-pull-requests/setup/playbooks" 10 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 11 | 12 | cd "$WORKSPACE/chacra" 13 | $VENV/tox -rv 14 | -------------------------------------------------------------------------------- /shaman-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # run ansible to get this current host to meet our requirements, specifying 8 | # a local connection and 'localhost' as the host where to execute 9 | cd "$WORKSPACE/ceph-build/shaman-pull-requests/setup/playbooks" 10 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 11 | 12 | cd "$WORKSPACE/shaman" 13 | $VENV/tox -rv 14 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/jenkins.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include: repo.yml 3 | 4 | - include: dependencies.yml 5 | 6 | - include: nginx.yml 7 | 8 | - name: Install Jenkins with Yum 9 | yum: name=jenkins state=present 10 | register: jenkins_install 11 | when: ansible_pkg_mgr == "yum" 12 | 13 | - name: Install Jenkins with Apt 14 | apt: name=jenkins state=present 15 | register: jenkins_install 16 | when: ansible_pkg_mgr == "apt" 17 | 18 | - include: config.yml 19 | 20 | - include: plugins.yml 21 | 22 | - include: config.yml 23 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: epel-release install 4 | yum: name=epel-release 5 | state=present 6 | when: ansible_pkg_mgr == "yum" 7 | 8 | - name: Install RPM dependencies 9 | yum: name={{ item }} state=installed 10 | with_items: jenkins.{{ ansible_pkg_mgr }}.dependencies 11 | when: ansible_pkg_mgr == "yum" 12 | 13 | - name: Install DEB dependencies 14 | apt: name={{ item }} state=installed 15 | with_items: jenkins.{{ ansible_pkg_mgr }}.dependencies 16 | when: ansible_pkg_mgr == "apt" 17 | -------------------------------------------------------------------------------- /update_all_pbuilders.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | 3 | set -e 4 | 5 | bindir=`dirname $0` 6 | 7 | pids="" 8 | for rem in `cat $bindir/deb_hosts` 9 | do 10 | echo rem $rem 11 | ssh $rem sudo rm -rf /tmp/ceph-build.\* \; sudo mkdir -p /srv/debian-base 12 | ssh $rem git clone git://github.com/ceph/ceph-build /tmp/ceph-build.$$ 13 | ssh $rem sudo /tmp/ceph-build.$$/update_pbuilder.sh /srv/debian-base > update.$rem 2>&1 & 14 | pids="$pids $!" 15 | done 16 | 17 | echo pids $pids 18 | for p in $pids 19 | do 20 | wait $p 21 | done 22 | 23 | echo done. -------------------------------------------------------------------------------- /ansible/examples/hosts-static: -------------------------------------------------------------------------------- 1 | [smithi] 2 | smithi118.front.sepia.ceph.com nodename=smithi118 3 | smithi119.front.sepia.ceph.com nodename=smithi119 4 | smithi120.front.sepia.ceph.com nodename=smithi120 5 | smithi121.front.sepia.ceph.com nodename=smithi121 6 | smithi122.front.sepia.ceph.com nodename=smithi122 7 | smithi123.front.sepia.ceph.com nodename=smithi123 8 | smithi124.front.sepia.ceph.com nodename=smithi124 9 | smithi125.front.sepia.ceph.com nodename=smithi125 10 | smithi127.front.sepia.ceph.com nodename=smithi127 11 | smithi128.front.sepia.ceph.com nodename=smithi128 12 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/setup/playbooks/tasks/ubuntu.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: add deadsnakes ppa 3 | action: apt_repository repo=ppa:fkrull/deadsnakes state=present 4 | 5 | - name: "update apt repo" 6 | action: apt update_cache=yes 7 | 8 | - name: install python requirements 9 | action: apt pkg={{ item }} state=installed 10 | with_items: 11 | - python-software-properties 12 | - python-dev 13 | - python2.6-dev 14 | - python-setuptools 15 | - python2.6 16 | - python2.7 17 | 18 | - name: install pip 19 | action: easy_install name=pip 20 | -------------------------------------------------------------------------------- /ansible/templates/systemd/jenkins.service.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | [Unit] 3 | Description=Jenkins Slave 4 | Wants=network.target 5 | After=network.target 6 | 7 | [Install] 8 | WantedBy=multi-user.target 9 | 10 | [Service] 11 | Type=simple 12 | User={{ jenkins_user }} 13 | ExecStart=/usr/bin/java \ 14 | -jar /home/{{ jenkins_user }}/slave.jar \ 15 | -jnlpUrl {{ api_uri }}/computer/{{ ansible_default_ipv4.address }}+{{ nodename }}/slave-agent.jnlp \ 16 | -jnlpCredentials {{ api_user }}:{{ token }} 17 | StandardOutput=journal 18 | StandardError=journal 19 | Restart=always 20 | RestartSec=30 21 | StartLimitInterval=0 22 | -------------------------------------------------------------------------------- /ceph-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | ./admin/build-doc 6 | 7 | REV="$(git rev-parse HEAD)" 8 | OUTDIR="docs.raw/sha1/$REV" 9 | mkdir -p $OUTDIR 10 | 11 | cp -a build-doc/output/html/* $OUTDIR 12 | 13 | # Log this $OUTDIR's sha1 14 | printf '%s\n' "$REV" >"$OUTDIR/sha1" 15 | 16 | # Symlink the branch name 17 | BRANCH=${GIT_BRANCH#*/} 18 | mkdir -p docs.raw/ref/ 19 | ln -s ../sha1/$REV "docs.raw/ref/$BRANCH" 20 | 21 | # Publish this sha1's contents first: 22 | rsync -a -v docs.raw/sha1/$REV /var/docs.raw/sha1/ 23 | # Now point the ref symlink at the newly-uploaded sha1. 24 | rsync -a -v docs.raw/ref/$BRANCH /var/docs.raw/ref/ 25 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/systemd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure /var/log/graphite dir exists 4 | sudo: true 5 | file: 6 | path: /var/log/graphite 7 | state: directory 8 | owner: _graphite 9 | group: _graphite 10 | recurse: yes 11 | 12 | - name: install the systemd unit file for graphite 13 | template: 14 | src: systemd/graphite.service.j2 15 | dest: /etc/systemd/system/graphite.service 16 | sudo: true 17 | notify: 18 | - reload systemd 19 | 20 | - name: ensure graphite is enabled and running 21 | sudo: true 22 | service: 23 | name: graphite 24 | state: running 25 | enabled: yes 26 | -------------------------------------------------------------------------------- /ceph-deploy-tag/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | if [ "$TAG" = false ] ; then 6 | echo "Assuming tagging process has succeeded before because TAG was set to false" 7 | exit 0 8 | fi 9 | 10 | # the following two methods exist in scripts/build_utils.sh 11 | pkgs=( "ansible" ) 12 | install_python_packages "pkgs[@]" 13 | 14 | # run ansible to do all the tagging and release specifying 15 | # a local connection and 'localhost' as the host where to execute 16 | cd "$WORKSPACE/ceph-build/ansible/" 17 | $VENV/ansible-playbook -i "localhost," -c local release.yml --extra-vars="version=$VERSION branch=$BRANCH release=stable clean=true project=ceph-deploy" 18 | -------------------------------------------------------------------------------- /ceph-tag/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | if [ "$TAG" = false ] ; then 6 | echo "Assuming tagging process has succeeded before because TAG was set to false" 7 | exit 0 8 | fi 9 | 10 | # the following two methods exist in scripts/build_utils.sh 11 | pkgs=( "ansible" ) 12 | install_python_packages_no_binary "pkgs[@]" 13 | 14 | # run ansible to do all the tagging and release specifying 15 | # a local connection and 'localhost' as the host where to execute 16 | cd "$WORKSPACE/ceph-build/ansible/" 17 | $VENV/ansible-playbook -i "localhost," -c local release.yml --extra-vars="version=$VERSION branch=$BRANCH force_version=$FORCE_VERSION release=stable clean=true project=ceph" 18 | -------------------------------------------------------------------------------- /ceph-docker-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # XXX this might not be needed 8 | source $VENV/activate 9 | 10 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 11 | 12 | sudo apt-get install -y docker.io 13 | 14 | sudo gpasswd -a ${USER} docker 15 | sudo systemctl restart docker 16 | newgrp docker 17 | 18 | # adding groups on the fly doesn't guarantee their availability 19 | # so we must use `sg` to execute the tests as part of the docker group to avoid 20 | # 'Permission Denied` when tryin to talk over the socket 21 | sg docker -c "$VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR" 22 | -------------------------------------------------------------------------------- /ceph-docker-nightly/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | # XXX this might not be needed 8 | source $VENV/activate 9 | 10 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 11 | 12 | sudo apt-get install -y docker.io 13 | 14 | sudo gpasswd -a ${USER} docker 15 | sudo systemctl restart docker 16 | newgrp docker 17 | 18 | # adding groups on the fly doesn't guarantee their availability 19 | # so we must use `sg` to execute the tests as part of the docker group to avoid 20 | # 'Permission Denied` when tryin to talk over the socket 21 | sg docker -c "$VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR" 22 | -------------------------------------------------------------------------------- /ansible/roles/ceph-deploy-release/tasks/clear_version.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: undo last commit from failed release 4 | command: git reset --soft HEAD~1 chdir=ceph-deploy 5 | when: (clean and last_commit.stdout == tag_name) 6 | 7 | - name: git checkout {{ branch }} branch 8 | command: git checkout {{ branch }} chdir=ceph-deploy 9 | 10 | - name: remove local tag 11 | command: git tag -d v{{ version }} chdir=ceph-deploy 12 | ignore_errors: yes 13 | 14 | - name: remove remote tag 15 | command: git push jenkins :refs/tags/v{{ version }} chdir=ceph-deploy 16 | ignore_errors: yes 17 | 18 | - name: force push changes to jenkins git repo 19 | command: git push -f jenkins {{ branch }} chdir=ceph-deploy 20 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/jenkins.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | server { 4 | 5 | listen 80; 6 | # FIXME 7 | #server_name jenkins.domain.tld; 8 | 9 | location / { 10 | 11 | proxy_set_header Host $host; 12 | proxy_set_header X-Real-IP $remote_addr; 13 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 14 | proxy_set_header X-Forwarded-Proto $scheme; 15 | 16 | # Fix the "It appears that your reverse proxy set up is broken" error. 17 | proxy_pass http://127.0.0.1:8080; 18 | proxy_read_timeout 90; 19 | 20 | # FIXME 21 | #proxy_redirect http://127.0.0.1:8080 https://jenkins.domain.tld; 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/rewrite-rules.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # This file defines regular expression patterns that can be used to 3 | # rewrite metric names in a search & replace fashion. It consists of two 4 | # sections, [pre] and [post]. The rules in the pre section are applied to 5 | # metric names as soon as they are received. The post rules are applied 6 | # after aggregation has taken place. 7 | # 8 | # The general form of each rule is as follows: 9 | # 10 | # regex-pattern = replacement-text 11 | # 12 | # For example: 13 | # 14 | # [post] 15 | # _sum$ = 16 | # _avg$ = 17 | # 18 | # These rules would strip off a suffix of _sum or _avg from any metric names 19 | # after aggregation. 20 | 21 | [post] 22 | ^{{ graphite_api_key }} = 23 | -------------------------------------------------------------------------------- /ceph-deploy-build/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This is the script that runs inside Jenkins. 4 | # http://jenkins.ceph.com/job/ceph-deploy/ 5 | 6 | set -x 7 | set -e 8 | 9 | # ensure dependencies are installed for RPM hosts, because they are required and 10 | # we are not building with a contained environment like we do on DEB builds 11 | 12 | # TODO: use Mock to build ceph-deploy rpm's and avoid this 13 | 14 | rpm_deps="python-devel python-virtualenv python-mock python-tox pytest" 15 | 16 | if test -f /etc/redhat-release ; then 17 | sudo yum install -y $rpm_deps 18 | fi 19 | 20 | pkgs=( "chacractl>=0.0.4" ) 21 | install_python_packages "pkgs[@]" 22 | 23 | # create the .chacractl config file using global variables 24 | make_chacractl_config 25 | -------------------------------------------------------------------------------- /ceph-pr-submodules/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | cd "$WORKSPACE" 6 | 7 | # The command below has access to the variables $name, $path, $sha1 and $toplevel: 8 | # $name is the name of the relevant submodule section in .gitmodules, $path is 9 | # the name of the submodule directory relative to the superproject, $sha1 is 10 | # the commit as recorded in the superproject, and $toplevel is the absolute 11 | # path to the top-level of the superproject. 12 | modified_submodules=$(git submodule foreach 'cd $toplevel; git diff --ignore-submodules=dirty origin/master -- $path') 13 | 14 | if [[ $modified_submodules ]]; then 15 | echo "Project has modified submodules!" 16 | echo $modified_submodules 17 | exit 1 18 | fi 19 | 20 | exit 0 21 | -------------------------------------------------------------------------------- /ansible/files/ssh/hostkeys/docs.ceph.com.pub: -------------------------------------------------------------------------------- 1 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXvH756ZpNtOTJrg9Exypn5Yb3ABDTqhBvUOOtj7wxzBbXtOYwrvtmls6wMfuHgAOIxjvNIjADkpRYRAuf+TVA= 2 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpJb8vcpiCchL+gRfc749x7/xyo4Gr4VltXFwvK361zlFlfUgpULD1aBpLpAth060QzNBczJymuM6JLCW8d4t0nsKVDKL0dcOySvhZ86tnVKANwDt0S75q0Pd80ClesOZsn+awQ1Rq0eT4dMhLql9PWgSggLOTL+kT8NBFovEIvAiol0uv+L4pVWeJh7FhdMokAHGf7UrbZG/EbjCOvveQmSVLnCUnqxm71y8wQQxGOcLZYdzl2hvIHlI5mJPotx8Pl5QzEs34hF9rltiQ0LJp8gLlFL3ydPW7NK88HxZEgSIwx53v1wJqB6bsf/qxrXOy+Cg4i/i4RjT9ij38Ez8L 3 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFay6H/uhGZ9Z4n18vV87OgNp1sIQTuJ4TLMmqPe6Ulk 4 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | jenkins_dest: /opt/jenkins 3 | jenkins_lib: /var/lib/jenkins 4 | jenkins_jobs: '{{ jenkins_lib }}/jobs' 5 | jenkins: 6 | yum: 7 | #repo: 'deb http://pkg.jenkins-ci.org/debian binary/' # Jenkins repository 8 | dependencies: # Jenkins dependencies 9 | #- 'openjdk-7-jre' 10 | - 'java-1.7.0-openjdk' 11 | - 'git' 12 | - 'curl' 13 | - 'nginx' 14 | config_file: '/etc/sysconfig/jenkins' 15 | apt: 16 | dependencies: # Jenkins dependencies 17 | - 'git' 18 | - 'curl' 19 | - 'nginx' 20 | config_file: '/etc/default/jenkins' 21 | cli_dest: '{{ jenkins_dest }}/jenkins-cli.jar' # Jenkins CLI destination 22 | updates_dest: '{{ jenkins_dest }}/updates_jenkins.json' # Jenkins updates file 23 | -------------------------------------------------------------------------------- /ansible/release.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | vars: 5 | # should be passed in the CLI like `--extra-vars "version=1.23.45 branch=master"` 6 | version: 0-dev # e.g. 0.78 7 | branch: master # any existing branch on Github 8 | release: stable # stable, development, or rc are valid options 9 | tag_name: "v{{ version}}" 10 | project: "ceph" 11 | clean: true # if re-doing a deployment this deletes the remote branch in Jenkin's git repo 12 | force_dch: false # if coming from a rc and wanting to release a stable you need to force dch 13 | debemail: ceph-maintainers@ceph.com 14 | debfullname: "Ceph Release Team" 15 | roles: 16 | - { role: ceph-release, when: "project == 'ceph'" } 17 | - { role: ceph-deploy-release, when: "project == 'ceph-deploy'" } 18 | -------------------------------------------------------------------------------- /gen_debian_version.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | raw=$1 4 | dist=$2 5 | 6 | [ "$dist" = "sid" ] && dver="$raw" 7 | [ "$dist" = "jessie" ] && dver="$raw~bpo80+1" 8 | [ "$dist" = "wheezy" ] && dver="$raw~bpo70+1" 9 | [ "$dist" = "squeeze" ] && dver="$raw~bpo60+1" 10 | [ "$dist" = "lenny" ] && dver="$raw~bpo50+1" 11 | [ "$dist" = "xenial" ] && dver="$raw$dist" 12 | [ "$dist" = "trusty" ] && dver="$raw$dist" 13 | [ "$dist" = "saucy" ] && dver="$raw$dist" 14 | [ "$dist" = "raring" ] && dver="$raw$dist" 15 | [ "$dist" = "precise" ] && dver="$raw$dist" 16 | [ "$dist" = "oneiric" ] && dver="$raw$dist" 17 | [ "$dist" = "natty" ] && dver="$raw$dist" 18 | [ "$dist" = "maverick" ] && dver="$raw$dist" 19 | [ "$dist" = "lucid" ] && dver="$raw$dist" 20 | [ "$dist" = "karmic" ] && dver="$raw$dist" 21 | 22 | echo $dver 23 | 24 | -------------------------------------------------------------------------------- /kernel/build/prepare_config: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Prep the config 5 | echo "Updating the kernel config" 6 | make olddefconfig 7 | 8 | # This way we get the commit hash embedded into the package metadata 9 | # and 'uname -r' in all cases - whether it's some random commit or an 10 | # annotated tag. Example package names: 11 | # uname -r: 4.9.0-rc4-ceph-g156db39ecfbd 12 | # deb: linux-image-4.9.0-rc4-ceph-g156db39ecfbd_4.9.0-rc4-ceph-g156db39ecfbd-1_amd64.deb 13 | # rpm: kernel-4.9.0_rc4_ceph_g156db39ecfbd-2.x86_64.rpm 14 | if ! grep -q "^CONFIG_LOCALVERSION_AUTO=y" .config; then 15 | echo "CONFIG_LOCALVERSION_AUTO is not set, check kernel-config-*.sh" 16 | exit 1 17 | fi 18 | printf -- '-ceph-g%s' ${GIT_COMMIT:0:12} > .scmversion 19 | 20 | kernelrelease=$(make -s kernelrelease) 21 | -------------------------------------------------------------------------------- /ceph-ansible-pr-syntax-check/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible") 7 | install_python_packages "pkgs[@]" 8 | 9 | cd $WORKSPACE/ceph-ansible 10 | 11 | $VENV/ansible-playbook -i '127.0.0.1,' site.yml.sample --syntax-check --list-tasks -vv 12 | $VENV/ansible-playbook -i '127.0.0.1,' site-docker.yml.sample --syntax-check --list-tasks -vv 13 | 14 | # move roles and group_vars into the 15 | # infrastructure-playbooks directory for easier syntax checking 16 | cp -r roles infrastructure-playbooks/ 17 | cp -r group_vars infrastructure-playbooks/ 18 | mv infrastructure-playbooks/group_vars/all.yml.sample infrastructure-playbooks/group_vars/all.yml 19 | 20 | $VENV/ansible-playbook -i '127.0.0.1,' infrastructure-playbooks/*.yml --syntax-check --list-tasks -vv 21 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/ssl.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure ssl certs directory 4 | file: 5 | dest: /etc/ssl/certs 6 | state: directory 7 | sudo: true 8 | 9 | - name: ensure ssl private directory 10 | file: 11 | dest: /etc/ssl/private 12 | state: directory 13 | sudo: true 14 | 15 | - name: copy SSL cert 16 | copy: 17 | src: "{{ ssl_cert_path }}" 18 | dest: "/etc/ssl/certs/{{ item.fqdn }}-bundled.crt" 19 | mode: 0777 20 | force: no 21 | sudo: true 22 | notify: restart nginx 23 | when: nginx_hosts is defined 24 | with_items: nginx_hosts 25 | 26 | - name: copy SSL key 27 | copy: 28 | src: "{{ ssl_key_path }}" 29 | dest: "/etc/ssl/private/{{ item.fqdn }}.key" 30 | force: no 31 | sudo: true 32 | notify: restart nginx 33 | when: nginx_hosts is defined 34 | with_items: nginx_hosts 35 | -------------------------------------------------------------------------------- /get_rpm_dist.sh: -------------------------------------------------------------------------------- 1 | #! /bin/sh -x 2 | 3 | LSB_RELEASE=/usr/bin/lsb_release 4 | [ ! -x $LSB_RELEASE ] && echo unknown && exit 5 | 6 | ID=`$LSB_RELEASE --short --id` 7 | 8 | case $ID in 9 | RedHatEnterpriseServer) 10 | RELEASE=`$LSB_RELEASE --short --release | cut -d. -f1` 11 | DIST=rhel$RELEASE 12 | ;; 13 | CentOS) 14 | RELEASE=`$LSB_RELEASE --short --release | cut -d. -f1` 15 | DIST=el$RELEASE 16 | ;; 17 | Fedora) 18 | RELEASE=`$LSB_RELEASE --short --release` 19 | DIST=fc$RELEASE 20 | ;; 21 | SUSE\ LINUX) 22 | DESC=`$LSB_RELEASE --short --description` 23 | RELEASE=`$LSB_RELEASE --short --release` 24 | case $DESC in 25 | *openSUSE*) 26 | DIST=opensuse$RELEASE 27 | ;; 28 | *Enterprise*) 29 | DIST=sles$RELEASE 30 | ;; 31 | esac 32 | ;; 33 | *) 34 | DIST=unknown 35 | ;; 36 | esac 37 | 38 | echo $DIST 39 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/test-client/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create directory for self-signed cert of docker-registry 3 | file: 4 | path: /etc/docker/certs.d/{{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000 5 | state: directory 6 | 7 | - name: copy self-signed cert of docker-registry 8 | copy: 9 | src: fetch/certs/self.crt 10 | dest: /etc/docker/certs.d/{{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/ca.crt 11 | 12 | - name: pull a small image from docker hub 13 | command: docker pull busybox 14 | 15 | - name: tag image 16 | command: docker tag busybox {{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/mybusybox 17 | 18 | - name: push tagged image to private registry 19 | command: docker push {{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/mybusybox 20 | -------------------------------------------------------------------------------- /gen_reprepro_conf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | 5 | bindir=`dirname $0` 6 | path="$1" 7 | key="$2" 8 | [ -z "$key" ] && echo "usage: $0 " && exit 1 9 | 10 | if [ ! -d $path -o ! -d $path/conf ] ; then 11 | mkdir -p $path/conf 12 | fi 13 | 14 | echo "$bindir" | grep -v -q '^/' && bindir=`pwd`"/$bindir" 15 | dists=`cat $bindir/deb_dists` 16 | components="main" 17 | 18 | echo "dists $dists" 19 | echo "components $components" 20 | 21 | [ -e $path/conf/distributions ] && rm $path/conf/distributions 22 | 23 | for dist in $dists 24 | do 25 | cat <> $path/conf/distributions 26 | Codename: $dist 27 | Suite: stable 28 | Components: $components 29 | Architectures: amd64 armhf arm64 i386 source 30 | Origin: Inktank 31 | Description: Ceph distributed file system 32 | DebIndices: Packages Release . .gz .bz2 33 | DscIndices: Sources Release .gz .bz2 34 | Contents: .gz .bz2 35 | SignWith: $key 36 | 37 | EOF 38 | done 39 | 40 | echo done 41 | -------------------------------------------------------------------------------- /ansible/roles/grafana/templates/nginx_site.conf: -------------------------------------------------------------------------------- 1 | server { 2 | listen 443 default_server ssl; 3 | server_name {{ fqdn }}; 4 | 5 | ssl_certificate /etc/ssl/certs/{{ fqdn }}-bundled.crt; 6 | ssl_certificate_key /etc/ssl/private/{{ fqdn }}.key; 7 | ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 8 | add_header Strict-Transport-Security "max-age=31536000"; 9 | 10 | access_log /var/log/nginx/{{ app_name }}-access.log; 11 | error_log /var/log/nginx/{{ app_name }}-error.log; 12 | 13 | # Some binaries are gigantic 14 | client_max_body_size 2048m; 15 | 16 | location / { 17 | proxy_set_header Host $host; 18 | proxy_set_header X-Real-IP $remote_addr; 19 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 20 | proxy_set_header X-Forwarded-Proto $scheme; 21 | 22 | proxy_pass http://127.0.0.1:3000; 23 | proxy_read_timeout 500; 24 | } 25 | 26 | } 27 | -------------------------------------------------------------------------------- /ceph-deploy-docs/config/definitions/ceph-deploy-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-deploy-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-deploy: docs build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/ceph-deploy 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: 10 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-deploy 26 | branches: 27 | - master 28 | browser: auto 29 | skip-tag: true 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | -------------------------------------------------------------------------------- /teuthology-docs/config/definitions/teuthology-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: teuthology-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'Teuthology: Docs Build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/teuthology 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: -1 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/teuthology.git 26 | branches: 27 | - master 28 | browser: auto 29 | timeout: 20 30 | 31 | builders: 32 | - shell: 33 | !include-raw: 34 | - ../../../scripts/build_utils.sh 35 | - ../../setup/setup 36 | - ../../build/build 37 | -------------------------------------------------------------------------------- /ceph-docs/config/definitions/ceph-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - github: 11 | url: https://github.com/ceph/ceph 12 | discard-old-builds: true 13 | logrotate: 14 | daysToKeep: 1 15 | numToKeep: 10 16 | artifactDaysToKeep: -1 17 | artifactNumToKeep: -1 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph 25 | browser: auto 26 | # The default is to build and publish every branch. 27 | # Uncomment this for testing: 28 | #branches: 29 | # - 'origin/master' 30 | skip-tag: true 31 | timeout: 20 32 | wipe-workspace: true 33 | 34 | builders: 35 | - shell: 36 | !include-raw ../../build/build 37 | -------------------------------------------------------------------------------- /ceph-installer-docs/config/definitions/ceph-installer-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-installer-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-installer: docs build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/ceph-installer 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: 10 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-installer 26 | branches: 27 | - master 28 | browser: auto 29 | skip-tag: true 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | -------------------------------------------------------------------------------- /radosgw-agent-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | install_python_packages "pkgs[@]" 6 | 7 | 8 | # run ansible to get this current host to meet our requirements, specifying 9 | # a local connection and 'localhost' as the host where to execute. This might 10 | # look odd because we are using ceph-deploy playbooks. But the job-specific 11 | # requirements are the same: install different versions of Python (including 12 | # 2.6 and 2.7) 13 | # 14 | # These job-specific requirements aren't met by the services in charge of 15 | # creating slaves (mainly prado.ceph.com) because those slaves have "generic" 16 | # requirements and usually do not care about specific needs like Python 2.6 17 | 18 | cd "$WORKSPACE/ceph-build/ceph-deploy-pull-requests/setup/playbooks" 19 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 20 | 21 | 22 | # create the build with tox 23 | cd $WORKSPACE/radosgw-agent 24 | $VENV/tox -rv 25 | -------------------------------------------------------------------------------- /scripts/sync-pull: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script will pull repository files from the binary API (chacra.ceph.com) 3 | # so that they can be signed. This is only useful in the context of the 4 | # "signer" server. 5 | # By default it will sync all releases defined, but can optionally take one or more 6 | # releases to sync: 7 | # 8 | # sync-pull hammer infernali 9 | 10 | releases=${*:-"firefly giant hammer infernalis testing"} 11 | 12 | ceph_sync() { 13 | release=$1 14 | deb_cmd="admin@chacra.ceph.com:/opt/repos/ceph/$release/debian/jessie/* /opt/repos/ceph/$release/debian/jessie/" 15 | rsync -Lavh -e 'ssh -p 2222' --progress $deb_cmd 16 | 17 | el6_cmd="admin@chacra.ceph.com:/opt/repos/ceph/$release/centos/6/* /opt/repos/ceph/$release/centos/6/" 18 | el7_cmd="admin@chacra.ceph.com:/opt/repos/ceph/$release/centos/7/* /opt/repos/ceph/$release/centos/7/" 19 | rsync -Lavh -e 'ssh -p 2222' --progress $el6_cmd 20 | rsync -Lavh -e 'ssh -p 2222' --progress $el7_cmd 21 | } 22 | 23 | for i in "${releases[@]}" 24 | do 25 | ceph_sync $i 26 | done 27 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/plugins.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: List plugins 3 | shell: java -jar {{ jenkins.cli_dest }} -s http://localhost:{{ port }} list-plugins | cut -f 1 -d ' ' 4 | when: plugins is defined 5 | register: plugins_installed 6 | 7 | - name: Install/update plugins 8 | shell: java -jar {{ jenkins.cli_dest }} -s http://localhost:{{ port }} install-plugin {{ item }} 9 | when: plugins_installed.changed and plugins_installed.stdout.find('{{ item }}') == -1 10 | with_items: plugins 11 | notify: 12 | - 'restart jenkins' 13 | 14 | - name: List plugins to be updated 15 | shell: java -jar {{ jenkins.cli_dest }} -s http://localhost:{{ port }} list-plugins | grep ')$' | cut -f 1 -d ' ' | sed ':a;N;$!ba;s/\n/ /g' 16 | register: plugins_updates 17 | 18 | - name: Update plugins 19 | shell: java -jar {{ jenkins.cli_dest }} -s http://localhost:{{ port }} install-plugin {{ item }} 20 | with_items: plugins_updates.stdout.split() 21 | when: plugins_updates.stdout != '' 22 | ignore_errors: yes 23 | notify: 24 | - 'restart jenkins' 25 | -------------------------------------------------------------------------------- /sign_and_index_rpm_repo.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | releasedir=$1 6 | repo=$2 7 | cephvers=$3 8 | 9 | keyid=460F3994 10 | 11 | usage() { 12 | echo "usage: $0 releasedir repodir version" 13 | } 14 | 15 | [ -z "$releasedir" ] && echo specify releasedir && usage && exit 1 16 | [ -z "$repo" ] && echo specify reprepro dir && usage && exit 1 17 | [ -z "$cephvers" ] && echo specify version && usage && exit 1 18 | [ ! -d "$releasedir/$cephvers" ] && echo missing $releasedir/$cephvers && usage && exit 1 19 | 20 | bindir=`dirname $0` 21 | 22 | echo version $cephvers 23 | 24 | # 25 | # Sign rpms and create repo index 26 | 27 | #echo "signing rpms" 28 | $bindir/sign_rpms.sh $repo $cephvers $keyid 29 | 30 | # Construct repodata 31 | # repo/dist/* 32 | #for dir in $repo/*/* 33 | for dir in $repo/$cephvers/*/* 34 | do 35 | echo "indexing $dir" 36 | if [ -d $dir ] ; then 37 | createrepo --update --checkts $dir 38 | gpg --batch --yes --detach-sign --armor -u $keyid $dir/repodata/repomd.xml 39 | fi 40 | done 41 | 42 | echo done 43 | -------------------------------------------------------------------------------- /kernel-trigger/config/definitions/kernel-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: kernel-trigger 3 | node: master 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - github: 11 | url: https://github.com/ceph/ceph-client 12 | discard-old-builds: true 13 | logrotate: 14 | daysToKeep: 1 15 | numToKeep: 10 16 | artifactDaysToKeep: -1 17 | artifactNumToKeep: -1 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-client.git 25 | branches: 26 | - 'origin/testing*' 27 | - 'origin/master*' 28 | - 'origin/for-linus' 29 | skip-tag: true 30 | timeout: 20 31 | wipe-workspace: true 32 | 33 | builders: 34 | - trigger-builds: 35 | - project: 'kernel' 36 | predefined-parameters: | 37 | BRANCH=${GIT_BRANCH} 38 | FORCE=True 39 | -------------------------------------------------------------------------------- /sign_debs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | set -e 4 | 5 | usage() { 6 | echo "usage: $0 releasedir vers key [filetypetosign] [resultdir]" 7 | } 8 | 9 | releasedir=$1 10 | cephvers=$2 11 | keyid=$3 12 | what=$4 13 | resultdir=$5 14 | 15 | [ -z "$what" ] && what="dsc changes" 16 | 17 | [ -z "$releasedir" ] && echo specify releasedir && usage && exit 1 18 | [ -z "$cephvers" ] && echo specify version && usage && exit 1 19 | [ ! -d "$releasedir/$cephvers" ] && echo missing $releasedir/$cephvers && usage && exit 1 20 | if [ -n "$resultdir" ] ; then 21 | resultdir=$releasedir/$cephvers/$resultdir 22 | [ ! -d "$resultdir" ] && echo missing $resultdir && usage && exit 1 23 | fi 24 | 25 | 26 | [ -z "$keyid" ] && echo specify keyid && exit 1 27 | 28 | echo version $cephvers 29 | 30 | echo "signing $releasedir/$cephvers/*.$w" 31 | for w in $what 32 | do 33 | if [ -n "$resultdir" -a -d "$resultdir" ] ; then 34 | ( cd $resultdir ; yes | debsign -k$keyid *.$w ) 35 | else 36 | yes | debsign -k$keyid $releasedir/$cephvers/*.$w 37 | fi 38 | done 39 | 40 | -------------------------------------------------------------------------------- /scripts/sync-push: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script will push repository files from the signer box to the upstream repositories. 3 | # By default it will sync all releases defined, but can optionally take one or more 4 | # releases to sync: 5 | # 6 | # sync-push hammer infernalis 7 | # 8 | # Since the binaries are created with a different repository layout, this 9 | # script maps directories like "centos/6" to "rpm-$release/el6" 10 | 11 | releases=${*:-"firefly giant hammer infernalis testing"} 12 | 13 | ceph_sync() { 14 | release=$1 15 | deb_cmd="/opt/repos/ceph/$release/debian/jessie/* dhc-user@download.ceph.com:/home/dhc-user/repos/debian-$release/" 16 | rsync --progress -avr $deb_cmd 17 | 18 | el6_cmd="/opt/repos/ceph/$release/centos/6/* dhc-user@download.ceph.com:/home/dhc-user/repos/rpm-$release/el6/" 19 | el7_cmd="/opt/repos/ceph/$release/centos/7/* dhc-user@download.ceph.com:/home/dhc-user/repos/rpm-$release/el7/" 20 | rsync --progress -avr $el6_cmd 21 | rsync --progress -avr $el7_cmd 22 | } 23 | 24 | for i in "${releases[@]}" 25 | do 26 | ceph_sync $i 27 | done 28 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/repo.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | # Yum 4 | 5 | - name: Install python-devel 6 | yum: name=python-devel state=present 7 | when: ansible_pkg_mgr == "yum" 8 | 9 | - name: Install python-pycurl 10 | yum: name=python-pycurl state=present 11 | when: ansible_pkg_mgr == "yum" 12 | 13 | - name: Add Jenkins GPG Key 14 | rpm_key: state=present key=https://jenkins-ci.org/redhat/jenkins-ci.org.key 15 | when: ansible_pkg_mgr == "yum" 16 | 17 | - name: Create the repo file for Jenkins 18 | copy: src=jenkins.repo 19 | dest=/etc/yum.repos.d/ 20 | owner=root 21 | group=root 22 | mode=0644 23 | when: ansible_pkg_mgr == "yum" 24 | 25 | # Apt 26 | 27 | - name: Add Jenkins GPG Key 28 | apt_key: 29 | url: "https://jenkins-ci.org/debian/jenkins-ci.org.key" 30 | state: present 31 | when: ansible_pkg_mgr == "apt" 32 | 33 | - name: Add the jenkins repo 34 | apt_repository: 35 | repo: 'deb http://pkg.jenkins-ci.org/debian binary/' 36 | state: present 37 | update_cache: true 38 | when: ansible_pkg_mgr == "apt" 39 | -------------------------------------------------------------------------------- /sign_rpms.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | usage() { 6 | echo "usage: $0 repodir vers key" 7 | } 8 | 9 | repodir=$1 10 | cephvers=$2 11 | keyid=$3 12 | bindir=`dirname $0` 13 | 14 | [ -z "$repodir" ] && echo specify repodir && usage && exit 1 15 | [ -z "$cephvers" ] && echo specify version && usage && exit 1 16 | [ ! -d "$repodir/$cephvers" ] && echo missing $repodir/$cephvers && usage && exit 1 17 | 18 | [ -z "$keyid" ] && echo specify keyid && exit 1 19 | 20 | echo "signing rpms, version $cephvers key $keyid" 21 | 22 | # Sign all the RPMs for this release 23 | #rpm --addsign --define "_gpg_name $keyid" $rpm 24 | #use expect wrapper to supply null passphrase 25 | 26 | #shopt -s nocasematch 27 | for rpm in `find ${repodir}/${cephvers} -name "*.rpm"` 28 | do 29 | signature=$(rpm -qi -p $rpm 2>/dev/null | grep ^Signature) 30 | if grep -iq $keyid <<< "$signature" ; then 31 | echo "skipping: $rpm" 32 | else 33 | echo "signing: $rpm" 34 | $bindir/rpm-autosign.exp --define "_gpg_name $keyid" $rpm 35 | fi 36 | done 37 | #shopt -u nocasematch 38 | 39 | echo done 40 | -------------------------------------------------------------------------------- /ansible/examples/deploy_grafana.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - nginx 7 | - grafana 8 | vars_files: 9 | - vars/load-balance-vars.yml 10 | vars: 11 | fqdn: "grafana.ceph.com" 12 | app_name: "grafana" 13 | development_server: true 14 | ansible_ssh_port: 22 15 | # These are defined in `vars/load-balance-vars.yml` but defaulting to what 16 | # is used in production. For local development, update these to match the 17 | # hosts used in your local development environment 18 | #nginx_hosts: 19 | # - fqdn: "grafana.ceph.com" 20 | # app_name: "grafana" 21 | # proxy_pass: "http://127.0.0.1:3000" 22 | # - fqdn: "shaman.ceph.com" 23 | # app_name: "shaman" 24 | # upstreams: 25 | # - name: "shaman" 26 | # strategy: "least_conn" 27 | # servers: 28 | # - "1.shaman.ceph.com" 29 | # - "2.shaman.ceph.com" 30 | 31 | # only needed when enabling Github Auth 32 | # github_client_id: "111aaa222" 33 | # github_client_secret: "qwerty1234" 34 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/jenkins.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | server { 4 | listen 80 default_server; 5 | listen 443 default_server ssl; 6 | 7 | server_name localhost {{ ansible_fqdn }}; 8 | 9 | ssl_certificate /etc/ssl/certs/{{ ansible_fqdn }}-bundled.crt; 10 | ssl_certificate_key /etc/ssl/private/{{ ansible_fqdn }}.key; 11 | ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 12 | add_header Strict-Transport-Security "max-age=31536000"; 13 | 14 | access_log /var/log/nginx/jenkins_access.log; 15 | error_log /var/log/nginx/jenkins_error.log; 16 | 17 | location / { 18 | proxy_set_header Host $host; 19 | proxy_set_header X-Real-IP $remote_addr; 20 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 21 | proxy_set_header X-Forwarded-Proto $scheme; 22 | 23 | proxy_pass http://127.0.0.1:8080; 24 | proxy_read_timeout 90; 25 | 26 | # Redirect all plaintext HTTP to HTTPS 27 | if ($scheme != "https") { 28 | rewrite ^ https://$host$uri permanent; 29 | } 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /ansible/examples/master.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: jenkins 4 | user: admin 5 | sudo: true 6 | roles: 7 | - ansible-jenkins 8 | vars: 9 | - nginx_processor_count: 20 10 | - nginx_connections: 2048 11 | - ansible_ssh_port: 2222 12 | - plugins: 13 | - 'github' 14 | - 'translation' 15 | - 'preSCMbuildstep' 16 | - 'copyartifact' 17 | - 'ssh-slaves' 18 | - 'scm-api' 19 | - 'multiple-scms' 20 | - 'rebuild' 21 | - 'publish-over-ssh' 22 | - 'nested-view' 23 | - 'jenkins-multijob-plugin' 24 | - 'dynamic-axis' 25 | - 'ghprb' 26 | - 'github' 27 | - 'github-api' 28 | - 'git' 29 | - 'git-client' 30 | - 'envinject' 31 | - 'credentials' 32 | - 'copyartifact' 33 | - 'github-oauth' 34 | - 'mask-passwords' 35 | - 'description-setter' 36 | - 'postbuildscript' 37 | - 'jobConfigHistory' 38 | 39 | - port: 8080 40 | - prefix: '/build' 41 | - xmx: 8192 42 | # Email support 43 | #- email: 44 | # smtp_host: 'mail.example.com' 45 | # smtp_ssl: 'true' 46 | # default_email_suffix: '@example.com' 47 | -------------------------------------------------------------------------------- /push_to_rpm_repo.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | releasedir=$1 6 | repo=$2 7 | cephvers=$3 8 | 9 | keyid=460F3994 10 | 11 | usage() { 12 | echo "usage: $0 releasedir repodir version component" 13 | } 14 | 15 | [ -z "$releasedir" ] && echo specify releasedir && usage && exit 1 16 | [ -z "$repo" ] && echo specify reprepro dir && usage && exit 1 17 | [ -z "$cephvers" ] && echo specify version && usage && exit 1 18 | [ ! -d "$releasedir/$cephvers" ] && echo missing $releasedir/$cephvers && usage && exit 1 19 | 20 | bindir=`dirname $0` 21 | 22 | echo version $cephvers 23 | #echo component $component 24 | 25 | mkdir -p $repo 26 | 27 | # For each distribution that we've built 28 | for dist in `ls -1 $releasedir/$cephvers/rpm` 29 | do 30 | echo dist $dist 31 | # copy binary and source rpms to repo 32 | for dir in $releasedir/$cephvers/rpm/$dist/RPMS/* $releasedir/$cephvers/rpm/$dist/SRPMS 33 | do 34 | mkdir -p $repo/$cephvers/$dist 35 | cp -a $dir $repo/$cephvers/$dist/. 36 | done 37 | # Add a yum or zypper release rpm to repo 38 | $bindir/gen_yum_zypper_repo_rpm.sh $releasedir $repo $cephvers $dist 39 | done 40 | 41 | echo done 42 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/nginx.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure sites-available for nginx 4 | file: path=/etc/nginx/sites-available state=directory 5 | 6 | - name: ensure sites-enable for nginx 7 | file: path=/etc/nginx/sites-enabled state=directory 8 | 9 | - name: remove default nginx site 10 | action: file path=/etc/nginx/sites-enabled/default state=absent 11 | 12 | - name: write nginx.conf 13 | action: template src=../templates/nginx.conf dest=/etc/nginx/nginx.conf 14 | 15 | - name: write jenkins SSL public cert 16 | copy: 17 | src: jenkins.ceph.com-bundled.crt 18 | dest: /etc/ssl/certs/ 19 | owner: root 20 | group: root 21 | mode: 0644 22 | when: ansible_pkg_mgr == 'apt' 23 | 24 | - name: create nginx site config 25 | action: template src=../templates/jenkins.conf dest=/etc/nginx/sites-available/jenkins.conf 26 | notify: 27 | - restart nginx 28 | 29 | - name: link nginx config 30 | action: file src=/etc/nginx/sites-available/jenkins.conf dest=/etc/nginx/sites-enabled/jenkins.conf state=link 31 | 32 | - name: Enable Nginx service 33 | service: name=nginx enabled=yes state=started 34 | -------------------------------------------------------------------------------- /ansible/vars/sensu-vars.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Set the rabbitmq address 3 | rabbitmq_conf_tcp_listeners_address: '0.0.0.0' 4 | 5 | sensu_server_rabbitmq_insecure: true 6 | 7 | # Sensu client variable 8 | 9 | sensu_checks: 10 | cpu: 11 | command: "check-cpu.rb" 12 | interval: 10 13 | subscribers: 14 | - common 15 | disk: 16 | command: "check-disk-usage.rb -w 85 -c 95" 17 | interval: 10 18 | subscribers: 19 | - common 20 | load: 21 | command: "check-load.rb" 22 | interval: 10 23 | subscribers: 24 | - common 25 | memory: 26 | command: "check-memory.rb -w 85 -c 95" 27 | interval: 10 28 | subscribers: 29 | - common 30 | rabbitmq-alive: 31 | command: "check-rabbitmq-amqp-alive.rb -u :::rabbitmq.user|guest::: -p :::rabbitmq.password|guest::: -v :::rabbitmq.vhost|%2F:::" 32 | interval: 10 33 | subscribers: 34 | - rabbitmq 35 | 36 | sensu_ruby_gem_plugins: 37 | - "sensu-plugins-load-checks" 38 | - "sensu-plugins-memory-checks" 39 | - "sensu-plugins-disk-checks" 40 | - "sensu-plugins-cpu-checks" 41 | - "sensu-plugins-rabbitmq" 42 | 43 | # Dummy sensu_handlers 44 | sensu_handlers: 45 | test_handler: 46 | type : pipe 47 | command: "echo" 48 | -------------------------------------------------------------------------------- /ceph-ansible-galaxy/config/definitions/ceph-ansible-galaxy.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-ansible-galaxy 3 | node: small && trusty 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-ansible: Update galaxy roles' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/ceph-ansible 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: -1 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-ansible.git 26 | branches: 27 | - master 28 | browser: auto 29 | basedir: "ceph-ansible" 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | 38 | wrappers: 39 | - ssh-agent-credentials: 40 | # "jenkins-build" SSH key, needed for access to ceph-ansible.git 41 | users: 42 | - '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 43 | -------------------------------------------------------------------------------- /ceph-ansible-rpm/config/definitions/ceph-ansible-rpm.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-ansible-rpm 3 | node: 'centos7 && x86_64 && small && !sepia' 4 | project-type: freestyle 5 | defaults: global 6 | disabled: false 7 | display-name: 'ceph-ansible: RPMs' 8 | description: 'Build RPMs for every ceph-ansible Git branch' 9 | concurrent: true 10 | quiet-period: 5 11 | block-downstream: false 12 | block-upstream: false 13 | retry-count: 3 14 | properties: 15 | - github: 16 | url: https://github.com/ceph/ceph-ansible 17 | discard-old-builds: true 18 | logrotate: 19 | daysToKeep: 1 20 | numToKeep: 10 21 | artifactDaysToKeep: -1 22 | artifactNumToKeep: -1 23 | 24 | triggers: 25 | - github 26 | 27 | scm: 28 | - git: 29 | url: https://github.com/ceph/ceph-ansible 30 | browser: auto 31 | skip-tag: true 32 | timeout: 20 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | -------------------------------------------------------------------------------- /ceph-installer-rpm/config/definitions/ceph-installer-rpm.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-installer-rpm 3 | node: 'centos7 && x86_64 && small && !sepia' 4 | project-type: freestyle 5 | defaults: global 6 | disabled: false 7 | display-name: 'ceph-installer: RPMs' 8 | description: 'Build RPMs for every ceph-installer Git branch' 9 | concurrent: true 10 | quiet-period: 5 11 | block-downstream: false 12 | block-upstream: false 13 | retry-count: 3 14 | properties: 15 | - github: 16 | url: https://github.com/ceph/ceph-installer 17 | discard-old-builds: true 18 | logrotate: 19 | daysToKeep: 1 20 | numToKeep: 10 21 | artifactDaysToKeep: -1 22 | artifactNumToKeep: -1 23 | 24 | triggers: 25 | - github 26 | 27 | scm: 28 | - git: 29 | url: https://github.com/ceph/ceph-installer 30 | browser: auto 31 | skip-tag: true 32 | timeout: 20 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | -------------------------------------------------------------------------------- /ceph-deploy-build/config/definitions/ceph-deploy-build.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-deploy-build 3 | node: small && trusty 4 | project-type: matrix 5 | defaults: global 6 | display-name: 'ceph-deploy-build' 7 | concurrent: true 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | 13 | scm: 14 | - git: 15 | url: https://github.com/ceph/ceph-deploy.git 16 | branches: 17 | - $BRANCH 18 | browser: auto 19 | skip-tag: true 20 | timeout: 20 21 | wipe-workspace: true 22 | 23 | axes: 24 | - axis: 25 | type: label-expression 26 | name: ARCH 27 | values: 28 | - x86_64 29 | - axis: 30 | type: label-expression 31 | name: DIST 32 | values: 33 | - trusty 34 | - jessie 35 | - centos6 36 | - centos7 37 | 38 | builders: 39 | - shell: 40 | !include-raw: 41 | - ../../../scripts/build_utils.sh 42 | - ../../build/setup 43 | - ../../build/build 44 | 45 | wrappers: 46 | - inject-passwords: 47 | global: true 48 | mask-password-params: true 49 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ensure database service is up 3 | service: 4 | name: postgresql 5 | state: started 6 | enabled: yes 7 | sudo: yes 8 | 9 | - name: allow users to connect locally 10 | sudo: yes 11 | lineinfile: 12 | # TODO: should not hardcode that version 13 | dest: /etc/postgresql/9.5/main/pg_hba.conf 14 | regexp: '^host\s+all\s+all\s+127.0.0.1/32' 15 | line: 'host all all 127.0.0.1/32 md5' 16 | backrefs: yes 17 | register: pg_hba_conf 18 | 19 | - service: 20 | name: postgresql 21 | state: restarted 22 | sudo: true 23 | when: pg_hba_conf.changed 24 | 25 | - name: make {{ app_name }} user 26 | postgresql_user: 27 | name: "{{ app_name }}" 28 | password: "{{ db_password.stdout }}" 29 | role_attr_flags: SUPERUSER 30 | login_user: postgres 31 | become_user: postgres 32 | become: yes 33 | 34 | - name: Make {{ app_name }} database 35 | postgresql_db: 36 | name: "{{ app_name }}" 37 | owner: "{{ app_name }}" 38 | state: present 39 | login_user: postgres 40 | sudo_user: postgres 41 | sudo: yes 42 | 43 | - name: ensure database service is up 44 | service: 45 | name: postgresql 46 | state: started 47 | enabled: yes 48 | sudo: yes 49 | -------------------------------------------------------------------------------- /push_to_deb_repo.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | releasedir=$1 6 | repo=$2 7 | cephvers=$3 8 | component=$4 9 | 10 | usage() { 11 | echo "usage: $0 releasedir repodir version component" 12 | } 13 | 14 | [ -z "$releasedir" ] && echo specify releasedir && usage && exit 1 15 | [ -z "$repo" ] && echo specify reprepro dir && usage && exit 1 16 | [ -z "$cephvers" ] && echo specify version && usage && exit 1 17 | [ ! -d "$releasedir/$cephvers" ] && echo missing $releasedir/$cephvers && usage && exit 1 18 | [ -z "$component" ] && echo "must specify repo component" && usage && exit 1 19 | 20 | bindir=`dirname $0` 21 | 22 | echo version $cephvers 23 | echo component $component 24 | 25 | [ -z "$dists" ] && dists=`cat $releasedir/$cephvers/debian_dists` 26 | dvers=`cat $releasedir/$cephvers/debian_version` 27 | echo deb vers $dvers 28 | echo dists $dists 29 | 30 | 31 | for dist in $dists 32 | do 33 | bpvers=`$bindir/gen_debian_version.sh $dvers $dist` 34 | echo dist $dist 35 | echo vers $bpvers 36 | #for f in $releasedir/$cephvers/*${bpvers}_*.changes 37 | for f in `find $releasedir/$cephvers/ -name "*${bpvers}_*.changes"` 38 | do 39 | echo file $f 40 | reprepro --ask-passphrase -b $repo -C $component --ignore=undefinedtarget --ignore=wrongdistribution include $dist $f 41 | done 42 | done 43 | -------------------------------------------------------------------------------- /ceph-installer-coverage/config/definitions/ceph-installer-coverage.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-installer-coverage 3 | node: small && (trusty || centos) 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-installer: coverage' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/ceph-installer 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: 10 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-installer 26 | branches: 27 | - master 28 | browser: auto 29 | skip-tag: true 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | 38 | publishers: 39 | - cobertura: 40 | results: coverage.xml 41 | only-stable: "true" 42 | fail-no-reports: "true" 43 | targets: 44 | - method: 45 | healthy: 50 46 | - junit: 47 | results: junit.xml 48 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: "Build hosts file" 4 | sudo: yes 5 | lineinfile: 6 | dest: /etc/hosts 7 | regexp: ".*{{ fqdn }}$" 8 | line: "127.0.1.1 {{ fqdn }}" 9 | state: present 10 | 11 | - name: Set Hostname with hostname command 12 | sudo: yes 13 | hostname: name="{{ fqdn }}" 14 | 15 | - name: update apt cache 16 | apt: 17 | update_cache: yes 18 | sudo: yes 19 | 20 | - name: install ssl system requirements 21 | sudo: yes 22 | apt: 23 | name: "{{ item }}" 24 | state: present 25 | with_items: ssl_requirements 26 | tags: 27 | - packages 28 | 29 | - name: install system packages 30 | sudo: yes 31 | apt: 32 | name: "{{ item }}" 33 | state: present 34 | with_items: system_packages 35 | tags: 36 | - packages 37 | 38 | - command: cp /usr/share/graphite-web/graphite.wsgi /usr/lib/python2.7/dist-packages/graphite/graphite_web.py 39 | args: 40 | creates: "/usr/lib/python2.7/dist-packages/graphite/graphite_web.py" 41 | sudo: true 42 | 43 | - include: carbon.yml 44 | 45 | - include: systemd.yml 46 | tags: 47 | - systemd 48 | 49 | - include: postgresql.yml 50 | tags: 51 | - postgresql 52 | 53 | - name: ensure graphite is running 54 | sudo: true 55 | service: 56 | name: graphite 57 | state: restarted 58 | enabled: yes 59 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/carbon.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: enable carbon 4 | lineinfile: 5 | dest: /etc/default/graphite-carbon 6 | regexp: "^CARBON_CACHE_ENABLED=false" 7 | line: "CARBON_CACHE_ENABLED=true" 8 | state: present 9 | sudo: true 10 | 11 | - name: enable whitelisting in carbon 12 | lineinfile: 13 | dest: /etc/carbon/carbon.conf 14 | regexp: "^# USE_WHITELIST = False" 15 | line: "USE_WHITELIST = True" 16 | state: present 17 | backrefs: true 18 | sudo: true 19 | 20 | - name: create the rewrite config with the secret api key 21 | template: 22 | src: ../templates/rewrite-rules.conf.j2 23 | dest: "/etc/carbon/rewrite-rules.conf" 24 | notify: 25 | - restart carbon 26 | sudo: true 27 | 28 | - name: create the whitelist/blacklist config allowing the api key only 29 | template: 30 | src: ../templates/whitelist.conf.j2 31 | dest: "/etc/carbon/whitelist.conf" 32 | notify: 33 | - restart carbon 34 | sudo: true 35 | 36 | - name: define the storage schemas 37 | template: 38 | src: ../templates/storage-schemas.conf.j2 39 | dest: "/etc/carbon/storage-schemas.conf" 40 | notify: 41 | - restart carbon 42 | sudo: true 43 | 44 | - name: ensure database service is up 45 | service: 46 | name: carbon-cache 47 | state: restarted 48 | enabled: yes 49 | sudo: yes 50 | -------------------------------------------------------------------------------- /chacra-pull-requests/setup/playbooks/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: update apt cache 3 | apt: 4 | update_cache: yes 5 | sudo: yes 6 | 7 | - name: install postgresql requirements 8 | sudo: yes 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: 13 | - postgresql 14 | - postgresql-common 15 | - postgresql-contrib 16 | - postgresql-server-dev-9.3 17 | - python-psycopg2 18 | tags: 19 | - packages 20 | 21 | - name: ensure database service is up 22 | service: 23 | name: postgresql 24 | state: started 25 | enabled: yes 26 | sudo: yes 27 | 28 | - name: allow users to connect locally 29 | sudo: yes 30 | lineinfile: 31 | # TODO: should not hardcode that version 32 | # 9.3 is available on trusty, 9.5 on Xenial 33 | dest: /etc/postgresql/9.3/main/pg_hba.conf 34 | regexp: '^host\s+all\s+all\s+127.0.0.1/32' 35 | line: 'host all all 127.0.0.1/32 trust' 36 | backrefs: yes 37 | register: pg_hba_conf 38 | 39 | - name: make jenkins-build user 40 | postgresql_user: 41 | name: "jenkins-build" 42 | password: "secret" 43 | role_attr_flags: SUPERUSER 44 | login_user: postgres 45 | become_user: postgres 46 | become: yes 47 | 48 | - service: 49 | name: postgresql 50 | state: restarted 51 | sudo: true 52 | -------------------------------------------------------------------------------- /mita-deploy/config/definitions/mita-deploy.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: mita 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/mita.git 6 | branches: 7 | - master 8 | browser: auto 9 | timeout: 20 10 | skip-tag: true 11 | wipe-workspace: true 12 | 13 | - job: 14 | name: mita-deploy 15 | node: master 16 | description: "This job clones mita and deploys it to its production server based on the BRANCH value" 17 | display-name: 'mita-deploy' 18 | logrotate: 19 | daysToKeep: -1 20 | numToKeep: 25 21 | artifactDaysToKeep: -1 22 | artifactNumToKeep: -1 23 | block-downstream: false 24 | block-upstream: false 25 | properties: 26 | - github: 27 | url: https://github.com/ceph/mita 28 | 29 | parameters: 30 | - string: 31 | name: BRANCH 32 | description: "The git branch (or tag) to build, defaults to 'master'" 33 | default: "master" 34 | scm: 35 | - mita 36 | 37 | triggers: 38 | - github 39 | 40 | builders: 41 | - shell: 42 | !include-raw: 43 | - ../../../scripts/build_utils.sh 44 | - ../../build/build 45 | 46 | wrappers: 47 | - inject-passwords: 48 | global: true 49 | mask-password-params: true 50 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: update apt cache 3 | apt: 4 | update_cache: yes 5 | sudo: yes 6 | 7 | - name: install postgresql requirements 8 | sudo: yes 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: 13 | - postgresql 14 | - postgresql-common 15 | - postgresql-contrib 16 | - postgresql-server-dev-9.3 17 | - python-psycopg2 18 | tags: 19 | - packages 20 | 21 | - name: ensure database service is up 22 | service: 23 | name: postgresql 24 | state: started 25 | enabled: yes 26 | sudo: yes 27 | 28 | - name: allow users to connect locally 29 | sudo: yes 30 | lineinfile: 31 | # TODO: should not hardcode that version 32 | # 9.3 is available on trusty, 9.5 on Xenial 33 | dest: /etc/postgresql/9.3/main/pg_hba.conf 34 | regexp: '^host\s+all\s+all\s+127.0.0.1/32' 35 | line: 'host all all 127.0.0.1/32 trust' 36 | backrefs: yes 37 | register: pg_hba_conf 38 | 39 | - name: make jenkins-build user 40 | postgresql_user: 41 | name: "jenkins-build" 42 | password: "secret" 43 | role_attr_flags: SUPERUSER 44 | login_user: postgres 45 | become_user: postgres 46 | become: yes 47 | 48 | - service: 49 | name: postgresql 50 | state: restarted 51 | sudo: true 52 | -------------------------------------------------------------------------------- /samba/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.4" ) 39 | install_python_packages "pkgs[@]" 40 | 41 | # ask shaman which chacra instance to use 42 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 43 | # create the .chacractl config file using global variables 44 | make_chacractl_config $chacra_url 45 | -------------------------------------------------------------------------------- /calamari/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | sudo rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.4" ) 39 | install_python_packages "pkgs[@]" 40 | 41 | # ask shaman which chacra instance to use 42 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 43 | # create the .chacractl config file using global variables 44 | make_chacractl_config $chacra_url 45 | -------------------------------------------------------------------------------- /ceph-dev-new-trigger/config/definitions/ceph-dev-new-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-dev-new-trigger 3 | node: master 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - github: 11 | url: https://github.com/ceph/ceph-ci 12 | discard-old-builds: true 13 | logrotate: 14 | daysToKeep: 1 15 | numToKeep: 10 16 | artifactDaysToKeep: -1 17 | artifactNumToKeep: -1 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-ci 25 | browser: auto 26 | skip-tag: true 27 | timeout: 20 28 | wipe-workspace: true 29 | 30 | builders: 31 | - shell: 32 | !include-raw: 33 | - ../../../scripts/build_utils.sh 34 | - ../../build/notify 35 | - trigger-builds: 36 | - project: 'ceph-dev-new' 37 | predefined-parameters: | 38 | BRANCH=${GIT_BRANCH} 39 | FORCE=True 40 | - project: 'ceph-dev-new' 41 | predefined-parameters: | 42 | BRANCH=${GIT_BRANCH} 43 | FORCE=True 44 | FLAVOR=notcmalloc 45 | 46 | wrappers: 47 | - inject-passwords: 48 | global: true 49 | mask-password-params: true 50 | -------------------------------------------------------------------------------- /ansible/roles/ceph-deploy-release/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: check if ceph-deploy repo exists 4 | stat: path='./ceph-deploy' 5 | register: 'cdep_repo' 6 | 7 | - name: clone the ceph-deploy repository 8 | git: repo=git@github.com:ceph/ceph-deploy dest=ceph-deploy 9 | when: cdep_repo.stat.exists is defined and cdep_repo.stat.exists == false 10 | 11 | - name: rename origin to jenkins 12 | command: git remote rename origin jenkins chdir=ceph-deploy 13 | ignore_errors: yes 14 | 15 | - name: fetch the latest from remote 16 | command: git fetch jenkins chdir=ceph-deploy 17 | 18 | - name: ensure local repo is in sync with remote 19 | command: git reset --hard jenkins/{{ branch }} chdir=ceph-deploy 20 | 21 | - name: check if we are re-pushing the release commit 22 | command: git log -1 --pretty=%B chdir=ceph-deploy 23 | register: 'last_commit' 24 | 25 | # we probably messed up the previous commit+tag, so we chose to use 'clean' 26 | # that will rollback that commit, delete the local and remote tag, and force 27 | # push the new changes 28 | - include: clear_version.yml 29 | when: (clean and last_commit.stdout == tag_name) 30 | 31 | # if the last commit wasn't one that we already did, then go ahead and make 32 | # the changes + tag for the release. Otherwise, just skip because it was 33 | # already done for this release 34 | - include: release.yml 35 | when: (tag_name != last_commit.stdout) 36 | -------------------------------------------------------------------------------- /calamari/build/build_deb: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test "$DISTRO" != "debian" -a "$DISTRO" != "ubuntu"; then 6 | exit 0 7 | fi 8 | 9 | 10 | ## Install any setup-time deps 11 | # We need this for mk-build-deps 12 | sudo apt-get install -y equivs 13 | 14 | # Run the install-deps.sh upstream script if it exists 15 | if [ -x install-deps.sh ]; then 16 | echo "Ensuring dependencies are installed" 17 | sudo ./install-deps.sh 18 | fi 19 | 20 | 21 | ## Get some basic information about the system and the repository 22 | DEB_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) 23 | # Get VERSION, REVISION 24 | eval $(./get-versions.sh) 25 | 26 | 27 | ## Install debian build-time dependencies 28 | yes | sudo mk-build-deps --install debian/control 29 | 30 | ## Clean any old and build the new packages 31 | rm -f ../calamari-server*.deb ../calamari-server*.changes 32 | make dpkg 33 | 34 | 35 | ## Upload the created DEBs to chacra 36 | chacra_endpoint="calamari/${BRANCH}/${GIT_COMMIT}/${DISTRO}/${DIST}" 37 | 38 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 39 | 40 | # push binaries to chacra 41 | find ../calamari-server*.deb ../calamari-server*.changes | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/${DEB_ARCH}/ 42 | 43 | # start repo creation 44 | $VENV/chacractl repo update ${chacra_endpoint} 45 | 46 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 47 | -------------------------------------------------------------------------------- /nfs-ganesha/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | cd $WORKSPACE/nfs-ganesha 24 | git clean -fxd 25 | 26 | cd $WORKSPACE/nfs-ganesha-debian 27 | git clean -fxd 28 | 29 | # Make sure the dist directory is clean 30 | cd $WORKSPACE 31 | rm -rf dist 32 | mkdir -p dist 33 | 34 | # Print some basic system info 35 | HOST=$(hostname --short) 36 | echo "Building on $(hostname) with the following env" 37 | echo "*****" 38 | env 39 | echo "*****" 40 | 41 | export LC_ALL=C # the following is vulnerable to i18n 42 | 43 | pkgs=( "chacractl>=0.0.4" ) 44 | install_python_packages "pkgs[@]" 45 | 46 | # ask shaman which chacra instance to use 47 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 48 | # create the .chacractl config file using global variables 49 | make_chacractl_config $chacra_url 50 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/nginx.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | user www-data; 3 | worker_processes {{ nginx_processor_count }}; 4 | worker_rlimit_nofile 8192; 5 | 6 | pid /var/run/nginx.pid; 7 | 8 | events { 9 | worker_connections {{ nginx_connections }} ; 10 | # multi_accept on; 11 | } 12 | 13 | http { 14 | 15 | ## 16 | # Basic Settings 17 | ## 18 | 19 | #sendfile on; 20 | tcp_nopush on; 21 | tcp_nodelay on; 22 | keepalive_timeout 65; 23 | types_hash_max_size 2048; 24 | server_tokens off; 25 | 26 | # server_names_hash_bucket_size 64; 27 | # server_name_in_redirect off; 28 | 29 | include /etc/nginx/mime.types; 30 | default_type application/octet-stream; 31 | 32 | ## 33 | # Logging Settings 34 | ## 35 | 36 | access_log /var/log/nginx/access.log; 37 | error_log /var/log/nginx/error.log; 38 | 39 | ## 40 | # Gzip Settings 41 | ## 42 | 43 | gzip on; 44 | gzip_disable "msie6"; 45 | 46 | # gzip_vary on; 47 | # gzip_proxied any; 48 | # gzip_comp_level 6; 49 | # gzip_buffers 16 8k; 50 | # gzip_http_version 1.1; 51 | # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; 52 | 53 | ## 54 | # If HTTPS, then set a variable so it can be passed along. 55 | ## 56 | 57 | map $scheme $server_https { 58 | default off; 59 | https on; 60 | } 61 | 62 | ## 63 | # Virtual Host Configs 64 | ## 65 | 66 | include /etc/nginx/conf.d/*.conf; 67 | include /etc/nginx/sites-enabled/*; 68 | } 69 | 70 | -------------------------------------------------------------------------------- /ceph-ansible-rpm/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # Sanity-check: 6 | [ -z "$GIT_BRANCH" ] && echo Missing GIT_BRANCH variable && exit 1 7 | [ -z "$JOB_NAME" ] && echo Missing JOB_NAME variable && exit 1 8 | 9 | # Strip "-rpm" off the job name to get our package's name 10 | PACKAGE=${JOB_NAME%-rpm} 11 | 12 | sudo yum -y install epel-release 13 | sudo yum -y install fedpkg 14 | 15 | # Add the Jenkins slave UID to the mock group. 16 | sudo usermod -a -G mock $(whoami) 17 | newgrp mock 18 | 19 | # Attempt the build. If it fails, print the mock logs to STDOUT. 20 | make rpm || ( tail -n +1 {root,build}.log && exit 1 ) 21 | 22 | # Chacra time 23 | 24 | pkgs=( "chacractl>=0.0.4" ) 25 | install_python_packages "pkgs[@]" 26 | 27 | # ask shaman which chacra instance to use 28 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 29 | # create the .chacractl config file using global variables 30 | make_chacractl_config $chacra_url 31 | 32 | BRANCH=`branch_slash_filter $GIT_BRANCH` 33 | 34 | ## Upload the created RPMs to chacra 35 | chacra_endpoint="${PACKAGE}/${BRANCH}/${GIT_COMMIT}/centos/7" 36 | 37 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 38 | 39 | # push binaries to chacra 40 | ls *.rpm | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/noarch/ 41 | 42 | # start repo creation 43 | $VENV/chacractl repo update ${chacra_endpoint} 44 | 45 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 46 | -------------------------------------------------------------------------------- /ceph-installer-rpm/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # Sanity-check: 6 | [ -z "$GIT_BRANCH" ] && echo Missing GIT_BRANCH variable && exit 1 7 | [ -z "$JOB_NAME" ] && echo Missing JOB_NAME variable && exit 1 8 | 9 | # Strip "-rpm" off the job name to get our package's name 10 | PACKAGE=${JOB_NAME%-rpm} 11 | 12 | sudo yum -y install epel-release 13 | sudo yum -y install fedpkg 14 | 15 | # Add the Jenkins slave UID to the mock group. 16 | sudo usermod -a -G mock $(whoami) 17 | newgrp mock 18 | 19 | # Attempt the build. If it fails, print the mock logs to STDOUT. 20 | make rpm || ( tail -n +1 {root,build}.log && exit 1 ) 21 | 22 | # Chacra time 23 | 24 | pkgs=( "chacractl>=0.0.4" ) 25 | install_python_packages "pkgs[@]" 26 | 27 | # ask shaman which chacra instance to use 28 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 29 | # create the .chacractl config file using global variables 30 | make_chacractl_config $chacra_url 31 | 32 | BRANCH=`branch_slash_filter $GIT_BRANCH` 33 | 34 | ## Upload the created RPMs to chacra 35 | chacra_endpoint="${PACKAGE}/${BRANCH}/${GIT_COMMIT}/centos/7" 36 | 37 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 38 | 39 | # push binaries to chacra 40 | ls *.rpm | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/noarch/ 41 | 42 | # start repo creation 43 | $VENV/chacractl repo update ${chacra_endpoint} 44 | 45 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 46 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: update apt cache 3 | apt: 4 | update_cache: yes 5 | sudo: yes 6 | 7 | - name: install ssl system requirements 8 | sudo: yes 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: ssl_requirements 13 | tags: 14 | - packages 15 | 16 | - name: install system packages 17 | sudo: yes 18 | apt: 19 | name: "{{ item }}" 20 | state: present 21 | with_items: system_packages 22 | tags: 23 | - packages 24 | 25 | - name: generate pseudo-random password for admin user 26 | shell: python -c "exec 'import os; print os.urandom(30).encode(\'base64\')[:${length}]'" 27 | register: admin_password 28 | changed_when: false 29 | 30 | - name: generate pseudo-random password for the database connection 31 | shell: python -c "exec 'import os; print os.urandom(30).encode(\'base64\')[:${length}]'" 32 | register: db_password 33 | changed_when: false 34 | 35 | - name: configure grafana 36 | template: 37 | src: ../templates/grafana.ini.j2 38 | dest: "/etc/grafana/grafana.ini" 39 | notify: 40 | - restart app 41 | sudo: true 42 | 43 | - include: postgresql.yml 44 | tags: 45 | - postgresql 46 | 47 | - include: nginx.yml 48 | 49 | - name: ensure nginx is running 50 | sudo: true 51 | service: 52 | name: nginx 53 | state: started 54 | enabled: yes 55 | 56 | - name: ensure grafana is restarted 57 | sudo: true 58 | service: 59 | name: grafana-server 60 | state: restarted 61 | enabled: yes 62 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/docker-registry/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create directory for self-signed SSL cert 3 | file: 4 | path: /var/registry/certs 5 | state: directory 6 | 7 | - name: create self-signed cfssl json file 8 | template: 9 | src: "{{ role_path }}/templates/self-csr.json.j2" 10 | dest: ./self-csr.json 11 | 12 | - name: get cfssl 13 | get_url: 14 | url: https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 15 | dest: ./cfssl 16 | mode: 0755 17 | 18 | - name: get cfssljson 19 | get_url: 20 | url: https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 21 | dest: ./cfssljson 22 | mode: 0755 23 | 24 | - name: gencert 25 | shell: ./cfssl gencert -initca self-csr.json | ./cfssljson -bare ca 26 | 27 | - name: push self-signed cfssl cert to the ansible server 28 | fetch: 29 | src: ca.pem 30 | dest: fetch/certs/self.crt 31 | flat: yes 32 | 33 | - name: mv the cert to be accessible by container 34 | command: mv ca.pem /var/registry/certs/self.crt 35 | 36 | - name: mv the key to be accessible by container 37 | command: mv ca-key.pem /var/registry/certs/self.key 38 | 39 | - name: start registry container 40 | command: docker run -d --name=docker-registry \ 41 | -p 5000:5000 \ 42 | --privileged=true \ 43 | --restart=unless-stopped \ 44 | -v /var/registry:/var/registry \ 45 | -e STORAGE_PATH=/var/registry/data \ 46 | -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/registry/certs/self.crt \ 47 | -e REGISTRY_HTTP_TLS_KEY=/var/registry/certs/self.key \ 48 | registry 49 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ensure sites-available for nginx 3 | file: 4 | path: /etc/nginx/sites-available 5 | state: directory 6 | sudo: true 7 | 8 | - name: ensure there is an nginx user 9 | user: 10 | name: nginx 11 | comment: "Nginx user" 12 | sudo: true 13 | 14 | - name: ensure sites-enable for nginx 15 | file: 16 | path: /etc/nginx/sites-enabled 17 | state: directory 18 | sudo: true 19 | 20 | - name: remove default nginx site 21 | file: 22 | path: /etc/nginx/sites-enabled/default 23 | state: absent 24 | sudo: true 25 | 26 | - name: write nginx.conf 27 | template: 28 | src: nginx.conf 29 | dest: /etc/nginx/nginx.conf 30 | sudo: true 31 | 32 | - name: enable nginx 33 | sudo: true 34 | service: 35 | name: nginx 36 | enabled: true 37 | 38 | - name: create nginx site config 39 | template: 40 | src: "nginx_site.conf" 41 | dest: "/etc/nginx/sites-available/{{ item.app_name }}.conf" 42 | sudo: true 43 | with_items: nginx_hosts 44 | notify: 45 | - restart nginx 46 | 47 | - include: ssl.yml 48 | when: development_server == true 49 | 50 | - include: letsencrypt.yml 51 | when: development_server == false 52 | 53 | - name: link nginx config 54 | file: 55 | src: "/etc/nginx/sites-available/{{ item.app_name }}.conf" 56 | dest: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 57 | state: link 58 | sudo: true 59 | with_items: nginx_hosts 60 | 61 | - name: ensure nginx is restarted 62 | sudo: true 63 | service: 64 | name: nginx 65 | state: restarted 66 | -------------------------------------------------------------------------------- /ceph-release-rpm/config/definitions/ceph-release-rpm.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-release-rpm 3 | project-type: matrix 4 | defaults: global 5 | description: Builds the repository configuration package for ceph-release. RPMS Only 6 | block-downstream: false 7 | block-upstream: false 8 | 9 | parameters: 10 | - string: 11 | name: RELEASE 12 | default: firefly 13 | 14 | - bool: 15 | name: TEST 16 | description: " 17 | If this is unchecked, then the builds will be pushed to chacra with the correct ref. This is the default. 18 | 19 | If this is checked, then the builds will be pushed to chacra under the 'test' ref." 20 | 21 | - bool: 22 | name: FORCE 23 | description: " 24 | If this is unchecked, then then nothing is built or pushed if they already exist in chacra. This is the default. 25 | 26 | If this is checked, then the binaries will be built and pushed to chacra even if they already exist in chacra." 27 | 28 | axes: 29 | - axis: 30 | type: label-expression 31 | name: ARCH 32 | values: 33 | - x86_64 34 | 35 | - axis: 36 | type: label-expression 37 | name: DIST 38 | values: 39 | - centos6 40 | - centos7 41 | 42 | builders: 43 | - shell: 44 | !include-raw: 45 | - ../../../scripts/build_utils.sh 46 | - ../../build/build 47 | 48 | wrappers: 49 | - inject-passwords: 50 | global: true 51 | mask-password-params: true 52 | -------------------------------------------------------------------------------- /ceph-build-pull-requests/config/definitions/ceph-build-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-build-pull-requests 3 | node: trusty 4 | project-type: freestyle 5 | defaults: global 6 | concurrent: true 7 | display-name: 'ceph-build: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/ceph-build 15 | logrotate: 16 | daysToKeep: 15 17 | numToKeep: 30 18 | artifactDaysToKeep: -1 19 | artifactNumToKeep: -1 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | admin-list: 29 | - alfredodeza 30 | - ktdreyer 31 | - andrewschoen 32 | org-list: 33 | - ceph 34 | trigger-phrase: '' 35 | only-trigger-phrase: false 36 | github-hooks: true 37 | permit-all: false 38 | auto-close-on-fail: false 39 | 40 | scm: 41 | - git: 42 | url: https://github.com/ceph/ceph-build.git 43 | branches: 44 | - ${sha1} 45 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 46 | browser: auto 47 | timeout: 20 48 | skip-tag: true 49 | wipe-workspace: false 50 | 51 | builders: 52 | - shell: 53 | !include-raw: 54 | - ../../../scripts/build_utils.sh 55 | - ../../build/build 56 | -------------------------------------------------------------------------------- /ceph-tag/config/definitions/ceph-tag.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: ceph-build 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/ceph-build.git 6 | browser: auto 7 | timeout: 20 8 | skip-tag: true 9 | wipe-workspace: true 10 | basedir: "ceph-build" 11 | branches: 12 | - origin/master 13 | 14 | - job: 15 | name: ceph-tag 16 | node: trusty 17 | description: "This job clones from upstream Ceph, sets the right version from the tag and pushes changes to ceph-releases" 18 | display-name: 'ceph-tag' 19 | logrotate: 20 | daysToKeep: -1 21 | numToKeep: 25 22 | artifactDaysToKeep: -1 23 | artifactNumToKeep: -1 24 | block-downstream: false 25 | block-upstream: false 26 | properties: 27 | - github: 28 | url: https://github.com/ceph/ceph 29 | 30 | parameters: 31 | - string: 32 | name: BRANCH 33 | description: "The git branch (or tag) to build" 34 | - string: 35 | name: VERSION 36 | description: "The version for release, e.g. 0.94.4" 37 | scm: 38 | - ceph-build 39 | 40 | builders: 41 | - shell: 42 | !include-raw: 43 | - ../../../scripts/build_utils.sh 44 | - ../../build/build 45 | 46 | wrappers: 47 | - inject-passwords: 48 | global: true 49 | mask-password-params: true 50 | - ssh-agent-credentials: 51 | # "jenkins-build" SSH key, needed so we can push to 52 | # ceph-deploy.git 53 | user: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 54 | -------------------------------------------------------------------------------- /ceph-deploy-tag/config/definitions/ceph-tag.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: ceph-build 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/ceph-build.git 6 | browser: auto 7 | timeout: 20 8 | skip-tag: true 9 | wipe-workspace: true 10 | basedir: "ceph-build" 11 | 12 | - job: 13 | name: ceph-deploy-tag 14 | description: "This job clones ceph-deploy and sets the right version from the tag, pushing back to ceph-deploy.git" 15 | display-name: 'ceph-deploy-tag' 16 | node: 'trusty&&small' 17 | logrotate: 18 | daysToKeep: -1 19 | numToKeep: 25 20 | artifactDaysToKeep: -1 21 | artifactNumToKeep: -1 22 | block-downstream: false 23 | block-upstream: false 24 | properties: 25 | - github: 26 | url: https://github.com/ceph/ceph-deploy 27 | 28 | parameters: 29 | - string: 30 | name: BRANCH 31 | description: "The git branch (or tag) to build" 32 | default: "master" 33 | - string: 34 | name: VERSION 35 | description: "The version for release, e.g. 1.5.30" 36 | scm: 37 | - ceph-build 38 | 39 | builders: 40 | - shell: 41 | !include-raw: 42 | - ../../../scripts/build_utils.sh 43 | - ../../build/build 44 | 45 | wrappers: 46 | - inject-passwords: 47 | global: true 48 | mask-password-params: true 49 | - ssh-agent-credentials: 50 | # "jenkins-build" SSH key, needed so we can push to 51 | # ceph-deploy.git 52 | user: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 53 | -------------------------------------------------------------------------------- /ceph-dev-trigger/config/definitions/ceph-dev-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-dev-trigger 3 | node: master 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - github: 11 | url: https://github.com/ceph/ceph 12 | discard-old-builds: true 13 | logrotate: 14 | daysToKeep: 1 15 | numToKeep: 10 16 | artifactDaysToKeep: -1 17 | artifactNumToKeep: -1 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph 25 | browser: auto 26 | # The default is to build and publish every branch. 27 | # Uncomment this for testing: 28 | branches: 29 | - 'origin/master' 30 | - 'origin/jewel' 31 | - 'origin/kraken' 32 | - 'origin/hammer' 33 | skip-tag: true 34 | timeout: 20 35 | wipe-workspace: true 36 | 37 | builders: 38 | - shell: 39 | !include-raw: 40 | - ../../../scripts/build_utils.sh 41 | - ../../build/notify 42 | - trigger-builds: 43 | - project: 'ceph-dev' 44 | predefined-parameters: | 45 | BRANCH=${GIT_BRANCH} 46 | FORCE=True 47 | - project: 'ceph-dev' 48 | predefined-parameters: | 49 | BRANCH=${GIT_BRANCH} 50 | FORCE=True 51 | FLAVOR=notcmalloc 52 | 53 | wrappers: 54 | - inject-passwords: 55 | global: true 56 | mask-password-params: true 57 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/letsencrypt.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: install system packages 4 | sudo: yes 5 | apt: 6 | name: "letsencrypt" 7 | state: present 8 | 9 | - name: ensure letsencrypt acme-challenge path 10 | file: 11 | path: "{{ ssl_webroot_base_path }}/{{ item.fqdn }}" 12 | state: "directory" 13 | mode: 0755 14 | sudo: yes 15 | with_items: nginx_hosts 16 | 17 | - name: unlink nginx configs 18 | file: 19 | path: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 20 | state: "absent" 21 | sudo: true 22 | with_items: nginx_hosts 23 | 24 | - name: create temporary nginx config 25 | template: 26 | src: "nginx_tmp_site.conf" 27 | dest: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 28 | sudo: true 29 | with_items: nginx_hosts 30 | 31 | - name: restart nginx 32 | sudo: yes 33 | service: 34 | name: nginx 35 | state: restarted 36 | 37 | - name: create (or renew) letsencrypt ssl cert 38 | command: "letsencrypt certonly --webroot -w {{ ssl_webroot_base_path }}/{{ item.fqdn }} -d {{ item.fqdn }} --email {{ ssl_support_email }} --agree-tos --renew-by-default" 39 | sudo: yes 40 | with_items: nginx_hosts 41 | 42 | - name: setup a cron to renew the SSL cert every day 43 | cron: 44 | name: "renew letsencrypt cert for {{ item.app_name }}" 45 | minute: "21" 46 | hour: "6,18" 47 | job: "letsencrypt renew --agree-tos --email {{ ssl_support_email }}" 48 | sudo: yes 49 | with_items: nginx_hosts 50 | 51 | - name: unlink tmp nginx config 52 | file: 53 | path: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 54 | state: "absent" 55 | sudo: true 56 | with_items: nginx_hosts 57 | -------------------------------------------------------------------------------- /calamari-clients-setup/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | HOST=$(hostname --short) 4 | echo "Building on ${HOST}" 5 | echo " DIST=${DIST}" 6 | echo " BPTAG=${BPTAG}" 7 | echo " WS=$WORKSPACE" 8 | echo " PWD=$(pwd)" 9 | echo " BRANCH=$BRANCH" 10 | echo " SHA1=$GIT_COMMIT" 11 | 12 | if [ -x "$BRANCH" ] ; then 13 | echo "No git branch was supplied" 14 | exit 1 15 | fi 16 | 17 | # Make sure the repo is clean 18 | echo "Cleaning up the repo" 19 | git clean -fxd 20 | 21 | # Make sure the dist directory is clean 22 | rm -rf dist build 23 | mkdir -p dist build 24 | 25 | echo "Building version $(git describe) Branch $BRANCH" 26 | 27 | # Install any setup-time deps 28 | if [ -x install-deps.sh ]; then 29 | echo "Ensuring dependencies are installed" 30 | ./install-deps.sh 31 | fi 32 | 33 | # Get the version 34 | eval $(./get-versions.sh) 35 | RPM_RELEASE=$(echo $REVISION | tr '-' '_') 36 | 37 | # Create the source tarball and move it to dist 38 | echo "Building source distribution" 39 | make DESTDIR=build/ REAL_BUILD=y VERSION=$VERSION REVISION=$REVISION RPM_REVISION=$RPM_RELEASE install 40 | tar -cvzf dist/calamari-clients-$VERSION-$RPM_RELEASE.tar.gz -C build opt 41 | 42 | # Prepare the spec file for build 43 | sed -e "s/@VERSION@/${VERSION}/g" -e "s/@RELEASE@/${RPM_RELEASE}/g" < calamari-clients.spec.in > dist/calamari-clients.spec 44 | 45 | # Save these so that we can later inject them into the build script 46 | cat > dist/sha1 << EOF 47 | SHA1=${GIT_COMMIT} 48 | EOF 49 | 50 | cat > dist/branch << EOF 51 | BRANCH=${BRANCH} 52 | EOF 53 | 54 | cat > dist/version << EOF 55 | VERSION=${VERSION} 56 | REVISION=${REVISION} 57 | RPM_RELEASE=${RPM_RELEASE} 58 | EOF 59 | -------------------------------------------------------------------------------- /jenkins-job-builder/config/definitions/jjb.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: jenkins-job-builder 3 | node: small 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'Jenkins Job Builder' 7 | concurrent: true 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/ceph-build 15 | 16 | parameters: 17 | - bool: 18 | name: FORCE 19 | default: false 20 | description: " 21 | If this is unchecked, then JJB will use its cache to update jobs. This makes this JJB job run faster, but it could cause JJB to fail to update some Jenkins jobs if the jobs have been changed outside of this JJB job's workflow. (This is the default.) 22 | 23 | If this is checked, JJB will wipe out its cache and force each job to align with the configurations in master." 24 | 25 | triggers: 26 | - github 27 | 28 | wrappers: 29 | - inject-passwords: 30 | global: true 31 | mask-password-params: true 32 | 33 | scm: 34 | - git: 35 | url: https://github.com/ceph/ceph-build 36 | branches: 37 | - master 38 | browser: auto 39 | timeout: 20 40 | 41 | builders: 42 | - shell: 43 | !include-raw: 44 | - ../../../scripts/build_utils.sh 45 | - ../../build/build 46 | 47 | publishers: 48 | - postbuildscript: 49 | builders: 50 | - shell: 'rm $HOME/.jenkins_jobs.ini' 51 | script-only-if-succeeded: false 52 | script-only-if-failed: false 53 | -------------------------------------------------------------------------------- /ceph-installer-pull-requests/config/definitions/ceph-installer-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-installer-pull-requests 3 | description: Unit-test the ceph-installer for each GitHub PR 4 | project-type: freestyle 5 | node: trusty && small 6 | block-downstream: false 7 | block-upstream: false 8 | defaults: global 9 | display-name: 'ceph-installer: Pull Requests' 10 | quiet-period: 5 11 | retry-count: 3 12 | 13 | logrotate: 14 | daysToKeep: 15 15 | numToKeep: 30 16 | artifactDaysToKeep: 15 17 | artifactNumToKeep: 15 18 | 19 | properties: 20 | - github: 21 | url: https://github.com/ceph/ceph-installer/ 22 | 23 | parameters: 24 | - string: 25 | name: sha1 26 | description: "A pull request ID, like 'origin/pr/72/head'" 27 | 28 | triggers: 29 | - github-pull-request: 30 | admin-list: 31 | - alfredodeza 32 | - dmick 33 | - ktdreyer 34 | - andrewschoen 35 | org-list: 36 | - ceph 37 | only-trigger-phrase: false 38 | github-hooks: true 39 | permit-all: false 40 | auto-close-on-fail: false 41 | 42 | scm: 43 | - git: 44 | url: https://github.com/ceph/ceph-installer 45 | branches: 46 | - ${sha1} 47 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 48 | browser: auto 49 | timeout: 20 50 | skip-tag: true 51 | wipe-workspace: true 52 | 53 | builders: 54 | - shell: 55 | !include-raw: 56 | - ../../../scripts/build_utils.sh 57 | - ../../build/build 58 | -------------------------------------------------------------------------------- /ceph-installer-docs-pull-requests/config/definitions/ceph-installer-docs-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-installer-docs-pull-requests 3 | node: small && (centos7 || trusty) 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-installer: docs pull requests' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - github: 13 | url: https://github.com/ceph/ceph-installer 14 | logrotate: 15 | daysToKeep: -1 16 | numToKeep: 10 17 | artifactDaysToKeep: -1 18 | artifactNumToKeep: -1 19 | 20 | parameters: 21 | - string: 22 | name: sha1 23 | description: "A pull request ID, like 'origin/pr/72/head'" 24 | 25 | triggers: 26 | - github-pull-request: 27 | allow-whitelist-orgs-as-admins: true 28 | org-list: 29 | - ceph 30 | trigger-phrase: '' 31 | only-trigger-phrase: false 32 | github-hooks: true 33 | permit-all: true 34 | auto-close-on-fail: false 35 | status-context: "Docs" 36 | started-status: "checking if docs build" 37 | success-status: "docs built successfully " 38 | failure-status: "docs could not build correctly" 39 | 40 | scm: 41 | - git: 42 | url: https://github.com/ceph/ceph-installer 43 | branches: 44 | - master 45 | browser: auto 46 | skip-tag: true 47 | timeout: 20 48 | 49 | builders: 50 | - shell: 51 | !include-raw: 52 | - ../../../scripts/build_utils.sh 53 | - ../../build/build 54 | -------------------------------------------------------------------------------- /ceph-pull-requests/config/definitions/ceph-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-pull-requests 3 | project-type: freestyle 4 | defaults: global 5 | concurrent: true 6 | node: huge && (centos7 || trusty) && rebootable 7 | display-name: 'ceph: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/ceph/ 15 | logrotate: 16 | daysToKeep: 15 17 | numToKeep: 300 18 | artifactDaysToKeep: -1 19 | artifactNumToKeep: -1 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | allow-whitelist-orgs-as-admins: true 29 | org-list: 30 | - ceph 31 | trigger-phrase: '' 32 | only-trigger-phrase: false 33 | github-hooks: true 34 | permit-all: true 35 | auto-close-on-fail: false 36 | 37 | scm: 38 | - git: 39 | url: https://github.com/ceph/ceph.git 40 | branches: 41 | - ${sha1} 42 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 43 | browser: auto 44 | timeout: 20 45 | skip-tag: true 46 | wipe-workspace: true 47 | 48 | builders: 49 | - shell: "export NPROC=$(nproc); timeout 7200 ./run-make-check.sh" 50 | 51 | publishers: 52 | - postbuildscript: 53 | script-only-if-succeeded: False 54 | script-only-if-failed: True 55 | builders: 56 | - shell: "sudo reboot" 57 | -------------------------------------------------------------------------------- /ceph-build-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible" "jenkins-job-builder" "urllib3" "pyopenssl" "ndg-httpsclient" "pyasn1" ) 7 | install_python_packages "pkgs[@]" 8 | 9 | 10 | # Test every definition if available in the current repository and update the jobs 11 | # if they do define one (they should always define their definitions) 12 | for dir in `find . -maxdepth 1 -path ./.git -prune -o -type d -print`; do 13 | definitions_dir="$dir/config/definitions" 14 | if [ -d "$definitions_dir" ]; then 15 | echo "found definitions directory: $definitions_dir" 16 | 17 | # Test the definitions 18 | $VENV/jenkins-jobs test $definitions_dir -o /tmp/output 19 | fi 20 | done 21 | 22 | # install ansible-galaxy roles for playbook syntax check 23 | for reqs in $WORKSPACE/ansible/requirements/*; do 24 | $VENV/ansible-galaxy install -r $reqs -p $WORKSPACE/ansible/roles --force 25 | done 26 | 27 | # To avoid moving everything into examples, including stuff that is not relevant 28 | # as an example, we copy them on the fly here 29 | cp -r $WORKSPACE/ansible/vars $WORKSPACE/ansible/examples/ 30 | cp -r $WORKSPACE/ansible/roles $WORKSPACE/ansible/examples/ 31 | cp -r $WORKSPACE/ansible/files $WORKSPACE/ansible/examples/ 32 | cp -r $WORKSPACE/ansible/library $WORKSPACE/ansible/examples/ 33 | cp -r $WORKSPACE/ansible/templates $WORKSPACE/ansible/examples/ 34 | cp $WORKSPACE/ansible/release.yml $WORKSPACE/ansible/examples/ 35 | 36 | 37 | # Syntax-check each Ansible playbook 38 | for playbook in $WORKSPACE/ansible/examples/*.yml; do 39 | $VENV/ansible-playbook -i '127.0.0.1,' $playbook --syntax-check 40 | done 41 | -------------------------------------------------------------------------------- /do_rpm_release.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | 3 | set -e 4 | trap cleanup INT EXIT 5 | 6 | default_dists="centos6" # just 1 so far 7 | subversion="1" # not used so far 8 | gpgkey='3CF7ABC8' # ceph rpm test key 9 | #rhosts="localhost 192.168.106.235" 10 | rhosts="localhost" 11 | versionfile="" 12 | 13 | cleanup() { 14 | [ -n "$pids" ] && kill $pids 15 | rm $versionfile 16 | [ -n "$vers" ] && rm -rf $releasedir/$vers 17 | } 18 | 19 | usage() { 20 | echo "usage: $0 releasedir [dists]" 21 | } 22 | 23 | bindir=`dirname $0` 24 | releasedir=$1 25 | shift || true 26 | dists="$*" 27 | 28 | [ -z "$releasedir" ] && usage && exit 1 29 | [ -z "$dists" ] && dists=$default_dists 30 | 31 | versionfile=`mktemp` 32 | rm $releasedir/* || true 33 | 34 | $bindir/release_rpm_tarball.sh $releasedir $versionfile 35 | vers=`cat $versionfile` 36 | echo "vers = $vers" 37 | 38 | cp ceph.spec $releasedir/$vers/. 39 | $bindir/build_rpms.sh $releasedir $vers $subversion $dists 40 | 41 | $bindir/sign_rpms.sh $releasedir $vers $gpgkey 42 | 43 | $bindir/gen_yum_repo.sh $releasedir/$vers $gpgkey $dists 44 | 45 | # stage the results 46 | rpmdir=$releasedir/$vers/rpmbuild 47 | for dir in $dists 48 | do 49 | distdir=$releasedir/$vers/$dir 50 | mkdir -p $distdir 51 | cp -a $rpmdir/RPMS/* $distdir/. 52 | done 53 | rm -rf $rpmdir 54 | 55 | # Notes: The debian version at this points run the build on several remote 56 | # hosts (pbuilders ?) and syncs the results back. 57 | # rsync -auv root@$rem:/tmp/release/$vers/\*.\{changes\,deb\} $releasedir/$vers 58 | # This version builds locally. We will sort out what machine we actually 59 | # build on later. 60 | 61 | rm $versionfile 62 | 63 | trap true INT EXIT 64 | 65 | exit 0 66 | -------------------------------------------------------------------------------- /ceph-pr-submodules/config/definitions/ceph-pr-commits.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-pr-submodules 3 | node: centos || trusty 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph: Pull Request modified submodules' 7 | concurrent: true 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/ceph/ 15 | logrotate: 16 | daysToKeep: 15 17 | numToKeep: 30 18 | artifactDaysToKeep: -1 19 | artifactNumToKeep: -1 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | allow-whitelist-orgs-as-admins: true 29 | org-list: 30 | - ceph 31 | trigger-phrase: '' 32 | only-trigger-phrase: false 33 | github-hooks: true 34 | permit-all: true 35 | auto-close-on-fail: false 36 | status-context: "Unmodifed Submodules" 37 | started-status: "checking if PR has modified submodules" 38 | success-status: "submodules for project are unmodified" 39 | failure-status: "Approval needed: modified submodules found" 40 | 41 | scm: 42 | - git: 43 | url: https://github.com/ceph/ceph.git 44 | branches: 45 | - ${sha1} 46 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 47 | browser: auto 48 | timeout: 20 49 | skip-tag: true 50 | wipe-workspace: false 51 | 52 | builders: 53 | - shell: 54 | !include-raw: 55 | - ../../build/build 56 | -------------------------------------------------------------------------------- /jenkins-job-builder/build/build: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # 4 | # This script uses Jenkins Job Builder to generate the configuration for its own 5 | # job so that it automatically configures all other jobs that have their YAML 6 | # definitions. 7 | # 8 | 9 | set -ex 10 | 11 | # the following two methods exist in scripts/build_utils.sh 12 | pkgs=( "jenkins-job-builder" ) 13 | install_python_packages "pkgs[@]" 14 | 15 | # Wipe out JJB's cache if $FORCE is set. 16 | [ "$FORCE" = true ] && rm -rf "$HOME/.cache/jenkins_jobs/" 17 | 18 | # workaround for https://issues.jenkins-ci.org/browse/JENKINS-16225 19 | JENKINS_URL=${JENKINS_URL:-"https://jenkins.ceph.com/"} 20 | JJB_CONFIG="$HOME/.jenkins_jobs.ini" 21 | 22 | # slap the programatically computed JJB config using env vars from Jenkins 23 | cat > $JJB_CONFIG << EOF 24 | [jenkins] 25 | user=$JOB_BUILDER_USER 26 | password=$JOB_BUILDER_PASS 27 | url=$JENKINS_URL 28 | EOF 29 | 30 | # Test every definition if available in the current repository and update the jobs 31 | # if they do define one (they should always define their definitions) 32 | for dir in `find . -maxdepth 1 -path ./.git -prune -o -type d -print`; do 33 | definitions_dir="$dir/config/definitions" 34 | if [ -d "$definitions_dir" ]; then 35 | echo "found definitions directory: $definitions_dir" 36 | 37 | # Test the definitions first 38 | $VENV/jenkins-jobs --log_level DEBUG --conf $JJB_CONFIG test $definitions_dir -o /tmp/output 39 | 40 | # Update Jenkins with the output if they passed the test phase 41 | # Note that this needs proper permissions with the right credentials to the 42 | # correct Jenkins instance. 43 | $VENV/jenkins-jobs --log_level DEBUG --conf $JJB_CONFIG update $definitions_dir 44 | fi 45 | done 46 | -------------------------------------------------------------------------------- /diamond-setup/config/definitions/diamond-setup.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: diamond-setup 3 | description: "This job step checks out the branch and builds the tarballs, diffs, and dsc that are passed to the diamond-build step.\r\n\r\nNotes:\r\nJob needs to run on a releatively recent debian system. The Restrict where run feature is used to specifiy an appropriate label.\r\nThe clear workspace before checkout box for the git plugin is used." 4 | # we do not need to pin this to trusty anymore for the new jenkins instance 5 | # FIXME: unpin when this gets ported over 6 | node: small && trusty 7 | display-name: 'diamond-setup' 8 | logrotate: 9 | daysToKeep: -1 10 | numToKeep: 25 11 | artifactDaysToKeep: -1 12 | artifactNumToKeep: -1 13 | block-downstream: false 14 | block-upstream: false 15 | concurrent: true 16 | properties: 17 | - github: 18 | url: https://github.com/ceph/Diamond 19 | 20 | parameters: 21 | - string: 22 | name: BRANCH 23 | description: "The git branch (or tag) to build" 24 | 25 | scm: 26 | - git: 27 | url: git@github.com:ceph/Diamond.git 28 | # Use the SSH key attached to the ceph-jenkins GitHub account. 29 | credentials-id: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 30 | branches: 31 | - $BRANCH 32 | skip-tag: true 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw ../../build/build 38 | 39 | publishers: 40 | - archive: 41 | artifacts: 'dist/**' 42 | allow-empty: false 43 | latest-only: false 44 | 45 | wrappers: 46 | - inject-passwords: 47 | global: true 48 | mask-password-params: true 49 | -------------------------------------------------------------------------------- /ansible/roles/ceph-deploy-release/tasks/release.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: fetch the latest from remote 4 | command: git fetch jenkins chdir=ceph-deploy 5 | 6 | - name: ensure local repo is in sync with remote 7 | command: git reset --hard jenkins/{{ branch }} chdir=ceph-deploy 8 | 9 | - name: git checkout {{ branch }} branch 10 | command: git checkout {{ branch }} chdir=ceph-deploy 11 | 12 | - name: set the debian version 13 | command: dch -v {{ version }} -D stable "New upstream release" chdir=ceph-deploy 14 | environment: 15 | DEBEMAIL: "{{ debemail }}" 16 | DEBFULLNAME: "{{ debfullname }}" 17 | 18 | - name: set the version in the spec file 19 | lineinfile: dest=ceph-deploy/ceph-deploy.spec 20 | regexp="Version{{':'}}\s+" 21 | line="Version{{':'}} {{ version }}" 22 | state=present 23 | 24 | - name: commit the version changes 25 | command: git commit -a -m "{{ version }}" chdir=ceph-deploy 26 | 27 | # from script: /srv/ceph-build/tag_release.sh 28 | # Contents of tag_release.sh 29 | # FIXME: this used to be a signed tag: 30 | # command: git tag -s "v{{ version }}" -u 17ED316D -m "v{{ version }}" chdir=ceph-deploy 31 | - name: tag and commit the version 32 | command: git tag "v{{ version }}" -m "v{{ version }}" chdir=ceph-deploy 33 | environment: 34 | GNUPGHOME: ~/build/gnupg.ceph-release 35 | DEBEMAIL: "{{ debemail }}" 36 | DEBFULLNAME: "{{ debfullname }}" 37 | 38 | - name: push changes to jenkins git repo 39 | command: git push jenkins {{ branch }} chdir=ceph-deploy 40 | 41 | # FIXME: this used to be set when signing the tag: 42 | # environment: 43 | # GNUPGHOME: ~/build/gnupg.ceph-release 44 | - name: push the newly created tag 45 | command: git push jenkins v{{ version }} chdir=ceph-deploy 46 | -------------------------------------------------------------------------------- /ansible/roles/nginx/templates/nginx_site.conf: -------------------------------------------------------------------------------- 1 | server { 2 | server_name {{ item.fqdn }}; 3 | location '/.well-known/acme-challenge' { 4 | default_type "text/plain"; 5 | root {{ ssl_webroot_base_path }}/{{ item.fqdn }}; 6 | } 7 | location / { 8 | add_header Strict-Transport-Security max-age=31536000; 9 | return 301 https://$server_name$request_uri; 10 | } 11 | } 12 | 13 | server { 14 | listen 443 ssl; 15 | server_name {{ item.fqdn }}; 16 | {% if development_server %} 17 | ssl_certificate /etc/ssl/certs/{{ item.fqdn }}-bundled.crt; 18 | ssl_certificate_key /etc/ssl/private/{{ item.fqdn }}.key; 19 | {% else %} 20 | ssl_certificate /etc/letsencrypt/live/{{ item.fqdn }}/fullchain.pem; 21 | ssl_certificate_key /etc/letsencrypt/live/{{ item.fqdn }}/privkey.pem; 22 | {% endif %} 23 | ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 24 | ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH'; 25 | ssl_prefer_server_ciphers on; 26 | add_header Strict-Transport-Security "max-age=31536000"; 27 | 28 | access_log /var/log/nginx/{{ item.app_name }}-access.log upstreamlog; 29 | error_log /var/log/nginx/{{ item.app_name }}-error.log; 30 | 31 | 32 | location / { 33 | proxy_set_header Host $host; 34 | proxy_set_header X-Real-IP $remote_addr; 35 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 36 | proxy_set_header X-Forwarded-Proto $scheme; 37 | 38 | 39 | {% if item.upstreams is defined %} 40 | proxy_pass https://{{ item.upstreams.name }}; 41 | {% elif item.proxy_pass is defined %} 42 | proxy_pass {{ item.proxy_pass }}; 43 | {% endif %} 44 | proxy_read_timeout 30; 45 | } 46 | 47 | } 48 | -------------------------------------------------------------------------------- /ceph-installer-tests/config/definitions/ceph-installer-tests.yml: -------------------------------------------------------------------------------- 1 | - project: 2 | name: ceph-installer-tests 3 | scenario: 4 | - ansible2.2-nightly_centos7 5 | - ansible2.2-nightly_xenial 6 | jobs: 7 | - 'ceph-installer-tests-{scenario}' 8 | 9 | - job-template: 10 | name: 'ceph-installer-tests-{scenario}' 11 | node: vagrant && libvirt 12 | project-type: freestyle 13 | defaults: global 14 | display-name: 'ceph-installer: Tests [{scenario}]' 15 | quiet-period: 5 16 | block-downstream: false 17 | block-upstream: false 18 | retry-count: 3 19 | properties: 20 | - github: 21 | url: https://github.com/ceph/ceph-installer 22 | logrotate: 23 | daysToKeep: -1 24 | numToKeep: -1 25 | artifactDaysToKeep: -1 26 | artifactNumToKeep: -1 27 | 28 | parameters: 29 | - string: 30 | name: INSTALLER_BRANCH 31 | description: "The ceph-installer branch (or tag) to test" 32 | default: master 33 | 34 | - string: 35 | name: CEPH_ANSIBLE_BRANCH 36 | description: "The ceph-ansible branch (or tag) to test" 37 | default: master 38 | 39 | triggers: 40 | - timed: '@daily' 41 | 42 | scm: 43 | - git: 44 | url: https://github.com/ceph/ceph-installer.git 45 | branches: 46 | - $INSTALLER_BRANCH 47 | browser: auto 48 | timeout: 20 49 | 50 | builders: 51 | - inject: 52 | properties-content: | 53 | SCENARIO={scenario} 54 | - shell: 55 | !include-raw-escape: 56 | - ../../../scripts/build_utils.sh 57 | - ../../build/build 58 | 59 | publishers: 60 | - email: 61 | recipients: aschoen@redhat.com adeza@redhat.com 62 | -------------------------------------------------------------------------------- /ansible/examples/sensu.yml: -------------------------------------------------------------------------------- 1 | # This playbook requires that you install the roles defined 2 | # in ./requirements/sensu-requirements.yml. Do this by running: 3 | # 4 | # ansible-galaxy install -r requirements/sensu-requirements.yml 5 | # 6 | - hosts: sensu-server 7 | sudo: true 8 | vars_files: 9 | - vars/sensu-vars.yml 10 | roles: 11 | - role: Mayeu.RabbitMQ 12 | rabbitmq_vhost_definitions: 13 | - name: "/sensu" 14 | rabbitmq_users_definitions: 15 | - vhost: "/sensu" 16 | user: sensu 17 | password: secret 18 | configure_priv: ".*" 19 | read_priv: ".*" 20 | write_priv: ".*" 21 | rabbitmq_ssl: false 22 | - redis 23 | - role: Mayeu.sensu 24 | sensu_server_rabbitmq_port: 5672 25 | sensu_server_api_password: secret 26 | sensu_server_rabbitmq_password: secret 27 | sensu_server_dashboard_password: secret 28 | sensu_client_subscription_names: 29 | - common 30 | - rabbitmq 31 | # these are custom settings for this client 32 | sensu_settings: 33 | # used in the rabbitmq-alive check 34 | rabbitmq: 35 | user: sensu 36 | password: secret 37 | # we need to escape the / in /sensu 38 | vhost: "%2Fsensu" 39 | 40 | - hosts: sensu-clients 41 | sudo: true 42 | vars_files: 43 | - vars/sensu-vars.yml 44 | roles: 45 | - role: Mayeu.sensu 46 | sensu_server_rabbitmq_hostname: "{{ rabbitmq_client_address|mandatory }}" 47 | sensu_server_rabbitmq_port: 5672 48 | sensu_server_rabbitmq_password: secret 49 | sensu_install_server: false 50 | sensu_install_uchiwa: false 51 | # sensu_client_subscription_names need to be defined in a hosts_var file 52 | # relative to your inventory 53 | -------------------------------------------------------------------------------- /update_pbuilder.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | 3 | set -e 4 | 5 | usage() { 6 | echo "usage: $0 basedir [dists...]" 7 | } 8 | 9 | bindir=`dirname $0` 10 | echo "$bindir" | grep -v -q '^/' && bindir=`pwd`"/$bindir" 11 | 12 | basedir=$1 13 | shift 14 | dists=$* 15 | 16 | [ ! -d "$basedir" ] && echo specify dir for pbuilder images && usage && exit 1 17 | [ -z "$dists" ] && dists=`cat $bindir/deb_dists` 18 | 19 | for dist in $dists 20 | do 21 | os="debian" 22 | [ "$dist" = "raring" ] && os="ubuntu" 23 | [ "$dist" = "saucy" ] && os="ubuntu" 24 | [ "$dist" = "xenial" ] && os="ubuntu" 25 | [ "$dist" = "trusty" ] && os="ubuntu" 26 | [ "$dist" = "precise" ] && os="ubuntu" 27 | [ "$dist" = "oneiric" ] && os="ubuntu" 28 | [ "$dist" = "natty" ] && os="ubuntu" 29 | [ "$dist" = "maverick" ] && os="ubuntu" 30 | [ "$dist" = "lucid" ] && os="ubuntu" 31 | 32 | if [ $os = "debian" ]; then 33 | mirror="http://apt-mirror.sepia.ceph.com/ftp.us.debian.org/debian" 34 | othermirror="" 35 | else 36 | mirror="" 37 | othermirror="deb http://apt-mirror.sepia.ceph.com/archive.ubuntu.com/ubuntu $dist main restricted universe multiverse" 38 | fi 39 | 40 | pbuilder --clean 41 | 42 | if [ -e $basedir/$dist.tgz ]; then 43 | echo updating $dist base.tgz 44 | # savelog -l -n $basedir/$dist.tgz 45 | # cp $basedir/$dist.tgz.0 $basedir/$dist.tgz 46 | pbuilder update \ 47 | --basetgz $basedir/$dist.tgz \ 48 | --distribution $dist \ 49 | --mirror "$mirror" \ 50 | --othermirror "$othermirror" 51 | else 52 | echo building $dist base.tgz 53 | pbuilder create \ 54 | --basetgz $basedir/$dist.tgz \ 55 | --distribution $dist \ 56 | --mirror "$mirror" \ 57 | --othermirror "$othermirror" 58 | fi 59 | done 60 | -------------------------------------------------------------------------------- /diamond-setup/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | ## Get the basic setup/info 4 | HOST=$(hostname --short) 5 | echo "Building on ${HOST}" 6 | echo " DIST=${DIST}" 7 | echo " BPTAG=${BPTAG}" 8 | echo " WS=$WORKSPACE" 9 | echo " PWD=$(pwd)" 10 | echo " BRANCH=$BRANCH" 11 | echo " SHA1=$GIT_COMMIT" 12 | 13 | if [ -x "$BRANCH" ] ; then 14 | echo "No git branch was supplied" 15 | exit 1 16 | fi 17 | 18 | echo "Building version $(git describe) Branch $BRANCH" 19 | 20 | ## Make sure the repo is clean 21 | # Remove all untracked files 22 | echo "Cleaning up the repo" 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | ## Install any setup-time deps 30 | # We need this for mk-build-deps 31 | sudo apt-get install equivs 32 | # Run the install-deps.sh upstream script if it exists 33 | if [ -x install-deps.sh ]; then 34 | echo "Ensuring dependencies are installed" 35 | ./install-deps.sh 36 | fi 37 | 38 | ## Get the version 39 | VERSION=$(./version.sh) 40 | 41 | ## Build the source tarball 42 | echo "Building source distribution" 43 | python setup.py sdist 44 | 45 | ## Prepare the spec file for build 46 | sed -e "s/@VERSION@/${VERSION}/g" < diamond.spec.in > dist/diamond.spec 47 | 48 | ## Prepare the debian files 49 | # Bump the changelog 50 | dch -v "$VERSION" "New release ($VERSION)" 51 | 52 | # Install build-time dependencies 53 | yes | sudo mk-build-deps --install debian/control 54 | 55 | # Create .dsc and source tarball 56 | sudo dpkg-buildpackage -S -us -uc 57 | 58 | cp ../diamond_$VERSION* dist/ 59 | 60 | ## Save these so that we can later inject them into the build script 61 | cat > dist/sha1 << EOF 62 | SHA1=${GIT_COMMIT} 63 | EOF 64 | 65 | cat > dist/branch << EOF 66 | BRANCH=${BRANCH} 67 | EOF 68 | 69 | cat > dist/version << EOF 70 | VERSION=${VERSION} 71 | EOF 72 | -------------------------------------------------------------------------------- /calamari-clients-setup/config/definitions/calamari-clients-setup.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: calamari-clients-setup 3 | description: "This job step checks out the branch and builds the tarballs, diffs, and dsc that are passed to the calamari-clients-build step.\r\n\r\nNotes:\r\nJob needs to run on a releatively recent debian system. The Restrict where run feature is used to specifiy an appropriate label.\r\nThe clear workspace before checkout box for the git plugin is used." 4 | # we do not need to pin this to trusty anymore for the new jenkins instance 5 | # FIXME: unpin when this gets ported over 6 | node: small && centos7 7 | display-name: 'calamari-clients-setup' 8 | logrotate: 9 | daysToKeep: -1 10 | numToKeep: 25 11 | artifactDaysToKeep: -1 12 | artifactNumToKeep: -1 13 | block-downstream: false 14 | block-upstream: false 15 | concurrent: true 16 | properties: 17 | - github: 18 | url: https://github.com/ceph/calamari-clients 19 | 20 | parameters: 21 | - string: 22 | name: BRANCH 23 | description: "The git branch (or tag) to build" 24 | 25 | scm: 26 | - git: 27 | url: git@github.com:ceph/calamari-clients.git 28 | # Use the SSH key attached to the ceph-jenkins GitHub account. 29 | credentials-id: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 30 | branches: 31 | - $BRANCH 32 | skip-tag: true 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw ../../build/build 38 | 39 | publishers: 40 | - archive: 41 | artifacts: 'dist/**' 42 | allow-empty: false 43 | latest-only: false 44 | 45 | wrappers: 46 | - inject-passwords: 47 | global: true 48 | mask-password-params: true 49 | -------------------------------------------------------------------------------- /merfi-pull-requests/config/definitions/merfi-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: merfi-pull-requests 3 | project-type: freestyle 4 | defaults: global 5 | display-name: 'Merfi: Pull Requests' 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | retry-count: 3 10 | properties: 11 | - github: 12 | url: https://github.com/alfredodeza/merfi/ 13 | logrotate: 14 | daysToKeep: 15 15 | numToKeep: 30 16 | artifactDaysToKeep: -1 17 | artifactNumToKeep: -1 18 | 19 | parameters: 20 | - string: 21 | name: sha1 22 | description: "A pull request ID, like 'origin/pr/72/head'" 23 | 24 | triggers: 25 | - github-pull-request: 26 | admin-list: 27 | - alfredodeza 28 | - ktdreyer 29 | - zmc 30 | - andrewschoen 31 | - dmick 32 | org-list: 33 | - ceph 34 | white-list: 35 | - jcsp 36 | - gregsfortytwo 37 | - GregMeno 38 | - dzafman 39 | - dillaman 40 | - dachary 41 | - liewegas 42 | - idryomov 43 | - vasukulkarni 44 | only-trigger-phrase: false 45 | github-hooks: true 46 | permit-all: false 47 | auto-close-on-fail: false 48 | 49 | scm: 50 | - git: 51 | url: https://github.com/alfredodeza/merfi.git 52 | branches: 53 | - ${sha1} 54 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 55 | browser: auto 56 | timeout: 20 57 | skip-tag: true 58 | wipe-workspace: true 59 | 60 | builders: 61 | - shell: 62 | !include-raw: 63 | - ../../../scripts/build_utils.sh 64 | - ../../build/build 65 | -------------------------------------------------------------------------------- /ceph-qa-suite-pull-requests/config/definitions/ceph-qa-suite-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-qa-suite-pull-requests 3 | node: trusty 4 | project-type: freestyle 5 | defaults: global 6 | concurrent: true 7 | display-name: 'ceph-qa-suite: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/ceph-qa-suite/ 15 | logrotate: 16 | daysToKeep: 15 17 | numToKeep: 30 18 | artifactDaysToKeep: -1 19 | artifactNumToKeep: -1 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | admin-list: 29 | - alfredodeza 30 | - ktdreyer 31 | - trhoden 32 | org-list: 33 | - ceph 34 | white-list: 35 | - jcsp 36 | - gregsfortytwo 37 | - dmick 38 | - GregMeno 39 | - dzafman 40 | - dillaman 41 | - dachary 42 | - liewegas 43 | trigger-phrase: '' 44 | only-trigger-phrase: false 45 | github-hooks: true 46 | permit-all: false 47 | auto-close-on-fail: false 48 | 49 | scm: 50 | - git: 51 | url: https://github.com/ceph/ceph-qa-suite.git 52 | branches: 53 | - ${sha1} 54 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 55 | browser: auto 56 | timeout: 20 57 | skip-tag: true 58 | wipe-workspace: false 59 | 60 | builders: 61 | - shell: 62 | !include-raw: 63 | - ../../../scripts/build_utils.sh 64 | - ../../build/build 65 | -------------------------------------------------------------------------------- /release_tarball.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | set -e 4 | 5 | usage() { 6 | echo "usage: $0 releasedir [versionfile]" 7 | } 8 | 9 | releasedir=$1 10 | versionfile=$2 11 | 12 | [ -z "$releasedir" ] && echo specify releasedir && usage && exit 1 13 | 14 | if git diff --quiet ; then 15 | echo repository is clean 16 | else 17 | echo 18 | echo "**** REPOSITORY IS DIRTY ****" 19 | echo 20 | if [ "$force" != "force" ]; then 21 | echo "add 'force' argument if you really want to continue." 22 | exit 1 23 | fi 24 | echo "forcing." 25 | fi 26 | 27 | cephver=`git describe --match "v*" | sed s/^v//` 28 | echo current version $cephver 29 | 30 | srcdir=`pwd` 31 | 32 | if [ -d "$releasedir/$cephver" ]; then 33 | echo "$releasedir/$cephver already exists; reuse that release tarball" 34 | else 35 | echo building tarball 36 | rm ceph-*.tar.gz || true 37 | rm ceph-*.tar.bz2 || true 38 | 39 | if [ -x make-dist ]; then 40 | ./make-dist 41 | else 42 | make dist-bzip2 43 | fi 44 | 45 | vers=`ls ceph-*.tar.bz2 | cut -c 6- | sed 's/.tar.bz2//'` 46 | echo tarball vers $vers 47 | 48 | echo extracting 49 | mkdir -p $releasedir/$cephver/rpm 50 | cp rpm/*.patch $releasedir/$cephver/rpm || true 51 | cd $releasedir/$cephver 52 | 53 | tar jxf $srcdir/ceph-$vers.tar.bz2 54 | [ "$vers" != "$cephver" ] && mv ceph-$vers ceph-$cephver 55 | 56 | tar zcf ceph_$cephver.orig.tar.gz ceph-$cephver 57 | cp -a ceph_$cephver.orig.tar.gz ceph-$cephver.tar.gz 58 | 59 | tar jcf ceph-$cephver.tar.bz2 ceph-$cephver 60 | 61 | # copy debian dir, too 62 | cp -a $srcdir/debian debian 63 | cd $srcdir 64 | 65 | # copy in spec file, too 66 | cp ceph.spec $releasedir/$cephver 67 | fi 68 | 69 | if [ -n "$versionfile" ]; then 70 | echo $cephver > $versionfile 71 | echo "wrote $cephver to $versionfile" 72 | fi 73 | 74 | echo "done." 75 | -------------------------------------------------------------------------------- /diamond-build/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | HOST=$(hostname --short) 16 | echo "Building on $(hostname)" 17 | echo " DIST=${DIST}" 18 | echo " BPTAG=${BPTAG}" 19 | echo " KEYID=${KEYID}" 20 | echo " WS=$WORKSPACE" 21 | echo " PWD=$(pwd)" 22 | echo " BUILD SOURCE=$COPYARTIFACT_BUILD_NUMBER_CEPH_SETUP" 23 | echo "*****" 24 | env 25 | echo "*****" 26 | 27 | if test $(id -u) != 0 ; then 28 | SUDO=sudo 29 | fi 30 | export LC_ALL=C # the following is vulnerable to i18n 31 | 32 | if test -f /etc/redhat-release ; then 33 | $SUDO yum install -y redhat-lsb-core 34 | fi 35 | 36 | if which apt-get > /dev/null ; then 37 | $SUDO apt-get install -y lsb-release 38 | fi 39 | 40 | case $(lsb_release -si) in 41 | CentOS|Fedora|SUSE*|RedHatEnterpriseServer) 42 | case $(lsb_release -si) in 43 | SUSE*) 44 | $SUDO zypper -y yum-utils 45 | ;; 46 | *) 47 | $SUDO yum install -y yum-utils mock 48 | ;; 49 | esac 50 | ;; 51 | *) 52 | echo "$(lsb_release -si) is unknown, dependencies will have to be installed manually." 53 | ;; 54 | esac 55 | 56 | pkgs=( "chacractl>=0.0.4" ) 57 | install_python_packages "pkgs[@]" 58 | 59 | # ask shaman which chacra instance to use 60 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 61 | # create the .chacractl config file using global variables 62 | make_chacractl_config $chacra_url 63 | -------------------------------------------------------------------------------- /calamari-clients-build/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | HOST=$(hostname --short) 16 | echo "Building on $(hostname)" 17 | echo " DIST=${DIST}" 18 | echo " BPTAG=${BPTAG}" 19 | echo " KEYID=${KEYID}" 20 | echo " WS=$WORKSPACE" 21 | echo " PWD=$(pwd)" 22 | echo " BUILD SOURCE=$COPYARTIFACT_BUILD_NUMBER_CEPH_SETUP" 23 | echo "*****" 24 | env 25 | echo "*****" 26 | 27 | if test $(id -u) != 0 ; then 28 | SUDO=sudo 29 | fi 30 | export LC_ALL=C # the following is vulnerable to i18n 31 | 32 | if test -f /etc/redhat-release ; then 33 | $SUDO yum install -y redhat-lsb-core 34 | fi 35 | 36 | if which apt-get > /dev/null ; then 37 | $SUDO apt-get install -y lsb-release 38 | fi 39 | 40 | case $(lsb_release -si) in 41 | CentOS|Fedora|SUSE*|RedHatEnterpriseServer) 42 | case $(lsb_release -si) in 43 | SUSE*) 44 | $SUDO zypper -y yum-utils 45 | ;; 46 | *) 47 | $SUDO yum install -y yum-utils mock 48 | ;; 49 | esac 50 | ;; 51 | *) 52 | echo "$(lsb_release -si) is unknown, dependencies will have to be installed manually." 53 | ;; 54 | esac 55 | 56 | pkgs=( "chacractl>=0.0.4" ) 57 | install_python_packages "pkgs[@]" 58 | 59 | # ask shaman which chacra instance to use 60 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 61 | # create the .chacractl config file using global variables 62 | make_chacractl_config $chacra_url 63 | -------------------------------------------------------------------------------- /ceph-docker-nightly/config/definitions/ceph-docker-nightly.yml: -------------------------------------------------------------------------------- 1 | - project: 2 | name: ceph-docker-nightly 3 | scenario: 4 | - ceph_ansible2.1-jewel-centos7_cluster 5 | - ceph_ansible2.1-jewel-xenial_cluster 6 | - ceph_ansible-jewel-centos7_cluster 7 | - ceph_ansible-jewel-xenial_cluster 8 | - ceph_ansible-kraken-centos7_cluster 9 | - ceph_ansible-kraken-xenial_cluster 10 | jobs: 11 | - 'ceph-docker-nightly-{scenario}' 12 | 13 | 14 | - job-template: 15 | name: 'ceph-docker-nightly-{scenario}' 16 | node: vagrant&&libvirt 17 | concurrent: true 18 | defaults: global 19 | display-name: 'ceph-docker: Nightly tests [{scenario}]' 20 | quiet-period: 5 21 | block-downstream: false 22 | block-upstream: false 23 | retry-count: 3 24 | properties: 25 | - github: 26 | url: https://github.com/ceph/ceph-docker 27 | logrotate: 28 | daysToKeep: 15 29 | numToKeep: 30 30 | artifactDaysToKeep: -1 31 | artifactNumToKeep: -1 32 | 33 | parameters: 34 | - string: 35 | name: BRANCH 36 | description: "A ceph-docker branch to test" 37 | default: master 38 | 39 | triggers: 40 | - timed: '@daily' 41 | 42 | scm: 43 | - git: 44 | url: https://github.com/ceph/ceph-docker.git 45 | branches: 46 | - ${{BRANCH}} 47 | browser: auto 48 | timeout: 20 49 | 50 | builders: 51 | - inject: 52 | properties-content: | 53 | SCENARIO={scenario} 54 | - shell: 55 | !include-raw-escape: 56 | - ../../../scripts/build_utils.sh 57 | - ../../build/build 58 | 59 | publishers: 60 | - postbuildscript: 61 | script-only-if-succeeded: False 62 | script-only-if-failed: True 63 | builders: 64 | - shell: !include-raw ../../build/teardown 65 | -------------------------------------------------------------------------------- /ceph-ansible-pr-syntax-check/config/definitions/ceph-ansible-pr-syntax-check.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: ceph-ansible 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/ceph-ansible.git 6 | branches: 7 | - ${sha1} 8 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 9 | browser: auto 10 | timeout: 20 11 | skip-tag: true 12 | wipe-workspace: false 13 | basedir: "ceph-ansible" 14 | 15 | - job: 16 | name: ceph-ansible-pr-syntax-check 17 | node: (centos7 || trusty) && x86_64 18 | project-type: freestyle 19 | defaults: global 20 | concurrent: true 21 | display-name: 'ceph-ansible: Pull Requests Syntax Check' 22 | quiet-period: 5 23 | block-downstream: false 24 | block-upstream: false 25 | retry-count: 3 26 | properties: 27 | - github: 28 | url: https://github.com/ceph/ceph-ansible 29 | logrotate: 30 | daysToKeep: 15 31 | numToKeep: 30 32 | artifactDaysToKeep: -1 33 | artifactNumToKeep: -1 34 | 35 | parameters: 36 | - string: 37 | name: sha1 38 | description: "A pull request ID, like 'origin/pr/72/head'" 39 | 40 | triggers: 41 | - github-pull-request: 42 | allow-whitelist-orgs-as-admins: true 43 | org-list: 44 | - ceph 45 | trigger-phrase: 'jenkins test playbook syntax' 46 | only-trigger-phrase: false 47 | github-hooks: true 48 | permit-all: true 49 | auto-close-on-fail: false 50 | status-context: "Testing: playbook syntax" 51 | started-status: "Running syntax checks on all playbooks" 52 | success-status: "OK - syntax checks completed" 53 | failure-status: "Syntax check failed with errors" 54 | 55 | scm: 56 | - ceph-ansible 57 | 58 | builders: 59 | - shell: 60 | !include-raw: 61 | - ../../../scripts/build_utils.sh 62 | - ../../build/build 63 | -------------------------------------------------------------------------------- /kernel/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | HOST=$(hostname --short) 16 | echo "Building on $(hostname)" 17 | echo " DIST=${DIST}" 18 | echo " BPTAG=${BPTAG}" 19 | echo " KEYID=${KEYID}" 20 | echo " WS=$WORKSPACE" 21 | echo " PWD=$(pwd)" 22 | echo " BUILD SOURCE=$COPYARTIFACT_BUILD_NUMBER_CEPH_SETUP" 23 | echo "*****" 24 | env 25 | echo "*****" 26 | 27 | if test $(id -u) != 0 ; then 28 | SUDO=sudo 29 | fi 30 | export LC_ALL=C # the following is vulnerable to i18n 31 | 32 | if test -f /etc/redhat-release ; then 33 | $SUDO yum install -y redhat-lsb-core 34 | fi 35 | 36 | if which apt-get > /dev/null ; then 37 | $SUDO apt-get install -y lsb-release 38 | fi 39 | 40 | case $(lsb_release -si) in 41 | CentOS|Fedora|SUSE*|RedHatEnterpriseServer) 42 | case $(lsb_release -si) in 43 | SUSE*) 44 | $SUDO zypper -y yum-utils 45 | ;; 46 | *) 47 | $SUDO yum install -y yum-utils mock 48 | ;; 49 | esac 50 | ;; 51 | *) 52 | echo "$(lsb_release -si) is unknown, dependencies will have to be installed manually." 53 | ;; 54 | esac 55 | 56 | pkgs=( "chacractl>=0.0.4" ) 57 | install_python_packages "pkgs[@]" 58 | 59 | # ask shaman which chacra instance to use 60 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 61 | make_chacractl_config $chacra_url 62 | 63 | BRANCH=$(branch_slash_filter $BRANCH) 64 | 65 | # Make sure we execute at the top level directory 66 | cd "$WORKSPACE" 67 | 68 | # Clean the git repo 69 | git clean -fxd 70 | -------------------------------------------------------------------------------- /teuthology-pull-requests/config/definitions/teuthology-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: teuthology-pull-requests 3 | node: trusty && huge 4 | project-type: freestyle 5 | defaults: global 6 | concurrent: true 7 | display-name: 'Teuthology: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - github: 14 | url: https://github.com/ceph/teuthology/ 15 | logrotate: 16 | daysToKeep: 15 17 | numToKeep: 30 18 | artifactDaysToKeep: -1 19 | artifactNumToKeep: -1 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | admin-list: 29 | - alfredodeza 30 | - ktdreyer 31 | - trhoden 32 | - zmc 33 | - andrewschoen 34 | - dmick 35 | org-list: 36 | - ceph 37 | white-list: 38 | - jcsp 39 | - gregsfortytwo 40 | - GregMeno 41 | - dzafman 42 | - dillaman 43 | - dachary 44 | - liewegas 45 | - idryomov 46 | - vasukulkarni 47 | - ivotron 48 | only-trigger-phrase: false 49 | github-hooks: true 50 | permit-all: false 51 | auto-close-on-fail: false 52 | 53 | scm: 54 | - git: 55 | url: https://github.com/ceph/teuthology.git 56 | branches: 57 | - ${sha1} 58 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 59 | browser: auto 60 | timeout: 20 61 | skip-tag: true 62 | wipe-workspace: true 63 | 64 | builders: 65 | - shell: 66 | !include-raw: 67 | - ../../../scripts/build_utils.sh 68 | - ../../setup/setup 69 | - shell: 70 | !include-raw: 71 | - ../../../scripts/build_utils.sh 72 | - ../../build/build 73 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | require 'yaml' 5 | VAGRANTFILE_API_VERSION = '2' 6 | 7 | config_file=File.expand_path(File.join(File.dirname(__FILE__), 'vagrant_variables.yml')) 8 | settings=YAML.load_file(config_file) 9 | 10 | BOX = settings['vagrant_box'] 11 | SYNC_DIR = settings['vagrant_sync_dir'] 12 | MEMORY = settings['memory'] 13 | TEST_CLIENT_VM = settings['provision_test_client_vm'] 14 | 15 | ansible_provision = proc do |ansible| 16 | ansible.playbook = 'site.yml' 17 | ansible.groups = { 18 | "registry" => ["docker-registry"] 19 | } 20 | if TEST_CLIENT_VM then 21 | ansible.groups['testclient'] = "docker-reg-test" 22 | end 23 | 24 | ansible.limit = 'all' 25 | end 26 | 27 | Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| 28 | config.vm.box = BOX 29 | config.ssh.insert_key = false # workaround for https://github.com/mitchellh/vagrant/issues/5048 30 | 31 | # Faster bootup. Disable if you need this for libvirt 32 | config.vm.provider :libvirt do |v,override| 33 | override.vm.synced_folder '.', SYNC_DIR, disabled: true 34 | end 35 | 36 | if TEST_CLIENT_VM then 37 | config.vm.define "docker-reg-test" do |regtest| 38 | regtest.vm.hostname = "docker-reg-test" 39 | end 40 | end 41 | 42 | config.vm.define "docker-registry" do |registry| 43 | registry.vm.hostname = "docker-registry" 44 | # Virtualbox 45 | registry.vm.provider :virtualbox do |vb| 46 | vb.customize ['modifyvm', :id, '--memory', "#{MEMORY}"] 47 | end 48 | 49 | # VMware 50 | registry.vm.provider :vmware_fusion do |v| 51 | v.vmx['memsize'] = "#{MEMORY}" 52 | end 53 | 54 | # Libvirt 55 | registry.vm.provider :libvirt do |lv| 56 | lv.memory = MEMORY 57 | end 58 | 59 | # Parallels 60 | registry.vm.provider "parallels" do |prl| 61 | prl.name = "docker-registry" 62 | prl.memory = "#{MEMORY}" 63 | end 64 | 65 | # Run the provisioner after the machine comes up 66 | registry.vm.provision 'ansible', &ansible_provision 67 | end 68 | end 69 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ensure database service is up 3 | service: 4 | name: postgresql 5 | state: started 6 | enabled: yes 7 | sudo: yes 8 | 9 | - name: allow users to connect locally 10 | sudo: yes 11 | lineinfile: 12 | # TODO: should not hardcode that version 13 | dest: /etc/postgresql/9.5/main/pg_hba.conf 14 | regexp: '^host\s+all\s+all\s+127.0.0.1/32' 15 | line: 'host all all 127.0.0.1/32 md5' 16 | backrefs: yes 17 | register: pg_hba_conf 18 | 19 | - service: 20 | name: postgresql 21 | state: restarted 22 | sudo: true 23 | when: pg_hba_conf.changed 24 | 25 | - name: generate pseudo-random password for the database connection 26 | shell: python -c "exec 'import os; print os.urandom(30).encode(\'base64\')[:${length}]'" 27 | register: db_password 28 | changed_when: false 29 | 30 | - name: make {{ app_name }} user 31 | postgresql_user: 32 | name: "{{ app_name }}" 33 | password: "{{ db_password.stdout }}" 34 | role_attr_flags: SUPERUSER 35 | login_user: postgres 36 | become_user: postgres 37 | become: yes 38 | 39 | - name: Make {{ app_name }} database 40 | postgresql_db: 41 | name: "{{ app_name }}" 42 | owner: "{{ app_name }}" 43 | state: present 44 | login_user: postgres 45 | sudo_user: postgres 46 | sudo: yes 47 | 48 | - name: ensure database service is up 49 | service: 50 | name: postgresql 51 | state: started 52 | enabled: yes 53 | sudo: yes 54 | 55 | - name: create the config file with the db password 56 | template: 57 | src: ../templates/local_settings.py.j2 58 | dest: "/etc/graphite/local_settings.py" 59 | notify: 60 | - restart app 61 | sudo: true 62 | 63 | # there is a bug where if you don't migrate auth first only it will fail 64 | # with "ProgrammingError: relation "auth_user" does not exist" 65 | - name: run migrate for auth first 66 | command: graphite-manage migrate --noinput auth 67 | sudo: true 68 | 69 | - name: run migrate to ensure database schema 70 | command: graphite-manage migrate --noinput 71 | sudo: true 72 | -------------------------------------------------------------------------------- /chacra-pull-requests/config/definitions/chacra-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: chacra 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/chacra 6 | branches: 7 | - ${sha1} 8 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 9 | browser: auto 10 | timeout: 20 11 | basedir: "chacra" 12 | skip-tag: true 13 | wipe-workspace: true 14 | 15 | - scm: 16 | name: ceph-build 17 | scm: 18 | - git: 19 | url: https://github.com/ceph/ceph-build.git 20 | browser-url: https://github.com/ceph/ceph-build 21 | timeout: 20 22 | skip-tag: true 23 | wipe-workspace: false 24 | basedir: "ceph-build" 25 | branches: 26 | - origin/master 27 | 28 | 29 | - job: 30 | name: chacra-pull-requests 31 | description: Runs tox tests for chacra on each GitHub PR 32 | project-type: freestyle 33 | node: trusty && small 34 | block-downstream: false 35 | block-upstream: false 36 | defaults: global 37 | display-name: 'chacra: Pull Requests' 38 | quiet-period: 5 39 | retry-count: 3 40 | 41 | logrotate: 42 | daysToKeep: 15 43 | numToKeep: 30 44 | artifactDaysToKeep: 15 45 | artifactNumToKeep: 15 46 | 47 | properties: 48 | - github: 49 | url: https://github.com/ceph/chacra/ 50 | 51 | parameters: 52 | - string: 53 | name: sha1 54 | description: "A pull request ID, like 'origin/pr/72/head'" 55 | 56 | triggers: 57 | - github-pull-request: 58 | admin-list: 59 | - alfredodeza 60 | - dmick 61 | - ktdreyer 62 | - andrewschoen 63 | org-list: 64 | - ceph 65 | only-trigger-phrase: false 66 | github-hooks: true 67 | permit-all: false 68 | auto-close-on-fail: false 69 | 70 | scm: 71 | - chacra 72 | - ceph-build 73 | 74 | builders: 75 | - shell: 76 | !include-raw: 77 | - ../../../scripts/build_utils.sh 78 | - ../../build/build 79 | -------------------------------------------------------------------------------- /shaman-pull-requests/config/definitions/shaman-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: shaman 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/shaman 6 | branches: 7 | - ${sha1} 8 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 9 | browser: auto 10 | timeout: 20 11 | basedir: "shaman" 12 | skip-tag: true 13 | wipe-workspace: true 14 | 15 | - scm: 16 | name: ceph-build 17 | scm: 18 | - git: 19 | url: https://github.com/ceph/ceph-build.git 20 | browser-url: https://github.com/ceph/ceph-build 21 | timeout: 20 22 | skip-tag: true 23 | wipe-workspace: false 24 | basedir: "ceph-build" 25 | branches: 26 | - origin/master 27 | 28 | 29 | - job: 30 | name: shaman-pull-requests 31 | description: Runs tox tests for shaman on each GitHub PR 32 | project-type: freestyle 33 | node: trusty && small 34 | block-downstream: false 35 | block-upstream: false 36 | defaults: global 37 | display-name: 'shaman: Pull Requests' 38 | quiet-period: 5 39 | retry-count: 3 40 | 41 | logrotate: 42 | daysToKeep: 15 43 | numToKeep: 30 44 | artifactDaysToKeep: 15 45 | artifactNumToKeep: 15 46 | 47 | properties: 48 | - github: 49 | url: https://github.com/ceph/shaman/ 50 | 51 | parameters: 52 | - string: 53 | name: sha1 54 | description: "A pull request ID, like 'origin/pr/72/head'" 55 | 56 | triggers: 57 | - github-pull-request: 58 | admin-list: 59 | - alfredodeza 60 | - dmick 61 | - ktdreyer 62 | - andrewschoen 63 | org-list: 64 | - ceph 65 | only-trigger-phrase: false 66 | github-hooks: true 67 | permit-all: false 68 | auto-close-on-fail: false 69 | 70 | scm: 71 | - shaman 72 | - ceph-build 73 | 74 | builders: 75 | - shell: 76 | !include-raw: 77 | - ../../../scripts/build_utils.sh 78 | - ../../build/build 79 | -------------------------------------------------------------------------------- /ceph-dev-setup/config/definitions/ceph-dev-setup.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-dev-setup 3 | description: "This job step checks out the branch and builds the tarballs, diffs, and dsc that are passed to the ceph-dev-build step.\r\n\r\nNotes:\r\nJob needs to run on a releatively recent debian system. The Restrict where run feature is used to specifiy an appropriate label.\r\nThe clear workspace before checkout box for the git plugin is used." 4 | # we do not need to pin this to trusty anymore for the new jenkins instance 5 | # FIXME: unpin when this gets ported over 6 | node: huge && trusty && x86_64 7 | display-name: 'ceph-dev-setup' 8 | logrotate: 9 | daysToKeep: -1 10 | numToKeep: 25 11 | artifactDaysToKeep: -1 12 | artifactNumToKeep: -1 13 | block-downstream: false 14 | block-upstream: false 15 | concurrent: true 16 | properties: 17 | - github: 18 | url: https://github.com/ceph/ceph 19 | 20 | parameters: 21 | - string: 22 | name: BRANCH 23 | description: "The git branch (or tag) to build" 24 | 25 | scm: 26 | - git: 27 | url: git@github.com:ceph/ceph.git 28 | # Use the SSH key attached to the ceph-jenkins GitHub account. 29 | credentials-id: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 30 | branches: 31 | - $BRANCH 32 | skip-tag: true 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | publishers: 42 | - archive: 43 | artifacts: 'dist/**' 44 | allow-empty: false 45 | latest-only: false 46 | 47 | - postbuildscript: 48 | script-only-if-failed: True 49 | script-only-if-succeeded: False 50 | builders: 51 | - shell: 52 | !include-raw: 53 | - ../../../scripts/build_utils.sh 54 | - ../../build/failure 55 | 56 | wrappers: 57 | - inject-passwords: 58 | global: true 59 | mask-password-params: true 60 | -------------------------------------------------------------------------------- /ceph-dev-new-setup/config/definitions/ceph-dev-new-setup.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-dev-new-setup 3 | description: "This job step checks out the branch and builds the tarballs, diffs, and dsc that are passed to the ceph-dev-build step.\r\n\r\nNotes:\r\nJob needs to run on a releatively recent debian system. The Restrict where run feature is used to specifiy an appropriate label.\r\nThe clear workspace before checkout box for the git plugin is used." 4 | # we do not need to pin this to trusty anymore for the new jenkins instance 5 | # FIXME: unpin when this gets ported over 6 | node: huge && trusty && x86_64 7 | display-name: 'ceph-dev-new-setup' 8 | logrotate: 9 | daysToKeep: -1 10 | numToKeep: 25 11 | artifactDaysToKeep: -1 12 | artifactNumToKeep: -1 13 | block-downstream: false 14 | block-upstream: false 15 | concurrent: true 16 | properties: 17 | - github: 18 | url: https://github.com/ceph/ceph-ci 19 | 20 | parameters: 21 | - string: 22 | name: BRANCH 23 | description: "The git branch (or tag) to build" 24 | 25 | scm: 26 | - git: 27 | url: git@github.com:ceph/ceph-ci.git 28 | # Use the SSH key attached to the ceph-jenkins GitHub account. 29 | credentials-id: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 30 | branches: 31 | - $BRANCH 32 | skip-tag: true 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | publishers: 42 | - archive: 43 | artifacts: 'dist/**' 44 | allow-empty: false 45 | latest-only: false 46 | 47 | - postbuildscript: 48 | script-only-if-failed: True 49 | script-only-if-succeeded: False 50 | builders: 51 | - shell: 52 | !include-raw: 53 | - ../../../scripts/build_utils.sh 54 | - ../../build/failure 55 | 56 | wrappers: 57 | - inject-passwords: 58 | global: true 59 | mask-password-params: true 60 | -------------------------------------------------------------------------------- /ansible/roles/nginx/templates/nginx.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | user nginx; 3 | worker_processes 20; 4 | worker_rlimit_nofile 8192; 5 | 6 | pid /var/run/nginx.pid; 7 | 8 | events { 9 | worker_connections 1024; 10 | # multi_accept on; 11 | } 12 | 13 | http { 14 | 15 | ## 16 | # Basic Settings 17 | ## 18 | 19 | #sendfile on; 20 | tcp_nopush on; 21 | tcp_nodelay on; 22 | keepalive_timeout 65; 23 | types_hash_max_size 2048; 24 | server_tokens off; 25 | 26 | # server_names_hash_bucket_size 64; 27 | # server_name_in_redirect off; 28 | include /etc/nginx/mime.types; 29 | default_type application/octet-stream; 30 | 31 | ## 32 | # Logging Settings 33 | ## 34 | # Useful for logging load balancing requests 35 | log_format upstreamlog '[$time_local] $remote_addr - backend: $upstream_addr $request upstream_response_time $upstream_response_time request_time $request_time'; 36 | access_log /var/log/nginx/access.log; 37 | error_log /var/log/nginx/error.log; 38 | 39 | ## 40 | # Gzip Settings 41 | ## 42 | 43 | gzip on; 44 | gzip_disable "msie6"; 45 | 46 | # gzip_vary on; 47 | # gzip_proxied any; 48 | # gzip_comp_level 6; 49 | # gzip_buffers 16 8k; 50 | # gzip_http_version 1.1; 51 | # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; 52 | 53 | ## 54 | # If HTTPS, then set a variable so it can be passed along. 55 | ## 56 | 57 | map $scheme $server_https { 58 | default off; 59 | https on; 60 | } 61 | 62 | {% for host in nginx_hosts %} 63 | {% if host.upstreams is defined %} 64 | upstream {{ host.upstreams.name }} { 65 | {% if host.upstreams.strategy is defined %} 66 | {{ host.upstreams.strategy }}; 67 | {% endif %} 68 | {% for server in host.upstreams.servers %} 69 | server {{ server }}:443; 70 | {% endfor %} 71 | } 72 | {% endif %} 73 | {% endfor %} 74 | 75 | ## 76 | # Virtual Host Configs 77 | ## 78 | 79 | include /etc/nginx/conf.d/*.conf; 80 | include /etc/nginx/sites-enabled/*; 81 | } 82 | -------------------------------------------------------------------------------- /radosgw-agent/config/definitions/radosgw-agent.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: radosgw-agent 3 | node: small && trusty 4 | project-type: matrix 5 | defaults: global 6 | display-name: 'radosgw-agent' 7 | concurrent: true 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | 13 | parameters: 14 | - string: 15 | name: BRANCH 16 | description: "The git branch or tag to build" 17 | default: master 18 | 19 | - bool: 20 | name: RELEASE 21 | description: "If checked, it will use the key for releases, otherwise it will use the autosign one." 22 | default: true 23 | 24 | - bool: 25 | name: TEST 26 | description: " 27 | If this is unchecked, then the builds will be pushed to chacra with the correct ref. This is the default. 28 | 29 | If this is checked, then the builds will be pushed to chacra under the 'test' ref." 30 | 31 | - bool: 32 | name: FORCE 33 | description: " 34 | If this is unchecked, then then nothing is built or pushed if they already exist in chacra. This is the default. 35 | 36 | If this is checked, then the binaries will be built and pushed to chacra even if they already exist in chacra." 37 | default: true 38 | 39 | scm: 40 | - git: 41 | skip-tag: true 42 | url: https://github.com/ceph/radosgw-agent.git 43 | branches: 44 | - $BRANCH 45 | browser: auto 46 | timeout: 20 47 | 48 | axes: 49 | - axis: 50 | type: label-expression 51 | name: ARCH 52 | values: 53 | - x86_64 54 | - axis: 55 | type: label-expression 56 | name: DIST 57 | values: 58 | - wheezy 59 | - precise 60 | - trusty 61 | - jessie 62 | - centos6 63 | - centos7 64 | 65 | builders: 66 | - shell: 67 | !include-raw: 68 | - ../../../scripts/build_utils.sh 69 | - ../../build/build 70 | 71 | wrappers: 72 | - inject-passwords: 73 | global: true 74 | mask-password-params: true 75 | -------------------------------------------------------------------------------- /ceph-setup/config/definitions/ceph-setup.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-setup 3 | description: "This job step checks out the branch and builds the tarballs, diffs, and dsc that are passed to the ceph-build step.\r\n\r\nNotes:\r\nJob needs to run on a releatively recent debian system. The Restrict where run feature is used to specifiy an appropriate label.\r\nThe clear workspace before checkout box for the git plugin is used." 4 | # we do not need to pin this to trusty anymore for the new jenkins instance 5 | # FIXME: unpin when this gets ported over 6 | node: huge && trusty && x86_64 7 | display-name: 'ceph-setup' 8 | logrotate: 9 | daysToKeep: -1 10 | numToKeep: 25 11 | artifactDaysToKeep: -1 12 | artifactNumToKeep: -1 13 | block-downstream: false 14 | block-upstream: false 15 | concurrent: true 16 | properties: 17 | - github: 18 | url: https://github.com/ceph/ceph 19 | 20 | parameters: 21 | - string: 22 | name: BRANCH 23 | description: "The git branch (or tag) to build" 24 | 25 | scm: 26 | - git: 27 | url: git@github.com:ceph/ceph-releases.git 28 | # Use the SSH key attached to the ceph-jenkins GitHub account. 29 | credentials-id: '39fa150b-b2a1-416e-b334-29a9a2c0b32d' 30 | # not really a branch, this builds the TAG that was previously 31 | # created by the ceph-tag job which concatenates 'v' + $VERSION 32 | branches: 33 | - v$VERSION 34 | skip-tag: true 35 | wipe-workspace: true 36 | 37 | builders: 38 | - shell: 39 | !include-raw ../../build/build 40 | 41 | publishers: 42 | - archive: 43 | artifacts: 'dist/**' 44 | allow-empty: false 45 | latest-only: false 46 | 47 | - postbuildscript: 48 | script-only-if-failed: True 49 | script-only-if-succeeded: False 50 | builders: 51 | - shell: 52 | !include-raw: 53 | - ../../../scripts/build_utils.sh 54 | - ../../build/failure 55 | 56 | wrappers: 57 | - inject-passwords: 58 | global: true 59 | mask-password-params: true 60 | -------------------------------------------------------------------------------- /calamari-clients-build/config/definitions/calamari-clients-build.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: calamari-clients-build 3 | project-type: matrix 4 | defaults: global 5 | display-name: 'calamari-clients-build' 6 | block-downstream: false 7 | block-upstream: false 8 | concurrent: true 9 | properties: 10 | - github: 11 | url: https://github.com/ceph/calamari-clients 12 | execution-strategy: 13 | combination-filter: DIST==AVAILABLE_DIST && ARCH==AVAILABLE_ARCH && (ARCH=="x86_64" || (ARCH == "arm64" && (DIST == "xenial" || DIST == "centos7"))) 14 | axes: 15 | - axis: 16 | type: label-expression 17 | name: MACHINE_SIZE 18 | values: 19 | - small 20 | - axis: 21 | type: label-expression 22 | name: AVAILABLE_ARCH 23 | values: 24 | - x86_64 25 | - arm64 26 | - axis: 27 | type: label-expression 28 | name: AVAILABLE_DIST 29 | values: 30 | - centos6 31 | - centos7 32 | - axis: 33 | type: dynamic 34 | name: DIST 35 | values: 36 | - DISTROS 37 | - axis: 38 | type: dynamic 39 | name: ARCH 40 | values: 41 | - ARCHS 42 | 43 | 44 | 45 | builders: 46 | - shell: | 47 | echo "Cleaning up top-level workarea (shared among workspaces)" 48 | rm -rf dist 49 | rm -rf venv 50 | rm -rf release 51 | - copyartifact: 52 | project: calamari-clients-setup 53 | filter: 'dist/**' 54 | which-build: last-successful 55 | - inject: 56 | properties-file: ${WORKSPACE}/dist/sha1 57 | - inject: 58 | properties-file: ${WORKSPACE}/dist/branch 59 | - inject: 60 | properties-file: ${WORKSPACE}/dist/version 61 | # rpm build scripts 62 | - shell: 63 | !include-raw: 64 | - ../../build/validate_rpm 65 | - ../../../scripts/build_utils.sh 66 | - ../../build/setup 67 | - ../../build/build_rpm 68 | 69 | wrappers: 70 | - inject-passwords: 71 | global: true 72 | mask-password-params: true 73 | -------------------------------------------------------------------------------- /radosgw-agent-pull-requests/config/definitions/radosgw-agent-pull-requests.yml: -------------------------------------------------------------------------------- 1 | # multiple scm requires definition of each scm with `name` so that they can be 2 | # referenced later in `job` 3 | # reference: http://docs.openstack.org/infra/jenkins-job-builder/scm.html 4 | - scm: 5 | name: radosgw-agent 6 | scm: 7 | - git: 8 | url: https://github.com/ceph/radosgw-agent.git 9 | branches: 10 | - ${sha1} 11 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 12 | browser: auto 13 | timeout: 20 14 | skip-tag: true 15 | wipe-workspace: false 16 | basedir: "radosgw-agent" 17 | 18 | - scm: 19 | name: ceph-build 20 | scm: 21 | - git: 22 | url: https://github.com/ceph/ceph-build.git 23 | browser-url: https://github.com/ceph/ceph-build 24 | timeout: 20 25 | skip-tag: true 26 | wipe-workspace: false 27 | basedir: "ceph-build" 28 | 29 | 30 | - job: 31 | name: radosgw-agent-pull-requests 32 | node: trusty 33 | project-type: freestyle 34 | defaults: global 35 | display-name: 'radosgw-agent: Pull Requests' 36 | quiet-period: 5 37 | block-downstream: false 38 | block-upstream: false 39 | retry-count: 3 40 | properties: 41 | - github: 42 | url: https://github.com/ceph/radosgw-agent 43 | logrotate: 44 | daysToKeep: 15 45 | numToKeep: 30 46 | artifactDaysToKeep: -1 47 | artifactNumToKeep: -1 48 | 49 | parameters: 50 | - string: 51 | name: sha1 52 | description: "A pull request ID, like 'origin/pr/72/head'" 53 | 54 | triggers: 55 | - github-pull-request: 56 | admin-list: 57 | - alfredodeza 58 | - jdurgin 59 | org-list: 60 | - ceph 61 | trigger-phrase: '' 62 | only-trigger-phrase: false 63 | github-hooks: true 64 | permit-all: false 65 | auto-close-on-fail: false 66 | 67 | scm: 68 | - radosgw-agent 69 | - ceph-build 70 | 71 | builders: 72 | - shell: 73 | !include-raw: 74 | - ../../../scripts/build_utils.sh 75 | - ../../build/build 76 | -------------------------------------------------------------------------------- /ansible/examples/init.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Public-facing machines get the port changed to prevent a bit of abuse on the 3 | # standard one. There are some caveats to this approach, since we are changing 4 | # the default port we now need to instruct everything else to use the alternate 5 | # one. This should be run against newly brought up hosts when they are going to 6 | # be publicly accessible. 7 | 8 | # install python2.7 on xenial nodes 9 | - hosts: all 10 | sudo: yes 11 | user: admin 12 | gather_facts: false 13 | tasks: 14 | - name: install python-simplejson 15 | raw: sudo apt-get -y install python-simplejson 16 | # so that this is ignored on rpm nodes 17 | failed_when: false 18 | 19 | - hosts: all 20 | user: admin 21 | sudo: true 22 | tasks: 23 | 24 | - name: uncomment SSH port 25 | lineinfile: 26 | dest: /etc/ssh/sshd_config 27 | regexp: '^#Port ' 28 | line: 'Port 2222' 29 | backrefs: yes 30 | 31 | - name: change default port from 22 if set 32 | lineinfile: 33 | dest: /etc/ssh/sshd_config 34 | regexp: '^Port ' 35 | line: 'Port 2222' 36 | backrefs: yes 37 | 38 | # this requires the firewalld module that 39 | # I couldn't get to work. It exists in the extras modules 40 | #- name: enable the port in the firewall 41 | # firewalld: 42 | # port: 2222/tcp 43 | # permanent: true 44 | # state: enabled 45 | 46 | # this is far from ideal, we ignore errors because we can't 47 | # condition this if the port was already opened 48 | - name: tell selinux that ssh uses a new port 49 | command: semanage port -a -t ssh_port_t -p tcp 2222 50 | ignore_errors: yes 51 | 52 | # The CentOS Wiki says this should be run but I couldn't find 53 | # a firewall-cmd in the remote CentOS 7 box 54 | #- name: configure firewall to add new port 55 | # command: firewall-cmd --add-port 2222/tcp --permanent 56 | 57 | # Example action to start service httpd, if not running 58 | - name: restart ssh 59 | service: name=ssh state=restarted 60 | when: ansible_pkg_mgr == "apt" 61 | 62 | - name: restart sshd 63 | service: name=sshd state=restarted 64 | when: ansible_pkg_mgr == "yum" 65 | -------------------------------------------------------------------------------- /ceph-deploy/config/definitions/ceph-deploy.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-deploy 3 | project-type: multijob 4 | defaults: global 5 | display-name: 'ceph-deploy' 6 | concurrent: true 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | 12 | parameters: 13 | - string: 14 | name: BRANCH 15 | description: "The git branch or tag to build. Defaults to master" 16 | default: "master" 17 | 18 | - bool: 19 | name: TEST 20 | description: " 21 | If this is unchecked, then the builds will be pushed to chacra with the correct ref. This is the default. 22 | 23 | If this is checked, then the builds will be pushed to chacra under the 'test' ref." 24 | 25 | - bool: 26 | name: TAG 27 | description: "When this is checked, Jenkins will remove the previous tag and recreate it again, changing the control files and committing again. When this is unchecked, Jenkins will not do any commit or tag operations. If you've already created the private tag separately, then leave this unchecked. 28 | Defaults to checked." 29 | default: true 30 | 31 | - bool: 32 | name: FORCE 33 | description: " 34 | If this is unchecked, then then nothing is built or pushed if they already exist in chacra. This is the default. 35 | 36 | If this is checked, then the binaries will be built and pushed to chacra even if they already exist in chacra." 37 | 38 | - string: 39 | name: VERSION 40 | description: "The version for release, e.g. 0.94.4" 41 | 42 | builders: 43 | - multijob: 44 | name: 'ceph-deploy tag phase' 45 | condition: SUCCESSFUL 46 | projects: 47 | - name: ceph-deploy-tag 48 | current-parameters: true 49 | exposed-scm: false 50 | 51 | - multijob: 52 | name: 'ceph-deploy build phase' 53 | condition: SUCCESSFUL 54 | projects: 55 | - name: ceph-deploy-build 56 | current-parameters: true 57 | exposed-scm: false 58 | 59 | wrappers: 60 | - inject-passwords: 61 | global: true 62 | mask-password-params: true 63 | -------------------------------------------------------------------------------- /ceph-pr-commits/config/definitions/ceph-pr-commits.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: ceph 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/ceph.git 6 | branches: 7 | - ${sha1} 8 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 9 | browser: auto 10 | timeout: 20 11 | skip-tag: true 12 | wipe-workspace: false 13 | basedir: "ceph" 14 | 15 | - scm: 16 | name: ceph-build 17 | scm: 18 | - git: 19 | url: https://github.com/ceph/ceph-build.git 20 | branches: 21 | - origin/master 22 | browser-url: https://github.com/ceph/ceph-build 23 | timeout: 20 24 | skip-tag: true 25 | wipe-workspace: false 26 | basedir: "ceph-build" 27 | 28 | 29 | - job: 30 | name: ceph-pr-commits 31 | node: small 32 | project-type: freestyle 33 | defaults: global 34 | display-name: 'ceph: Pull Request commits' 35 | concurrent: true 36 | quiet-period: 5 37 | block-downstream: false 38 | block-upstream: false 39 | retry-count: 3 40 | properties: 41 | - github: 42 | url: https://github.com/ceph/ceph/ 43 | logrotate: 44 | daysToKeep: 15 45 | numToKeep: 30 46 | artifactDaysToKeep: -1 47 | artifactNumToKeep: -1 48 | 49 | parameters: 50 | - string: 51 | name: sha1 52 | description: "A pull request ID, like 'origin/pr/72/head'" 53 | 54 | triggers: 55 | - github-pull-request: 56 | allow-whitelist-orgs-as-admins: true 57 | org-list: 58 | - ceph 59 | trigger-phrase: '' 60 | only-trigger-phrase: false 61 | github-hooks: true 62 | permit-all: true 63 | auto-close-on-fail: false 64 | status-context: "Signed-off-by" 65 | started-status: "checking if commits are signed" 66 | success-status: "all commits in this PR are signed" 67 | failure-status: "one or more commits in this PR are not signed" 68 | 69 | scm: 70 | - ceph 71 | - ceph-build 72 | 73 | 74 | builders: 75 | - shell: 76 | !include-raw: 77 | - ../../../scripts/build_utils.sh 78 | - ../../build/build 79 | 80 | publishers: 81 | - junit: 82 | results: report.xml 83 | -------------------------------------------------------------------------------- /ceph-docker-prs/config/definitions/ceph-docker-prs.yml: -------------------------------------------------------------------------------- 1 | - project: 2 | name: ceph-docker-prs 3 | scenario: 4 | - ceph_ansible-jewel-centos7_cluster 5 | - ceph_ansible-jewel-xenial_cluster 6 | - ceph_ansible-kraken-centos7_cluster 7 | - ceph_ansible-kraken-xenial_cluster 8 | jobs: 9 | - 'ceph-docker-prs-{scenario}' 10 | 11 | 12 | - job-template: 13 | name: 'ceph-docker-prs-{scenario}' 14 | node: vagrant&&libvirt 15 | concurrent: true 16 | defaults: global 17 | display-name: 'ceph-docker: Pull Requests [{scenario}]' 18 | quiet-period: 5 19 | block-downstream: false 20 | block-upstream: false 21 | retry-count: 3 22 | properties: 23 | - github: 24 | url: https://github.com/ceph/ceph-docker 25 | logrotate: 26 | daysToKeep: 15 27 | numToKeep: 30 28 | artifactDaysToKeep: -1 29 | artifactNumToKeep: -1 30 | 31 | parameters: 32 | - string: 33 | name: sha1 34 | description: "A pull request ID, like 'origin/pr/72/head'" 35 | 36 | triggers: 37 | - github-pull-request: 38 | allow-whitelist-orgs-as-admins: true 39 | org-list: 40 | - ceph 41 | trigger-phrase: 'jenkins test {scenario}' 42 | only-trigger-phrase: false 43 | github-hooks: true 44 | permit-all: true 45 | auto-close-on-fail: false 46 | status-context: "Testing: {scenario}" 47 | started-status: "Running: {scenario}" 48 | success-status: "OK - {scenario}" 49 | failure-status: "FAIL - {scenario}" 50 | 51 | scm: 52 | - git: 53 | url: https://github.com/ceph/ceph-docker.git 54 | branches: 55 | - ${{sha1}} 56 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 57 | browser: auto 58 | timeout: 20 59 | skip-tag: true 60 | wipe-workspace: false 61 | 62 | builders: 63 | - inject: 64 | properties-content: | 65 | SCENARIO={scenario} 66 | - shell: 67 | !include-raw-escape: 68 | - ../../../scripts/build_utils.sh 69 | - ../../build/build 70 | 71 | publishers: 72 | - postbuildscript: 73 | script-only-if-succeeded: False 74 | script-only-if-failed: True 75 | builders: 76 | - shell: !include-raw ../../build/teardown 77 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/config/definitions/ceph-deploy-pull-requests.yml: -------------------------------------------------------------------------------- 1 | # multiple scm requires definition of each scm with `name` so that they can be 2 | # referenced later in `job` 3 | # reference: http://docs.openstack.org/infra/jenkins-job-builder/scm.html 4 | - scm: 5 | name: ceph-deploy 6 | scm: 7 | - git: 8 | url: https://github.com/ceph/ceph-deploy.git 9 | branches: 10 | - ${sha1} 11 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 12 | browser: auto 13 | timeout: 20 14 | skip-tag: true 15 | wipe-workspace: false 16 | basedir: "ceph-deploy" 17 | 18 | - scm: 19 | name: ceph-build 20 | scm: 21 | - git: 22 | url: https://github.com/ceph/ceph-build.git 23 | browser-url: https://github.com/ceph/ceph-build 24 | timeout: 20 25 | skip-tag: true 26 | wipe-workspace: false 27 | basedir: "ceph-build" 28 | 29 | 30 | - job: 31 | name: ceph-deploy-pull-requests 32 | node: trusty 33 | project-type: freestyle 34 | defaults: global 35 | concurrent: true 36 | display-name: 'ceph-deploy: Pull Requests' 37 | quiet-period: 5 38 | block-downstream: false 39 | block-upstream: false 40 | retry-count: 3 41 | properties: 42 | - github: 43 | url: https://github.com/ceph/ceph-deploy/ 44 | logrotate: 45 | daysToKeep: 15 46 | numToKeep: 30 47 | artifactDaysToKeep: -1 48 | artifactNumToKeep: -1 49 | 50 | parameters: 51 | - string: 52 | name: sha1 53 | description: "A pull request ID, like 'origin/pr/72/head'" 54 | 55 | triggers: 56 | - github-pull-request: 57 | admin-list: 58 | - alfredodeza 59 | - ktdreyer 60 | white-list: 61 | - xarses 62 | - angdraug 63 | - mbroz 64 | - ddiss 65 | - osynge 66 | org-list: 67 | - ceph 68 | trigger-phrase: '' 69 | only-trigger-phrase: false 70 | github-hooks: true 71 | permit-all: false 72 | auto-close-on-fail: false 73 | 74 | scm: 75 | - ceph-deploy 76 | - ceph-build 77 | 78 | builders: 79 | - shell: 80 | !include-raw: 81 | - ../../../scripts/build_utils.sh 82 | - ../../build/setup 83 | - ../../build/build 84 | -------------------------------------------------------------------------------- /calamari/build/build_rpm: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are an RPM distro 5 | if test "$DISTRO" != "fedora" -a "$DISTRO" != "centos" -a "$DISTRO" != "rhel"; then 6 | exit 0 7 | fi 8 | 9 | 10 | ## Install any setup-time deps (to make dist package) 11 | # We need this to get the major version from lsb_release 12 | sudo yum install -y redhat-lsb-core mock 13 | 14 | # Run the install-deps.sh upstream script if it exists 15 | if [ -x install-deps.sh ]; then 16 | echo "Ensuring dependencies are installed" 17 | sudo ./install-deps.sh 18 | fi 19 | 20 | 21 | ## Get some basic information about the system and the repository 22 | RELEASE="$(lsb_release --short -r | cut -d '.' -f 1)" # sytem release 23 | # Get VERSION, REVISION (and RPM_RELEASE) 24 | eval $(./get-versions.sh) 25 | RPM_RELEASE=$(echo $REVISION | tr '-' '_') # the '-' has a special meaning 26 | 27 | 28 | ## Build the source tarball 29 | echo "Building source distribution" 30 | make dist 31 | mv ../calamari-server_*.tar.gz dist/ 32 | 33 | 34 | ## Prepare the spec file for build 35 | sed -e "s/@VERSION@/${VERSION}/g" -e "s/@RELEASE@/${RPM_RELEASE}/g" < calamari-server.spec.in > dist/calamari-server.spec 36 | 37 | 38 | ## Install the build-time dependencies (for rpmbuild -bs) 39 | sudo yum-builddep -y dist/calamari-server.spec 40 | 41 | 42 | ## Create the source rpm 43 | echo "Building SRPM" 44 | rpmbuild \ 45 | --define "_sourcedir ./dist" \ 46 | --define "_specdir ." \ 47 | --define "_builddir ." \ 48 | --define "_srcrpmdir ." \ 49 | --define "_rpmdir ." \ 50 | --define "dist .any" \ 51 | --define "fedora 21" \ 52 | --define "rhel 7" \ 53 | --nodeps -bs dist/calamari-server.spec 54 | SRPM=$(readlink -f *.src.rpm) 55 | 56 | 57 | ## Build the binaries with mock 58 | echo "Building RPMs" 59 | sudo mock -r ${MOCK_TARGET}-${RELEASE}-${ARCH} --resultdir=./dist/rpm/ ${SRPM} || cat ./dist/rpm/build.log 60 | 61 | 62 | ## Upload the created RPMs to chacra 63 | chacra_endpoint="calamari/${BRANCH}/${GIT_COMMIT}/${DISTRO}/${RELEASE}" 64 | 65 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 66 | 67 | # push binaries to chacra 68 | find ./dist/rpm/ | egrep '\.rpm$' | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/$ARCH/ 69 | 70 | # start repo creation 71 | $VENV/chacractl repo update ${chacra_endpoint} 72 | 73 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 74 | -------------------------------------------------------------------------------- /ceph-pr-commits/build/test_commits.py: -------------------------------------------------------------------------------- 1 | from subprocess import Popen, PIPE 2 | import os 3 | from os.path import dirname 4 | import pytest 5 | 6 | # ceph-pr-commits/build 7 | current_directory = dirname(os.path.abspath(__file__)) 8 | 9 | # workspace directory 10 | workspace = os.getenv('WORKSPACE', None) or dirname(dirname(dirname(current_directory))) 11 | 12 | # ceph checkout path 13 | ceph_checkout = os.path.join(workspace, 'ceph') 14 | 15 | 16 | def run(command): 17 | print "Running command: %s" % ' '.join(command) 18 | process = Popen( 19 | command, 20 | cwd=ceph_checkout, 21 | stdout=PIPE, 22 | stderr=PIPE, 23 | close_fds=True 24 | ) 25 | 26 | returncode = process.wait() 27 | 28 | return process.stdout.read() 29 | 30 | 31 | def get_commits(): 32 | target_branch = os.getenv('ghprbTargetBranch', 'master') 33 | # ensure that we have the latest commits from master 34 | command = ['git', 'fetch', 'origin', '+refs/heads/{target_branch}:refs/remotes/origin/{target_branch}'.format(target_branch=target_branch)] 35 | run(command) 36 | # we use 'HEAD' here because the PR is already checked out on the right branch 37 | source_branch = 'HEAD' 38 | command = ['git', 'log', '--no-merges', 'origin/%s..%s' % (target_branch, source_branch)] 39 | output = run(command) 40 | chunked_commits = [] 41 | for chunk in output.split('\n\ncommit'): 42 | if not chunk: 43 | continue 44 | chunked_commits.append(chunk) 45 | return chunked_commits 46 | 47 | 48 | 49 | class TestSignedOffByCommits(object): 50 | 51 | def test_signed_off_by(self): 52 | for commit in get_commits(): 53 | if 'Signed-off-by:' not in commit: 54 | msg = ( 55 | "\nFollowing commit is not signed, please make sure all commits", 56 | "\nare signed following the 'Submitting Patches' guide:", 57 | "\nhttps://github.com/ceph/ceph/blob/master/SubmittingPatches#L61", 58 | "\n", 59 | commit) 60 | raise AssertionError, ' '.join(msg) 61 | 62 | def extract_sha(self, lines): 63 | # XXX Unused for now, if py.test can spit out the hashes in verbose 64 | # mode this should be removed, otherwise put to good use 65 | trim = lines.split() 66 | for i in trim: 67 | if i and 'commit' not in i: 68 | return i 69 | -------------------------------------------------------------------------------- /samba/build/build_deb: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test "$DISTRO" != "debian" -a "$DISTRO" != "ubuntu"; then 6 | exit 0 7 | fi 8 | 9 | 10 | ## Install any setup-time deps 11 | # We need these for the build 12 | sudo apt-get install -y build-essential equivs libgnutls-dev libacl1-dev libldap2-dev ruby ruby-dev 13 | 14 | # We use fpm to create the deb package 15 | sudo gem install fpm 16 | 17 | 18 | ## Do the actual build 19 | # Prepare the build 20 | DESTDIR="install.tmp" 21 | install -d -m0755 -- "$DESTDIR" 22 | ./configure --without-lttng 23 | 24 | # Perform the build and install the files to DESTDIR 25 | NCPU=$(grep -c processor /proc/cpuinfo) 26 | make -j$NCPU 27 | make -j$NCPU install DESTDIR=${DESTDIR} 28 | 29 | 30 | ## Get some basic information about the system and the repository 31 | # Get version 32 | export LD_LIBRARY_PATH=${DESTDIR}/usr/local/samba/lib/:${DESTDIR}/usr/local/samba/lib/private/ 33 | DEB_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) 34 | VERSION=$(${DESTDIR}/usr/local/samba/sbin/smbd --version | sed -e "s|Version ||") 35 | REVISION="$(git rev-parse HEAD)" 36 | 37 | 38 | ## Create the deb package 39 | # Make sure there are no other deb packages, first 40 | rm -f *.deb 41 | 42 | # Create the deb package 43 | fpm -s dir -t deb -n samba -v ${VERSION} -C ${DESTDIR} -d krb5-user usr 44 | 45 | 46 | ## Upload the created DEB to chacra 47 | chacra_endpoint="samba/${BRANCH}/${GIT_COMMIT}/${DISTRO}/${DIST}" 48 | 49 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 50 | 51 | # push binaries to chacra 52 | find *.deb | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/${DEB_ARCH}/ 53 | PACKAGE_MANAGER_VERSION=$(dpkg-deb -f $(find *"$DEB_ARCH".deb | head -1) Version) 54 | 55 | # write json file with build info 56 | cat > $WORKSPACE/repo-extra.json << EOF 57 | { 58 | "version":"$VERSION", 59 | "package_manager_version":"$PACKAGE_MANAGER_VERSION", 60 | "build_url":"$BUILD_URL", 61 | "root_build_cause":"$ROOT_BUILD_CAUSE", 62 | "node_name":"$NODE_NAME", 63 | "job_name":"$JOB_NAME" 64 | } 65 | EOF 66 | # post the json to repo-extra json to chacra 67 | curl -X POST -H "Content-Type:application/json" --data "@$WORKSPACE/repo-extra.json" -u $CHACRACTL_USER:$CHACRACTL_KEY ${chacra_url}repos/${chacra_endpoint}/extra/ 68 | 69 | # start repo creation 70 | $VENV/chacractl repo update ${chacra_endpoint} 71 | 72 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 73 | --------------------------------------------------------------------------------