├── .gitignore ├── ansible ├── tmp │ └── .empty ├── files │ ├── ssl │ │ ├── dev │ │ │ └── .empty │ │ └── prod │ │ │ └── .empty │ ├── rpmmacros.j2 │ └── ssh │ │ ├── keys │ │ └── jenkins_build.pub │ │ └── hostkeys │ │ ├── github.com.pub │ │ └── docs.ceph.com.pub ├── private-docker-registry │ ├── group_vars │ │ └── all │ ├── site.yml │ ├── vagrant_variables.yml │ └── roles │ │ ├── docker-registry │ │ ├── templates │ │ │ └── self-csr.json.j2 │ │ └── tasks │ │ │ └── main.yml │ │ └── test-client │ │ └── tasks │ │ └── main.yml ├── roles │ ├── ansible-jenkins │ │ ├── .gitignore │ │ ├── tasks │ │ │ ├── main.yml │ │ │ ├── dependencies.yml │ │ │ ├── repo.yml │ │ │ ├── ufw.yml │ │ │ ├── jenkins.yml │ │ │ └── nginx.yml │ │ ├── files │ │ │ ├── jenkins.repo │ │ │ ├── ssh │ │ │ │ ├── jenkins_build.pub │ │ │ │ └── keys │ │ │ │ │ └── jenkins_build.pub │ │ │ └── jenkins.conf │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── templates │ │ │ ├── hudson.tasks.Mailer.xml.j2 │ │ │ ├── jenkins.conf │ │ │ └── nginx.conf │ │ ├── vars │ │ │ └── main.yml │ │ └── README.rst │ ├── grafana │ │ ├── defaults │ │ │ └── main.yml │ │ ├── handlers │ │ │ └── main.yml │ │ ├── vars │ │ │ └── main.yml │ │ ├── tasks │ │ │ ├── nginx.yml │ │ │ ├── postgresql.yml │ │ │ └── main.yml │ │ └── templates │ │ │ └── nginx_site.conf │ ├── graphite │ │ ├── defaults │ │ │ └── main.yml │ │ ├── templates │ │ │ ├── systemd │ │ │ │ ├── 80-graphite.preset.j2 │ │ │ │ └── graphite.service.j2 │ │ │ ├── storage-schemas.conf.j2 │ │ │ ├── whitelist.conf.j2 │ │ │ └── rewrite-rules.conf.j2 │ │ ├── handlers │ │ │ └── main.yml │ │ ├── vars │ │ │ └── main.yml │ │ └── tasks │ │ │ ├── systemd.yml │ │ │ ├── carbon.yml │ │ │ └── main.yml │ ├── ceph-release │ │ ├── handlers │ │ │ └── main.yml │ │ └── tasks │ │ │ ├── release │ │ │ ├── candidate.yml │ │ │ ├── development.yml │ │ │ └── stable.yml │ │ │ ├── main.yml │ │ │ └── write_sha1_file.yml │ ├── kraken │ │ ├── templates │ │ │ ├── helga.sysconfig.j2 │ │ │ └── helga.service.j2 │ │ ├── handlers │ │ │ └── main.yml │ │ ├── tasks │ │ │ └── systemd.yml │ │ └── defaults │ │ │ └── main.yml │ ├── nginx │ │ ├── handlers │ │ │ └── main.yml │ │ ├── defaults │ │ │ └── main.yml │ │ ├── templates │ │ │ └── nginx_tmp_site.conf │ │ └── tasks │ │ │ ├── ssl.yml │ │ │ ├── main.yml │ │ │ └── letsencrypt.yml │ ├── redis │ │ └── tasks │ │ │ └── main.yml │ ├── remoto-release │ │ └── tasks │ │ │ ├── clear_version.yml │ │ │ └── main.yml │ └── ceph-deploy-release │ │ └── tasks │ │ ├── clear_version.yml │ │ └── main.yml ├── templates │ ├── systemd │ │ ├── jenkins.secret.j2 │ │ └── jenkins.service.j2 │ └── yum-repos │ │ └── epel.repo ├── host_vars │ ├── prado.ceph.com.yml │ └── all.yml ├── sensu-hosts ├── ansible.cfg ├── examples │ ├── deploy_graphite.yml │ ├── shaman_load_balancer.yml │ ├── hosts-static │ └── deploy_grafana.yml ├── .gitignore ├── requirements │ └── sensu-requirements.yml └── vars │ ├── load-balance-vars.yml │ └── sensu-vars.yml ├── sync-images ├── config │ ├── JENKINS_URL │ └── definitions │ │ └── sync-images.yml └── build │ └── build ├── ceph-ansible-prs ├── config │ └── JENKINS_URL └── build │ └── teardown ├── ceph-ansible-rpm ├── config │ ├── JENKINS_URL │ └── definitions │ │ └── ceph-ansible-rpm.yml └── build │ └── build ├── ceph-ansible-galaxy ├── config │ ├── JENKINS_URL │ └── definitions │ │ └── ceph-ansible-galaxy.yml └── build │ └── build ├── ceph-ansible-scenario ├── config │ └── JENKINS_URL └── build │ ├── teardown │ └── build ├── cephadm-ansible-prs ├── config │ └── JENKINS_URL └── build │ ├── teardown │ └── build ├── attic ├── ceph-container-lint │ └── config │ │ └── JENKINS_URL ├── ceph-container-prs │ ├── config │ │ └── JENKINS_URL │ └── build │ │ ├── teardown │ │ └── build ├── ceph-ansible-docs-prs │ ├── config │ │ └── JENKINS_URL │ └── build │ │ └── build ├── ceph-container-flake8 │ └── config │ │ └── JENKINS_URL ├── ceph-container-build-push-imgs │ ├── config │ │ ├── JENKINS_URL │ │ └── definitions │ │ │ └── ceph-container-build-push-imgs.yml │ └── build │ │ └── build ├── ceph-container-prerelease-build │ ├── config │ │ └── JENKINS_URL │ └── build │ │ └── build ├── ceph-container-build-ceph-base-push-imgs │ ├── config │ │ └── JENKINS_URL │ └── build │ │ └── build ├── ceph-container-build-push-imgs-arm64 │ ├── config │ │ ├── JENKINS_URL │ │ └── definitions │ │ │ └── ceph-container-build-push-imgs-arm64.yml │ └── build │ │ └── build ├── ceph-container-build-push-imgs-devel-nightly │ ├── config │ │ ├── JENKINS_URL │ │ └── definitions │ │ │ └── ceph-container-build-push-imgs-devel-nightly.yml │ └── build │ │ └── build ├── ceph-container-build-ceph-base-push-imgs-arm64 │ ├── config │ │ └── JENKINS_URL │ └── build │ │ └── build ├── diamond-build │ └── build │ │ ├── validate_deb │ │ ├── validate_rpm │ │ └── setup_pbuilder ├── teuthology-docs │ ├── setup │ │ └── setup │ ├── build │ │ └── build │ └── config │ │ └── definitions │ │ └── teuthology-docs.yml ├── ceph-dev-trigger │ └── build │ │ └── notify ├── ceph-medic-pull-requests │ └── build │ │ └── build ├── ceph-medic-tests │ └── build │ │ └── build ├── ceph-medic-docs │ ├── build │ │ └── build │ └── config │ │ └── definitions │ │ └── ceph-medic-docs.yml ├── ceph-dev-container-only │ └── build │ │ └── failure ├── ceph-docs │ ├── config │ │ └── definitions │ │ │ └── ceph-docs.yml │ └── build │ │ └── build ├── ceph-pr-render-docs │ └── build │ │ └── build ├── ceph-medic-release │ └── build │ │ └── build_rpm └── ceph-medic-rpm │ ├── config │ └── definitions │ │ └── ceph-medic-rpm.yml │ └── build │ └── build ├── ceph-dev-new-build └── build │ ├── build_osc │ ├── setup_osc │ ├── validate_deb │ ├── validate_osc │ ├── validate_rpm │ ├── build_deb │ ├── validate_mingw │ ├── failure │ └── setup_pbuilder ├── ceph-deploy-pull-requests ├── build │ ├── build │ └── setup └── setup │ └── playbooks │ ├── setup.yml │ └── tasks │ └── ubuntu.yml ├── chacra-pull-requests ├── setup │ └── playbooks │ │ ├── ansible.cfg │ │ └── setup.yml └── build │ └── build ├── shaman-pull-requests ├── setup │ └── playbooks │ │ ├── ansible.cfg │ │ ├── setup.yml │ │ ├── templates │ │ └── pg_hba.conf.j2 │ │ └── tasks │ │ └── postgresql.yml └── build │ └── build ├── ceph-iscsi-stable ├── build │ ├── failure │ ├── validate_rpm │ └── setup └── README.rst ├── ceph-pull-requests └── build │ ├── cleanup │ └── kill-tests ├── ceph-build └── build │ ├── build_deb │ ├── validate_deb │ ├── validate_rpm │ ├── failure │ └── setup_pbuilder ├── ceph-dev-build └── build │ ├── build_deb │ ├── validate_mingw │ ├── validate_deb │ ├── validate_osc │ ├── validate_rpm │ ├── failure │ └── setup_pbuilder ├── ceph-pr-api └── build │ ├── api │ └── build ├── ceph-grafana ├── build │ └── build └── config │ └── definitions │ └── ceph-grafana.yml ├── kernel └── build │ ├── validate_deb │ ├── validate_rpm │ ├── failure │ └── prepare_config ├── scripts ├── status_completed ├── dashboard │ ├── install-backend-api-test-deps.sh │ └── install-e2e-test-deps.sh ├── ceph-windows │ ├── cleanup │ ├── cleanup_libvirt_ubuntu_vm │ ├── setup_libvirt_windows_vm │ └── win32_build ├── setup_uv.sh ├── setup_chacractl.sh ├── get-tarballs.sh ├── ceph-website │ └── install-deps.sh ├── nfs-ganesha │ ├── sync-push-auto │ ├── pull-sign-push │ └── sign-rpm-auto └── rook-orch │ └── install-rook-e2e-deps.sh ├── nfs-ganesha └── build │ ├── validate_deb │ ├── validate_rpm │ └── failure ├── ceph-iscsi-cli └── build │ ├── validate_deb │ ├── validate_rpm │ ├── build_deb │ └── setup ├── ceph-iscsi-tools └── build │ ├── validate_deb │ ├── validate_rpm │ ├── build_deb │ └── setup ├── ceph-iscsi-config └── build │ ├── validate_deb │ ├── validate_rpm │ ├── build_deb │ └── setup ├── nfs-ganesha-stable └── build │ ├── validate_deb │ ├── validate_rpm │ └── failure ├── quay-pruner ├── build │ └── build └── config │ └── definitions │ └── quay-pruner.yml ├── ceph-rook-e2e └── build │ └── cleanup ├── ceph-iscsi └── build │ ├── validate_rpm │ └── setup ├── cephmetrics-pull-requests ├── build │ └── build └── setup │ └── setup ├── ceph-volume-scenario └── build │ ├── teardown │ └── build ├── ceph-iscsi-tox └── build │ └── build ├── wnbd-build └── build │ ├── failure │ └── setup ├── ceph-dashboard-cephadm-e2e └── build │ └── cleanup ├── ceph-iscsi-cli-flake8 └── build │ └── build ├── ceph-iscsi-config-flake8 └── build │ └── build ├── ceph-windows-installer-build └── build │ └── failure ├── ceph-dev-cron └── build │ └── notify ├── ceph-dev-new-trigger └── build │ └── notify ├── ceph-setup └── build │ ├── failure │ └── create_tag ├── ceph-dev-setup └── build │ └── failure ├── ceph-dev-new-setup └── build │ └── failure ├── paddles-pull-requests ├── build │ └── build └── config │ └── definitions │ └── paddles-pull-requests.yml ├── merfi-pull-requests └── build │ └── build ├── ceph-volume-nightly └── build │ ├── teardown │ └── build ├── ceph-windows-pull-requests └── build │ └── check_docs_pr_only ├── ceph-deploy-docs ├── build │ └── build └── config │ └── definitions │ └── ceph-deploy-docs.yml ├── ceph-qa-suite-pull-requests ├── build │ └── build └── config │ └── definitions │ └── ceph-qa-suite-pull-requests.yml ├── mita-deploy ├── build │ └── build └── config │ └── definitions │ └── mita-deploy.yml ├── lab-cop ├── config │ └── definitions │ │ └── lab-cop.yml └── build │ └── build ├── ceph-volume-unit-tests └── build │ ├── teardown │ └── build ├── ceph-website ├── build │ └── build └── config │ └── definitions │ └── ceph-website.yml ├── ceph-multibranch-pipeline └── config │ └── definitions │ └── ceph-multibranch-pipeline.yml ├── ceph-api-nightly └── build │ └── cleanup ├── ceph-windows-image-build ├── build │ ├── cleanup │ └── install-windows-updates.ps1 └── README.md ├── ceph-volume-cephadm-prs └── build │ ├── teardown │ └── build ├── ceph-pr-commits └── build │ ├── conftest.py │ └── build ├── ceph-deploy-tag └── build │ └── build ├── ceph-tag └── build │ └── build ├── ceph-ansible-docs ├── build │ └── build └── config │ └── definitions │ └── ceph-ansible-docs.yml ├── ceph-iscsi-trigger └── config │ └── definitions │ └── ceph-iscsi-trigger.yml ├── samba-trigger └── config │ └── definitions │ └── samba-trigger.yml ├── radosgw-agent-pull-requests └── build │ └── build ├── ceph-iscsi-cli-trigger └── config │ └── definitions │ └── ceph-iscsi-cli-trigger.yml ├── rtslib-fb-trigger └── config │ └── definitions │ └── rtslib-fb-trigger.yml ├── ceph-iscsi-tools-trigger └── config │ └── definitions │ └── ceph-iscsi-tools-trigger.yml ├── ceph-iscsi-config-trigger └── config │ └── definitions │ └── ceph-iscsi-config-trigger.yml ├── tcmu-runner-trigger └── config │ └── definitions │ └── tcmu-runner-trigger.yml ├── ceph-grafana-trigger └── build │ └── build ├── ceph-deploy-build ├── build │ └── setup └── config │ └── definitions │ └── ceph-deploy-build.yml ├── ceph-website-prs └── build │ └── build ├── cephmetrics-release └── build │ └── setup ├── ceph-dev-pipeline-trigger ├── config │ └── definitions │ │ └── ceph-dev-pipeline-trigger.yml └── build │ └── Jenkinsfile ├── rtslib-fb └── build │ └── setup ├── configshell-fb └── build │ └── setup ├── tcmu-runner └── build │ └── setup ├── cephmetrics └── build │ └── setup ├── ceph-pull-requests-arm64 └── build │ └── kill-tests ├── ceph-devstack └── config │ └── definitions │ └── ceph-devstack.yml ├── samba └── build │ └── setup ├── ceph-build-pull-requests └── config │ └── definitions │ └── ceph-build-pull-requests.yml └── ceph-pr-docs └── build └── build /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | -------------------------------------------------------------------------------- /ansible/tmp/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ansible/files/ssl/dev/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ansible/files/ssl/prod/.empty: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /sync-images/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-ansible-prs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-ansible-rpm/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-ansible-galaxy/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-ansible-scenario/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /cephadm-ansible-prs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/group_vars/all: -------------------------------------------------------------------------------- 1 | --- 2 | dummy: 3 | -------------------------------------------------------------------------------- /attic/ceph-container-lint/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /attic/ceph-container-prs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | .vagrant 3 | Vagrantfile -------------------------------------------------------------------------------- /attic/ceph-ansible-docs-prs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /attic/ceph-container-flake8/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/build_osc: -------------------------------------------------------------------------------- 1 | ../../ceph-dev-build/build/build_osc -------------------------------------------------------------------------------- /ceph-dev-new-build/build/setup_osc: -------------------------------------------------------------------------------- 1 | ../../ceph-dev-build/build/setup_osc -------------------------------------------------------------------------------- /ansible/templates/systemd/jenkins.secret.j2: -------------------------------------------------------------------------------- 1 | {{ api_user }}:{{ token }} 2 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | ../../ceph-dev-build/build/validate_deb -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_osc: -------------------------------------------------------------------------------- 1 | ../../ceph-dev-build/build/validate_osc -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | ../../ceph-dev-build/build/validate_rpm -------------------------------------------------------------------------------- /ansible/files/rpmmacros.j2: -------------------------------------------------------------------------------- 1 | %dist .el{{ ansible_distribution_major_version }} 2 | -------------------------------------------------------------------------------- /attic/ceph-container-prerelease-build/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /attic/ceph-container-build-ceph-base-push-imgs/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-arm64/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | cd $WORKSPACE/ceph-deploy && $VENV/tox -rv 2 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-devel-nightly/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /chacra-pull-requests/setup/playbooks/ansible.cfg: -------------------------------------------------------------------------------- 1 | [ssh_connection] 2 | pipelining=True 3 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/ansible.cfg: -------------------------------------------------------------------------------- 1 | [ssh_connection] 2 | pipelining=True 3 | -------------------------------------------------------------------------------- /ansible/host_vars/prado.ceph.com.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sensu_client_subscription_names: 3 | - common 4 | -------------------------------------------------------------------------------- /attic/ceph-container-build-ceph-base-push-imgs-arm64/config/JENKINS_URL: -------------------------------------------------------------------------------- 1 | 2.jenkins.ceph.com 2 | -------------------------------------------------------------------------------- /ansible/host_vars/all.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sensu_client_subscription_names: 3 | - common 4 | - rabbitmq 5 | -------------------------------------------------------------------------------- /ansible/roles/grafana/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | app_name: "grafana" 4 | fqdn: "grafana.local" 5 | -------------------------------------------------------------------------------- /ceph-iscsi-stable/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | rm -f ~/.chacractl 4 | rm -rf $WORKSPACE/* 5 | -------------------------------------------------------------------------------- /ceph-pull-requests/build/cleanup: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | podman unshare chown -R 0:0 "${WORKSPACE}" 4 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: jenkins.yml 3 | tags: jenkins 4 | -------------------------------------------------------------------------------- /ansible/roles/graphite/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | app_name: "graphite" 4 | fqdn: "graphite.local" 5 | -------------------------------------------------------------------------------- /ceph-build/build/build_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | build_debs ${VENV} ${vers} ${debian_version} 5 | -------------------------------------------------------------------------------- /ceph-dev-build/build/build_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | build_debs $VENV ${vers} ${debian_version} 5 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/build_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | build_debs ${VENV} ${vers} ${debian_version} 5 | -------------------------------------------------------------------------------- /ceph-pr-api/build/api: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | cd src/pybind/mgr/dashboard 3 | timeout 7200 ./run-backend-api-tests.sh 4 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: "update apt cache" 3 | action: apt update-cache=yes 4 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/systemd/80-graphite.preset.j2: -------------------------------------------------------------------------------- 1 | # graphite web service 2 | 3 | enable graphite.service 4 | -------------------------------------------------------------------------------- /ansible/roles/kraken/templates/helga.sysconfig.j2: -------------------------------------------------------------------------------- 1 | HOME=/home/{{ ansible_ssh_user }} 2 | HELGA_SETTINGS=custom_settings 3 | -------------------------------------------------------------------------------- /ceph-grafana/build/build: -------------------------------------------------------------------------------- 1 | sudo yum install -y buildah 2 | cd monitoring/grafana/build 3 | make build 4 | make push 5 | make clean 6 | -------------------------------------------------------------------------------- /ansible/sensu-hosts: -------------------------------------------------------------------------------- 1 | [sensu-server] 2 | 158.69.67.168 3 | 4 | [sensu-clients] 5 | jenkins.ceph.com 6 | chacra.ceph.com 7 | prado.ceph.com 8 | -------------------------------------------------------------------------------- /ansible/roles/nginx/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: restart nginx 4 | become: yes 5 | action: service name=nginx state=restarted enabled=yes 6 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/storage-schemas.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # 3 | [default] 4 | pattern = .* 5 | retentions = 5s:30d,1m:120d,10m:1y 6 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/jenkins.repo: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | [jenkins] 4 | name=Jenkins 5 | baseurl=http://pkg.jenkins-ci.org/redhat 6 | gpgcheck=1 7 | -------------------------------------------------------------------------------- /kernel/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /scripts/status_completed: -------------------------------------------------------------------------------- 1 | # update shaman with the completed build status 2 | update_build_status "completed" "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 3 | -------------------------------------------------------------------------------- /ansible/ansible.cfg: -------------------------------------------------------------------------------- 1 | [defaults] 2 | callback_plugins = callbacks 3 | retry_files_enabled = False 4 | stdout_callback=debug 5 | 6 | [ssh_connection] 7 | pipelining=True 8 | 9 | -------------------------------------------------------------------------------- /ceph-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /nfs-ganesha/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | cd "$WORKSPACE"/ceph-container/ || exit 6 | bash -x contrib/build-push-ceph-container-imgs.sh 7 | -------------------------------------------------------------------------------- /ceph-iscsi-cli/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-tools/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /attic/diamond-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-config/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /nfs-ganesha-stable/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test -f /etc/redhat-release ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /quay-pruner/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | python3 -mvenv ./v 3 | ./v/bin/pip install requests 4 | ./v/bin/python3 ./ceph-build/quay-pruner/build/prune-quay.py -v 5 | rm -rf ./v 6 | 7 | -------------------------------------------------------------------------------- /ceph-rook-e2e/build/cleanup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set +x 3 | echo "Starting cleanup..." 4 | docker container prune -f 5 | minikube stop 6 | minikube delete 7 | echo "Cleanup completed." 8 | -------------------------------------------------------------------------------- /chacra-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | become: True 6 | 7 | tasks: 8 | - import_tasks: tasks/postgresql.yml 9 | -------------------------------------------------------------------------------- /kernel/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | sudo: True 6 | 7 | tasks: 8 | - import_tasks: tasks/postgresql.yml 9 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-arm64/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | cd "$WORKSPACE"/ceph-container/ || exit 6 | bash -x contrib/build-push-ceph-container-imgs-arm64.sh 7 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-devel-nightly/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | cd "$WORKSPACE"/ceph-container/ || exit 6 | bash -x contrib/build-push-ceph-container-imgs.sh 7 | -------------------------------------------------------------------------------- /ceph-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /nfs-ganesha/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-cli/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-config/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-stable/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ceph-iscsi-tools/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /cephmetrics-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | # run tox by recreating the environment and in verbose mode 4 | # by default this will run all environments defined 5 | $VENV/tox -rv 6 | -------------------------------------------------------------------------------- /nfs-ganesha-stable/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/dependencies.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install DEB dependencies 3 | apt: 4 | name: "{{ item }}" 5 | state: installed 6 | with_items: "{{ jenkins.dependencies }}" 7 | -------------------------------------------------------------------------------- /attic/diamond-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | if [[ ! -f /etc/redhat-release && ! -f /usr/bin/zypper ]] ; then 6 | exit 0 7 | fi 8 | -------------------------------------------------------------------------------- /ansible/roles/nginx/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | ssl_support_email: "adeza@redhat.com" 3 | ssl_webroot_base_path: "/var/www" 4 | 5 | ssl_cert: "files/ssl/dev/ssl/ssl.crt" 6 | ssl_key: "files/ssl/dev/ssl/ssl.key" 7 | -------------------------------------------------------------------------------- /ansible/templates/yum-repos/epel.repo: -------------------------------------------------------------------------------- 1 | [epel] 2 | baseurl = http://download.fedoraproject.org/pub/epel/$releasever/$basearch/ 3 | gpgkey = https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7 4 | name = EPEL YUM repo 5 | -------------------------------------------------------------------------------- /ceph-iscsi-cli/build/build_deb: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test "$DISTRO" != "debian" -a "$DISTRO" != "ubuntu"; then 6 | exit 0 7 | fi 8 | 9 | exit 1 10 | -------------------------------------------------------------------------------- /ceph-iscsi-config/build/build_deb: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test "$DISTRO" != "debian" -a "$DISTRO" != "ubuntu"; then 6 | exit 0 7 | fi 8 | 9 | exit 1 10 | -------------------------------------------------------------------------------- /ceph-iscsi-tools/build/build_deb: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | if test "$DISTRO" != "debian" -a "$DISTRO" != "ubuntu"; then 6 | exit 0 7 | fi 8 | 9 | exit 1 10 | -------------------------------------------------------------------------------- /ceph-ansible-galaxy/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # propagates the change in the necessary Ansible Galaxy repos. 4 | # i.e. https://github.com/ceph/ansible-ceph-common 5 | bash "$WORKSPACE"/ceph-ansible/contrib/push-roles-to-ansible-galaxy.sh 6 | -------------------------------------------------------------------------------- /ansible/roles/nginx/templates/nginx_tmp_site.conf: -------------------------------------------------------------------------------- 1 | server { 2 | server_name {{ item.fqdn }}; 3 | location '/.well-known/acme-challenge' { 4 | default_type "text/plain"; 5 | root {{ ssl_webroot_base_path }}/{{ item.fqdn }}; 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /ceph-volume-scenario/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd $WORKSPACE/src/ceph-volume/ceph_volume/tests/functional 4 | 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | # the method exists in scripts/build_utils.sh 8 | teardown_vagrant_tests $VENV 9 | -------------------------------------------------------------------------------- /ansible/examples/deploy_graphite.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - graphite 7 | vars: 8 | fqdn: "graphite.ceph.com" 9 | app_name: "graphite" 10 | development_server: true 11 | graphite_api_key: "secret" 12 | -------------------------------------------------------------------------------- /ansible/.gitignore: -------------------------------------------------------------------------------- 1 | # these should never make it to version control 2 | files/ssl/dev 3 | files/ssl/prod 4 | tmp 5 | 6 | # Top level YAML files are ignored, so that users can simply copy 7 | # from examples/* to the top level dir and not worry about committing them 8 | /*.yml 9 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/candidate.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "New upstream release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/development.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "Development release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/release/stable.yml: -------------------------------------------------------------------------------- 1 | 2 | - name: set the debian version 3 | command: dch -v {{ version }}-1 -D stable "New upstream release" chdir=ceph 4 | environment: 5 | DEBEMAIL: "{{ debemail }}" 6 | DEBFULLNAME: "{{ debfullname }}" 7 | 8 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/site.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Defines deployment design and assigns role to server groups 3 | 4 | - hosts: registry 5 | become: True 6 | roles: 7 | - docker-registry 8 | 9 | - hosts: testclient 10 | become: True 11 | roles: 12 | - test-client 13 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | placeholder: 'placeholder' 3 | jenkins_port: 8080 4 | letsencrypt_email: 'ceph-infra@redhat.com' 5 | # plugins: 6 | # - 'ldap' 7 | # - 'github' 8 | # - 'translation' 9 | # - 'preSCMbuildstep' 10 | # - 'gravatar' 11 | -------------------------------------------------------------------------------- /attic/teuthology-docs/setup/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | APT_DEPS="libmysqlclient-dev libevent-dev libffi-dev libssl-dev pkg-config libvirt-dev" 6 | # We don't have tty-less sudo on docs.ceph.com; these deps must be installed 7 | # manually. 8 | #sudo apt install $APT_DEPS 9 | -------------------------------------------------------------------------------- /ansible/roles/kraken/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: restart helga service 4 | service: 5 | name: helga 6 | status: restarted 7 | 8 | # prevents issues when updating systemd files 9 | - name: reload systemd 10 | become: yes 11 | command: systemctl daemon-reload 12 | -------------------------------------------------------------------------------- /ceph-ansible-scenario/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd $WORKSPACE/tests 4 | 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | # the method exists in scripts/build_utils.sh 8 | teardown_vagrant_tests $VENV 9 | 10 | # clean fact cache 11 | rm -rf $HOME/ansible/facts/* 12 | -------------------------------------------------------------------------------- /scripts/dashboard/install-backend-api-test-deps.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | if grep -q debian /etc/*-release; then 5 | sudo apt-get install -y python3-scipy python3-routes 6 | elif grep -q rhel /etc/*-release; then 7 | sudo yum install -y scipy python-routes python3-routes 8 | fi 9 | -------------------------------------------------------------------------------- /attic/ceph-ansible-docs-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | cd $WORKSPACE/docs/ 10 | $VENV/tox -rv 11 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/setup/playbooks/setup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: localhost 4 | user: jenkins-build 5 | become: yes 6 | 7 | tasks: 8 | - include: tasks/ubuntu.yml 9 | when: ansible_distribution == "Ubuntu" 10 | 11 | # TODO: maybe add RPM or Debian handling? 12 | -------------------------------------------------------------------------------- /ceph-iscsi-tox/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | cd "$WORKSPACE/ceph-iscsi" 10 | 11 | $VENV/tox -rv 12 | -------------------------------------------------------------------------------- /cephadm-ansible-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd "${WORKSPACE}"/tests 4 | 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | # the method exists in scripts/build_utils.sh 8 | teardown_vagrant_tests $VENV 9 | 10 | # clean fact cache 11 | rm -rf "${HOME}"/ansible/facts/* 12 | -------------------------------------------------------------------------------- /wnbd-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | FLAVOR="default" 6 | 7 | BRANCH=`branch_slash_filter $BRANCH` 8 | 9 | # update shaman with the failed build status 10 | failed_build_status "wnbd" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 11 | -------------------------------------------------------------------------------- /ceph-dashboard-cephadm-e2e/build/cleanup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set +x 3 | echo "Starting cleanup..." 4 | kcli delete plan -y ceph || true 5 | kcli delete network ceph-dashboard -y 6 | kcli delete pool ceph-dashboard -y 7 | sudo rm -rf ${HOME}/.kcli 8 | docker container prune -f 9 | echo "Cleanup completed." 10 | -------------------------------------------------------------------------------- /ceph-iscsi-cli-flake8/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | cd "$WORKSPACE/ceph-iscsi-cli" 10 | 11 | $VENV/tox -rv -e flake8 12 | -------------------------------------------------------------------------------- /ceph-iscsi-config-flake8/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | cd "$WORKSPACE/ceph-iscsi-config" 10 | 11 | $VENV/tox -rv -e flake8 12 | -------------------------------------------------------------------------------- /ceph-windows-installer-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | FLAVOR="default" 6 | 7 | BRANCH=`branch_slash_filter $BRANCH` 8 | 9 | # update shaman with the failed build status 10 | failed_build_status "ceph-windows-installer" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 11 | -------------------------------------------------------------------------------- /ansible/requirements/sensu-requirements.yml: -------------------------------------------------------------------------------- 1 | - src: https://github.com/dmick/ansible-playbook-sensu 2 | scm: git 3 | name: Mayeu.sensu 4 | version: remotes/origin/plugin-gem-install 5 | 6 | - src: https://github.com/dmick/ansible-playbook-rabbitmq 7 | scm: git 8 | name: Mayeu.RabbitMQ 9 | version: remotes/origin/master 10 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_mingw: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # We're currently using pbuilder. 5 | ( source /etc/os-release 6 | case $ID in 7 | ubuntu) 8 | exit 0 9 | ;; 10 | *) 11 | exit 1 12 | ;; 13 | esac) || exit 0 14 | 15 | if [ "${DIST}" != "windows" ]; then 16 | exit 0 17 | fi 18 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/vagrant_variables.yml: -------------------------------------------------------------------------------- 1 | --- 2 | provision_test_client_vm: true 3 | memory: 1024 4 | vagrant_box: centos/atomic-host 5 | # The sync directory changes based on vagrant box 6 | # Set to /home/vagrant/sync for Centos/7, /home/{ user }/vagrant for openstack and defaults to /vagrant 7 | vagrant_sync_dir: /home/vagrant/sync 8 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/validate_mingw: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # We're currently using pbuilder. 5 | ( source /etc/os-release 6 | case $ID in 7 | ubuntu) 8 | exit 0 9 | ;; 10 | *) 11 | exit 1 12 | ;; 13 | esac) || exit 0 14 | 15 | if [ "${DIST}" != "windows" ]; then 16 | exit 0 17 | fi 18 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_deb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Only do actual work when we are a DEB distro 5 | ( source /etc/os-release 6 | case $ID in 7 | debian|ubuntu) 8 | exit 0 9 | ;; 10 | *) 11 | exit 1 12 | ;; 13 | esac) || exit 0 14 | 15 | if [ "${DIST}" == "windows" ]; then 16 | exit 0 17 | fi 18 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_osc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a SUSE distro 5 | ( source /etc/os-release 6 | case $ID in 7 | opensuse*|suse|sles) 8 | exit 0 9 | ;; 10 | *) 11 | exit 1 12 | ;; 13 | esac) || exit 0 14 | 15 | if [ "${DIST}" == "windows" ]; then 16 | exit 0 17 | fi 18 | -------------------------------------------------------------------------------- /ceph-dev-build/build/validate_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # only do work if we are a RPM distro 5 | ( source /etc/os-release 6 | case $ID in 7 | centos|rhel|fedora) 8 | exit 0 9 | ;; 10 | *) 11 | exit 1 12 | ;; 13 | esac) || exit 0 14 | 15 | if [ "${DIST}" == "windows" ]; then 16 | exit 0 17 | fi 18 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/handlers/main.yml: -------------------------------------------------------------------------------- 1 | - name: restart jenkins 2 | service: 3 | name: jenkins 4 | state: restarted 5 | when: okay_with_restart == "y" 6 | 7 | - name: restart nginx 8 | service: 9 | name: nginx 10 | state: restarted 11 | 12 | - name: stop nginx 13 | service: 14 | name: nginx 15 | state: stopped 16 | -------------------------------------------------------------------------------- /ceph-dev-cron/build/notify: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the triggered build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # build information 6 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 7 | SHA1=${GIT_COMMIT} 8 | 9 | update_build_status "queued" "ceph" 10 | -------------------------------------------------------------------------------- /scripts/ceph-windows/cleanup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | 6 | # Cleanup libvirt VMs / networks 7 | delete_libvirt_vms 8 | clear_libvirt_networks 9 | 10 | # Cleanup remaining files / directories 11 | sudo rm -rf \ 12 | $WORKSPACE/ceph $WORKSPACE/ceph_vstart $WORKSPACE/ceph.zip \ 13 | $WORKSPACE/libvirt 14 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/repo.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Add Jenkins GPG Key 3 | apt_key: 4 | url: "https://jenkins-ci.org/debian/jenkins-ci.org.key" 5 | state: present 6 | 7 | - name: Add the jenkins repo 8 | apt_repository: 9 | repo: 'deb https://pkg.jenkins.io/debian-stable binary/' 10 | state: present 11 | update_cache: true 12 | -------------------------------------------------------------------------------- /scripts/ceph-windows/cleanup_libvirt_ubuntu_vm: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | if [[ -z $UBUNTU_VM_NAME ]]; then echo "ERROR: The env variable UBUNTU_VM_NAME is not set"; exit 1; fi 6 | 7 | # Destroy and undefine the VM 8 | sudo virsh destroy $UBUNTU_VM_NAME 9 | sudo virsh undefine $UBUNTU_VM_NAME --remove-all-storage 10 | -------------------------------------------------------------------------------- /attic/ceph-dev-trigger/build/notify: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the triggered build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # build information 6 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 7 | SHA1=${GIT_COMMIT} 8 | 9 | update_build_status "queued" "ceph" 10 | -------------------------------------------------------------------------------- /ceph-dev-new-trigger/build/notify: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the triggered build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # build information 6 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 7 | SHA1=${GIT_COMMIT} 8 | 9 | update_build_status "queued" "ceph" 10 | 11 | -------------------------------------------------------------------------------- /attic/ceph-container-build-ceph-base-push-imgs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | sudo apt-get install jq -y 6 | 7 | cd "$WORKSPACE"/ceph-container/ || exit 8 | export PRERELEASE=false 9 | ARCH=x86_64 bash -x contrib/build-ceph-base.sh 10 | 11 | echo "Now running manifest script" 12 | BUILD_SERVER_GOARCH=amd64 bash -x contrib/make-ceph-base-manifests.sh 13 | -------------------------------------------------------------------------------- /attic/ceph-container-build-ceph-base-push-imgs-arm64/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | sudo apt-get install jq -y 6 | 7 | cd "$WORKSPACE"/ceph-container/ || exit 8 | export PRERELEASE=false 9 | ARCH=aarch64 bash -x contrib/build-ceph-base.sh 10 | 11 | echo "Now running manifest script" 12 | BUILD_SERVER_GOARCH=arm64 bash -x contrib/make-ceph-base-manifests.sh 13 | -------------------------------------------------------------------------------- /ceph-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /ceph-dev-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /ansible/roles/redis/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: install Redis server 3 | apt: 4 | name: redis-server 5 | state: latest 6 | update_cache: yes 7 | 8 | - name: check if Redis is running 9 | service: 10 | name: redis-server 11 | state: started 12 | 13 | - name: enable redis-server to survive reboot 14 | service: 15 | name: redis-server 16 | enabled: yes 17 | -------------------------------------------------------------------------------- /ceph-dev-new-setup/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # update shaman with the failed build status. At this point there aren't any 4 | # architectures or distro information, so we just report this with the current 5 | # (ceph-dev-setup) build information that includes log and build urls 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | SHA1=${GIT_COMMIT} 8 | 9 | failed_build_status "ceph" 10 | -------------------------------------------------------------------------------- /paddles-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | sudo yum install -y epel-release 10 | sudo yum --enablerepo epel install -y python36 11 | 12 | cd "$WORKSPACE/paddles" 13 | 14 | $VENV/tox -rv 15 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/hudson.tasks.Mailer.xml.j2: -------------------------------------------------------------------------------- 1 | 2 | 3 | {{ email.default_email_suffix }} 4 | {{ email.smtp_host }} 5 | {{ email.smtp_ssl }} 6 | UTF-8 7 | 8 | -------------------------------------------------------------------------------- /merfi-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # run tox by recreating the environment and in verbose mode 12 | # by default this will run all environments defined 13 | $VENV/tox -rv 14 | -------------------------------------------------------------------------------- /ansible/roles/grafana/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: reload systemd 4 | become: yes 5 | command: systemctl daemon-reload 6 | 7 | - name: restart app 8 | become: true 9 | service: 10 | name: grafana-server 11 | state: restarted 12 | enabled: yes 13 | 14 | - name: restart nginx 15 | become: true 16 | service: 17 | name: nginx 18 | state: restarted 19 | enabled: yes 20 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/setup/playbooks/tasks/ubuntu.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: "update apt repo" 3 | action: apt update_cache=yes 4 | 5 | - name: install python requirements 6 | action: apt pkg={{ item }} 7 | with_items: 8 | - python-software-properties 9 | - python-dev 10 | - python2.7 11 | - python3.5 12 | 13 | - name: install pip 14 | action: easy_install name=pip 15 | -------------------------------------------------------------------------------- /ansible/roles/grafana/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | system_packages: 4 | - grafana 5 | - git 6 | - g++ 7 | - gcc 8 | - libpq-dev 9 | - postgresql 10 | - postgresql-common 11 | - postgresql-contrib 12 | - python-psycopg2 13 | - nginx 14 | - vim 15 | # needed for the ansible apt_repository module 16 | - python-apt 17 | - python 18 | 19 | ssl_requirements: 20 | - openssl 21 | - libssl-dev 22 | -------------------------------------------------------------------------------- /ansible/examples/shaman_load_balancer.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - nginx 7 | vars_files: 8 | - vars/load-balance-vars.yml 9 | vars: 10 | app_name: "shaman" 11 | fqdn: "shaman.ceph.com" 12 | development_server: true 13 | # only needed when enabling Github Auth 14 | # github_client_id: "111aaa222" 15 | # github_client_secret: "qwerty1234" 16 | -------------------------------------------------------------------------------- /ansible/roles/graphite/handlers/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: reload systemd 4 | become: yes 5 | command: systemctl daemon-reload 6 | 7 | - name: restart app 8 | become: true 9 | service: 10 | name: graphite 11 | state: restarted 12 | enabled: yes 13 | 14 | - name: restart carbon 15 | service: 16 | name: carbon-cache 17 | state: restarted 18 | enabled: yes 19 | become: yes 20 | 21 | -------------------------------------------------------------------------------- /ansible/roles/kraken/templates/helga.service.j2: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=helga (kraken) bot service 3 | After=network.target 4 | 5 | [Service] 6 | Type=simple 7 | ExecStart={{ helga_home }}/bin/helga 8 | EnvironmentFile=/etc/sysconfig/helga 9 | User={{ ansible_ssh_user }} 10 | WorkingDirectory={{ helga_home }}/src/ 11 | StandardOutput=journal 12 | StandardError=journal 13 | 14 | [Install] 15 | WantedBy=multi-user.target 16 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/nginx.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create nginx site config 3 | action: template src=../templates/nginx_site.conf dest=/etc/nginx/sites-available/{{ app_name }}.conf 4 | become: true 5 | notify: 6 | - restart nginx 7 | 8 | - name: link nginx config 9 | action: file src=/etc/nginx/sites-available/{{ app_name }}.conf dest=/etc/nginx/sites-enabled/{{ app_name }}.conf state=link 10 | become: true 11 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/whitelist.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # This file takes a single regular expression per line 3 | # If USE_WHITELIST is set to True in carbon.conf, only metrics received which 4 | # match one of these expressions will be persisted. If this file is empty or 5 | # missing, all metrics will pass through. 6 | # This file is reloaded automatically when changes are made 7 | ^{{ graphite_api_key }}.* 8 | -------------------------------------------------------------------------------- /ceph-volume-nightly/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | 6 | cd $WORKSPACE/src/ceph-volume/ceph_volume/tests/functional 7 | 8 | TEMPVENV=$(create_venv_dir) 9 | VENV=${TEMPVENV}/bin 10 | # the method exists in scripts/build_utils.sh 11 | teardown_vagrant_tests $VENV 12 | -------------------------------------------------------------------------------- /ceph-windows-pull-requests/build/check_docs_pr_only: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | docs_pr_only 6 | container_pr_only 7 | gha_pr_only 8 | qa_pr_only 9 | if [[ "$DOCS_ONLY" = true || "$CONTAINER_ONLY" = true || "$GHA_ONLY" == true || "$QA_ONLY" == true ]]; then 10 | echo "Only the doc/, container/, qa/ or .github/ dir changed. No need to run Ceph Windows tests." 11 | exit 0 12 | fi 13 | -------------------------------------------------------------------------------- /ansible/files/ssh/keys/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD4l25l32QJxD9aWZxXcUaVPAClY53yuLwRvnn4V5ZzE3Wd5ddrbbsBhON6iPdb5Vv4d9Q3b1SfKhWjAIZgSeHwD1vcTPt/Kd2nIpwv2qQYjKAvnByS21NNdd5kf1Iz+NFieJlWi+x3vBaDR53OY9OpVnwZj7pZ4jQXHJ4bhVOEG9auxncpy4/HJi70IpnqRFg9GidqcaGsiFUu4+ji1euU2WKnX2hVmIxK/47vKTGSVpDEAPM8OqBueKXU78fh2xeFR76hVWkKpLwhhdr5OAY1dDzuADjFNUT/x76eceXZ8B1aPBsQuhQ9W2oPtQZRHqu8ElXFuhrwQ30pm5IAzimb jenkins-build@jenkins 2 | -------------------------------------------------------------------------------- /attic/ceph-medic-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | sudo yum install -y epel-release 10 | sudo yum --enablerepo epel install -y python36 11 | 12 | cd "$WORKSPACE/ceph-medic" 13 | 14 | export TOX_SKIP_ENV=py37 15 | $VENV/tox -rv 16 | -------------------------------------------------------------------------------- /ceph-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /kernel/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "kernel" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ceph-dev-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/docker-registry/templates/self-csr.json.j2: -------------------------------------------------------------------------------- 1 | { 2 | "CN": "docker-registry", 3 | "hosts": [ 4 | "{{ ansible_default_ipv4.address }}", 5 | "127.0.0.1" 6 | ], 7 | "key": { 8 | "algo": "rsa", 9 | "size": 2048 10 | }, 11 | "names": [ 12 | { 13 | "C": "XX", 14 | "L": "Default City", 15 | "O": "Default Company Ltd", 16 | "ST": "." 17 | } 18 | ] 19 | } 20 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/ssh/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI0tHxJQ7n+uMiLpsoR6CKAVd0xatgVQuqp/gmnGpZU0kE54a29vPNnEt7/aLitbfyhc57rrbHOT09H3ov74GZKkoVBSbMJUSsK3drbN+58wcuk+HK0htRewmwCfcfi9AkrVbyw6pbPXW/pbjxnxLep52fKmpJJnImZ5eHRV5le9OSAcLA1LHYR4y9R3IOrTp7jgpE205UxZi5OopAx7gkyTsmfydvmq4MjaSwbVOJ7aW/Fdt5FVxNJP3Zl/OrvDoo/1WovoRIDbVQH8JFpLikMSnCqtBVIHDeW6imAKl6dpn9Gf4FxD94+OcurhXo2p0pvSzC4Strg4d2Sxqh4wph jenkins-build 2 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - import_tasks: create.yml 3 | when: "stage == 'create'" 4 | 5 | - import_tasks: push.yml 6 | when: 7 | - "stage == 'push'" 8 | - "release != 'SECURITY'" 9 | - tag|bool is true 10 | 11 | # This only runs to prevent the Archive Artifacts plugin from hanging 12 | - import_tasks: write_sha1_file.yml 13 | when: 14 | - "stage == 'push'" 15 | - "release == 'SECURITY'" 16 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | BRANCH=`branch_slash_filter $BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/ssh/keys/jenkins_build.pub: -------------------------------------------------------------------------------- 1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI0tHxJQ7n+uMiLpsoR6CKAVd0xatgVQuqp/gmnGpZU0kE54a29vPNnEt7/aLitbfyhc57rrbHOT09H3ov74GZKkoVBSbMJUSsK3drbN+58wcuk+HK0htRewmwCfcfi9AkrVbyw6pbPXW/pbjxnxLep52fKmpJJnImZ5eHRV5le9OSAcLA1LHYR4y9R3IOrTp7jgpE205UxZi5OopAx7gkyTsmfydvmq4MjaSwbVOJ7aW/Fdt5FVxNJP3Zl/OrvDoo/1WovoRIDbVQH8JFpLikMSnCqtBVIHDeW6imAKl6dpn9Gf4FxD94+OcurhXo2p0pvSzC4Strg4d2Sxqh4wph jenkins-build 2 | -------------------------------------------------------------------------------- /attic/ceph-container-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | cd $WORKSPACE/ceph-ansible/tests/functional 6 | 7 | scenarios=$(find . | grep Vagrantfile | xargs -r dirname) 8 | 9 | for scenario in $scenarios; do 10 | cd $scenario 11 | vagrant destroy -f 12 | cd - 13 | done 14 | -------------------------------------------------------------------------------- /ceph-deploy-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # create the docs build with tox 12 | $VENV/tox -rv -e docs 13 | 14 | # publish docs to http://docs.ceph.com/docs/ceph-deploy 15 | rsync -auv --delete .tox/docs/tmp/html/* /var/ceph-deploy/docs/ 16 | -------------------------------------------------------------------------------- /ceph-qa-suite-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | 12 | # run tox by recreating the environment and in verbose mode 13 | # by default this will run all environments defined, although currently 14 | # it is just flake8 15 | $VENV/tox -rv 16 | -------------------------------------------------------------------------------- /attic/teuthology-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # create the docs build with tox 12 | $VENV/tox -rv -e docs 13 | 14 | # publish docs to http://docs.ceph.com/docs/teuthology 15 | rsync -auv --delete .tox/docs/tmp/html/* /var/teuthology/docs/ 16 | -------------------------------------------------------------------------------- /mita-deploy/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | cd "$WORKSPACE/deploy/playbooks/" 12 | $VENV/ansible-playbook -i "localhost," -c local local_deploy.yml --extra-vars="branch=$BRANCH jenkins_prado_token=$JENKINS_PRADO_TOKEN prado_token=$PRADO_TOKEN" 13 | -------------------------------------------------------------------------------- /nfs-ganesha/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # note: the failed_build_status call relies on normalized variable names that 4 | # are infered by the builds themselves. If the build fails before these are 5 | # set, they will be posted with empty values 6 | NFS_GANESHA_BRANCH=`branch_slash_filter $NFS_GANESHA_BRANCH` 7 | 8 | # update shaman with the failed build status 9 | failed_build_status "nfs-ganesha" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 10 | -------------------------------------------------------------------------------- /lab-cop/config/definitions/lab-cop.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: lab-cop 3 | node: small && xenial 4 | defaults: global 5 | display-name: 'lab-cop' 6 | properties: 7 | - build-discarder: 8 | days-to-keep: 15 9 | num-to-keep: 30 10 | artifact-days-to-keep: -1 11 | artifact-num-to-keep: -1 12 | 13 | builders: 14 | - shell: 15 | !include-raw-verbatim: 16 | - ../../build/build 17 | 18 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/systemd/graphite.service.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | [Unit] 3 | Description=graphite gunicorn service 4 | After=network.target 5 | 6 | [Service] 7 | Type=simple 8 | ExecStart=/usr/bin/gunicorn -b 127.0.0.1:8000 -w 10 -t 30 graphite_web:application 9 | User=_graphite 10 | WorkingDirectory=/usr/lib/python2.7/dist-packages/graphite 11 | StandardOutput=journal 12 | StandardError=journal 13 | 14 | [Install] 15 | WantedBy=multi-user.target 16 | -------------------------------------------------------------------------------- /ceph-iscsi-stable/README.rst: -------------------------------------------------------------------------------- 1 | ceph-iscsi-stable 2 | ================= 3 | This job is used to build and push RPMs to chacra.ceph.com so they can be synced, signed, then pushed to download.ceph.com. 4 | 5 | There are scripts in ``~/ceph-iscsi/bin`` on the signer box for pulling, signing, and pushing the RPMs. 6 | 7 | .. code:: 8 | 9 | # Example 10 | cd /home/ubuntu/ceph-iscsi/bin 11 | ./sync-pull 2 0784eb00a859501f90f2b1c92354ae7242d5be3d 12 | ./sign-rpms 13 | ./sync-push 2 14 | -------------------------------------------------------------------------------- /ceph-volume-unit-tests/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "github-status>0.0.3" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | GITHUB_STATUS_STATE="failure" $VENV/github-status create 12 | -------------------------------------------------------------------------------- /ansible/vars/load-balance-vars.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | # These values allow the nginx role to fully configure load balancing and 4 | # regular hosts. 5 | nginx_hosts: 6 | - fqdn: "grafana.ceph.com" 7 | app_name: "grafana" 8 | proxy_pass: "http://127.0.0.1:3000" 9 | - fqdn: "shaman.ceph.com" 10 | app_name: "shaman" 11 | upstreams: 12 | name: "shaman" 13 | strategy: "least_conn" 14 | servers: 15 | - "1.shaman.ceph.com" 16 | - "2.shaman.ceph.com" 17 | -------------------------------------------------------------------------------- /ceph-website/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | env 5 | 6 | set +e 7 | export NVM_DIR="$HOME/.nvm" 8 | [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" 9 | [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" 10 | set -e 11 | 12 | echo "Using node version $(node -v)" 13 | 14 | npm install 15 | 16 | npm run build:production 17 | 18 | if [ ! -d /opt/www/main ]; then 19 | mkdir -p /opt/www/main 20 | fi 21 | 22 | rsync -av --delete-after dist/ /opt/www/main/ 23 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | jenkins_dest: /opt/jenkins 3 | jenkins_lib: /var/lib/jenkins 4 | jenkins_jobs: '{{ jenkins_lib }}/jobs' 5 | jenkins: 6 | dependencies: # Jenkins dependencies 7 | - 'git' 8 | - 'curl' 9 | - 'nginx' 10 | - 'default-jdk' 11 | config_file: '/etc/default/jenkins' 12 | cli_dest: '{{ jenkins_dest }}/jenkins-cli.jar' # Jenkins CLI destination 13 | updates_dest: '{{ jenkins_dest }}/updates_jenkins.json' # Jenkins updates file 14 | -------------------------------------------------------------------------------- /ansible/roles/graphite/vars/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | system_packages: 4 | - grafana 5 | - graphite-web 6 | - graphite-api 7 | - graphite-carbon 8 | - git 9 | - g++ 10 | - gcc 11 | - libpq-dev 12 | - postgresql 13 | - postgresql-common 14 | - postgresql-contrib 15 | - python-psycopg2 16 | - nginx 17 | - vim 18 | # needed for the ansible apt_repository module 19 | - python-apt 20 | - python 21 | - gunicorn 22 | 23 | ssl_requirements: 24 | - openssl 25 | - libssl-dev 26 | -------------------------------------------------------------------------------- /cephmetrics-pull-requests/setup/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | if test $(id -u) != 0 ; then 4 | SUDO=sudo 5 | fi 6 | 7 | deb_deps="python-dev python-virtualenv" 8 | rpm_deps="python-devel python-virtualenv" 9 | if test -f /etc/redhat-release ; then 10 | $SUDO yum install -y $rpm_deps 11 | elif test -f /etc/debian_version ; then 12 | $SUDO apt install -y $deb_deps 13 | fi 14 | 15 | pkgs=( "tox" ) 16 | TEMPVENV=$(create_venv_dir) 17 | VENV=${TEMPVENV}/bin 18 | install_python_packages $TEMPVENV "pkgs[@]" 19 | -------------------------------------------------------------------------------- /shaman-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | install_python_packages $TEMPVENV "pkgs[@]" 6 | 7 | # run ansible to get this current host to meet our requirements, specifying 8 | # a local connection and 'localhost' as the host where to execute 9 | cd "$WORKSPACE/ceph-build/shaman-pull-requests/setup/playbooks" 10 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 11 | 12 | cd "$WORKSPACE/shaman" 13 | $VENV/tox -rv 14 | -------------------------------------------------------------------------------- /scripts/setup_uv.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # vim: ts=4 sw=4 expandtab 3 | 4 | function setup_pipx () { 5 | command -v pipx || ( 6 | command -v apt && sudo apt install -y pipx 7 | command -v dnf && sudo dnf install -y pipx 8 | ) 9 | pipx ensurepath 10 | } 11 | 12 | function setup_uv () { 13 | setup_pipx 14 | pipx install uv 15 | } 16 | 17 | # If the script is executed (as opposed to sourced), run the function now 18 | if [ "$(basename -- "${0#-}")" = "$(basename -- "${BASH_SOURCE[0]}")" ]; then 19 | setup_uv 20 | fi 21 | -------------------------------------------------------------------------------- /attic/ceph-medic-tests/build/build: -------------------------------------------------------------------------------- 1 | # the following two methods exist in scripts/build_utils.sh 2 | pkgs=( "tox" ) 3 | TEMPVENV=$(create_venv_dir) 4 | VENV=${TEMPVENV}/bin 5 | install_python_packages $TEMPVENV "pkgs[@]" 6 | 7 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 8 | 9 | cd $WORKSPACE/tests/functional 10 | 11 | delete_libvirt_vms 12 | clear_libvirt_networks 13 | restart_libvirt_services 14 | 15 | CEPH_MEDIC_DEV_BRANCH=$CEPH_MEDIC_BRANCH CEPH_ANSIBLE_BRANCH=$CEPH_ANSIBLE_BRANCH $VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR -- --provider=libvirt 16 | -------------------------------------------------------------------------------- /ceph-deploy-pull-requests/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | 10 | # run ansible to get this current host to meet our requirements, specifying 11 | # a local connection and 'localhost' as the host where to execute 12 | cd "$WORKSPACE/ceph-build/ceph-deploy-pull-requests/setup/playbooks" 13 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 14 | -------------------------------------------------------------------------------- /ansible/examples/hosts-static: -------------------------------------------------------------------------------- 1 | [smithi] 2 | smithi118.front.sepia.ceph.com nodename=smithi118 3 | smithi119.front.sepia.ceph.com nodename=smithi119 4 | smithi120.front.sepia.ceph.com nodename=smithi120 5 | smithi121.front.sepia.ceph.com nodename=smithi121 6 | smithi122.front.sepia.ceph.com nodename=smithi122 7 | smithi123.front.sepia.ceph.com nodename=smithi123 8 | smithi124.front.sepia.ceph.com nodename=smithi124 9 | smithi125.front.sepia.ceph.com nodename=smithi125 10 | smithi127.front.sepia.ceph.com nodename=smithi127 11 | smithi128.front.sepia.ceph.com nodename=smithi128 12 | -------------------------------------------------------------------------------- /ceph-build/build/setup_pbuilder: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | # This file will set the tgz images needed for pbuilder on a given host. It has 3 | # some hard-coded values like `/srv/debian-base` because it gets built every 4 | # time this file is executed - completely ephemeral. If a Debian host will use 5 | # pbuilder, then it will need this. Since it is not idempotent it makes 6 | # everything a bit slower. ## FIXME ## 7 | 8 | set -e 9 | 10 | # Only run when we are a Debian or Debian-based distro 11 | if test -f /etc/redhat-release ; then 12 | exit 0 13 | fi 14 | 15 | setup_pbuilder use_gcc 16 | -------------------------------------------------------------------------------- /ansible/files/ssh/hostkeys/github.com.pub: -------------------------------------------------------------------------------- 1 | github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk= 2 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/ufw.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: install ufw 3 | apt: 4 | name: ufw 5 | state: latest 6 | 7 | - name: only listen to localhost on port 8080 8 | ufw: 9 | port: 8080 10 | src: 127.0.0.1 11 | rule: allow 12 | 13 | - name: allow custom ssh, http, https, and JNLP slave port 14 | ufw: 15 | port: "{{ item }}" 16 | rule: allow 17 | with_items: 18 | - 2222 19 | - 80 20 | - 443 21 | - 49187 22 | 23 | - name: reload ufw 24 | ufw: 25 | state: reloaded 26 | 27 | - name: start ufw 28 | ufw: 29 | state: enabled 30 | -------------------------------------------------------------------------------- /attic/diamond-build/build/setup_pbuilder: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | # This file will set the tgz images needed for pbuilder on a given host. It has 3 | # some hard-coded values like `/srv/debian-base` because it gets built every 4 | # time this file is executed - completely ephemeral. If a Debian host will use 5 | # pbuilder, then it will need this. Since it is not idempotent it makes 6 | # everything a bit slower. ## FIXME ## 7 | 8 | set -e 9 | 10 | # Only run when we are a Debian or Debian-based distro 11 | if test -f /etc/redhat-release ; then 12 | exit 0 13 | fi 14 | 15 | setup_pbuilder 16 | -------------------------------------------------------------------------------- /ceph-dev-build/build/setup_pbuilder: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | # This file will set the tgz images needed for pbuilder on a given host. It has 3 | # some hard-coded values like `/srv/debian-base` because it gets built every 4 | # time this file is executed - completely ephemeral. If a Debian host will use 5 | # pbuilder, then it will need this. Since it is not idempotent it makes 6 | # everything a bit slower. ## FIXME ## 7 | 8 | set -e 9 | 10 | # Only run when we are a Debian or Debian-based distro 11 | if test -f /etc/redhat-release ; then 12 | exit 0 13 | fi 14 | 15 | setup_pbuilder use_gcc 16 | -------------------------------------------------------------------------------- /scripts/setup_chacractl.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | # vim: ts=4 sw=4 expandtab 3 | command -v pipx || ( 4 | command -v apt && sudo apt install -y pipx 5 | command -v dnf && sudo dnf install -y pipx 6 | ) 7 | pipx ensurepath 8 | pipx install uv 9 | ~/.local/bin/uv tool install chacractl 10 | 11 | if [ -z "$chacra_url" ]; then 12 | chacra_url=$(curl -u "$SHAMAN_API_USER:$SHAMAN_API_KEY" https://shaman.ceph.com/api/nodes/next/) 13 | fi 14 | cat > $HOME/.chacractl << EOF 15 | url = "$chacra_url" 16 | user = "$CHACRACTL_USER" 17 | key = "$CHACRACTL_KEY" 18 | ssl_verify = True 19 | EOF 20 | echo $chacra_url 21 | -------------------------------------------------------------------------------- /ceph-volume-unit-tests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # set up variables needed for 5 | # githubstatus to report back to the github PR 6 | # if this project was started manually 7 | github_status_setup 8 | 9 | # the following two methods exist in scripts/build_utils.sh 10 | pkgs=( "tox" "github-status>0.0.3") 11 | TEMPVENV=$(create_venv_dir) 12 | VENV=${TEMPVENV}/bin 13 | install_python_packages $TEMPVENV "pkgs[@]" 14 | 15 | cd src/ceph-volume 16 | 17 | GITHUB_STATUS_STATE="pending" $VENV/github-status create 18 | 19 | $VENV/tox -vr 20 | 21 | GITHUB_STATUS_STATE="success" $VENV/github-status create 22 | -------------------------------------------------------------------------------- /ceph-multibranch-pipeline/config/definitions/ceph-multibranch-pipeline.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - job: 3 | name: ceph-multibranch-pipeline 4 | project-type: multibranch 5 | number-to-keep: 300 6 | days-to-keep: 30 7 | scm: 8 | - github: 9 | repo: ceph 10 | repo-owner: ceph 11 | ssh-checkout: 12 | credentials: 'jenkins-build' 13 | credentials-id: 8cffdeb4-283c-4d96-a190-05d5645bcc2f 14 | clean: 15 | before: true 16 | shallow-clone: true 17 | do-not-fetch-tags: true 18 | discover-pr-forks-trust: contributors 19 | -------------------------------------------------------------------------------- /ceph-api-nightly/build/cleanup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | use_percentage=$(df -m ~ | grep -v Filesystem | awk '{ print $5 }' | cut -d '%' -f1) 3 | if [ $use_percentage -gt 90 ]; then 4 | rm -rf ~/.ccache 5 | for dir in $(ls ~/build/workspace/); do 6 | # Used "${var}" instead of ${var+x} here because I also want to check if the string is empty 7 | if [ -z "${dir}" ] || [ -z "${JOB_NAME}" ]; then 8 | echo "Either \$dir or \$JOB_NAME aren't set. Not cleaning up job directories." 9 | else 10 | if [ "$dir" != "$JOB_NAME" ]; then 11 | rm -rf ~/build/workspace/$dir 12 | fi 13 | fi 14 | done 15 | fi 16 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/templates/pg_hba.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # Database administrative login by Unix domain socket 3 | local all postgres peer 4 | 5 | # TYPE DATABASE USER ADDRESS METHOD 6 | 7 | # "local" is for Unix domain socket connections only 8 | local all all peer 9 | # IPv4 local connections: 10 | host all all 127.0.0.1/0 trust 11 | # IPv6 local connections: 12 | host all all ::1/128 trust 13 | -------------------------------------------------------------------------------- /ceph-windows-image-build/build/cleanup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | BUILD_DIR="$(cd $(dirname "${BASH_SOURCE[0]}") && pwd)" 6 | 7 | source ${BUILD_DIR}/../../scripts/build_utils.sh 8 | 9 | if mountpoint -q -- "${BUILD_DIR}/floppy"; then 10 | sudo umount ${BUILD_DIR}/floppy 11 | fi 12 | 13 | delete_libvirt_vms 14 | clear_libvirt_networks 15 | 16 | sudo rm -rf "${BUILD_DIR}/virtio-win.iso" "${BUILD_DIR}/windows-server-2019.iso" \ 17 | "${BUILD_DIR}/floppy" "${BUILD_DIR}/floppy.img" "${BUILD_DIR}/ceph-win-ltsc2019-ci-image.qcow2" \ 18 | "${BUILD_DIR}/known_hosts" "${BUILD_DIR}/id_rsa.pub" 19 | -------------------------------------------------------------------------------- /ceph-volume-cephadm-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # There has to be a better way to do this than this script which just looks 3 | # for every Vagrantfile in scenarios and then just destroys whatever is left. 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" "github-status>0.0.3" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | GITHUB_STATUS_STATE="failure" $VENV/github-status create 12 | 13 | cd $WORKSPACE/src/ceph-volume/ceph_volume/tests/functional 14 | 15 | # the method exists in scripts/build_utils.sh 16 | teardown_vagrant_tests $VENV 17 | -------------------------------------------------------------------------------- /lab-cop/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -x 5 | 6 | function main() { 7 | # install some of our dependencies 8 | pushd "$WORKSPACE" 9 | git clone gti@github.com:ceph/teuthology 10 | pushd "$WORKSPACE/teuthology" 11 | git remote -v 12 | ./bootstrap 13 | curl -XGET -L paddles.front.sepia.ceph.com/nodes | jq '[.[] | select(.description == null or .description == "None") | select(.locked == true)] | group_by(.locked_by) | .[] | {locked_by: .[0].locked_by, name: [ .[].name | tostring] | join(" ")} | select(.locked_by | tostring| test("scheduled")|not)' 14 | popd 15 | exit $? 16 | } 17 | 18 | main "$@" 19 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/systemd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure /var/log/graphite dir exists 4 | become: true 5 | file: 6 | path: /var/log/graphite 7 | state: directory 8 | owner: _graphite 9 | group: _graphite 10 | recurse: yes 11 | 12 | - name: install the systemd unit file for graphite 13 | template: 14 | src: systemd/graphite.service.j2 15 | dest: /etc/systemd/system/graphite.service 16 | become: true 17 | notify: 18 | - reload systemd 19 | 20 | - name: ensure graphite is enabled and running 21 | become: true 22 | service: 23 | name: graphite 24 | state: running 25 | enabled: yes 26 | -------------------------------------------------------------------------------- /ceph-ansible-prs/build/teardown: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd $WORKSPACE/tests 4 | 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | 8 | echo "========= VAGRANT DEBUGGING =========" 9 | sudo virsh list --all 10 | for net in $(sudo virsh net-list --name); do sudo virsh net-dhcp-leases ${net}; done 11 | sudo journalctl -u libvirtd --pager-end --no-pager 12 | echo "======= END VAGRANT DEBUGGING =======" 13 | 14 | # the method exists in scripts/build_utils.sh 15 | COLLECT_LOGS_PLAYBOOK_PATH="$WORKSPACE/tests/functional/collect-logs.yml" 16 | teardown_vagrant_tests $VENV $COLLECT_LOGS_PLAYBOOK_PATH 17 | 18 | # clean fact cache 19 | rm -rf $HOME/ansible/facts/* 20 | -------------------------------------------------------------------------------- /ceph-ansible-scenario/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | set_centos_python3_version "python3.9" 8 | install_python_packages $TEMPVENV "pkgs[@]" "pip==22.0.4" 9 | 10 | # XXX this might not be needed 11 | source $VENV/activate 12 | 13 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 14 | 15 | prune_stale_vagrant_running_vms 16 | delete_libvirt_vms 17 | clear_libvirt_networks 18 | restart_libvirt_services 19 | update_vagrant_boxes 20 | 21 | # In the same logic, clean fact cache 22 | rm -rf $HOME/ansible/facts/* 23 | 24 | start_tox $TEMPVENV 25 | -------------------------------------------------------------------------------- /scripts/get-tarballs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -x 3 | # Helper to get tarball for releases 4 | 5 | : ${2?"Usage: $0 \$release \$sha1 \$version"} 6 | 7 | release=$1 8 | sha1=$2 9 | version=$3 10 | 11 | pushd /data/download.ceph.com/www/prerelease/ceph/tarballs 12 | 13 | if [[ ! -f ceph_${version}.tar.gz ]]; then 14 | wget -q https://chacra.ceph.com/binaries/ceph/${release}/${sha1}/ubuntu/noble/x86_64/flavors/default/ceph_${version}-1noble.tar.gz \ 15 | || wget -q https://chacra.ceph.com/binaries/ceph/${release}/${sha1}/ubuntu/jammy/x86_64/flavors/default/ceph_${version}-1jammy.tar.gz 16 | 17 | mv ceph_${version}*.tar.gz ceph-${version}.tar.gz 18 | fi 19 | 20 | popd 21 | -------------------------------------------------------------------------------- /ansible/roles/remoto-release/tasks/clear_version.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: undo last commit from failed release 4 | command: git reset --soft HEAD~1 chdir=remoto 5 | when: (clean and last_commit.stdout == tag_name) 6 | 7 | - name: git checkout {{ branch }} branch 8 | command: git checkout {{ branch }} chdir=remoto 9 | 10 | - name: remove local tag 11 | command: git tag -d v{{ version }} chdir=remoto 12 | ignore_errors: yes 13 | 14 | - name: remove remote tag 15 | command: git push jenkins :refs/tags/v{{ version }} chdir=remoto 16 | ignore_errors: yes 17 | 18 | - name: force push changes to jenkins git repo 19 | command: git push -f jenkins {{ branch }} chdir=remoto 20 | -------------------------------------------------------------------------------- /ceph-pr-commits/build/conftest.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | 4 | def pytest_configure(config): 5 | config.addinivalue_line( 6 | "markers", "code_test: mark test to run against code related changes" 7 | ) 8 | config.addinivalue_line( 9 | "markers", "doc_test: mark test to run against doc only changes" 10 | ) 11 | 12 | 13 | def pytest_addoption(parser): 14 | parser.addoption("--skip-code-test", action="store_true", 15 | help="skip code tests") 16 | 17 | 18 | def pytest_runtest_setup(item): 19 | if "code_test" in item.keywords and item.config.getoption("--skip-code-test"): 20 | pytest.skip("skipping due to --skip-code-test") 21 | -------------------------------------------------------------------------------- /sync-images/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # shellcheck disable=SC2034 5 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 6 | 7 | cat << EOF > ./sync.yml 8 | docker.io: 9 | images-by-semver: 10 | nginx: ">= 1.26.0" 11 | grafana/grafana: ">= 9.0.0" 12 | grafana/loki: "= 3.0.0" 13 | grafana/promtail: "= 3.0.0" 14 | maxwo/snmp-notifier: "= v1.2.1" 15 | EOF 16 | # make sure we pull the last stable image 17 | podman pull quay.io/skopeo/stable 18 | podman run --rm --security-opt label=disable -v ./sync.yml:/sync.yml:ro quay.io/skopeo/stable sync --all --src yaml --dest docker /sync.yml "${DEST_REGISTRY}" --dest-username "${DEST_USERNAME}" --dest-password "${DEST_PASSWORD}" 19 | -------------------------------------------------------------------------------- /ansible/templates/systemd/jenkins.service.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | [Unit] 3 | Description=Jenkins Builder 4 | Wants=network.target 5 | After=network.target 6 | 7 | [Install] 8 | WantedBy=multi-user.target 9 | 10 | [Service] 11 | Type=simple 12 | User={{ jenkins_user }} 13 | ExecStart=/usr/bin/java \ 14 | -Dfile.encoding=UTF8 \ 15 | -jar /home/{{ jenkins_user }}/agent.jar \ 16 | -jnlpUrl {{ api_uri }}/computer/{{ ansible_default_ipv4.address }}+{{ nodename }}/slave-agent.jnlp \ 17 | -jnlpCredentials @/etc/systemd/system/jenkins.secret 18 | StandardOutput=journal 19 | StandardError=journal 20 | Restart=always 21 | RestartSec=30 22 | StartLimitInterval=0 23 | -------------------------------------------------------------------------------- /ceph-volume-scenario/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox==4.2.8" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | 12 | delete_libvirt_vms 13 | clear_libvirt_networks 14 | restart_libvirt_services 15 | update_vagrant_boxes 16 | 17 | cd src/ceph-volume/ceph_volume/tests/functional/${DISTRO}/${OBJECTSTORE}/${METHOD}/${SCENARIO} 18 | 19 | CEPH_DEV_BRANCH=$CEPH_BRANCH CEPH_DEV_SHA1=$CEPH_SHA1 $VENV/tox --workdir=$WORKDIR -vre ${DISTRO}-${OBJECTSTORE}-${METHOD}-${OBJECTSTORE}-${SCENARIO} -- --provider=libvirt 20 | -------------------------------------------------------------------------------- /scripts/ceph-website/install-deps.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -x 3 | 4 | # install nvm 5 | if [[ ! $(command -v nvm) ]]; then 6 | # install nvm 7 | LATEST_NVM_VERSION=$(curl -s https://api.github.com/repos/nvm-sh/nvm/releases/latest | jq -r '.tag_name') 8 | echo "Installing nvm version ${LATEST_NVM_VERSION}" 9 | 10 | curl -fsSL https://raw.githubusercontent.com/nvm-sh/nvm/${LATEST_NVM_VERSION}/install.sh | bash 11 | 12 | export NVM_DIR="$HOME/.nvm" 13 | [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" 14 | [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" 15 | fi 16 | 17 | echo "Installing nodejs from nvm with version $(cat .nvmrc)" 18 | nvm install 19 | nvm use 20 | -------------------------------------------------------------------------------- /ceph-dev-new-build/build/setup_pbuilder: -------------------------------------------------------------------------------- 1 | #!/bin/sh -x 2 | # This file will set the tgz images needed for pbuilder on a given host. It has 3 | # some hard-coded values like `/srv/debian-base` because it gets built every 4 | # time this file is executed - completely ephemeral. If a Debian host will use 5 | # pbuilder, then it will need this. Since it is not idempotent it makes 6 | # everything a bit slower. ## FIXME ## 7 | 8 | set -e 9 | 10 | # Only run when we are a Debian or Debian-based distro 11 | if test -f /etc/redhat-release ; then 12 | exit 0 13 | fi 14 | 15 | setup_pbuilder use_gcc 16 | 17 | if [ "$SCCACHE" = true ] ; then 18 | setup_pbuilderrc 19 | setup_sccache_pbuilder_hook 20 | fi 21 | -------------------------------------------------------------------------------- /ansible/roles/ceph-deploy-release/tasks/clear_version.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: undo last commit from failed release 4 | command: git reset --soft HEAD~1 chdir=ceph-deploy 5 | when: (clean and last_commit.stdout == tag_name) 6 | 7 | - name: git checkout {{ branch }} branch 8 | command: git checkout {{ branch }} chdir=ceph-deploy 9 | 10 | - name: remove local tag 11 | command: git tag -d v{{ version }} chdir=ceph-deploy 12 | ignore_errors: yes 13 | 14 | - name: remove remote tag 15 | command: git push jenkins :refs/tags/v{{ version }} chdir=ceph-deploy 16 | ignore_errors: yes 17 | 18 | - name: force push changes to jenkins git repo 19 | command: git push -f jenkins {{ branch }} chdir=ceph-deploy 20 | -------------------------------------------------------------------------------- /nfs-ganesha-stable/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # this job adds custom shaman repositories which can cause issues at build time 4 | # for other jobs so they need to be properly removed 5 | sudo rm -f /etc/yum.repos.d/shaman* 6 | sudo rm -f /etc/apt/sources.list.d/shaman* 7 | 8 | 9 | # note: the failed_build_status call relies on normalized variable names that 10 | # are infered by the builds themselves. If the build fails before these are 11 | # set, they will be posted with empty values 12 | NFS_GANESHA_BRANCH=`branch_slash_filter $NFS_GANESHA_BRANCH` 13 | 14 | # update shaman with the failed build status 15 | failed_build_status "nfs-ganesha-stable" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 16 | -------------------------------------------------------------------------------- /ceph-deploy-tag/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | if [ "$TAG" = false ] ; then 6 | echo "Assuming tagging process has succeeded before because TAG was set to false" 7 | exit 0 8 | fi 9 | 10 | # the following two methods exist in scripts/build_utils.sh 11 | pkgs=( "ansible" ) 12 | TEMPVENV=$(create_venv_dir) 13 | VENV=${TEMPVENV}/bin 14 | install_python_packages $TEMPVENV "pkgs[@]" 15 | 16 | # run ansible to do all the tagging and release specifying 17 | # a local connection and 'localhost' as the host where to execute 18 | cd "$WORKSPACE/ceph-build/ansible/" 19 | $VENV/ansible-playbook -i "localhost," -c local release.yml --extra-vars="version=$VERSION branch=$BRANCH release=stable clean=true project=ceph-deploy" 20 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/files/jenkins.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | server { 4 | 5 | listen 80; 6 | # FIXME 7 | #server_name jenkins.domain.tld; 8 | 9 | location / { 10 | 11 | proxy_set_header Host $host; 12 | proxy_set_header X-Real-IP $remote_addr; 13 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 14 | proxy_set_header X-Forwarded-Proto $scheme; 15 | 16 | # Fix the "It appears that your reverse proxy set up is broken" error. 17 | proxy_pass http://127.0.0.1:8080; 18 | proxy_read_timeout 90; 19 | 20 | # FIXME 21 | #proxy_redirect http://127.0.0.1:8080 https://jenkins.domain.tld; 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /attic/ceph-container-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | set_centos_python3_version "python3.9" 8 | install_python_packages $TEMPVENV "pkgs[@]" "pip==22.0.4" 9 | 10 | # XXX this might not be needed 11 | source $VENV/activate 12 | 13 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 14 | 15 | prune_stale_vagrant_running_vms 16 | delete_libvirt_vms 17 | clear_libvirt_networks 18 | restart_libvirt_services 19 | update_vagrant_boxes 20 | 21 | if ! timeout 3h $VENV/tox -rv -e=$SCENARIO --workdir=$WORKDIR; then 22 | echo "ERROR: Job didn't complete successfully or got stuck for more than 3h." 23 | exit 1 24 | fi 25 | -------------------------------------------------------------------------------- /ansible/roles/graphite/templates/rewrite-rules.conf.j2: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | # This file defines regular expression patterns that can be used to 3 | # rewrite metric names in a search & replace fashion. It consists of two 4 | # sections, [pre] and [post]. The rules in the pre section are applied to 5 | # metric names as soon as they are received. The post rules are applied 6 | # after aggregation has taken place. 7 | # 8 | # The general form of each rule is as follows: 9 | # 10 | # regex-pattern = replacement-text 11 | # 12 | # For example: 13 | # 14 | # [post] 15 | # _sum$ = 16 | # _avg$ = 17 | # 18 | # These rules would strip off a suffix of _sum or _avg from any metric names 19 | # after aggregation. 20 | 21 | [post] 22 | ^{{ graphite_api_key }} = 23 | -------------------------------------------------------------------------------- /chacra-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | # run ansible to get this current host to meet our requirements, specifying 10 | # a local connection and 'localhost' as the host where to execute 11 | cd "$WORKSPACE/ceph-build/chacra-pull-requests/setup/playbooks" 12 | 13 | # make sure any shaman list file is removed. At some point if all nodes 14 | # are clean this will not be needed. 15 | sudo rm -f /etc/apt/sources.list.d/shaman* 16 | 17 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 18 | 19 | cd "$WORKSPACE/chacra" 20 | $VENV/tox -rv 21 | -------------------------------------------------------------------------------- /ansible/roles/ceph-release/tasks/write_sha1_file.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # These tasks write the version commit's sha1 to a file for 3 | # ceph-source-dist and ceph-dev-pipeline to later consume. 4 | # 5 | # We write it again when running the playbook with stage=push only so 6 | # the Archive Artifacts plugin doesn't hang indefinitely. We don't want 7 | # to configure the plugin to allow an empty or missing file. 8 | 9 | - name: record commit sha1 10 | command: git rev-parse HEAD 11 | args: 12 | chdir: ceph 13 | register: commit_sha 14 | 15 | - name: ensure ceph/dist dir exists 16 | file: 17 | path: ceph/dist 18 | state: directory 19 | 20 | - name: save commit sha1 to file 21 | copy: 22 | content: "{{ commit_sha.stdout }}" 23 | dest: ceph/dist/sha1 24 | -------------------------------------------------------------------------------- /ceph-tag/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # remove "-release" from $BRANCH variable in case it was accidentally passed in the Jenkins UI 12 | BRANCH=${BRANCH//-release/} 13 | 14 | # run ansible to do all the tagging and release specifying 15 | # a local connection and 'localhost' as the host where to execute 16 | cd "$WORKSPACE/ceph-build/ansible/" 17 | $VENV/ansible-playbook -i "localhost," -c local release.yml --extra-vars="stage=$TAG_PHASE version=$VERSION branch=$BRANCH force_version=$FORCE_VERSION release=$RELEASE_TYPE tag=$TAG project=ceph token=$GITHUB_TOKEN" 18 | -------------------------------------------------------------------------------- /ceph-setup/build/create_tag: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "ansible" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # remove "-release" from $BRANCH variable in case it was accidentally passed in the Jenkins UI 12 | BRANCH=${BRANCH//-release/} 13 | 14 | # run ansible to do all the tagging and release specifying 15 | # a local connection and 'localhost' as the host where to execute 16 | cd "$WORKSPACE/ceph-build/ansible/" 17 | $VENV/ansible-playbook -i "localhost," -c local release.yml -vvv --extra-vars="stage=create version=$VERSION branch=$BRANCH force_version=$FORCE_VERSION release=$RELEASE_TYPE tag=$TAG throwaway=$THROWAWAY project=ceph" 18 | -------------------------------------------------------------------------------- /ansible/files/ssh/hostkeys/docs.ceph.com.pub: -------------------------------------------------------------------------------- 1 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXvH756ZpNtOTJrg9Exypn5Yb3ABDTqhBvUOOtj7wxzBbXtOYwrvtmls6wMfuHgAOIxjvNIjADkpRYRAuf+TVA= 2 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpJb8vcpiCchL+gRfc749x7/xyo4Gr4VltXFwvK361zlFlfUgpULD1aBpLpAth060QzNBczJymuM6JLCW8d4t0nsKVDKL0dcOySvhZ86tnVKANwDt0S75q0Pd80ClesOZsn+awQ1Rq0eT4dMhLql9PWgSggLOTL+kT8NBFovEIvAiol0uv+L4pVWeJh7FhdMokAHGf7UrbZG/EbjCOvveQmSVLnCUnqxm71y8wQQxGOcLZYdzl2hvIHlI5mJPotx8Pl5QzEs34hF9rltiQ0LJp8gLlFL3ydPW7NK88HxZEgSIwx53v1wJqB6bsf/qxrXOy+Cg4i/i4RjT9ij38Ez8L 3 | docs.ceph.com,newdocs.ceph.com,173.236.253.149 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFay6H/uhGZ9Z4n18vV87OgNp1sIQTuJ4TLMmqPe6Ulk 4 | -------------------------------------------------------------------------------- /kernel/build/prepare_config: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | # Prep the config 5 | echo "Updating the kernel config" 6 | make olddefconfig 7 | 8 | # This way we get the commit hash embedded into the package metadata 9 | # and 'uname -r' in all cases - whether it's some random commit or an 10 | # annotated tag. Example package names: 11 | # uname -r: 4.9.0-rc4-ceph-g156db39ecfbd 12 | # deb: linux-image-4.9.0-rc4-ceph-g156db39ecfbd_4.9.0-rc4-ceph-g156db39ecfbd-1_amd64.deb 13 | # rpm: kernel-4.9.0_rc4_ceph_g156db39ecfbd-2.x86_64.rpm 14 | if ! grep -q "^CONFIG_LOCALVERSION_AUTO=y" .config; then 15 | echo "CONFIG_LOCALVERSION_AUTO is not set, check kernel-config-*.sh" 16 | exit 1 17 | fi 18 | printf -- '-ceph-g%s' ${GIT_COMMIT:0:12} > .scmversion 19 | 20 | kernelrelease=$(make -s kernelrelease) 21 | -------------------------------------------------------------------------------- /ceph-ansible-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # trims leading slashes 12 | BRANCH=${GIT_BRANCH#*/} 13 | 14 | # create the docs build with tox 15 | cd $WORKSPACE/docs/ 16 | $VENV/tox -rv 17 | 18 | # publish docs to http://docs.ceph.com/docs/ceph-ansible/$BRANCH/ create 19 | # a `$BRANCH` dir because the project has stable branches that will 20 | # publish docs that might be different from other versions (similar, 21 | # but not exactly the same to what the Ceph project does) 22 | mkdir -p "/var/ceph-ansible/docs/$BRANCH" 23 | rsync -auv --delete .tox/docs/tmp/html/* "/var/ceph-ansible/docs/$BRANCH/" 24 | -------------------------------------------------------------------------------- /attic/ceph-medic-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | # trims leading slashes 12 | BRANCH=`branch_slash_filter ${GIT_BRANCH}` 13 | 14 | # create the docs build with tox 15 | $VENV/tox -rv -e docs 16 | 17 | # publish docs to http://docs.ceph.com/ceph-medic/$BRANCH/ create 18 | # a `$BRANCH` dir because the project has stable branches that will 19 | # publish docs that might be different from other versions (similar, 20 | # but not exactly the same to what the Ceph project does) 21 | mkdir -p "/var/ceph-medic/docs/$BRANCH" 22 | rsync -auv --delete .tox/docs/tmp/html/* "/var/ceph-medic/docs/$BRANCH/" 23 | 24 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/ssl.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure ssl certs directory 4 | file: 5 | dest: /etc/ssl/certs 6 | state: directory 7 | become: true 8 | 9 | - name: ensure ssl private directory 10 | file: 11 | dest: /etc/ssl/private 12 | state: directory 13 | become: true 14 | 15 | - name: copy SSL cert 16 | copy: 17 | src: "{{ ssl_cert_path }}" 18 | dest: "/etc/ssl/certs/{{ item.fqdn }}-bundled.crt" 19 | mode: 0777 20 | force: no 21 | become: true 22 | notify: restart nginx 23 | when: nginx_hosts is defined 24 | with_items: nginx_hosts 25 | 26 | - name: copy SSL key 27 | copy: 28 | src: "{{ ssl_key_path }}" 29 | dest: "/etc/ssl/private/{{ item.fqdn }}.key" 30 | force: no 31 | become: true 32 | notify: restart nginx 33 | when: nginx_hosts is defined 34 | with_items: nginx_hosts 35 | -------------------------------------------------------------------------------- /attic/ceph-container-prerelease-build/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | sudo apt-get install jq -y 5 | 6 | cd "$WORKSPACE"/ceph-container/ || exit 7 | TMPNAME=$(mktemp) 8 | 9 | ARCH=x86_64 \ 10 | TEST_BUILD_ONLY=true \ 11 | PRERELEASE=true \ 12 | FORCE_BUILD=true \ 13 | X86_64_FLAVORS_TO_BUILD=${X86_64_FLAVORS_TO_BUILD} \ 14 | AARCH64_FLAVORS_TO_BUILD="" \ 15 | FULL_BUILD_TAG_TMPFILE=${TMPNAME} \ 16 | bash -x contrib/build-ceph-base.sh 17 | 18 | imagename=$(<${TMPNAME}) 19 | 20 | # strip leading path components, sub _ for : in name 21 | imagetag=${imagename##*/} 22 | imagetag=${imagetag//:/_} 23 | imagetag=quay.ceph.io/ceph/prerelease:${imagetag} 24 | 25 | docker tag ${imagename} ${imagetag} 26 | docker login --username ${QUAY_CEPH_IO_USERNAME} --password ${QUAY_CEPH_IO_PASSWORD} quay.ceph.io 27 | docker push ${imagetag} 28 | docker rmi ${imagename} 29 | -------------------------------------------------------------------------------- /ceph-volume-nightly/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 4 | 5 | # the following two methods exist in scripts/build_utils.sh 6 | pkgs=( "tox" ) 7 | TEMPVENV=$(create_venv_dir) 8 | VENV=${TEMPVENV}/bin 9 | install_python_packages $TEMPVENV "pkgs[@]" 10 | 11 | prune_stale_vagrant_vms $WORKSPACE/../**/tests 12 | delete_libvirt_vms 13 | clear_libvirt_networks 14 | restart_libvirt_services 15 | update_vagrant_boxes 16 | 17 | cd src/ceph-volume/ceph_volume/tests/functional/$SUBCOMMAND 18 | 19 | if [[ "$CEPH_BRANCH" == "reef" ]]; then 20 | CEPH_ANSIBLE_BRANCH="stable-8.0" 21 | else 22 | CEPH_ANSIBLE_BRANCH="main" 23 | fi 24 | 25 | 26 | VAGRANT_RELOAD_FLAGS="--debug --no-provision" CEPH_ANSIBLE_BRANCH=$CEPH_ANSIBLE_BRANCH CEPH_DEV_BRANCH=$CEPH_BRANCH $VENV/tox --workdir=$WORKDIR -vre $DISTRO-$OBJECTSTORE-$SCENARIO -- --provider=libvirt 27 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/jenkins.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: repo.yml 3 | 4 | - include_tasks: dependencies.yml 5 | 6 | - include_tasks: nginx.yml 7 | 8 | - include_tasks: letsencrypt.yml 9 | tags: 10 | - letsencrypt 11 | 12 | - include_tasks: ufw.yml 13 | tags: 14 | - ufw 15 | 16 | - name: Install Jenkins 17 | apt: 18 | name: jenkins 19 | state: present 20 | register: jenkins_install 21 | 22 | - include_tasks: config.yml 23 | 24 | - include_tasks: plugins.yml 25 | when: okay_with_restart == "y" 26 | tags: 27 | - plugins 28 | 29 | # This should only get run the first time the role is run. 30 | # The variables should be passed as --extra-vars via ansible-playbook command 31 | - include_tasks: auth.yml 32 | when: github_oauth_client is defined and github_oauth_secret is defined 33 | tags: 34 | - auth 35 | 36 | - include_tasks: config.yml 37 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/test-client/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create directory for self-signed cert of docker-registry 3 | file: 4 | path: /etc/docker/certs.d/{{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000 5 | state: directory 6 | 7 | - name: copy self-signed cert of docker-registry 8 | copy: 9 | src: fetch/certs/self.crt 10 | dest: /etc/docker/certs.d/{{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/ca.crt 11 | 12 | - name: pull a small image from docker hub 13 | command: docker pull busybox 14 | 15 | - name: tag image 16 | command: docker tag busybox {{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/mybusybox 17 | 18 | - name: push tagged image to private registry 19 | command: docker push {{ hostvars['docker-registry']['ansible_default_ipv4']['address'] }}:5000/mybusybox 20 | -------------------------------------------------------------------------------- /ansible/roles/kraken/tasks/systemd.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | 4 | - name: ensure /etc/sysconfig/ dir exists 5 | become: true 6 | file: 7 | path: /etc/sysconfig 8 | state: directory 9 | 10 | # prevents issues when updating systemd files 11 | - name: reload systemd 12 | become: yes 13 | command: systemctl daemon-reload 14 | 15 | - name: install the systemd configuration file for celery 16 | template: 17 | src: helga.sysconfig.j2 18 | dest: /etc/sysconfig/helga 19 | become: true 20 | notify: 21 | - reload systemd 22 | 23 | - name: install the systemd unit file for helga 24 | template: 25 | src: helga.service.j2 26 | dest: /etc/systemd/system/helga.service 27 | become: true 28 | notify: 29 | - reload systemd 30 | 31 | - name: ensure helga is enabled and running 32 | become: true 33 | service: 34 | name: helga 35 | state: running 36 | enabled: yes 37 | -------------------------------------------------------------------------------- /cephadm-ansible-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | set_centos_python3_version "python3.9" 8 | install_python_packages $TEMPVENV "pkgs[@]" "pip==22.0.4" 9 | 10 | # XXX this might not be needed 11 | source $VENV/activate 12 | 13 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 14 | 15 | delete_libvirt_vms 16 | clear_libvirt_networks 17 | restart_libvirt_services 18 | update_vagrant_boxes 19 | 20 | rm -rf "${HOME}"/ansible/facts/* 21 | 22 | if [[ -n "$DISTRIBUTION" ]]; then 23 | ENVIRONMENT="${DISTRIBUTION}"-"${SCENARIO}" 24 | else 25 | ENVIRONMENT="${SCENARIO}" 26 | fi 27 | 28 | # Skip the following scenarios as they are not valid: 29 | 30 | [[ "$ghprbTargetBranch" == pacific && "$DISTRIBUTION" == el9 ]] || 31 | "${VENV}"/tox --workdir="${TEMPVENV}" -c tox.ini -e "${ENVIRONMENT}" -r -v -- --provider=libvirt 32 | -------------------------------------------------------------------------------- /ansible/roles/kraken/defaults/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | helga_home: /opt/helga 3 | helga_settings_path: '{{ helga_home }}/bin/settings.d' 4 | helga_nick: helga 5 | helga_irc_host: localhost 6 | helga_irc_port: 6667 7 | helga_use_ssl: yes 8 | helga_operators: [] 9 | helga_irc_channels: 10 | - "#bots" 11 | helga_timezone: 'UTC' 12 | helga_default_plugins: 13 | - dubstep 14 | - facts 15 | - help 16 | - manager 17 | - meant_to_say 18 | - oneliner 19 | - operator 20 | - poems 21 | - reminders 22 | - stfu 23 | 24 | helga_external_plugins: [] 25 | helga_cmd_prefix: '!' 26 | helga_webhooks_port: 8080 27 | helga_twitter_api_key: null 28 | helga_twitter_api_secret: null 29 | helga_twitter_oauth_token: null 30 | helga_twitter_oauth_secret: null 31 | helga_twitter_username: null 32 | helga_system_packages: 33 | - python-devel 34 | - git 35 | - python-virtualenv 36 | - mongodb-server 37 | - gcc-c++ 38 | 39 | helga_ssl_requirements: 40 | - openssl 41 | - openssl-devel 42 | -------------------------------------------------------------------------------- /ansible/roles/grafana/templates/nginx_site.conf: -------------------------------------------------------------------------------- 1 | server { 2 | listen 443 default_server ssl; 3 | server_name {{ fqdn }}; 4 | 5 | ssl_certificate /etc/ssl/certs/{{ fqdn }}-bundled.crt; 6 | ssl_certificate_key /etc/ssl/private/{{ fqdn }}.key; 7 | ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 8 | add_header Strict-Transport-Security "max-age=31536000"; 9 | 10 | access_log /var/log/nginx/{{ app_name }}-access.log; 11 | error_log /var/log/nginx/{{ app_name }}-error.log; 12 | 13 | # Some binaries are gigantic 14 | client_max_body_size 2048m; 15 | 16 | location / { 17 | proxy_set_header Host $host; 18 | proxy_set_header X-Real-IP $remote_addr; 19 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 20 | proxy_set_header X-Forwarded-Proto $scheme; 21 | 22 | proxy_pass http://127.0.0.1:3000; 23 | proxy_read_timeout 500; 24 | } 25 | 26 | } 27 | -------------------------------------------------------------------------------- /ceph-pr-api/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | 3 | if [[ "${ROOT_BUILD_CAUSE}" == "MANUALTRIGGER" ]]; then 4 | ghprbGhRepository="ceph/ceph" 5 | fi 6 | 7 | docs_pr_only 8 | container_pr_only 9 | gha_pr_only 10 | qa_pr_only 11 | if [[ "$DOCS_ONLY" = true || "$CONTAINER_ONLY" = true || "$GHA_ONLY" == true || "$QA_ONLY" == true ]]; then 12 | echo "Only the doc/, container/, qa/ or .github/ dir changed. No need to run make check or API tests." 13 | mkdir -p $WORKSPACE/build/out 14 | echo "File created to avoid Jenkins' Artifact Archiving plugin from hanging" > $WORKSPACE/build/out/mgr.foo.log 15 | exit 0 16 | fi 17 | 18 | n_build_jobs=$(get_nr_build_jobs) 19 | n_test_jobs=$(($(nproc) / 4)) 20 | export CHECK_MAKEOPTS="-j${n_test_jobs} -N -Q" 21 | export BUILD_MAKEOPTS="-j${n_build_jobs}" 22 | export FOR_MAKE_CHECK=1 23 | timeout 2h ./src/script/run-make.sh \ 24 | --cmake-args '-DWITH_TESTS=OFF -DENABLE_GIT_VERSION=OFF' 25 | sleep 5 26 | ps -ef | grep ceph || true 27 | -------------------------------------------------------------------------------- /attic/ceph-dev-container-only/build/failure: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # The ceph-container dir is supposed to get deleted in the build_rpm script. 4 | # We used to add '|| true' to the container build so the dir would still get 5 | # deleted even if it failed. This changed in https://github.com/ceph/ceph-build/pull/1603 6 | # So now we need to delete the directory or the Wipe Workspace plugin will fail on the next build. 7 | cd $WORKSPACE 8 | sudo rm -rf ceph-container 9 | 10 | get_rpm_dist 11 | # note: the failed_build_status call relies on normalized variable names that 12 | # are infered by the builds themselves. If the build fails before these are 13 | # set, they will be posted with empty values 14 | BRANCH=`branch_slash_filter $BRANCH` 15 | 16 | # Normalize variables across rpm/deb builds 17 | NORMAL_DISTRO=$DISTRO 18 | NORMAL_DISTRO_VERSION=$RELEASE 19 | NORMAL_ARCH=$ARCH 20 | 21 | # update shaman with the failed build status 22 | failed_build_status "ceph" $NORMAL_DISTRO $NORMAL_DISTRO_VERSION $NORMAL_ARCH 23 | -------------------------------------------------------------------------------- /wnbd-build/build/setup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | DISTRO="windows" 6 | DISTRO_VERSION="1809" 7 | ARCH="x86_64" 8 | FLAVOR="default" 9 | 10 | BRANCH=`branch_slash_filter $BRANCH` 11 | SHA1="$GIT_COMMIT" 12 | 13 | pkgs=( "chacractl>=0.0.21" ) 14 | TEMPVENV=$(create_venv_dir) 15 | VENV=${TEMPVENV}/bin 16 | install_python_packages $TEMPVENV "pkgs[@]" 17 | 18 | # ask shaman which chacra instance to use 19 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 20 | # create the .chacractl config file using global variables 21 | make_chacractl_config $chacra_url 22 | 23 | chacra_endpoint="wnbd/${BRANCH}/${SHA1}/${DISTRO}/${DISTRO_VERSION}" 24 | chacra_binary_endpoint="${chacra_endpoint}/${ARCH}/flavors/${FLAVOR}" 25 | chacra_repo_endpoint="${chacra_endpoint}/flavors/${FLAVOR}" 26 | chacra_check_url="${chacra_binary_endpoint}/wnbd.zip" 27 | 28 | # create build status in shaman 29 | update_build_status "started" "wnbd" $DISTRO $DISTRO_VERSION $ARCH 30 | -------------------------------------------------------------------------------- /ceph-deploy-docs/config/definitions/ceph-deploy-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-deploy-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-deploy: docs build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: 10 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-deploy 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-deploy 26 | branches: 27 | - main 28 | browser: auto 29 | skip-tag: true 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw-verbatim: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | -------------------------------------------------------------------------------- /attic/ceph-docs/config/definitions/ceph-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/ceph 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph 25 | browser: auto 26 | # The default is to build and publish every branch. 27 | # Uncomment this for testing: 28 | #branches: 29 | # - 'origin/main' 30 | skip-tag: true 31 | timeout: 20 32 | wipe-workspace: true 33 | 34 | builders: 35 | - shell: 36 | !include-raw-verbatim: ../../build/build 37 | -------------------------------------------------------------------------------- /attic/teuthology-docs/config/definitions/teuthology-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: teuthology-docs 3 | disabled: true 4 | node: docs 5 | project-type: freestyle 6 | defaults: global 7 | display-name: 'Teuthology: Docs Build' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - build-discarder: 14 | days-to-keep: -1 15 | num-to-keep: -1 16 | artifact-days-to-keep: -1 17 | artifact-num-to-keep: -1 18 | - github: 19 | url: https://github.com/ceph/teuthology 20 | 21 | triggers: 22 | - github 23 | 24 | scm: 25 | - git: 26 | url: https://github.com/ceph/teuthology.git 27 | branches: 28 | - main 29 | browser: auto 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../setup/setup 37 | - ../../build/build 38 | -------------------------------------------------------------------------------- /ceph-pr-commits/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Don't require signed commits if only docs changed. 4 | # I tried using the excluded-regions parameter for the ghprb plugin but since 5 | # this job/check is required, it hung with 'Expected - Waiting for status to be reported' 6 | docs_pr_only 7 | if [ "$DOCS_ONLY" = false ]; then 8 | echo "Not a docs only change. Will proceed with signed commit check." 9 | pytest_mark="code_test" 10 | elif [ "$DOCS_ONLY" = true ]; then 11 | echo "Only the doc/ dir changed. No need to check for signed commits." 12 | pytest_mark="doc_test" 13 | else 14 | echo "Could not determine if this is a docs only change. Failing job." 15 | exit 1 16 | fi 17 | 18 | # the following two methods exist in scripts/build_utils.sh 19 | pkgs=( "pytest" ) 20 | TEMPVENV=$(create_venv_dir) 21 | VENV=${TEMPVENV}/bin 22 | install_python_packages $TEMPVENV "pkgs[@]" 23 | 24 | cd "$WORKSPACE" 25 | timeout 600 $VENV/py.test -m "${pytest_mark}" -vs --junitxml="$WORKSPACE/report.xml" "$WORKSPACE/ceph-build/ceph-pr-commits/build/test_commits.py" 26 | -------------------------------------------------------------------------------- /ceph-iscsi-trigger/config/definitions/ceph-iscsi-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-iscsi-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/ceph-iscsi 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-iscsi.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/wip*' 28 | skip-tag: true 29 | timeout: 20 30 | wipe-workspace: true 31 | 32 | builders: 33 | - trigger-builds: 34 | - project: 'ceph-iscsi' 35 | predefined-parameters: | 36 | BRANCH=${{GIT_BRANCH}} 37 | FORCE=True 38 | -------------------------------------------------------------------------------- /ceph-volume-cephadm-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | env 4 | WORKDIR=$(mktemp -td tox.XXXXXXXXXX) 5 | 6 | # set up variables needed for 7 | # githubstatus to report back to the github PR 8 | # if this project was started manually 9 | github_status_setup 10 | 11 | # the following two methods exist in scripts/build_utils.sh 12 | pkgs=( "tox" "github-status>0.0.3" ) 13 | TEMPVENV=$(create_venv_dir) 14 | VENV=${TEMPVENV}/bin 15 | set_centos_python3_version "python3.9" 16 | install_python_packages $TEMPVENV "pkgs[@]" "pip==22.0.4" 17 | 18 | GITHUB_STATUS_STATE="pending" $VENV/github-status create 19 | 20 | prune_stale_vagrant_vms $WORKSPACE/../**/tests 21 | delete_libvirt_vms 22 | clear_libvirt_networks 23 | restart_libvirt_services 24 | update_vagrant_boxes 25 | 26 | cd src/ceph-volume/ceph_volume/tests/functional/${DISTRO}/${OBJECTSTORE}/${METHOD}/${SCENARIO} 27 | 28 | CEPH_DEV_BRANCH=$ghprbTargetBranch $VENV/tox --workdir=$WORKDIR -vre ${DISTRO}-${OBJECTSTORE}-${METHOD}-${SCENARIO} -- --provider=libvirt 29 | 30 | GITHUB_STATUS_STATE="success" $VENV/github-status create 31 | -------------------------------------------------------------------------------- /samba-trigger/config/definitions/samba-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: samba-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/samba 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/samba.git 25 | branches: 26 | - 'origin/ceph*' 27 | - 'origin/main*' 28 | - 'origin/wip-*' 29 | skip-tag: true 30 | timeout: 20 31 | wipe-workspace: true 32 | 33 | builders: 34 | - trigger-builds: 35 | - project: 'samba' 36 | predefined-parameters: | 37 | BRANCH=${{GIT_BRANCH}} 38 | FORCE=True 39 | -------------------------------------------------------------------------------- /attic/ceph-pr-render-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # This job is meant to be triggered from a Github Pull Request, only when the 6 | # job is executed in that way a few "special" variables become available. So 7 | # this build script tries to use those first but then it will try to figure it 8 | # out using Git directly so that if triggered manually it can attempt to 9 | # actually work. 10 | PR_ID=$ghprbPullId 11 | 12 | # fallback to just using 'manual' if that ID is not available, for manually 13 | # triggered builds 14 | if [ -z "$ghprbPullId" ]; then 15 | PR_ID="manual" 16 | fi 17 | 18 | ./admin/build-doc 19 | 20 | # publish docs to http://docs.ceph.com/ceph-prs/$PR_ID/ 21 | mkdir -p "/var/ceph-prs/$PR_ID" 22 | rsync -auv --delete build-doc/output/html/* "/var/ceph-prs/$PR_ID/" 23 | 24 | set +e 25 | set +x 26 | 27 | # Cleanup docs rendered 90+ days ago 28 | find /var/ceph-prs/ -mindepth 1 -maxdepth 1 -mtime +90 -exec rm -rvf {} \; 29 | 30 | echo 31 | echo "Docs available to preview at:" 32 | echo 33 | echo " http://docs.ceph.com/ceph-prs/$PR_ID/" 34 | echo 35 | -------------------------------------------------------------------------------- /radosgw-agent-pull-requests/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # the following two methods exist in scripts/build_utils.sh 4 | pkgs=( "ansible" "tox" ) 5 | TEMPVENV=$(create_venv_dir) 6 | VENV=${TEMPVENV}/bin 7 | install_python_packages $TEMPVENV "pkgs[@]" 8 | 9 | 10 | # run ansible to get this current host to meet our requirements, specifying 11 | # a local connection and 'localhost' as the host where to execute. This might 12 | # look odd because we are using ceph-deploy playbooks. But the job-specific 13 | # requirements are the same: install different versions of Python (including 14 | # 2.6 and 2.7) 15 | # 16 | # These job-specific requirements aren't met by the services in charge of 17 | # creating Jenkins builders (mainly prado.ceph.com) because those slaves have "generic" 18 | # requirements and usually do not care about specific needs like Python 2.6 19 | 20 | cd "$WORKSPACE/ceph-build/ceph-deploy-pull-requests/setup/playbooks" 21 | $VENV/ansible-playbook -i "localhost," -c local setup.yml 22 | 23 | 24 | # create the build with tox 25 | cd $WORKSPACE/radosgw-agent 26 | $VENV/tox -rv 27 | -------------------------------------------------------------------------------- /ceph-iscsi-cli-trigger/config/definitions/ceph-iscsi-cli-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-iscsi-cli-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/ceph-iscsi-cli 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-iscsi-cli.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/wip*' 28 | skip-tag: true 29 | timeout: 20 30 | wipe-workspace: true 31 | 32 | builders: 33 | - trigger-builds: 34 | - project: 'ceph-iscsi-cli' 35 | predefined-parameters: | 36 | BRANCH=${{GIT_BRANCH}} 37 | FORCE=True 38 | -------------------------------------------------------------------------------- /ceph-website/config/definitions/ceph-website.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-website 3 | description: This job builds the main branch of https://github.com/ceph/ceph.io and keeps the website up to date 4 | node: www 5 | project-type: freestyle 6 | defaults: global 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | properties: 11 | - build-discarder: 12 | days-to-keep: -1 13 | num-to-keep: 20 14 | artifact-days-to-keep: -1 15 | artifact-num-to-keep: -1 16 | - github: 17 | url: https://github.com/ceph/ceph.io 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph.io 25 | branches: 26 | - main 27 | browser: auto 28 | skip-tag: true 29 | timeout: 20 30 | wipe-workspace: true 31 | 32 | builders: 33 | - shell: 34 | !include-raw-verbatim: 35 | - ../../../scripts/ceph-website/install-deps.sh 36 | - ../../build/build 37 | -------------------------------------------------------------------------------- /shaman-pull-requests/setup/playbooks/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: update apt cache 3 | apt: 4 | update_cache: yes 5 | become: yes 6 | 7 | - name: install postgresql requirements 8 | sudo: yes 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: 13 | - postgresql 14 | - postgresql-common 15 | - postgresql-contrib 16 | - postgresql-server-dev-9.5 17 | - python-psycopg2 18 | tags: 19 | - packages 20 | 21 | - name: ensure database service is up 22 | service: 23 | name: postgresql 24 | state: started 25 | enabled: yes 26 | become: yes 27 | 28 | - name: "Build pg_hba.conf file" 29 | become: true 30 | template: 31 | src: pg_hba.conf.j2 32 | dest: "/etc/postgresql/9.5/main/pg_hba.conf" 33 | 34 | - name: make jenkins-build user 35 | postgresql_user: 36 | name: "jenkins-build" 37 | password: "secret" 38 | role_attr_flags: SUPERUSER 39 | login_user: postgres 40 | become_user: postgres 41 | become: yes 42 | 43 | - service: 44 | name: postgresql 45 | state: restarted 46 | become: yes 47 | -------------------------------------------------------------------------------- /rtslib-fb-trigger/config/definitions/rtslib-fb-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: rtslib-fb-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/rtslib-fb 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/rtslib-fb.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/testing*' 28 | - 'origin/wip*' 29 | skip-tag: true 30 | timeout: 20 31 | wipe-workspace: true 32 | 33 | builders: 34 | - trigger-builds: 35 | - project: 'rtslib-fb' 36 | predefined-parameters: | 37 | BRANCH=${{GIT_BRANCH}} 38 | FORCE=True 39 | -------------------------------------------------------------------------------- /ceph-iscsi-tools-trigger/config/definitions/ceph-iscsi-tools-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-iscsi-tools-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/ceph-iscsi-tools 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-iscsi-tools.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/wip*' 28 | skip-tag: true 29 | timeout: 20 30 | wipe-workspace: true 31 | 32 | builders: 33 | - trigger-builds: 34 | - project: 'ceph-iscsi-tools' 35 | predefined-parameters: | 36 | BRANCH=${{GIT_BRANCH}} 37 | FORCE=True 38 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/tasks/nginx.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: ensure sites-available for nginx 4 | file: 5 | path: /etc/nginx/sites-available 6 | state: directory 7 | 8 | - name: ensure sites-enable for nginx 9 | file: 10 | path: /etc/nginx/sites-enabled 11 | state: directory 12 | 13 | - name: remove default nginx site 14 | file: 15 | path: /etc/nginx/sites-enabled/default 16 | state: absent 17 | 18 | - name: write nginx.conf 19 | template: 20 | src: templates/nginx.conf 21 | dest: /etc/nginx/nginx.conf 22 | 23 | - name: create nginx site config 24 | template: 25 | src: templates/jenkins.conf 26 | dest: /etc/nginx/sites-available/jenkins.conf 27 | notify: 28 | - restart nginx 29 | 30 | - name: link nginx config 31 | file: 32 | src: /etc/nginx/sites-available/jenkins.conf 33 | dest: /etc/nginx/sites-enabled/jenkins.conf 34 | state: link 35 | force: yes 36 | 37 | - name: Enable Nginx service 38 | service: 39 | name: nginx 40 | enabled: yes 41 | state: started 42 | -------------------------------------------------------------------------------- /ceph-iscsi-config-trigger/config/definitions/ceph-iscsi-config-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-iscsi-config-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/ceph-iscsi-config 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/ceph-iscsi-config.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/wip*' 28 | skip-tag: true 29 | timeout: 20 30 | wipe-workspace: true 31 | 32 | builders: 33 | - trigger-builds: 34 | - project: 'ceph-iscsi-config' 35 | predefined-parameters: | 36 | BRANCH=${{GIT_BRANCH}} 37 | FORCE=True 38 | -------------------------------------------------------------------------------- /tcmu-runner-trigger/config/definitions/tcmu-runner-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: tcmu-runner-trigger 3 | node: built-in 4 | project-type: freestyle 5 | defaults: global 6 | quiet-period: 5 7 | block-downstream: false 8 | block-upstream: false 9 | properties: 10 | - build-discarder: 11 | days-to-keep: 1 12 | num-to-keep: 10 13 | artifact-days-to-keep: -1 14 | artifact-num-to-keep: -1 15 | - github: 16 | url: https://github.com/ceph/tcmu-runner 17 | discard-old-builds: true 18 | 19 | triggers: 20 | - github 21 | 22 | scm: 23 | - git: 24 | url: https://github.com/ceph/tcmu-runner.git 25 | branches: 26 | - 'origin/main*' 27 | - 'origin/testing*' 28 | - 'origin/wip*' 29 | skip-tag: true 30 | timeout: 20 31 | wipe-workspace: true 32 | 33 | builders: 34 | - trigger-builds: 35 | - project: 'tcmu-runner' 36 | predefined-parameters: | 37 | BRANCH=${{GIT_BRANCH}} 38 | FORCE=True 39 | -------------------------------------------------------------------------------- /ansible/examples/deploy_grafana.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - hosts: all 4 | user: vagrant 5 | roles: 6 | - nginx 7 | - grafana 8 | vars_files: 9 | - vars/load-balance-vars.yml 10 | vars: 11 | fqdn: "grafana.ceph.com" 12 | app_name: "grafana" 13 | development_server: true 14 | ansible_ssh_port: 22 15 | # These are defined in `vars/load-balance-vars.yml` but defaulting to what 16 | # is used in production. For local development, update these to match the 17 | # hosts used in your local development environment 18 | #nginx_hosts: 19 | # - fqdn: "grafana.ceph.com" 20 | # app_name: "grafana" 21 | # proxy_pass: "http://127.0.0.1:3000" 22 | # - fqdn: "shaman.ceph.com" 23 | # app_name: "shaman" 24 | # upstreams: 25 | # - name: "shaman" 26 | # strategy: "least_conn" 27 | # servers: 28 | # - "1.shaman.ceph.com" 29 | # - "2.shaman.ceph.com" 30 | 31 | # only needed when enabling Github Auth 32 | # github_client_id: "111aaa222" 33 | # github_client_secret: "qwerty1234" 34 | -------------------------------------------------------------------------------- /ceph-grafana-trigger/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | CONTAINER_VERSION=${CONTAINER_VERSION:-9.4.12} 4 | CONTAINER=ceph/ceph-grafana:${CONTAINER_VERSION} 5 | sudo dnf install -y podman 6 | sudo podman login quay.io -u ${CONTAINER_REPO_USERNAME} -p ${CONTAINER_REPO_PASSWORD} 7 | 8 | for repohost in quay.io; do 9 | sudo podman rmi -f grafana:${CONTAINER_VERSION}-combined ${repohost}/${CONTAINER}-x86_64 ${repohost}/${CONTAINER}-aarch64 || true 10 | 11 | sudo podman pull ${repohost}/${CONTAINER}-x86_64 12 | sudo podman pull ${repohost}/${CONTAINER}-aarch64 13 | sudo podman manifest create grafana:${CONTAINER_VERSION}-combined 14 | sudo podman manifest add grafana:${CONTAINER_VERSION}-combined ${repohost}/${CONTAINER}-x86_64 15 | sudo podman manifest add grafana:${CONTAINER_VERSION}-combined ${repohost}/${CONTAINER}-aarch64 16 | 17 | sudo podman manifest push grafana:${CONTAINER_VERSION}-combined ${repohost}/${CONTAINER} 18 | 19 | sudo podman rmi -f grafana:${CONTAINER_VERSION}-combined ${repohost}/${CONTAINER}-x86_64 ${repohost}/${CONTAINER}-aarch64 || true 20 | done 21 | -------------------------------------------------------------------------------- /attic/ceph-medic-docs/config/definitions/ceph-medic-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-medic-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-medic: docs build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: 10 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-medic 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-medic 26 | branches: 27 | - main 28 | # as more stable branches are published, they need to be 29 | # added here 30 | #- stable-1.0 31 | browser: auto 32 | skip-tag: true 33 | timeout: 20 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | -------------------------------------------------------------------------------- /ceph-deploy-build/build/setup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This is the script that runs inside Jenkins. 4 | # http://jenkins.ceph.com/job/ceph-deploy/ 5 | 6 | set -x 7 | set -e 8 | 9 | # ensure dependencies are installed for RPM hosts, because they are required and 10 | # we are not building with a contained environment like we do on DEB builds 11 | 12 | # TODO: use Mock to build ceph-deploy rpm's and avoid this 13 | 14 | if test -f /etc/redhat-release ; then 15 | get_rpm_dist 16 | if [ "$RELEASE" = 7 ]; then 17 | rpm_deps="python-devel python-virtualenv python-mock python-tox pytest" 18 | else 19 | rpm_deps="python3-devel python3-virtualenv python3-mock python3-tox python3-pytest" 20 | fi 21 | sudo yum install -y $rpm_deps 22 | fi 23 | 24 | pkgs=( "chacractl>=0.0.21" ) 25 | TEMPVENV=$(create_venv_dir) 26 | VENV=${TEMPVENV}/bin 27 | install_python_packages $TEMPVENV "pkgs[@]" 28 | 29 | # ask shaman which chacra instance to use 30 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 31 | # create the .chacractl config file using global variables 32 | make_chacractl_config $chacra_url 33 | -------------------------------------------------------------------------------- /ceph-windows-image-build/build/install-windows-updates.ps1: -------------------------------------------------------------------------------- 1 | $ErrorActionPreference = "Stop" 2 | $ProgressPreference='SilentlyContinue' 3 | 4 | Write-Output "Installing PSWindowsUpdate PowerShell module" 5 | Install-PackageProvider -Name "NuGet" -Force -Confirm:$false 6 | Set-PSRepository -Name "PSGallery" -InstallationPolicy Trusted 7 | Install-Module -Name "PSWindowsUpdate" -Force -Confirm:$false 8 | 9 | Write-Output "Installing latest Windows updates" 10 | $updateScript = { 11 | Import-Module "PSWindowsUpdate" 12 | Install-WindowsUpdate -AcceptAll -IgnoreReboot | Out-File -FilePath "${env:SystemDrive}\PSWindowsUpdate.log" -Encoding ascii 13 | } 14 | Invoke-WUJob -Script $updateScript -Confirm:$false -RunNow 15 | while($true) { 16 | $task = Get-ScheduledTask -TaskName "PSWindowsUpdate" 17 | if($task.State -eq "Ready") { 18 | break 19 | } 20 | Start-Sleep -Seconds 10 21 | } 22 | Get-Content "${env:SystemDrive}\PSWindowsUpdate.log" 23 | Remove-Item -Force -Path "${env:SystemDrive}\PSWindowsUpdate.log" 24 | Unregister-ScheduledTask -TaskName "PSWindowsUpdate" -Confirm:$false 25 | Write-Output "Windows updates successfully installed" 26 | -------------------------------------------------------------------------------- /attic/ceph-medic-release/build/build_rpm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # Sanity-check: 6 | [ -z "$GIT_BRANCH" ] && echo Missing GIT_BRANCH variable && exit 1 7 | [ -z "$JOB_NAME" ] && echo Missing JOB_NAME variable && exit 1 8 | 9 | 10 | sudo yum -y install epel-release 11 | sudo yum -y install fedpkg mock 12 | 13 | # Attempt the build. If it fails, print the mock logs to STDOUT. 14 | make rpm || ( tail -n +1 {root,build}.log && exit 1 ) 15 | 16 | # Chacra time 17 | 18 | pkgs=( "chacractl>=0.0.21" ) 19 | TEMPVENV=$(create_venv_dir) 20 | VENV=${TEMPVENV}/bin 21 | install_python_packages $TEMPVENV "pkgs[@]" 22 | 23 | make_chacractl_config 24 | 25 | BRANCH=`branch_slash_filter $GIT_BRANCH` 26 | 27 | ## Upload the created RPMs to chacra 28 | chacra_endpoint="ceph-medic/${BRANCH}/${GIT_COMMIT}/centos/7" 29 | 30 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 31 | 32 | # push binaries to chacra 33 | ls *.rpm | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/noarch/ 34 | 35 | # start repo creation 36 | $VENV/chacractl repo update ${chacra_endpoint} 37 | 38 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 39 | -------------------------------------------------------------------------------- /scripts/nfs-ganesha/sync-push-auto: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script will push repository files from the signer box to the upstream repositories. 3 | # By default it will push all releases and ceph_versions defined in the releases and ceph_version varibles to download.ceph.com 4 | 5 | releases=( V3.5 V2.7 ) 6 | ceph_version=( octopus ceph_pacific ) 7 | 8 | repodirs=$( find /opt/nfs-ganesha/new-repos/ -type d -name x86_64 | cut -d/ -f 13 --complement ) 9 | for dir in "$repodirs"; do 10 | for i in "${releases[@]}"; do 11 | for v in "${ceph_version[@]}"; do 12 | find_release=$( ls -ld "$dir" | grep "$i" | wc -l ) 13 | find_version=$( ls -ld "$dir" | grep "$v" | wc -l ) 14 | if [ $find_release == '1' ] && [ $find_version == '1' ]; then 15 | release=$i 16 | version=$v 17 | ssh signer@download.ceph.com "mkdir -p /data/download.ceph.com/www/nfs-ganesha/rpm-$release-stable/$version/el8" && el8_cmd="$dir/* signer@download.ceph.com:/data/download.ceph.com/www/nfs-ganesha/rpm-$release-stable/$version/el8" && rsync --progress -avr $el8_cmd 18 | rm -rf /opt/nfs-ganesha/new-repos/* 19 | fi 20 | done 21 | done 22 | done 23 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/README.rst: -------------------------------------------------------------------------------- 1 | ansible-jenkins 2 | =============== 3 | 4 | This role will allow you to install a new Jenkins server from scratch or manage an existing instance. 5 | 6 | It assumes the following: 7 | 8 | 1. You've installed a VM with Ubuntu Xenial (16.04) 9 | 2. You're using _`https://github.com/ceph/ceph-sepia-secrets/` as your ansible inventory 10 | 3. You've already run the ``ansible_managed`` and ``common`` roles from https://github.com/ceph/ceph-cm-ansible 11 | 4. You've already generated github oauth application credentials under the Ceph org 12 | 13 | The role is idempotent but it should be noted that the Jenkins service will be restarted when updating or installing plugins. You will be prompted at the beginning of the playbook run if you're okay with restarting the service. 14 | 15 | Initial Installation 16 | -------------------- 17 | 18 | To set up a new Jenkins server from scratch: 19 | 20 | 1. ``cd ceph-build/ansible`` 21 | 2. ``cp examples/controller.yml .`` 22 | 3. ``ansible-playbook controller.yml --limit="new.jenkins.example.com" --extra-vars="{github_oauth_client: 'foo',github_oauth_secret: 'bar'}"`` 23 | 4. Continue with https://github.com/ceph/ceph-sepia-secrets/blob/main/jenkins-controller.rst 24 | -------------------------------------------------------------------------------- /ansible/vars/sensu-vars.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Set the rabbitmq address 3 | rabbitmq_conf_tcp_listeners_address: '0.0.0.0' 4 | 5 | sensu_server_rabbitmq_insecure: true 6 | 7 | # Sensu client variable 8 | 9 | sensu_checks: 10 | cpu: 11 | command: "check-cpu.rb" 12 | interval: 10 13 | subscribers: 14 | - common 15 | disk: 16 | command: "check-disk-usage.rb -w 85 -c 95" 17 | interval: 10 18 | subscribers: 19 | - common 20 | load: 21 | command: "check-load.rb" 22 | interval: 10 23 | subscribers: 24 | - common 25 | memory: 26 | command: "check-memory.rb -w 85 -c 95" 27 | interval: 10 28 | subscribers: 29 | - common 30 | rabbitmq-alive: 31 | command: "check-rabbitmq-amqp-alive.rb -u :::rabbitmq.user|guest::: -p :::rabbitmq.password|guest::: -v :::rabbitmq.vhost|%2F:::" 32 | interval: 10 33 | subscribers: 34 | - rabbitmq 35 | 36 | sensu_ruby_gem_plugins: 37 | - "sensu-plugins-load-checks" 38 | - "sensu-plugins-memory-checks" 39 | - "sensu-plugins-disk-checks" 40 | - "sensu-plugins-cpu-checks" 41 | - "sensu-plugins-rabbitmq" 42 | 43 | # Dummy sensu_handlers 44 | sensu_handlers: 45 | test_handler: 46 | type : pipe 47 | command: "echo" 48 | -------------------------------------------------------------------------------- /sync-images/config/definitions/sync-images.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: sync-images 3 | id: sync-images 4 | node: small && centos9 5 | defaults: global 6 | display-name: sync-images 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: -1 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | 18 | triggers: 19 | - timed: '@daily' 20 | 21 | parameters: 22 | - string: 23 | name: DEST_REGISTRY 24 | description: "The destination registry hostname. Eg: quay.io" 25 | default: "quay.io/ceph" 26 | 27 | builders: 28 | - shell: 29 | !include-raw-verbatim: 30 | - ../../../scripts/build_utils.sh 31 | - ../../build/build 32 | 33 | wrappers: 34 | - inject-passwords: 35 | global: true 36 | mask-password-params: true 37 | - credentials-binding: 38 | - username-password-separated: 39 | credential-id: sync-images-from-docker-to-quay 40 | username: DEST_USERNAME 41 | password: DEST_PASSWORD -------------------------------------------------------------------------------- /scripts/nfs-ganesha/pull-sign-push: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # The script runnings on the signer box will pull nfs-ganesha packags that wore created in the last 24 hours on chacra.ceph.com to /opt/new-repos 3 | # After this the sign-rpms-auto script will run and sign the nfs-ganesha packages 4 | # And finally the sync-push-auto script will run and will push the signed packages to download.ceph.com 5 | 6 | today_items=$(ssh ubuntu@chacra.ceph.com 'find /opt/repos/nfs-ganesha-stable -newermt "-24 hours" -ls' | awk '{ print $11 }' ) 7 | if [ -n "$today_items" ]; then 8 | echo "pulling nfs-ganesha packages from chacra" 9 | echo "********************************************" 10 | [[ -d /opt/nfs-ganesha/new-repos/ ]] | mkdir -p /opt/nfs-ganesha/new-repos/ 11 | for item in $today_items; do 12 | sync_cmd="ubuntu@chacra.ceph.com:$item /opt/nfs-ganesha/new-repos/" 13 | rsync -Lavh --progress --relative $sync_cmd 14 | done 15 | 16 | # sign the rpm's that wore pulled today 17 | 18 | echo "signing rpms" 19 | bash /home/ubuntu/ceph-build/scripts/nfs-ganesha/sign-rpms-auto 20 | 21 | # syncing the singed rpm's to download.ceph.com 22 | 23 | echo "pushing rpms to download.ceph.com" 24 | bash /home/ubuntu/ceph-build/scripts/nfs-ganesha/sync-push-auto 25 | 26 | fi 27 | -------------------------------------------------------------------------------- /ceph-ansible-galaxy/config/definitions/ceph-ansible-galaxy.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-ansible-galaxy 3 | node: small && trusty 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-ansible: Update galaxy roles' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: -1 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-ansible 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-ansible.git 26 | branches: 27 | - main 28 | browser: auto 29 | basedir: "ceph-ansible" 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw-verbatim: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | 38 | wrappers: 39 | - ssh-agent-credentials: 40 | # "jenkins-build" SSH key, needed for access to ceph-ansible.git 41 | users: 42 | - 'jenkins-build' 43 | -------------------------------------------------------------------------------- /ceph-ansible-docs/config/definitions/ceph-ansible-docs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-ansible-docs 3 | node: docs 4 | project-type: freestyle 5 | defaults: global 6 | display-name: 'ceph-ansible: docs build' 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: 10 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-ansible 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-ansible 26 | branches: 27 | - main 28 | - stable-2.1 29 | - stable-2.2 30 | - stable-3.0 31 | - stable-3.1 32 | - stable-3.2 33 | - stable-4.0 34 | - stable-5.0 35 | - stable-6.0 36 | browser: auto 37 | skip-tag: true 38 | timeout: 20 39 | 40 | builders: 41 | - shell: 42 | !include-raw-verbatim: 43 | - ../../../scripts/build_utils.sh 44 | - ../../build/build 45 | -------------------------------------------------------------------------------- /quay-pruner/config/definitions/quay-pruner.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: ceph-build 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/ceph-build.git 6 | branches: 7 | - origin/main 8 | browser-url: https://github.com/ceph/ceph-build 9 | timeout: 20 10 | basedir: "ceph-build" 11 | 12 | 13 | - job: 14 | name: quay-pruner 15 | node: small 16 | project-type: freestyle 17 | defaults: global 18 | display-name: 'Quay: prune container images' 19 | concurrent: true 20 | quiet-period: 5 21 | block-downstream: false 22 | block-upstream: false 23 | retry-count: 3 24 | properties: 25 | - build-discarder: 26 | days-to-keep: 15 27 | artifact-days-to-keep: 15 28 | 29 | triggers: 30 | - timed: '@daily' 31 | 32 | scm: 33 | - ceph-build 34 | 35 | 36 | builders: 37 | - shell: 38 | !include-raw-verbatim: 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | - credentials-binding: 46 | - text: 47 | credential-id: quay-dot-ceph-dot-io-pruner-token 48 | variable: QUAYTOKEN 49 | -------------------------------------------------------------------------------- /ceph-website-prs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | env 5 | 6 | BRANCH=$(echo $GIT_BRANCH | sed 's:.*/::') 7 | 8 | set +e 9 | export NVM_DIR="$HOME/.nvm" 10 | [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" 11 | [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" 12 | set -e 13 | 14 | echo "Using node version $(node -v)" 15 | 16 | # https://docs.npmjs.com/cli/v7/commands/npm-ci 17 | npm ci 18 | 19 | npm run build:development 20 | 21 | if [ "$BRANCH" = "main" ]; then 22 | echo "branch must not be named 'main', exiting" 23 | exit 1 24 | fi 25 | 26 | if [ ! -d /opt/www/${BRANCH} ]; then 27 | mkdir -p /opt/www/${BRANCH} 28 | fi 29 | 30 | rsync -av --delete-after dist/ /opt/www/${BRANCH}/ 31 | 32 | echo "===== Begin pruning old builds =====" 33 | old_builds=$(find /opt/www/ -maxdepth 1 -not -path "/opt/www/main" -type d -mtime +90 | sed 's:.*/::') 34 | for old_build in $old_builds; do 35 | echo $old_build 36 | if [ ! -z "$old_build" ]; then # So we don't accidentally wipe out /opt/www somehow 37 | rm -rf "/opt/www/$old_build" 38 | fi 39 | done 40 | echo "===== Done pruning old builds =====" 41 | 42 | # This just makes the last `echo` line not repeat 43 | { set +x; } 2>/dev/null 44 | 45 | echo "Success! This site is available at https://${BRANCH}.ceph.io." 46 | -------------------------------------------------------------------------------- /cephmetrics-release/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | sudo git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | sudo rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # create the .chacractl config file using global variables 44 | make_chacractl_config 45 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/postgresql.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ensure database service is up 3 | service: 4 | name: postgresql 5 | state: started 6 | enabled: yes 7 | become: yes 8 | 9 | - name: allow users to connect locally 10 | become: yes 11 | lineinfile: 12 | # TODO: should not hardcode that version 13 | dest: /etc/postgresql/9.5/main/pg_hba.conf 14 | regexp: '^host\s+all\s+all\s+127.0.0.1/32' 15 | line: 'host all all 127.0.0.1/32 md5' 16 | backrefs: yes 17 | register: pg_hba_conf 18 | 19 | - service: 20 | name: postgresql 21 | state: restarted 22 | become: true 23 | when: pg_hba_conf.changed 24 | 25 | - name: make {{ app_name }} user 26 | postgresql_user: 27 | name: "{{ app_name }}" 28 | password: "{{ db_password.stdout }}" 29 | role_attr_flags: SUPERUSER 30 | login_user: postgres 31 | become_user: postgres 32 | become: yes 33 | 34 | - name: Make {{ app_name }} database 35 | postgresql_db: 36 | name: "{{ app_name }}" 37 | owner: "{{ app_name }}" 38 | state: present 39 | login_user: postgres 40 | become_user: postgres 41 | become: yes 42 | 43 | - name: ensure database service is up 44 | service: 45 | name: postgresql 46 | state: started 47 | enabled: yes 48 | become: yes 49 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/jenkins.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | server { 4 | listen 80 default_server; 5 | listen 443 default_server ssl; 6 | 7 | server_name {{ inventory_hostname }}; 8 | 9 | ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 10 | add_header Strict-Transport-Security "max-age=31536000"; 11 | 12 | access_log /var/log/nginx/jenkins_access.log; 13 | error_log /var/log/nginx/jenkins_error.log; 14 | 15 | location '/.well-known/acme-challenge' { 16 | default_type "text/plain"; 17 | root /var/www/{{ inventory_hostname }}; 18 | } 19 | 20 | location / { 21 | proxy_set_header Host $host; 22 | proxy_set_header X-Real-IP $remote_addr; 23 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 24 | proxy_set_header X-Forwarded-Proto $scheme; 25 | 26 | proxy_pass http://127.0.0.1:{{ jenkins_port }}; 27 | proxy_read_timeout 180; 28 | 29 | # http://tracker.ceph.com/issues/18176 30 | proxy_buffer_size 128k; 31 | proxy_buffers 4 256k; 32 | proxy_busy_buffers_size 256k; 33 | 34 | # Redirect all plaintext HTTP to HTTPS 35 | if ($scheme != "https") { 36 | rewrite ^ https://$host$uri permanent; 37 | } 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/carbon.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: enable carbon 4 | lineinfile: 5 | dest: /etc/default/graphite-carbon 6 | regexp: "^CARBON_CACHE_ENABLED=false" 7 | line: "CARBON_CACHE_ENABLED=true" 8 | state: present 9 | become: true 10 | 11 | - name: enable whitelisting in carbon 12 | lineinfile: 13 | dest: /etc/carbon/carbon.conf 14 | regexp: "^# USE_WHITELIST = False" 15 | line: "USE_WHITELIST = True" 16 | state: present 17 | backrefs: true 18 | become: true 19 | 20 | - name: create the rewrite config with the secret api key 21 | template: 22 | src: ../templates/rewrite-rules.conf.j2 23 | dest: "/etc/carbon/rewrite-rules.conf" 24 | notify: 25 | - restart carbon 26 | become: true 27 | 28 | - name: create the whitelist/blacklist config allowing the api key only 29 | template: 30 | src: ../templates/whitelist.conf.j2 31 | dest: "/etc/carbon/whitelist.conf" 32 | notify: 33 | - restart carbon 34 | become: true 35 | 36 | - name: define the storage schemas 37 | template: 38 | src: ../templates/storage-schemas.conf.j2 39 | dest: "/etc/carbon/storage-schemas.conf" 40 | notify: 41 | - restart carbon 42 | become: true 43 | 44 | - name: ensure database service is up 45 | service: 46 | name: carbon-cache 47 | state: restarted 48 | enabled: yes 49 | become: yes 50 | -------------------------------------------------------------------------------- /ansible/roles/graphite/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: "Build hosts file" 4 | become: yes 5 | lineinfile: 6 | dest: /etc/hosts 7 | regexp: ".*{{ fqdn }}$" 8 | line: "127.0.1.1 {{ fqdn }}" 9 | state: present 10 | 11 | - name: Set Hostname with hostname command 12 | become: yes 13 | hostname: name="{{ fqdn }}" 14 | 15 | - name: update apt cache 16 | apt: 17 | update_cache: yes 18 | become: yes 19 | 20 | - name: install ssl system requirements 21 | become: yes 22 | apt: 23 | name: "{{ item }}" 24 | state: present 25 | with_items: ssl_requirements 26 | tags: 27 | - packages 28 | 29 | - name: install system packages 30 | become: yes 31 | apt: 32 | name: "{{ item }}" 33 | state: present 34 | with_items: system_packages 35 | tags: 36 | - packages 37 | 38 | - command: cp /usr/share/graphite-web/graphite.wsgi /usr/lib/python2.7/dist-packages/graphite/graphite_web.py 39 | args: 40 | creates: "/usr/lib/python2.7/dist-packages/graphite/graphite_web.py" 41 | become: true 42 | 43 | - include_tasks: carbon.yml 44 | 45 | - include_tasks: systemd.yml 46 | tags: 47 | - systemd 48 | 49 | - include_tasks: postgresql.yml 50 | tags: 51 | - postgresql 52 | 53 | - name: ensure graphite is running 54 | become: true 55 | service: 56 | name: graphite 57 | state: restarted 58 | enabled: yes 59 | -------------------------------------------------------------------------------- /ansible/roles/remoto-release/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: check if remoto repo exists 4 | stat: path='./remoto' 5 | register: 'cdep_repo' 6 | 7 | - name: clone the remoto repository 8 | git: repo=git@github.com:ceph/remoto dest=remoto 9 | when: cdep_repo.stat.exists is defined and cdep_repo.stat.exists == false 10 | 11 | - name: rename origin to jenkins 12 | command: git remote rename origin jenkins chdir=remoto 13 | ignore_errors: yes 14 | 15 | - name: fetch the latest from remote 16 | command: git fetch jenkins chdir=remoto 17 | 18 | - name: ensure local repo is in sync with remote 19 | command: git reset --hard jenkins/{{ branch }} chdir=remoto 20 | 21 | - name: check if we are re-pushing the release commit 22 | command: git log -1 --pretty=%B chdir=remoto 23 | register: 'last_commit' 24 | 25 | # we probably messed up the previous commit+tag, so we chose to use 'clean' 26 | # that will rollback that commit, delete the local and remote tag, and force 27 | # push the new changes 28 | - include_tasks: clear_version.yml 29 | when: (clean and last_commit.stdout == tag_name) 30 | 31 | # if the last commit wasn't one that we already did, then go ahead and make 32 | # the changes + tag for the release. Otherwise, just skip because it was 33 | # already done for this release 34 | - include_tasks: release.yml 35 | when: (tag_name != last_commit.stdout) 36 | -------------------------------------------------------------------------------- /mita-deploy/config/definitions/mita-deploy.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: mita 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/mita.git 6 | branches: 7 | - main 8 | browser: auto 9 | timeout: 20 10 | skip-tag: true 11 | wipe-workspace: true 12 | 13 | - job: 14 | name: mita-deploy 15 | node: built-in 16 | description: "This job clones mita and deploys it to its production server based on the BRANCH value" 17 | display-name: 'mita-deploy' 18 | block-downstream: false 19 | block-upstream: false 20 | properties: 21 | - build-discarder: 22 | days-to-keep: -1 23 | num-to-keep: 25 24 | artifact-days-to-keep: -1 25 | artifact-num-to-keep: -1 26 | - github: 27 | url: https://github.com/ceph/mita 28 | 29 | parameters: 30 | - string: 31 | name: BRANCH 32 | description: "The git branch (or tag) to build, defaults to 'main'" 33 | default: "main" 34 | scm: 35 | - mita 36 | 37 | triggers: 38 | - github 39 | 40 | builders: 41 | - shell: 42 | !include-raw-verbatim: 43 | - ../../../scripts/build_utils.sh 44 | - ../../build/build 45 | 46 | wrappers: 47 | - inject-passwords: 48 | global: true 49 | mask-password-params: true 50 | -------------------------------------------------------------------------------- /ceph-dev-pipeline-trigger/config/definitions/ceph-dev-pipeline-trigger.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-dev-pipeline-trigger 3 | project-type: pipeline 4 | quiet-period: 1 5 | concurrent: true 6 | pipeline-scm: 7 | scm: 8 | - git: 9 | url: https://github.com/ceph/ceph-build 10 | branches: 11 | - main 12 | shallow-clone: true 13 | submodule: 14 | disable: true 15 | wipe-workspace: true 16 | script-path: ceph-dev-pipeline-trigger/build/Jenkinsfile 17 | lightweight-checkout: true 18 | do-not-fetch-tags: true 19 | 20 | triggers: 21 | - generic-webhook-trigger: 22 | token: ceph-dev-pipeline-trigger 23 | token-credential-id: pipeline-trigger-token 24 | print-contrib-var: true 25 | header-params: 26 | - key: X_GitHub_Event 27 | value: "" 28 | post-content-params: 29 | - type: JSONPath 30 | key: head_commit 31 | value: $.head_commit.message 32 | - type: JSONPath 33 | key: ref 34 | value: $.ref 35 | - type: JSONPath 36 | key: pusher 37 | value: $.pusher.name 38 | regex-filter-text: $head_commit 39 | regex-filter-expression: "(?i)CI-PIPELINE: true" 40 | cause: "Push to $ref by $pusher" 41 | -------------------------------------------------------------------------------- /ansible/roles/ceph-deploy-release/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: check if ceph-deploy repo exists 4 | stat: path='./ceph-deploy' 5 | register: 'cdep_repo' 6 | 7 | - name: clone the ceph-deploy repository 8 | git: repo=git@github.com:ceph/ceph-deploy dest=ceph-deploy 9 | when: cdep_repo.stat.exists is defined and cdep_repo.stat.exists == false 10 | 11 | - name: rename origin to jenkins 12 | command: git remote rename origin jenkins chdir=ceph-deploy 13 | ignore_errors: yes 14 | 15 | - name: fetch the latest from remote 16 | command: git fetch jenkins chdir=ceph-deploy 17 | 18 | - name: ensure local repo is in sync with remote 19 | command: git reset --hard jenkins/{{ branch }} chdir=ceph-deploy 20 | 21 | - name: check if we are re-pushing the release commit 22 | command: git log -1 --pretty=%B chdir=ceph-deploy 23 | register: 'last_commit' 24 | 25 | # we probably messed up the previous commit+tag, so we chose to use 'clean' 26 | # that will rollback that commit, delete the local and remote tag, and force 27 | # push the new changes 28 | - include_tasks: clear_version.yml 29 | when: (clean and last_commit.stdout == tag_name) 30 | 31 | # if the last commit wasn't one that we already did, then go ahead and make 32 | # the changes + tag for the release. Otherwise, just skip because it was 33 | # already done for this release 34 | - include_tasks: release.yml 35 | when: (tag_name != last_commit.stdout) 36 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs/config/definitions/ceph-container-build-push-imgs.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-container-build-push-imgs 3 | node: huge && trusty && x86_64 4 | project-type: freestyle 5 | defaults: global 6 | name: ceph-container-build-push-imgs 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: -1 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-container 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-container.git 26 | branches: 27 | - main 28 | browser: auto 29 | basedir: "ceph-container" 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | 38 | wrappers: 39 | - inject-passwords: 40 | global: true 41 | mask-password-params: true 42 | - credentials-binding: 43 | - username-password-separated: 44 | credential-id: ceph-container-quay-io 45 | username: REGISTRY_USERNAME 46 | password: REGISTRY_PASSWORD 47 | -------------------------------------------------------------------------------- /attic/ceph-medic-rpm/config/definitions/ceph-medic-rpm.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-medic-rpm 3 | node: 'centos7 && x86_64 && small && !sepia' 4 | project-type: freestyle 5 | defaults: global 6 | disabled: false 7 | display-name: 'ceph-medic: RPMs' 8 | description: 'Build RPMs for every ceph-medic Git branch' 9 | concurrent: true 10 | quiet-period: 5 11 | block-downstream: false 12 | block-upstream: false 13 | retry-count: 3 14 | properties: 15 | - build-discarder: 16 | days-to-keep: 1 17 | num-to-keep: 10 18 | artifact-days-to-keep: -1 19 | artifact-num-to-keep: -1 20 | - github: 21 | url: https://github.com/ceph/ceph-medic 22 | discard-old-builds: true 23 | 24 | triggers: 25 | - github 26 | 27 | scm: 28 | - git: 29 | url: https://github.com/ceph/ceph-medic 30 | browser: auto 31 | skip-tag: true 32 | timeout: 20 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | - credentials-binding: 46 | - text: 47 | credential-id: shaman-api-key 48 | variable: SHAMAN_API_KEY 49 | -------------------------------------------------------------------------------- /rtslib-fb/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /ceph-iscsi-cli/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /ceph-iscsi/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /configshell-fb/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /tcmu-runner/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /ceph-iscsi-config/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /ceph-iscsi-tools/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-arm64/config/definitions/ceph-container-build-push-imgs-arm64.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-container-build-push-imgs-arm64 3 | node: arm64 && xenial 4 | project-type: freestyle 5 | defaults: global 6 | name: ceph-container-build-push-imgs-arm64 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: -1 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-container 19 | 20 | triggers: 21 | - github 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-container.git 26 | branches: 27 | - main 28 | browser: auto 29 | basedir: "ceph-container" 30 | timeout: 20 31 | 32 | builders: 33 | - shell: 34 | !include-raw: 35 | - ../../../scripts/build_utils.sh 36 | - ../../build/build 37 | 38 | wrappers: 39 | - inject-passwords: 40 | global: true 41 | mask-password-params: true 42 | - credentials-binding: 43 | - username-password-separated: 44 | credential-id: ceph-container-quay-io 45 | username: REGISTRY_USERNAME 46 | password: REGISTRY_PASSWORD 47 | -------------------------------------------------------------------------------- /cephmetrics/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | sudo git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | sudo rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | # ask shaman which chacra instance to use 44 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 45 | # create the .chacractl config file using global variables 46 | make_chacractl_config $chacra_url 47 | -------------------------------------------------------------------------------- /scripts/ceph-windows/setup_libvirt_windows_vm: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | if [[ -z $LIBVIRT_DIR ]]; then echo "ERROR: The env variable LIBVIRT_DIR is not set"; exit 1; fi 6 | 7 | export WINDOWS_VM_IMAGE_URL=${WINDOWS_VM_IMAGE_URL:-"https://apt-mirror.sepia.ceph.com/windows/ceph-win-ltsc2019-ci-image.qcow2"} 8 | export WINDOWS_VM_NAME=${WINDOWS_VM_NAME:-"ceph-windows-client-${JOB_NAME}-${BUILD_ID}"} 9 | export WINDOWS_VM_VCPUS="32" 10 | export WINDOWS_VM_MEMORY="32768" # 8GB 11 | export WINDOWS_SSH_USER="administrator" 12 | 13 | # 14 | # Setup the Windows VM to run Ceph client 15 | # 16 | mkdir -p $LIBVIRT_DIR 17 | echo "Downloading VM image from $WINDOWS_VM_IMAGE_URL" 18 | curl -s -L $WINDOWS_VM_IMAGE_URL -o ${LIBVIRT_DIR}/ceph-windows-client.qcow2 19 | 20 | sudo virt-install \ 21 | --name $WINDOWS_VM_NAME \ 22 | --os-variant win2k19 \ 23 | --import \ 24 | --virt-type kvm \ 25 | --graphics spice \ 26 | --cpu host \ 27 | --vcpus $WINDOWS_VM_VCPUS \ 28 | --memory $WINDOWS_VM_MEMORY \ 29 | --disk ${LIBVIRT_DIR}/ceph-windows-client.qcow2,bus=virtio \ 30 | --network network=default,model=virtio \ 31 | --controller type=virtio-serial \ 32 | --channel unix,target_type=virtio,name=org.qemu.guest_agent.0 \ 33 | --noautoconsol 34 | 35 | # 36 | # Get the VM SSH address 37 | # 38 | export WINDOWS_VM_IP=$(VM_NAME=$WINDOWS_VM_NAME SSH_USER=$WINDOWS_SSH_USER get_libvirt_vm_ssh_address) 39 | -------------------------------------------------------------------------------- /scripts/ceph-windows/win32_build: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o errexit 3 | set -o pipefail 4 | 5 | if [[ -z $SSH_KEY ]]; then echo "ERROR: The SSH_KEY env variable is not set"; exit 1; fi 6 | if [[ -z $SSH_KNOWN_HOSTS_FILE ]]; then echo "ERROR: The SSH_KNOWN_HOSTS_FILE env variable is not set"; exit 1; fi 7 | 8 | if [[ -z $UBUNTU_SSH_USER ]]; then echo "ERROR: The UBUNTU_SSH_USER env variable is not set"; exit 1; fi 9 | if [[ -z $UBUNTU_VM_IP ]]; then echo "ERROR: The UBUNTU_VM_IP env variable is not set"; exit 1; fi 10 | 11 | export SSH_USER=$UBUNTU_SSH_USER 12 | export SSH_ADDRESS=$UBUNTU_VM_IP 13 | 14 | function rsync_cmd() { 15 | rsync -a --delete -e "ssh -i $SSH_KEY -o UserKnownHostsFile=$SSH_KNOWN_HOSTS_FILE" "${@}" 16 | } 17 | 18 | # 19 | # Build Ceph Windows 20 | # 21 | cat > ${WORKSPACE}/build-ceph-windows.sh << EOF 22 | #!/usr/bin/env bash 23 | set -o errexit 24 | set -o pipefail 25 | 26 | cd ~/ceph 27 | 28 | sudo apt-get update -o Acquire::Languages=none -o Acquire::Translation=none || true 29 | sudo apt-get install -y git 30 | git submodule update --init --recursive 31 | 32 | ZIP_DEST=~/ceph.zip $CEPH_WIN32_BUILD_FLAGS timeout 3h ./win32_build.sh 33 | EOF 34 | chmod +x ${WORKSPACE}/build-ceph-windows.sh 35 | time rsync_cmd $WORKSPACE/ceph ${WORKSPACE}/build-ceph-windows.sh ${UBUNTU_SSH_USER}@${UBUNTU_VM_IP}: 36 | 37 | time SSH_TIMEOUT=3h ssh_exec ./build-ceph-windows.sh 38 | time rsync_cmd ${UBUNTU_SSH_USER}@${UBUNTU_VM_IP}:~/ceph.zip $WORKSPACE/ceph.zip 39 | -------------------------------------------------------------------------------- /ceph-ansible-rpm/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # Sanity-check: 6 | [ -z "$GIT_BRANCH" ] && echo Missing GIT_BRANCH variable && exit 1 7 | [ -z "$JOB_NAME" ] && echo Missing JOB_NAME variable && exit 1 8 | 9 | # Strip "-rpm" off the job name to get our package's name 10 | PACKAGE=${JOB_NAME%-rpm} 11 | 12 | sudo yum -y install epel-release 13 | sudo yum -y install fedpkg mock 14 | 15 | # Attempt the build. If it fails, print the mock logs to STDOUT. 16 | make rpm || ( tail -n +1 {root,build}.log && exit 1 ) 17 | 18 | # Chacra time 19 | 20 | pkgs=( "chacractl>=0.0.21" ) 21 | TEMPVENV=$(create_venv_dir) 22 | VENV=${TEMPVENV}/bin 23 | install_python_packages $TEMPVENV "pkgs[@]" 24 | 25 | # ask shaman which chacra instance to use 26 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 27 | # create the .chacractl config file using global variables 28 | make_chacractl_config $chacra_url 29 | 30 | BRANCH=`branch_slash_filter $GIT_BRANCH` 31 | 32 | ## Upload the created RPMs to chacra 33 | chacra_endpoint="${PACKAGE}/${BRANCH}/${GIT_COMMIT}/centos/7" 34 | 35 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 36 | 37 | # push binaries to chacra 38 | ls *.rpm | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/noarch/ 39 | 40 | # start repo creation 41 | $VENV/chacractl repo update ${chacra_endpoint} 42 | 43 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 44 | -------------------------------------------------------------------------------- /attic/ceph-medic-rpm/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | # Sanity-check: 6 | [ -z "$GIT_BRANCH" ] && echo Missing GIT_BRANCH variable && exit 1 7 | [ -z "$JOB_NAME" ] && echo Missing JOB_NAME variable && exit 1 8 | 9 | # Strip "-rpm" off the job name to get our package's name 10 | PACKAGE=${JOB_NAME%-rpm} 11 | 12 | sudo yum -y install epel-release 13 | sudo yum -y install fedpkg mock 14 | 15 | # Attempt the build. If it fails, print the mock logs to STDOUT. 16 | make rpm || ( tail -n +1 {root,build}.log && exit 1 ) 17 | 18 | # Chacra time 19 | 20 | pkgs=( "chacractl>=0.0.21" ) 21 | TEMPVENV=$(create_venv_dir) 22 | VENV=${TEMPVENV}/bin 23 | install_python_packages $TEMPVENV "pkgs[@]" 24 | 25 | # ask shaman which chacra instance to use 26 | chacra_url=`curl -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 27 | # create the .chacractl config file using global variables 28 | make_chacractl_config $chacra_url 29 | 30 | BRANCH=`branch_slash_filter $GIT_BRANCH` 31 | 32 | ## Upload the created RPMs to chacra 33 | chacra_endpoint="${PACKAGE}/${BRANCH}/${GIT_COMMIT}/centos/7" 34 | 35 | [ "$FORCE" = true ] && chacra_flags="--force" || chacra_flags="" 36 | 37 | # push binaries to chacra 38 | ls *.rpm | $VENV/chacractl binary ${chacra_flags} create ${chacra_endpoint}/noarch/ 39 | 40 | # start repo creation 41 | $VENV/chacractl repo update ${chacra_endpoint} 42 | 43 | echo Check the status of the repo at: https://shaman.ceph.com/api/repos/${chacra_endpoint} 44 | -------------------------------------------------------------------------------- /ceph-pull-requests/build/kill-tests: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | # kill all descendant processes of ctest 4 | 5 | # ceph-pull-requests/build/build is killed by jenkins when the ceph-pull-requests job is aborted or 6 | # canceled, see https://www.jenkins.io/doc/book/using/aborting-a-build/ . but build/build does not 7 | # wait until all its children processes quit. after ctest is killed by SIGTERM, there is chance 8 | # that some tests are still running as ctest does not get a chance to kill them before it terminates. 9 | # if these tests had timed out, ctest would kill them using SIGKILL. so we need to kill them 10 | # manually after the job is aborted. 11 | 12 | # if ctest is still running, get its pid, otherwise we are done. 13 | ctest_pid=$(pgrep ctest) || exit 0 14 | # the parent process of ctest should have been terminated, but this might not be true when 15 | # it comes to some of its descendant processes, for instance, unittest-seastar-messenger 16 | ctest_pgid=$(ps --no-headers --format 'pgid:1' --pid $ctest_pid) 17 | kill -SIGTERM -- -"$ctest_pgid" 18 | # try harder 19 | for seconds in 0 1 1 2 3; do 20 | sleep $seconds 21 | if pgrep --pgroup $ctest_pgid > /dev/null; then 22 | # kill only if we've waited for a while 23 | if test $seconds != 0; then 24 | pgrep --pgroup $ctest_pgid 25 | echo 'try harder' 26 | kill -SIGKILL -- -"$ctest_pgid" 27 | fi 28 | else 29 | echo 'killed' 30 | break 31 | fi 32 | done 33 | -------------------------------------------------------------------------------- /scripts/dashboard/install-e2e-test-deps.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -ex 3 | 4 | if [[ ! $(arch) =~ (i386|x86_64|amd64) ]]; then 5 | # google chrome is only available on amd64 6 | exit 7 | fi 8 | 9 | if grep -q debian /etc/*-release; then 10 | sudo bash -c 'echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list' 11 | curl -fsSL https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add - 12 | sudo apt-get update 13 | sudo apt-get install -y google-chrome-stable 14 | sudo apt-get install -y python3-requests python3-openssl python3-jinja2 \ 15 | python3-jwt python3-scipy python3-routes 16 | sudo apt-get install -y xvfb libxss1 17 | sudo rm /etc/apt/sources.list.d/google-chrome.list 18 | elif grep -q rhel /etc/*-release; then 19 | sudo dd of=/etc/yum.repos.d/google-chrome.repo status=none < /dev/null; then 22 | # kill only if we've waited for a while 23 | if test $seconds != 0; then 24 | pgrep --pgroup $ctest_pgid 25 | echo 'try harder' 26 | kill -SIGKILL -- -"$ctest_pgid" 27 | fi 28 | else 29 | echo 'killed' 30 | break 31 | fi 32 | done 33 | -------------------------------------------------------------------------------- /ansible/roles/ansible-jenkins/templates/nginx.conf: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | user www-data; 3 | worker_processes {{ nginx_processor_count }}; 4 | worker_rlimit_nofile 8192; 5 | 6 | pid /var/run/nginx.pid; 7 | 8 | events { 9 | worker_connections {{ nginx_connections }} ; 10 | # multi_accept on; 11 | } 12 | 13 | http { 14 | 15 | ## 16 | # Basic Settings 17 | ## 18 | 19 | #sendfile on; 20 | tcp_nopush on; 21 | tcp_nodelay on; 22 | keepalive_timeout 65; 23 | types_hash_max_size 2048; 24 | server_tokens off; 25 | 26 | # server_names_hash_bucket_size 64; 27 | # server_name_in_redirect off; 28 | 29 | include /etc/nginx/mime.types; 30 | default_type application/octet-stream; 31 | 32 | ## 33 | # Logging Settings 34 | ## 35 | 36 | access_log /var/log/nginx/access.log; 37 | error_log /var/log/nginx/error.log; 38 | 39 | ## 40 | # Gzip Settings 41 | ## 42 | 43 | gzip on; 44 | gzip_disable "msie6"; 45 | 46 | # gzip_vary on; 47 | # gzip_proxied any; 48 | # gzip_comp_level 6; 49 | # gzip_buffers 16 8k; 50 | # gzip_http_version 1.1; 51 | # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; 52 | 53 | ## 54 | # If HTTPS, then set a variable so it can be passed along. 55 | ## 56 | 57 | map $scheme $server_https { 58 | default off; 59 | https on; 60 | } 61 | 62 | ## 63 | # Virtual Host Configs 64 | ## 65 | 66 | include /etc/nginx/conf.d/*.conf; 67 | include /etc/nginx/sites-enabled/*; 68 | } 69 | 70 | -------------------------------------------------------------------------------- /ceph-windows-image-build/README.md: -------------------------------------------------------------------------------- 1 | # Ceph Windows Image Build 2 | 3 | The Windows image can be generated, fully unattended, via the build script: 4 | 5 | ```bash 6 | ./build/build 7 | ``` 8 | 9 | It accepts the following environment variables: 10 | 11 | * `SSH_PRIVATE_KEY` (required) - The SSH private key path that will be authorized to access the VMs using the new image. 12 | * `WINDOWS_SERVER_2019_ISO_URL` (optional) - URL to the Windows Server 2019 ISO image. It defaults to the official Microsoft evaluation ISO. 13 | * `VIRTIO_WIN_ISO_URL` (optional) - URL to the virtio-win guest tools ISO image. It defaults to the stable ISO from the [official docs](https://github.com/virtio-win/virtio-win-pkg-scripts/blob/master/README.md#downloads). 14 | 15 | The build script assumes that the host is a KVM enabled machine, and it will do the following: 16 | 17 | 1. Download the ISOs from the URLs specified in the environment variables (or the defaults if not given). 18 | 19 | 2. Start a libvirt virtual machine and install the Windows Server 2019 from the ISO. 20 | 21 | * The process is fully unattended, via the `autounattended.xml` file with the input needed to install the operating system. 22 | 23 | * The virtio drivers and the guest tools are installed from the ISO. 24 | 25 | * SSH is configured and the given SSH private key is authorized. 26 | 27 | 3. Install the latest Windows updates. 28 | 29 | 4. Run the `setup.ps1` script to prepare the CI environment. 30 | 31 | 5. Generalize the VM image via `Sysprep`. 32 | -------------------------------------------------------------------------------- /ceph-iscsi-stable/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | 15 | set -ex 16 | 17 | # Make sure we execute at the top level directory before we do anything 18 | cd $WORKSPACE 19 | 20 | # This will set the DISTRO and MOCK_TARGET variables. 21 | get_distro_and_target 22 | echo "DISTRO: $DISTRO MOCK_TARGET: $MOCK_TARGET" 23 | 24 | # Make sure the dist directory is clean 25 | rm -rf dist 26 | mkdir -p dist 27 | 28 | # Perform a clean-up 29 | for dir in $(ls -h | grep -v dist); do 30 | cd $WORKSPACE/$dir 31 | git clean -fxd 32 | done 33 | 34 | cd $WORKSPACE 35 | 36 | # Print some basic system info 37 | HOST=$(hostname --short) 38 | echo "Building on $(hostname) with the following env" 39 | echo "*****" 40 | env 41 | echo "*****" 42 | 43 | export LC_ALL=C # the following is vulnerable to i18n 44 | 45 | pkgs=( "chacractl>=0.0.21" ) 46 | TEMPVENV=$(create_venv_dir) 47 | VENV=${TEMPVENV}/bin 48 | install_python_packages $TEMPVENV "pkgs[@]" 49 | 50 | chacra_url="https://chacra.ceph.com/" 51 | # create the .chacractl config file using global variables 52 | make_chacractl_config $chacra_url 53 | -------------------------------------------------------------------------------- /scripts/rook-orch/install-rook-e2e-deps.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -ex 4 | 5 | install_docker(){ 6 | DISTRO="$(lsb_release -cs)" 7 | if [[ $(command -v docker) == '' ]]; then 8 | # Set up docker official repo and install docker. 9 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg 10 | echo \ 11 | "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \ 12 | ${DISTRO} stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null 13 | sudo apt update -y 14 | sudo apt install -y docker-ce docker-ce-cli containerd.io 15 | fi 16 | sudo groupadd docker || true 17 | sudo usermod -aG docker $(id -un) 18 | sudo systemctl unmask docker 19 | sudo systemctl restart docker 20 | sudo chgrp "$(id -un)" /var/run/docker.sock 21 | 22 | # wait for docker 23 | sleep 10 24 | 25 | docker info 26 | docker container prune -f 27 | } 28 | 29 | # install dependencies 30 | sudo apt update -y 31 | sudo apt install --reinstall -y qemu-kvm libvirt-daemon-driver-qemu libvirt-clients libvirt-daemon-system libvirt-daemon runc python3 32 | sudo apt install --reinstall -y python3-pip 33 | install_docker 34 | 35 | # install minikube 36 | curl -LO https://storage.googleapis.com/minikube/releases/v1.31.2/minikube-linux-amd64 37 | sudo install minikube-linux-amd64 /usr/local/bin/minikube 38 | 39 | # delete any existing minikube setup 40 | minikube delete 41 | -------------------------------------------------------------------------------- /ceph-deploy-build/config/definitions/ceph-deploy-build.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-deploy-build 3 | node: small && trusty 4 | project-type: matrix 5 | defaults: global 6 | display-name: 'ceph-deploy-build' 7 | concurrent: true 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | 13 | scm: 14 | - git: 15 | url: https://github.com/ceph/ceph-deploy.git 16 | branches: 17 | - $BRANCH 18 | browser: auto 19 | skip-tag: true 20 | timeout: 20 21 | wipe-workspace: true 22 | 23 | axes: 24 | - axis: 25 | type: label-expression 26 | name: ARCH 27 | values: 28 | - x86_64 29 | - axis: 30 | type: label-expression 31 | name: DIST 32 | values: 33 | - bionic 34 | - centos7 35 | - centos8 36 | 37 | builders: 38 | - shell: 39 | !include-raw-verbatim: 40 | - ../../../scripts/build_utils.sh 41 | - ../../build/setup 42 | - ../../build/build 43 | 44 | wrappers: 45 | - inject-passwords: 46 | global: true 47 | mask-password-params: true 48 | - credentials-binding: 49 | - text: 50 | credential-id: chacractl-key 51 | variable: CHACRACTL_KEY 52 | - text: 53 | credential-id: shaman-api-key 54 | variable: SHAMAN_API_KEY 55 | -------------------------------------------------------------------------------- /ceph-devstack/config/definitions/ceph-devstack.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-devstack 3 | description: Run ceph-devstack 4 | project-type: pipeline 5 | concurrent: false 6 | pipeline-scm: 7 | scm: 8 | - git: 9 | url: https://github.com/zmc/ceph-devstack 10 | branches: 11 | - origin/${{CEPH_DEVSTACK_BRANCH}} 12 | parameters: 13 | - string: 14 | name: CEPH_DEVSTACK_BRANCH 15 | default: "main" 16 | - string: 17 | name: TEUTHOLOGY_BRANCH 18 | default: "main" 19 | - string: 20 | name: TEUTHOLOGY_CEPH_BRANCH 21 | default: "main" 22 | - string: 23 | name: TEUTHOLOGY_CEPH_REPO 24 | default: "https://github.com/ceph/ceph.git" 25 | - string: 26 | name: TEUTHOLOGY_SUITE 27 | default: "teuthology:no-ceph" 28 | - string: 29 | name: TEUTHOLOGY_SUITE_BRANCH 30 | default: "main" 31 | - string: 32 | name: TEUTHOLOGY_SUITE_REPO 33 | default: "https://github.com/ceph/ceph.git" 34 | 35 | triggers: 36 | - github-pull-request: 37 | admin-list: 38 | - zmc 39 | - dmick 40 | - kamoltat 41 | - amathuria 42 | org-list: 43 | - ceph 44 | trigger-phrase: 'jenkins test.*|jenkins retest.*' 45 | only-trigger-phrase: false 46 | github-hooks: true 47 | permit-all: false 48 | auto-close-on-fail: false 49 | -------------------------------------------------------------------------------- /ansible/roles/grafana/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: update apt cache 3 | apt: 4 | update_cache: yes 5 | become: yes 6 | 7 | - name: install ssl system requirements 8 | become: yes 9 | apt: 10 | name: "{{ item }}" 11 | state: present 12 | with_items: ssl_requirements 13 | tags: 14 | - packages 15 | 16 | - name: install system packages 17 | become: yes 18 | apt: 19 | name: "{{ item }}" 20 | state: present 21 | with_items: system_packages 22 | tags: 23 | - packages 24 | 25 | - name: generate pseudo-random password for admin user 26 | shell: python -c "exec 'import os; print os.urandom(30).encode(\'base64\')[:${length}]'" 27 | register: admin_password 28 | changed_when: false 29 | 30 | - name: generate pseudo-random password for the database connection 31 | shell: python -c "exec 'import os; print os.urandom(30).encode(\'base64\')[:${length}]'" 32 | register: db_password 33 | changed_when: false 34 | 35 | - name: configure grafana 36 | template: 37 | src: ../templates/grafana.ini.j2 38 | dest: "/etc/grafana/grafana.ini" 39 | notify: 40 | - restart app 41 | become: true 42 | 43 | - include_tasks: postgresql.yml 44 | tags: 45 | - postgresql 46 | 47 | - include_tasks: nginx.yml 48 | 49 | - name: ensure nginx is running 50 | become: true 51 | service: 52 | name: nginx 53 | state: started 54 | enabled: yes 55 | 56 | - name: ensure grafana is restarted 57 | become: true 58 | service: 59 | name: grafana-server 60 | state: restarted 61 | enabled: yes 62 | -------------------------------------------------------------------------------- /ansible/private-docker-registry/roles/docker-registry/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: create directory for self-signed SSL cert 3 | file: 4 | path: /var/registry/certs 5 | state: directory 6 | 7 | - name: create self-signed cfssl json file 8 | template: 9 | src: "{{ role_path }}/templates/self-csr.json.j2" 10 | dest: ./self-csr.json 11 | 12 | - name: get cfssl 13 | get_url: 14 | url: https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 15 | dest: ./cfssl 16 | mode: 0755 17 | 18 | - name: get cfssljson 19 | get_url: 20 | url: https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 21 | dest: ./cfssljson 22 | mode: 0755 23 | 24 | - name: gencert 25 | shell: ./cfssl gencert -initca self-csr.json | ./cfssljson -bare ca 26 | 27 | - name: push self-signed cfssl cert to the ansible server 28 | fetch: 29 | src: ca.pem 30 | dest: fetch/certs/self.crt 31 | flat: yes 32 | 33 | - name: mv the cert to be accessible by container 34 | command: mv ca.pem /var/registry/certs/self.crt 35 | 36 | - name: mv the key to be accessible by container 37 | command: mv ca-key.pem /var/registry/certs/self.key 38 | 39 | - name: start registry container 40 | command: docker run -d --name=docker-registry \ 41 | -p 5000:5000 \ 42 | --privileged=true \ 43 | --restart=unless-stopped \ 44 | -v /var/registry:/var/registry \ 45 | -e STORAGE_PATH=/var/registry/data \ 46 | -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/registry/certs/self.crt \ 47 | -e REGISTRY_HTTP_TLS_KEY=/var/registry/certs/self.key \ 48 | registry 49 | -------------------------------------------------------------------------------- /ceph-ansible-rpm/config/definitions/ceph-ansible-rpm.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-ansible-rpm 3 | node: 'centos8 && x86_64 && small && !sepia' 4 | project-type: freestyle 5 | defaults: global 6 | disabled: false 7 | display-name: 'ceph-ansible: RPMs' 8 | description: 'Build RPMs for every ceph-ansible Git branch' 9 | concurrent: true 10 | quiet-period: 5 11 | block-downstream: false 12 | block-upstream: false 13 | retry-count: 3 14 | properties: 15 | - build-discarder: 16 | days-to-keep: 1 17 | num-to-keep: 10 18 | artifact-days-to-keep: -1 19 | artifact-num-to-keep: -1 20 | - github: 21 | url: https://github.com/ceph/ceph-ansible 22 | discard-old-builds: true 23 | 24 | triggers: 25 | - github 26 | 27 | scm: 28 | - git: 29 | url: https://github.com/ceph/ceph-ansible 30 | browser: auto 31 | skip-tag: true 32 | timeout: 20 33 | wipe-workspace: true 34 | 35 | builders: 36 | - shell: 37 | !include-raw-verbatim: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | - credentials-binding: 46 | - text: 47 | credential-id: shaman-api-key 48 | variable: SHAMAN_API_KEY 49 | - text: 50 | credential-id: chacractl-key 51 | variable: CHACRACTL_KEY 52 | -------------------------------------------------------------------------------- /ceph-qa-suite-pull-requests/config/definitions/ceph-qa-suite-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-qa-suite-pull-requests 3 | node: trusty 4 | project-type: freestyle 5 | defaults: global 6 | concurrent: true 7 | display-name: 'ceph-qa-suite: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - build-discarder: 14 | days-to-keep: 15 15 | num-to-keep: 30 16 | artifact-days-to-keep: -1 17 | artifact-num-to-keep: -1 18 | - github: 19 | url: https://github.com/ceph/ceph-qa-suite/ 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "A pull request ID, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | org-list: 29 | - ceph 30 | trigger-phrase: '' 31 | only-trigger-phrase: false 32 | github-hooks: true 33 | permit-all: false 34 | auto-close-on-fail: false 35 | 36 | scm: 37 | - git: 38 | url: https://github.com/ceph/ceph-qa-suite.git 39 | branches: 40 | - ${{sha1}} 41 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 42 | browser: auto 43 | timeout: 20 44 | skip-tag: true 45 | wipe-workspace: false 46 | 47 | builders: 48 | - shell: 49 | !include-raw-verbatim: 50 | - ../../../scripts/build_utils.sh 51 | - ../../build/build 52 | -------------------------------------------------------------------------------- /samba/build/setup: -------------------------------------------------------------------------------- 1 | #! /usr/bin/bash 2 | # 3 | # Ceph distributed storage system 4 | # 5 | # Copyright (C) 2016 Red Hat 6 | # 7 | # Author: Boris Ranto 8 | # 9 | # This library is free software; you can redistribute it and/or 10 | # modify it under the terms of the GNU Lesser General Public 11 | # License as published by the Free Software Foundation; either 12 | # version 2.1 of the License, or (at your option) any later version. 13 | # 14 | set -ex 15 | 16 | # Make sure we execute at the top level directory before we do anything 17 | cd $WORKSPACE 18 | 19 | # This will set the DISTRO and MOCK_TARGET variables 20 | get_distro_and_target 21 | 22 | # Perform a clean-up 23 | git clean -fxd 24 | 25 | # Make sure the dist directory is clean 26 | rm -rf dist 27 | mkdir -p dist 28 | 29 | # Print some basic system info 30 | HOST=$(hostname --short) 31 | echo "Building on $(hostname) with the following env" 32 | echo "*****" 33 | env 34 | echo "*****" 35 | 36 | export LC_ALL=C # the following is vulnerable to i18n 37 | 38 | pkgs=( "chacractl>=0.0.21" ) 39 | TEMPVENV=$(create_venv_dir) 40 | VENV=${TEMPVENV}/bin 41 | install_python_packages $TEMPVENV "pkgs[@]" 42 | 43 | SAMBA_BRANCH=$(branch_slash_filter $SAMBA_BRANCH) 44 | CEPH_BRANCH=$(branch_slash_filter $CEPH_BRANCH) 45 | BRANCH=${SAMBA_BRANCH} 46 | 47 | # ask shaman which chacra instance to use 48 | chacra_url=`curl -f -u $SHAMAN_API_USER:$SHAMAN_API_KEY https://shaman.ceph.com/api/nodes/next/` 49 | # create the .chacractl config file using global variables 50 | make_chacractl_config $chacra_url 51 | -------------------------------------------------------------------------------- /attic/ceph-container-build-push-imgs-devel-nightly/config/definitions/ceph-container-build-push-imgs-devel-nightly.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-container-build-push-imgs-devel-nightly 3 | node: huge && trusty && x86_64 4 | project-type: freestyle 5 | defaults: global 6 | name: ceph-container-build-push-imgs-devel-nightly 7 | quiet-period: 5 8 | block-downstream: false 9 | block-upstream: false 10 | retry-count: 3 11 | properties: 12 | - build-discarder: 13 | days-to-keep: -1 14 | num-to-keep: -1 15 | artifact-days-to-keep: -1 16 | artifact-num-to-keep: -1 17 | - github: 18 | url: https://github.com/ceph/ceph-container 19 | 20 | triggers: 21 | - timed: '@daily' 22 | 23 | scm: 24 | - git: 25 | url: https://github.com/ceph/ceph-container.git 26 | branches: 27 | - main 28 | browser: auto 29 | basedir: "ceph-container" 30 | timeout: 20 31 | 32 | builders: 33 | - inject: 34 | properties-content: | 35 | DEVEL=true 36 | - shell: 37 | !include-raw: 38 | - ../../../scripts/build_utils.sh 39 | - ../../build/build 40 | 41 | wrappers: 42 | - inject-passwords: 43 | global: true 44 | mask-password-params: true 45 | - credentials-binding: 46 | - username-password-separated: 47 | credential-id: ceph-container-quay-io 48 | username: REGISTRY_USERNAME 49 | password: REGISTRY_PASSWORD 50 | -------------------------------------------------------------------------------- /attic/ceph-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # vim: ts=4 sw=4 expandtab 3 | 4 | set -ex 5 | 6 | # did any doc/ files change? 7 | # If $GIT_COMMIT is a merge commit (it is usually, because that's how 8 | # we manage ceph, with PRs that include merges): noting the three dots, 9 | # this git diff invocation compares the common ancestor between first parent 10 | # (original state of the branch) to the second parent (the branch being merged 11 | # in) i.e., same as git diff $(git merge-base p1 p2) p2 and outputs filenames 12 | # only (--name-only). 13 | # 14 | # Skip this optimization for non-merge commits 15 | 16 | if git rev-parse --verify ${GIT_COMMIT}^2; then 17 | # enclosing doublequotes prevent newlines from disappearing, for grep below 18 | files="$(git diff --name-only ${GIT_COMMIT}^1...${GIT_COMMIT}^2)" 19 | echo -e "changed files:\n$files" 20 | if ! (echo "$files" | grep -sq '^doc/'); then 21 | echo "No doc files changed, skipping build" 22 | exit 0 23 | fi 24 | fi 25 | 26 | ./admin/build-doc 27 | 28 | REV="$(git rev-parse HEAD)" 29 | OUTDIR="docs.raw/sha1/$REV" 30 | mkdir -p $OUTDIR 31 | 32 | cp -a build-doc/output/html/* $OUTDIR 33 | 34 | # Log this $OUTDIR's sha1 35 | printf '%s\n' "$REV" >"$OUTDIR/sha1" 36 | 37 | # Symlink the branch name 38 | BRANCH=${GIT_BRANCH#*/} 39 | mkdir -p docs.raw/ref/ 40 | ln -s ../sha1/$REV "docs.raw/ref/$BRANCH" 41 | 42 | # Publish this sha1's contents first: 43 | rsync -a -v docs.raw/sha1/$REV /var/docs.raw/sha1/ 44 | # Now point the ref symlink at the newly-uploaded sha1. 45 | rsync -a -v docs.raw/ref/$BRANCH /var/docs.raw/ref/ 46 | -------------------------------------------------------------------------------- /ceph-build-pull-requests/config/definitions/ceph-build-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-build-pull-requests 3 | node: huge 4 | project-type: freestyle 5 | defaults: global 6 | concurrent: true 7 | display-name: 'ceph-build: Pull Requests' 8 | quiet-period: 5 9 | block-downstream: false 10 | block-upstream: false 11 | retry-count: 3 12 | properties: 13 | - build-discarder: 14 | days-to-keep: 15 15 | num-to-keep: 30 16 | artifact-days-to-keep: -1 17 | artifact-num-to-keep: -1 18 | - github: 19 | url: https://github.com/ceph/ceph-build 20 | 21 | parameters: 22 | - string: 23 | name: sha1 24 | description: "commit id or a refname, like 'origin/pr/72/head'" 25 | 26 | triggers: 27 | - github-pull-request: 28 | org-list: 29 | - ceph 30 | trigger-phrase: '.*retest.*' 31 | only-trigger-phrase: false 32 | github-hooks: true 33 | permit-all: false 34 | auto-close-on-fail: false 35 | 36 | scm: 37 | - git: 38 | url: https://github.com/ceph/ceph-build.git 39 | branches: 40 | - ${{sha1}} 41 | refspec: +refs/pull/${{ghprbPullId}}/*:refs/remotes/origin/pr/${{ghprbPullId}}/* 42 | browser: auto 43 | timeout: 20 44 | skip-tag: true 45 | wipe-workspace: false 46 | 47 | builders: 48 | - shell: 49 | !include-raw-verbatim: 50 | - ../../../scripts/build_utils.sh 51 | - ../../build/build 52 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: ensure sites-available for nginx 3 | file: 4 | path: /etc/nginx/sites-available 5 | state: directory 6 | become: true 7 | 8 | - name: ensure there is an nginx user 9 | user: 10 | name: nginx 11 | comment: "Nginx user" 12 | become: true 13 | 14 | - name: ensure sites-enable for nginx 15 | file: 16 | path: /etc/nginx/sites-enabled 17 | state: directory 18 | become: true 19 | 20 | - name: remove default nginx site 21 | file: 22 | path: /etc/nginx/sites-enabled/default 23 | state: absent 24 | become: true 25 | 26 | - name: write nginx.conf 27 | template: 28 | src: nginx.conf 29 | dest: /etc/nginx/nginx.conf 30 | become: true 31 | 32 | - name: enable nginx 33 | become: true 34 | service: 35 | name: nginx 36 | enabled: true 37 | 38 | - name: create nginx site config 39 | template: 40 | src: "nginx_site.conf" 41 | dest: "/etc/nginx/sites-available/{{ item.app_name }}.conf" 42 | become: true 43 | with_items: nginx_hosts 44 | notify: 45 | - restart nginx 46 | 47 | - include_tasks: ssl.yml 48 | when: development_server == true 49 | 50 | - include_tasks: letsencrypt.yml 51 | when: development_server == false 52 | 53 | - name: link nginx config 54 | file: 55 | src: "/etc/nginx/sites-available/{{ item.app_name }}.conf" 56 | dest: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 57 | state: link 58 | become: true 59 | with_items: nginx_hosts 60 | 61 | - name: ensure nginx is restarted 62 | become: true 63 | service: 64 | name: nginx 65 | state: restarted 66 | -------------------------------------------------------------------------------- /scripts/nfs-ganesha/sign-rpm-auto: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script will the rpm files pulled from the chacra machines. 3 | 4 | 5 | keyid=460F3994 6 | GPG_PASSPHRASE='' 7 | 8 | path="/opt/nfs-ganesha/new-repos/" 9 | echo $path 10 | update_repo=0 11 | cd $path 12 | 13 | for rpm in `find -name "*.rpm"` 14 | do 15 | signature=$(rpm -qi -p $rpm 2>/dev/null | grep ^Signature) 16 | if ! grep -iq $keyid <<< "$signature" ; then 17 | rpm_path=`readlink -f $rpm` 18 | echo "signing: $rpm_path" 19 | update_repo=1 20 | 21 | echo "yes" | setsid rpm \ 22 | --define "_gpg_name '$keyid'" \ 23 | --define '_signature gpg' \ 24 | --define '__gpg_check_password_cmd /bin/true' \ 25 | --define "__gpg_sign_cmd %{__gpg} gpg --no-tty --yes --batch --no-armor --passphrase '$GPG_PASSPHRASE' --no-secmem-warning -u "%{_gpg_name}" --sign --detach-sign --output %{__signature_filename} %{__plaintext_filename}" \ 26 | --resign "$rpm_path" 27 | 28 | fi 29 | done 30 | 31 | # now sign the repomd.xml files 32 | if [[ $update_repo -eq 1 ]]; then 33 | for repomd in `find -name repomd.xml` 34 | do 35 | echo "signing repomd: $repomd" 36 | gpg --batch --yes --passphrase "$GPG_PASSPHRASE" --detach-sign --armor -u $keyid $repomd 37 | done 38 | fi 39 | 40 | # finally, update the repo metadata 41 | repodirs=$( find /opt/nfs-ganesha/new-repos/ -type d -name x86_64 | cut -d/ -f 13 --complement ) 42 | if [ -n "$repodirs" ]; then 43 | for directory in $repodirs 44 | do 45 | cd $directory 46 | createrepo . 47 | cd - 48 | done 49 | fi 50 | -------------------------------------------------------------------------------- /ceph-grafana/config/definitions/ceph-grafana.yml: -------------------------------------------------------------------------------- 1 | - job: 2 | name: ceph-grafana 3 | description: 'Builds the ceph-grafana container.' 4 | project-type: freestyle 5 | concurrent: true 6 | display-name: 'ceph-grafana' 7 | properties: 8 | - groovy-label: 9 | script: return ARCH + '&¢os8' 10 | - build-discarder: 11 | days-to-keep: -1 12 | num-to-keep: 25 13 | artifact-days-to-keep: 25 14 | artifact-num-to-keep: 25 15 | - github: 16 | url: https://github.com/ceph/ceph 17 | 18 | scm: 19 | - git: 20 | url: https://github.com/ceph/ceph 21 | branches: 22 | - $BRANCH 23 | wipe-workspace: true 24 | 25 | parameters: 26 | - string: 27 | name: BRANCH 28 | description: "The git branch (or tag) to build" 29 | default: main 30 | - string: 31 | name: ARCH 32 | description: "Architecture to build for. Available options are: x86_64, arm64" 33 | default: "x86_64" 34 | 35 | builders: 36 | - shell: 37 | !include-raw-verbatim: 38 | ../../build/build 39 | 40 | wrappers: 41 | - inject-passwords: 42 | global: true 43 | mask-password-params: true 44 | - credentials-binding: 45 | - username-password-separated: 46 | credential-id: ceph-container-quay-io 47 | username: CONTAINER_REPO_USERNAME 48 | password: CONTAINER_REPO_PASSWORD 49 | - build-name: 50 | name: "#${{BUILD_NUMBER}} ${{BRANCH}}, ${{ARCH}}" 51 | -------------------------------------------------------------------------------- /ceph-pr-docs/build/build: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -xo pipefail 4 | 5 | # make sure any shaman list file is removed. At some point if all nodes 6 | # are clean this will not be needed. 7 | sudo rm -f /etc/apt/sources.list.d/shaman* 8 | sudo rm -f /etc/apt/sources.list.d/ubuntu-toolchain-r* 9 | sudo rm -f /etc/apt/sources.list.d/ceph-boost* 10 | 11 | # Ceph doc build deps, Ubuntu only because ditaa is not packaged for CentOS 12 | sudo apt-get update -o Acquire::Languages=none -o Acquire::Translation=none 13 | sudo apt-get install -y gcc python3-dev python3-pip python3-virtualenv libxml2-dev libxslt-dev doxygen graphviz ant ditaa cython3 14 | 15 | virtualenv -q --python python3 venv 16 | . venv/bin/activate 17 | pip install tox 18 | pip install git+https://github.com/ceph/githubcheck.git 19 | sha1=$(git rev-parse refs/remotes/origin/pr/${ghprbPullId}/head) 20 | 21 | output=$(mktemp $PWD/build-doc-XXX.out) 22 | 23 | if timeout 3600 ./admin/build-doc 2>&1 | tee ${output}; then 24 | succeed=true 25 | else 26 | succeed=false 27 | fi 28 | 29 | if ! $succeed; then 30 | cat ${output} | github-check \ 31 | --sphinx \ 32 | --sphinx-root=. \ 33 | --owner "ceph" \ 34 | --repo "ceph" \ 35 | --pkey-file $GITHUB_CHECK_PKEY_PEM \ 36 | --app-id "62865" \ 37 | --install-id "8465036" \ 38 | --name "ceph-pr-docs" \ 39 | --sha $sha1 \ 40 | --external-id $BUILD_ID \ 41 | --details-url $BUILD_URL \ 42 | --title sphinx-build 43 | fi 44 | 45 | $succeed 46 | -------------------------------------------------------------------------------- /ansible/roles/nginx/tasks/letsencrypt.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: install system packages 4 | become: yes 5 | apt: 6 | name: "letsencrypt" 7 | state: present 8 | 9 | - name: ensure letsencrypt acme-challenge path 10 | file: 11 | path: "{{ ssl_webroot_base_path }}/{{ item.fqdn }}" 12 | state: "directory" 13 | mode: 0755 14 | become: yes 15 | with_items: nginx_hosts 16 | 17 | - name: unlink nginx configs 18 | file: 19 | path: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 20 | state: "absent" 21 | become: true 22 | with_items: nginx_hosts 23 | 24 | - name: create temporary nginx config 25 | template: 26 | src: "nginx_tmp_site.conf" 27 | dest: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 28 | become: true 29 | with_items: nginx_hosts 30 | 31 | - name: restart nginx 32 | become: yes 33 | service: 34 | name: nginx 35 | state: restarted 36 | 37 | - name: create (or renew) letsencrypt ssl cert 38 | command: "letsencrypt certonly --webroot -w {{ ssl_webroot_base_path }}/{{ item.fqdn }} -d {{ item.fqdn }} --email {{ ssl_support_email }} --agree-tos --renew-by-default" 39 | become: yes 40 | with_items: nginx_hosts 41 | 42 | - name: setup a cron to renew the SSL cert every day 43 | cron: 44 | name: "renew letsencrypt cert for {{ item.app_name }}" 45 | minute: "21" 46 | hour: "6,18" 47 | job: "letsencrypt renew --agree-tos --email {{ ssl_support_email }}" 48 | become: yes 49 | with_items: nginx_hosts 50 | 51 | - name: unlink tmp nginx config 52 | file: 53 | path: "/etc/nginx/sites-enabled/{{ item.app_name }}.conf" 54 | state: "absent" 55 | become: true 56 | with_items: nginx_hosts 57 | -------------------------------------------------------------------------------- /paddles-pull-requests/config/definitions/paddles-pull-requests.yml: -------------------------------------------------------------------------------- 1 | - scm: 2 | name: paddles 3 | scm: 4 | - git: 5 | url: https://github.com/ceph/paddles 6 | branches: 7 | - ${{sha1}} 8 | refspec: +refs/pull/*:refs/remotes/origin/pr/* 9 | browser: auto 10 | timeout: 20 11 | basedir: "paddles" 12 | skip-tag: true 13 | wipe-workspace: true 14 | 15 | 16 | - job: 17 | name: paddles-pull-requests 18 | description: Runs tox tests for paddles on each GitHub PR 19 | project-type: freestyle 20 | node: python3 && centos7 21 | block-downstream: false 22 | block-upstream: false 23 | defaults: global 24 | display-name: 'paddles: Pull Requests' 25 | quiet-period: 5 26 | retry-count: 3 27 | 28 | 29 | properties: 30 | - build-discarder: 31 | days-to-keep: 15 32 | num-to-keep: 30 33 | artifact-days-to-keep: 15 34 | artifact-num-to-keep: 15 35 | - github: 36 | url: https://github.com/ceph/paddles/ 37 | 38 | parameters: 39 | - string: 40 | name: sha1 41 | description: "A pull request ID, like 'origin/pr/72/head'" 42 | 43 | triggers: 44 | - github-pull-request: 45 | org-list: 46 | - ceph 47 | only-trigger-phrase: false 48 | github-hooks: true 49 | permit-all: false 50 | auto-close-on-fail: false 51 | 52 | scm: 53 | - paddles 54 | 55 | builders: 56 | - shell: 57 | !include-raw-verbatim: 58 | - ../../../scripts/build_utils.sh 59 | - ../../build/build 60 | -------------------------------------------------------------------------------- /ceph-dev-pipeline-trigger/build/Jenkinsfile: -------------------------------------------------------------------------------- 1 | JOB = "ceph-dev-pipeline" 2 | VALID_PARAMETERS = [ 3 | "CEPH_BUILD_BRANCH", 4 | "ARCHS", 5 | "CI_COMPILE", 6 | "CI_CONTAINER", 7 | "CI_PIPELINE", 8 | "DISTROS", 9 | "DWZ", 10 | "FLAVOR", 11 | "SCCACHE", 12 | ] 13 | def params = [] 14 | 15 | pipeline { 16 | agent any 17 | stages { 18 | stage("Prepare parameters") { 19 | steps { 20 | script { 21 | def trailer = sh( 22 | script: "echo \"$head_commit\" | git interpret-trailers --parse", 23 | returnStdout: true, 24 | ) 25 | println("trailer: ${trailer}") 26 | def paramsMap = [:] 27 | for (item in trailer.split("\n")) { 28 | def matcher = item =~ /(.+): (.+)/ 29 | if (matcher.matches()) { 30 | key = matcher[0][1].replace("-", "_").toUpperCase() 31 | value = matcher[0][2] 32 | paramsMap[key] = value 33 | } 34 | } 35 | def branch = env.ref.replace("refs/heads/", "") 36 | params.push(string(name: "BRANCH", value: branch)) 37 | println("Looking for parameters: ${VALID_PARAMETERS}") 38 | for (key in VALID_PARAMETERS) { 39 | value = paramsMap[key] 40 | if ( value ) { 41 | params.push(string(name: key, value: value)) 42 | println("${key}=${value}") 43 | } 44 | } 45 | } 46 | } 47 | } 48 | stage("Trigger job") { 49 | steps { 50 | script { 51 | build( 52 | job: JOB, 53 | parameters: params 54 | ) 55 | } 56 | } 57 | } 58 | } 59 | } 60 | --------------------------------------------------------------------------------