├── .output.gif ├── pip-requirements.txt ├── apt-requirements.txt ├── jenkins-apt-sudoers ├── .gitignore ├── .travis.yml ├── init.groovy ├── LICENSE ├── jenkins.sh ├── .checkJava.sh ├── jenkins-swarm-client.sh ├── Dockerfile ├── Makefile ├── fabfile.py └── readme.md /.output.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ska-sa/CBF-Tests-Automation/master/.output.gif -------------------------------------------------------------------------------- /pip-requirements.txt: -------------------------------------------------------------------------------- 1 | jenkinsapi==0.3.6 2 | python-dotenv==0.9.1 3 | fabric==1.12.1 4 | -------------------------------------------------------------------------------- /apt-requirements.txt: -------------------------------------------------------------------------------- 1 | apt-transport-https 2 | wget 3 | git 4 | curl 5 | zip 6 | sudo 7 | nano 8 | -------------------------------------------------------------------------------- /jenkins-apt-sudoers: -------------------------------------------------------------------------------- 1 | jenkins ALL=(root)NOPASSWD:/usr/bin/apt-get update,/usr/bin/apt-get install * 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | # Python droppings 3 | *.pyc 4 | .ropeproject 5 | .env 6 | .venv 7 | *.jar 8 | oldstuff 9 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: c 2 | sudo: required 3 | 4 | services: 5 | - docker 6 | notifications: 7 | email: false 8 | 9 | install: 10 | - make testbootstrap 11 | 12 | script: 13 | - docker ps -a 14 | -------------------------------------------------------------------------------- /init.groovy: -------------------------------------------------------------------------------- 1 | import hudson.model.*; 2 | import jenkins.model.*; 3 | 4 | 5 | Thread.start { 6 | sleep 10000 7 | println "--> setting agent port for jnlp" 8 | Jenkins.instance.setSlaveAgentPort(50000) 9 | } 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 SKA Africa 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /jenkins.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | # Copy files from /usr/share/jenkins/ref into /var/jenkins_home 4 | # So the initial JENKINS-HOME is set with expected content. 5 | # Don't override, as this is just a reference setup, and use from UI 6 | # can then change this, upgrade plugins, etc. 7 | copy_reference_file() { 8 | f=${1%/} 9 | echo "$f" 10 | rel=${f:23} 11 | dir=$(dirname ${f}) 12 | echo " $f -> $rel" 13 | if [[ ! -e /var/jenkins_home/${rel} ]] 14 | then 15 | echo "copy $rel to JENKINS_HOME" 16 | mkdir -p /var/jenkins_home/${dir:23} 17 | cp -r /usr/share/jenkins/ref/${rel} /var/jenkins_home/${rel}; 18 | fi; 19 | } 20 | export -f copy_reference_file 21 | find /usr/share/jenkins/ref/ -type f -exec bash -c 'copy_reference_file {}' \; 22 | 23 | # if `docker run` first argument start with `--` the user is passing jenkins launcher arguments 24 | if [[ $# -lt 1 ]] || [[ "$1" == "--"* ]]; then 25 | exec java $JAVA_OPTS -jar /usr/share/jenkins/jenkins.war $JENKINS_OPTS "$@" 26 | fi 27 | 28 | # As argument is not jenkins, assume user want to run his own process, for sample a `bash` shell to explore this image 29 | exec "$@" 30 | 31 | -------------------------------------------------------------------------------- /.checkJava.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # Script that checks Java version if its JAVA8 else it will install it. 3 | # Author: Mpho Mphego 4 | 5 | 6 | printf "Checking Java version\n" 7 | # Allows us to read user input below, assigns stdin to keyboard 8 | exec < /dev/tty 9 | JAVA_VER=$(java -version 2>&1 | grep -i version) 10 | if [ $(echo ${JAVA_VER} | grep 1.8 > /dev/null; printf $?) -ne 0 ]; then 11 | printf "Java Version is out-of-date: ${JAVA_VER}\n" 12 | printf "Do you wish update your JAVA runtime?\n" 13 | printf "NOTE: This installer assumes you are running Debian 7 Wheezy\n"; 14 | printf "Enter Yes -> 1 or No -> 2\n" 15 | select yn in "Yes" "No"; do 16 | case $yn in 17 | Yes ) sudo sh -c 'echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" >> /etc/apt/sources.list'; 18 | sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886; 19 | printf "Running apt update\n" 20 | sudo apt-get update -qq; 21 | sudo apt-get install -y --force-yes oracle-java8-installer; 22 | break;; 23 | No ) printf "Oh well then! I guess you have a better idea!!!"; 24 | exit 1;; 25 | esac 26 | done 27 | else 28 | printf "Java Version is supported: ${JAVA_VER}\n\n\n" 29 | exit 0; 30 | fi; 31 | 32 | -------------------------------------------------------------------------------- /jenkins-swarm-client.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ### BEGIN INIT INFO 4 | # Provides: jenkinsvmfarm 5 | # Required-Start: $remote_fs $syslog 6 | # Required-Stop: $remote_fs $syslog 7 | # Should-Start: $named 8 | # Default-Start: 2 3 4 5 9 | # Default-Stop: 0 1 6 10 | # Short-Description: jenkins swarm build slave 11 | # Description: jenkins swarm build slave assigned to a Jenkins master 12 | ### END INIT INFO 13 | set -e 14 | 15 | . /lib/lsb/init-functions 16 | 17 | USER=cbf-test 18 | USER_HOME="/home/${USER}" 19 | SWARM_HOME="${USER_HOME}/jenkinsswarm" 20 | JAR="${SWARM_HOME}/swarm-client-jar-with-dependencies.jar" 21 | LOG="${SWARM_HOME}/jenkins-swarm-client.log" 22 | # PASSWORD_FILE="${SWARM_HOME}/jenkins_password" 23 | MASTER="http://localhost:8080" 24 | # Serious security risk 25 | USERNAME="user" 26 | PASSWORD="bitnami" 27 | 28 | # Swarm client option 29 | DESCRIPTION="CMC Swarm client" 30 | EXECUTORS=1 31 | FSROOT="${SWARM_HOME}/fsroot" 32 | LABELS="cmc" 33 | NAME="cmc_slave" 34 | 35 | OPTS="-description \"${DESCRIPTION}\" \ 36 | -name ${NAME} \ 37 | -master ${MASTER} \ 38 | -executors ${EXECUTORS} \ 39 | -fsroot ${FSROOT} \ 40 | -labels \"${LABELS}\" \ 41 | -username ${USERNAME} \ 42 | -password ${PASSWORD}" 43 | 44 | # Note, the --password @PASSWORD_FILE option to read the password from a file 45 | # was added in v 2.0 of the swarm plugin 46 | # 47 | # See https://issues.jenkins-ci.org/browse/JENKINS-26620 48 | # 49 | # Seems the option is undocumented, so it may break in the future? 50 | 51 | PIDFILE="/var/run/jenkins-swarm-client.pid" 52 | ARGS="-server -Djava.awt.headless=true -jar $JAR $OPTS" 53 | # JAVA_HOME="/etc/alternatives/java" 54 | DAEMON="/usr/bin/java" 55 | 56 | test -x $DAEMON || exit 1 57 | 58 | case $1 in 59 | start) 60 | log_daemon_msg "Starting jenkins-swarm-client" 61 | echo "${DAEMON} ${ARGS} > ${LOG}" 62 | start-stop-daemon --start --chuid $USER --background --make-pidfile --pidfile $PIDFILE --startas /bin/bash -- -c "exec $DAEMON $ARGS > $LOG 2>&1" 63 | 64 | log_end_msg $? 65 | ;; 66 | stop) 67 | if [ -e $PIDFILE ]; then 68 | log_daemon_msg "Stopping jenkins-swarm-client" 69 | start-stop-daemon --stop --quiet --pidfile $PIDFILE 70 | log_end_msg $? 71 | rm -f $PIDFILE 72 | fi 73 | ;; 74 | restart) 75 | $0 stop 76 | sleep 2 77 | $0 start 78 | ;; 79 | status) 80 | status_of_proc -p $PIDFILE "$DAEMON" jenkins-swarm-client 81 | # RETVAL=$? 82 | ;; 83 | 84 | *) 85 | echo "Usage: $0 {start|stop|restart|status}" 86 | exit 1 87 | 88 | esac 89 | 90 | exit 0 91 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # Usage 2 | # export JENKINS_USER="jenkins" 3 | # docker run --restart=on-failure:10 -d --name=${JENKINS_USER} -p 8080:8080 -p 50000:50000 -v /home/${JENKINS_USER}:/var/jenkins_home ska-sa-cbf/${JENKINS_USER} 4 | 5 | FROM java:openjdk-8-jdk 6 | LABEL maintainer="Mpho Mphego " 7 | ARG DEBIAN_FRONTEND=noninteractive 8 | # RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections 9 | COPY apt-requirements.txt / 10 | RUN apt-get update && \ 11 | apt-get install -y --no-install-recommends apt-utils && \ 12 | apt-get install -y --no-install-recommends $(grep -vE "^\s*#" apt-requirements.txt | tr "\n" " ") && \ 13 | apt-get clean && \ 14 | rm -rf /var/lib/apt/lists/* || true 15 | 16 | # Handle python deps 17 | RUN curl https://bootstrap.pypa.io/get-pip.py | python && \ 18 | pip install --no-cache-dir --disable-pip-version-check -U virtualenv 19 | 20 | ENV JENKINS_HOME /var/jenkins_home 21 | ENV JENKINS_UID 2000 22 | # Jenkins is ran with user `jenkins`, uid = JENKINS_UID 23 | # If you bind mount a volume from host/volume from a data container, 24 | # ensure you use same uid 25 | RUN useradd -d "${JENKINS_HOME}" -u "${JENKINS_UID}" -m -s /bin/bash jenkins 26 | 27 | ## Allow jenkins user to use sudo to perform a limited subset of commands 28 | # without a password such as installing new packages. 29 | RUN adduser jenkins sudo 30 | 31 | COPY jenkins-apt-sudoers /etc/sudoers.d/ 32 | RUN chmod 0440 /etc/sudoers.d/jenkins-apt-sudoers 33 | 34 | # Jenkins home directory is a volume, so configuration and build history 35 | # can be persisted and survive image upgrades 36 | VOLUME /var/jenkins_home 37 | 38 | # `/usr/share/jenkins/ref/` contains all reference configuration we want 39 | # to set on a fresh new installation. Use it to bundle additional plugins 40 | # or config file with your custom jenkins Docker image. 41 | RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d 42 | COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-angent-port.groovy 43 | # Copy in local config files 44 | # COPY plugins.sh /usr/local/bin/plugins.sh 45 | # COPY install-plugins.sh /usr/local/bin/install-plugins.sh 46 | # RUN chmod +x /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy \ 47 | # && chmod +x /usr/local/bin/plugins.sh \ 48 | # && chmod +x /usr/local/bin/jenkins.sh \ 49 | # && chmod +x /usr/local/bin/install-plugins.sh 50 | 51 | # Jenkins Mirrors: http://mirrors.jenkins.io/status.html 52 | # LTS version 53 | ENV JENKINS_VERSION 2.107.1 54 | RUN bash -c "ping -c 5 www.google.com" 55 | 56 | # Backup links 57 | # ----------------------------------------- 58 | # RUN bash -c "wget 'https://mirrors.tuna.tsinghua.edu.cn/jenkins/war/${JENKINS_VERSION}/jenkins.war' -O /usr/share/jenkins/jenkins.war" 59 | # RUN bash -c "wget 'http://mirrors.jenkins-ci.org/war/${JENKINS_VERSION}/jenkins.war' -O /usr/share/jenkins/jenkins.war" 60 | # ----------------------------------------- 61 | RUN wget "http://mirrors.jenkins.io/war-stable/${JENKINS_VERSION}/jenkins.war" -O /usr/share/jenkins/jenkins.war 62 | #RUN axel "http://mirrors.jenkins-ci.org/war/${JENKINS_VERSION}/jenkins.war" -o /usr/share/jenkins/jenkins.war 63 | ENV JENKINS_UC https://updates.jenkins-ci.org 64 | RUN chown -R jenkins "${JENKINS_HOME}" /usr/share/jenkins/ref 65 | 66 | # for main web interface: 67 | EXPOSE 8080 68 | # will be used by attached slave agents: 69 | EXPOSE 50000 70 | 71 | USER jenkins 72 | COPY jenkins.sh /usr/local/bin/jenkins.sh 73 | #RUN chmod +x /usr/local/bin/jenkins.sh 74 | ENTRYPOINT ["/bin/bash", "/usr/local/bin/jenkins.sh"] 75 | 76 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | export JENKINS_USER=jenkins 2 | export HOSTNAME=$(shell hostname -i) 3 | 4 | help: 5 | @echo "Please use \`make ' where is one of" 6 | @echo " bootstrap One-liner to make everything work!!!" 7 | @echo " checkJava to check/install Java runtime." 8 | @echo " docker to build a cbf-test/jenkins docker container" 9 | @echo " build to build a docker container, configure jenkins local volume, configure sonarqube and portainer" 10 | @echo " install to check and install Java and Python dependencies." 11 | @echo " fabric to configure jenkins local volume and other dependencies" 12 | @echo "" 13 | @echo " run to run pre-built jenkins container" 14 | @echo " start to start an existing jenkins container" 15 | @echo " stop to stop an existing jenkins container" 16 | @echo "" 17 | @echo " log to see the logs of a running container" 18 | @echo " shell to execute a shell on jenkins container" 19 | @echo "" 20 | @echo " sonar to run sonarqube container" 21 | @echo " sonar_start to start sonarqube container" 22 | @echo " sonar_stop to stop sonarqube container" 23 | @echo "" 24 | @echo " portainer to run portainer container" 25 | @echo " portainer_start to start portainer container" 26 | @echo " portainer_stop to stop portainer container" 27 | @echo "" 28 | @echo " start_all to start all containers defined" 29 | @echo " stop_all to stop all containers defined" 30 | @echo "" 31 | @echo " clean to stop and delete jenkins container" 32 | @echo " superclean to clean and delete jenkins user and /home/jenkins" 33 | 34 | checkJava: 35 | bash -c "./.checkJava.sh" || true 36 | 37 | install: checkJava 38 | sudo pip install fabric==1.12.2 39 | @echo 40 | 41 | docker: 42 | @docker build -t ska-sa-cbf/${JENKINS_USER} . 43 | 44 | sonar: 45 | @docker run --restart=on-failure:10 -d --name sonarqube -p 9000:9000 -p 9092:9092 sonarqube 46 | 47 | portainer: 48 | @docker volume create --name=portainer_data 49 | @docker run --restart=on-failure:10 -d --name portainer -p 9001:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer 50 | 51 | fabric: 52 | @echo "Running Fabric on $(HOSTNAME)" 53 | @fab -H ${HOSTNAME} setup_cbftest_user 54 | @fab -H ${HOSTNAME} setup_jenkins_user 55 | @fab -H ${HOSTNAME} -u ${USER} checkout_cbf_jenkins_config 56 | 57 | build: docker fabric sonar portainer 58 | 59 | run: 60 | @sudo /etc/init.d/jenkins-swarm-client.sh start || true; 61 | @docker run --restart=on-failure:10 -d --name=${JENKINS_USER} -p 8080:8080 -p 50000:50000 -v /home/${JENKINS_USER}:/var/jenkins_home ska-sa-cbf/${JENKINS_USER} 62 | 63 | bootstrap: install build run 64 | 65 | testbootstrap: install docker sonar portainer run 66 | 67 | start: 68 | @sudo /etc/init.d/jenkins-swarm-client.sh start || true 69 | @docker start ${JENKINS_USER} || true 70 | 71 | stop: 72 | @sudo /etc/init.d/jenkins-swarm-client.sh stop || true 73 | @docker stop ${JENKINS_USER} || true 74 | 75 | portainer_start: 76 | @docker start portainer || true 77 | 78 | portainer_stop: 79 | @docker stop portainer || true 80 | 81 | sonar_start: 82 | @docker start sonarqube || true 83 | 84 | sonar_stop: 85 | @docker stop sonarqube || true 86 | 87 | stop_all: stop portainer_stop sonar_stop 88 | 89 | start_all: start portainer_start sonar_start 90 | 91 | clean: stop_all 92 | @docker rm -v ${JENKINS_USER} || true 93 | @docker rm -v sonarqube || true 94 | @docker rm -v portainer || true 95 | @docker volume rm portainer || true 96 | 97 | superclean: clean 98 | @docker rmi ska-sa-cbf/${JENKINS_USER} || true 99 | @docker rmi portainer/portainer || true 100 | @sudo userdel -f -r ${JENKINS_USER} || true 101 | @sudo rm -rf /etc/init.d/jenkins-swarm-client.sh || true 102 | 103 | log: 104 | @docker logs -f ${JENKINS_USER} 105 | 106 | shell: 107 | @docker exec -it ${JENKINS_USER} /bin/bash -------------------------------------------------------------------------------- /fabfile.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import absolute_import 4 | from __future__ import print_function 5 | 6 | import __builtin__ 7 | import os 8 | # import time 9 | # import signal 10 | from requests import get as geturl 11 | try: 12 | from fabric.api import task, sudo, run, cd, local, env, warn_only, get 13 | from fabric.context_managers import settings 14 | except ImportError: 15 | from pip._internal import main as pip 16 | pip(['install', '--user', 'fabric==1.12.1']) 17 | from fabric.api import task, sudo, run, cd, local, env, warn_only, get 18 | from fabric.context_managers import settings 19 | 20 | UPSTART_DIR = '/etc/init.d' 21 | CBFTEST_USER = "cbf-test" 22 | CBFTEST_HOME = "/home/{CBFTEST_USER}".format(**locals()) 23 | 24 | JENKINS_UID = 2000 25 | JENKINS_USER = os.environ.get('JENKINS_USER', 'jenkins') 26 | JENKINS_HOME = '/home/{JENKINS_USER}'.format(**locals()) 27 | 28 | JENKINS_SWARM = "jenkins-swarm-client.sh" 29 | JENKINS_SWARM_VERSION = 2.0 30 | JENKINS_SWARM_CLIENT = ( 31 | 'https://raw.githubusercontent.com/ska-sa/CBF-Tests-Automation/master/' 32 | '{JENKINS_SWARM}'.format(**locals())) 33 | 34 | CONFIG_GIT_REPO = 'git@github.com:ska-sa/CBF-Jenkins-home.git' 35 | 36 | JENKINS_LOCALPORT = 8080 37 | 38 | class bcolors: 39 | BoldGreen = '\033[92m\033[1m' 40 | EndCmd = '\033[0m' 41 | 42 | def print(*args, **kwargs): 43 | __builtin__.print('{0}{2}{1}'.format(bcolors.BoldGreen, bcolors.EndCmd, *args, **kwargs)) 44 | 45 | @task 46 | def setup_cbftest_user(): 47 | print ("NOTE: You will need to enter your SUDO password and hostname") 48 | with settings(warn_only=True): 49 | print ("Creating new user: {CBFTEST_USER}".format(**globals())) 50 | sudo('useradd -d "{CBFTEST_HOME}" -m -s /bin/bash {CBFTEST_USER}'.format(**globals())) 51 | print ('Adding user to staff and docker groups.') 52 | sudo('groupadd docker') 53 | sudo('usermod -aG staff {CBFTEST_USER}'.format(**globals())) 54 | sudo('usermod -aG docker {CBFTEST_USER}'.format(**globals())) 55 | sudo('chgrp docker /usr/bin/docker') 56 | sudo('chgrp docker /var/run/docker.sock/') 57 | sudo('chmod a=rwX,o+t /tmp -R') 58 | print ("Setup a simple password for user: {CBFTEST_USER}".format(**globals())) 59 | sudo('passwd {CBFTEST_USER}'.format(**globals())) 60 | print("Generating SSH Keys") 61 | sudo('ssh-keygen -t rsa -f {CBFTEST_HOME}/.ssh/id_rsa -q -N ""'.format(**globals()), 62 | user="{CBFTEST_USER}".format(**globals())) 63 | sudo('mkdir -p {CBFTEST_HOME}/jenkinsswarm/fsroot'.format(**globals()), 64 | user="{CBFTEST_USER}".format(**globals())) 65 | print ("Downloading JAVA Jenkins swarm client v{JENKINS_SWARM_VERSION}".format(**globals())) 66 | sudo( 67 | "curl --create-dirs -sSLo {CBFTEST_HOME}/jenkinsswarm/swarm-client-jar-with-dependencies.jar" 68 | " https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/swarm-client/" 69 | "{JENKINS_SWARM_VERSION}/swarm-client-{JENKINS_SWARM_VERSION}-jar-with-" 70 | "dependencies.jar".format(**globals()), user="{CBFTEST_USER}".format(**globals())) 71 | 72 | if geturl(JENKINS_SWARM_CLIENT).status_code == 200: 73 | print ("Downloading Jenkins Swarm Client shell from github.com") 74 | sudo('wget {JENKINS_SWARM_CLIENT} -O {UPSTART_DIR}/{JENKINS_SWARM}'.format(**globals())) 75 | sudo('chmod a+x {UPSTART_DIR}/{JENKINS_SWARM}'.format(**globals())) 76 | print ("Starting Jenkins Swarm client automagically....") 77 | sudo('{UPSTART_DIR}/{JENKINS_SWARM} start'.format(**globals())) 78 | sudo("echo '{JENKINS_USER}:2345:respawn:/bin/bash {UPSTART_DIR}/{JENKINS_SWARM}' >> /etc/inittab".format(**globals())) 79 | 80 | @task 81 | def setup_jenkins_user(): 82 | print ("NOTE: You will need to enter your SUDO password and hostname") 83 | with settings(warn_only=True): 84 | sudo('useradd -d "{JENKINS_HOME}" -u {JENKINS_UID} -m -s /bin/bash {JENKINS_USER}'.format( 85 | **globals())) 86 | sudo('usermod -aG docker {JENKINS_USER}'.format(**globals())) 87 | print ("Setup a password for user: {JENKINS_USER}".format(**globals())) 88 | sudo('passwd {JENKINS_USER}'.format(**globals())) 89 | sudo('chmod -R o-xrw {JENKINS_HOME}'.format(**globals())) 90 | print("Generating SSH Keys") 91 | sudo('ssh-keygen -t rsa -f {JENKINS_HOME}/.ssh/id_rsa -q -N ""'.format(**globals()), 92 | user="{JENKINS_USER}".format(**globals())) 93 | print("\n\nCopy your `public key` and add new SSH keys to your GitHub profile.") 94 | sudo('cat {JENKINS_HOME}/.ssh/id_rsa.pub'.format(**globals())) 95 | print("Paste your ssh public key to your `GitHub ssh keys` in order to access private repos...") 96 | print("Open link in your browser: https://github.com/settings/ssh/new\n") 97 | while True: 98 | if raw_input("Press any Keys & Enter to continue..."): 99 | break 100 | 101 | @task 102 | def checkout_cbf_jenkins_config(): 103 | print ("This section assumes that {} has SUDO rights and, ".format(os.environ['USER'])) 104 | print ("That you successfully added jenkins ssh public keys to your Github account.") 105 | with settings(warn_only=True): 106 | with cd(JENKINS_HOME): 107 | sudo('git init .', user="{JENKINS_USER}".format(**globals())) 108 | sudo('git config --local user.email "fake-jenkins-user@ska.ac.za.fake"', 109 | user="{JENKINS_USER}".format(**globals())) 110 | sudo('git config --local user.name "CBF Jenkins automaton"', 111 | user="{JENKINS_USER}".format(**globals())) 112 | sudo('git remote add origin {CONFIG_GIT_REPO}'.format(**globals()), 113 | user="{JENKINS_USER}".format(**globals())) 114 | print("This will take a while, go make yourself a cup of coffee!!!") 115 | sudo('git fetch', user="{JENKINS_USER}".format(**globals())) 116 | sudo('git checkout master -f ', user="{JENKINS_USER}".format(**globals())) 117 | sudo('git pull -f ', user="{JENKINS_USER}".format(**globals())) 118 | sudo('git config --global push.default simple ', user="{JENKINS_USER}".format(**globals())) 119 | 120 | @task 121 | def kill_jenkins_http(): 122 | print ('Kill http exposing process') 123 | run( 124 | "pgrep -f 'ssh -R 8080:localhost:{JENKINS_LOCALPORT} {PORTFORWARDING_URL}' | " 125 | "xargs kill -9".format(**globals())) 126 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | 2 | # CBF Automated Test Framework 3 | 4 | [![Codacy Badge](https://api.codacy.com/project/badge/Grade/fbfe4e35e4584901a2916a55ca6fd994)](https://app.codacy.com/app/mmphego/CBF-Tests-Automation?utm_source=github.com&utm_medium=referral&utm_content=ska-sa/CBF-Tests-Automation&utm_campaign=Badge_Grade_Dashboard) 5 | [![Build Status](https://travis-ci.org/ska-sa/CBF-Tests-Automation.svg?branch=master)](https://travis-ci.org/ska-sa/CBF-Tests-Automation) 6 | [![LICENSE](https://img.shields.io/github/license/ska-sa/CBF-Tests-Automation.svg?style=flat)](LICENSE) 7 | 8 | Define a Dockerfiles that can build a docker instances useful for running Automated CBF Tests running on Jenkins CI. 9 | 10 | The files are: 11 | - Dockerfile 12 | Builds a Jenkins master-server instance. This runs the Jenkins web frontend, 13 | schedules builds, checks source repositories for changes, and can run certain 14 | jobs on slaves. 15 | It assumes that the jenkins user config/home directory is mounted on the container volume `/var/jenkins_home`, which should be owned by UID=2000. 16 | 17 | - Jenkins-Swarm-Client 18 | Script that running as a service, for self organizing slave nodes; auto-discovers a nearbby Jenkins master and join it automatically. 19 | 20 | More documentation: [https://wiki.jenkins.io/display/JENKINS/Swarm+Plugin](https://wiki.jenkins.io/display/JENKINS/Swarm+Plugin) 21 | 22 | ## Howto 23 | 24 | Starting from scratch, the subsection steps need to be performed in order. The 25 | `host` referred to below is the server/VM that will be hosting the Jenkins 26 | docker instance. 27 | 28 | ### Set up the host 29 | 30 | 1. Install [Docker](https://docs.docker.com/installation/) 31 | 32 | **Note:** Extra steps only needed when installing on Debian Wheezy; newer versions have per-packaged docker.io. 33 | 34 | Add the wheezy backports repository so that a new-enough kernel can be installed for docker: 35 | 36 | ```shell 37 | echo "deb http://http.debian.net/debian wheezy-backports main" >> /etc/apt/sources.list.d/wheezy-backports.list 38 | ``` 39 | 40 | Perform the following steps in a shell 41 | ```shell 42 | sudo apt-get update 43 | sudo apt-get install -t wheezy-backports linux-image-amd64 44 | # Apparmor info: https://wiki.debian.org/AppArmor/HowToUse 45 | sudo apt-get install apparmor apparmor-profiles apparmor-utils 46 | sudo perl -pi -e \ 47 | 's,GRUB_CMDLINE_LINUX="(.*)"$,GRUB_CMDLINE_LINUX="$1 apparmor=1 security=apparmor",' /etc/default/grub 48 | sudo update-grub 49 | 50 | sudo shutdown -rf now 51 | 52 | sudo apt-get install apt-transport-https 53 | # From https://get.docker.com/ script 54 | sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 \ 55 | --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9 56 | sudo sh -c "echo deb https://get.docker.com/ubuntu docker main \ 57 | > /etc/apt/sources.list.d/docker.list" 58 | sudo apt-get update && sudo apt-get install lxc-docker 59 | ``` 60 | 61 | 2. Fix DNS config if you use a localhost dnsmasq (like Ubuntu's using NetworkManager) by adding the *SKA SA* DNS servers to the `/etc/default/docker` 62 | 63 | 64 | ```shell 65 | DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 196.24.41.8 --dns 196.24.41.9 --dns 192.168.1.21" 66 | ``` 67 | 68 | if using [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) to find the SKARABS with dnsmasq bound to the SKARABS control network interface as on the correlator controller servers, where `10.103.0.1` is the IP address of the SKARABS control network interface (usually eth2). 69 | 70 | ```shell 71 | DOCKER_OPTS="--dns 10.103.0.1" 72 | ``` 73 | 74 | ### Deploy the Jenkins container on the host 75 | 76 | This section is only a suggestion, there are many ways to manage docker images, 77 | containers and volumes, but this is a simple one :) 78 | 79 | Clone the current repository out somewhere on the host, change to the repository directory and run: 80 | 81 | ```shell 82 | make bootstrap 83 | ``` 84 | 85 | This will check & possibly install Java 8 JDK and Python [Fabric](http://fabfile.org/), Creates *jenkins* and *cbf-test* users for test automation, Build a Jenkins Docker container, Runs [SonarQube](sonarqube.org) and [Portainer](portainer.io). 86 | 87 | It should be visible in the list if the `sudo docker ps` command is run. 88 | 89 | Jenkins container will start automatically at boot-time, and it will automatically be restarted up to 10 times if it exits with an error condition. 90 | 91 | The Jenkins web interface is exposed on port `8080` of all the host network interfaces. If you need to modify the port consult docker documentation. 92 | 93 | The container data volume `/var/jenkins_home` will be linked to the `/home/jenkins` directory on the host. This directory should be owned by the `jenkins` user. It is recommended that it not be world readable since the jenkins configuration might contain sensitive information and it seems `Jenkins` itself is not too smart about using appropriate permissions. 94 | 95 | 96 | **Note**: This Jenkins image exports a data volume `/var/jenkins_home` that is required to store the Jenkins configuration. This volume should be owned by a `jenkins` user with `UID 2000` (as configured in `Dockerfile`). It is recommended that the volume (if stored on the host) should not be world readable since the jenkins configuration might contain sensitive information and it seems Jenkins itself is not too smart about using appropriate permissions. 97 | 98 | A *fabfile.py* for use with the [python Fabric](http://fabfile.org) package is included. It has tasks to set up a Jenkins/CBF-Test user with the correct UID on a host, and a task to do the git checkout. 99 | 100 | 101 | **Note:** Edit the `CONFIG_GIT_REPO` variable to change the git repository to clone. Your host needs to have `sudo` installed, and the user used to connect to host must have `sudo` rights. Also the Python `fabric` package must be installed. 102 | 103 | Example for setting up a Jenkins user and checking out our Jenkins configuration on a host in the home directory of the `jenkins` user, run in shell in the current repository directory: 104 | 105 | ```shell 106 | make fabric 107 | ``` 108 | 109 | ## Demo 110 | 111 | [![Demo](.output.gif)](https://www.youtube.com/watch?v=uK2Qlv3v6jk) 112 | 113 | ## Conclusion 114 | 115 | Hopefully one can easily see how easy it is to set-up and run Jenkins with Docker. I’ve tried to make it as basic as possible with minimal options to take the default Jenkins container and make it a little more usable. There are other options where one can initialise Jenkins container with pre-installed plugins and etc. 116 | 117 | By making our own Dockerfile file, we were able to make our life a little easier. We set up a convenient place to store the configuration and by running a swarm client we were able to have our artifacts and files stored in our user who is designed to run tests. We moved our default settings into the Dockerfile and now we can store this in source control as a good piece of self-documentation. 118 | 119 | By making our own Dockerfile we have explored and achieved these concepts: 120 | 121 | - Preserving Jenkins Job and Plugin data 122 | - Docker Data Persistence with Volumes 123 | - Making a Data-Volume container 124 | - Sharing data in volumes with other containers 125 | - Control over Jenkins and Swarm client versions, custom apt packages, Python packages and plug-in installations 126 | 127 | Taking control of your Docker images isn’t that hard. At a minimum, paying attention to your dependencies and where they come from can help you understand what goes into making your containers work. Additionally, if you do so, you can find some opportunities to make your image lighter weight and save some disk space by removing things you don’t need to install. You also lower your risk of a dependency breaking on you. 128 | 129 | On the other hand, you take on significant responsibility - you won’t get automatic updates and you’ll have to follow along with changes to things like the Jenkins image. Whether or not this is a benefit to you depends on your personal development policies. 130 | 131 | Regardless of whether you choose to own your image, I do recommend that you follow the same basic practices here whenever you pick up a new Dockerfile. Find the Docker image it inherits from by using Dockerhub to help you follow the inheritance chain. Be aware of all the Dockerfiles in the chain and what they use. You should always be aware of what your images contain - after all this is stuff running on your network, on your servers. At a minimum it helps you find the base operating system but you can also learn a lot about the ecosystem of Docker images out there and learn some interesting practices. 132 | 133 | At this point you should have a fully functional Jenkins master server image set and the basics of your own Jenkins environment. 134 | 135 | - [Ref: How I configured JenkinsCI server in a Docker container](https://blog.mphomphego.co.za/blog/2018/10/23/How-I-configured-JenkinsCI-server-in-a-Docker-container-2.html) 136 | 137 | ### Useful Links 138 | 139 | - [https://thepracticalsysadmin.com/setting-up-a-github-webhook-in-jenkins/](https://thepracticalsysadmin.com/setting-up-a-github-webhook-in-jenkins/) 140 | - [https://live-rg-engineering.pantheon.io/news/putting-jenkins-docker-container](https://live-rg-engineering.pantheon.io/news/putting-jenkins-docker-container) 141 | - [https://github.com/boxboat/jenkins-demo/blob/develop/docs/part1-jenkins-setup.md](https://github.com/boxboat/jenkins-demo/blob/develop/docs/part1-jenkins-setup.md) 142 | - [www.donaldsimpson.co.uk/category/jenkins/page/2/](www.donaldsimpson.co.uk/category/jenkins/page/2/) 143 | 144 | 145 | ## Feedback 146 | 147 | Feel free to fork it or send me PR to improve it. 148 | --------------------------------------------------------------------------------