├── LICENSE ├── README.md ├── src ├── com │ └── invoca │ │ ├── ci │ │ └── DeprecationWarnings.groovy │ │ ├── docker │ │ ├── Docker.groovy │ │ └── Image.groovy │ │ ├── github │ │ ├── GitHubApi.groovy │ │ └── GitHubStatus.groovy │ │ └── util │ │ ├── CacheUtil.groovy │ │ ├── FileUtil.groovy │ │ └── JenkinsFolder.groovy └── io │ └── invoca │ └── Docker.groovy └── vars ├── assetCacheKeys.groovy ├── bundleCacheKeys.groovy ├── cacheRestore.groovy ├── cacheStore.groovy ├── chefPipeline.groovy ├── gemfileLockChecksum.groovy ├── gitHubStatus.groovy ├── githubShallowCloneCommit.groovy ├── notifySlack.groovy ├── runWithDeprecationWarningCheck.groovy └── shWithSSHAgent.groovy /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Invoca 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Jenkins Pipeline 2 | 3 | Jenkins Pipeline Shared Library. Contains helper functions to be used with the Jenkins Pipeline Plugin. 4 | 5 | ## Image Class 6 | 7 | The Docker `Image` class expects 3 or 4 arguments in its constructor call 8 | 9 | | # | Type | Required | Description | 10 | |---|----------|:---------:|-------------| 11 | | 1 | Script | Y | A reference to the Script object, always `this` when instantiated from the Jenkinsfile. | 12 | | 2 | String | Y | The name of the image, including the Docker Hub organization. i.e. `invocaops/ruby`. | 13 | | 3 | String[] | Y | An array of tags to apply to the image. | 14 | | 4 | String | N | The directory that the Dockerfile is in. Useful when multiple versions of the image need to be built. Defaults to the directory the Jenkinsfile is in. | 15 | 16 | #### Example 17 | Example for Ruby 2.4.2, which is in a directory named `2.4.2` and being built from the master branch with SHA `12345`: 18 | ```groovy 19 | image = new Image(this, "invocaops/ruby", ["2.4.2-12345", "2.4.2-master"], "2.4.2") 20 | ``` 21 | 22 | ### Usage 23 | 24 | The `Image` class has 4 main methods to perform operations 25 | 26 | | Method | Arguments | Action | 27 | |---------|--------------------------------|----------------------------------------------| 28 | | build() | [buildArgs (Map)](#build-args) | Buils the Docker image. | 29 | | tag() | None | Tags the image. | 30 | | push() | None | Pushes the image and its tags to Docker Hub. | 31 | 32 | Each method returns a reference to the `Image` object, so chaining is possible. 33 | 34 | ### Build Args 35 | 36 | The `Image#build` method takes a `Map` of build arguments. 37 | 38 | | Argument | Type | Required | Description | 39 | |------------|--------|:--------:|------------------------------------------------------| 40 | | gitUrl     | String | Y | URL to remote Git repository. Set to `env.GIT_URL`. | 41 | | buildArgs | Map   |     N | `foo=bar` pairings for `docker build --build-arg`. | 42 | | dockerFile | String | N | Name of `Dockerfile` file, defaults to `Dockerfile`. | 43 | | target | String | N | Target stage to build to in the docker build. | 44 | 45 | ### Environment 46 | 47 | In addition to the included Git environment variables, we currently assume access to credentials for DockerHub. You'll need to explicitly set these in your environment. 48 | 49 | | Variable | Available By Default | Description | 50 | |--------------------|:--------------------:|---------------------------------------| 51 | | DOCKERHUB_USER | N | Username for DockerHub. | 52 | | DOCKERHUB_PASSWORD | N | Password for DockerHub. | 53 | | GIT_COMMIT | Y | SHA of current build. | 54 | | GIT_URL | Y | URL of GitHub repository being built. | 55 | | GIT_BRANCH         |           Y         | The name of the checked out branch. | 56 | 57 | ## Getting Started 58 | 59 | To use this library, start your `Jenkinsfile` with: 60 | 61 | ```groovy 62 | @Library('github.com/invoca/jenkins-pipeline@v0.1.0') 63 | ``` 64 | 65 | After, parts of the library can be imported and used. Below is an example of a `Jenkinsfile` that builds multiple versions of the Ruby image. 66 | 67 | ```groovy 68 | @Library('github.com/invoca/jenkins-pipeline@v0.1.0') 69 | 70 | import com.invoca.docker.*; 71 | 72 | pipeline { 73 | agent any 74 | stages { 75 | stage('Setup') { 76 | steps { 77 | script { 78 | def imageName = "invocaops/ruby" 79 | def directories = sh(script: 'ls **/Dockerfile | while read dir; do echo $(dirname $dir); done', returnStdout: true).split("\n") 80 | def sha = env.GIT_COMMIT 81 | def branchName = env.GIT_BRANCH 82 | 83 | images = directories.collect { 84 | String[] tags = ["${it}-${branchName}", "${it}-${sha}"] 85 | new Image(this, imageName, tags, it) 86 | } 87 | } 88 | } 89 | } 90 | 91 | stage('Build') { 92 | steps { 93 | script { 94 | for (Image image : images) { 95 | image.build(gitUrl: env.GIT_URL).tag() 96 | } 97 | } 98 | } 99 | } 100 | 101 | stage('Push') { 102 | environment { 103 | DOCKERHUB_USER = credentials('dockerhub_user') 104 | DOCKERHUB_PASSWORD = credentials('dockerhub_password') 105 | } 106 | steps { 107 | script { 108 | new Docker().hubLogin(env.DOCKERHUB_USER, env.DOCKERHUB_PASSWORD) 109 | for (Image image : images) { 110 | image.push() 111 | } 112 | } 113 | } 114 | } 115 | } 116 | 117 | post { 118 | always { 119 | notifySlack(currentBuild.result) 120 | } 121 | } 122 | } 123 | ``` 124 | 125 | Please read more about libraries in the [Jenkins documentation](https://jenkins.io/doc/book/pipeline/shared-libraries/). 126 | -------------------------------------------------------------------------------- /src/com/invoca/ci/DeprecationWarnings.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | 3 | package com.invoca.ci 4 | 5 | import com.invoca.github.GitHubStatus 6 | 7 | class DeprecationWarnings { 8 | static void checkAndUpdateGithub(Script script, String testOutput, Map githubStatusConfig) { 9 | (new DeprecationWarnings(script, testOutput, githubStatusConfig)).checkAndUpdateGithub() 10 | } 11 | 12 | public static String GITHUB_STATUS_CONTEXT = 'deprecation-warning-check' 13 | public static String GITHUB_STATUS_FAILURE_MESSAGE = 'Unexpected deprecation warnings encountered' 14 | public static String GITHUB_STATUS_SUCCESS_MESSAGE = 'No unexpected deprecation warnings encountered' 15 | public static String WARNINGS_ARCHIVE_FILE_NAME = 'deprecation_warnings.txt' 16 | public static String DEPRECATION_WARNING_PREFIX = 'DEPRECATION WARNING' 17 | public static String UNEXPECTED_DEPRECATIONS_START = 'Unexpected Deprecation Warnings Encountered' 18 | public static String UNEXPECTED_DEPRECATIONS_END = '=====' 19 | 20 | private Script script 21 | private String testOutput 22 | private Map githubStatusConfig 23 | 24 | public DeprecationWarnings(Script script, String testOutput, Map githubStatusConfig) { 25 | this.script = script 26 | this.testOutput = testOutput 27 | this.githubStatusConfig = githubStatusConfig 28 | } 29 | 30 | public void checkAndUpdateGithub() { 31 | Map githubStatusConfig = this.githubStatusConfig 32 | 33 | githubStatusConfig.context = GITHUB_STATUS_CONTEXT 34 | 35 | if (this.warningsExist()) { 36 | githubStatusConfig.targetURL = this.archiveWarnings() 37 | githubStatusConfig.status = 'failure' 38 | githubStatusConfig.description = GITHUB_STATUS_FAILURE_MESSAGE 39 | } else { 40 | githubStatusConfig.targetURL = this.script.env.RUN_DISPLAY_URL 41 | githubStatusConfig.status = 'success' 42 | githubStatusConfig.description = GITHUB_STATUS_SUCCESS_MESSAGE 43 | } 44 | 45 | GitHubStatus.update(githubStatusConfig) 46 | } 47 | 48 | public boolean warningsExist() { 49 | return this.testOutput.contains(UNEXPECTED_DEPRECATIONS_START) || this.testOutput.contains(DEPRECATION_WARNING_PREFIX) 50 | } 51 | 52 | private String extractWarnings() { 53 | List warnings = new ArrayList(); 54 | boolean withinUnexpectedDeprecationOutput = false 55 | 56 | this.testOutput.split("\n").each { 57 | if (it == UNEXPECTED_DEPRECATIONS_START) { 58 | warnings.add("=====\n" + UNEXPECTED_DEPRECATIONS_START) 59 | withinUnexpectedDeprecationOutput = true 60 | } else if (withinUnexpectedDeprecationOutput && it == UNEXPECTED_DEPRECATIONS_END) { 61 | warnings.add(it) 62 | withinUnexpectedDeprecationOutput = false 63 | } else if (withinUnexpectedDeprecationOutput || it.contains(DEPRECATION_WARNING_PREFIX)) { 64 | warnings.add(it) 65 | } 66 | } 67 | 68 | return warnings.join("\n") 69 | } 70 | 71 | // Archives the extracted warnings and returns the URL for the artifact 72 | private String archiveWarnings() { 73 | this.script.writeFile(file: WARNINGS_ARCHIVE_FILE_NAME, text: this.extractWarnings()) 74 | this.script.archiveArtifacts(WARNINGS_ARCHIVE_FILE_NAME) 75 | 76 | return this.script.env.BUILD_URL + "/artifact/" + WARNINGS_ARCHIVE_FILE_NAME 77 | } 78 | } 79 | -------------------------------------------------------------------------------- /src/com/invoca/docker/Docker.groovy: -------------------------------------------------------------------------------- 1 | package com.invoca.docker 2 | 3 | def hubLogin(String username, String password) { 4 | sh "docker login -u ${username} -p ${password}" 5 | } 6 | 7 | def imageExistsLocally(String imageWithTag) { 8 | return sh(script: "docker image inspect ${imageWithTag}", returnStatus: true) == 0 9 | } 10 | 11 | def downloadImage(String imageWithTag) { 12 | return sh(script: "docker pull ${imageWithTag}", returnStatus: true) == 0 13 | } 14 | 15 | def getImage(String imageWithTag) { 16 | // Check if image is present locally, or try downloading it from remote. 17 | // Return true if image is now present locally, false otherwise. 18 | return imageExistsLocally(imageWithTag) || downloadImage(imageWithTag) 19 | } 20 | -------------------------------------------------------------------------------- /src/com/invoca/docker/Image.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package com.invoca.docker 3 | 4 | class Image implements Serializable { 5 | public static String LABEL_SCHEMA_VERSION = "org.label-schema.schema-version" 6 | public static String LABEL_BUILD_DATE = "org.label-schema.build-date" 7 | public static String LABEL_VCS_URL = "org.label-schema.vcs-url" 8 | 9 | private String imageName 10 | private String[] tags 11 | private String baseDir 12 | private Script script 13 | private String[] imageNameWithTags 14 | private String[] labels 15 | 16 | public Image(Script script, String imageName, String tag, String baseDir = ".") { 17 | this(script, imageName, [tag], baseDir) 18 | } 19 | 20 | public Image(Script script, String imageName, ArrayList tags, String baseDir = ".") { 21 | this(script, imageName, tags as String[], baseDir) 22 | } 23 | 24 | public Image(Script script, String imageName, String[] tags, String baseDir = ".") { 25 | this.script = script 26 | this.imageName = imageName 27 | this.tags = tags 28 | this.baseDir = baseDir 29 | } 30 | 31 | public Image build(Map args) { 32 | this.imageNameWithTags = this.buildImageNameWithTags() 33 | 34 | def gitUrl = args.gitUrl 35 | def buildArgs = args.buildArgs ?: [:] 36 | def dockerFile = args.dockerFile ?: "Dockerfile" 37 | def target = args.target ?: null 38 | 39 | sh buildCommand(gitUrl, buildArgs, "${this.baseDir}/${dockerFile}", target) 40 | this 41 | } 42 | 43 | public Image tag() { 44 | this.imageNameWithTags = this.buildImageNameWithTags() 45 | 46 | if (this.imageNameWithTags.size() > 1) { 47 | for (int i = 1; i < this.imageNameWithTags.size(); i++) { 48 | sh tagCommand(this.imageNameWithTags[i]) 49 | } 50 | } 51 | this 52 | } 53 | 54 | public Image push() { 55 | this.imageNameWithTags.each { sh pushCommand(it) } 56 | this 57 | } 58 | 59 | public Image remove() { 60 | this.imageNameWithTags.each { sh removeCommand(it) } 61 | this 62 | } 63 | 64 | private void sh(String command) { 65 | this.script.sh(command) 66 | } 67 | 68 | private void sh(Map args) { 69 | this.script.sh(args) 70 | } 71 | 72 | private String buildCommand(String gitUrl, Map buildArgs, String dockerFile, String target) { 73 | def buildArgList = buildArgs.collect { k, v -> "--build-arg ${k}=\"${v}\"" } 74 | def command = [ 75 | "docker", 76 | "build", 77 | "--pull", 78 | "-t ${this.imageName}:${baseTag()}", 79 | "-f ${dockerFile}" 80 | ] + getLabels(gitUrl) + buildArgList 81 | 82 | if (target) { 83 | command.add("--target ${target}") 84 | } 85 | 86 | command.add(this.baseDir) 87 | 88 | return command.join(" ") 89 | } 90 | 91 | private String tagCommand(String imageNameWithTag) { 92 | "docker tag ${this.imageName}:${baseTag()} ${imageNameWithTag}" 93 | } 94 | 95 | private String pushCommand(String imageNameWithTag) { 96 | "docker push ${imageNameWithTag}" 97 | } 98 | 99 | private String removeCommand(String imageNameWithTag) { 100 | "docker rmi ${imageNameWithTag}" 101 | } 102 | 103 | private String baseTag() { 104 | sanitizeTag(this.tags[0]) 105 | } 106 | 107 | private String sanitizeTag(String tag) { 108 | tag.replaceAll("[/:]", "_") 109 | } 110 | 111 | private String[] buildImageNameWithTags() { 112 | this.tags.collect { "${this.imageName}:${sanitizeTag(it)}" } 113 | } 114 | 115 | private String[] getLabels(String gitUrl) { 116 | [ 117 | (LABEL_SCHEMA_VERSION): "1.0", 118 | (LABEL_BUILD_DATE): currentDate(), 119 | (LABEL_VCS_URL): gitUrl 120 | ].collect { k, v -> "--label ${k}=${v}" } 121 | } 122 | 123 | private String currentDate() { 124 | sh(script: 'date -u +"%Y-%m-%dT%H:%M:%SZ"', returnStdout: true).toString().trim() 125 | } 126 | } 127 | -------------------------------------------------------------------------------- /src/com/invoca/github/GitHubApi.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package com.invoca.github 3 | 4 | import java.net.HttpURLConnection 5 | import java.net.* 6 | import groovy.json.JsonOutput 7 | import hudson.AbortException 8 | 9 | class GitHubApi implements Serializable { 10 | final static String GITHUB_API_URL_TEMPLATE = "https://api.github.com/repos/%s/%s" 11 | 12 | private String repoSlug 13 | private String token 14 | private String script 15 | 16 | public GitHubApi(Script script, String repoSlug, String token) { 17 | this.script = script 18 | this.repoSlug = repoSlug 19 | this.token = token 20 | } 21 | 22 | public void post(String resource, String jsonBody) { 23 | log("GitHubApi POST %s to %s", jsonBody, buildGitHubURL(resource)) 24 | 25 | HttpURLConnection connection = (HttpURLConnection) buildHttpConnectionForResource(resource) 26 | connection.setRequestMethod("POST") 27 | connection.setRequestProperty("Content-Type", "application/json") 28 | 29 | OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream(), "UTF-8") 30 | writer.write(jsonBody) 31 | writer.flush() 32 | 33 | def responseMessage = String.format("%d %s", connection.getResponseCode(), connection.getResponseMessage()) 34 | 35 | if (connection.getResponseCode() == HttpURLConnection.HTTP_CREATED) { 36 | log("Received response: %s", responseMessage) 37 | } else { 38 | abort(responseMessage) 39 | } 40 | } 41 | 42 | private HttpURLConnection buildHttpConnectionForResource(String resource) { 43 | HttpURLConnection connection = (HttpURLConnection) buildGitHubURL(resource).openConnection() 44 | connection.setRequestProperty("Authorization", "token " + token) 45 | connection.setDoOutput(true) 46 | return connection 47 | } 48 | 49 | private URL buildGitHubURL(String resource) { 50 | new URI(String.format(GITHUB_API_URL_TEMPLATE, this.repoSlug, resource)).toURL() 51 | } 52 | 53 | private void log(String format, Object... args) { 54 | script.println(String.format(format, args)) 55 | } 56 | 57 | private void abort(String message) { 58 | throw new AbortException("Failed to set GitHub status: " + message) 59 | } 60 | } -------------------------------------------------------------------------------- /src/com/invoca/github/GitHubStatus.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package com.invoca.github 3 | 4 | import java.net.HttpURLConnection 5 | import java.net.* 6 | import groovy.json.JsonOutput 7 | import hudson.AbortException 8 | 9 | import com.invoca.github.GitHubApi 10 | 11 | class GitHubStatus implements Serializable { 12 | final static String GITHUB_API_URL_TEMPLATE = "statuses/%s" 13 | 14 | private String context 15 | private String description 16 | private String sha 17 | private String targetURL 18 | private Script script 19 | private GitHubApi githubAPI 20 | 21 | static void update(Map config) { 22 | GitHubStatus.fromConfig(config).update(config.status) 23 | } 24 | 25 | static GitHubStatus fromConfig(Map config) { 26 | def githubApi = new GitHubApi(config.script, config.repoSlug, config.token) 27 | return new GitHubStatus(githubApi, config) 28 | } 29 | 30 | public GitHubStatus(GitHubApi githubAPI, Map config) { 31 | this.script = config.script 32 | this.githubAPI = githubAPI 33 | this.sha = config.sha 34 | this.context = config.context 35 | this.description = config.description 36 | this.targetURL = config.targetURL 37 | } 38 | 39 | public void update(String status) { 40 | log("Attempting to set GitHub status to %s='%s' for %s", context, status, sha) 41 | this.githubAPI.post(buildGitHubResource(), buildPayload(status)) 42 | } 43 | 44 | private void log(String format, Object... args) { 45 | script.println(String.format(format, args)) 46 | } 47 | 48 | private String buildGitHubResource() { 49 | String.format(GITHUB_API_URL_TEMPLATE, sha) 50 | } 51 | 52 | private String buildPayload(String status) { 53 | def payload = [ 54 | state: status, 55 | target_url: targetURL, 56 | description: description, 57 | context: context 58 | ] 59 | 60 | JsonOutput.toJson(payload) 61 | } 62 | 63 | private void abort(String msg) { 64 | throw new AbortException("Failed to set GitHub status: " + msg) 65 | } 66 | } 67 | -------------------------------------------------------------------------------- /src/com/invoca/util/CacheUtil.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package com.invoca.util 3 | 4 | class CacheUtil { 5 | static String sanitizeCacheKey(String cacheKey) { 6 | return cacheKey.replaceAll("\\W", "-"); 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /src/com/invoca/util/FileUtil.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package com.invoca.util 3 | 4 | class FileUtil { 5 | private Script script 6 | 7 | public FileUtil(Script script) { 8 | this.script = script 9 | } 10 | 11 | def sha256sum(ArrayList filePaths) { 12 | return this.script.sh(script: "cat ${filePaths.join(" ")} | sha256sum -b | cut -d ' ' -f 1", returnStdout: true).trim() 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /src/com/invoca/util/JenkinsFolder.groovy: -------------------------------------------------------------------------------- 1 | package com.invoca.util 2 | 3 | import com.cloudbees.hudson.plugins.folder.Folder 4 | import hudson.model.AbstractProject 5 | 6 | class JenkinsFolder { 7 | def folderName 8 | 9 | JenkinsFolder(folderName) { 10 | this.folderName = folderName 11 | } 12 | 13 | def getProjectNames() { 14 | getFolder().items.collect { it.name } 15 | } 16 | 17 | def getFolder() { 18 | for (item in Jenkins.instance.items) { 19 | if (item instanceof Folder && item.name == folderName) { 20 | return item 21 | } 22 | } 23 | 24 | throw new RuntimeException("Could not locate folder ${folderName}") 25 | } 26 | } 27 | -------------------------------------------------------------------------------- /src/io/invoca/Docker.groovy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/groovy 2 | package io.invoca; 3 | 4 | /* args 5 | build_args list (optional) 6 | dockerfile string 7 | image_name string 8 | */ 9 | 10 | @NonCPS 11 | def buildCommand(Map args) { 12 | def String cmd 13 | 14 | // http://label-schema.org/rc1/ 15 | cmd = "docker build -t ${args.image_name}:$GIT_COMMIT \ 16 | --label org.label-schema.schema-version=1.0 \ 17 | --label org.label-schema.build-date=`date -u +\"%Y-%m-%dT%H:%M:%SZ\"` \ 18 | --label org.label-schema.vcs-url=$GIT_URL" 19 | 20 | if (args.build_args) { 21 | cmd += args.build_args.collect { " --build-arg ${it}" }.join(" ") 22 | } 23 | 24 | cmd += " ${args.dockerfile}" 25 | return cmd 26 | } 27 | 28 | def imageBuild(Map args) { 29 | sh buildCommand(args) 30 | } 31 | 32 | def imagePush(String image_name, String tag) { 33 | sh "docker push ${image_name}:${tag}" 34 | } 35 | 36 | def imageRemove(String image_name, String tag) { 37 | sh "docker rmi ${image_name}:${tag}" 38 | } 39 | 40 | def imageTag(String image_name, String current_tag, String new_tag) { 41 | sh "docker tag ${image_name}:${current_tag} ${image_name}:${new_tag}" 42 | } 43 | 44 | def imageTagPush(String image_name) { 45 | hubLogin() 46 | imagePush(image_name, env.GIT_COMMIT) 47 | 48 | if (env.GIT_BRANCH == 'master') { 49 | imageTag(image_name, env.GIT_COMMIT, 'latest') 50 | imagePush(image_name, 'latest') 51 | } 52 | 53 | if (env.GIT_BRANCH == 'production') { 54 | imageTag(image_name, env.GIT_COMMIT, 'production') 55 | imagePush(image_name, 'production') 56 | } 57 | } 58 | 59 | def hubLogin() { 60 | sh "docker login -u $DOCKERHUB_USER -p $DOCKERHUB_PASSWORD" 61 | } 62 | 63 | return this 64 | -------------------------------------------------------------------------------- /vars/assetCacheKeys.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for generating s3 cache keys 3 | * 4 | * @param cacheKey String 5 | * @param exhaustive Boolean 6 | */ 7 | 8 | ArrayList call(String cacheKey, Boolean exhaustive = true) { 9 | if (exhaustive) { 10 | return [ 11 | "assets-${cacheKey}", 12 | "assets-master", 13 | "assets-production" 14 | ].unique(false); 15 | } 16 | 17 | return ["assets-${cacheKey}"]; 18 | } 19 | -------------------------------------------------------------------------------- /vars/bundleCacheKeys.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for generating s3 cache keys 3 | * 4 | * @param cacheKey String 5 | * @param exhaustive Boolean 6 | */ 7 | 8 | ArrayList call(String cacheKey, Boolean exhaustive = true) { 9 | if (exhaustive) { 10 | return [ 11 | "bundle-${gemfileLockChecksum()}", 12 | "bundle-${cacheKey}", 13 | "bundle-master" 14 | ].unique(false); 15 | } 16 | 17 | return [ 18 | "bundle-${gemfileLockChecksum()}", 19 | "bundle-${cacheKey}" 20 | ]; 21 | } 22 | -------------------------------------------------------------------------------- /vars/cacheRestore.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for restoring cached artifacts 3 | * 4 | * @param 5 | */ 6 | 7 | import com.invoca.util.CacheUtil 8 | 9 | void call(String s3Bucket, ArrayList cacheKeys, Boolean global = false) { 10 | String cacheDirectory = global ? "s3://${s3Bucket}/jenkins_cache" : "s3://${s3Bucket}/jenkins_cache/${env.JOB_NAME.replaceAll("\\W", "")}"; 11 | // Verify that aws-cli is installed before proceeding 12 | sh 'which aws' 13 | 14 | echo 'Pulling cache from AWS' 15 | Boolean cacheExists = false 16 | String cacheTarball = "" 17 | for (String cacheKey : cacheKeys) { 18 | String serializedCacheKey = CacheUtil.sanitizeCacheKey(cacheKey) 19 | echo "Serialized cacheKey from ${cacheKey} => ${serializedCacheKey}" 20 | 21 | cacheTarball = "${serializedCacheKey}.tar.gz" 22 | String cacheLocation = "${cacheDirectory}/${cacheTarball}" 23 | 24 | cacheExists = sh(script: "aws s3 ls ${cacheLocation}", returnStatus: true) == 0 25 | if (cacheExists) { 26 | try { 27 | echo "Found cache at key ${serializedCacheKey}" 28 | sh "aws s3 cp ${cacheLocation} ${cacheTarball} --content-type application/x-gzip" 29 | 30 | echo "Unpacking cache tarball from ${serializedCacheKey}" 31 | sh "tar -xzf ${cacheTarball}" 32 | 33 | echo "Cleaning up local cache tarball from ${serializedCacheKey}" 34 | sh "rm -rf ${cacheTarball}" 35 | 36 | echo 'Cache restored!' 37 | break; 38 | } catch(Exception ex) { 39 | echo "Error occurred while unpacking cache from ${serializedCacheKey}" 40 | echo "${ex.toString()}\n${ex.getStackTrace().join("\n")}" 41 | cacheExists = false 42 | sh "rm -rf ${cacheTarball}" 43 | } 44 | } 45 | } 46 | 47 | if (!cacheExists) { 48 | echo 'Unable to find cache stored for any of the provided keys' 49 | } 50 | } 51 | -------------------------------------------------------------------------------- /vars/cacheStore.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for caching artifacts that will be shared across builds 3 | * 4 | * @param 5 | */ 6 | 7 | import com.invoca.util.CacheUtil 8 | 9 | void call(String s3Bucket, ArrayList cacheKeys, ArrayList itemsToCache, Boolean global = false) { 10 | String cacheDirectory = global ? "s3://${s3Bucket}/jenkins_cache" : "s3://${s3Bucket}/jenkins_cache/${env.JOB_NAME.replaceAll("\\W", "")}"; 11 | String serializedTarballKey = CacheUtil.sanitizeCacheKey(cacheKeys[0]) 12 | String cacheTarball = "${serializedTarballKey}.tar.gz" 13 | 14 | // Verify that aws-cli is installed before proceeding 15 | sh 'which aws' 16 | 17 | echo 'Creating local cache tarball' 18 | sh "tar -czf ${cacheTarball} ${itemsToCache.join(' ')}" 19 | 20 | echo 'Pushing cache to AWS' 21 | cacheKeys.each { cacheKey -> 22 | String serializedCacheKey = CacheUtil.sanitizeCacheKey(cacheKey) 23 | echo "Serialized cacheKey from ${cacheKey} => ${serializedCacheKey}" 24 | try { 25 | String cacheLocation = "${cacheDirectory}/${serializedCacheKey}.tar.gz" 26 | sh "aws s3 cp ${cacheTarball} ${cacheLocation} --content-type application/x-gzip" 27 | } catch(Exception ex) { 28 | echo "Error occurred while pushing cache to ${serializedCacheKey}" 29 | echo "${ex.toString()}\n${ex.getStackTrace().join("\n")}" 30 | } 31 | } 32 | 33 | echo 'Cleaning up local cache tarball' 34 | sh "rm -rf ${cacheTarball}" 35 | 36 | echo 'Caching complete' 37 | } 38 | -------------------------------------------------------------------------------- /vars/chefPipeline.groovy: -------------------------------------------------------------------------------- 1 | def call(Closure body = null) { 2 | def TEST_RUNNER_CONTAINER_NAME = 'chef' 3 | def TEST_RUNNER_IMAGE_NAME = 'invocaops/chef-ci:master' 4 | 5 | def uuid = UUID.randomUUID().toString() 6 | def pipelineParams = [:] 7 | 8 | if (body != null) { 9 | body.resolveStrategy = Closure.DELEGATE_FIRST 10 | body.delegate = pipelineParams 11 | body() 12 | } 13 | 14 | pipeline { 15 | agent none 16 | 17 | environment { 18 | GEMFURY_TOKEN = credentials("gemfury-token") 19 | GITHUB_SSH_KEY = credentials('github-ssh-key') 20 | TEST_KITCHEN_SSH_KEY = credentials('test-kitchen-ssh-key') 21 | AWS_CREDENTIALS = credentials('aws-test-kitchen') 22 | AWS_ACCESS_KEY_ID = "${env.AWS_CREDENTIALS_USR}" 23 | AWS_SECRET_ACCESS_KEY = "${env.AWS_CREDENTIALS_PSW}" 24 | AWS_DEFAULT_REGION = 'us-east-1' 25 | } 26 | stages { 27 | stage('Run tests') { 28 | parallel { 29 | stage('Run unit tests') { 30 | agent { 31 | kubernetes { 32 | label "leroy-unit-${uuid}" 33 | containerTemplate { 34 | name TEST_RUNNER_CONTAINER_NAME 35 | image TEST_RUNNER_IMAGE_NAME 36 | alwaysPullImage true 37 | ttyEnabled true 38 | command 'cat' 39 | resourceRequestCpu '500m' 40 | resourceLimitMemory '1500Mi' 41 | } 42 | } 43 | } 44 | steps { 45 | container(TEST_RUNNER_CONTAINER_NAME) { 46 | sh """ 47 | eval `ssh-agent -s` 48 | echo "$GITHUB_SSH_KEY" | ssh-add - 49 | mkdir -p /root/.ssh 50 | ssh-keyscan -t rsa github.com > /root/.ssh/known_hosts 51 | bundle install 52 | bundle exec berks install 53 | bundle exec rake jenkins:unit 54 | """ 55 | } 56 | } 57 | } 58 | stage('Run integration tests') { 59 | agent { 60 | kubernetes { 61 | label "leroy-integration-${uuid}" 62 | containerTemplate { 63 | name TEST_RUNNER_CONTAINER_NAME 64 | image TEST_RUNNER_IMAGE_NAME 65 | alwaysPullImage true 66 | ttyEnabled true 67 | command 'cat' 68 | resourceRequestCpu '500m' 69 | resourceLimitMemory '1Gi' 70 | } 71 | } 72 | } 73 | steps { 74 | retry(3) { 75 | container(TEST_RUNNER_CONTAINER_NAME) { 76 | sh """ 77 | eval `ssh-agent -s` 78 | echo "$GITHUB_SSH_KEY" | ssh-add - 79 | echo "$TEST_KITCHEN_SSH_KEY" | ssh-add - 80 | mkdir -p /root/.ssh 81 | ssh-keyscan -t rsa github.com > /root/.ssh/known_hosts 82 | bundle install 83 | bundle exec berks install 84 | bundle exec rake jenkins:integration 85 | """ 86 | } 87 | } 88 | } 89 | } 90 | } 91 | } 92 | } 93 | } 94 | } 95 | -------------------------------------------------------------------------------- /vars/gemfileLockChecksum.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for a checksum against the contents of the Gemfile.lock 3 | * 4 | * @param 5 | */ 6 | 7 | String call() { 8 | return sh(script: "sha1sum -b Gemfile.lock | cut -d ' ' -f 1", returnStdout: true).trim(); 9 | } 10 | -------------------------------------------------------------------------------- /vars/gitHubStatus.groovy: -------------------------------------------------------------------------------- 1 | import com.invoca.github.GitHubStatus 2 | 3 | def call(Map config) { 4 | config.script = this 5 | GitHubStatus.update(config) 6 | } 7 | -------------------------------------------------------------------------------- /vars/githubShallowCloneCommit.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for doing a quick and clean clone of a single commit on GitHub 3 | * 4 | * @param repo String The repository to pull the commit from (ex. Invoca/web) 5 | * @param commit String The sha or branch to pull down from GitHub 6 | * @param githubToken String The GitHub Access Token to use for authenticating with GitHub 7 | */ 8 | 9 | void call(String repo, String commit, String githubToken) { 10 | echo "Checking out ${commit}" 11 | sh "git init" 12 | sh "git remote add origin https://${githubToken}:x-oauth-basic@github.com/${repo}" 13 | sh "git fetch --depth 1 origin ${commit}" 14 | sh "git checkout ${commit}" 15 | sh "git rev-parse HEAD" 16 | } 17 | -------------------------------------------------------------------------------- /vars/notifySlack.groovy: -------------------------------------------------------------------------------- 1 | def call (String buildStatus, String message = "", String channel = "#dev-jenkins") { 2 | // build status of null means SUCCESS 3 | buildStatus = buildStatus ?: 'SUCCESS' 4 | 5 | def String color 6 | switch(buildStatus) { 7 | case 'ABORTED': 8 | color = '#D3D3D3' 9 | break 10 | case 'FAILURE': 11 | color = 'danger' 12 | break 13 | case 'ONGOING': 14 | color = '#D3D3D3' 15 | break 16 | case 'SUCCESS': 17 | color = 'good' 18 | break 19 | case 'UNSTABLE': 20 | color = 'warning' 21 | break 22 | default: 23 | color = 'warning' 24 | break 25 | } 26 | 27 | def String msg 28 | if (message) { 29 | msg = message 30 | } else { 31 | // SUCCESS: Job 'web/PR-7750 [42]' (https://jenkins.instance.com/job/web/job/PR-7750/42/) 32 | msg = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})" 33 | } 34 | 35 | try { 36 | slackSend(channel: channel, color: color, message: msg) 37 | } 38 | catch (exc) { 39 | echo "Unable to notify Slack!" 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /vars/runWithDeprecationWarningCheck.groovy: -------------------------------------------------------------------------------- 1 | import com.invoca.ci.DeprecationWarnings 2 | 3 | def call(String script, Map githubStatusConfig) { 4 | def testOutput = sh(returnStdout: true, script: "${script} 2>&1") 5 | echo testOutput 6 | 7 | githubStatusConfig.script = this 8 | DeprecationWarnings.checkAndUpdateGithub(this, testOutput, githubStatusConfig) 9 | } 10 | -------------------------------------------------------------------------------- /vars/shWithSSHAgent.groovy: -------------------------------------------------------------------------------- 1 | /** 2 | * Helper method for running a sh command after having added an SSH agent. 3 | * Requires a valid GITHUB_KEY environment variable. 4 | * 5 | * @param command The sh command to run 6 | */ 7 | 8 | void call(command) { 9 | sh """ 10 | # Get SSH setup inside the container 11 | eval `ssh-agent -s` 12 | echo "$GITHUB_KEY" | ssh-add - 13 | mkdir -p /root/.ssh 14 | ssh-keyscan -t rsa github.com > /root/.ssh/known_hosts 15 | $command 16 | """ 17 | } --------------------------------------------------------------------------------