├── CONTRIBUTING.md ├── LICENSE.md ├── README.md ├── cf-cron.js ├── count3.sh ├── crontab.json ├── crontab.yml ├── jobs └── cf-quota-perm-set │ ├── README.md │ ├── cf │ ├── cf-cron.js │ ├── job.sh │ ├── manifest.yml │ ├── package.json │ ├── prep.sh │ └── set-quota-auditor.sh ├── manifest.yml └── package.json /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | ## Welcome! 2 | 3 | We're so glad you're thinking about contributing to an 18F open source project! If you're unsure about anything, just ask -- or submit the issue or pull request anyway. The worst that can happen is you'll be politely asked to change something. We love all friendly contributions. 4 | 5 | We want to ensure a welcoming environment for all of our projects. Our staff follow the [18F Code of Conduct](https://github.com/18F/code-of-conduct/blob/master/code-of-conduct.md) and all contributors should do the same. 6 | 7 | We encourage you to read this project's CONTRIBUTING policy (you are here), its [LICENSE](LICENSE.md), and its [README](README.md). 8 | 9 | If you have any questions or want to read more, check out the [18F Open Source Policy GitHub repository]( https://github.com/18f/open-source-policy), or just [shoot us an email](mailto:18f@gsa.gov). 10 | 11 | ## Public domain 12 | 13 | This project is in the public domain within the United States, and 14 | copyright and related rights in the work worldwide are waived through 15 | the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/). 16 | 17 | All contributions to this project will be released under the CC0 18 | dedication. By submitting a pull request, you are agreeing to comply 19 | with this waiver of copyright interest. 20 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | As a work of the United States Government, this project is in the 2 | public domain within the United States. 3 | 4 | Additionally, we waive copyright and related rights in the work 5 | worldwide through the CC0 1.0 Universal public domain dedication. 6 | 7 | ## CC0 1.0 Universal Summary 8 | 9 | This is a human-readable summary of the [Legal Code (read the full text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode). 10 | 11 | ### No Copyright 12 | 13 | The person who associated a work with this deed has dedicated the work to 14 | the public domain by waiving all of his or her rights to the work worldwide 15 | under copyright law, including all related and neighboring rights, to the 16 | extent allowed by law. 17 | 18 | You can copy, modify, distribute and perform the work, even for commercial 19 | purposes, all without asking permission. 20 | 21 | ### Other Information 22 | 23 | In no way are the patent or trademark rights of any person affected by CC0, 24 | nor are the rights that other persons may have in the work or in how the 25 | work is used, such as publicity or privacy rights. 26 | 27 | Unless expressly stated otherwise, the person who associated a work with 28 | this deed makes no warranties about the work, and disclaims liability for 29 | all uses of the work, to the fullest extent permitted by applicable law. 30 | When using or citing the work, you should not imply endorsement by the 31 | author or the affirmer. 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # cf-cron 2 | Run cron jobs in a Cloud Foundry app. 3 | 4 | ### Usage: 5 | 6 | This app expects a bound service holding credentials and a `crontab.yml` or `crontab.json` with the following format: 7 | 8 | **crontab.yml** 9 | 10 | ``` 11 | --- 12 | jobs: 13 | - name: job-1 14 | schedule: "0 * * * * *" 15 | command: "/bin/bash, job.sh, creds.key" 16 | tz: "America/Los_Angeles" 17 | prep: 18 | name: job-1-prep 19 | command: "/bin/bash, prep.sh, creds.username, creds.password" 20 | - name: job-2 21 | schedule: "0 * * * * *" 22 | command: "/bin/bash, job2.sh, creds.key" 23 | ``` 24 | 25 | ##### Job Parameters: 26 | 27 | * `prep:`
28 | **Description:** An optional job which runs once, prior to the recurring job. Takes an array-formatted command with optional parameters. Specify a script here to avoid unexpected issues with variable expansion.
29 | 30 | **Example:** `"/bin/bash, prep.sh"` 31 | 32 | * `command:`
33 | **Description:** The recurring job. An array-formatted command with optional parameters. Specify a script here to avoid unexpected issues with variable expansion.
34 | **Example:** `"/bin/bash, job.sh, creds.username, creds.password"` 35 | 36 | Parameters passed to `PREP_JOB` and `CRON_JOB` with the `creds.` prefix will eval to variables held in the service specified by `CF_CREDS`. 37 | 38 | * `schedule:`
39 | **Description:** A cron schedule. For more information see: [Cron Ranges](https://www.npmjs.com/package/cron#cron-ranges). This parameter only applies to cronjobs, prep jobs run immediately.
40 | **Example:** `"0 * * * * *"` 41 | 42 | * `tz: [optional]`
43 | **Description:** Time Zone for the cronjob.
44 | **Example:** `"America/Los_Angeles"` 45 | 46 | ##### Credential Service: 47 | 48 | This app will take advantage of secrets held in the credentials of either a bound service or a user-provided service. To store arbitrary credentials or non-public variables, use the syntax below to set up a user-provided service. 49 | 50 | ``` 51 | cf cups cf-cron-creds -p '{"username":"user", "password":"password"}' 52 | ``` 53 | 54 | ### Running the Example: 55 | 56 | Create the credential service. 57 | 58 | ``` 59 | cf cups cf-cron-creds -p '{"username":"user", "password":"password"}' 60 | ``` 61 | 62 | Push the app. 63 | 64 | ``` 65 | cf push 66 | ``` 67 | 68 | The app will: 69 | 70 | 1. Start the first prep job which counts to 3. 71 | 2. Start the second cronjob which echoes betelgeuse every 3 seconds. 72 | 3. When the first prep completes, start the first cronjob which echoes sirius every 5 seconds. 73 | 74 | **Output:** 75 | 76 | ``` 77 | Found crontab.yml. 78 | cf-cron started... 79 | Found 2 jobs. 80 | 0:job-1 81 | 1:job-2 82 | Preparing for: job-1 with job-1-prep 83 | Creating Job: job-2 84 | Prep: job-1-prep - Out: Count 1/3 85 | Prep: job-1-prep - Out: Count 2/3 86 | Job: job-2 - Out: betelgeuse 87 | Job: job-2 - Exit: 0 88 | Prep: job-1-prep - Out: Count 3/3 89 | Prep: job-1-prep - Exit: 0 90 | Finished: job-1-prep 91 | Creating Job: job-1 92 | Job: job-2 - Out: betelgeuse 93 | Job: job-2 - Exit: 0 94 | Job: job-1 - Out: sirius 95 | Job: job-1 - Exit: 0 96 | ``` 97 | -------------------------------------------------------------------------------- /cf-cron.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var CronJob = require('cron').CronJob, 4 | 5 | // Get the name of our user-provided credential service. 6 | cf_creds = process.env.CF_CREDS, 7 | 8 | // Use cfenv to grab our credentials from the credential service. 9 | cfenv = require("cfenv"), 10 | appEnv = cfenv.getAppEnv(), 11 | creds = appEnv.getServiceCreds(cf_creds), 12 | 13 | // For reading crontab.json 14 | fs = require('fs'), 15 | 16 | // For running all of our jobs. 17 | spawn = require('child_process').spawn, 18 | 19 | // YAML to parse the crontab. 20 | YAML = require('yamljs'); 21 | 22 | // Does the parameter look like the name of a service credential. 23 | function credEval(cred) { 24 | if (cred.split(".")[0] === 'creds') { 25 | // Is there a better way do this? Evaluating single strings 26 | // doesn't seem terrible. 27 | return eval(cred); 28 | } 29 | return cred; 30 | } 31 | 32 | // Clean up parameters and check if they're service credentials. 33 | function evalJob(job) { 34 | var comma = ','; 35 | job = job.split(comma); 36 | job = job.map(Function.prototype.call, String.prototype.trim); 37 | job = job.map(credEval); 38 | return job; 39 | } 40 | 41 | // Make a new cronjob. 42 | function makeJob(entry) { 43 | 44 | // Say something about which job we're on. 45 | console.log('Creating Job: ' + entry.name); 46 | 47 | // Set the timezone null if not provided. 48 | if (!entry.hasOwnProperty('tz')) { 49 | entry.tz = null; 50 | } 51 | 52 | // Create a new cronjob. 53 | new CronJob(entry.schedule, function () { 54 | // Carve up the prep job into command and params. 55 | var job = evalJob(entry.command), 56 | job_run; 57 | 58 | if (job.length > 1) { 59 | job_run = spawn(job[0], job.slice(1)); 60 | } else { 61 | job_run = spawn(job[0]); 62 | } 63 | 64 | // Handle and label job output. 65 | job_run.stdout.on('data', function (data) { 66 | console.log('Job: ' + entry.name + ' - Out: ' + data); 67 | }); 68 | 69 | job_run.stderr.on('data', function (data) { 70 | console.log('Job: ' + entry.name + ' - Err: ' + data); 71 | }); 72 | 73 | job_run.on('close', function (code) { 74 | console.log('Job: ' + entry.name + ' - Exit: ' + code); 75 | }); 76 | }, 77 | null, 78 | true, 79 | entry.tz); 80 | } 81 | 82 | // Make a new prep job. 83 | function makePrep(entry) { 84 | // Say something about the job in progress. 85 | console.log('Preparing for: ' + entry.name + ' with ' + entry.prep.name); 86 | 87 | // Run our prep commands. 88 | var spawn = require('child_process').spawn, 89 | prep = evalJob(entry.prep.command), 90 | prep_run; 91 | 92 | // Carve up the prep job into command and params. 93 | if (prep.length > 1) { 94 | prep_run = spawn(prep[0], prep.slice(1)); 95 | } else { 96 | prep_run = spawn(prep[0]); 97 | } 98 | 99 | // Handle and label job output. 100 | prep_run.stdout.on('data', function (data) { 101 | console.log('Prep: ' + entry.prep.name + ' - Out: ' + data); 102 | }); 103 | 104 | prep_run.stderr.on('data', function (data) { 105 | console.log('Prep: ' + entry.prep.name + ' - Err: ' + data); 106 | }); 107 | 108 | prep_run.on('close', function (code) { 109 | console.log('Prep: ' + entry.prep.name + ' - Exit: ' + code); 110 | 111 | // Set up cron and run the job. 112 | console.log('Finished: ' + entry.prep.name); 113 | if (code === 0) { 114 | // Now run the job. 115 | makeJob(entry); 116 | } else { 117 | console.log('Prep job failed. Stopping.'); 118 | process.exit(); 119 | } 120 | }); 121 | } 122 | 123 | // Look for a crontab. Try yaml then json. 124 | var crontab; 125 | 126 | try { 127 | crontab = YAML.parse(fs.readFileSync('crontab.yml', 'utf8')); 128 | console.log('Found crontab.yml.'); 129 | } catch (e) { 130 | try { 131 | crontab = JSON.parse(fs.readFileSync('crontab.json', 'utf8')); 132 | console.log('Found crontab.json.'); 133 | } catch (e) { 134 | console.log('No crontabs found.'); 135 | console.log('Please add crontab.yml or crontab.json.'); 136 | console.log('The error was: ' + e); 137 | process.exit(); 138 | } 139 | } 140 | 141 | // Lets begin. 142 | console.log("cf-cron started..."); 143 | 144 | // Summarize the crontab. 145 | console.log('Found ' + crontab.jobs.length + ' jobs.'); 146 | crontab.jobs.forEach(function (item, index) { 147 | console.log(index + ':' + item.name); 148 | }); 149 | 150 | // Run jobs. 151 | crontab.jobs.forEach(function (item, index) { 152 | if (item.hasOwnProperty('prep')) { 153 | makePrep(item); 154 | } else { 155 | makeJob(item); 156 | } 157 | }); -------------------------------------------------------------------------------- /count3.sh: -------------------------------------------------------------------------------- 1 | for i in `seq 3`; do echo "Count ${i}/3 "; sleep 1; done -------------------------------------------------------------------------------- /crontab.json: -------------------------------------------------------------------------------- 1 | { 2 | "jobs": [ 3 | { 4 | "name": "job-1", 5 | "schedule": "*/3 * * * * *", 6 | "command": "/bin/echo, sirius", 7 | "prep": { 8 | "name": "job-1-prep", 9 | "command": "/bin/bash, count3.sh" 10 | } 11 | }, 12 | { 13 | "name": "job-2", 14 | "schedule": "*/5 * * * * *", 15 | "command": "/bin/echo, betelgeuse" 16 | } 17 | ] 18 | } 19 | -------------------------------------------------------------------------------- /crontab.yml: -------------------------------------------------------------------------------- 1 | --- 2 | jobs: 3 | - name: job-1 4 | schedule: "*/3 * * * * *" 5 | command: "/bin/echo, sirius" 6 | prep: 7 | name: job-1-prep 8 | command: "/bin/bash, count3.sh" 9 | - name: job-2 10 | schedule: "*/5 * * * * *" 11 | command: "/bin/echo, betelgeuse" -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/README.md: -------------------------------------------------------------------------------- 1 | # cf-quota-perm-set 2 | Periodically set OrgAuditor and SpaceAuditor perms across a Cloud Foundry environment. 3 | 4 | ### Usage: 5 | 6 | See [cf-cron](https://github.com/18F/cf-cron) for general usage. 7 | 8 | ### Running the Job: 9 | 10 | The `set-quota-auditor.sh` script expects to find all the parameters it needs in order to authenticate and set permissions. 11 | 12 | Create a credential service with following keys and appropriate values. 13 | 14 | ``` 15 | cf cups cf-cron-creds -p '{"cf_api":"API_URL", "username":"ORGMANAGER", "password":"PASSWORD", "cf_org":"DEFAULT_ORG", "cf_space":"DEFAULT_SPACE", "auditor":"AUDITOR_USERNAME"}' 16 | ``` 17 | 18 | Push the app. 19 | 20 | ``` 21 | cf push 22 | ``` 23 | The app will set *Auditor permissions for every Org / Space in the environment. 24 | 25 | ``` 26 | Started... 27 | Prep_Out: API endpoint: API_URL 28 | Prep_Out: Authenticating... 29 | Prep_Out: OK 30 | Prep_Out: Targeted org DEFAULT_ORG 31 | Prep_Out: Targeted space DEFAULT_SPACE 32 | Prep_Out: 33 | API endpoint: API_URL (API version: x.x.x) 34 | User: USERNAME 35 | Org: DEFAULT_ORG 36 | Space: DEFAULT_SPACE 37 | Prep_Exit:0 38 | Scheduling... 39 | Setting: org1 40 | Setting: org1 / space1 41 | Setting: org1 / space2 42 | ... 43 | Setting: org43 / space4 44 | Setting: org43 / space4 45 | Set: Orgs - 43 / Spaces - 211 / Errors - 0 46 | ``` -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/cf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cloud-gov/cg-cron/8232814f4d2969da2ce420f61d8901a565494da5/jobs/cf-quota-perm-set/cf -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/cf-cron.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | // Give a little help when failing. 4 | function friendlyExit(reason) { 5 | if (reason === 'env_job') { 6 | console.log('You must set the environment variable CRON_JOB.'); 7 | process.exit(); 8 | } 9 | } 10 | 11 | // Does the parameter look like the name of a service credential. 12 | function credEval(cred) { 13 | if (cred.split(".")[0] === 'creds') { 14 | // Is there a better way do this? Evaluating single strings 15 | // doesn't seem terrible. 16 | return eval(cred); 17 | } 18 | return cred; 19 | } 20 | 21 | // Clean up parameters and check if they're service credentials. 22 | function evalJob(job) { 23 | var comma = ','; 24 | job = job.split(comma); 25 | job = job.map(Function.prototype.call, String.prototype.trim); 26 | job = job.map(credEval); 27 | return job; 28 | } 29 | 30 | // Get the name of our user-provided credential service. 31 | var cf_creds = process.env.CF_CREDS; 32 | 33 | // Use cfenv to grab our credentials from the credential service. 34 | var cfenv = require("cfenv"); 35 | var appEnv = cfenv.getAppEnv(); 36 | var creds = appEnv.getServiceCreds(cf_creds); 37 | 38 | // Get the command. 39 | var env_job = process.env.CRON_JOB || friendlyExit('env_job'); 40 | 41 | // Clean up extra whitespace here to give some leeway in job formatting. 42 | var prep_job = process.env.PREP_JOB || false; 43 | 44 | // If there's a prep job, parse it. Otherwise, the job is bash NOP. 45 | if (prep_job) { 46 | var prep = evalJob(prep_job); 47 | } else { 48 | var prep = ":"; 49 | } 50 | 51 | // Parse the cron job. 52 | var job = evalJob(env_job); 53 | 54 | // Get the command schedule. 55 | var schedule = process.env.CRON_SCHEDULE; 56 | 57 | // Lets begin. 58 | console.log("Started..."); 59 | 60 | // Run our prep commands. 61 | var spawn = require('child_process').spawn; 62 | 63 | // Carve up the prep job into command and params. 64 | if (prep.length > 1) { 65 | var prep_run = spawn(prep[0], prep.slice(1)); 66 | } else { 67 | var prep_run = spawn(prep[0]); 68 | } 69 | 70 | // Handle and label job output. 71 | prep_run.stdout.on('data', function (data) { 72 | console.log('Prep_Out: ' + data); 73 | }); 74 | 75 | prep_run.stderr.on('data', function (data) { 76 | console.log('Prep_Err: ' + data); 77 | }); 78 | 79 | prep_run.on('close', function (code) { 80 | console.log('Prep_Exit:' + code); 81 | 82 | // Set up cron and run the job. 83 | console.log('Scheduling...'); 84 | 85 | var CronJob = require('cron').CronJob; 86 | 87 | new CronJob(schedule, function () { 88 | // Carve up the prep job into command and params. 89 | var job_run; 90 | if (job.length > 1) { 91 | job_run = spawn(job[0], job.slice(1)); 92 | } else { 93 | job_run = spawn(job[0]); 94 | } 95 | 96 | // Handle and label job output. 97 | job_run.stdout.on('data', function (data) { 98 | console.log('Job_Out: ' + data); 99 | }); 100 | 101 | job_run.stderr.on('data', function (data) { 102 | console.log('Job_Err: ' + data); 103 | }); 104 | 105 | job_run.on('close', function (code) { 106 | console.log('Job_Exit:' + code); 107 | }); 108 | }, null, true, 'America/New_York'); 109 | }); 110 | -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/job.sh: -------------------------------------------------------------------------------- 1 | # Lets make sure we're prepped. 2 | set -e 3 | 4 | export PATH=~/:$PATH 5 | 6 | if [ -e .prepped ] 7 | then 8 | /bin/bash set-quota-auditor.sh $1 9 | else 10 | echo "Nope. We're unprepaed." 11 | fi 12 | -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/manifest.yml: -------------------------------------------------------------------------------- 1 | --- 2 | applications: 3 | - name: cf-quota-perm-set 4 | no-route: true 5 | disk_quota: 128M 6 | memory: 128M 7 | env: 8 | CF_CREDS: cf-quota-perm-set-creds 9 | PREP_JOB: "/bin/bash, prep.sh, creds.cf_api, creds.username, creds.password, creds.cf_org, creds.cf_space" 10 | CRON_JOB: "/bin/bash, job.sh, creds.auditor" 11 | CRON_SCHEDULE: "0 0 * * * *" 12 | services: 13 | - cf-quota-perm-set-creds 14 | -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "cf-cron", 3 | "version": "0.0.1", 4 | "description": "Run cron jobs in a Cloud Foundry app.", 5 | "main": "cf-cron.js", 6 | "scripts": { 7 | "start": "node cf-cron.js", 8 | "test": "echo \"Error: no test specified\" && exit 1" 9 | }, 10 | "author": "", 11 | "license": "BSD-2-Clause", 12 | "dependencies" : 13 | { "cron" : "1.0.9", 14 | "cfenv" : "1.0.0" 15 | }, 16 | "engines": { 17 | "node": "0.12.x" 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/prep.sh: -------------------------------------------------------------------------------- 1 | # Lets get prepped. 2 | set -e 3 | 4 | export PATH=~/:$PATH 5 | 6 | cf login -a $1 -u $2 -p $3 -o $4 -s $5 7 | 8 | touch .prepped 9 | -------------------------------------------------------------------------------- /jobs/cf-quota-perm-set/set-quota-auditor.sh: -------------------------------------------------------------------------------- 1 | # This script walks through each org in a Cloud Foundry instance setting 2 | # OrgAuditor and SpaceAuditor permissions at each level. 3 | # 4 | # Usage: 5 | # 6 | # set-quota-auditor.sh AUDIT_USER 7 | USER_EMAIL=$1 8 | 9 | # Strip the host portion of the email supplied. 10 | USER_NAME="${USER_EMAIL%%@*}" 11 | 12 | # Initialize counters for success / failure. 13 | org_count=0 14 | spc_count=0 15 | err_count=0 16 | 17 | # Loop over org and spaces. 18 | # Use process substitution to keep track of the counts in this loop. 19 | # http://mywiki.wooledge.org/ProcessSubstitution 20 | while read ORG_NAME 21 | do 22 | echo "Setting: $ORG_NAME" 23 | cf set-org-role $USER_NAME $ORG_NAME OrgAuditor > /dev/null 24 | if [ $? -ne 0 ]; then 25 | err_count=$((err_count+=1)) 26 | else 27 | org_count=$((org_count+=1)) 28 | fi 29 | 30 | cf target -o $ORG_NAME > /dev/null 31 | 32 | while read SPACE_NAME 33 | do 34 | echo "Setting: $ORG_NAME / $SPACE_NAME" 35 | cf set-space-role $USER_NAME $ORG_NAME $SPACE_NAME SpaceAuditor > /dev/null 36 | if [ $? -ne 0 ]; then 37 | err_count=$((err_count+=1)) 38 | else 39 | spc_count=$((spc_count+=1)) 40 | fi 41 | done < <(cf spaces | awk 'm;/^name/{m=1}') 42 | done < <(cf orgs | awk 'm;/^name/{m=1}') 43 | 44 | # Print a summary. 45 | echo "Set: Orgs - $org_count / Spaces - $spc_count / Errors - $err_count" 46 | -------------------------------------------------------------------------------- /manifest.yml: -------------------------------------------------------------------------------- 1 | --- 2 | applications: 3 | - name: cf-cron 4 | no-route: true 5 | disk_quota: 256M 6 | memory: 128M 7 | env: 8 | CF_CREDS: cf-cron-creds 9 | services: 10 | - cf-cron-creds 11 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "cf-cron", 3 | "version": "0.0.1", 4 | "description": "Run cron jobs in a Cloud Foundry app.", 5 | "main": "cf-cron.js", 6 | "scripts": { 7 | "start": "node cf-cron.js", 8 | "test": "echo \"Error: no test specified\" && exit 1" 9 | }, 10 | "author": "", 11 | "license": "BSD-2-Clause", 12 | "dependencies" : 13 | { "cron" : "1.0.9", 14 | "cfenv" : "1.0.0", 15 | "yamljs" : "0.2.3" 16 | }, 17 | "engines": { 18 | "node": "0.12.x" 19 | } 20 | } 21 | --------------------------------------------------------------------------------