├── .circleci └── config.yml ├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md └── pull_request_template.md ├── .gitignore ├── CODEOWNERS ├── LICENSE.txt ├── README.md ├── _ci └── run-tests.sh ├── bootstrap-gruntwork-installer.sh ├── examples └── packer-file-copy │ ├── files │ └── foo │ │ └── bar │ │ └── test.txt │ └── packer-example.json ├── gruntwork-install ├── modules ├── README.md ├── args-test │ ├── README.md │ └── install.sh ├── dummy-module │ ├── README.md │ └── install.sh ├── mixed-args-module │ ├── README.md │ └── install.sh └── packer-file-copy │ ├── README.md │ ├── install-scripts │ └── copy-packer-files.sh │ └── install.sh └── test ├── amazonlinux ├── Dockerfile └── docker-compose.yml ├── centos ├── Dockerfile └── docker-compose.yml ├── integration-test.sh ├── no-sudo-test.sh ├── no_sudo ├── Dockerfile └── docker-compose.yml ├── ubuntu ├── Dockerfile └── docker-compose.yml └── ubuntu18 ├── Dockerfile └── docker-compose.yml /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | defaults: &defaults 2 | working_directory: /home/circleci/.go_workspace/src/github.com/gruntwork-io/gruntwork-installer 3 | machine: 4 | enabled: true 5 | image: ubuntu-2204:2022.04.1 6 | version: 2 7 | jobs: 8 | test: 9 | <<: *defaults 10 | steps: 11 | - checkout 12 | - run: 13 | name: run tests 14 | command: | 15 | ./_ci/run-tests.sh 16 | workflows: 17 | version: 2 18 | install-and-test: 19 | jobs: 20 | - test: 21 | filters: 22 | tags: 23 | only: /^v.*/ 24 | context: 25 | - AWS__PHXDEVOPS__circle-ci-test 26 | - GITHUB__PAT__gruntwork-ci 27 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a bug report to help us improve. 4 | title: '' 5 | labels: bug 6 | assignees: '' 7 | 8 | --- 9 | 10 | 14 | 15 | **Describe the bug** 16 | A clear and concise description of what the bug is. 17 | 18 | **To Reproduce** 19 | Steps to reproduce the behavior including the relevant Terraform/Terragrunt/Packer version number and any code snippets and module inputs you used. 20 | 21 | ```hcl 22 | // paste code snippets here 23 | ``` 24 | 25 | **Expected behavior** 26 | A clear and concise description of what you expected to happen. 27 | 28 | **Nice to have** 29 | - [ ] Terminal output 30 | - [ ] Screenshots 31 | 32 | **Additional context** 33 | Add any other context about the problem here. 34 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Submit a feature request for this repo. 4 | title: '' 5 | labels: enhancement 6 | assignees: '' 7 | 8 | --- 9 | 10 | 14 | 15 | **Describe the solution you'd like** 16 | A clear and concise description of what you want to happen. 17 | 18 | **Describe alternatives you've considered** 19 | A clear and concise description of any alternative solutions or features you've considered. 20 | 21 | **Additional context** 22 | Add any other context or screenshots about the feature request here. 23 | -------------------------------------------------------------------------------- /.github/pull_request_template.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Description 4 | 5 | Fixes #000. 6 | 7 | 8 | 9 | ## TODOs 10 | 11 | Read the [Gruntwork contribution guidelines](https://gruntwork.notion.site/Gruntwork-Coding-Methodology-02fdcd6e4b004e818553684760bf691e). 12 | 13 | - [ ] Update the docs. 14 | - [ ] Run the relevant tests successfully, including pre-commit checks. 15 | - [ ] Ensure any 3rd party code adheres with our [license policy](https://www.notion.so/gruntwork/Gruntwork-licenses-and-open-source-usage-policy-f7dece1f780341c7b69c1763f22b1378) or delete this line if it's not applicable. 16 | - [ ] Include release notes. If this PR is backward incompatible, include a migration guide. 17 | 18 | ## Release Notes (draft) 19 | 20 | 21 | Added / Removed / Updated [X]. 22 | 23 | ### Migration Guide 24 | 25 | 26 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.iml -------------------------------------------------------------------------------- /CODEOWNERS: -------------------------------------------------------------------------------- 1 | * @brikis98 @denis256 2 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Gruntwork, LLC 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Maintained by Gruntwork.io](https://img.shields.io/badge/maintained%20by-gruntwork.io-%235849a6.svg)](https://gruntwork.io/?ref=repo_gruntwork_installer) 2 | # Gruntwork Installer 3 | 4 | `gruntwork-install` is a bash script you run to easily download and install Gruntwork Modules. 5 | 6 | ## Compatibility 7 | 8 | Tested under CentOS 7, latest Amazon Linux, and Ubuntu 16.04. 9 | 10 | ## Quick Start 11 | 12 | ### Install gruntwork-install 13 | 14 | If `gruntwork-install` is our approach for installing Gruntwork Modules, how do we install `gruntwork-install` itself? 15 | 16 | Our solution is to make the `gruntwork-install` tool open source and to publish a `bootstrap-gruntwork-installer.sh` 17 | script that anyone can use to install `gruntwork-install` itself. To use it, execute the following: 18 | 19 | ``` 20 | curl -LsS https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/v0.0.38/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.38 21 | ``` 22 | 23 | Notice the `--version` parameter at the end where you specify which version of `gruntwork-install` to install. See the 24 | [releases](https://github.com/gruntwork-io/gruntwork-installer/releases) page for all available versions. 25 | 26 | For those concerned about security, see [is it safe to pipe URLs into bash?](#is-it-safe-to-pipe-urls-into-bash) below. 27 | 28 | ### Use gruntwork-install 29 | 30 | #### Authentication 31 | 32 | To install scripts and binaries from private GitHub repos, you must create a [GitHub access 33 | token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) and set it as the environment 34 | variable `GITHUB_OAUTH_TOKEN` so `gruntwork-install` can use it to access the repo: 35 | 36 | ``` 37 | export GITHUB_OAUTH_TOKEN="(your secret token)" 38 | ``` 39 | 40 | #### Options 41 | 42 | Once that environment variable is set, you can run `gruntwork-install` with the following options: 43 | 44 | Option | Required | Description 45 | --------------------------- | -------- | ------------ 46 | `--repo` | Yes | The GitHub repo to install from. 47 | `--tag` | Yes | The version of the `--repo` to install from.
Follows the syntax described at [Tag Constraint Expressions](https://github.com/gruntwork-io/fetch#tag-constraint-expressions). This value is exposed to module install scripts as GRUNTWORK_INSTALL_TAG. 48 | `--module-name` | XOR | The name of a module to install.
Can be any folder within the `modules` directory of `--repo`.
You must specify exactly one of `--module-name` or `--binary-name`. 49 | `--binary-name` | XOR | The name of a binary to install.
Can be any file uploaded as a release asset in `--repo`.
You must specify exactly one of `--module-name` or `--binary-name`. 50 | `--binary-sha256-checksum` | No | The SHA256 checksum of the binary specified by `--binary-name`. Should be exactly 64 characters.. 51 | `--binary-sha512-checksum` | No | The SHA512 checksum of the binary specified by `--binary-name`. Should be exactly 128 characters.. 52 | `--module-param` | No | A key-value pair of the format `key=value` you wish to pass to the module as a parameter. May be used multiple times.
Note: a `--` will automatically be appended to the `key` when your module is invoked
See the documentation for each module to find out what parameters it accepts. 53 | `--download-dir` | No | The directory to which the module will be downloaded and from which it will be installed. 54 | `--binary-install-dir` | No | The directory to which the binary will be installed. Only applies to binaries (not modules). Default: DEFAULT_BIN_DIR. 55 | `--no-sudo` | No | When true, don't use sudo to install the binary into the install directory. Only applies to binaries (not modules). Default: false. 56 | `--branch` | No | Download the latest commit from this branch in --repo. This is an alternative to --tag,
and is used only for testing. This value is exposed to module install scripts as GRUNTWORK_INSTALL_BRANCH. 57 | `--ref` | No | Download the latest commit from this ref in --repo. This is an alternative to --tag,
and is used only for testing. This value is exposed to module install scripts as GRUNTWORK_INSTALL_BRANCH. 58 | `--help` | No | Show the help text and exit. 59 | 60 | #### Examples 61 | 62 | ##### Example 1: Download and Install a Script Module with No Parameters 63 | 64 | Install the [ecs-scripts 65 | module](https://github.com/gruntwork-io/terraform-aws-ecs/tree/main/modules/ecs-scripts) from the [terraform-aws-ecs 66 | repo](https://github.com/gruntwork-io/terraform-aws-ecs), version `v0.0.1`: 67 | 68 | ``` 69 | gruntwork-install --module-name 'ecs-scripts' --repo 'https://github.com/gruntwork-io/terraform-aws-ecs' --tag 'v0.0.1' 70 | ``` 71 | 72 | ##### Example 2: Download and Install a Script Module with Parameters 73 | 74 | Install the [fail2ban 75 | module](https://github.com/gruntwork-io/terraform-aws-security/tree/main/modules/fail2ban) from the [terraform-aws-security 76 | repo](https://github.com/gruntwork-io/terraform-aws-security), passing two custom parameters to it: 77 | 78 | 79 | ``` 80 | gruntwork-install --module-name 'fail2ban' --repo 'terraform-aws-security' --module-param 'ban-time=3600' 81 | ``` 82 | 83 | ##### Example 3: Download and Install a Binary Module 84 | 85 | Install the `gruntkms` binary from the `v0.0.1` release of the [gruntkms 86 | repo](https://github.com/gruntwork-io/gruntkms): 87 | 88 | ``` 89 | gruntwork-install --binary-name 'gruntkms' --repo 'https://github.com/gruntwork-io/gruntkms' --tag 'v0.0.1' 90 | ``` 91 | 92 | Note that the [v0.0.1 release of the gruntkms repo](https://github.com/gruntwork-io/gruntkms/releases/tag/v0.0.1) has 93 | multiple binaries (`gruntkms_linux_amd64`, `gruntkms_darwin_386`, etc): `gruntwork-install` automatically picks the 94 | right binary for your OS and copies it to `/usr/local/bin/gruntkms`. 95 | 96 | ##### Example 4: Use `gruntwork-install` in a Packer template 97 | 98 | Finally, to put all the pieces together, here is an example of a Packer template that installs `gruntwork-install` 99 | and then uses it to install several modules: 100 | 101 | ```json 102 | { 103 | "variables": { 104 | "github_auth_token": "{{env `GITHUB_OAUTH_TOKEN`}}" 105 | }, 106 | "builders": [ 107 | { 108 | "ami_name": "gruntwork-install-example-{{isotime | clean_ami_name}}", 109 | "instance_type": "t2.micro", 110 | "region": "us-east-1", 111 | "type": "amazon-ebs", 112 | "source_ami": "ami-fce3c696", 113 | "ssh_username": "ubuntu" 114 | } 115 | ], 116 | "provisioners": [ 117 | { 118 | "type": "shell", 119 | "inline": 120 | "curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/v0.0.38/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.16" 121 | }, 122 | { 123 | "type": "shell", 124 | "inline": [ 125 | "gruntwork-install --module-name 'ecs-scripts' --repo 'https://github.com/gruntwork-io/terraform-aws-ecs' --tag 'v0.0.1'", 126 | "gruntwork-install --module-name 'fail2ban' --repo 'https://github.com/gruntwork-io/terraform-aws-security' -module-param 'ban-time=3600'", 127 | "gruntwork-install --binary-name 'gruntkms' --repo 'https://github.com/gruntwork-io/gruntkms' --tag 'v0.0.1'" 128 | ], 129 | "environment_vars": ["GITHUB_OAUTH_TOKEN={{user `github_auth_token`}}"] 130 | } 131 | ] 132 | } 133 | ``` 134 | 135 | ## Motivation 136 | 137 | At [Gruntwork](http://www.gruntwork.io/), we've developed a number of scripts and binaries, most of them in private GitHub 138 | repos, that perform common infrastructure tasks such as setting up continuous integration, monitoring, log aggregation, 139 | and SSH access. Being able to use these "modules" of code typically involves many steps: you download the files 140 | (possibly from a private GitHub repo), change their permissions, and run them with the parameters that make sense for 141 | your environment. 142 | 143 | That basically means lots of custom `bash` code copied differently across multiple software teams in multiple different 144 | contexts. Worse, if we want to update a binary or script to add a new parameter, each team has to modify their own custom 145 | code, which can be painful. 146 | 147 | We believe we can do better by writing our scripts and binaries in a standardized way, and including a minimal tool that 148 | streamlines the process of downloading and installing them. Also, since we give you 100% of the source code, we want it 149 | to be clear exactly what happens when you install a Gruntwork Module. 150 | 151 | Finally, installation should be streamlined no matter what platform (Windows, MacOS, Linux) you're on. Indeed, our goal 152 | is to make installing Gruntwork Script Modules as easy as installing a typical package using `apt-get`, `yum`, `npm`, 153 | or similar tools. We would have just used these existing tools, but none offer multi-platform compatibility. 154 | 155 | ## What's a Gruntwork Module? 156 | 157 | A Gruntwork Module is a collection of one or more bash scripts and/or binaries maintained by Gruntwork that can be used to 158 | add functionality to or configure an environment. There are multiple types of Gruntwork Modules: 159 | 160 | * **Script Modules:** A collection of one or more files and scripts; installed with an `install.sh` script. 161 | * **Binary Modules:** A single OS-specific executable binary. 162 | 163 | Additional module types may be introduced in the future. 164 | 165 | As an example, we have Script Modules for installing a CloudWatch Logs agent, optimizing syslog settings, and setting up 166 | automatic security updates. We have a Binary Module for streamlining the use of Amazon Key Management Service (KMS). 167 | 168 | Gruntwork sells [Infrastructure Packages](https://blog.gruntwork.io/gruntwork-infrastructure-packages-7434dc77d0b1#.6bwor6wxc). 169 | Each Infrastructure Package corresponds to a specific GitHub repo and contains one or more Gruntwork Modules. The `/modules` 170 | folder in the repo lists all Modules included with that Package. 171 | 172 | ### Freely Available Script Modules 173 | 174 | Some Script Modules are so common that we've made them freely available in the [modules/](modules) folder of this repo. 175 | 176 | ### How `gruntwork-install` Works 177 | 178 | `gruntwork-install` helps you install a Gruntwork Module. Here's how it works: 179 | 180 | 1. It uses [fetch](https://github.com/gruntwork-io/fetch) to download the specified version of the scripts or binary from 181 | the (public or private) git repo specified via the `--repo` option. 182 | 1. You need to specify either a module name or a binary name. 183 | - If you use the `--module-name` parameter, it downloads the files from the `modules` folder of `--repo` and runs 184 | the `install.sh` script of that module. 185 | - If you use the `--binary-name` parameter, it downloads the right binary for your OS, copies it to `/usr/local/bin`, 186 | and gives it execute permissions. 187 | 188 | ## Create Your Own Gruntwork Modules 189 | 190 | You can use `gruntwork-install` with any GitHub repo, not just repos maintained by Gruntwork. 191 | 192 | That means that to create an installable Script Module, all you have to do is put it in the `modules` folder of 193 | a GitHub repo to which you have access and include an `install.sh` script. To create a Binary Module, you just publish 194 | it to a GitHub release with the name format `__`. 195 | 196 | ### Example 197 | 198 | For example, in your Packer and Docker templates, you can use `gruntwork-install` to install the [ecs-scripts 199 | module](https://github.com/gruntwork-io/terraform-aws-ecs/tree/main/modules/ecs-scripts) as follows: 200 | 201 | ``` 202 | gruntwork-install --module-name 'ecs-scripts' --repo 'https://github.com/gruntwork-io/terraform-aws-ecs' --tag 'v0.0.1' 203 | ``` 204 | 205 | In https://github.com/gruntwork-io/module-ecs, we download the contents of `/modules/ecs-scripts` and run 206 | `/modules/ecs-scripts/install.sh`. 207 | 208 | ## Running tests 209 | 210 | The tests for this repo are defined in the `test` folder. They are designed to run in a Docker container so that you 211 | do not repeatedly dirty up your local OS while testing. We've defined a `test/docker-compose.yml` file as a convenient 212 | way to expose the environment variables we need for testing and to mount local directories as volumes for rapid 213 | iteration. 214 | 215 | To run the tests: 216 | 217 | 1. Set your [GitHub access token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) as 218 | the environment variable `GITHUB_OAUTH_TOKEN`. 219 | 1. `./_ci/run-tests.sh` 220 | 221 | ## Security 222 | 223 | ### Validate the Downloaded Binary 224 | 225 | `gruntwork-install` will retrieve the desired GitHub Release Asset specified by the `--binary-name` property, but how 226 | can we confirm that this binary has not been tampered with? In short, we trust that the maintainer has been responsible 227 | and not allowed a malicious third-party to corrupt the Release Asset. 228 | 229 | You can narrow the scope of this trust by computing a checksum on a Release Asset using a UNIX command like 230 | `shasum -a 256 /path/to/file` when you first download the release. You can then feed this value (e.g. `b0b30cc24aed1b8cded2df903183b884c77f086efffc36ef19876d1c55fef93d`) 231 | to `--binary-sha256-checksum` or `--binary-sha512-checksum`. If the checksum does not match, gruntwork-install will fail 232 | with an error. This way, you are at least notified if the Release Asset you initially downloaded has since been changed. 233 | 234 | ### Is it safe to pipe URLs into bash? 235 | 236 | Are you worried that our install instructions tell you to pipe a URL into bash? Although this approach has seen some 237 | [backlash](https://news.ycombinator.com/item?id=6650987), we believe that the convenience of a one-line install 238 | outweighs the minimal security risks. Below is a brief discussion of the most commonly discussed risks and what you can 239 | do about them. 240 | 241 | #### Risk #1: You don't know what the script is doing, so you shouldn't blindly execute it. 242 | 243 | This is true of _all_ installers. For example, have you ever inspected the install code before running `apt-get install` 244 | or `brew install` or double clicking a `.dmg` or `.exe` file? If anything, a shell script is the most transparent 245 | installer out there, as it's one of the few that allows you to inspect the code (feel free to do so, as this script is 246 | open source!). The reality is that you either trust the developer or you don't. And eventually, you automate the 247 | install process anyway, at which point manual inspection isn't a possibility anyway. 248 | 249 | #### Risk #2: The download URL could be hijacked for malicious code. 250 | 251 | This is unlikely, as it is an https URL, and your download program (e.g. `curl`) should be verifying SSL certs. That 252 | said, Certificate Authorities have been hacked in the past, and perhaps the Gruntwork GitHub account could be hacked 253 | in the future, so if that is a major concern for you, feel free to copy the bootstrap code into your own codebase and 254 | execute it from there. Alternatively, in the future we will publish checksums of all of our releases, so you could 255 | optionally verify the checksum before executing the script. 256 | 257 | #### Risk #3: The script may not download fully and executing it could cause errors. 258 | 259 | We wrote our [bootstrap-gruntwork-installer.sh](bootstrap-gruntwork-installer.sh) as a series of bash functions that 260 | are only executed by the very last line of the script. Therefore, if the script doesn't fully download, the worst 261 | that'll happen when you execute it is a harmless syntax error. 262 | 263 | ## TODO 264 | 265 | 1. Configure a CI build to automatically set the `--version` flag for each release. 266 | 1. Add an `uninstall` command that uses an `uninstall.sh` script in each module. 267 | 1. Add support for modules declaring their dependencies. Alternatively, consider Nix again as a dependency manager. 268 | 269 | -------------------------------------------------------------------------------- /_ci/run-tests.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Runs the automated tests for this repo 3 | 4 | readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" 5 | cd "$SCRIPT_DIR/.." 6 | 7 | echo "Building docker containers for test" 8 | docker build -t gruntwork/gruntwork-installer-ubuntu test/ubuntu 9 | docker build -t gruntwork/gruntwork-installer-ubuntu18 test/ubuntu18 10 | docker build -t gruntwork/gruntwork-installer-amazonlinux test/amazonlinux 11 | docker build -t gruntwork/gruntwork-installer-centos test/centos 12 | docker build -t gruntwork/gruntwork-installer-no-sudo-ubuntu test/no_sudo 13 | 14 | echo "Running integration tests using docker-compose" 15 | docker-compose -f test/ubuntu/docker-compose.yml run installer /test/integration-test.sh 16 | docker-compose -f test/ubuntu18/docker-compose.yml run installer /test/integration-test.sh 17 | docker-compose -f test/amazonlinux/docker-compose.yml run installer /test/integration-test.sh 18 | docker-compose -f test/centos/docker-compose.yml run installer /test/integration-test.sh 19 | docker-compose -f test/no_sudo/docker-compose.yml run installer /test/no-sudo-test.sh 20 | -------------------------------------------------------------------------------- /bootstrap-gruntwork-installer.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # A bootstrap script to install the Gruntwork Installer. 4 | # 5 | # Usage: 6 | # 7 | # curl -LsS https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/v0.0.38/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.38 8 | # 9 | # Rationale: 10 | # 11 | # The Gruntwork Installer makes installing Gruntwork Script Modules as easy as installing a package using apt-get, 12 | # brew, or yum. However, something has to install the Gruntwork Installer first. One option is for each Gruntwork 13 | # client to do so manually, which would basically entail copying and pasting all the code below. This is tedious and 14 | # gives us no good way to push updates to this bootstrap script. 15 | # 16 | # So instead, we recommend that clients use this tiny bootstrap script. 17 | # 18 | # In your Packer and Docker templates, you can use the above one-liner to install the Gruntwork Installer, and then 19 | # start using the `gruntwork-install` command immediately. 20 | 21 | set -e 22 | 23 | readonly BIN_DIR="/usr/local/bin" 24 | readonly USER_DATA_DIR="/etc/user-data" 25 | 26 | readonly DEFAULT_FETCH_VERSION="v0.4.6" 27 | readonly FETCH_DOWNLOAD_URL_BASE="https://github.com/gruntwork-io/fetch/releases/download" 28 | readonly FETCH_INSTALL_PATH="$BIN_DIR/fetch" 29 | 30 | readonly GRUNTWORK_INSTALLER_DOWNLOAD_URL_BASE="https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer" 31 | readonly GRUNTWORK_INSTALLER_INSTALL_PATH="$BIN_DIR/gruntwork-install" 32 | readonly GRUNTWORK_INSTALLER_SCRIPT_NAME="gruntwork-install" 33 | 34 | function print_usage { 35 | echo 36 | echo "Usage: bootstrap-gruntwork-installer.sh [OPTIONS]" 37 | echo 38 | echo "A bootstrap script to install the Gruntwork Installer ($GRUNTWORK_INSTALLER_SCRIPT_NAME)." 39 | echo 40 | echo "Options:" 41 | echo 42 | echo -e " --version\t\tRequired. The version of $GRUNTWORK_INSTALLER_SCRIPT_NAME to install (e.g. v0.0.38)." 43 | echo -e " --fetch-version\tOptional. The version of fetch to install. Default: $DEFAULT_FETCH_VERSION." 44 | echo -e " --user-data-owner\tOptional. The user who shown own the $USER_DATA_DIR folder. Default: (current user)." 45 | echo -e " --download-url\tOptional. The URL from where to download $GRUNTWORK_INSTALLER_SCRIPT_NAME. Mostly used for automated tests. Default: $GRUNTWORK_INSTALLER_DOWNLOAD_URL_BASE/(version)/$GRUNTWORK_INSTALLER_SCRIPT_NAME." 46 | echo -e " --no-sudo\tOptional. When true, don't use sudo to install binaries. Default: false." 47 | echo 48 | echo "Examples:" 49 | echo 50 | echo " Install version v0.0.38:" 51 | echo " bootstrap-gruntwork-installer.sh --version v0.0.38" 52 | echo 53 | echo " One-liner to download this bootstrap script from GitHub and run it to install version v0.0.38:" 54 | echo " curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/v0.0.38/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.38" 55 | } 56 | 57 | function maybe_sudo { 58 | local -r no_sudo="$1" 59 | shift 60 | 61 | if [[ "$no_sudo" == "true" ]]; then 62 | "$@" 63 | else 64 | sudo "$@" 65 | fi 66 | } 67 | 68 | # http://stackoverflow.com/questions/592620/check-if-a-program-exists-from-a-bash-script 69 | function command_exists { 70 | local -r cmd="$1" 71 | type "$cmd" > /dev/null 2>&1 72 | } 73 | 74 | function download_url_to_file { 75 | local -r url="$1" 76 | local -r file="$2" 77 | local -r tmp_path=$(mktemp "/tmp/gruntwork-bootstrap-download-XXXXXX") 78 | local -r no_sudo="$3" 79 | 80 | echo "Downloading $url to $tmp_path" 81 | if command_exists "curl"; then 82 | local -r status_code=$(curl -L -s -w '%{http_code}' -o "$tmp_path" "$url") 83 | assert_successful_status_code "$status_code" "$url" 84 | 85 | echo "Moving $tmp_path to $file" 86 | maybe_sudo "$no_sudo" mv -f "$tmp_path" "$file" 87 | else 88 | echo "ERROR: curl is not installed. Cannot download $url." 89 | exit 1 90 | fi 91 | } 92 | 93 | function assert_successful_status_code { 94 | local -r status_code="$1" 95 | local -r url="$2" 96 | 97 | if [[ "$status_code" == "200" ]]; then 98 | echo "Got expected status code 200" 99 | elif string_starts_with "$url" "file://" && [[ "$status_code" == "000" ]]; then 100 | echo "Got expected status code 000 for local file URL" 101 | else 102 | echo "ERROR: Expected status code 200 but got $status_code when downloading $url" 103 | exit 1 104 | fi 105 | } 106 | 107 | function string_starts_with { 108 | local -r str="$1" 109 | local -r prefix="$2" 110 | 111 | [[ "$str" == "$prefix"* ]] 112 | } 113 | 114 | function string_contains { 115 | local -r str="$1" 116 | local -r contains="$2" 117 | 118 | [[ "$str" == *"$contains"* ]] 119 | } 120 | 121 | # http://stackoverflow.com/a/2264537/483528 122 | function to_lower_case { 123 | tr '[:upper:]' '[:lower:]' 124 | } 125 | 126 | function get_os_name { 127 | uname | to_lower_case 128 | } 129 | 130 | function get_os_arch { 131 | uname -m 132 | } 133 | 134 | function get_os_arch_gox_format { 135 | local -r arch=$(get_os_arch) 136 | 137 | if string_contains "$arch" "arm64"; then 138 | echo "arm64" 139 | elif string_contains "$arch" "aarch64"; then 140 | echo "arm64" 141 | elif string_contains "$arch" "64"; then 142 | echo "amd64" 143 | elif string_contains "$arch" "386"; then 144 | echo "386" 145 | elif string_contains "$arch" "686"; then 146 | echo "386" # Not a typo; 686 is also 32-bit and should work with 386 binaries 147 | elif string_contains "$arch" "arm"; then 148 | echo "arm" 149 | fi 150 | } 151 | 152 | function download_and_install { 153 | local -r url="$1" 154 | local -r install_path="$2" 155 | local -r no_sudo="$3" 156 | 157 | download_url_to_file "$url" "$install_path" "$no_sudo" 158 | maybe_sudo "$no_sudo" chmod 0755 "$install_path" 159 | } 160 | 161 | function install_fetch { 162 | local -r install_path="$1" 163 | local -r version="$2" 164 | local -r no_sudo="$3" 165 | 166 | local -r os=$(get_os_name) 167 | local -r os_arch=$(get_os_arch_gox_format) 168 | 169 | if [[ -z "$os_arch" ]]; then 170 | echo "ERROR: Unrecognized OS architecture: $(get_os_arch)" 171 | exit 1 172 | fi 173 | 174 | echo "Installing fetch version $version to $install_path" 175 | local -r url="${FETCH_DOWNLOAD_URL_BASE}/${version}/fetch_${os}_${os_arch}" 176 | download_and_install "$url" "$install_path" "$no_sudo" 177 | } 178 | 179 | function install_gruntwork_installer { 180 | local -r install_path="$1" 181 | local -r version="$2" 182 | local -r download_url="$3" 183 | local -r no_sudo="$4" 184 | 185 | echo "Installing $GRUNTWORK_INSTALLER_SCRIPT_NAME version $version to $install_path" 186 | download_and_install "$download_url" "$install_path" "$no_sudo" 187 | } 188 | 189 | function assert_not_empty { 190 | local -r arg_name="$1" 191 | local -r arg_value="$2" 192 | 193 | if [[ -z "$arg_value" ]]; then 194 | echo "ERROR: The value for '$arg_name' cannot be empty" 195 | print_usage 196 | exit 1 197 | fi 198 | } 199 | 200 | function create_user_data_folder { 201 | local -r user_data_folder="$1" 202 | local -r user_data_folder_owner="$2" 203 | local -r user_data_folder_readme="$user_data_folder/README.txt" 204 | local -r no_sudo="$3" 205 | 206 | echo "Creating $user_data_folder as a place to store scripts intended to be run in the User Data of an EC2 instance during boot" 207 | maybe_sudo "$no_sudo" mkdir -p "$user_data_folder" 208 | 209 | maybe_sudo "$no_sudo" tee "$user_data_folder_readme" > /dev/null <&2 echo -e "${timestamp} [${level}] [$scriptname] ${message[*]}" 63 | } 64 | 65 | function log_info { 66 | local -ra message=("$@") 67 | log "INFO" "${message[@]}" 68 | } 69 | 70 | function log_warn { 71 | local -ra message=("$@") 72 | log "WARN" "${message[@]}" 73 | } 74 | 75 | function log_error { 76 | local -ra message=("$@") 77 | log "ERROR" "${message[@]}" 78 | } 79 | 80 | function maybe_sudo { 81 | local -r no_sudo="$1" 82 | shift 83 | 84 | if [[ "$no_sudo" == "true" ]]; then 85 | "$@" 86 | else 87 | sudo "$@" 88 | fi 89 | } 90 | 91 | # Assert that a given binary is installed on this box 92 | function assert_is_installed { 93 | local -r name="$1" 94 | 95 | if [[ ! "$(command -v "${name}")" ]]; then 96 | log_error "The binary '$name' is required by this script but is not installed or in the system's PATH." 97 | exit 1 98 | fi 99 | } 100 | 101 | function assert_not_empty { 102 | local -r arg_name="$1" 103 | local -r arg_value="$2" 104 | 105 | if [[ -z "$arg_value" ]]; then 106 | log_error "The value for '$arg_name' cannot be empty" 107 | print_usage 108 | exit 1 109 | fi 110 | } 111 | 112 | function assert_env_var_not_empty { 113 | local -r var_name="$1" 114 | local -r var_value="${!var_name}" 115 | 116 | if [[ -z "$var_value" ]]; then 117 | log_error "Required environment variable $var_name not set." 118 | exit 1 119 | fi 120 | } 121 | 122 | # Download the files of the given Script Module using fetch (https://github.com/gruntwork-io/fetch) 123 | function fetch_script_module { 124 | local -r module_name="$1" 125 | local -r tag="$2" 126 | local -r branch="$3" 127 | local -r ref="$4" 128 | local -r download_dir="$5" 129 | local -r repo="$6" 130 | 131 | # We want to make sure that all folders down to $download_path/$module_name exists, but that $download_path/$module_name itself is empty. 132 | mkdir -p "$download_dir/$module_name/" 133 | rm -Rf "${download_dir:?}/${module_name:?}/" 134 | 135 | # Note that fetch can safely handle blank arguments for --tag or --branch 136 | # If both --tag and --branch are specified, --branch will be used 137 | log_info "Downloading module $module_name from $repo" 138 | fetch --ref="$ref" --repo="$repo" --tag="$tag" --branch="$branch" --source-path="/modules/$module_name" "$download_dir/$module_name" 139 | } 140 | 141 | # Download a binary asset from a GitHub release using fetch (https://github.com/gruntwork-io/fetch) 142 | function fetch_binary { 143 | local -r binary_name="$1" 144 | local -r tag="$2" 145 | local -r download_dir="$3" 146 | local -r repo="$4" 147 | local -r sha256_checksum="$5" 148 | local -r sha512_checksum="$6" 149 | local -r binary_install_dir="$7" 150 | local -r no_sudo="$8" 151 | 152 | local binary_name_full="" 153 | binary_name_full=$(determine_binary_name "$binary_name") 154 | 155 | local -r full_download_path="$download_dir/$binary_name_full" 156 | local -r full_dest_path="$binary_install_dir/$binary_name" 157 | 158 | # We want to make sure that all folders down to $download_path exist, but that $download_path/$binary_name_full does not 159 | mkdir -p "$download_dir" 160 | rm -f "$download_dir/$binary_name_full" 161 | 162 | if [[ -n "$sha256_checksum" ]]; then 163 | fetch --repo="$repo" --tag="$tag" --release-asset="$binary_name_full" "$download_dir" --release-asset-checksum-algo "sha256" --release-asset-checksum "$sha256_checksum" 164 | elif [[ -n "$sha512_checksum" ]]; then 165 | fetch --repo="$repo" --tag="$tag" --release-asset="$binary_name_full" "$download_dir" --release-asset-checksum-algo "sha512" --release-asset-checksum "$sha512_checksum" 166 | else 167 | fetch --repo="$repo" --tag="$tag" --release-asset="$binary_name_full" "$download_dir" 168 | fi 169 | 170 | log_info "Moving $full_download_path to $full_dest_path and setting execute permissions" 171 | maybe_sudo "$no_sudo" mv "$full_download_path" "$full_dest_path" 172 | maybe_sudo "$no_sudo" chmod u+x "$full_dest_path" 173 | } 174 | 175 | # Validate that at least one file was downloaded from the module; otherwise throw an error. 176 | function validate_module { 177 | local -r module_name="$1" 178 | local -r download_dir="$2" 179 | local -r tag="$3" 180 | local -r branch="$4" 181 | local -r repo="$5" 182 | 183 | if [[ ! -e "$download_dir/$module_name" ]]; then 184 | log_error "No files were downloaded. Are you sure \"$module_name\" is a valid Script Module in $repo (tag = $tag, branch = $branch)?" 185 | exit 1 186 | fi 187 | } 188 | 189 | # http://stackoverflow.com/a/2264537/483528 190 | function to_lower_case { 191 | tr '[:upper:]' '[:lower:]' 192 | } 193 | 194 | function get_os_name { 195 | uname | to_lower_case 196 | } 197 | 198 | function get_os_arch { 199 | uname -m 200 | } 201 | 202 | function string_contains { 203 | local -r str="$1" 204 | local -r contains="$2" 205 | 206 | [[ "$str" == *"$contains"* ]] 207 | } 208 | 209 | function get_os_arch_gox_format { 210 | local -r arch=$(get_os_arch) 211 | 212 | if string_contains "$arch" "arm64"; then 213 | echo "arm64" 214 | elif string_contains "$arch" "aarch64"; then 215 | echo "arm64" 216 | elif string_contains "$arch" "64"; then 217 | echo "amd64" 218 | elif string_contains "$arch" "386"; then 219 | echo "386" 220 | elif string_contains "$arch" "arm"; then 221 | echo "arm" 222 | fi 223 | } 224 | 225 | # We release binaries with the name following the format __ (e.g. foo_linux_amd64). Given the NAME of 226 | # a binary, this function adds the proper OS and ARCH to it for the current OS. 227 | function determine_binary_name { 228 | local -r binary_name="$1" 229 | local -r os_name=$(get_os_name) 230 | local -r os_arch=$(get_os_arch_gox_format) 231 | echo "${binary_name}_${os_name}_${os_arch}" 232 | } 233 | 234 | # Check if the repo is anonymously accessible. This indicates that the repo is public, and will skip checks for the 235 | # token. 236 | function repo_is_public { 237 | local -r repo_url="$1" 238 | 239 | curl --silent --fail "$repo_url" > /dev/null 240 | } 241 | 242 | function run_module { 243 | local -r module_name="$1" 244 | local -r download_dir="$2" 245 | local -r tag="$3" 246 | local -r branch="$4" 247 | shift 4 248 | local -ra module_params=("$@") 249 | local -r install_script_path="${download_dir}/${module_name}/${MODULE_INSTALL_FILE_NAME}" 250 | 251 | log_info "Setting GRUNTWORK_INSTALL_TAG to $tag" 252 | export GRUNTWORK_INSTALL_TAG="$tag" 253 | 254 | log_info "Setting GRUNTWORK_INSTALL_BRANCH to $branch" 255 | export GRUNTWORK_INSTALL_BRANCH="$branch" 256 | 257 | log_info "Executing $install_script_path ${module_params[*]}" 258 | chmod u+x "$install_script_path" 259 | "$install_script_path" "${module_params[@]}" 260 | } 261 | 262 | function install_script_module { 263 | local tag="" 264 | local branch="" 265 | local ref="" 266 | local module_name="" 267 | local binary_name="" 268 | local binary_sha256_checksum="" 269 | local binary_sha512_checksum="" 270 | local binary_install_dir="$DEFAULT_BIN_DIR" 271 | local repo="" 272 | local download_dir="$DEFAULT_MODULES_DOWNLOAD_DIR" 273 | local module_params=() 274 | local no_sudo="false" 275 | 276 | while [[ $# -gt 0 ]]; do 277 | local key="$1" 278 | 279 | case "$key" in 280 | --tag) 281 | tag="$2" 282 | shift 283 | ;; 284 | --branch) 285 | branch="$2" 286 | shift 287 | ;; 288 | --ref) 289 | ref="$2" 290 | shift 291 | ;; 292 | --module-name) 293 | module_name="$2" 294 | shift 295 | ;; 296 | --binary-name) 297 | binary_name="$2" 298 | shift 299 | ;; 300 | --binary-sha256-checksum) 301 | binary_sha256_checksum="$2" 302 | shift 303 | ;; 304 | --binary-sha512-checksum) 305 | binary_sha512_checksum="$2" 306 | shift 307 | ;; 308 | --repo) 309 | repo="$2" 310 | shift 311 | ;; 312 | --module-param) 313 | if [[ $2 == *"="* ]]; then 314 | local module_param_key="${2%%=*}" 315 | local module_param_val="${2#*=}" 316 | module_params+=( "--$module_param_key" "$module_param_val" ) 317 | else 318 | module_params+=( "--$2" ) 319 | fi 320 | shift 321 | ;; 322 | --download-dir) 323 | download_dir="$2" 324 | shift 325 | ;; 326 | --binary-install-dir) 327 | binary_install_dir="$2" 328 | shift 329 | ;; 330 | --no-sudo) 331 | no_sudo="$2" 332 | shift 333 | ;; 334 | --help) 335 | print_usage 336 | exit 337 | ;; 338 | *) 339 | echo "ERROR: Unrecognized option: $key" 340 | print_usage 341 | exit 1 342 | ;; 343 | esac 344 | 345 | shift 346 | done 347 | 348 | assert_not_empty "--repo" "$repo" 349 | assert_is_installed fetch 350 | 351 | if ! repo_is_public "$repo"; then 352 | log_info "Repository is not public. GITHUB_OAUTH_TOKEN environment variable is required." 353 | assert_env_var_not_empty "GITHUB_OAUTH_TOKEN" 354 | fi 355 | 356 | if [[ ( -z "$module_name" && -z "$binary_name" ) || ( -n "$module_name" && -n "$binary_name" ) ]]; then 357 | log_error "You must specify exactly one of --module-name or --binary-name." 358 | exit 1 359 | fi 360 | 361 | if [[ -n "$binary_name" && -z "$tag" ]]; then 362 | log_error "--binary-name can only be used if you specify a release via --tag." 363 | exit 1 364 | fi 365 | 366 | if [[ -n "$binary_sha256_checksum" && -n "$binary_sha512_checksum" ]]; then 367 | log_error "You must specify at most one of --binary-sha256-checksum and --binary-sha512-checksum" 368 | exit 1 369 | fi 370 | 371 | if [[ -n "$module_name" ]]; then 372 | log_info "Installing from $module_name..." 373 | fetch_script_module "$module_name" "$tag" "$branch" "$ref" "$download_dir" "$repo" 374 | validate_module "$module_name" "$download_dir" "$tag" "$branch" "$repo" 375 | run_module "$module_name" "$download_dir" "$tag" "$branch" "${module_params[@]}" 376 | else 377 | log_info "Installing $binary_name..." 378 | fetch_binary "$binary_name" "$tag" "$download_dir" "$repo" "$binary_sha256_checksum" "$binary_sha512_checksum" "$binary_install_dir" "$no_sudo" 379 | fi 380 | 381 | log_info "Success!" 382 | } 383 | 384 | install_script_module "$@" 385 | -------------------------------------------------------------------------------- /modules/README.md: -------------------------------------------------------------------------------- 1 | # Freely Available Gruntwork Script Modules 2 | 3 | This folder contains Gruntwork Script Modules that are commonly used and made freely available. 4 | 5 | See [Gruntwork Modules](packer-file-copy/README.md) to learn more about what a Gruntwork Script Module is. -------------------------------------------------------------------------------- /modules/args-test/README.md: -------------------------------------------------------------------------------- 1 | # Dummy module 2 | 3 | This module is used solely for running automated tests against the Gruntwork Installer. -------------------------------------------------------------------------------- /modules/args-test/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # A dirt simple install script that expects a single argument, --test-args, and validates that it equals '1 2 3 *' 3 | # in order to test that the --module-param args in gruntwork-install works correctly. 4 | 5 | set -e 6 | 7 | if [[ "${#@}" -ne "2" ]]; then 8 | echo "ERROR: Expected exactly two arguments to install.sh but received ${#@}" 9 | exit 1 10 | fi 11 | 12 | if [[ "$1" != "--test-args" ]]; then 13 | echo "ERROR: Expected first argument to be '--test-args' but received '$1'" 14 | exit 1 15 | fi 16 | 17 | if [[ "$2" != '1 2 3 *' ]]; then 18 | echo "ERROR: Expected second argument to be '1 2 3 *' but received '$2'" 19 | exit 1 20 | fi 21 | 22 | echo "ok" 23 | -------------------------------------------------------------------------------- /modules/dummy-module/README.md: -------------------------------------------------------------------------------- 1 | # Dummy module 2 | 3 | This module is used solely for running automated tests against the Gruntwork Installer. -------------------------------------------------------------------------------- /modules/dummy-module/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # A dirt simple install script that expects a single argument, --file-to-cat, that it runs cat on. This is used to 3 | # test that the --module-param args in gruntwork-install work correctly. 4 | 5 | set -e 6 | 7 | function assert_not_empty { 8 | local -r arg_name="$1" 9 | local -r arg_value="$2" 10 | 11 | if [[ -z "$arg_value" ]]; then 12 | echo "ERROR: The value for '$arg_name' cannot be empty" 13 | exit 1 14 | fi 15 | } 16 | 17 | function install { 18 | local file_to_cat 19 | 20 | while [[ $# > 0 ]]; do 21 | local key="$1" 22 | 23 | case "$key" in 24 | --file-to-cat) 25 | file_to_cat="$2" 26 | shift 27 | ;; 28 | *) 29 | echo "Unrecognized argument: $key" 30 | exit 1 31 | ;; 32 | esac 33 | 34 | shift 35 | done 36 | 37 | assert_not_empty "--file-to-cat" "$file_to_cat" 38 | cat "$file_to_cat" 39 | } 40 | 41 | install "$@" -------------------------------------------------------------------------------- /modules/mixed-args-module/README.md: -------------------------------------------------------------------------------- 1 | # Mixed args module 2 | 3 | This module is used solely for running automated tests against the Gruntwork Installer. -------------------------------------------------------------------------------- /modules/mixed-args-module/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # A dirt simple install script that expects one madatory argument, --message-to-echo, that it echoes. The echo message can be overridden with 3 | # a flag, --echo-override. This is used to test that the --module-param args in gruntwork-install can handle flags without values. 4 | 5 | set -e 6 | 7 | function assert_not_empty { 8 | local -r arg_name="$1" 9 | local -r arg_value="$2" 10 | 11 | if [[ -z "$arg_value" ]]; then 12 | echo "ERROR: The value for '$arg_name' cannot be empty" 13 | exit 1 14 | fi 15 | } 16 | 17 | function install { 18 | local echomessage 19 | 20 | while [[ $# > 0 ]]; do 21 | local key="$1" 22 | 23 | case "$key" in 24 | --message-to-echo) 25 | echomessage="$2" 26 | shift 27 | ;; 28 | --echo-override) 29 | echomessage="Override" 30 | ;; 31 | *) 32 | echo "Unrecognized argument: $key" 33 | exit 1 34 | ;; 35 | esac 36 | 37 | shift 38 | done 39 | 40 | assert_not_empty "--message-to-echo" "$echomessage" 41 | echo "$echomessage" 42 | } 43 | 44 | install "$@" -------------------------------------------------------------------------------- /modules/packer-file-copy/README.md: -------------------------------------------------------------------------------- 1 | # Packer File Copy 2 | 3 | This module is intended to be used in a [Packer template](https://www.packer.io/) to move all files that template 4 | copies into `/tmp/packer-files/XXX/YYY` into `/XXX/YYY`. For example, if you used the `file` provisioner to upload a 5 | file to `/tmp/packer-files/foo/bar`, it will be moved to `/foo/bar`. 6 | 7 | Why not just upload these files directly with Packer? Because: 8 | 9 | 1. If the destination folder doesn't exist, the `file` provisioner won't create it. Instead, you just get an error. 10 | 1. The `file` provisioner may not have permissions to write to certain folders (e.g. `/opt/my-app`) and it can't use 11 | `sudo`. 12 | 13 | As a result, using the `file` provisioner is often a multi-step process where you first copy the files to a temporary 14 | folder, then run scripts to create the real destination folder, move your files there, and update permissions. This 15 | packer-file-copy module automates all these steps. 16 | 17 | For an example of this module in action, see [examples/packer-file-copy](../../examples/packer-file-copy/). 18 | 19 | IMPORTANT: The packer file provisioner should look like the following: 20 | 21 | ```json 22 | { 23 | "type": "file", 24 | "source": "{{template_dir}}/files", 25 | "destination": "/tmp/packer-files" 26 | } 27 | ``` 28 | 29 | Note how both the source and destination have no trailing slash. Adding a trailing slash will mean Packer will upload 30 | files differently than intended. -------------------------------------------------------------------------------- /modules/packer-file-copy/install-scripts/copy-packer-files.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # Copy all files located in the $DEFAULT_PACKER_FILES_PATH to their corresponding location on the file systemm. For 4 | # example, copy /tmp/packer-files/foo/bar.txt to /foo/bar.txt. 5 | # 6 | # Detailed Example: 7 | # $ DEFAULT_PACKER_FILES_PATH="/tmp/packer-files" 8 | # $ tree /tmp/packer-files: 9 | # . 10 | # ├── etc 11 | # │   └── foo.config 12 | # └── opt 13 | # └── bar.sh 14 | # 15 | # $ ./copy-packer-files.sh 16 | # $ ls /etc/ 17 | # foo.config 18 | # $ ls /opt/ 19 | # bar.sh 20 | 21 | set -e 22 | 23 | # Declare an array of paths to which Packer uploaded files 24 | readonly DEFAULT_PACKER_FILES_PATH="/tmp/packer-files" 25 | 26 | function copy_packer_files { 27 | local -ra file_upload_paths=("$DEFAULT_PACKER_FILES_PATH") 28 | local -r script_path="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" 29 | 30 | # For each file upload path, if there are any files there, relocate the files to their proper directories 31 | for upload_path in "${file_upload_paths[@]}"; do 32 | if [[ -e "$upload_path" ]]; then 33 | # Get the length of the string that $upload_path resolves to 34 | local -r upload_path_length=${#upload_path} 35 | local file="" 36 | 37 | for file in $(find "$upload_path" -type f); do 38 | local -r absolute_filename="${file:$upload_path_length}"; 39 | echo "The packer-file-copy module is copying $file to $absolute_filename" 40 | sudo mkdir -p $(dirname "$absolute_filename"); 41 | sudo mv "$file" "$absolute_filename"; 42 | done 43 | fi 44 | done 45 | 46 | # Write README in case a future user is confused about /tmp/packer-files 47 | if [[ -e "$DEFAULT_PACKER_FILES_PATH" ]]; then 48 | cp "$script_path/../README.md" "$DEFAULT_PACKER_FILES_PATH" 49 | fi 50 | } 51 | 52 | copy_packer_files -------------------------------------------------------------------------------- /modules/packer-file-copy/install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # Script used by gruntwork-install to install the packer-file-copy module. 4 | # 5 | 6 | set -e 7 | 8 | # Locate the directory in which this script is located 9 | readonly script_path="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" 10 | 11 | # Execute the install script 12 | chmod u+x "${script_path}/install-scripts/copy-packer-files.sh" 13 | eval "${script_path}/install-scripts/copy-packer-files.sh $@" -------------------------------------------------------------------------------- /test/amazonlinux/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM amazonlinux 2 | MAINTAINER Gruntwork 3 | 4 | RUN yum install -y curl sudo 5 | 6 | COPY . /test 7 | 8 | CMD ["echo", "This container is used for testing. Consider running one of the test scripts under the /test folder."] 9 | -------------------------------------------------------------------------------- /test/amazonlinux/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | installer: 3 | image: gruntwork/gruntwork-installer-amazonlinux 4 | volumes: 5 | # Mount the scripts in the root directory under /src 6 | - ../../:/src 7 | # Mount the test code in this directory under /test 8 | - ../:/test 9 | environment: 10 | # Forward the GITHUB_OAUTH_TOKEN as an environment variable from the user's environment 11 | - GITHUB_OAUTH_TOKEN 12 | -------------------------------------------------------------------------------- /test/centos/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM centos:7 2 | MAINTAINER Gruntwork 3 | 4 | RUN yum install -y curl sudo 5 | 6 | COPY . /test 7 | 8 | CMD ["echo", "This container is used for testing. Consider running one of the test scripts under the /test folder."] 9 | -------------------------------------------------------------------------------- /test/centos/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | installer: 3 | image: gruntwork/gruntwork-installer-centos 4 | volumes: 5 | # Mount the scripts in the root directory under /src 6 | - ../../:/src 7 | # Mount the test code in this directory under /test 8 | - ../:/test 9 | environment: 10 | # Forward the GITHUB_OAUTH_TOKEN as an environment variable from the user's environment 11 | - GITHUB_OAUTH_TOKEN 12 | -------------------------------------------------------------------------------- /test/integration-test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Some basic automated tests for gruntwork-installer 4 | 5 | set -e 6 | 7 | readonly LOCAL_INSTALL_URL="file:///src/gruntwork-install" 8 | readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" 9 | 10 | echo "Using local copy of bootstrap installer to install local copy of gruntwork-install" 11 | ./src/bootstrap-gruntwork-installer.sh --download-url "$LOCAL_INSTALL_URL" --version "ignored-for-local-install" 12 | 13 | echo "Using gruntwork-install to install a module from the terraform-aws-ecs repo using branch" 14 | gruntwork-install --module-name "ecs-scripts" --repo "https://github.com/gruntwork-io/terraform-aws-ecs" --branch "v0.0.1" 15 | 16 | echo "Using gruntwork-install to install a module from the terraform-aws-ecs repo with --download-dir option" 17 | gruntwork-install --module-name "ecs-scripts" --repo "https://github.com/gruntwork-io/terraform-aws-ecs" --branch "v0.0.1" --download-dir ~/tmp 18 | 19 | echo "Checking that the ecs-scripts installed correctly" 20 | configure-ecs-instance --help 21 | 22 | echo "Using gruntwork-install to install a module from the gruntwork-install repo and passing args to it via --module-param" 23 | gruntwork-install --module-name "dummy-module" --repo "https://github.com/gruntwork-io/gruntwork-installer" --tag "v0.0.25" --module-param "file-to-cat=$SCRIPT_DIR/integration-test.sh" 24 | 25 | echo "Using gruntwork-install to install a module from the gruntwork-install repo and passing mixed args to it via --module-param" 26 | gruntwork-install --module-name "mixed-args-module" --repo "https://github.com/gruntwork-io/gruntwork-installer" --ref "fix/mixed-args" --module-param "message-to-echo=Hello" --module-param "echo-override" 27 | 28 | echo "Using gruntwork-install to install a module from the gruntwork-install repo with branch as ref" 29 | gruntwork-install --module-name "dummy-module" --repo "https://github.com/gruntwork-io/gruntwork-installer" --ref "for-testing-dont-delete" --module-param "file-to-cat=$SCRIPT_DIR/integration-test.sh" 30 | 31 | echo "Using gruntwork-install to install a module from the gruntwork-install repo with tag as ref" 32 | gruntwork-install --module-name "dummy-module" --repo "https://github.com/gruntwork-io/gruntwork-installer" --ref "v0.0.25" --module-param "file-to-cat=$SCRIPT_DIR/integration-test.sh" 33 | 34 | echo "Using gruntwork-install to install a test module from the gruntwork-install repo and test that it's args are maintained via --module-param" 35 | gruntwork-install --module-name "args-test" --repo "https://github.com/gruntwork-io/gruntwork-installer" --tag "v0.0.25" --module-param 'test-args=1 2 3 *' 36 | 37 | echo "Using gruntwork-install to install a binary from the gruntkms repo" 38 | gruntwork-install --binary-name "gruntkms" --repo "https://github.com/gruntwork-io/gruntkms" --tag "v0.0.1" 39 | 40 | echo "Checking that gruntkms installed correctly" 41 | gruntkms --help 42 | 43 | echo "Unsetting GITHUB_OAUTH_TOKEN to test installing from public repo (terragrunt)" 44 | unset GITHUB_OAUTH_TOKEN 45 | 46 | echo "Verifying private repo access is denied" 47 | if gruntwork-install --binary-name "gruntkms" --repo "https://github.com/gruntwork-io/gruntkms" --tag "v0.0.1" ; then 48 | echo "ERROR: was able to access private repo" 49 | exit 1 50 | fi 51 | 52 | echo "Verifying public repo access is allowed" 53 | gruntwork-install --repo 'https://github.com/gruntwork-io/terragrunt' --binary-name terragrunt --tag '~>v0.21.0' 54 | 55 | echo "Checking that terragrunt installed correctly" 56 | terragrunt --help 57 | -------------------------------------------------------------------------------- /test/no-sudo-test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Some basic automated tests for gruntwork-installer 4 | 5 | set -e 6 | 7 | readonly LOCAL_INSTALL_URL="file:///src/gruntwork-install" 8 | 9 | echo "Using local copy of bootstrap installer to install local copy of gruntwork-install" 10 | ./src/bootstrap-gruntwork-installer.sh --download-url "$LOCAL_INSTALL_URL" --version "ignored-for-local-install" --no-sudo "true" 11 | 12 | echo "Using gruntwork-install to install a binary from the gruntkms repo into a different folder without using sudo" 13 | gruntwork-install \ 14 | --binary-name "gruntkms" \ 15 | --repo "https://github.com/gruntwork-io/gruntkms" \ 16 | --tag "v0.0.1" \ 17 | --binary-install-dir "$HOME" \ 18 | --no-sudo "true" 19 | 20 | echo "Checking that gruntkms installed correctly into home dir" 21 | "$HOME/gruntkms" --help 22 | -------------------------------------------------------------------------------- /test/no_sudo/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | MAINTAINER Gruntwork 3 | 4 | RUN apt-get update 5 | RUN apt-get install -y curl 6 | 7 | COPY . /test 8 | 9 | CMD ["echo", "This container is used for testing. Consider running one of the test scripts under the /test folder."] 10 | -------------------------------------------------------------------------------- /test/no_sudo/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | installer: 3 | image: gruntwork/gruntwork-installer-no-sudo-ubuntu 4 | volumes: 5 | # Mount the scripts in the root directory under /src 6 | - ../../:/src 7 | # Mount the test code in this directory under /test 8 | - ../:/test 9 | environment: 10 | # Forward the GITHUB_OAUTH_TOKEN as an environment variable from the user's environment 11 | - GITHUB_OAUTH_TOKEN 12 | -------------------------------------------------------------------------------- /test/ubuntu/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | MAINTAINER Gruntwork 3 | 4 | RUN apt-get update 5 | RUN apt-get install -y curl sudo 6 | 7 | COPY . /test 8 | 9 | CMD ["echo", "This container is used for testing. Consider running one of the test scripts under the /test folder."] 10 | -------------------------------------------------------------------------------- /test/ubuntu/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | installer: 3 | image: gruntwork/gruntwork-installer-ubuntu 4 | volumes: 5 | # Mount the scripts in the root directory under /src 6 | - ../../:/src 7 | # Mount the test code in this directory under /test 8 | - ../:/test 9 | environment: 10 | # Forward the GITHUB_OAUTH_TOKEN as an environment variable from the user's environment 11 | - GITHUB_OAUTH_TOKEN 12 | -------------------------------------------------------------------------------- /test/ubuntu18/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:18.04 2 | MAINTAINER Gruntwork 3 | 4 | RUN apt-get update 5 | RUN apt-get install -y curl sudo 6 | 7 | COPY . /test 8 | 9 | CMD ["echo", "This container is used for testing. Consider running one of the test scripts under the /test folder."] 10 | -------------------------------------------------------------------------------- /test/ubuntu18/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | installer: 3 | image: gruntwork/gruntwork-installer-ubuntu18 4 | volumes: 5 | # Mount the scripts in the root directory under /src 6 | - ../../:/src 7 | # Mount the test code in this directory under /test 8 | - ../:/test 9 | environment: 10 | # Forward the GITHUB_OAUTH_TOKEN as an environment variable from the user's environment 11 | - GITHUB_OAUTH_TOKEN 12 | --------------------------------------------------------------------------------