├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── Dockerfile.ansible ├── LICENSE ├── README.md ├── aws.env.encrypted ├── base └── Dockerfile ├── codeship-services.yml ├── codeship-steps.yml ├── deployment ├── Dockerfile ├── scripts │ ├── codeship_aws │ ├── codeship_aws_codedeploy_deploy │ ├── codeship_aws_codedeploy_deploy_validation │ ├── codeship_aws_eb_deploy │ ├── codeship_aws_eb_deploy_validation │ ├── eb_deployment_check │ ├── elastic_beanstalk_environment_update.sh │ └── utils └── test │ ├── code_deploy │ ├── app │ │ ├── Codeship.png │ │ ├── appspec.yml │ │ ├── index.html │ │ └── scripts │ │ │ ├── install_dependencies │ │ │ ├── start_server │ │ │ └── stop_server │ └── integration-test │ ├── elastic_beanstalk │ ├── Dockerfile │ ├── integration-test │ └── options.txt │ ├── test_tools_available.sh │ ├── upload_to_s3 │ └── utils ├── dockercfg-generator ├── Dockerfile ├── Dockerfile.test ├── README.md ├── aws_docker_creds.sh └── codeship-steps.yml ├── dockercfg.encrypted ├── site.yml └── tasks ├── install.yml └── main.yml /.gitignore: -------------------------------------------------------------------------------- 1 | aws.env 2 | *.aes 3 | dockercfg 4 | tmp 5 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of experience, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting one of the project maintainers listed below. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Project Maintainers 69 | 70 | * Codeship Engineering Team 71 | 72 | ## Attribution 73 | 74 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 75 | available at [http://contributor-covenant.org/version/1/4][version] 76 | 77 | [homepage]: http://contributor-covenant.org 78 | [version]: http://contributor-covenant.org/version/1/4/ 79 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | *Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.* 4 | 5 | In the spirit of [free software][free-sw], **everyone** is encouraged to help improve this project. Here are some ways *you* can contribute: 6 | 7 | * by reporting errors on our documentation pages 8 | * by suggesting new articles or enhancements to existing ones 9 | * by writing documentation (**no patch is too small** : fix typos, add comments, clean up inconsistent whitespace) 10 | * by closing [issues][issues] 11 | * by reviewing patches and pull requests 12 | 13 | ## Submitting an Issue 14 | We use the [GitHub issue tracker][issues] to track bugs and features. Before submitting a bug report or feature request, check to make sure it hasn't already been submitted. When submitting a bug report, please include any details that may be necessary to reproduce the bug. 15 | 16 | ## Submitting a Pull Request 17 | 1. [Fork the repository.][fork] 18 | 2. [Create a topic branch.][branch] 19 | 3. Implement your feature or bug fix. 20 | 4. Add, commit, and push your changes. 21 | 5. [Submit a pull request.][pr] 22 | 23 | ## When not to submit an issue 24 | 25 | Please don't create new issues on this repository when you experience issues with your builds on https://codeship.com, want to request new features for the CI system or have general questions on how to use https://codeship.com for your projects. 26 | 27 | In all of the above cases, reach out to our support team at support@codeship.com or on Twitter at [@CodeshipSupport](https://twitter.com/CodeshipSupport). 28 | 29 | [free-sw]: http://www.fsf.org/licensing/essays/free-sw.html 30 | [issues]: https://github.com/codeship-library/aws/issues 31 | [gist]: https://gist.github.com/ 32 | [fork]: http://help.github.com/fork-a-repo/ 33 | [branch]: https://github.com/blog/1377-create-and-delete-branches 34 | [pr]: http://help.github.com/send-pull-requests/ 35 | -------------------------------------------------------------------------------- /Dockerfile.ansible: -------------------------------------------------------------------------------- 1 | FROM ubuntu:24.04 2 | 3 | ENV LANG=C.UTF-8 4 | ENV PATH="/root/bin:/root/.local/bin:${PATH}" 5 | 6 | RUN \ 7 | apt-get update && \ 8 | apt-get install -y --no-install-recommends pipx python3-setuptools apt-transport-https ca-certificates wget rsync unzip jq zip curl && \ 9 | pipx install --include-deps ansible==11.4.0 awscli 10 | 11 | RUN mkdir /root/bin 12 | 13 | COPY tasks /aws/tasks 14 | COPY site.yml site.yml 15 | COPY deployment /aws/deployment 16 | 17 | RUN ansible-playbook -i localhost -c local site.yml 18 | 19 | ENV CODESHIP_VIRTUALENV="/root/.codeship-venv" 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS related Docker images for Codeship Jet 2 | 3 | ## AWS Deployment 4 | 5 | A Docker image with scripts to deploy to various AWS services, including S3, CodeDeploy and ElasticBeanstalk. 6 | 7 | ## AWS dockercfg Generator 8 | 9 | This container allows you to generate a temporary dockercfg using your AWS credentials and writes it to a specified filename. Typical usage of this image would be to run it with a volume attached, and write the dockercfg to that volume. 10 | 11 | See the [dockercfg-generator ReadMe](dockercfg-generator/README.md) for more details and usage instructions. 12 | 13 | ## Contributing 14 | 15 | We are happy to hear your feedback. Please read our [contributing guidelines](CONTRIBUTING.md) and the [code of conduct](CODE_OF_CONDUCT.md) before you submit a pull request or open a ticket. 16 | 17 | ## License 18 | 19 | see [LICENSE](LICENSE) 20 | -------------------------------------------------------------------------------- /aws.env.encrypted: -------------------------------------------------------------------------------- 1 | codeship:v2 2 | iLfF3czYVlzAx3rUWoPjUblDYMd5jcC/ghHaFBX4ABJ7dCRv2qIWBzd/Bm7GTb+me1HwvxgD8hTiu6/1i0sM2xaCgB1ojQHXxblATsEJhpoYArLWmhkGVjXxOgDNly6s0fCWGIuy3W20f1TjlqAq9Mud2CtdZ5a16kN+HYdIwwer8VyW/swQo9fUiyeB5EKQvpcZax+x1wTCefOCIVKXZK2kr1DpgTo3mC6meaA8qHt177XqVwO9uPJXLeSh6wSLJfe18TKpsEmMOVS9UYnQS0bqgewnm9d2tyc3naCKmQyXVIbCgMAv1yXs3wfuzG+lCCnFlV/ip85Jx2bE7KuEXg== -------------------------------------------------------------------------------- /base/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.13-alpine 2 | LABEL maintainer='Codeship Inc., ' 3 | 4 | # which version of the AWS CLI to install. 5 | # https://pypi.python.org/pypi/awscli 6 | ARG AWS_CLI_VERSION="1.40.12" 7 | 8 | ENV PIP_DISABLE_PIP_VERSION_CHECK=true 9 | 10 | RUN \ 11 | pip install awscli==${AWS_CLI_VERSION} && \ 12 | mkdir -p "${HOME}/.aws" 13 | -------------------------------------------------------------------------------- /codeship-services.yml: -------------------------------------------------------------------------------- 1 | base: 2 | build: 3 | image: codeship/aws-base 4 | path: ./base 5 | dockerfile_path: Dockerfile 6 | deployment-ansible: 7 | build: 8 | dockerfile: Dockerfile.ansible 9 | encrypted_env_file: aws.env.encrypted 10 | volumes: 11 | - ./deployment:/deploy 12 | deployment: 13 | build: 14 | image: codeship/aws-deployment 15 | path: ./deployment 16 | dockerfile_path: Dockerfile 17 | encrypted_env_file: aws.env.encrypted 18 | volumes: 19 | - ./deployment:/deploy 20 | dockercfg-generator: 21 | build: 22 | image: codeship/aws-ecr-dockercfg-generator 23 | path: ./dockercfg-generator 24 | dockerfile_path: Dockerfile 25 | dockercfg-test: 26 | build: 27 | image: 870119404647.dkr.ecr.us-east-1.amazonaws.com/codeship-testing 28 | path: ./dockercfg-generator 29 | dockerfile_path: Dockerfile.test 30 | dockercfg-test-generator: 31 | image: codeship/aws-ecr-dockercfg-generator 32 | add_docker: true 33 | encrypted_env_file: aws.env.encrypted 34 | -------------------------------------------------------------------------------- /codeship-steps.yml: -------------------------------------------------------------------------------- 1 | - name: Build Base Image 2 | service: base 3 | command: true 4 | - name: Test Deployment Ansible 5 | type: serial 6 | service: deployment-ansible 7 | steps: 8 | - name: Check for tooling Ansible 9 | command: /deploy/test/test_tools_available.sh 10 | - name: Tests Ansible 11 | type: parallel 12 | steps: 13 | - name: Util Scripts Ansible 14 | command: /deploy/test/utils 15 | - name: S3, LS Ansible 16 | command: aws s3 ls s3://codeship-aws-deployment-integration-test 17 | - name: S3, CP Archive Ansible 18 | command: aws s3 cp /deploy/tmp/upload_to_s3.zip s3://codeship-aws-deployment-integration-test 19 | - name: S3, CP Ansible 20 | command: aws s3 cp /deploy/test/upload_to_s3 s3://codeship-aws-deployment-integration-test 21 | - name: Elastic Beanstalk Ansible 22 | command: /deploy/test/elastic_beanstalk/integration-test 23 | - name: Code Deploy Ansible 24 | command: /deploy/test/code_deploy/integration-test 25 | - name: Test 26 | type: parallel 27 | steps: 28 | - name: Test Deployment 29 | type: serial 30 | service: deployment 31 | steps: 32 | - name: Check for tooling 33 | command: /deploy/test/test_tools_available.sh 34 | - name: Check for kubectl tool 35 | command: kubectl version --client 36 | - name: Tests 37 | type: parallel 38 | steps: 39 | - name: S3, LS 40 | command: aws s3 ls s3://codeship-aws-deployment-integration-test 41 | - name: S3, CP Archive 42 | command: aws s3 cp /deploy/tmp/upload_to_s3.zip s3://codeship-aws-deployment-integration-test 43 | - name: S3, CP 44 | command: aws s3 cp /deploy/test/upload_to_s3 s3://codeship-aws-deployment-integration-test 45 | - name: Elastic Beanstalk 46 | command: /deploy/test/elastic_beanstalk/integration-test 47 | - name: Code Deploy 48 | command: /deploy/test/code_deploy/integration-test 49 | - name: Test dockercfg Generator 50 | service: dockercfg-test 51 | type: push 52 | image_name: 870119404647.dkr.ecr.us-east-1.amazonaws.com/codeship-testing 53 | registry: https://870119404647.dkr.ecr.us-east-1.amazonaws.com 54 | dockercfg_service: dockercfg-test-generator 55 | - name: Deploy (to Docker Hub) 56 | type: serial 57 | steps: 58 | - name: Push Deployment 59 | service: deployment 60 | tag: master 61 | type: push 62 | image_name: codeship/aws-deployment 63 | encrypted_dockercfg_path: dockercfg.encrypted 64 | - name: Push dockercfg Generator 65 | service: dockercfg-generator 66 | tag: master 67 | type: push 68 | image_name: codeship/aws-ecr-dockercfg-generator 69 | encrypted_dockercfg_path: dockercfg.encrypted 70 | -------------------------------------------------------------------------------- /deployment/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM codeship/aws-base:latest 2 | LABEL maintainer='Codeship Inc., ' 3 | 4 | RUN \ 5 | apk --no-cache add \ 6 | bash \ 7 | curl \ 8 | groff \ 9 | jq \ 10 | less \ 11 | zip \ 12 | python3 \ 13 | python3-dev \ 14 | py3-pip \ 15 | && pip install virtualenv 16 | 17 | # Install kubectl from AWS 18 | # https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html#linux_amd64_kubectl 19 | ARG KUBECTL_VERSION="1.33.0" 20 | ARG KUBECTL_BUILD_DATE="2025-05-01" 21 | 22 | RUN curl -L https://s3.us-west-2.amazonaws.com/amazon-eks/${KUBECTL_VERSION}/${KUBECTL_BUILD_DATE}/bin/linux/amd64/kubectl > /usr/local/bin/kubectl \ 23 | && chmod +x /usr/local/bin/kubectl 24 | 25 | # Install aws-iam-authenticator 26 | RUN curl -L https://s3.us-west-2.amazonaws.com/amazon-eks/${KUBECTL_VERSION}/${KUBECTL_BUILD_DATE}/bin/linux/amd64/aws-iam-authenticator > /usr/local/bin/aws-iam-authenticator \ 27 | && chmod +x /usr/local/bin/aws-iam-authenticator 28 | 29 | RUN virtualenv root/.codeship-venv 30 | 31 | ENV CODESHIP_VIRTUALENV="/root/.codeship-venv" 32 | 33 | COPY scripts/ /usr/bin/ 34 | -------------------------------------------------------------------------------- /deployment/scripts/codeship_aws: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) 6 | 7 | : ${AWS_ACCESS_KEY_ID:?'Set the AWS_ACCESS_KEY_ID environment variable'} 8 | : ${AWS_SECRET_ACCESS_KEY:?'Set the AWS_SECRET_ACCESS_KEY environment variable'} 9 | : ${AWS_DEFAULT_REGION:?'Set the AWS_DEFAULT_REGION environment variable'} 10 | 11 | SCRIPT_NAME=${1:?'You need to provide the action as the first parameter'} 12 | 13 | SCRIPT_PATH=$DIR/codeship_aws_$SCRIPT_NAME 14 | 15 | if [ -x $SCRIPT_PATH ]; then 16 | if [ ! -z "${CODESHIP_VIRTUALENV}" ]; then 17 | #deactivate old virtualenv if one was activated and store in variable 18 | if [ "" != "${VIRTUAL_ENV}" ]; then 19 | OLD_VIRTUAL_ENV=${VIRTUAL_ENV} 20 | if [ "$(type -t deactivate)" == "function" ]; then 21 | deactivate 22 | fi 23 | fi 24 | source "${CODESHIP_VIRTUALENV}/bin/activate" 25 | fi 26 | 27 | $SCRIPT_PATH "${@:2}" 28 | 29 | deactivate 30 | if [ ! -x "$(command -v aws)" ]; then 31 | echo "Linking aws command for global usage." 32 | ln -s "${CODESHIP_VIRTUALENV}/bin/aws" "${HOME}/bin/aws" 33 | fi 34 | 35 | if [ "" != "${OLD_VIRTUAL_ENV}" ]; then 36 | # shellcheck disable=SC1090 37 | source "${OLD_VIRTUAL_ENV}/bin/activate" 38 | fi 39 | else 40 | if [ ! -z "${CODESHIP_VIRTUALENV}" ]; then 41 | #deactivate old virtualenv if one was activated and store in variable 42 | if [ "" != "${VIRTUAL_ENV}" ]; then 43 | OLD_VIRTUAL_ENV=${VIRTUAL_ENV} 44 | if [ "$(type -t deactivate)" == "function" ]; then 45 | deactivate 46 | fi 47 | fi 48 | source "${CODESHIP_VIRTUALENV}/bin/activate" 49 | fi 50 | aws "$@" 51 | deactivate 52 | if [ ! -x "$(command -v aws)" ]; then 53 | echo "Linking aws command for global usage." 54 | ln -s "${CODESHIP_VIRTUALENV}/bin/aws" "${HOME}/bin/aws" 55 | fi 56 | 57 | if [ "" != "${OLD_VIRTUAL_ENV}" ]; then 58 | # shellcheck disable=SC1090 59 | source "${OLD_VIRTUAL_ENV}/bin/activate" 60 | fi 61 | fi 62 | -------------------------------------------------------------------------------- /deployment/scripts/codeship_aws_codedeploy_deploy: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | APPLICATION_FOLDER=${1:?'You need to provide the directory with your code as the second parameter'} 7 | APPLICATION_NAME=${2:?'You need to provide the CodeDeploy application name'} 8 | DEPLOYMENT_GROUP_NAME=${3:?'You need to provide the Deployment Group name'} 9 | S3_BUCKET=${4:?'You need to provide the S3 Bucket to upload the artefact to'} 10 | DEPLOYMENT_CONFIG_NAME=$5 11 | 12 | source utils 13 | S3_BUCKET_SUBFOLDER_PATH=$(s3_bucket_subfolder_path $S3_BUCKET) 14 | S3_BUCKET=$(s3_bucket_name $S3_BUCKET) 15 | 16 | echo "Starting CodeDeploy deployment to $APPLICATION_NAME:$DEPLOYMENT_GROUP_NAME" 17 | 18 | VERSION_NAME=${CI_COMMIT_ID}-${CI_BUILD_ID}-$(date +%Y-%m-%d.%H:%M:%S) 19 | FILE_NAME=$VERSION_NAME.zip 20 | 21 | if [[ -z $S3_BUCKET_SUBFOLDER_PATH ]]; then 22 | S3_KEY=$FILE_NAME 23 | S3_PATH="s3://$S3_BUCKET/$FILE_NAME" 24 | else 25 | S3_KEY="$S3_BUCKET_SUBFOLDER_PATH/$FILE_NAME" 26 | S3_PATH="s3://$S3_BUCKET/$S3_BUCKET_SUBFOLDER_PATH/$FILE_NAME" 27 | fi 28 | 29 | DESCRIPTION="${CI_COMMIT_MESSAGE:0:512}" 30 | 31 | aws deploy push --application-name "$APPLICATION_NAME" --description "$DESCRIPTION" --ignore-hidden-files --s3-location "$S3_PATH" --source "$APPLICATION_FOLDER" > /dev/null 32 | 33 | echo "Creating new Deployment: $VERSION_NAME" 34 | 35 | deployment=("--application-name" "$APPLICATION_NAME" "--deployment-group-name" "$DEPLOYMENT_GROUP_NAME" "--description" "$DESCRIPTION" "--s3-location" "bucket=$S3_BUCKET,key=$S3_KEY,bundleType=zip") 36 | 37 | if [ ! -z "$DEPLOYMENT_CONFIG_NAME" ]; then 38 | deployment+=("--deployment-config-name" "$DEPLOYMENT_CONFIG_NAME") 39 | fi 40 | 41 | echo "Deployment Command: ${deployment[@]}" 42 | 43 | deployment_id=$(aws deploy create-deployment "${deployment[@]}" | jq -r .deploymentId) 44 | 45 | echo "Validating deployment: $deployment_id" 46 | codeship_aws codedeploy_deploy_validation "$deployment_id" 47 | -------------------------------------------------------------------------------- /deployment/scripts/codeship_aws_codedeploy_deploy_validation: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | DEPLOYMENT_ID=${1:?'You need to provide the Deployment Id that should be validated'} 7 | 8 | function get_deployment(){ 9 | aws deploy get-deployment --deployment-id "$DEPLOYMENT_ID" 10 | } 11 | 12 | function environment_status(){ 13 | get_deployment | jq -r .deploymentInfo.status 14 | } 15 | 16 | function environment_output(){ 17 | get_deployment | jq -r .deploymentInfo 18 | } 19 | 20 | echo "Waiting for Deployment to finish" 21 | until [[ "$(environment_status)" =~ (Succeeded|Failed|Stopped) ]] ; do 22 | echo "Environment Output: $(environment_output)" 23 | sleep 10 24 | echo "." 25 | done 26 | 27 | echo "Deployment process finished, checking success of deployment" 28 | 29 | echo "Environment Output: $(environment_output)" 30 | 31 | if [ "$(environment_status)" == "Succeeded" ] 32 | then 33 | exit_status=0 34 | else 35 | exit_status=1 36 | fi 37 | 38 | echo "Deployment $(environment_status)" 39 | 40 | exit $exit_status 41 | -------------------------------------------------------------------------------- /deployment/scripts/codeship_aws_eb_deploy: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | APPLICATION_FOLDER=${1:?'You need to provide the directory with your code as the second parameter'} 7 | APPLICATION_NAME=${2:?'You need to provide the Elastic Beanstalk application name'} 8 | ENVIRONMENT_NAME=${3:?'You need to provide the Elastic Beanstalk environment name'} 9 | S3_BUCKET=${4:?'You need to provide the S3 Bucket to upload the artefact to'} 10 | 11 | source utils 12 | S3_BUCKET_SUBFOLDER_PATH=$(s3_bucket_subfolder_path $S3_BUCKET) 13 | S3_BUCKET=$(s3_bucket_name $S3_BUCKET) 14 | 15 | echo "Starting ElasticBeanstalk deployment to $APPLICATION_NAME:$ENVIRONMENT_NAME" 16 | 17 | echo "CHANGING Directory to $APPLICATION_FOLDER" 18 | cd "$APPLICATION_FOLDER" 19 | 20 | VERSION_NAME=${CI_COMMIT_ID}-${CI_BUILD_ID}-$(date +%Y-%m-%d.%H:%M:%S) 21 | FILE_NAME=$VERSION_NAME.zip 22 | DEPLOYMENT_FILE_NAME=/tmp/$FILE_NAME 23 | 24 | if [[ -z $S3_BUCKET_SUBFOLDER_PATH ]]; then 25 | S3_KEY=$FILE_NAME 26 | S3_PATH="s3://$S3_BUCKET/$FILE_NAME" 27 | else 28 | S3_KEY="$S3_BUCKET_SUBFOLDER_PATH/$FILE_NAME" 29 | S3_PATH="s3://$S3_BUCKET/$S3_BUCKET_SUBFOLDER_PATH/$FILE_NAME" 30 | fi 31 | 32 | echo "Creating Zip file to deploy in $DEPLOYMENT_FILE_NAME" 33 | zip -x */.git* -x .git* -x *.hg* -r "$DEPLOYMENT_FILE_NAME" ./ 34 | 35 | echo "Uploading to S3 for deployment" 36 | 37 | aws s3 cp "$DEPLOYMENT_FILE_NAME" "$S3_PATH" 38 | 39 | echo "Creating ElasticBeanstalk version: $VERSION_NAME" 40 | 41 | aws elasticbeanstalk create-application-version --application-name "$APPLICATION_NAME" --description "${CI_COMMIT_MESSAGE:0:200}" --version-label "$VERSION_NAME" --source-bundle "S3Bucket=$S3_BUCKET,S3Key=$S3_KEY" 42 | 43 | echo "Updating ElasticBeanstalk environment" 44 | aws elasticbeanstalk update-environment --environment-name "$ENVIRONMENT_NAME" --version-label "$VERSION_NAME" 45 | 46 | echo "Validating deployment" 47 | codeship_aws eb_deploy_validation "$APPLICATION_NAME" "$ENVIRONMENT_NAME" "$VERSION_NAME" 48 | -------------------------------------------------------------------------------- /deployment/scripts/codeship_aws_eb_deploy_validation: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | 5 | UPDATING_STATUS="Updating" 6 | 7 | APPLICATION_NAME=${1:?'You need to provide the Elastic Beanstalk application name'} 8 | ENVIRONMENT_NAME=${2:?'You need to provide the Elastic Beanstalk environment name'} 9 | ENVIRONMENT_VERSION=${3:?'You need to provide the version identifier that should be deployed'} 10 | 11 | source $DIR/elastic_beanstalk_environment_update.sh 12 | 13 | echo "Waiting for Environment to finish deployment" 14 | while [ "$(environment_status)" == "$UPDATING_STATUS" ] ; do 15 | sleep 10 16 | echo "." 17 | done 18 | 19 | echo "Deployment process finished, checking success of deployment" 20 | 21 | if [ "$(environment_health)" == "Green" ] 22 | then 23 | echo "Environment in a healthy state" 24 | else 25 | echo "Environment in an unhealthy state" 26 | exit 1 27 | fi 28 | 29 | if [ "$(environment_version)" == "$ENVIRONMENT_VERSION" ] 30 | then 31 | echo "New Version deployed" 32 | else 33 | echo "Deployment failed, still running old version" 34 | exit 1 35 | fi 36 | -------------------------------------------------------------------------------- /deployment/scripts/eb_deployment_check: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Check the status of an ElasticBeanstalk deployment by checking both, environment 3 | # health as well as the version the environment is running after the deployment. 4 | # 5 | # See http://docs.aws.amazon.com/cli/latest/reference/elasticbeanstalk/describe-environments.html 6 | # for a description of the output of the "describe-environments" API call. 7 | EB_APP_NAME=${1:?'You need to provide the Elastic Beanstalk application name'} 8 | EB_ENV_NAME=${2:?'You need to provide the Elastic Beanstalk environment name'} 9 | EB_ENV_VERSION=${3:?'You need to provide the version identifier that should be deployed'} 10 | 11 | WAIT=10 12 | 13 | function eb_describe_environment(){ 14 | codeship_aws elasticbeanstalk describe-environments --application-name "${EB_APP_NAME}" --environment-names "${EB_ENV_NAME}" 15 | } 16 | 17 | echo -e "\033[0;36mWaiting for environment ${EB_ENV_NAME} to finish the deployment.\033[0m" 18 | while [ -z "${status}" ] || [ "${status}" == "Updating" ] || [ "${status}" == "Launching" ]; do 19 | echo -en '\033[0;37m.\033[0m' 20 | sleep ${WAIT} 21 | status=$(eb_describe_environment | jq -r '.Environments[0].Status') 22 | done 23 | echo -e "\033[0;36m\nDeployment for environment \"${EB_ENV_NAME}\" finished.\033[0m" 24 | 25 | health=$(eb_describe_environment | jq -r '.Environments[0].Health') 26 | if [ "${health}" == "Green" ] || [ "${health}" == "Yellow" ]; then 27 | echo -e "\033[0;32mEnvironment \"${EB_ENV_NAME}\" is ${health}.\033[0m" 28 | else 29 | echo -e "\033[0;31mEnvironment \"${EB_ENV_NAME}\" is in an unhealthy state (\"${health}\").\033[0m" 30 | echo -e "\033[0;31mDeployment Failed.\033[0m" 31 | exit 1 32 | fi 33 | 34 | version=$(eb_describe_environment | jq -r '.Environments[0].VersionLabel') 35 | if [ "${version}" == "${EB_ENV_VERSION}" ]; then 36 | echo -e "\033[0;32mEnvironment \"${EB_ENV_NAME}\" is now running version \"${version}\".\033[0m" 37 | else 38 | echo -e "\033[0;31mEnvironment \"${EB_ENV_NAME}\" is running a different version (\"${version}\").\033[0m" 39 | echo -e "\033[0;31mDeployment Failed.\033[0m" 40 | exit 2 41 | fi 42 | 43 | -------------------------------------------------------------------------------- /deployment/scripts/elastic_beanstalk_environment_update.sh: -------------------------------------------------------------------------------- 1 | function eb_describe(){ 2 | aws elasticbeanstalk describe-environments --application-name "$APPLICATION_NAME" --environment-names "$ENVIRONMENT_NAME" 3 | } 4 | 5 | function environment_status(){ 6 | echo "$(eb_describe)" | jq -r .Environments[0].Status 7 | } 8 | 9 | function environment_health(){ 10 | echo "$(eb_describe)" | jq -r .Environments[0].Health 11 | } 12 | 13 | function environment_version(){ 14 | echo "$(eb_describe)" | jq -r .Environments[0].VersionLabel 15 | } 16 | 17 | function environment_cname(){ 18 | echo "$(eb_describe)" | jq -r .Environments[0].CNAME 19 | } 20 | -------------------------------------------------------------------------------- /deployment/scripts/utils: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | function remove_trailing_slashes() { 7 | s3_bucket=$1 8 | 9 | if [[ ${s3_bucket:0:1} == "/" ]]; then 10 | s3_bucket="${s3_bucket:1}" 11 | elif [[ ${s3_bucket: -1} == "/" ]]; then 12 | echo "hey" 13 | s3_bucket="${s3_bucket:0:-1}" 14 | fi 15 | 16 | echo $s3_bucket 17 | } 18 | 19 | function s3_bucket_name() { 20 | s3_bucket="$(remove_trailing_slashes "$1")" 21 | 22 | echo ${s3_bucket%%/*} 23 | } 24 | 25 | function s3_bucket_subfolder_path() { 26 | s3_bucket="$(remove_trailing_slashes "$1")" 27 | 28 | if [[ $s3_bucket =~ / ]]; then 29 | echo ${s3_bucket#*/} 30 | else 31 | echo "" 32 | fi 33 | } 34 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/Codeship.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/codeship-library/aws-utilities/7dc242c30a59659fb791eb186e19cdba2452c2a3/deployment/test/code_deploy/app/Codeship.png -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/appspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.0 2 | os: linux 3 | files: 4 | - source: /index.html 5 | destination: /var/www/html/ 6 | - source: /Codeship.png 7 | destination: /var/www/html/ 8 | hooks: 9 | BeforeInstall: 10 | - location: scripts/install_dependencies 11 | timeout: 300 12 | runas: root 13 | - location: scripts/start_server 14 | timeout: 300 15 | runas: root 16 | ApplicationStop: 17 | - location: scripts/stop_server 18 | timeout: 300 19 | runas: root 20 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 13 | 14 | Welcome to the Codeship 15 | 83 | 84 | 85 |
86 |

Congratulations

87 |

This application was deployed from Codeship to AWS CodeDeploy

88 |

What to do next? Take a look through the AWS CodeDeploy Documentation.

89 | 90 |
91 | 92 | 93 | 94 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/scripts/install_dependencies: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | yum install -y httpd 3 | 4 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/scripts/start_server: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | service httpd start 3 | 4 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/app/scripts/stop_server: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | isExistApp = `pgrep httpd` 3 | if [[ -n $isExistApp ]]; then 4 | service httpd stop 5 | fi 6 | 7 | -------------------------------------------------------------------------------- /deployment/test/code_deploy/integration-test: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | function debug() { echo -e "\033[0;37m$*\033[0m"; } 4 | function info() { echo -e "\033[0;36m$*\033[0m"; } 5 | function error() { >&2 echo -e "\033[0;31m$*\033[0m"; } 6 | function fail() { error ${1}; exit ${2:-1}; } 7 | 8 | function describe-stack(){ 9 | aws cloudformation describe-stacks --stack-name "${STACK_NAME}" 10 | } 11 | function stack_status(){ 12 | echo "$(describe-stack)" | jq -r .Stacks[0].StackStatus 13 | } 14 | function security_group_arn(){ 15 | echo "$(describe-stack)" | jq -r .Stacks[0].Outputs[0].OutputValue 16 | } 17 | 18 | function cleanup() { 19 | info "Delete CodeDeploy deployment group called ${DEPLOYMENT_GROUP_NAME}" 20 | aws deploy delete-deployment-group \ 21 | --application-name "${APPLICATION_NAME}" \ 22 | --deployment-group-name "${DEPLOYMENT_GROUP_NAME}" 23 | 24 | info "Delete CodeDeploy application called ${APPLICATION_NAME}" 25 | aws deploy delete-application \ 26 | --application-name "${APPLICATION_NAME}" 27 | 28 | info "Delete CloudFormation stack called ${STACK_NAME}" 29 | aws cloudformation delete-stack \ 30 | --stack-name "${STACK_NAME}" 31 | } 32 | trap cleanup EXIT SIGINT SIGTERM 33 | 34 | CODEDEPLOY_UNIX_TIME="$(date +%s)" 35 | STACK_NAME="aws-utilities-stack-${CODEDEPLOY_UNIX_TIME}" 36 | CODEDEPLOY_TAG_NAME='CodeDeployIntegrationTestTag' 37 | CODEDEPLOY_TAG_VALUE='CodeDeployIntegrationTestTagValue' 38 | APPLICATION_NAME="aws-utilities-cd-${CODEDEPLOY_UNIX_TIME}" 39 | DEPLOYMENT_GROUP_NAME="deployment-${CODEDEPLOY_UNIX_TIME}" 40 | S3_BUCKET="codeship-codedeploy-bucket" 41 | 42 | set -eo pipefail 43 | 44 | info "Creating a new CloudFormation stack called ${STACK_NAME}" 45 | aws cloudformation create-stack \ 46 | --stack-name "${STACK_NAME}" \ 47 | --template-url "http://s3.amazonaws.com/aws-codedeploy-us-east-1/templates/latest/CodeDeploy_SampleCF_Template.json" \ 48 | --parameters \ 49 | ParameterKey=InstanceCount,ParameterValue=1 \ 50 | ParameterKey=InstanceType,ParameterValue=t1.micro \ 51 | ParameterKey=KeyPairName,ParameterValue=CodeDeploy \ 52 | ParameterKey=OperatingSystem,ParameterValue=Linux \ 53 | ParameterKey=SSHLocation,ParameterValue=0.0.0.0/0 \ 54 | ParameterKey=TagKey,ParameterValue=${CODEDEPLOY_TAG_NAME} \ 55 | ParameterKey=TagValue,ParameterValue=${CODEDEPLOY_TAG_VALUE} \ 56 | --capabilities CAPABILITY_IAM 57 | 58 | info "Waiting for CloudFormation Stack to be ready" 59 | while [[ "$(stack_status)" =~ .*IN_PROGRESS$ ]] ; do 60 | sleep 10 61 | echo "." 62 | done 63 | 64 | info "Create CodeDeploy application called ${APPLICATION_NAME}" 65 | aws deploy create-application \ 66 | --application-name "${APPLICATION_NAME}" 67 | 68 | info "Create CodeDeploy deployment group called ${DEPLOYMENT_GROUP_NAME}" 69 | aws deploy create-deployment-group \ 70 | --application-name "${APPLICATION_NAME}" \ 71 | --deployment-group-name "${DEPLOYMENT_GROUP_NAME}" \ 72 | --deployment-config-name CodeDeployDefault.OneAtATime \ 73 | --ec2-tag-filters "Key=${CODEDEPLOY_TAG_NAME},Value=${CODEDEPLOY_TAG_VALUE},Type=KEY_AND_VALUE" \ 74 | --service-role-arn "$(security_group_arn)" 75 | 76 | info "Run integrated CodeDeploy deployment" 77 | codeship_aws codedeploy_deploy \ 78 | /deploy/test/code_deploy/app \ 79 | "${APPLICATION_NAME}" \ 80 | "${DEPLOYMENT_GROUP_NAME}" \ 81 | "${S3_BUCKET}" 82 | -------------------------------------------------------------------------------- /deployment/test/elastic_beanstalk/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:20.04 2 | 3 | EXPOSE 80 4 | 5 | RUN apt-get update \ 6 | && apt-get install -y --no-install-recommends \ 7 | nginx \ 8 | && echo "daemon off;" >> /etc/nginx/nginx.conf 9 | 10 | COPY index.html /var/www/html/ 11 | 12 | CMD ["/usr/sbin/nginx", "-c", "/etc/nginx/nginx.conf"] 13 | -------------------------------------------------------------------------------- /deployment/test/elastic_beanstalk/integration-test: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | function debug() { echo -e "\033[0;37m$*\033[0m"; } 4 | function info() { echo -e "\033[0;36m$*\033[0m"; } 5 | function error() { >&2 echo -e "\033[0;31m$*\033[0m"; } 6 | function fail() { error ${1}; exit ${2:-1}; } 7 | 8 | function cleanup() { 9 | info "Collecting debug information for application ${APPLICATION_NAME}, environment ${ENVIRONMENT_NAME}" 10 | aws elasticbeanstalk describe-environments \ 11 | --application-name "${APPLICATION_NAME}" \ 12 | --environment-names "${ENVIRONMENT_NAME}" 13 | 14 | info "Delete Elastic Beanstalk application named ${APPLICATION_NAME}" 15 | until aws elasticbeanstalk delete-application \ 16 | --terminate-env-by-force \ 17 | --application-name "${APPLICATION_NAME}"; do 18 | echo "Unable do delete elastic beanstalk application, retrying until success." 19 | done 20 | } 21 | trap cleanup EXIT SIGINT SIGTERM 22 | 23 | APPLICATION_NAME="aws-utilities-eb-$(date +%s)" 24 | ENVIRONMENT_NAME="env-$(date +%s)" 25 | S3_BUCKET="elasticbeanstalkcodeupload" 26 | 27 | set -eo pipefail 28 | 29 | # switch to the top level directory 30 | dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 31 | cd "${dir}/../.." 32 | 33 | # load helper functions to abstract some calls to the AWS CLI 34 | source ./scripts/elastic_beanstalk_environment_update.sh 35 | 36 | info "Create Elastic Beanstalk application named ${APPLICATION_NAME}" 37 | aws elasticbeanstalk create-application \ 38 | --application-name "${APPLICATION_NAME}" 39 | 40 | info "Create Elastic Beanstalk environment called ${ENVIRONMENT_NAME}" 41 | aws elasticbeanstalk create-environment \ 42 | --application-name "${APPLICATION_NAME}" \ 43 | --environment-name "${ENVIRONMENT_NAME}" \ 44 | --option-settings file://test/elastic_beanstalk/options.txt \ 45 | --solution-stack-name "64bit Amazon Linux 2018.03 v2.16.7 running Docker 19.03.13-ce" 46 | 47 | info "Waiting for environment to start" 48 | while [ "$(environment_status)" != "Ready" ] ; do 49 | sleep 10 50 | echo "." 51 | done 52 | 53 | info "Creating deployment assets" 54 | mkdir -p ./tmp/deployment 55 | cp test/elastic_beanstalk/Dockerfile ./tmp/deployment 56 | echo "${CI_COMMIT_ID}" > ./tmp/deployment/index.html 57 | 58 | codeship_aws eb_deploy \ 59 | "/deploy/tmp/deployment" \ 60 | "${APPLICATION_NAME}" \ 61 | "${ENVIRONMENT_NAME}" \ 62 | "${S3_BUCKET}" 63 | 64 | info "Checking Environment if deployment matches ci commit hash" 65 | deployed_commit_id=$(curl -s "$(environment_cname)") 66 | 67 | if [ "${deployed_commit_id}" == "${CI_COMMIT_ID}" ]; then 68 | info "Matching commit hashes ${CI_COMMIT_ID}, deployment successful" 69 | else 70 | error "Commit Hashes did not match:" 71 | error "CI_COMMIT_ID: ${CI_COMMIT_ID}" 72 | error "DEPLOYED: ${deployed_commit_id}" 73 | exit 1 74 | fi 75 | -------------------------------------------------------------------------------- /deployment/test/elastic_beanstalk/options.txt: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "Namespace": "aws:autoscaling:launchconfiguration", 4 | "OptionName": "IamInstanceProfile", 5 | "Value": "aws-elasticbeanstalk-ec2-role" 6 | } 7 | ] 8 | -------------------------------------------------------------------------------- /deployment/test/test_tools_available.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | mkdir -p /deploy/tmp 7 | 8 | # Test that Zip is Available 9 | zip -r /deploy/tmp/upload_to_s3.zip /deploy/test/upload_to_s3 10 | 11 | # Test that JQ is available and works 12 | echo "{\"test\": \"result\"}" | jq ".test" | grep "result" 13 | -------------------------------------------------------------------------------- /deployment/test/upload_to_s3: -------------------------------------------------------------------------------- 1 | File to test S3 upload 2 | -------------------------------------------------------------------------------- /deployment/test/utils: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -o pipefail 5 | 6 | source "$(dirname "$0")/../scripts/utils" 7 | 8 | echo "Testing for s3_bucket" 9 | test $(s3_bucket_name "s3_bucket") == "s3_bucket" 10 | test -z $(s3_bucket_subfolder_path "s3_bucket") 11 | 12 | echo "Testing for /s3_bucket" 13 | test $(s3_bucket_name "/s3_bucket") == "s3_bucket" 14 | test -z $(s3_bucket_subfolder_path "/s3_bucket") 15 | 16 | echo "Testing for s3_bucket/subdir" 17 | test $(s3_bucket_name "s3_bucket/subdir") == "s3_bucket" 18 | test $(s3_bucket_subfolder_path "s3_bucket/subdir") == "subdir" 19 | 20 | echo "Testing for /s3_bucket/subdir" 21 | test $(s3_bucket_name "/s3_bucket/subdir") == "s3_bucket" 22 | test $(s3_bucket_subfolder_path "/s3_bucket/subdir") == "subdir" 23 | 24 | echo "Testing for s3_bucket/subdir/another" 25 | test $(s3_bucket_name "s3_bucket/subdir/another") == "s3_bucket" 26 | test $(s3_bucket_subfolder_path "s3_bucket/subdir/another") == "subdir/another" 27 | 28 | echo "Testing for /s3_bucket/subdir/another" 29 | test $(s3_bucket_name "/s3_bucket/subdir/another") == "s3_bucket" 30 | test $(s3_bucket_subfolder_path "/s3_bucket/subdir/another") == "subdir/another" 31 | 32 | echo "done." 33 | -------------------------------------------------------------------------------- /dockercfg-generator/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM codeship/aws-base:latest 2 | LABEL maintainer='Codeship Inc., ' 3 | 4 | RUN \ 5 | apk --no-cache add \ 6 | bash \ 7 | docker \ 8 | jq 9 | 10 | COPY aws_docker_creds.sh / 11 | 12 | ENTRYPOINT ["/aws_docker_creds.sh"] 13 | -------------------------------------------------------------------------------- /dockercfg-generator/Dockerfile.test: -------------------------------------------------------------------------------- 1 | FROM alpine:3.21 2 | LABEL maintainer='Codeship Inc., ' 3 | 4 | RUN \ 5 | echo "Hello ECR at $(date)" > hello.txt && \ 6 | cat hello.txt 7 | 8 | CMD ["cat hello.txt"] 9 | -------------------------------------------------------------------------------- /dockercfg-generator/README.md: -------------------------------------------------------------------------------- 1 | # Dockercfg Generation Service for AWS ECR 2 | 3 | This container allows you to generate a temporary dockercfg using your AWS credentials 4 | and writes it to a specified filename. Typical usage of this image would be to run it 5 | with a volume attached, and write the dockercfg to that volume. 6 | 7 | In order to export a dockercfg, the container needs access to a docker instance, so 8 | you must mount a docker socket, or provide access to a docker host in some way. 9 | 10 | ```bash 11 | $ cat aws_creds.env 12 | AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXX 13 | AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 14 | AWS_REGION=us-east-1 15 | $ docker run -it -v /home/myuser/data:/opt/data --env-file=aws_creds.env -v /var/run/docker.sock:/var/run/docker.sock codeship/aws-ecr-dockercfg-generator /opt/data/aws.dockercfg 16 | Logging into AWS ECR 17 | WARNING: login credentials saved in /root/.docker/config.json 18 | Login Succeeded 19 | Writing Docker creds to /opt/data/aws.dockerccfg 20 | $ cat /home/myuser/data/aws.dockercfg # file is available locally 21 | ``` 22 | 23 | As per AWS documentation, this dockercfg should be valid for 48 hours. 24 | 25 | ## Using with Codeship 26 | 27 | Codeship supports using custom images to generate dockercfg files during the build process. To use this image to integrate with AWS ECR, simply define a entry in your services file for this image, and reference it from any steps or services which need to interact with ECR repositories with the `dockercfg_service` field. You'll also need to provide the following environment variables using an [encrypted env file](https://codeship.com/documentation/docker/encryption/): 28 | 29 | * `AWS_REGION` - Your selected AWS Region, ensure this is a region supporting AWS ECR 30 | * `AWS_ACCESS_KEY_ID` - Your AWS Access Key 31 | * `AWS_SECRET_ACCESS_KEY` - Your AWS Access Secret 32 | 33 | Optionally, you can also set the following variables to assume a role across accounts before generating the dockercfg: 34 | 35 | * `AWS_STS_ROLE` - The AWS role to assume 36 | * `AWS_STS_ACCOUNT` - The AWS account the role exists in 37 | 38 | Here is an example of using and ECR Dockercfg generator to authenticate pushing an image. 39 | 40 | ```yaml 41 | # codeship-services.yml 42 | app: 43 | build: 44 | image: 874084658176.dkr.ecr.us-east-1.amazonaws.com/myorg/myapp 45 | dockerfile_path: ./Dockerfile 46 | aws_dockercfg: 47 | image: codeship/aws-ecr-dockercfg-generator 48 | add_docker: true 49 | encrypted_env_file: aws.env.encrypted 50 | ``` 51 | 52 | ```yaml 53 | # codeship-steps.yml 54 | - service: app 55 | type: push 56 | tag: master 57 | image_name: 874084658176.dkr.ecr.us-east-1.amazonaws.com/myorg/myapp 58 | registry: https://874084658176.dkr.ecr.us-east-1.amazonaws.com 59 | dockercfg_service: aws_dockercfg 60 | ``` 61 | 62 | You can also use this authentication to pull images, or use with caching, by defining the `dockercfg_service` field on groups of steps, or each individual step that pulls or pushes an image, or by adding the field to specific services. 63 | 64 | ## ECR - public vs private repositories 65 | 66 | By default, this tool will generate docker configurations for pushing to an AWS ECR private registry, eg `123456789012.dkr.ecr.us-east-1.amazonaws.com/your-image`. These images are only visible within (I think) your AWS account via valid IAM users. 67 | 68 | AWS recently released public [ECR public repositories](https://docs.aws.amazon.com/AmazonECR/latest/public/public-repositories.html), eg `public.ecr.aws/abcd1234/your-image` that uses a different API to get credentials. In order to generate credentials for a public ECR repository, set the `ECR_PUBLIC` environment variable. An example push would be: 69 | 70 | ``` 71 | # codeship-steps.yml 72 | - service: app 73 | type: push 74 | tag: master 75 | image_name: public.ecr.aws/abcd1234/my-app 76 | registry: https://public.ecr.aws 77 | dockercfg_service: aws_dockercfg 78 | ``` 79 | 80 | ## Troubleshooting 81 | 82 | #### "No basic auth credentials" error on push 83 | Make sure the registry entry in your steps does not contain a trailing slash. 84 | -------------------------------------------------------------------------------- /dockercfg-generator/aws_docker_creds.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | echo 'AWS ECR dockercfg generator' 6 | 7 | : "${AWS_REGION:?Need to set AWS_REGION}" 8 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID}" 9 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY}" 10 | 11 | cat << EOF > ~/.aws/config 12 | [default] 13 | region = $AWS_REGION 14 | EOF 15 | 16 | # For multi account aws setups, use primary credentials to assume the role in 17 | # the target account 18 | AWS_ACCOUNT="" 19 | if [[ -n $AWS_STS_ROLE || -n $AWS_STS_ACCOUNT ]]; then 20 | : "${AWS_STS_ROLE:?Need to set AWS_STS_ROLE}" 21 | : "${AWS_STS_ACCOUNT:?Need to set AWS_STS_ACCOUNT}" 22 | 23 | role="arn:aws:iam::${AWS_STS_ACCOUNT}:role/${AWS_STS_ROLE}" 24 | echo "Using STS to get credentials for ${role}" 25 | 26 | aws_tmp=$(mktemp -t aws-json-XXXXXX) 27 | 28 | aws sts assume-role --role-arn "${role}" --role-session-name aws_docker_creds > "${aws_tmp}" 29 | 30 | export AWS_ACCESS_KEY_ID=$(cat ${aws_tmp} | jq -r ".Credentials.AccessKeyId") 31 | export AWS_SECRET_ACCESS_KEY=$(cat ${aws_tmp} | jq -r ".Credentials.SecretAccessKey") 32 | export AWS_SESSION_TOKEN=$(cat ${aws_tmp} | jq -r ".Credentials.SessionToken") 33 | export AWS_SESSION_EXPIRATION=$(cat ${aws_tmp} | jq -r ".Credentials.Expiration") 34 | 35 | AWS_ACCOUNT=$AWS_STS_ACCOUNT 36 | else 37 | AWS_ACCOUNT=$(aws sts get-caller-identity | jq -r ".Account") 38 | fi 39 | 40 | # For public images stored in public.ecr.aws, we need to use a different CLI call 41 | if [[ -v ECR_PUBLIC ]]; then 42 | COMMAND=ecr-public 43 | SERVER=public.ecr.aws 44 | else 45 | COMMAND=ecr 46 | SERVER=${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com 47 | fi 48 | 49 | # fetching aws docker login 50 | # aws deprecated the get-login function in favor of get-login-password 51 | # https://docs.aws.amazon.com/cli/latest/reference/ecr/get-login.html 52 | echo "Logging into AWS ECR with account ${AWS_ACCOUNT}" 53 | aws ${COMMAND} get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${SERVER} 54 | 55 | # writing aws docker creds to desired path 56 | echo "Writing Docker creds to $1" 57 | chmod 544 ~/.docker/config.json 58 | cp ~/.docker/config.json $1 59 | -------------------------------------------------------------------------------- /dockercfg-generator/codeship-steps.yml: -------------------------------------------------------------------------------- 1 | - service: demo 2 | type: push 3 | image_name: 870119404647.dkr.ecr.us-east-1.amazonaws.com/codeship-testing 4 | registry: https://870119404647.dkr.ecr.us-east-1.amazonaws.com 5 | dockercfg_service: aws_dockercfg 6 | - service: aws 7 | tag: master 8 | type: push 9 | image_name: codeship/aws-ecr-dockercfg-generator 10 | encrypted_dockercfg_path: dockercfg.encrypted 11 | -------------------------------------------------------------------------------- /dockercfg.encrypted: -------------------------------------------------------------------------------- 1 | codeship:v2 2 | rJm/UUjqQ9KEyvvpK1UFzz0IDjqsXqR44PkkvWds1hSqP16O2o2WCLk1m7BofIIx7XleZdciI1UduVIPgKB/DGINrUPjcXbTfOF60473DJMOnbAI0HHYh3CEYQH4KYvglbJ3DM/LGJefSabvChH59C8ZkbPI1eIUfMt7gzX0ZewNlqG3a+gqnQifJeLGvs3+bXj9NTZmk7rLwTiZO7TGjk6f/pHrse5MlqY3KRsnAWcjTFuvZk3GyEFXEJ0mMlY3lelJSzhcTRJE0Z9AnNE7LiByHaJRMyVotUCi6rh3weu3ijARAtP04fEVw6HN5br69588aQBSv817n6p6AXsNz4jgLpU= -------------------------------------------------------------------------------- /site.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - hosts: localhost 3 | roles: 4 | - role: aws 5 | aws_user: root 6 | aws_home: /root 7 | codeship_python: /usr/bin/python3 8 | codeship_venv: "{{ aws_home }}/.codeship-venv" 9 | -------------------------------------------------------------------------------- /tasks/install.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # Mothership calls a local AWS CLI inside a Basic container for AWS deployment tasks. 3 | # We preconfigure a virtualenv with AWS CLI to avoid upstream dependencies. 4 | 5 | - name: Stat 6 | ansible.builtin.stat: 7 | path: "{{ codeship_venv }}" 8 | register: directory 9 | 10 | - name: Install virtualenv (Focal) 11 | ansible.builtin.pip: 12 | name: virtualenv 13 | state: present 14 | when: 15 | - not directory.stat.exists 16 | - ansible_distribution_release == 'focal' 17 | 18 | - name: Install virtualenv (Noble) 19 | ansible.builtin.apt: 20 | name: python3-virtualenv 21 | state: present 22 | install_recommends: false 23 | when: 24 | - not directory.stat.exists 25 | - ansible_distribution_release == 'noble' 26 | 27 | - name: Create virtualenv 28 | ansible.builtin.command: virtualenv --python={{ codeship_python }} {{ codeship_venv }} 29 | become: true 30 | become_user: "{{ aws_user }}" 31 | when: not directory.stat.exists 32 | 33 | - name: Install AWS CLI into virtualenv 34 | ansible.builtin.pip: 35 | name: awscli 36 | virtualenv: "{{ codeship_venv }}" 37 | become: true 38 | become_user: "{{ aws_user }}" 39 | when: not directory.stat.exists 40 | 41 | - name: Uninstall virtualenv (Focal) 42 | ansible.builtin.pip: 43 | name: virtualenv 44 | state: absent 45 | when: 46 | - not directory.stat.exists 47 | - ansible_distribution_release == 'focal' 48 | 49 | - name: Uninstall virtualenv (Noble) 50 | ansible.builtin.apt: 51 | name: python3-virtualenv 52 | state: absent 53 | when: 54 | - not directory.stat.exists 55 | - ansible_distribution_release == 'noble' 56 | 57 | - name: Move scripts into the container 58 | ansible.posix.synchronize: 59 | src: "{{ role_path }}/deployment/scripts/" 60 | dest: "{{ aws_home }}/bin/" 61 | use_ssh_args: true 62 | owner: false 63 | group: false 64 | become: true 65 | become_user: "{{ aws_user }}" 66 | 67 | - name: Fix file permissions 68 | ansible.builtin.file: 69 | dest: "{{ aws_home }}/bin/" 70 | owner: "{{ aws_user }}" 71 | recurse: true 72 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Install AWS 3 | ansible.builtin.include_tasks: install.yml 4 | --------------------------------------------------------------------------------