├── Container-Root ├── requirements.txt ├── function │ ├── startup.sh │ ├── lambda_function.py │ ├── test1.sh │ └── test2.sh ├── job │ ├── startup.sh │ ├── test1.sh │ └── test2.sh ├── setup.sh ├── startup.sh ├── test1.sh ├── test2.sh └── app │ └── server.py ├── docs └── img │ ├── aws-do-docker-demo.gif │ └── aws-do-docker-flow.png ├── to ├── kubernetes │ └── template │ │ ├── 010-namespace.yaml │ │ ├── 030-service.yaml │ │ ├── 050-ingress.yaml │ │ ├── 040-hpa.yaml │ │ └── 020-deployment.yaml ├── lambda │ ├── template │ │ ├── lambda-trust.json │ │ └── lambda-policy.json │ └── test │ │ ├── test1.sh │ │ └── test2.sh └── compose │ ├── template │ ├── docker-compose.yaml │ ├── ecs-trust.json │ ├── ecs-params-EC2.yaml │ ├── ecs-params-FARGATE.yaml │ ├── ecs-compose.yaml │ └── ecs-exec-policy.json │ └── test │ ├── test1.sh │ └── test2.sh ├── .dockerignore ├── .gitignore ├── CODE_OF_CONDUCT.md ├── pull.sh ├── push.sh ├── install ├── install-kubectl.sh ├── install-docker-compose.sh ├── install-aws-cli.sh ├── install-ecs-cli.sh ├── install-ssm-plugin.sh └── gpg-public-key ├── Dockerfile ├── LICENSE ├── login.sh ├── Dockerfile-lambda ├── config.sh ├── test.sh ├── status.sh ├── .fun ├── logs.sh ├── CONTRIBUTING.md ├── exec.sh ├── stop.sh ├── .env ├── run.sh └── README.md /Container-Root/requirements.txt: -------------------------------------------------------------------------------- 1 | fastapi 2 | hypercorn 3 | -------------------------------------------------------------------------------- /docs/img/aws-do-docker-demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-do-docker/HEAD/docs/img/aws-do-docker-demo.gif -------------------------------------------------------------------------------- /docs/img/aws-do-docker-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-do-docker/HEAD/docs/img/aws-do-docker-flow.png -------------------------------------------------------------------------------- /to/kubernetes/template/010-namespace.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Namespace 3 | metadata: 4 | name: ${NAMESPACE} 5 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | *.sh 2 | .env 3 | .fun 4 | build.sh 5 | exec.sh 6 | logs.sh 7 | pull.sh 8 | push.sh 9 | run.sh 10 | status.sh 11 | stop.sh 12 | 13 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | install/ecs-cli.asc 2 | *.swp 3 | to/compose/app 4 | to/kubernetes/app 5 | install/sessionmanager-bundle 6 | install/*.rpm 7 | install/*/deb 8 | *.bak 9 | -------------------------------------------------------------------------------- /to/lambda/template/lambda-trust.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Principal": { 7 | "Service": "lambda.amazonaws.com" 8 | }, 9 | "Action": "sts:AssumeRole" 10 | } 11 | ] 12 | } 13 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /to/compose/template/docker-compose.yaml: -------------------------------------------------------------------------------- 1 | version: "3.9" 2 | 3 | services: 4 | 5 | ${CONTAINER}: 6 | image: ${REGISTRY}${IMAGE}${TAG} 7 | environment: 8 | - PORT_INTERNAL=${PORT_INTERNAL} 9 | ports: 10 | - ${PORT_EXTERNAL}:${PORT_INTERNAL} 11 | # volumes: 12 | # - ${VOLUME_PATH_EXTERNAL}:${VOLUME_PATH_INTERNAL} 13 | 14 | -------------------------------------------------------------------------------- /to/kubernetes/template/030-service.yaml: -------------------------------------------------------------------------------- 1 | kind: Service 2 | apiVersion: v1 3 | metadata: 4 | name: ${APP_NAME} 5 | namespace: ${NAMESPACE} 6 | labels: 7 | app: ${APP_NAME} 8 | spec: 9 | ports: 10 | - name: svc-port 11 | port: ${PORT_EXTERNAL} 12 | targetPort: pod-port 13 | type: ClusterIP 14 | selector: 15 | app: ${APP_NAME} 16 | -------------------------------------------------------------------------------- /to/compose/template/ecs-trust.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": { 4 | "Effect": "Allow", 5 | "Principal": { 6 | "Service": [ 7 | "ecs-tasks.amazonaws.com", 8 | "ecs.amazonaws.com", 9 | "batch.amazonaws.com", 10 | "ec2.amazonaws.com" 11 | ] 12 | }, 13 | "Action": "sts:AssumeRole" 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /to/lambda/template/lambda-policy.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Action": [ 7 | "logs:CreateLogGroup", 8 | "logs:CreateLogStream", 9 | "logs:PutLogEvents" 10 | ], 11 | "Resource": "*" 12 | } 13 | ] 14 | } 15 | -------------------------------------------------------------------------------- /pull.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | docker image pull ${REGISTRY}${IMAGE}${TAG} 11 | 12 | -------------------------------------------------------------------------------- /push.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | docker image push ${REGISTRY}${IMAGE}${TAG} 11 | 12 | -------------------------------------------------------------------------------- /to/kubernetes/template/050-ingress.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: networking.k8s.io/v1 2 | kind: Ingress 3 | metadata: 4 | name: ${APP_NAME} 5 | namespace: ${NAMESPACE} 6 | spec: 7 | rules: 8 | - host: "${APP_DNS_NAME}" 9 | http: 10 | paths: 11 | - pathType: Prefix 12 | path: "/" 13 | backend: 14 | service: 15 | name: ${APP_NAME} 16 | port: 17 | number: ${PORT_EXTERNAL} 18 | -------------------------------------------------------------------------------- /to/compose/template/ecs-params-EC2.yaml: -------------------------------------------------------------------------------- 1 | version: 1 2 | task_definition: 3 | ecs_network_mode: bridge 4 | task_role_arn: ecsTaskRole 5 | task_execution_role: ecsTaskExecutionRole 6 | task_size: 7 | mem_limit: ${ECS_MEM_LIMIT} 8 | cpu_limit: ${ECS_CPU_LIMIT} 9 | run_params: 10 | network_configuration: 11 | bridge_configuration: 12 | subnets: 13 | - "${SUBNET_ID}" 14 | security_groups: 15 | - "${SG_ID}" 16 | assign_public_ip: ${ECS_ASSIGN_PUBLIC_IP} 17 | -------------------------------------------------------------------------------- /to/compose/template/ecs-params-FARGATE.yaml: -------------------------------------------------------------------------------- 1 | version: 1 2 | task_definition: 3 | ecs_network_mode: awsvpc 4 | task_role_arn: ecsTaskRole 5 | task_execution_role: ecsTaskExecutionRole 6 | task_size: 7 | mem_limit: ${ECS_MEM_LIMIT} 8 | cpu_limit: ${ECS_CPU_LIMIT} 9 | run_params: 10 | network_configuration: 11 | awsvpc_configuration: 12 | subnets: 13 | - "${SUBNET_ID}" 14 | security_groups: 15 | - "${SG_ID}" 16 | assign_public_ip: ${ECS_ASSIGN_PUBLIC_IP} 17 | -------------------------------------------------------------------------------- /Container-Root/function/startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then 9 | exec /usr/local/bin/aws-lambda-rie /usr/local/bin/python -m awslambdaric $@ 10 | else 11 | exec /usr/local/bin/python -m awslambdaric $@ 12 | fi 13 | -------------------------------------------------------------------------------- /to/lambda/test/test1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | RESULT=$(./exec.sh) 9 | 10 | echo "$RESULT" 11 | 12 | echo "$RESULT" | grep Healthy > /dev/null 13 | 14 | if [ $? -eq 0 ]; then 15 | echo "" 16 | echo "Test1 succeeded" 17 | else 18 | echo "" 19 | echo "Test1 failed" 20 | fi 21 | echo "" 22 | -------------------------------------------------------------------------------- /to/compose/template/ecs-compose.yaml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | 3 | services: 4 | 5 | ${CONTAINER}: 6 | image: ${REGISTRY}${IMAGE}${TAG} 7 | environment: 8 | - PORT_INTERNAL=${PORT_INTERNAL} 9 | ports: 10 | - ${PORT_EXTERNAL}:${PORT_INTERNAL} 11 | healthcheck: 12 | test: ["bash","-c","curl http://localhost:${PORT_INTERNAL}/"] 13 | interval: 10s 14 | timeout: 5s 15 | retries: 2 16 | logging: 17 | driver: awslogs 18 | options: 19 | awslogs-group: /aws/ecs/${ECS_CLUSTER} 20 | awslogs-region: ${REGION} 21 | awslogs-stream-prefix: ${ECS_CLUSTER} 22 | 23 | -------------------------------------------------------------------------------- /Container-Root/function/lambda_function.py: -------------------------------------------------------------------------------- 1 | ###################################################################### 2 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 3 | # SPDX-License-Identifier: MIT-0 # 4 | ###################################################################### 5 | 6 | import json 7 | 8 | def handler(event, context): 9 | 10 | print(f"event = {event}") 11 | response = { "Status": "Healthy" } 12 | 13 | if "text" in event.keys(): 14 | response ={ "Message": event['text'] } 15 | 16 | print(f"response = {response}") 17 | return response 18 | -------------------------------------------------------------------------------- /install/install-kubectl.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -e .env ]; then 9 | . .env 10 | elif [ -e ../.env ]; then 11 | pushd .. 12 | . .env 13 | popd 14 | fi 15 | 16 | # Install kubectl 17 | curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl 18 | chmod +x ./kubectl 19 | mv ./kubectl /usr/local/bin 20 | kubectl version --client 21 | -------------------------------------------------------------------------------- /Container-Root/job/startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Job startup script 9 | echo "Container-Root/job/startup.sh executed" 10 | 11 | # Start job 12 | cd /job 13 | 14 | LIMIT=3 15 | if [ ! "${ITERATION_LIMIT}" == "" ]; then 16 | LIMIT=${ITERATION_LIMIT} 17 | fi 18 | 19 | i=0 20 | while [ $i -lt $LIMIT ]; do 21 | echo "Iteration $i: $(date)" 22 | sleep 5 23 | i=$((i+1)) 24 | done 25 | 26 | -------------------------------------------------------------------------------- /Container-Root/setup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -d /etc/apt ]; then 9 | [ -n "$http_proxy" ] && echo "Acquire::http::proxy \"${http_proxy}\";" > /etc/apt/apt.conf; \ 10 | [ -n "$https_proxy" ] && echo "Acquire::https::proxy \"${https_proxy}\";" >> /etc/apt/apt.conf; \ 11 | [ -f /etc/apt/apt.conf ] && cat /etc/apt/apt.conf 12 | fi 13 | 14 | pip install -r /requirements.txt 15 | 16 | -------------------------------------------------------------------------------- /Container-Root/startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Container startup script 9 | echo "Container-Root/startup.sh executed" 10 | 11 | # Start sample web api in background 12 | PORT=80 13 | if [ ! "$PORT_INTERNAL" == "" ]; then 14 | PORT=$PORT_INTERNAL 15 | fi 16 | cd /app 17 | hypercorn server:app -b 0.0.0.0:${PORT} & 18 | 19 | # Start sample ininite loop to produce periodic log output 20 | while true; do date; sleep 10; done 21 | 22 | -------------------------------------------------------------------------------- /to/lambda/test/test2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | payload=$(echo '{"text": "This is a Docker container running as a lambda function!"}' | base64 -w 0) 9 | 10 | RESULT=$(./exec.sh ${payload}) 11 | 12 | echo "$RESULT" 13 | 14 | echo "$RESULT" | grep Docker > /dev/null 15 | 16 | if [ $? -eq 0 ]; then 17 | echo "" 18 | echo "Test2 succeeded" 19 | else 20 | echo "" 21 | echo "Test2 failed" 22 | fi 23 | echo "" 24 | -------------------------------------------------------------------------------- /Container-Root/job/test1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 1 9 | 10 | CMD="echo {\"Status\": \"Successful\"} | tee /tmp/status.txt > /dev/null; cat /tmp/status.txt" 11 | 12 | #echo "$CMD" 13 | 14 | RESULT=$(eval "$CMD") 15 | 16 | echo "$RESULT" 17 | 18 | echo "$RESULT" | grep Successful > /dev/null 19 | 20 | if [ $? -eq 0 ]; then 21 | echo "" 22 | echo "Test1 succeeded" 23 | else 24 | echo "" 25 | echo "Test1 failed" 26 | fi 27 | echo "" 28 | -------------------------------------------------------------------------------- /Container-Root/function/test1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 1 9 | 10 | CMD="curl -i -s -XPOST 'http://localhost:8080/2015-03-31/functions/function/invocations' -d '{}'" 11 | 12 | echo "$CMD" 13 | 14 | RESULT=$(eval "$CMD") 15 | 16 | echo "$RESULT" 17 | 18 | echo "$RESULT" | grep Healthy > /dev/null 19 | 20 | if [ $? -eq 0 ]; then 21 | echo "" 22 | echo "Test1 succeeded" 23 | else 24 | echo "" 25 | echo "Test1 failed" 26 | fi 27 | echo "" 28 | -------------------------------------------------------------------------------- /Container-Root/job/test2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 2 9 | 10 | CMD="echo {\"Message\": \"Job container unit test successful\"} | tee /tmp/message.txt > /dev/null; cat /tmp/message.txt" 11 | 12 | #echo "$CMD" 13 | 14 | RESULT=$(eval "$CMD") 15 | 16 | echo "$RESULT" 17 | 18 | echo "$RESULT" | grep successful > /dev/null 19 | 20 | if [ $? -eq 0 ]; then 21 | echo "" 22 | echo "Test2 succeeded" 23 | else 24 | echo "" 25 | echo "Test2 failed" 26 | fi 27 | echo "" 28 | -------------------------------------------------------------------------------- /to/compose/test/test1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | endpoint="http://localhost" 9 | if [ ! -z "$1" ]; then 10 | endpoint="$1" 11 | fi 12 | 13 | CMD="curl -i -s ${endpoint}" 14 | 15 | echo "$CMD" 16 | 17 | RESULT=$(eval "$CMD") 18 | 19 | echo "$RESULT" 20 | 21 | echo "$RESULT" | grep Healthy > /dev/null 22 | 23 | if [ $? -eq 0 ]; then 24 | echo "" 25 | echo "Test1 succeeded" 26 | else 27 | echo "" 28 | echo "Test1 failed" 29 | fi 30 | echo "" 31 | -------------------------------------------------------------------------------- /Container-Root/function/test2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 2 9 | 10 | CMD="curl -i -s -XPOST 'http://localhost:8080/2015-03-31/functions/function/invocations' -d '{\"text\": \"DockerCon 2021 demo!\"}'" 11 | 12 | echo "$CMD" 13 | 14 | RESULT=$(eval "$CMD") 15 | 16 | echo "$RESULT" 17 | 18 | echo "$RESULT" | grep DockerCon > /dev/null 19 | 20 | if [ $? -eq 0 ]; then 21 | echo "" 22 | echo "Test2 succeeded" 23 | else 24 | echo "" 25 | echo "Test2 failed" 26 | fi 27 | echo "" 28 | -------------------------------------------------------------------------------- /Container-Root/test1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 1 9 | 10 | PORT=80 11 | if [ ! -z "$PORT_INTERNAL" ]; then 12 | PORT=$PORT_INTERNAL 13 | fi 14 | 15 | CMD="curl -i -s 'http://localhost:${PORT_INTERNAL}/'" 16 | 17 | echo "$CMD" 18 | 19 | RESULT=$(eval "$CMD") 20 | 21 | echo "$RESULT" 22 | 23 | echo "$RESULT" | grep Healthy > /dev/null 24 | 25 | if [ $? -eq 0 ]; then 26 | echo "" 27 | echo "Test1 succeeded" 28 | else 29 | echo "" 30 | echo "Test1 failed" 31 | fi 32 | echo "" 33 | -------------------------------------------------------------------------------- /Container-Root/test2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Unit test 2 9 | 10 | PORT=80 11 | if [ ! -z "$PORT_INTERNAL" ]; then 12 | PORT=$PORT_INTERNAL 13 | fi 14 | 15 | CMD="curl -i -s 'http://localhost:${PORT_INTERNAL}/say?text=Nice%20demo%21'" 16 | 17 | echo "$CMD" 18 | 19 | RESULT=$(eval "$CMD") 20 | 21 | echo "$RESULT" 22 | 23 | echo "$RESULT" | grep Nice > /dev/null 24 | 25 | if [ $? -eq 0 ]; then 26 | echo "" 27 | echo "Test2 succeeded" 28 | else 29 | echo "" 30 | echo "Test2 failed" 31 | fi 32 | echo "" 33 | -------------------------------------------------------------------------------- /to/compose/test/test2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | endpoint="http://localhost" 9 | if [ ! -z "$1" ]; then 10 | endpoint="$1" 11 | fi 12 | 13 | CMD="curl -i -s ${endpoint}/say?text=This%20is%20a%20Docker%20container%20running%20in%20ECS%21" 14 | 15 | echo "$CMD" 16 | 17 | RESULT=$(eval "$CMD") 18 | 19 | echo "$RESULT" 20 | 21 | echo "$RESULT" | grep Docker > /dev/null 22 | 23 | if [ $? -eq 0 ]; then 24 | echo "" 25 | echo "Test2 succeeded" 26 | else 27 | echo "" 28 | echo "Test2 failed" 29 | fi 30 | echo "" 31 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | ARG BASE_IMAGE_PATH="alpine:latest" 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | FROM $BASE_IMAGE_PATH 9 | 10 | ARG BUILD="202104" 11 | ARG MAINTAINER="Full Name " 12 | ARG DESCRIPTION="Depend on Docker Image" 13 | ARG http_proxy 14 | ARG https_proxy 15 | ARG no_proxy 16 | 17 | LABEL MAINTAINER="$MAINTAINER" 18 | 19 | LABEL DESCRIPTION="$DESCRIPTION" 20 | 21 | LABEL BUILD="$BUILD" 22 | 23 | ADD Container-Root / 24 | 25 | RUN export http_proxy=$http_proxy; export https_proxy=$https_proxy; export no_proxy=$no_proxy; /setup.sh; rm -f /setup.sh 26 | 27 | CMD /startup.sh 28 | 29 | -------------------------------------------------------------------------------- /to/compose/template/ecs-exec-policy.json: -------------------------------------------------------------------------------- 1 | { 2 | "Version": "2012-10-17", 3 | "Statement": [ 4 | { 5 | "Effect": "Allow", 6 | "Action": [ 7 | "ssmmessages:CreateControlChannel", 8 | "ssmmessages:CreateDataChannel", 9 | "ssmmessages:OpenControlChannel", 10 | "ssmmessages:OpenDataChannel" 11 | ], 12 | "Resource": "*" 13 | }, 14 | { 15 | "Effect": "Allow", 16 | "Action": [ 17 | "logs:DescribeLogGroups" 18 | ], 19 | "Resource": "*" 20 | }, 21 | { 22 | "Effect": "Allow", 23 | "Action": [ 24 | "logs:CreateLogStream", 25 | "logs:DescribeLogStreams", 26 | "logs:PutLogEvents" 27 | ], 28 | "Resource": "${LOG_GROUP_ARN}" 29 | } 30 | ] 31 | } 32 | -------------------------------------------------------------------------------- /Container-Root/app/server.py: -------------------------------------------------------------------------------- 1 | ###################################################################### 2 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 3 | # SPDX-License-Identifier: MIT-0 # 4 | ###################################################################### 5 | 6 | # Simple REST API 7 | from fastapi import FastAPI 8 | import logging 9 | 10 | global logger 11 | 12 | logger = logging.getLogger() 13 | 14 | app = FastAPI() 15 | 16 | @app.get("/") 17 | async def report_health(): 18 | logger.warning("Received healthcheck request") 19 | r = {"Status": "Healthy"} 20 | logger.warning("Returning " + str(r)) 21 | return r 22 | 23 | 24 | @app.get("/say") 25 | async def say_something(text: str = "Hello"): 26 | logger.warning("Received say request: " + text) 27 | r = {"Message": text} 28 | logger.warning("Returning " + str(r)) 29 | return r 30 | 31 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | -------------------------------------------------------------------------------- /to/kubernetes/template/040-hpa.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: autoscaling/v2beta2 2 | kind: HorizontalPodAutoscaler 3 | metadata: 4 | name: ${APP_NAME} 5 | namespace: ${NAMESPACE} 6 | labels: 7 | app: ${APP_NAME} 8 | spec: 9 | scaleTargetRef: 10 | apiVersion: apps/v1 11 | kind: Deployment 12 | name: ${APP_NAME} 13 | minReplicas: 1 14 | maxReplicas: 20 15 | metrics: 16 | - type: Resource 17 | resource: 18 | name: cpu 19 | target: 20 | type: Utilization 21 | averageUtilization: 50 22 | # - type: Object 23 | # object: 24 | # metric: 25 | # name: requests-per-second 26 | # describedObject: 27 | # apiVersion: networking.k8s.io/v1beta1 28 | # kind: Service 29 | # name: ${APP_NAME} 30 | # target: 31 | # type: Value 32 | # value: 10 33 | # - type: Pods 34 | # pods: 35 | # metric: 36 | # name: packets-per-second 37 | # target: 38 | # type: AverageValue 39 | # averageValue: 1k 40 | -------------------------------------------------------------------------------- /install/install-docker-compose.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -e .env ]; then 9 | . .env 10 | elif [ -e ../.env ]; then 11 | pushd .. 12 | . .env 13 | popd 14 | fi 15 | 16 | if [ "$OPERATING_SYSTEM" == "Linux" ]; then 17 | echo "Installing docker-compose on Linux ..." 18 | sudo curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose 19 | sudo chmod +x /usr/local/bin/docker-compose 20 | docker-compose --version 21 | elif [ "$OPERATING_SYSTEM" == "MacOS" ]; then 22 | echo "Docker-compose on MacOS is included in Docker Desktop. Please refer to https://docs.docker.com/docker-for-mac/install" 23 | else 24 | echo "Unrecognized OS $OPERATING_SYSTEM" 25 | fi 26 | -------------------------------------------------------------------------------- /install/install-aws-cli.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -e .env ]; then 9 | . .env 10 | elif [ -e ../.env ]; then 11 | pushd .. 12 | . .env 13 | popd 14 | fi 15 | 16 | if [ "$OPERATING_SYSTEM" == "Linux" ]; then 17 | echo "Installing aws-cli on Linux ..." 18 | pushd /tmp 19 | curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" 20 | unzip awscliv2.zip 21 | sudo ./aws/install 22 | rm -f awscliv2.zip 23 | rm -rf ./aws 24 | popd 25 | which aws 26 | aws --version 27 | elif [ "$OPERATING_SYSTEM" == "MacOS" ]; then 28 | echo "Installing aws-cli on MacOS ..." 29 | curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg" 30 | sudo installer -pkg AWSCLIV2.pkg -target / 31 | which aws 32 | aws --version 33 | else 34 | echo "Unrecognized OS $OPERATING_SYSTEM" 35 | fi 36 | 37 | 38 | -------------------------------------------------------------------------------- /to/kubernetes/template/020-deployment.yaml: -------------------------------------------------------------------------------- 1 | kind: Deployment 2 | apiVersion: apps/v1 3 | metadata: 4 | name: ${APP_NAME} 5 | namespace: ${NAMESPACE} 6 | labels: 7 | app: ${APP_NAME} 8 | spec: 9 | replicas: 1 10 | selector: 11 | matchLabels: 12 | app: ${APP_NAME} 13 | template: 14 | metadata: 15 | labels: 16 | app: ${APP_NAME} 17 | spec: 18 | #nodeSelector: 19 | # beta.kubernetes.io/instance-type: "${INSTANCE_TYPE}" 20 | containers: 21 | - name: main 22 | image: "${REGISTRY}${IMAGE}${TAG}" 23 | imagePullPolicy: Always 24 | #command: ["${command}"] 25 | #args: ["${args}"] 26 | env: 27 | - name: PORT_INTERNAL 28 | value: "${PORT_INTERNAL}" 29 | ports: 30 | - name: pod-port 31 | containerPort: ${PORT_INTERNAL} 32 | #resources: 33 | # limits: 34 | # cpu: 4 35 | # memory: 4Gi 36 | # nvidia.com/gpu: 1 37 | # k8s.amazonaws.com/vgpu: 5 38 | # requests: 39 | # cpu: "1" 40 | # memory: 1Gi 41 | #volumeMounts: 42 | #- name: nvidia-mps 43 | # mountPath: /tmp/nvidia-mps 44 | #volumes: 45 | #- name: nvidia-mps 46 | # hostPath: 47 | # path: /tmp/nvidia-mps 48 | -------------------------------------------------------------------------------- /install/install-ecs-cli.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -e .env ]; then 9 | . .env 10 | elif [ -e ../.env ]; then 11 | pushd .. 12 | . .env 13 | popd 14 | fi 15 | 16 | if [ "$OPERATING_SYSTEM" == "Linux" ]; then 17 | echo "Installing ecs-cli on Linux ..." 18 | sudo curl -Lo /usr/local/bin/ecs-cli https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-linux-amd64-latest 19 | gpg --import ./gpg-public-key 20 | curl -Lo ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-linux-amd64-latest.asc 21 | gpg --verify ecs-cli.asc /usr/local/bin/ecs-cli 22 | sudo chmod +x /usr/local/bin/ecs-cli 23 | ecs-cli --version 24 | elif [ "$OPERATING_SYSTEM" == "MacOS" ]; then 25 | echo "Installing ecs-cli on MacOS ..." 26 | sudo curl -Lo /usr/local/bin/ecs-cli https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-darwin-amd64-latest 27 | brew install gnupg 28 | gpg --import ./gpg-public-key 29 | curl -Lo ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-darwin-amd64-latest.asc 30 | gpg --verify ecs-cli.asc /usr/local/bin/ecs-cli 31 | sudo chmod +x /usr/local/bin/ecs-cli 32 | ecs-cli --version 33 | else 34 | echo "Unrecognized OS $OPERATING_SYSTEM" 35 | fi 36 | 37 | 38 | -------------------------------------------------------------------------------- /login.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ "${DEBUG}" == "true" ]; then 9 | set -x 10 | fi 11 | 12 | print_help() { 13 | echo "" 14 | echo "Usage: $0" 15 | echo "" 16 | echo " This script assists with logging in to a private container registry." 17 | echo " By default we use Amazon ECR, however the script can be extended to support other registries as needed." 18 | echo " In order to login successfully, the environment in which this script is running, must be configured" 19 | echo " with an IAM role allowing access to ECR in the target AWS account." 20 | echo "" 21 | } 22 | 23 | if [ "$1" == "" ]; then 24 | 25 | . ./.env 26 | 27 | # Login to container registry 28 | case "$REGISTRY_TYPE" in 29 | "ecr") 30 | echo "Logging in to $REGISTRY_TYPE $REGISTRY ..." 31 | CMD="aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $REGISTRY" 32 | if [ "${VERBOSE}" == "true" ]; then 33 | echo "${CMD}" 34 | fi 35 | if [ "${DRY_RUN}" == "false" ]; then 36 | eval "${CMD}" 37 | fi 38 | ;; 39 | *) 40 | echo "Automatic login for REGISTRY_TYPE=$REGISTRY_TYPE is not implemented. Please log in manually." 41 | ;; 42 | esac 43 | else 44 | print_help 45 | fi 46 | 47 | if [ "${DEBUG}" == "true" ]; then 48 | set +x 49 | fi 50 | -------------------------------------------------------------------------------- /Dockerfile-lambda: -------------------------------------------------------------------------------- 1 | ARG BASE_IMAGE_PATH="python:3.9" 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Multi-stage build: compile 9 | FROM $BASE_IMAGE_PATH as build-image 10 | 11 | # Install aws-lambda-cpp build dependencies 12 | RUN apt-get update && \ 13 | apt-get install -y \ 14 | g++ \ 15 | make \ 16 | cmake \ 17 | unzip \ 18 | libcurl4-openssl-dev 19 | 20 | # Install aws-mapbda-rie Runtime Interface Emulator for testing 21 | RUN curl -Lo /tmp/aws-lambda-rie \ 22 | https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie \ 23 | && chmod +x /tmp/aws-lambda-rie && mv /tmp/aws-lambda-rie /usr/local/bin 24 | 25 | # Create function directory 26 | RUN mkdir -p /function 27 | 28 | # Copy function code 29 | COPY Container-Root/function/* /function/ 30 | 31 | # Install the runtime interface client 32 | RUN pip install --target /function awslambdaric 33 | 34 | # Multi-stage build: runtime container 35 | FROM $BASE_IMAGE_PATH as runtime-image 36 | ARG BUILD="202105" 37 | ARG MAINTAINER="Full Name " 38 | ARG DESCRIPTION="Depend on Docker Image" 39 | ARG http_proxy 40 | ARG https_proxy 41 | ARG no_proxy 42 | 43 | LABEL MAINTAINER="$MAINTAINER" 44 | 45 | LABEL DESCRIPTION="$DESCRIPTION" 46 | 47 | LABEL BUILD="$BUILD" 48 | 49 | COPY --from=build-image /function /function 50 | RUN mv /function/test*.sh / 51 | COPY --from=build-image /usr/local/bin/aws-lambda-rie /usr/local/bin/aws-lambda-rie 52 | 53 | WORKDIR /function 54 | ENTRYPOINT [ "/function/startup.sh" ] 55 | CMD [ "lambda_function.handler" ] 56 | 57 | -------------------------------------------------------------------------------- /install/install-ssm-plugin.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | if [ -e .env ]; then 9 | . .env 10 | elif [ -e ../.env ]; then 11 | pushd .. 12 | . .env 13 | popd 14 | fi 15 | 16 | echo "" 17 | echo "Installing SSM plugin on $OPERATING_SYSTEM ..." 18 | echo "" 19 | 20 | if [ "$OPERATING_SYSTEM" == "Linux" ]; then 21 | command -v apt-get &> /dev/null 22 | APT_COMMAND=$? 23 | command -v yum &> /dev/null 24 | YUM_COMMAND=$? 25 | if [ "${APT_COMMAND}" == "0" ]; then 26 | curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb" -o "session-manager-plugin.deb" 27 | sudo dpkg -i session-manager-plugin.deb 28 | #rm -f ./session-manager-plugin.deb 29 | elif [ "${YUM_COMMAND}" == "0" ]; then 30 | curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm" -o "session-manager-plugin.rpm" 31 | sudo yum install -y session-manager-plugin.rpm 32 | #rm -f ./session-manager-plugin.rpm 33 | else 34 | echo "Unable to determine proper session manager plugin package for your OS" 35 | fi 36 | elif [ "$OPERATING_SYSTEM" == "MacOS" ]; then 37 | curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/mac/sessionmanager-bundle.zip" -o "sessionmanager-bundle.zip" 38 | unzip sessionmanager-bundle.zip 39 | sudo ./sessionmanager-bundle/install -i /usr/local/sessionmanagerplugin -b /usr/local/bin/session-manager-plugin 40 | rm -f ./sessionmanager-bundle.zip 41 | else 42 | echo "Operating system $OPERATING_SYSTEM not supported" 43 | fi 44 | 45 | 46 | -------------------------------------------------------------------------------- /config.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | . ./.env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | function usage 15 | { 16 | echo "" 17 | echo "Usage: $0 [setting] [value]" 18 | echo "" 19 | echo " $0 --help - display usage information" 20 | echo " $0 -i - edit selected settings interactively" 21 | echo " $0 - edit configuration file" 22 | echo " $0 setting - edit specified setting interactively" 23 | echo " $0 setting value - set the specified setting to the provided value" 24 | echo "" 25 | } 26 | 27 | if [ "$1" == "--help" ]; then 28 | usage 29 | CMD="" 30 | elif [ "$1" == "-i" ]; then 31 | ./config.sh TO 32 | ./config.sh BASE_IMAGE_PATH 33 | ./config.sh REGISTRY 34 | ./config.sh IMAGE_NAME 35 | ./config.sh VERSION 36 | elif [ "$1" == "" ]; then 37 | CMD="${CMD_EDIT} .env" 38 | else 39 | if [ "$2" == "" ]; then 40 | # Interactive 41 | echo "" 42 | cat ./.env | grep "^export $1=" -B 10 43 | echo "" 44 | CURRENT_VALUE=$(printenv $1) 45 | echo "Set $1 [ ${CURRENT_VALUE} ] to: " 46 | read NEW_VALUE 47 | if [ "${NEW_VALUE}" == "" ]; then 48 | NEW_VALUE="${CURRENT_VALUE}" 49 | fi 50 | else 51 | # Set value to $2 52 | NEW_VALUE="$2" 53 | fi 54 | CMD="cp -f ./.env ./.env.bak; cat ./.env | sed \"s#^export $1=.*#export $1=${NEW_VALUE}#\" | tee ./.env > /dev/null; echo ''; echo New value:; cat ./.env | grep \"^export $1=\"" 55 | fi 56 | 57 | 58 | if [ "${VERBOSE}" == "true" ]; then 59 | echo "" 60 | echo "${CMD}" 61 | fi 62 | 63 | if [ "${DRY_RUN}" == "false" ]; then 64 | eval "${CMD}" 65 | echo "" 66 | fi 67 | 68 | if [ "${DEBUG}" == "true" ]; then 69 | set +x 70 | fi 71 | 72 | -------------------------------------------------------------------------------- /test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | export MODE=-it 11 | 12 | echo "" 13 | echo "Testing container ${CONTAINER} on ${TO} ..." 14 | 15 | case ${TO} in 16 | "lambda") 17 | echo "Testing lambda function ${LAMBDA_FUNCTION_NAME} ..." 18 | for t in $(ls ${LAMBDA_TEST_PATH}/test*.sh); do echo Running test $t; $t; done 19 | ;; 20 | "compose") 21 | if [ "${COMPOSE_CONTEXT_TYPE}" == "moby" ]; then 22 | CONTAINER_INDEX=$1 23 | if [ "$CONTAINER_INDEX" == "" ]; then 24 | CONTAINER_INDEX=1 25 | fi 26 | docker exec ${MODE} ${COMPOSE_PROJECT_NAME}_${CONTAINER}_${CONTAINER_INDEX} sh -c "for t in \$(ls /test*.sh); do echo Running test \$t; \$t; done;" 27 | elif [ "${COMPOSE_CONTEXT_TYPE}" == "ecs" ]; then 28 | endpoint_host=$(./status.sh | grep ${CONTAINER} | grep Running | awk '{print $4}' | cut -d ':' -f 1) 29 | endpoint="http://${endpoint_host}:${PORT_EXTERNAL}" 30 | for t in $(ls ${COMPOSE_TEST_PATH}/test*.sh); do echo Running test $t; $t $endpoint; done 31 | else 32 | echo "" 33 | echo "Unrecognized COMPOSE_CONTEXT_TYPE ${COMPOSE_CONTEXT_TYPE}" 34 | echo "Supported context types: moby | ecs" 35 | fi 36 | ;; 37 | "ecs") 38 | endpoint_host=$(./status.sh | grep ${CONTAINER} | grep RUNNING | awk '{print $3}' | cut -d ':' -f 1) 39 | endpoint="http://${endpoint_host}:${PORT_EXTERNAL}" 40 | for t in $(ls ${COMPOSE_TEST_PATH}/test*.sh); do echo Running test $t; $t $endpoint; done 41 | ;; 42 | "kubernetes") 43 | CONTAINER_INDEX=$1 44 | if [ "$CONTAINER_INDEX" == "" ]; then 45 | CONTAINER_INDEX=1 46 | fi 47 | unset DEBUG; ${KUBECTL} -n ${NAMESPACE} exec -it $( ${KUBECTL} -n ${NAMESPACE} get pod | grep ${APP_NAME} | head -n ${CONTAINER_INDEX} | tail -n 1 | cut -d ' ' -f 1 ) -- bash -c "for t in \$(ls /test*.sh); do echo Running test \$t; \$t; done" 48 | ;; 49 | "batchlocal") 50 | BATCH_COMMAND="cd /job; for t in \$(ls test*.sh); do echo Running test \$t; ./\$t; done" 51 | echo "${BATCH_COMMAND}" 52 | docker container run ${RUN_OPTS} ${CONTAINER_NAME} -d ${NETWORK} ${VOL_MAP} ${REGISTRY}${IMAGE}${TAG} bash -c "${BATCH_COMMAND}" 53 | ./logs.sh 54 | ./stop.sh 55 | ;; 56 | "batch") 57 | BATCH_COMMAND="cd /job; ./test1.sh; ./test2.sh" 58 | ./run.sh "[\"/bin/bash\",\"-c\",\"${BATCH_COMMAND}\"]" 59 | ;; 60 | *) 61 | docker exec ${MODE} ${CONTAINER} sh -c "for t in \$(ls /test*.sh); do echo Running test \$t; \$t; done;" 62 | ;; 63 | esac 64 | 65 | 66 | -------------------------------------------------------------------------------- /status.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | echo "" 15 | echo "Showing status of container ${CONTAINER} on ${TO} ..." 16 | 17 | case "${TO}" in 18 | "compose") 19 | CMD="${DOCKER_COMPOSE} -f ${COMPOSE_FILE} ps -a" 20 | ;; 21 | "swarm") 22 | CMD="docker stack ps ${SWARM_STACK_NAME}" 23 | ;; 24 | "ecs") 25 | COMPOSE_FILE=${ECS_COMPOSE_FILE} 26 | CMD="${ECS_CLI} ps" 27 | if [ "${VERBOSE}" == "true" ]; then 28 | echo "" 29 | echo "${CMD}" 30 | fi 31 | if [ "${DRY_RUN}" == "false" ]; then 32 | eval "${CMD}" 33 | fi 34 | CMD="aws ecs describe-tasks --cluster ${ECS_CLUSTER} --query 'tasks[*].{TaskArn:taskArn,State:lastStatus,Health:healthStatus,LaunchType:launchType,PrivateIP:containers[0].networkInterfaces[0].privateIpv4Address}' --output table --tasks $(aws ecs list-tasks --cluster ${ECS_CLUSTER} --query 'taskArns' --output text)" 35 | ;; 36 | "kubernetes") 37 | CMD="${KUBECTL} -n ${NAMESPACE} get all" 38 | ;; 39 | "lambda") 40 | CMD="aws lambda get-function --function-name ${LAMBDA_FUNCTION_NAME} --query '{Name:Configuration.FunctionName,State:Configuration.State,Status:Configuration.LastUpdateStatus,Updated:Configuration.LastModified,Image:Code.ImageUri}' --output table" 41 | ;; 42 | "batch") 43 | if [ "$1" == "" ]; then 44 | STATUS_LIST=(SUBMITTED PENDING RUNNABLE STARTING RUNNING SUCCEEDED FAILED) 45 | for STATUS in ${STATUS_LIST[@]}; do 46 | CMD="aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status ${STATUS} --query 'jobSummaryList[*].{createdAt:createdAt,jobId:jobId,jobName:jobName,status:status,statusReason:statusReason,exitCode:container.exitCode}' --output table" 47 | if [ "${VERBOSE}" == "true" ]; then 48 | echo "" 49 | echo "${CMD}" 50 | fi 51 | if [ "${DRY_RUN}" == "false" ]; then 52 | eval "${CMD}" 53 | fi 54 | done 55 | CMD="" 56 | else 57 | CMD="aws batch describe-jobs --jobs $@ --query 'jobs[*].{createdAt:createdAt,jobId:jobId,jobName:jobName,status:status,statusReason:statusReason,exitCode:container.exitCode,platformCapabilities:platformCapabilities[0]}' --output table" 58 | fi 59 | ;; 60 | *) 61 | checkTO "${TO}" 62 | CMD="docker ps -a | grep ${CONTAINER}" 63 | ;; 64 | esac 65 | 66 | if [ "${VERBOSE}" == "true" ]; then 67 | echo "" 68 | echo "${CMD}" 69 | fi 70 | 71 | if [ "${DRY_RUN}" == "false" ]; then 72 | eval "${CMD}" 73 | fi 74 | 75 | if [ "${DEBUG}" == "true" ]; then 76 | set +x 77 | fi 78 | 79 | -------------------------------------------------------------------------------- /.fun: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Helper functions 9 | ## Detect current operating system 10 | function os 11 | { 12 | UNAME=$(uname -a) 13 | if [ $(echo $UNAME | awk '{print $1}') == "Darwin" ]; then 14 | export OPERATING_SYSTEM="MacOS" 15 | elif [ $(echo $UNAME | awk '{print $1}') == "Linux" ]; then 16 | export OPERATING_SYSTEM="Linux" 17 | elif [ ${UNAME:0:5} == "MINGW" ]; then 18 | export OPERATING_SYSTEM="Windows" 19 | export MSYS_NO_PATHCONV=1 # turn off path conversion 20 | else 21 | export OPERATING_SYSTEM="Other" 22 | fi 23 | } 24 | ## End os function 25 | os 26 | 27 | ## Determine current host IP address 28 | function hostip 29 | { 30 | case "${OPERATING_SYSTEM}" in 31 | "Linux") 32 | export HOST_IP=$(hostname -I | tr " " "\n" | head -1) # Linux 33 | ;; 34 | "MacOS") 35 | export HOST_IP=$(ifconfig | grep -v 127.0.0.1 | grep -v inet6 | grep inet | head -n 1 | awk '{print $2}') # Mac OS 36 | ;; 37 | "Windows") 38 | export HOST_IP=$( ((ipconfig | grep IPv4 | grep 10.187 | tail -1) && (ipconfig | grep IPv4 | grep 3. | head -1)) | tail -1 | awk '{print $14}' ) # Git bash 39 | ;; 40 | *) 41 | export HOST_IP=$(hostname) 42 | ;; 43 | esac 44 | } 45 | ## End hostip function 46 | hostip 47 | 48 | ## Check default target orchestrator 49 | function checkTO 50 | { 51 | KNOWN_TARGET_ORCHESTRATORS="docker compose ecs swarm kubernetes lambdalocal lambda batchlocal batch" 52 | echo "${KNOWN_TARGET_ORCHESTRATORS}" | grep $1 > /dev/null 53 | if [ "$?" == "1" ]; then 54 | echo "" 55 | echo "Supported target orchestrators:" 56 | echo "${KNOWN_TARGET_ORCHESTRATORS}" 57 | echo "WARNING: Unrecognized target orchestrator TO=$1. Defaulting to docker." 58 | echo "" 59 | fi 60 | } 61 | 62 | ## Return first available port starting with arg, return 0 if no port is available 63 | function firstAvailable() 64 | { 65 | if [ "$1" == "" ]; then 66 | PORT_DESIRED=80 67 | else 68 | PORT_DESIRED=$1 69 | fi 70 | PORT_AVAILABLE=${PORT_DESIRED} 71 | LOCAL_ORCHESTRATORS="docker compose lambdalocal batchlocal" 72 | TO_IS_LOCAL=$(echo ${LOCAL_ORCHESTRATORS} | grep -w -q ${TO}; echo $?) 73 | if [ "${TO_IS_LOCAL}" == "0" ]; then 74 | TEST_RESULT=$(timeout 1 bash -c "/dev/null 2>/dev/null; echo $?) 75 | while [ "${TEST_RESULT}" == "0" ]; do 76 | PORT_AVAILABLE=$((PORT_AVAILABLE+1)) 77 | if [ "${PORT_AVAILABLE}" -ge "65535" ]; then 78 | PORT_AVAILABLE="0" 79 | break 80 | fi 81 | TEST_RESULT=$(timeout 1 bash -c "/dev/null 2>/dev/null; echo $?) 82 | done 83 | fi 84 | echo "${PORT_AVAILABLE}" 85 | } 86 | -------------------------------------------------------------------------------- /logs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | echo "" 15 | echo "Showing logs from container ${CONTAINER} on ${TO} ..." 16 | 17 | case "${TO}" in 18 | "compose") 19 | if [ "$1" == "" ]; then 20 | CMD="${DOCKER_COMPOSE} -f ${COMPOSE_FILE} logs -f" 21 | else 22 | CMD="docker logs -f ${COMPOSE_PROJECT_NAME}_${CONTAINER}_$1" 23 | fi 24 | ;; 25 | "swarm") 26 | if [ "$1" == "" ]; then 27 | CMD="docker service logs -f ${SWARM_STACK_NAME}_${SWARM_SERVICE_NAME}" 28 | else 29 | CMD="docker service ps ${SWARM_STACK_NAME}_${SWARM_SERVICE_NAME} | grep ${SWARM_SERVICE_NAME}.$1 | cut -d ' ' -f 1 | xargs docker service logs -f" 30 | fi 31 | ;; 32 | "ecs") 33 | CONTAINER_INDEX=1 34 | if [ ! "$1" == "" ]; then 35 | CONTAINER_INDEX=$1 36 | fi 37 | CMD="${ECS_CLI} logs --task-id $(./status.sh | grep RUNNING | head -n ${CONTAINER_INDEX} | tail -n 1 | cut -d '/' -f 2) --follow" 38 | ;; 39 | "kubernetes") 40 | CMD="${KUBETAIL} ${APP_NAME} -n ${NAMESPACE} -s 30m" 41 | ;; 42 | "lambda") 43 | since="--since 1h" 44 | if [ ! "$1" == "" ]; then 45 | since="--since $1" 46 | fi 47 | CMD="aws logs tail ${since} --follow /aws/lambda/${LAMBDA_FUNCTION_NAME}" 48 | ;; 49 | "batch") 50 | if [ "$1" == "" ]; then 51 | JOB_IDS_RUNNING=$(aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status RUNNING --query 'jobSummaryList[*].jobId' --output text) 52 | JOB_IDS_SUCCEEDED=$(aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status SUCCEEDED --query 'jobSummaryList[*].jobId' --output text) 53 | JOB_IDS_FAILED=$(aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status FAILED --query 'jobSummaryLists[*].jobId' --output text) 54 | JOB_IDS="$JOB_IDS_RUNNING $JOB_IDS_SUCCEEDED $JOB_IDS_FAILED" 55 | else 56 | JOB_IDS="$@" 57 | fi 58 | for JOB_ID in ${JOB_IDS}; do 59 | if [ ! "$JOB_ID" == "None" ]; then 60 | echo "" 61 | echo "Getting log evewnts for job ${JOB_ID} ..." 62 | LOG_STREAM=$(aws batch describe-jobs --jobs ${JOB_ID} --query 'jobs[*].container.logStreamName' --output text) 63 | CMD="aws logs get-log-events --log-group-name /aws/batch/job --no-paginate --query 'events[*].{timestamp:timestamp,message:message}' --output text --log-stream-name ${LOG_STREAM}" 64 | if [ "${VERBOSE}" == "true" ]; then 65 | echo "${CMD}" 66 | fi 67 | if [ "${DRY_RUN}" == "false" ]; then 68 | eval "${CMD}" 69 | fi 70 | fi 71 | done 72 | CMD="" 73 | ;; 74 | *) 75 | checkTO "${TO}" 76 | CMD="docker container logs -f ${CONTAINER}" 77 | ;; 78 | esac 79 | 80 | if [ "${VERBOSE}" == "true" ]; then 81 | echo "${CMD}" 82 | fi 83 | 84 | if [ "${DRY_RUN}" == "false" ]; then 85 | eval "${CMD}" 86 | fi 87 | 88 | if [ "${DEBUG}" == "true" ]; then 89 | set +x 90 | fi 91 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /exec.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | echo "" 15 | echo "Executing command in container ${CONTAINER} on ${TO} ..." 16 | 17 | case "${TO}" in 18 | "compose") 19 | if [ "${COMPOSE_CONTEXT_TYPE}" == "moby" ]; then 20 | CONTAINER_INDEX=$1 21 | if [ "$CONTAINER_INDEX" == "" ]; then 22 | CONTAINER_INDEX=1 23 | fi 24 | CMD="docker exec -it ${COMPOSE_PROJECT_NAME}_${CONTAINER}_${CONTAINER_INDEX} sh -c 'if [ -e /bin/bash ]; then /bin/bash; else sh; fi'" 25 | elif [ "${COMPOSE_CONTEXT_TYPE}" == "ecs" ]; then 26 | echo "" 27 | echo "Exec into container running on ECS is not supported" 28 | echo "" 29 | else 30 | echo "" 31 | echo "Unrecognized COMPOSE_CONTEXT_TYPE ${COMPOSE_CONTEXT_TYPE}" 32 | echo "Supported context types are moby | ecs" 33 | fi 34 | ;; 35 | "swarm") 36 | SERVICE_INDEX=$1 37 | if [ "$SERVICE_INDEX" == "" ]; then 38 | SERVICE_INDEX=1 39 | fi 40 | TASK_ID=${SWARM_STACK_NAME}_${SWARM_SERVICE_NAME}.${SERVICE_INDEX} 41 | CONTAINER_ID=$(docker ps | grep ${TASK_ID} | cut -d ' ' -f 1) 42 | if [ "${CONTAINER_ID}" == "" ]; then 43 | CONTAINER_HOST=$(docker service ps ${SWARM_STACK_NAME}_${SWARM_SERVICE_NAME} -f desired-state=running --format="{{.Node}} {{.Name }} {{.ID}}" | grep ${TASK_ID} | cut -d ' ' -f 1) 44 | echo "" 45 | if [ "${CONTAINER_HOST}" == "" ]; then 46 | echo "Could not find running task ${TASK_ID}" 47 | else 48 | echo "Cannot connect to service task ${TASK_ID}" 49 | echo "Only connections to tasks running on the local host are supported" 50 | echo "You may connect to the node running this task ($CONTAINER_HOST) and use docker exec locally" 51 | echo " or configure remote docker daemon access." 52 | fi 53 | echo "" 54 | CMD="" 55 | else 56 | CMD="docker exec -it ${CONTAINER_ID} sh -c 'if [ -e /bin/bash ]; then /bin/bash; else sh; fi'" 57 | fi 58 | ;; 59 | "ecs") 60 | CONTAINER_INDEX=1 61 | if [ ! "$1" == "" ]; then 62 | CONTAINER_INDEX=$1 63 | fi 64 | TASK_ID=$(./status.sh | grep RUNNING | head -n ${CONTAINER_INDEX} | tail -n 1 | cut -d '/' -f 2) 65 | CMD="aws ecs execute-command --region ${REGION} --cluster ${ECS_CLUSTER} --command /bin/bash --task ${TASK_ID} --container ${CONTAINER} --interactive" 66 | ;; 67 | "kubernetes") 68 | CONTAINER_INDEX=$1 69 | if [ "$CONTAINER_INDEX" == "" ]; then 70 | CONTAINER_INDEX=1 71 | fi 72 | CMD="unset DEBUG; ${KUBECTL} -n ${NAMESPACE} exec -it $( ${KUBECTL} -n ${NAMESPACE} get pod | grep ${APP_NAME} | head -n ${CONTAINER_INDEX} | tail -n 1 | cut -d ' ' -f 1 ) -- sh -c 'if [ -e /bin/bash ]; then /bin/bash; else sh; fi'" 73 | ;; 74 | "lambda") 75 | LAMBDA_PAYLOAD="" 76 | if [ ! "$1" == "" ]; then 77 | LAMBDA_PAYLOAD="--payload '$1'" 78 | fi 79 | CMD="aws lambda invoke --function-name ${LAMBDA_FUNCTION_NAME} ${LAMBDA_PAYLOAD} /tmp/${LAMBDA_FUNCTION_NAME}.out; cat /tmp/${LAMBDA_FUNCTION_NAME}.out" 80 | ;; 81 | "batch") 82 | echo "" 83 | echo "exec operation is not supported for target orchestrator batch" 84 | ;; 85 | *) 86 | checkTO "${TO}" 87 | if [ "$1" == "" ]; then 88 | CMD="sh -c 'if [ -e /bin/bash ]; then /bin/bash; else sh; fi'" 89 | else 90 | CMD="$@" 91 | fi 92 | CMD="docker container exec -it ${CONTAINER} $CMD" 93 | ;; 94 | esac 95 | 96 | if [ "${VERBOSE}" == "true" ]; then 97 | echo "${CMD}" 98 | fi 99 | 100 | if [ "${DRY_RUN}" == "false" ]; then 101 | eval "${CMD}" 102 | fi 103 | 104 | if [ "${DEBUG}" == "true" ]; then 105 | set +x 106 | fi 107 | 108 | -------------------------------------------------------------------------------- /stop.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | echo "" 15 | echo "Stopping container ${CONTAINER} on ${TO} ..." 16 | 17 | case "${TO}" in 18 | "compose") 19 | CMD="${DOCKER_COMPOSE} -f ${COMPOSE_FILE} down" 20 | ;; 21 | "swarm") 22 | CMD="docker stack rm ${SWARM_STACK_NAME}" 23 | ;; 24 | "ecs") 25 | COMPOSE_FILE=${ECS_COMPOSE_FILE} 26 | TASK_ARNS=$(aws ecs list-tasks --cluster ${ECS_CLUSTER} --query 'taskArns' --output text) 27 | for task in $TASK_ARNS; do 28 | echo "Stopping task $task ..." 29 | CMD="aws ecs stop-task --cluster ${ECS_CLUSTER} --task $task" 30 | if [ "${VERBOSE}" == "true" ]; then 31 | echo "${CMD}" 32 | fi 33 | if [ "${DRY_RUN}" == "false" ]; then 34 | RESULT=$(eval "${CMD}") 35 | fi 36 | TASK_DEFINITION=$(aws ecs describe-tasks --cluster ${ECS_CLUSTER} --tasks $task --query 'tasks[].taskDefinitionArn' --output text) 37 | CMD="aws ecs deregister-task-definition --task-definition ${TASK_DEFINITION}" 38 | if [ "${VERBOSE}" == "true" ]; then 39 | echo "${CMD}" 40 | fi 41 | if [ "${DRY_RUN}" == "false" ]; then 42 | RESULT=$(eval "${CMD}") 43 | fi 44 | done 45 | if [ "${ECS_MANAGE_CLUSTER}" == "true" ]; then 46 | CMD="${ECS_CLI} down --force" 47 | else 48 | COMPOSE_FILE=${ECS_COMPOSE_FILE} 49 | CMD="${ECS_CLI} compose --file ${COMPOSE_FILE} --cluster ${ECS_CLUSTER} down" 50 | fi 51 | ;; 52 | "kubernetes") 53 | CMD="${KUBECTL} delete -f ${KUBERNETES_APP_PATH}" 54 | ;; 55 | "lambda") 56 | CMD="aws lambda delete-function --function-name ${LAMBDA_FUNCTION_NAME}" 57 | ;; 58 | "batch") 59 | echo "" 60 | echo "Stopping job ${BATCH_JOB_NAME} ..." 61 | # Cancel SUBMITTED, PENDING or RUNNABLE jobs 62 | for STATUS in "SUBMITTED" "PENDING" "RUNNABLE"; do 63 | JOB_IDS=$(aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status ${STATUS} --query 'jobSummaryList[*].jobId' --output text) 64 | if [ ! "${JOB_IDS}" == "" ]; then 65 | for JOB_ID in "${JOB_IDS}"; do 66 | echo "Cancelling job ${JOB_ID} ..." 67 | CMD="aws batch cancel-job --job-id ${JOB_ID} --reason 'Stopped by user'" 68 | if [ "${VERBOSE}" == "true" ]; then 69 | echo "${CMD}" 70 | fi 71 | if [ "${DRY_RUN}" == "false" ]; then 72 | RESULT=$(eval "${CMD}") 73 | fi 74 | done 75 | fi 76 | done 77 | # Terminate STARTING or RUNNING jobs 78 | for STATUS in "STARTING" "RUNNING"; do 79 | JOB_IDS=$(aws batch list-jobs --job-queue ${BATCH_JOB_QUEUE_NAME} --job-status ${STATUS} --query 'jobSummaryList[*].jobId' --output text) 80 | if [ ! "${JOB_IDS}" == "" ]; then 81 | for JOB_ID in "${JOB_IDS}"; do 82 | echo "Terminating job ${JOB_ID} ..." 83 | CMD="aws batch terminate-job --job-id ${JOB_ID} --reason 'Stopped by user'" 84 | if [ "${VERBOSE}" == "true" ]; then 85 | echo "${CMD}" 86 | fi 87 | if [ "${DRY_RUN}" == "false" ]; then 88 | RESULT=$(eval "${CMD}") 89 | fi 90 | done 91 | fi 92 | done 93 | # Deregister job definitions 94 | REVISIONS=$(aws batch describe-job-definitions --job-definition-name ${BATCH_JOB_DEFINITION_NAME} --query 'jobDefinitions[?status==`ACTIVE`].revision' --output text) 95 | for REVISION in ${REVISIONS}; do 96 | echo "Deregistering job definition ${BATCH_JOB_DEFINITION_NAME}:${REVISION} ..." 97 | CMD="aws batch deregister-job-definition --job-definition ${BATCH_JOB_DEFINITION_NAME}:${REVISION}" 98 | if [ "${VERBOSE}" == "true" ]; then 99 | echo "${CMD}" 100 | fi 101 | if [ "${DRY_RUN}" == "false" ]; then 102 | RESULT=$(eval "${CMD}") 103 | fi 104 | done 105 | if [ "${BATCH_MANAGE_COMPUTE_ENVIRONMENT}" == "true" ]; then 106 | JOB_QUEUE=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].jobQueueName' --output text) 107 | if [ "${JOB_QUEUE}" == "${BATCH_JOB_QUEUE_NAME}" ]; then 108 | echo "Deleting job queue ${BATCH_JOB_QUEUE_NAME} ..." 109 | CMD="aws batch update-job-queue --job-queue ${BATCH_JOB_QUEUE_NAME} --state DISABLED" 110 | if [ "${VERBOSE}" == "true" ]; then 111 | echo "${CMD}" 112 | fi 113 | if [ "${DRY_RUN}" == "false" ]; then 114 | RESULT=$(eval "${CMD}") 115 | fi 116 | sleep 2 117 | STATE=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].state' --output text) 118 | while [ ! "${STATE}" == "DISABLED" ]; do 119 | echo "Waiting for ${BATCH_JOB_QUEUE_NAME} state to change to DISABLED ..." 120 | sleep 2 121 | STATE=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].state' --output text) 122 | done 123 | CMD="aws batch delete-job-queue --job-queue ${BATCH_JOB_QUEUE_NAME}" 124 | if [ "${VERBOSE}" == "true" ]; then 125 | echo "${CMD}" 126 | fi 127 | if [ "${DRY_RUN}" == "false" ]; then 128 | RESULT=$(eval "${CMD}") 129 | fi 130 | # Wait for job queue to be deleted 131 | JOB_QUEUE=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].jobQueueName' --output text) 132 | while [ ! "${JOB_QUEUE}" == "" ]; do 133 | echo "Waiting for job queue ${JOB_QUEUE} to be deleted ..." 134 | sleep 2 135 | JOB_QUEUE=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].jobQueueName' --output text) 136 | done 137 | fi 138 | COMPUTE_ENVIRONMENT=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].computeEnvironmentName' --output text) 139 | if [ "${COMPUTE_ENVIRONMENT}" == "${BATCH_COMPUTE_ENVIRONMENT_NAME}" ]; then 140 | echo "Deleting compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} ..." 141 | CMD="aws batch update-compute-environment --compute-environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} --state DISABLED" 142 | if [ "${VERBOSE}" == "true" ]; then 143 | echo "${CMD}" 144 | fi 145 | if [ "${DRY_RUN}" == "false" ]; then 146 | RESULT=$(eval "${CMD}") 147 | fi 148 | sleep 2 149 | STATE=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].state' --output text) 150 | while [ ! "${STATE}" == "DISABLED" ]; do 151 | echo "Waiting for ${BATCH_COMPUTE_ENVIRONMENT_NAME} state to change to DISABLED ..." 152 | sleep 2 153 | STATE=$(aws batch describe-compute-environment --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].state' --output text) 154 | done 155 | STATUS=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].status' --output text) 156 | while [ ! "${STATUS}" == "VALID" ]; do 157 | echo "Waiting for ${BATCH_COMPUTE_ENVIRONMENT_NAME} status to change to VALID ..." 158 | sleep 2 159 | STATUS=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].status' --output text) 160 | done 161 | CMD="aws batch delete-compute-environment --compute-environment ${BATCH_COMPUTE_ENVIRONMENT_NAME}" 162 | fi 163 | else 164 | CMD="" 165 | fi 166 | ;; 167 | *) 168 | checkTO "${TO}" 169 | CMD="docker container rm -f ${CONTAINER}" 170 | ;; 171 | esac 172 | 173 | if [ "${VERBOSE}" == "true" ]; then 174 | echo "${CMD}" 175 | fi 176 | 177 | if [ "${DRY_RUN}" == "false" ]; then 178 | eval "${CMD}" 179 | fi 180 | 181 | if [ "${DEBUG}" == "true" ]; then 182 | set +x 183 | fi 184 | 185 | -------------------------------------------------------------------------------- /install/gpg-public-key: -------------------------------------------------------------------------------- 1 | -----BEGIN PGP PUBLIC KEY BLOCK----- 2 | Version: GnuPG v2 3 | 4 | mQINBFq1SasBEADliGcT1NVJ1ydfN8DqebYYe9ne3dt6jqKFmKowLmm6LLGJe7HU 5 | jGtqhCWRDkN+qPpHqdArRgDZAtn2pXY5fEipHgar4CP8QgRnRMO2fl74lmavr4Vg 6 | 7K/KH8VHlq2uRw32/B94XLEgRbGTMdWFdKuxoPCttBQaMj3LGn6Pe+6xVWRkChQu 7 | BoQAhjBQ+bEm0kNy0LjNgjNlnL3UMAG56t8E3LANIgGgEnpNsB1UwfWluPoGZoTx 8 | N+6pHBJrKIL/1v/ETU4FXpYw2zvhWNahxeNRnoYj3uycHkeliCrw4kj0+skizBgO 9 | 2K7oVX8Oc3j5+ZilhL/qDLXmUCb2az5cMM1mOoF8EKX5HaNuq1KfwJxqXE6NNIcO 10 | lFTrT7QwD5fMNld3FanLgv/ZnIrsSaqJOL6zRSq8O4LN1OWBVbndExk2Kr+5kFxn 11 | 5lBPgfPgRj5hQ+KTHMa9Y8Z7yUc64BJiN6F9Nl7FJuSsfqbdkvRLsQRbcBG9qxX3 12 | rJAEhieJzVMEUNl+EgeCkxj5xuSkNU7zw2c3hQZqEcrADLV+hvFJktOz9Gm6xzbq 13 | lTnWWCz4xrIWtuEBA2qE+MlDheVd78a3gIsEaSTfQq0osYXaQbvlnSWOoc1y/5Zb 14 | zizHTJIhLtUyls9WisP2s0emeHZicVMfW61EgPrJAiupgc7kyZvFt4YwfwARAQAB 15 | tCRBbWF6b24gRUNTIDxlY3Mtc2VjdXJpdHlAYW1hem9uLmNvbT6JAhwEEAECAAYF 16 | AlrjL0YACgkQHivRXs0TaQrg1g/+JppwPqHnlVPmv7lessB8I5UqZeD6p6uVpHd7 17 | Bs3pcPp8BV7BdRbs3sPLt5bV1+rkqOlw+0gZ4Q/ue/YbWtOAt4qY0OcEo0HgcnaX 18 | lsB827QIfZIVtGWMhuh94xzm/SJkvngml6KB3YJNnWP61A9qJ37/VbVVLzvcmazA 19 | McWB4HUMNrhd0JgBCo0gIpqCbpJEvUc02Bjn23eEJsS9kC7OUAHyQkVnx4d9UzXF 20 | 4OoISF6hmQKIBoLnRrAlj5Qvs3GhvHQ0ThYq0Grk/KMJJX2CSqt7tWJ8gk1n3H3Y 21 | SReRXJRnv7DsDDBwFgT6r5Q2HW1TBUvaoZy5hF6maD09nHcNnvBjqADzeT8Tr/Qu 22 | bBCLzkNSYqqkpgtwv7seoD2P4n1giRvDAOEfMZpVkUr+C252IaH1HZFEz+TvBVQM 23 | Y8OWWxmIJW+J6evjo3N1eO19UHv71jvoF8zljbI4bsL2c+QTJmOv7nRqzDQgCWyp 24 | Id/v2dUVVTk1j9omuLBBwNJzQCB+72LcIzJhYmaP1HC4LcKQG+/f41exuItenatK 25 | lEJQhYtyVXcBlh6Yn/wzNg2NWOwb3vqY/F7m6u9ixAwgtIMgPCDE4aJ86zrrXYFz 26 | N2HqkTSQh77Z8KPKmyGopsmN/reMuilPdINb249nA0dzoN+nj+tTFOYCIaLaFyjs 27 | Z0r1QAOJAjkEEwECACMFAlq1SasCGwMHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIX 28 | gAAKCRC86dmkLVF4T9iFEACEnkm1dNXsWUx34R3c0vamHrPxvfkyI1FlEUen8D1h 29 | uX9xy6jCEROHWEp0rjGK4QDPgM93sWJ+s1UAKg214QRVzft0y9/DdR+twApA0fzy 30 | uavIthGd6+03jAAo6udYDE+cZC3P7XBbDiYEWk4XAF9I1JjB8hTZUgvXBL046JhG 31 | eM17+crgUyQeetkiOQemLbsbXQ40Bd9V7zf7XJraFd8VrwNUwNb+9KFtgAsc9rk+ 32 | YIT/PEf+YOPysgcxI4sTWghtyCulVnuGoskgDv4v73PALU0ieUrvvQVqWMRvhVx1 33 | 0X90J7cC1KOyhlEQQ1aFTgmQjmXexVTwIBm8LvysFK6YXM41KjOrlz3+6xBIm/qe 34 | bFyLUnf4WoiuOplAaJhK9pRY+XEnGNxdtN4D26Kd0F+PLkm3Tr3Hy3b1Ok34FlGr 35 | KVHUq1TZD7cvMnnNKEELTUcKX+1mV3an16nmAg/my1JSUt6BNK2rJpY1s/kkSGSE 36 | XQ4zuF2IGCpvBFhYAlt5Un5zwqkwwQR3/n2kwAoDzonJcehDw/C/cGos5D0aIU7I 37 | K2X2aTD3+pA7Mx3IMe2hqmYqRt9X42yF1PIEVRneBRJ3HDezAgJrNh0GQWRQkhIx 38 | gz6/cTR+ekr5TptVszS9few2GpI5bCgBKBisZIssT89aw7mAKWut0Gcm4qM9/yK6 39 | 1bkCDQRatUmrARAAxNPvVwreJ2yAiFcUpdRlVhsuOgnxvs1QgsIw3H7+Pacr9Hpe 40 | 8uftYZqdC82KeSKhpHq7c8gMTMucIINtH25x9BCc73E33EjCL9Lqov1TL7+QkgHe 41 | T+JIhZwdD8Mx2K+LVVVu/aWkNrfMuNwyDUciSI4D5QHa8T+F8fgN4OTpwYjirzel 42 | 5yoICMr9hVcbzDNv/ozKCxjx+XKgnFc3wrnDfJfntfDAT7ecwbUTL+viQKJ646s+ 43 | psiqXRYtVvYInEhLVrJ0aV6zHFoigE/Bils6/g7ru1Q6CEHqEw++APs5CcE8VzJu 44 | WAGSVHZgun5Y9N4quR/M9Vm+IPMhTxrAg7rOvyRN9cAXfeSMf77I+XTifigNna8x 45 | t/MOdjXr1fjF4pThEi5u6WsuRdFwjY2azEv3vevodTi4HoJReH6dFRa6y8c+UDgl 46 | 2iHiOKIpQqLbHEfQmHcDd2fix+AaJKMnPGNku9qCFEMbgSRJpXz6BfwnY1QuKE+I 47 | R6jA0frUNt2jhiGG/F8RceXzohaaC/Cx7LUCUFWc0n7z32C9/Dtj7I1PMOacdZzz 48 | bjJzRKO/ZDv+UN/c9dwAkllzAyPMwGBkUaY68EBstnIliW34aWm6IiHhxioVPKSp 49 | VJfyiXPO0EXqujtHLAeChfjcns3I12YshT1dv2PafG53fp33ZdzeUgsBo+EAEQEA 50 | AYkCHwQYAQIACQUCWrVJqwIbDAAKCRC86dmkLVF4T+ZdD/9x/8APzgNJF3o3STrF 51 | jvnV1ycyhWYGAeBJiu7wjsNWwzMFOv15tLjB7AqeVxZn+WKDD/mIOQ45OZvnYZuy 52 | X7DR0JszaH9wrYTxZLVruAu+t6UL0y/XQ4L1GZ9QR6+r+7t1Mvbfy7BlHbvX/gYt 53 | Rwe/uwdibI0CagEzyX+2D3kTOlHO5XThbXaNf8AN8zha91Jt2Q2UR2X5T6JcwtMz 54 | FBvZnl3LSmZyE0EQehS2iUurU4uWOpGppuqVnbi0jbCvCHKgDGrqZ0smKNAQng54 55 | F365W3g8AfY48s8XQwzmcliowYX9bT8PZiEi0J4QmQh0aXkpqZyFefuWeOL2R94S 56 | XKzr+gRh3BAULoqF+qK+IUMxTip9KTPNvYDpiC66yBiT6gFDji5Ca9pGpJXrC3xe 57 | TXiKQ8DBWDhBPVPrruLIaenTtZEOsPc4I85yt5U9RoPTStcOr34s3w5yEaJagt6S 58 | Gc5r9ysjkfH6+6rbi1ujxMgROSqtqr+RyB+V9A5/OgtNZc8llK6u4UoOCde8jUUW 59 | vqWKvjJB/Kz3u4zaeNu2ZyyHaOqOuH+TETcW+jsY9IhbEzqN5yQYGi4pVmDkY5vu 60 | lXbJnbqPKpRXgM9BecV9AMbPgbDq/5LnHJJXg+G8YQOgp4lR/hC1TEFdIp5wM8AK 61 | CWsENyt2o1rjgMXiZOMF8A5oBLkCDQRatUuSARAAr77kj7j2QR2SZeOSlFBvV7oS 62 | mFeSNnz9xZssqrsm6bTwSHM6YLDwc7Sdf2esDdyzONETwqrVCg+FxgL8hmo9hS4c 63 | rR6tmrP0mOmptr+xLLsKcaP7ogIXsyZnrEAEsvW8PnfayoiPCdc3cMCR/lTnHFGA 64 | 7EuR/XLBmi7Qg9tByVYQ5Yj5wB9V4B2yeCt3XtzPqeLKvaxl7PNelaHGJQY/xo+m 65 | V0bndxf9IY+4oFJ4blD32WqvyxESo7vW6WBh7oqv3Zbm0yQrr8a6mDBpqLkvWwNI 66 | 3kpJR974tg5o5LfDu1BeeyHWPSGm4U/G4JB+JIG1ADy+RmoWEt4BqTCZ/knnoGvw 67 | D5sTCxbKdmuOmhGyTssoG+3OOcGYHV7pWYPhazKHMPm201xKCjH1RfzRULzGKjD+ 68 | yMLT1I3AXFmLmZJXikAOlvE3/wgMqCXscbycbLjLD/bXIuFWo3rzoezeXjgi/DJx 69 | jKBAyBTYO5nMcth1O9oaFd9d0HbsOUDkIMnsgGBE766Piro6MHo0T0rXl07Tp4pI 70 | rwuSOsc6XzCzdImj0Wc6axS/HeUKRXWdXJwno5awTwXKRJMXGfhCvSvbcbc2Wx+L 71 | IKvmB7EB4K3fmjFFE67yolmiw2qRcUBfygtH3eL5XZU28MiCpue8Y8GKJoBAUyvf 72 | KeM1rO8Jm3iRAc5a/D0AEQEAAYkEPgQYAQIACQUCWrVLkgIbAgIpCRC86dmkLVF4 73 | T8FdIAQZAQIABgUCWrVLkgAKCRDePL1hra+LjtHYD/9MucxdFe6bXO1dQR4tKhhQ 74 | P0LRqy6zlBY9ILCLowNdGZdqorogUiUymgn3VhEhVtxTOoHcN7qOuM01PNsRnOeS 75 | EYjf8Xrb1clzkD6xULwmOclTb9bBxnBc/4PFvHAbZW3QzusaZniNgkuxt6BTfloS 76 | Of4inq71kjmGK+TlzQ6mUMQUg228NUQC+a84EPqYyAeY1sgvgB7hJBhYL0QAxhcW 77 | 6m20Rd8iEc6HyzJ3yCOCsKip/nRWAbf0OvfHfRBp0+m0ZwnJM8cPRFjOqqzFpKH9 78 | HpDmTrC4wKP1+TL52LyEqNh4yZitXmZNV7giSRIkk0eDSko+bFy6VbMzKUMkUJK3 79 | D3eHFAMkujmbfJmSMTJOPGn5SB1HyjCZNx6bhIIbQyEUB9gKCmUFaqXKwKpF6rj0 80 | iQXAJxLR/shZ5Rk96VxzOphUl7T90m/PnUEEPwq8KsBhnMRgxa0RFidDP+n9fgtv 81 | HLmrOqX9zBCVXh0mdWYLrWvmzQFWzG7AoE55fkf8nAEPsalrCdtaNUBHRXA0OQxG 82 | AHMOdJQQvBsmqMvuAdjkDWpFu5y0My5ddU+hiUzUyQLjL5Hhd5LOUDdewlZgIw1j 83 | xrEAUzDKetnemM8GkHxDgg8koev5frmShJuce7vSjKpCNg3EIJSgqMOPFjJuLWtZ 84 | vjHeDNbJy6uNL65ckJy6WhGjEADS2WAW1D6Tfekkc21SsIXk/LqEpLMR/0g5OUif 85 | wcEN1rS9IJXBwIy8MelN9qr5KcKQLmfdfBNEyyceBhyVl0MDyHOKC+7PofMtkGBq 86 | 13QieRHv5GJ8LB3fclqHV8pwTTo3Bc8z2g0TjmUYAN/ixETdReDoKavWJYSE9yoM 87 | aaJu279ioVTrwpECse0XkiRyKToTjwOb73CGkBZZpJyqux/rmCV/fp4ALdSW8zbz 88 | FJVORaivhoWwzjpfQKhwcU9lABXi2UvVm14v0AfeI7oiJPSU1zM4fEny4oiIBXlR 89 | zhFNih1UjIu82X16mTm3BwbIga/s1fnQRGzyhqUIMii+mWra23EwjChaxpvjjcUH 90 | 5ilLc5Zq781aCYRygYQw+hu5nFkOH1R+Z50Ubxjd/aqUfnGIAX7kPMD3Lof4KldD 91 | Q8ppQriUvxVo+4nPV6rpTy/PyqCLWDjkguHpJsEFsMkwajrAz0QNSAU5CJ0G2Zu4 92 | yxvYlumHCEl7nbFrm0vIiA75Sa8KnywTDsyZsu3XcOcf3g+g1xWTpjJqy2bYXlqz 93 | 9uDOWtArWHOis6bq8l9RE6xr1RBVXS6uqgQIZFBGyq66b0dIq4D2JdsUvgEMaHbc 94 | e7tBfeB1CMBdA64e9Rq7bFR7Tvt8gasCZYlNr3lydh+dFHIEkH53HzQe6l88HEic 95 | +0jVnLkCDQRa55wJARAAyLya2Lx6gyoWoJN1a6740q3o8e9d4KggQOfGMTCflmeq 96 | ivuzgN+3DZHN+9ty2KxXMtn0mhHBerZdbNJyjMNT1gAgrhPNB4HtXBXum2wS57WK 97 | DNmade914L7FWTPAWBG2Wn448OEHTqsClICXXWy9IICgclAEyIq0Yq5mAdTEgRJS 98 | Z8t4GpwtDL9gNQyFXaWQmDmkAsCygQMvhAlmu9xOIzQG5CxSnZFk7zcuL60k14Z3 99 | Cmt49k4T/7ZU8goWi8tt+rU78/IL3J/fF9+1civ1OwuUidgfPCSvOUW1JojsdCQA 100 | L+RZJcoXq7lfOFj/eNjeOSstCTDPfTCL+kThE6E5neDtbQHBYkEX1BRiTedsV4+M 101 | ucgiTrdQFWKf89G72xdv8ut9AYYQ2BbEYU+JAYhUH8rYYui2dHKJIgjNvJscuUWb 102 | +QEqJIRleJRhrO+/CHgMs4fZAkWF1VFhKBkcKmEjLn1f7EJJUUW84ZhKXjO/AUPX 103 | 1CHsNjziRceuJCJYox1cwsoq6jTE50GiNzcIxTn9xUc0UMKFeggNAFys1K+TDTm3 104 | Bzo8H5ucjCUEmUm9lhkGwqTZgOlRX5eqPX+JBoSaObqhgqCa5IPinKRa6MgoFPHK 105 | 6sYKqroYwBGgZm6Js5chpNchvJMs/3WXNOEVg0J3z3vP0DMhxqWm+r+n9zlW8qsA 106 | EQEAAYkEPgQYAQgACQUCWuecCQIbAgIpCRC86dmkLVF4T8FdIAQZAQgABgUCWuec 107 | CQAKCRBQ3szEcQ5hr+ykD/4tOLRHFHXuKUcxgGaubUcVtsFrwBKma1cYjqaPms8u 108 | 6Sk0wfGRI32G/GhOrp0Ts/MOkbObq6VLTh8N5Yc/53MEl8zQFw9Y5AmRoW4PZXER 109 | ujs5s7p4oR7xHMihMjCCBn1bvrR+34YPfgzTcgLiOEFHYT8UTxwnGmXOvNkMM7md 110 | xD3CV5q6VAte8WKBo/220II3fcQlc9r/oWX4kXXkb0v9hoGwKbDJ1tzqTPrp/xFt 111 | yohqnvImpnlz+Q9zXmbrWYL9/g8VCmW/NN2gju2G3Lu/TlFUWIT4v/5OPK6TdeNb 112 | VKJO4+S8bTayqSG9CML1S57KSgCo5HUhQWeSNHI+fpe5oX6FALPT9JLDce8OZz1i 113 | cZZ0MELP37mOOQun0AlmHm/hVzf0f311PtbzcqWaE51tJvgUR/nZFo6Ta3O5Ezhs 114 | 3VlEJNQ1Ijf/6DH87SxvAoRIARCuZd0qxBcDK0avpFzUtbJd24lRA3WJpkEiMqKv 115 | RDVZkE4b6TW61f0o+LaVfK6E8oLpixegS4fiqC16mFrOdyRk+RJJfIUyz0WTDVmt 116 | g0U1CO1ezokMSqkJ7724pyjr2xf/r9/sC6aOJwB/lKgZkJfC6NqL7TlxVA31dUga 117 | LEOvEJTTE4gl+tYtfsCDvALCtqL0jduSkUo+RXcBItmXhA+tShW0pbS2Rtx/ixua 118 | KohVD/0R4QxiSwQmICNtm9mw9ydIl1yjYXX5a9x4wMJracNY/LBybJPFnZnT4dYR 119 | z4XjqysDwvvYZByaWoIe3QxjX84V6MlI2IdAT/xImu8gbaCI8tmyfpIrLnPKiR9D 120 | VFYfGBXuAX7+HgPPSFtrHQONCALxxzlbNpS+zxt9r0MiLgcLyspWxSdmoYGZ6nQP 121 | RO5Nm/ZVS+u2imPCRzNUZEMa+dlE6kHx0rS0dPiuJ4O7NtPeYDKkoQtNagspsDvh 122 | cK7CSqAiKMq06UBTxqlTSRkm62eOCtcs3p3OeHu5GRZF1uzTET0ZxYkaPgdrQknx 123 | ozjP5mC7X+45lcCfmcVt94TFNL5HwEUVJpmOgmzILCI8yoDTWzloo+i+fPFsXX4f 124 | kynhE83mSEcr5VHFYrTY3mQXGmNJ3bCLuc/jq7ysGq69xiKmTlUeXFm+aojcRO5i 125 | zyShIRJZ0GZfuzDYFDbMV9amA/YQGygLw//zP5ju5SW26dNxlf3MdFQE5JJ86rn9 126 | MgZ4gcpazHEVUsbZsgkLizRp9imUiH8ymLqAXnfRGlU/LpNSefnvDFTtEIRcpOHc 127 | bhayG0bk51Bd4mioOXnIsKy4j63nJXA27x5EVVHQ1sYRN8Ny4Fdr2tMAmj2O+X+J 128 | qX2yy/UX5nSPU492e2CdZ1UhoU0SRFY3bxKHKB7SDbVeav+K5g== 129 | =Gi5D 130 | -----END PGP PUBLIC KEY BLOCK----- 131 | -------------------------------------------------------------------------------- /.env: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | # Source helper functions 9 | source .fun 10 | 11 | # Proxy settings [optional] - set if your network requires a proxy to connect to the Internet 12 | export http_proxy= 13 | export https_proxy= 14 | export no_proxy=localhost 15 | 16 | # Project settings 17 | ## CMD_EDIT - text editor command used when editing config file. Default CMD_EDIT='vim -c ":syntax on"' 18 | export CMD_EDIT='vim -c ":syntax on"' 19 | ## Target orchestrator TO=docker|compose|ecs|swarm|kubernetes|eks|lambda|batchlocal|batch 20 | ## docker - choose when running locally or on ec2 21 | ## compose - choose when running locally, on ec2 22 | ## ecs - choose when running on ecs with or without Fargate 23 | ## swarm - choose when running on a docker swarm 24 | ## kubernetes - choose when running on a local, remote Kubernetes cluster or EKS, with or without Fargate 25 | ## lambdalocal - choose when developing and testing an AWS Lambda container locally 26 | ## lambda - choose when running containerized function on AWS Lambda 27 | ## batchlocal - choose when running containerized jobs locally 28 | ## batch - choose when running containerized jobs on AWS Batch 29 | export TO=docker 30 | ## DOCKERFILE - dockerfile to use for this project DOCKERFILE=./Dockerfile|./Dockerfile.lambda. If TO=lambda, then set DOCKERFILE=./Dockerfile.lambda 31 | export DOCKERFILE_EXT="" 32 | if [[ "$TO" == "lambda" || "$TO" == "lambdalocal" ]]; then 33 | export DOCKERFILE_EXT="-lambda" 34 | fi 35 | export DOCKERFILE=./Dockerfile${DOCKERFILE_EXT} 36 | ## VERBOSE - show verbose output, VERBOSE=true|false 37 | export VERBOSE=true 38 | ## DEBUG - turn on script debugging, DEBUG=false|true 39 | export DEBUG=false 40 | ## DRY_RUN - do not execute commands, DRY_RUN=false|true 41 | export DRY_RUN=false 42 | 43 | # Docker image settings 44 | ## BASE_IMAGE - your image will be built FROM this BASE_IMAGE as a starting point 45 | export BASE_IMAGE_PATH=python:3.9 46 | ## MAINTAINER - the user who owns and maintains this image 47 | export MAINTAINER="$(whoami)" 48 | ## DESCRIPTION - short description of what your image does 49 | export DESCRIPTION="Simple Parrot API - providing a healthcheck and repeating requested text" 50 | ## BUILD - a user-friendly build identifier to distinguish between two images that have the same tag 51 | export BUILD=$(date +%Y%m%d%H%M) 52 | ## REGISTRY: [optional] - Docker registry path including trailing "/". Example: registry.company.com/demo/ 53 | export REGISTRY= 54 | if [ -n "${REGISTRY}" ]; then 55 | if [ "${REGISTRY: -1}" != "/" ]; then 56 | export REGISTRY="${REGISTRY}/" 57 | fi 58 | fi 59 | ## REGISTRY_TYPE - type of container registry, used by login.sh. REGISTRY_TYPE=ecr 60 | export REGISTRY_TYPE=ecr 61 | ## IMAGE_NAME - name of the Docker image for this project. Example: parrot 62 | export IMAGE_NAME=parrot 63 | ## IMAGE - derived Docker image name, do not edit. 64 | export IMAGE=${IMAGE_NAME}${DOCKERFILE_EXT} 65 | ## VERSION: [optional] - Version tag for this Docker image. Example: v20180302 66 | #export VERSION=v$(date +%Y%m%d) 67 | export VERSION=latest 68 | export TAG=$(if [ -z "${VERSION}" ]; then echo ""; else echo ":${VERSION}"; fi) 69 | ## BUILD_OPTS: [optional] - arguments for the docker image build command 70 | export BUILD_OPTS="--file ${DOCKERFILE} --build-arg BUILD=${BUILD} --build-arg BASE_IMAGE_PATH=${BASE_IMAGE_PATH} --build-arg MAINTAINER=\"${MAINTAINER}\" --build-arg DESCRIPTION=\"${DESCRIPTION}\" --build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} --build-arg no_proxy=${no_proxy}" 71 | 72 | # Target Orchestrator (TO) Settings 73 | ## common 74 | ### CONTAINER_NAME: [optional] - Name of the Docker container including the --name switch. Example --name myapp 75 | export CONTAINER=${IMAGE}-${TO} 76 | export CONTAINER_NAME="--name ${CONTAINER}" 77 | ### PORT_INTERNAL - port on which the application runs inside the container, if more than one port is needed, define additioal variables here 78 | export PORT_INTERNAL=80 79 | ### PORT_EXTERNAL - port to expose outside the container, if more than one port is needed, define additional variables here 80 | ### firstAvailable is a function defined in .fun. For local orchestrators, it returns the first available port, starting from the one requested in the argument 81 | export PORT_EXTERNAL=$(firstAvailable 80) 82 | ### VOLUME_PATH_INTERNAL - mount path of volume inside the container 83 | export VOLUME_PATH_INTERNAL=/wd 84 | ## VOLUME_PATH_EXTERNAL - external path to mount 85 | export VOLUME_PATH_EXTERNAL=$(pwd) 86 | ### ENVSUBST - envsubst cli. ENVSUBST=envsubst | "env > /tmp/myanv && docker run --rm -it --env-file /tmp/myenv -v $(pwd):$(pwd) iankoulski/envsubst sh -c envsubst" 87 | export ENVSUBST=envsubst 88 | ### REGION - the region where container will be deployed. Example: REGION=us-west-2 89 | export REGION=us-west-2 90 | 91 | ## docker 92 | ### Port map [optional] - Mapping of external to internal ports including the -p switch. Example -p 80:8080 93 | export PORT_MAP="-p ${PORT_EXTERNAL}:${PORT_INTERNAL}" 94 | ### Volume map [optional] - Mapping of external to internal paths including the -v switch. Example $(pwd):/wd 95 | export VOL_MAP="-v ${VOLUME_PATH_EXTERNAL}:${VOLUME_PATH_INTERNAL}" 96 | ### Network [optional] - Network name including the --net switch. Example --net mynet 97 | export NETWORK= 98 | ### RUN_OPTS [optional] - additional options to specify with the run comman. Example -e POSTGRES_DB=dbname 99 | export RUN_OPTS="-e PORT_INTERNAL=$PORT_INTERNAL -e ITERATION_LIMIT=3 -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy" 100 | 101 | ## compose 102 | ### DOCKER_COMPOSE - docker-compose cli, DOCKER_COMPOSE=docker-compose | "docker compose", 103 | ### please use docker-compose with traditional docker-compose and "docker compose" with Docker Desktop 104 | export DOCKER_COMPOSE="docker compose" 105 | ### COMPOSE_CONTEXT_TYPE - docker compose context type, COMPOSE_CONTEXT_TYPE=moby|ecs, 106 | ### please use moby when running against a docker daemon, or ecs when depoying to an AWS ECS context. 107 | COMPOSE_CONTEXT_TYPE=moby 108 | ### COMPOSE_PROJECT_NAME - prefix for project containers, example COMPOSE_PROJECT_NAME=compose 109 | export COMPOSE_PROJECT_NAME=compose 110 | ### COMPOSE_TEMPLATE_PATH - folder containing compose file templates 111 | export COMPOSE_TEMPLATE_PATH=./to/compose/template 112 | ### COMPOSE_APP_PATH - folder containig generated compose files 113 | export COMPOSE_APP_PATH=./to/compose/app 114 | ### COMPOSE_TEST_PATH - folder containing deployment test scripts 115 | export COMPOSE_TEST_PATH=./to/compose/test 116 | ### COMPOSE_FILE - file path of the docker-compose file for the current app 117 | export COMPOSE_FILE=${COMPOSE_APP_PATH}/docker-compose.yaml 118 | 119 | ## ecs 120 | ### ECS_CLI - ecs-cli command, ECS_CLI=ecs-cli 121 | export ECS_CLI=ecs-cli 122 | ### ECS_CLUSTER_NAME - name of the ECS cluster to use, if the cluster does not exist, it will be created, ECS_CLUSTER_NAME=default 123 | export ECS_CLUSTER=default 124 | ### ECS_MANAGE_CLUSTER - when true create and remove ECS cluster upon run or stop of container, when false assume cluster exists, ECS_MANAGE_CLUSTER=false|true 125 | ECS_MANAGE_CLUSTER=true 126 | ### ECS_LAUNCH_TYPE - default launch type for ecs tasks, determines if tasks run on EC2 or Fargate, ECS_LAUNCH_TYPE=EC2|FARGATE 127 | export ECS_LAUNCH_TYPE=EC2 128 | ### ECS_COMPOSE_FILE - file path to the docker-compose file for the current ECS app, generated from template 129 | export ECS_COMPOSE_FILE=${COMPOSE_APP_PATH}/ecs-compose.yaml 130 | ### ECS_PARAMS_FILE - file with ECS-specific task definition parameters 131 | export ECS_PARAMS_FILE=${COMPOSE_APP_PATH}/ecs-params-${ECS_LAUNCH_TYPE}.yaml 132 | ### ECS_TRUST_FILE - file with IAM principal information for ecsTaskExecitonRole and ecsTaskRole 133 | export ECS_TRUST_FILE=${COMPOSE_APP_PATH}/ecs-trust.json 134 | ### ECS_EXEC_POLICY_FILE - file with definition of IAM policy ecsExecPolicy for ecsTaskRole 135 | export ECS_EXEC_POLICY_FILE=${COMPOSE_APP_PATH}/ecs-exec-policy.json 136 | ### ECS_SG_CIDR - security group CIDR from which to allow connections to the containers, ECS_SG_CIDR=0.0.0.0/0 137 | export ECS_SG_CIDR="0.0.0.0/0" 138 | ### ECS_MEM_LIMIT - memory limit for container, ECS_MEM_LIMIT="0.5GB" 139 | export ECS_MEM_LIMIT="0.5GB" 140 | ### ECS_CPU_LIMIT - cpu limit for container, 1 vCPU = 1024, ECS_CPU_LIMIT=256 141 | export ECS_CPU_LIMIT=256 142 | ### ECS_ASSIGN_PUBLIC_IP - auto-assign public ip to container, ECS_ASSIGN_PUBLIC_IP=ENABLED|DISABLED 143 | export ECS_ASSIGN_PUBLIC_IP=ENABLED 144 | 145 | ## swarm 146 | ### SWARM_STACK_NAME - name of the application stack to deploy, similar to COMPOSE_PROJECT_NAME 147 | export SWARM_STACK_NAME=mystack 148 | ### SWARM_SERVICE_NAME - a stack may contain many services. The action scripts are executed agains the service name configured here. 149 | export SWARM_SERVICE_NAME=${CONTAINER} 150 | 151 | ## kubernetes 152 | ### KUBECTL - kubectl cli, KUBECTL=kubectl 153 | export KUBECTL=kubectl 154 | ### KUBETAIL - kubetail cli, KUBETAIL=kubetail 155 | export KUBETAIL=kubetail 156 | ### KUBERNETES_TEMPLATE_PATH - folder containing Kubernetes manifest templates 157 | export KUBERNETES_TEMPLATE_PATH=./to/kubernetes/template 158 | ### KUBERNETES_APP_PATH - folder containing Kubernetes manifests 159 | export KUBERNETES_APP_PATH=./to/kubernetes/app 160 | ### APP_NAME - name of the appliation to deploy 161 | export APP_NAME=${CONTAINER} 162 | ### APP_DNS_NAME - fully qualified domain name for the application 163 | export APP_DNS_NAME=${CONTAINER}.domain.ext 164 | ### NAMESPACE - Kubernetes namespace where the app will run 165 | export NAMESPACE=${APP_NAME} 166 | ### INSTANCE_TYPE - optional instance type on which application pods should run. To use, uncomment deployment template nodeSelector section. 167 | export INSTANCE_TYPE=c5.2xlarge 168 | 169 | ## lambda 170 | ### see https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html 171 | ### LAMBDA_FUNCTION_NAME - name of the function to create, LAMBDA_FUNCTION_NAME=${CONTAINER} 172 | export LAMBDA_FUNCTION_NAME=${CONTAINER} 173 | ### LAMBDA_TEMPLATE_PATH - path to lambda-related templates LAMBDA_TEMPLATE_PATH=./to/lambda/template 174 | export LAMBDA_TEMPLATE_PATH=./to/lambda/template 175 | ### LAMBDA_APP_PATH - path to lambda related files generated from templates LAMBDA_APP_PATH=./to/lambda/app 176 | export LAMBDA_APP_PATH=./to/lambda/app 177 | ### LAMBDA_TEST_PATH - path to directory containing test scripts for deployed lambda function 178 | export LAMBDA_TEST_PATH=./to/lambda/test 179 | ### LAMBDA_TRUST_FILE - file with IAM principal information for lambdaFunctionRole 180 | export LAMBDA_TRUST_FILE=${LAMBDA_APP_PATH}/lambda-trust.json 181 | ## LAMBDA_POLICY_FILE - file with definition of IAM policy lambdaFunctionPolicy for lambdaFunctionRole 182 | export LAMBDA_POLICY_FILE=${LAMBDA_APP_PATH}/lambda-policy.json 183 | 184 | ## batch 185 | ### BATCH_NAME - app name for the current batch workload 186 | export BATCH_NAME=${CONTAINER} 187 | ### BATCH_COMPUTE_ENVIRONMENT_NAME - name of the compute environment to submit jobs to, BATCH_COMPUTE_ENVIRONMENT_NAME=${BATCH_NAME}-compute 188 | export BATCH_COMPUTE_ENVIRONMENT_NAME=${BATCH_NAME}-compute 189 | ### BATCH_JOB_QUEUE_NAME - name of the job queue for the current batch, BATCH_JOB_QUEUE_NAME=${BATCH_NAME}-job-queue-yyyymmdd 190 | #export BATCH_JOB_QUEUE_NAME=${BATCH_NAME}-job-queue-$(date +%Y%m%d) 191 | export BATCH_JOB_QUEUE_NAME=${BATCH_NAME}-job-queue 192 | ### BATCH_JOB_DEFINITION_NAME - name of the batch job definition, BATCH_JOB_DEFINITION_NAME=${BATCH_NAME}-job-definition 193 | export BATCH_JOB_DEFINITION_NAME=${BATCH_NAME}-job-definition 194 | ### BATCH_JOB_NAME - name of the batch job, BATCH_JOB_NAME=${BATCH_NAME}-job 195 | export BATCH_JOB_NAME=${BATCH_NAME}-job 196 | ### BATCH_COMMAND_DEFAULT - the default command to execute in the job container, BATCH_COMMAND_DEFAULT=["/job/startup.sh"] 197 | export BATCH_COMMAND_DEFAULT="/job/startup.sh" 198 | ### BATCH_JOB_ENV_VARS - environment variables to set in the job container, BATCH_JOB_ENV_VARS="[{name=NAME1,value=VALUE1},{name=NAME2,value=VALUE2}]" 199 | export BATCH_JOB_ENV_VARS="[{name=ITERATION_LIMIT,value=6}]" 200 | ### BATCH_MANAGE_COMPUTE_ENVIRONMENT - if true create and delete compute environment upon run or stop, if false assume compute environment exists, BATCH_MANAGE_COMPUTE_ENVIRONMENT=true|false 201 | export BATCH_MANAGE_COMPUTE_ENVIRONMENT=true 202 | ### BATCH_JOB_VCPUS - number of vCPUs to assign to the batch job, BATCH_JOB_VCPUS=1 203 | ### The combination of VCPUS and MEMORY has to be supported according to https://docs.aws.amazon.com/cli/latest/reference/batch/register-job-definition.html 204 | ### Examples: VCPUS=0.5, MEMORY=1024,2048,3072,4096; VCPUS=1, MEMORY=2048,3072,4096,5120,6144,7168,8192 205 | export BATCH_JOB_VCPUS=1 206 | ## BATCH_JOB_MEMORY - memory limit in MiB for the container, BATCH_JOB_MEMORY=2048 207 | export BATCH_JOB_MEMORY=2048 208 | ### BATCH_COMPUTE_ENVIRONMENT_TYPE - type of compute to use, BATCH_COMPUTE_ENVIRONMENT_TYPE=EC2|FARGATE 209 | export BATCH_COMPUTE_ENVIRONMENT_TYPE=EC2 210 | ### BATCH_COMPUTE_RESOURCES - compute environment configuration depending on type. 211 | ### It is assumed that a VPC already exists. Specify subnets where you would like batch jobs to run. 212 | ### Specify at least one valid security group. The vpc default security group is acceptable to use here. 213 | ### The full list of possible settings is below 214 | ### export BATCH_COMPUTE_RESOURCES="type=string,allocationStrategy=string,minvCpus=integer,maxvCpus=integer,desiredvCpus=integer,instanceTypes=string,string,imageId=string,subnets=string,string,securityGroupIds=string,string,ec2KeyPair=string,instanceRole=string,tags={KeyName1=string,KeyName2=string},placementGroup=string,bidPercentage=integer,spotIamFleetRole=string,launchTemplate={launchTemplateId=string,launchTemplateName=string,version=string},ec2Configuration=[{imageType=string,imageIdOverride=string},{imageType=string,imageIdOverride=string}]" 215 | if [ "${BATCH_COMPUTE_ENVIRONMENT_TYPE}" == "EC2" ]; then 216 | export BATCH_COMPUTE_RESOURCES="type=EC2,minvCpus=0,maxvCpus=256,instanceTypes=optimal,instanceRole=ecsInstanceRole,subnets=subnet-0e11612f928c54936,subnet-0003aacf876f0fac1,securityGroupIds=sg-097730877ae293efd" 217 | else 218 | export BATCH_COMPUTE_RESOURCES="type=FARGATE,maxvCpus=256,subnets=subnet-0e11612f928c54936,subnet-0003aacf876f0fac1,securityGroupIds=sg-097730877ae293efd" 219 | fi 220 | 221 | -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ###################################################################### 4 | # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # 5 | # SPDX-License-Identifier: MIT-0 # 6 | ###################################################################### 7 | 8 | source .env 9 | 10 | if [ "${DEBUG}" == "true" ]; then 11 | set -x 12 | fi 13 | 14 | if [ -z "$1" ]; then 15 | MODE=-d 16 | else 17 | MODE=-it 18 | fi 19 | 20 | function generate_compose_files 21 | { 22 | echo "Generating compose files ..." 23 | if [ ! -d "${COMPOSE_APP_PATH}" ]; then 24 | mkdir -p "${COMPOSE_APP_PATH}" 25 | fi 26 | CMD="BASE_PATH=$(pwd); cd ${COMPOSE_TEMPLATE_PATH}; for f in *.*; do cat \$f | envsubst > \${BASE_PATH}/\${COMPOSE_APP_PATH}/\$f; done; cd \${BASE_PATH}" 27 | if [ "${VERBOSE}" == "true" ]; then 28 | echo "${CMD}" 29 | fi 30 | if [ "${DRY_RUN}" == "false" ]; then 31 | eval "${CMD}" 32 | fi 33 | } 34 | 35 | function generate_lambda_files 36 | { 37 | echo "Generating lambda files ..." 38 | export ACCOUNT=$(aws sts get-caller-identity --query 'Account' --output text) 39 | if [ ! -d "${LAMBDA_APP_PATH}" ]; then 40 | mkdir -p "${LAMBDA_APP_PATH}" 41 | fi 42 | CMD="BASE_PATH=$(pwd); cd ${LAMBDA_TEMPLATE_PATH}; for f in *.*; do cat \$f | envsubst > \${BASE_PATH}/\${LAMBDA_APP_PATH}/\$f; done; cd \${BASE_PATH}" 43 | if [ "${VERBOSE}" == "true" ]; then 44 | echo "${CMD}" 45 | fi 46 | if [ "${DRY_RUN}" == "false" ]; then 47 | eval "${CMD}" 48 | fi 49 | } 50 | 51 | function generate_kubernetes_manifests 52 | { 53 | echo "Generating Kubernetes manifests ..." 54 | if [ ! -d "${KUBERNETES_APP_PATH}" ]; then 55 | mkdir -p "${KUBERNETES_APP_PATH}" 56 | fi 57 | CMD="BASE_PATH=$(pwd); cd ${KUBERNETES_TEMPLATE_PATH}; for f in *.yaml; do cat \$f | envsubst > \${BASE_PATH}/\${KUBERNETES_APP_PATH}/\$f; done; cd \${BASE_PATH}" 58 | if [ "${VERBOSE}" == "true" ]; then 59 | echo "${CMD}" 60 | fi 61 | if [ "${DRY_RUN}" == "false" ]; then 62 | eval "${CMD}" 63 | fi 64 | 65 | } 66 | 67 | function prepare_ecs_roles 68 | { 69 | LOG_GROUP=/aws/ecs/${ECS_CLUSTER} 70 | echo "Preparing LogGroup $LOG_GROUP ..." 71 | LOG_GROUPS=$(aws logs describe-log-groups | grep GroupName | grep ${LOG_GROUP}) 72 | if [ "$LOG_GROUPS" == "" ]; then 73 | echo "Creating log group ${LOG_GROUP} ..." 74 | RESULT=$(aws logs create-log-group --region ${REGION} --log-group-name ${LOG_GROUP}) 75 | fi 76 | export LOG_GROUP_ARN=$(aws logs describe-log-groups --query "logGroups[?logGroupName==\`${LOG_GROUP}\`].arn" --output text) 77 | generate_compose_files 78 | echo "Preparing ecsInstanceRole ..." 79 | role=$(aws iam list-roles --query 'Roles[?RoleName==`ecsInstanceRole`].RoleName' --output text) 80 | if [ "${role}" == "" ]; then 81 | RESULT=$(aws iam create-role --role-name ecsInstanceRole --assume-role-policy-document file://${ECS_TRUST_FILE}) 82 | RESULT=$(aws iam attach-role-policy --role-name ecsInstanceRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role) 83 | fi 84 | echo "Preparing ecsTaskRole ..." 85 | role=$(aws iam list-roles --query 'Roles[?RoleName==`ecsTaskRole`].RoleName' --output text) 86 | if [ "${role}" == "" ]; then 87 | RESULT=$(aws iam create-role --role-name ecsTaskRole --assume-role-policy-document file://${ECS_TRUST_FILE}) 88 | RESULT=$(aws iam put-role-policy --role-name ecsTaskRole --policy-name ecsExecPolicy --policy-document file://${ECS_EXEC_POLICY_FILE}) 89 | fi 90 | role_arn=$(aws iam list-roles --query 'Roles[?RoleName==`ecsTaskRole`].Arn' --output text) 91 | export ECS_TASK_ROLE_ARN="${role_arn}" 92 | echo "Preparing ecsTaskExecutionRole ..." 93 | role=$(aws iam list-roles --query 'Roles[?RoleName==`ecsTaskExecutionRole`].RoleName' --output text) 94 | if [ "${role}" == "" ]; then 95 | RESULT=$(aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://${ECS_TRUST_FILE}) 96 | fi 97 | policy=$(aws iam list-attached-role-policies --role-name ecsTaskExecutionRole --query 'AttachedPolicies[?PolicyName==`AmazonECSTaskExecutionRolePolicy`].PolicyName' --output text) 98 | if [ "${policy}" == "" ]; then 99 | RESULT=$(aws iam attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy) 100 | fi 101 | role_arn=$(aws iam list-roles --query 'Roles[?RoleName==`ecsTaskExecutionRole`].Arn' --output text) 102 | export ECS_EXEC_ROLE_ARN="${role_arn}" 103 | } 104 | 105 | function prepare_lambda_role 106 | { 107 | echo "Preparing AWSLambdaFunctionRole ..." 108 | role=$(aws iam list-roles --query "Roles[?RoleName==\`AWSLambdaFunctionRole\`].RoleName" --output text) 109 | if [ "${role}" == "" ]; then 110 | RESULT=$(aws iam create-role --role-name AWSLambdaFunctionRole --assume-role-policy-document file://${LAMBDA_TRUST_FILE}) 111 | fi 112 | policy=$(aws iam list-attached-role-policies --role-name AWSLambdaFunctionRole --query 'AttachedPolicies[?PolicyName==`AWSLambdaBasicExecutionRole`].PolicyName' --output text) 113 | if [ "${policy}" == "" ]; then 114 | RESULT=$(aws iam attach-role-policy --role-name AWSLambdaFunctionRole --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole) 115 | fi 116 | role_arn=$(aws iam list-roles --query "Roles[?RoleName==\`AWSLambdaFunctionRole\`].Arn" --output text) 117 | export LAMBDA_ROLE_ARN="${role_arn}" 118 | } 119 | 120 | function prepare_ecs_cluster 121 | { 122 | ECS_CLUSTERS=$(aws ecs list-clusters | grep -w ${ECS_CLUSTER}) 123 | if [ "${ECS_MANAGE_CLUSTER}" == "true" ]; then 124 | echo "Preparing ECS Cluster ${ECS_CLUSTER} ..." 125 | ${ECS_CLI} configure --region ${REGION} --cluster ${ECS_CLUSTER} 126 | if [ "${ECS_CLUSTERS}" == "" ]; then 127 | echo "Creating ECS cluster ${ECS_CLUSTER} ..." 128 | ${ECS_CLI} up --force --capability-iam --cidr ${ECS_SG_CIDR} 129 | else 130 | echo "ECS cluster ${ECS_CLUSTER} already exists" 131 | fi 132 | else 133 | if [ "${ECS_CLUSTERS}" == "" ]; then 134 | echo "ECS cluster ${ECS_CLUSTER} not found. Please set ECS_MANAGE_CLUSTER to true and execute run.sh to create the cluster, or create cluster manually first" 135 | exit 1 136 | else 137 | echo "ECS cluster ${ECS_CLUSTER} found" 138 | fi 139 | fi 140 | CONTAINER_ARN=$(aws ecs list-container-instances --cluster ${ECS_CLUSTER} --query 'containerInstanceArns[0]' --output text) 141 | while [ "$CONTAINER_ARN" == "None" ]; do 142 | echo "Waiting for container startup ..." 143 | sleep 10 144 | export CONTAINER_ARN=$(aws ecs list-container-instances --cluster ${ECS_CLUSTER} --query 'containerInstanceArns[0]' --output text) 145 | done 146 | echo "CONTAINER_ARN=$CONTAINER_ARN" 147 | if [ ! "${CONTAINER_ARN}" == "" ]; then 148 | export VPC_ID=$(aws ecs describe-container-instances --cluster default --container-instances ${CONTAINER_ARN} --query 'containerInstances[*].attributes[?name==`ecs.vpc-id`].value' --output text) 149 | echo "VPC_ID=${VPC_ID}" 150 | export SUBNET_ID=$(aws ecs describe-container-instances --cluster default --container-instances ${CONTAINER_ARN} --query 'containerInstances[*].attributes[?name==`ecs.subnet-id`].value' --output text) 151 | echo "SUBNET_ID=$SUBNET_ID" 152 | export SG_ID=$(aws ec2 describe-security-groups --filters Name=vpc-id,Values=${VPC_ID} --query 'SecurityGroups[?Description==`ECS Allowed Ports`].GroupId' --output text) 153 | echo "SG_ID=${SG_ID}" 154 | 155 | else 156 | echo "WARN: Could not find SUBNET_ID for cluster ${ECS_CLUSTER}. Please specify subnet manually in to/compose/template/ecs-params.yaml" 157 | fi 158 | } 159 | 160 | function prepare_batch_compute_environment 161 | { 162 | echo "Preparing Batch compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} ..." 163 | BATCH_COMPUTE_ENVIRONMENT_NAMES=$(aws batch describe-compute-environments --query 'computeEnvironments[*].computeEnvironmentName' --output text) 164 | echo ${BATCH_COMPUTE_ENVIRONMENT_NAMES} | grep -w -q ${BATCH_COMPUTE_ENVIRONMENT_NAME} 165 | if [ "$?" == "0" ]; then 166 | echo "Compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} exists" 167 | STATE=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].state' --output text) 168 | if [ ! "${STATE}" == "ENABLED" ]; then 169 | echo "Enabling compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} ..." 170 | CMD="aws batch update-compute-environment --compute-environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} --state ENABLED" 171 | RESULT=$(eval "${CMD}") 172 | fi 173 | else 174 | if [ "${BATCH_MANAGE_COMPUTE_ENVIRONMENT}" == "true" ]; then 175 | echo "Creating compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} ..." 176 | CMD="aws batch create-compute-environment --compute-environment-name ${BATCH_COMPUTE_ENVIRONMENT_NAME} --type MANAGED --compute-resources ${BATCH_COMPUTE_RESOURCES}" 177 | RESULT=$(eval "${CMD}") 178 | # Wait for compute environment to get created 179 | STATUS=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].status' --output text) 180 | while [ ! "${STATUS}" == "VALID" ]; do 181 | echo "Waiting for ${BATCH_COMPUTE_ENVIRONMENT_NAME} status to become VALID ..." 182 | sleep 2 183 | STATUS=$(aws batch describe-compute-environments --compute-environments ${BATCH_COMPUTE_ENVIRONMENT_NAME} --query 'computeEnvironments[*].status' --output text) 184 | done 185 | else 186 | echo "Compute environment ${BATCH_COMPUTE_ENVIRONMENT_NAME} not found!" 187 | exit 1 188 | fi 189 | fi 190 | } 191 | 192 | function prepare_batch_queue 193 | { 194 | echo "Preparing job queue ${BATCH_JOB_QUEUE_NAME} ..." 195 | BATCH_JOB_QUEUE_NAMES=$(aws batch describe-job-queues --query 'jobQueues[*].jobQueueName' --output text) 196 | echo ${BATCH_JOB_QUEUE_NAMES} | grep -w -q ${BATCH_JOB_QUEUE_NAME} 197 | if [ "$?" == "0" ]; then 198 | echo "Job queue ${BATCH_JOB_QUEUE_NAME} exists" 199 | else 200 | if [ "${BATCH_MANAGE_COMPUTE_ENVIRONMENT}" == "true" ]; then 201 | echo "Creating job queue ${BATCH_JOB_QUEUE_NAME} ..." 202 | CMD="aws batch create-job-queue --job-queue-name ${BATCH_JOB_QUEUE_NAME} --priority 1 --compute-environment-order order=0,computeEnvironment=${BATCH_COMPUTE_ENVIRONMENT_NAME}" 203 | RESULT=$(eval "${CMD}") 204 | # Wait for job queue to become valid 205 | STATUS=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].status' --output text) 206 | while [ ! "${STATUS}" == "VALID" ]; do 207 | echo "Waiting for ${BATCH_JOB_QUEUE_NAME} status to become VALID ..." 208 | sleep 2 209 | STATUS=$(aws batch describe-job-queues --job-queues ${BATCH_JOB_QUEUE_NAME} --query 'jobQueues[*].status' --output text) 210 | done 211 | else 212 | echo "Job queue ${BATCH_JOB_QUEUE_NAME} not found!" 213 | exit 1 214 | fi 215 | fi 216 | } 217 | 218 | function register_batch_job_definition 219 | { 220 | if [ "$1" == "" ]; then 221 | BATCH_JOB_COMMAND="${BATCH_COMMAND_DEFAULT}" 222 | else 223 | BATCH_JOB_COMMAND="$@" 224 | fi 225 | echo "Registering job definition ${BATCH_JOB_DEFINITION_NAME} ..." 226 | echo "BATCH_JOB_COMMAND=${BATCH_JOB_COMMAND}" 227 | if [ "${BATCH_COMPUTE_ENVIRONMENT_TYPE}" == "EC2" ]; then 228 | export BATCH_CONTAINER_PROPERTIES="image=${REGISTRY}${IMAGE}${TAG},vcpus=${BATCH_JOB_VCPUS},memory=${BATCH_JOB_MEMORY},jobRoleArn=${ECS_TASK_ROLE_ARN},executionRoleArn=${ECS_EXEC_ROLE_ARN},environment=\"${BATCH_JOB_ENV_VARS}\",command=${BATCH_JOB_COMMAND}" 229 | else 230 | export BATCH_CONTAINER_PROPERTIES="image=${REGISTRY}${IMAGE}${TAG},resourceRequirements=\"[{type=VCPU,value=${BATCH_JOB_VCPUS}},{type=MEMORY,value=${BATCH_JOB_MEMORY}}]\",jobRoleArn=${ECS_TASK_ROLE_ARN},executionRoleArn=${ECS_EXEC_ROLE_ARN},environment=\"${BATCH_JOB_ENV_VARS}\",command=${BATCH_JOB_COMMAND}" 231 | fi 232 | CMD="aws batch register-job-definition --type container --job-definition-name ${BATCH_JOB_DEFINITION_NAME} --platform-capabilities ${BATCH_COMPUTE_ENVIRONMENT_TYPE} --container-properties ${BATCH_CONTAINER_PROPERTIES}" 233 | if [ "${VERBOSE}" == "true" ]; then 234 | echo "${CMD}" 235 | fi 236 | if [ "${DRY_RUN}" == "false" ]; then 237 | RESULT=$(eval "${CMD}") 238 | fi 239 | } 240 | 241 | echo "" 242 | echo "Running container ${CONTAINER} on ${TO} ..." 243 | 244 | case "${TO}" in 245 | "compose") 246 | generate_compose_files 247 | CMD="${DOCKER_COMPOSE} -f ${COMPOSE_FILE} up -d" 248 | ;; 249 | "swarm") 250 | generate_compose_files 251 | CMD="docker stack deploy -c ${COMPOSE_FILE} ${SWARM_STACK_NAME}" 252 | ;; 253 | "ecs") 254 | prepare_ecs_roles 255 | prepare_ecs_cluster 256 | generate_compose_files 257 | COMPOSE_FILE=${ECS_COMPOSE_FILE} 258 | CMD="${ECS_CLI} compose --ecs-params ${ECS_PARAMS_FILE} --file ${ECS_COMPOSE_FILE} create --launch-type ${ECS_LAUNCH_TYPE}" 259 | if [ "${VERBOSE}" == "true" ]; then 260 | echo "${CMD}" 261 | fi 262 | 263 | if [ "${DRY_RUN}" == "false" ]; then 264 | TASK_DEFINITION_RESULT=$(eval "${CMD}") 265 | echo "" 266 | fi 267 | CMD="aws ecs run-task --cluster ${ECS_CLUSTER} --task-definition compose --enable-execute-command --launch-type ${ECS_LAUNCH_TYPE}" 268 | if [ "${ECS_LAUNCH_TYPE}" == "FARGATE" ]; then 269 | CMD="$CMD --network-configuration \"awsvpcConfiguration={subnets=[${SUBNET_ID}],securityGroups=[${SG_ID}],assignPublicIp=${ECS_ASSIGN_PUBLIC_IP}}\"" 270 | fi 271 | ;; 272 | "kubernetes") 273 | generate_kubernetes_manifests 274 | CMD="${KUBECTL} -n ${NAMESPACE} apply -f ${KUBERNETES_APP_PATH}" 275 | ;; 276 | "lambdalocal") 277 | CMD="docker run ${RUN_OPTS} ${CONTAINER_NAME} ${MODE} ${NETWORK} -p ${PORT_EXTERNAL}:8080 ${REGISTRY}${IMAGE}${TAG} $@" 278 | ;; 279 | "lambda") 280 | generate_lambda_files 281 | prepare_lambda_role 282 | sleep 5 # wait for lambda role to finish creating 283 | CMD="aws lambda create-function --region ${REGION} --function-name ${LAMBDA_FUNCTION_NAME} --package-type Image --code ImageUri=${REGISTRY}${IMAGE}${TAG} --role ${LAMBDA_ROLE_ARN}" 284 | ;; 285 | "batchlocal") 286 | if [ "$1" == "" ]; then 287 | BATCH_COMMAND=${BATCH_COMMAND_DEFAULT} 288 | else 289 | BATCH_COMMAND="$@" 290 | fi 291 | CMD="docker container run ${RUN_OPTS} ${CONTAINER_NAME} -d ${NETWORK} ${VOL_MAP} ${REGISTRY}${IMAGE}${TAG} ${BATCH_COMMAND}" 292 | ;; 293 | "batch") 294 | prepare_ecs_roles 295 | prepare_batch_compute_environment 296 | prepare_batch_queue 297 | register_batch_job_definition "$@" 298 | CMD="aws batch submit-job --job-name ${BATCH_JOB_NAME} --job-queue ${BATCH_JOB_QUEUE_NAME} --job-definition ${BATCH_JOB_DEFINITION_NAME}" 299 | ;; 300 | *) 301 | checkTO "${TO}" 302 | CMD="docker container run ${RUN_OPTS} ${CONTAINER_NAME} ${MODE} ${NETWORK} ${PORT_MAP} ${VOL_MAP} ${REGISTRY}${IMAGE}${TAG} $@" 303 | ;; 304 | esac 305 | 306 | if [ "${VERBOSE}" == "true" ]; then 307 | echo "${CMD}" 308 | fi 309 | 310 | if [ "${DRY_RUN}" == "false" ]; then 311 | RESULT=$(eval "${CMD}") 312 | echo "" 313 | fi 314 | 315 | if [ "${DEBUG}" == "true" ]; then 316 | set +x 317 | fi 318 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS DevOps for Docker (aws-do-docker) 2 | 3 | This project provides a simple pattern for building and deployment of Docker containers to AWS. It can be used "as is" to build and run a sample container, or can be customized and used to build, test, and deploy containerized microservices, jobs, or serverless functions. 4 | 5 | Containers built using this project can be run locally or on [EC2](https://aws.amazon.com/ec2), [ECS](https://aws.amazon.com/ecs), [EKS](https://aws.amazon.com/eks), [Fargate](https://aws.amazon.com/fargate), [AWS Batch](https://aws.amazon.com/batch), or [AWS Lambda](https://aws.amazon.com/lambda) just by configuring the target orchestrator ( [`./config.sh TO`](config.sh) ) and executing [`./run.sh`](run.sh). 6 | 7 |
workflow
8 |
Fig. 1 - Container workflow
9 | 10 | ## Prerequisites 11 | The following are minimum requirements to use this project: 12 | * [`bash` compatible Unix shell](https://en.wikipedia.org/wiki/Unix_shell) - required for running control scripts 13 | * [`Docker`](https://docs.docker.com/get-docker/) - Docker Desktop or Docker for Linux v 19.03 or newer 14 | * [`aws-cli v2`](install/install-aws-cli.sh) - install using script [`install/install-aws-cli.sh`](install/install-aws-cli.sh) 15 | * `aws configure` - provide credentials for your aws command line client 16 | 17 | Additional requirements are determined by your selection of a target orchestrator (TO). Install the tools corresponding to each container orchestrator you plan to use as described below: 18 | * [`docker-compose`](install/install-docker-compose.sh) - required when running in Docker Compose or Docker Swarm 19 | * [`ecs-cli`](install/install-ecs-cli.sh) - required when deploying to ECS 20 | * [`ssm-plugin`](install/install-ssm-plugin.sh) - optional, needed to enable exec into container running on ECS 21 | * [`kubectl`](install/install-kubectl.sh) - required when deploying to EKS 22 | * ~/.kube/config - it is assumed that a Kubernetes cluster exists and the local host is configured to access it via kubectl 23 | 24 | ## Using the project 25 | To use this repository without modification, just clone it from Git, then execute the [`./build.sh`](./build.sh) script. To verify that the sample container works as expected, execute [`./run.sh`](./run.sh), then [`./test.sh`](./test.sh) and [`./logs.sh`](./logs.sh). 26 | 27 | The following logs indicate the container works correctly: 28 | ``` 29 | Showing logs from container parrot-docker on docker ... 30 | Container-Root/startup.sh executed 31 | Thu May 13 05:02:41 UTC 2021 32 | [2021-05-13 05:02:41 +0000] [8] [INFO] Running on http://0.0.0.0:80 (CTRL + C to quit) 33 | Received healthcheck request 34 | Returning {'Status': 'Healthy'} 35 | Received say request: Nice demo! 36 | Returning {'Message': 'Nice demo!'} 37 | ``` 38 | Execute the [`./stop.sh`](./stop.sh) script to remove the container. 39 | 40 | To use this repository as a starting point for your own container project, clone the repo and name it accordingly. Then execute [`./config.sh -i`](./config.sh) to review and set the basic project settings interactively. For each setting, the current value will be displayed. You may type a new value, or just press `Enter` to preserve the current one. 41 | 42 | We recommend developing changes locally and testing them on a target orchestrator that runs the container on the local machine before deploying it elsewhere. Local target orchestrators are `docker`,`compose`,`lambdalocal`,and `batchlocal`. 43 | 44 | To customize your container, iteratively modify or add files in the `Conteiner-Root` folder. The content of this folder is added to the root of your Docker image. Use [`./build.sh`](build.sh) to rebuild your container image, [`./run.sh`](run.sh), [`./status.sh`](status.sh), [`./exec.sh`](exec.sh), [`./test.sh`](test.sh), [`./logs.sh`](logs.sh) to start and interact with the container, and [`./stop.sh`](stop.sh) to stop and remove it. For more details, please see the "Customizing the project"(#customizing-the-project) section. 45 | 46 | When you'd like to deploy the container to a remote environment, push the container image to a container registry by executing the [`./login.sh`](login.sh) and [`./push.sh`](push.sh) scripts, then configure the target orchestrator where you would like to run the container as follows: 47 | 48 | ``` 49 | local:~/environment/demo $ ./config.sh TO 50 | 51 | ## Target orchestrator TO=docker|compose|ecs|swarm|kubernetes|eks|lambda|batchlocal|batch 52 | ## docker - choose when running locally or on ec2 53 | ## compose - choose when running locally, on ec2 54 | ## ecs - choose when running on ecs with or without Fargate 55 | ## swarm - choose when running on a docker swarm 56 | ## kubernetes - choose when running on a local, remote Kubernetes cluster or EKS, with or without Fargate 57 | ## lambdalocal - choose when developing and testing an AWS Lambda container locally 58 | ## lambda - choose when running containerized function on AWS Lambda 59 | ## batchlocal - choose when running containerized jobs locally 60 | ## batch - choose when running containerized jobs on AWS Batch 61 | export TO=docker 62 | 63 | Set TO [ docker ] to: 64 | ecs 65 | 66 | New value: 67 | export TO=ecs 68 | ``` 69 | 70 | For each of the target orchestrators listed above, you may use the same sequence of commands: 71 | 72 | [`./config.sh TO`](config.sh), [`./run.sh`](run.sh), [`./status.sh`](status.sh), [`./test.sh`](test.sh), [`./logs.sh`](logs.sh), [`./stop.sh`](stop.sh) 73 | 74 | The workflow on each target orchestrator looks similar to the demo below: 75 | 76 |
demo
77 |
Fig. 2 - Demo - running and testing sample container on EC2 and EKS
78 | 79 | To run this project on distributed AWS container orchestrators, such as [ECS](https://aws.amazon.com/ecs), [Lambda](https://aws.amazon.com/lambda), [Batch](https://aws.amazon.com/batch), and [EKS](https://aws.amazon.com/eks), the credentials provided to the AWS CLI must have sufficient privileges. See the "[Advanced settings](#advanced-settings)" section for more information on how to enable `Fargate` for target orchestrators `ecs`, `batch`, or `kubernetes`. 80 | 81 | ## Customizing the project 82 | 83 | ### Project structure 84 | 85 | The project has the following structure, which can be reused or extended for containerizing other microservices/apps, functions, or jobs. 86 | 87 | 88 | ``` 89 | . 90 | ├── build.sh 91 | ├── config.sh 92 | ├── Container-Root 93 | │   ├── app 94 | │   ├── function 95 | │   ├── job 96 | │   ├── requirements.txt 97 | │   ├── setup.sh 98 | │   ├── startup.sh 99 | │   ├── test1.sh 100 | │   └── test2.sh 101 | ├── Dockerfile 102 | ├── Dockerfile-lambda 103 | ├── docs 104 | │   └── img 105 | ├── exec.sh 106 | ├── install 107 | │   ├── install-aws-cli.sh 108 | │   ├── install-docker-compose.sh 109 | │   ├── install-ecs-cli.sh 110 | │   ├── install-kubectl.sh 111 | │   └── install-ssm-plugin.sh 112 | ├── LICENSE 113 | ├── login.sh 114 | ├── logs.sh 115 | ├── pull.sh 116 | ├── push.sh 117 | ├── README.md 118 | ├── run.sh 119 | ├── status.sh 120 | ├── stop.sh 121 | ├── test.sh 122 | └── to 123 | ├── compose 124 | ├── kubernetes 125 | └── lambda 126 | ``` 127 | 128 | The main folder contains the configuration file [`.env`](.env), the control scripts `*.sh` and Dockerfiles. 129 | The install folder contains scripts to facilitate installation of dependent tools as needed, based on your target orchestrator. 130 | The `Container-Root` folder hosts the content that goes inside your container. Files and folder structures from this location get copied into the root of the container image. Implementation of different container functionality typically involves adding files or modifying content within this folder. 131 | 132 | Files that typically get modified in the `Container-Root` folder include: 133 | * [`setup.sh`](Container-Root/setup.sh) - this file gets executed at container build time and is responsible for installing software within your container as needed. 134 | * [`startup.sh`](Container-Root/startup.sh) - this file gets executed as a command at start time and is responsible for launching the main process in your container. 135 | * `test*.sh` - unit tests to verify your container works as expected 136 | * `app/*` - application files for service containers 137 | * `function/*` - function code, startup and test files for lambda-based containers 138 | * `job/*` - job code, startup and test files for containerized jobs 139 | 140 | To customize the deployment of your container for one of the supported target orchestrators, you can modify the content of the corresponding `to//template` folder. You can use the environment variable names that are defined in [`.env`](.env) within your template files. The environment variables get substituted at launch time and produce the files placed in the corresponding `to//app` folder. 141 | 142 | ### Control scripts 143 | 144 | The project contains the following scripts: 145 | 146 | * [`config.sh`](config.sh) - configure project settings 147 | * [`build.sh`](build.sh) - build the container image 148 | * [`test.sh`](test.sh) - run container unit tests 149 | * [`push.sh`](push.sh) - push the container image to a registry 150 | * [`pull.sh`](pull.sh) - pull the container image from a registry 151 | * [`run.sh [cmd]`](run.sh) - run the container, passing an argument overrides the default command 152 | * [`status.sh`](status.sh) - show container status - running, exited, etc. 153 | * [`logs.sh`](logs.sh) - tail container logs 154 | * [`exec.sh [cmd]`](exec.sh) - open a shell or execute a specified command in the running container 155 | 156 | ### Configuration 157 | 158 | All settings in the project are centralized in the [`.env`](.env) configuration file. The default settings can be used without any modifications to build and run a sample Docker container locally. 159 | 160 | Configuration is managed through the [`./config.sh`](config.sh) script. 161 | 162 | ``` 163 | Usage: ./config.sh [setting] [value] 164 | 165 | ./config.sh --help - display usage information 166 | ./config.sh -i - edit selected settings interactively 167 | ./config.sh - edit configuration file 168 | ./config.sh setting - edit specified setting interactively 169 | ./config.sh setting value - set the specified setting to the provided value 170 | ``` 171 | 172 | #### Sections 173 | 174 | The [`.env`](.env) file contains the following sections: 175 | * Proxy settings - if you are working behind a corporate proxy, these settings allow your container to connect to the Internet. 176 | * Project settings - project-wide settings to specify Target Orchestrator (TO), verbosity, debug flag, etc. 177 | * Docker image settings - define image name, docker registry, build options, etc 178 | * Target orchestrator settings - common and TO specific settings in corresponding subsections 179 | 180 | #### Basic settings 181 | To configure basic settings for the project interactively, execute [`./config.sh -i`](config.sh). You will be asked to set the following: 182 | * `TO`=docker|compose|ecs|swarm|kubernetes|lambdalocal|lambda|batchlocal|batch 183 | * `BASE_IMAGE_PATH` - base image from which your project's image will be built 184 | * `REGISTRY` - docker registry to which your image will be pushed 185 | * `IMAGE_NAME` - name of your image 186 | * `VERSION` - your docker image version tag 187 | 188 | #### Advanced settings 189 | * `PORT_INTERNAL` - if your container runs a server process, this is the port that process listens on inside the container 190 | * `PORT_EXTERNAL` - if the container listens on an internal port, this is the corresponding port exposed outside of the continer 191 | * `VOLUME_PATH_EXTERNAL` - external path that you wish to mount into the container, used primarily in local environments 192 | * `VOLUME_PATH_INTERNAL` - path inside the container that the external volume is mounted on 193 | * `REGION` - AWS region to use for deploying your container 194 | * `ECS_LAUNCH_TYPE`=EC2|FARGATE - select whether to use EC2 or FARGATE when deploying container to ECS 195 | * `BATCH_COMPUTE_ENVIRONMENT_TYPE`=EC2|FARGATE - select whether to use EC2 or FARGATE when deploying to AWS Batch 196 | * `NAMESPACE` - configure EC2 managed or FARGATE profile namespace when deploying container to EKS 197 | 198 | For reviewing and editing of all possible project settings, execute [`./config.sh`](config.sh) to open the [`.env`](.env) file in a text editor. 199 | 200 | #### GPU Support 201 | 202 | To run GPU workloads, the following additional settings need to be defined depending on your target orchestrator. 203 | 204 | EC2/docker: 205 | * The EC2 instance on which the container runs should be provisioned with GPU resources. Please refer to the Accelerated Computing section of the [Amazon EC2 Instance Types](https://aws.amazon.com/ec2/instance-types/) page for selection of the desired GPU type. 206 | * For Docker to make GPUs available within the container, the --gpus option must be specified in the docker run command. To achieve that, add the option to the `RUN_OPTS` setting in the configuration file `.env`. Examples: `--gpus all` or `--gpus 0` 207 | 208 | ECS: 209 | * [Provision the ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html) with [EC2 launch type](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_types.html#launch-type-ec2) through the AWS console and specify the desired GPU instance type 210 | * Set the `ECS_MANAGE_CLUSTER` value to `false` in the configuration file `.env`. This will allow the project to use an existing ECS cluster, instead of creating one 211 | * Set the `ECS_CLUSTER` value to the name of your GPU-enabled ECS cluster 212 | * Modify the ECS task template [to/compose/template/ecs-params-EC2.yaml](to/compose/template/ecs-params-EC2.yaml) to include a GPU resource requirement `gpu: 1` as described in the [Amazon ECS parameters schema](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-ecsparams.html) 213 | 214 | Batch: 215 | * [Provision the AWS Batch Compute Environment using EC2 resources](https://docs.aws.amazon.com/batch/latest/userguide/create-compute-environment.html#create-compute-environment-managed-ec2) through the AWS console and specify the desired GPU instance type 216 | * Set the `BATCH_MANAGE_COMPUTE_ENVIRONMENT` value to `false` in the configuration file `.env`. This will allow the project to use an existing compute environment, instead of creating one 217 | * Set the `BATCH_COMPUTE_ENVIRONMENT_NAME` value to the name of your GPU-enabled compute environment 218 | * Edit script [`./run.sh`](run.sh) (~line 223), modify variable `BATCH_CONTAINER_PROPERTIES` to include `resourceRequirements=\"[{type=GPU,value=1}]\"`. For more information about resourceRequirements and other job definition settings, refer to the [register-job-definition](https://docs.aws.amazon.com/cli/latest/reference/batch/register-job-definition.html) API documentation 219 | 220 | EKS/kubernetes: 221 | * In your EKS cluster [provision a node group](https://docs.aws.amazon.com/eks/latest/userguide/create-managed-node-group.html) of instances with the desired GPU type 222 | * Modify the deployment template [`to/kubernetes/template/020-deployment.yaml`](to/kubernetes/template/020-deployment.yaml), uncomment lines 32-36, specifying container resource limit `nvidia.com/gpu:`, and set the desired number of GPUs for your container. 223 | 224 | ## License 225 | This repository is released under the MIT-0 License. See the [LICENSE](LICENSE) file for details. 226 | 227 | ## References 228 | 229 | * [Docker](https://docker.com) 230 | * [Docker Compose](https://docs.docker.com/compose/) 231 | * [Docker Swarm](https://docs.docker.com/engine/swarm/) 232 | * [Kubernetes](https://kubernetes.io) 233 | * [Amazon Web Services (AWS)](https://aws.amazon.com/) 234 | * [Amazon EC2](https://aws.amazon.com/ec2) 235 | * [Amazon EC2 Instance Types](https://aws.amazon.com/ec2/instance-types/) 236 | * [AWS Lambda](https://aws.amazon.com/lambda/) 237 | * [Amazon Elastic Container Service (ECS)](https://aws.amazon.com/ecs) 238 | * [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks) 239 | * [AWS Batch](https://aws.amazon.com/batch) 240 | * [AWS Fargate](https://aws.amazon.com/fargate) 241 | * [Depend on Docker Project](https://github.com/iankoulski/depend-on-docker) 242 | * [DockerCon 2021 Presentation](https://www.docker.com/dockercon-live/2021/content/Videos/qYmjYNgD27hS8559G) 243 | --------------------------------------------------------------------------------