├── .gitignore ├── README.md ├── circle.yml └── fargate ├── Dockerfile ├── README.md ├── application ├── .gitignore ├── Dockerfile ├── Gopkg.lock ├── Gopkg.toml ├── deploy │ ├── cfn-master.yaml │ ├── cfn-release.yaml │ ├── edge │ │ └── buildspec.yml │ ├── production │ │ └── buildspec.yml │ └── staging │ │ └── buildspec.yml ├── diff │ └── iam.yaml └── main.go ├── infrastructure ├── .gitignore ├── diff │ ├── orig.yaml │ ├── production │ └── staging ├── notification │ └── index.js └── sam.yaml ├── notebooks ├── 00-overview.ipynb ├── 01-provision-aws-resources.ipynb ├── 02-develop-with-git.ipynb ├── 03-deploy-to-staging.ipynb ├── 04-deploy-to-production.ipynb ├── 05-teardown-resources.ipynb └── notebook-assets │ ├── app-1.png │ ├── infra-1.png │ ├── infra-2.png │ ├── whole-arch.png │ ├── workflow-1.png │ ├── workflow-2.png │ └── workflow-3.png └── notebooks_config ├── entrypoint.sh └── jupyter_notebook.py /.gitignore: -------------------------------------------------------------------------------- 1 | .env 2 | env.sh 3 | .ipynb_checkpoints 4 | packaged.yaml 5 | vendor 6 | docker-compose.yml 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ハンズオン 2 | 3 | [![CircleCI](https://circleci.com/gh/jawsug-container/hands-on.svg?style=svg)](https://circleci.com/gh/jawsug-container/hands-on) 4 | 5 | [![jawsug/container](http://dockeri.co/image/jawsug/container)](https://hub.docker.com/r/jawsug/container/) 6 | 7 | Supported tags and respective `Dockerfile` links: 8 | 9 | ・fargate-handson ([fargate/Dockerfile](https://github.com/jawsug-container/hands-on/blob/master/fargate/Dockerfile)) 10 | 11 | ## 前提条件 12 | 13 | - 端末（Windows, Mac, ..）で Docker が利用できること 14 | - 利用可能な AWS アカウント、Administrator 権限のある IAM ユーザ、そのアクセスキーがあること 15 | 16 | ### Docker が利用できること 17 | 18 | Docker Store からインストーラーをダウンロードし、お手もとの環境へ Docker をインストールしてください。 19 | Windows へのインストールや動作確認がうまくできない場合は、EC2 の利用をご検討ください。 20 | （EC2 を利用する場合、パブリック IP アドレスの取得、8080 番ポートの解放が必要です） 21 | 22 | - Mac は [こちら](https://store.docker.com/editions/community/docker-ce-desktop-mac) 23 | - Windows は [こちら](https://store.docker.com/editions/community/docker-ce-desktop-windows) 24 | - EC2 は [こちら](https://docs.aws.amazon.com/ja_jp/AmazonECS/latest/developerguide/docker-basics.html) 25 | 26 | 以下のコマンドで、Client と Server の情報が正常に返ってくることを確認してください。 27 | 28 | ``` 29 | $ docker version 30 | ``` 31 | 32 | 応答例） 33 | 34 | ``` 35 | Client: 36 | Version: 18.03.1-ce 37 | API version: 1.37 38 | Go version: go1.9.5 39 | Git commit: 9ee9f40 40 | Built: Thu Apr 26 07:13:02 2018 41 | OS/Arch: darwin/amd64 42 | Experimental: false 43 | Orchestrator: swarm 44 | 45 | Server: 46 | Engine: 47 | Version: 18.03.1-ce 48 | API version: 1.37 (minimum version 1.12) 49 | Go version: go1.9.5 50 | Git commit: 9ee9f40 51 | Built: Thu Apr 26 07:22:38 2018 52 | OS/Arch: linux/amd64 53 | Experimental: true 54 | ``` 55 | 56 | ### AWS の準備ができていること 57 | 58 | #### 1. AWS アカウントの開設 59 | 60 | 以下のサイトを参考に、AWS アカウントを用意してください。 61 | https://aws.amazon.com/jp/register-flow/ 62 | 63 | #### 2. 重要！「AWS 利用開始時に最低限おさえておきたい 10 のこと」の確認 64 | 65 | 以下のスライドを読み、必要に応じて初期設定を変更してください。 66 | https://www.slideshare.net/AmazonWebServicesJapan/20180403-aws-white-belt-online-seminar-aws10 67 | 68 | #### 3. Administrator 権限のある IAM ユーザと、そのアクセスキーの発行 69 | 70 | IAM（Identity and Access Management）とは、AWS でのユーザーや権限を管理するサービスです。 71 | AWS をより安全に利用するために、初期設定されたルートユーザーとは別に、IAM ユーザーを用意します。 72 | 73 | 3.1. IAM 管理者ユーザーを作成します 74 | 75 | https://docs.aws.amazon.com/ja_jp/IAM/latest/UserGuide/getting-started_create-admin-group.html 76 | 77 | 3.2. アクセスキーを発行します 78 | 79 | 発行したアクセスキー・シークレットキーはどこか安全ば場所に保管しておいてください。 80 | インターネット上に流出させないよう、[git-secrets](https://github.com/awslabs/git-secrets) などをご検討ください。 81 | https://docs.aws.amazon.com/ja_jp/IAM/latest/UserGuide/id_credentials_access-keys.html 82 | 83 | #### 4. EC2 を起動・停止する 84 | 85 | AWS では開設直後のアカウントの場合、仮想サーバーの起動数などに制限がかかっていることがあります。 86 | 以下の手順に従い、 **t2.micro** でのインスタンス起動、設定、接続、および終了を一度実行してください。 87 | 88 | https://aws.amazon.com/jp/getting-started/tutorials/launch-a-virtual-machine/ 89 | 90 | 91 | ## コンテンツ 92 | 93 | - [Fargate アプリケーションの継続的デリバリー](https://github.com/jawsug-container/hands-on/blob/master/fargate/README.md) 94 | -------------------------------------------------------------------------------- /circle.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | jobs: 3 | build: 4 | docker: 5 | - image: docker:18.06.0-ce-git 6 | steps: 7 | - checkout 8 | - setup_remote_docker 9 | - run: 10 | name: Install dependencies 11 | command: | 12 | apk add --no-cache py-pip 13 | pip install awscli 14 | - run: 15 | name: Run tests 16 | command: | 17 | docker run --rm -it -v $(pwd):/work -e IGNORE_PATH=/vendor/ supinf/shellcheck 18 | -------------------------------------------------------------------------------- /fargate/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM docker:18.06.0-ce-dind 2 | 3 | ENV AWSCLI_VERSION=1.15.66 \ 4 | JUPYTER_VERSION=1.0.0 5 | 6 | RUN apk --no-cache add python3 git 7 | 8 | # Install AWS CLI 9 | RUN apk --no-cache add groff less jq \ 10 | && apk --no-cache add --virtual build-deps py3-pip \ 11 | && pip3 install "awscli == ${AWSCLI_VERSION}" \ 12 | && pip3 install yq \ 13 | && find / -type d -name \__pycache__ -depth -exec rm -rf {} \; \ 14 | && rm -rf /root/.cache \ 15 | && apk del --purge -r build-deps 16 | 17 | # Install Jupyter notebook 18 | RUN apk --no-cache add bash tini \ 19 | && apk --no-cache add --virtual build-deps build-base python3-dev \ 20 | && pip3 install "jupyter == ${JUPYTER_VERSION}" \ 21 | && pip3 install backcall bash_kernel \ 22 | && python3 -m bash_kernel.install \ 23 | && find / -type d -name tests -depth -exec rm -rf {} \; \ 24 | && find / -type d -name \__pycache__ -depth -exec rm -rf {} \; \ 25 | && rm -rf /root/.cache 26 | 27 | # Configure git to use CodeCommit as git repositories 28 | RUN git config --global credential.helper '!aws codecommit credential-helper $@' \ 29 | && git config --global credential.UseHttpPath true 30 | 31 | WORKDIR /root/notebook 32 | ADD notebooks_config/jupyter_notebook.py /root/.jupyter/jupyter_notebook_config.py 33 | ADD notebooks /root/notebook 34 | ADD application /root/notebook/application 35 | ADD infrastructure /root/notebook/infrastructure 36 | 37 | ADD notebooks_config/entrypoint.sh / 38 | RUN chmod +x /entrypoint.sh 39 | 40 | VOLUME /root/config 41 | 42 | ENTRYPOINT ["/entrypoint.sh"] 43 | CMD ["tini", "--", "jupyter", "notebook"] 44 | -------------------------------------------------------------------------------- /fargate/README.md: -------------------------------------------------------------------------------- 1 | # Fargate アプリケーションの継続的デリバリー 2 | 3 | この git リポジトリを clone する必要はありませんし、任意のディレクトリで動作します。 4 | Docker クライアントを操作できる端末を起動し、以下のステップを実行してください。 5 | 6 | ## ハンズオン環境の起動 7 | 8 | ### 1. アクセスキーをそれぞれ変数に設定します 9 | 10 | ``` 11 | $ export AWS_ACCESS_KEY_ID=<あなたの AWS アクセスキー> 12 | $ export AWS_SECRET_ACCESS_KEY=<あなたの AWS シークレットキー> 13 | ``` 14 | 15 | ### 2. git を扱うためのユーザー名とメールアドレス（任意）を変数に設定します 16 | 17 | このメールアドレスには、ハンズオンの中で、パイプライン実行のためのメールが送信されます。 18 | 19 | ``` 20 | $ export GIT_USER_NAME=<あなたの git ユーザー名（任意）> 21 | $ export GIT_EMAIL_ADDRESS=<あなたの git ユーザーメールアドレス（有効なもの）> 22 | ``` 23 | 24 | ### 3. ハンズオン環境の設定置き場を作ります 25 | 26 | ``` 27 | $ docker volume create fargate-handson 28 | ``` 29 | 30 | ### 4. ハンズオン環境を起動します 31 | 32 | ``` 33 | $ docker run --rm -it -e AWS_DEFAULT_REGION=ap-northeast-1 \ 34 | -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \ 35 | -e GIT_USER_NAME -e GIT_EMAIL_ADDRESS \ 36 | -v /var/run/docker.sock:/var/run/docker.sock \ 37 | -v fargate-handson:/root/config \ 38 | -p 8080:8080 jawsug/container:fargate-handson 39 | ``` 40 | 41 | ### 5. ブラウザで環境に接続します 42 | 43 | ブラウザで http://localhost:8080 を開いてください。 44 | （EC2 を利用している場合は、localhost を EC2 のパブリック IP アドレスに読み替えてください） 45 | 46 | パスワードを聞かれるので **jawsug** と入力してください。 47 | 48 | ### 6. ハンズオンの実施 49 | 50 | ハンズオンはブラウザ内の Jupyter notebook で行います。 51 | 以下の順序でハンズオンを進めてください。 52 | 53 | ``` 54 | - 00-overview.ipynb 55 | - 01-provision-aws-resources.ipynb 56 | - 02-develop-with-git.ipynb 57 | - 03-deploy-to-staging.ipynb 58 | - 04-deploy-to-production.ipynb 59 | - 05-teardown-resources.ipynb 60 | ``` 61 | 62 | ### 7. 後片付け 63 | 64 | `Ctrl + C` でコンテナに停止シグナルを送ると、Jupyter notebook から 65 | `Shutdown this notebook server (y/[n])?` と聞かれます。 66 | `y` と入力してコンテナを終了しましょう。 67 | 68 | 最後に設定を保存したボリュームを削除します。 69 | 70 | ``` 71 | $ docker volume rm fargate-handson 72 | ``` 73 | 74 | ## 参考 75 | 76 | - [AWS 公式: startup-kit-templates](https://github.com/aws-samples/startup-kit-templates/) 77 | - [AWS 公式: codepipeline-nested-cfn](https://github.com/aws-samples/codepipeline-nested-cfn/) 78 | -------------------------------------------------------------------------------- /fargate/application/.gitignore: -------------------------------------------------------------------------------- 1 | vendor 2 | -------------------------------------------------------------------------------- /fargate/application/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM dockercloud/hello-world 2 | 3 | RUN sed -i -e "s/world/Fargate/g" /www/index.php 4 | RUN sed -i -e "s/hostname/project ID/g" /www/index.php 5 | RUN sed -i -e "s/HOSTNAME/PROJECT_ID/g" /www/index.php 6 | -------------------------------------------------------------------------------- /fargate/application/Gopkg.lock: -------------------------------------------------------------------------------- 1 | # This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'. 2 | 3 | 4 | [[projects]] 5 | digest = "1:354011ee62a1d69d78dc7558813b10e8692e35087d1cc68ef0f18da54d3c6509" 6 | name = "github.com/aws/aws-sdk-go-v2" 7 | packages = [ 8 | "aws", 9 | "aws/awserr", 10 | "aws/defaults", 11 | "aws/ec2metadata", 12 | "aws/ec2rolecreds", 13 | "aws/endpointcreds", 14 | "aws/endpoints", 15 | "aws/external", 16 | "aws/signer/v4", 17 | "aws/stscreds", 18 | "internal/awsutil", 19 | "internal/sdk", 20 | "private/protocol", 21 | "private/protocol/json/jsonutil", 22 | "private/protocol/jsonrpc", 23 | "private/protocol/query", 24 | "private/protocol/query/queryutil", 25 | "private/protocol/rest", 26 | "private/protocol/xml/xmlutil", 27 | "service/sts", 28 | "service/translate", 29 | ] 30 | pruneopts = "UT" 31 | revision = "ff1a530c31507c97cf5edbee226e604ca08661cc" 32 | version = "v2.0.0-preview.4" 33 | 34 | [[projects]] 35 | digest = "1:e361cb646a33874cf6521b5c4e77c9208b4b58c7fd3893b8c87f61f49b19d991" 36 | name = "github.com/caarlos0/env" 37 | packages = ["."] 38 | pruneopts = "UT" 39 | revision = "1cddc31c48c56ecd700d873edb9fd5b6f5df922a" 40 | version = "v3.3.0" 41 | 42 | [[projects]] 43 | digest = "1:fe8a03a8222d5b913f256972933d26d24ad7c8286692a42943bc01633cc8fce3" 44 | name = "github.com/go-ini/ini" 45 | packages = ["."] 46 | pruneopts = "UT" 47 | revision = "358ee7663966325963d4e8b2e1fbd570c5195153" 48 | version = "v1.38.1" 49 | 50 | [[projects]] 51 | digest = "1:e22af8c7518e1eab6f2eab2b7d7558927f816262586cd6ed9f349c97a6c285c4" 52 | name = "github.com/jmespath/go-jmespath" 53 | packages = ["."] 54 | pruneopts = "UT" 55 | revision = "0b12d6b5" 56 | 57 | [solve-meta] 58 | analyzer-name = "dep" 59 | analyzer-version = 1 60 | input-imports = [ 61 | "github.com/aws/aws-sdk-go-v2/aws", 62 | "github.com/aws/aws-sdk-go-v2/aws/endpoints", 63 | "github.com/aws/aws-sdk-go-v2/aws/external", 64 | "github.com/aws/aws-sdk-go-v2/service/translate", 65 | "github.com/caarlos0/env", 66 | ] 67 | solver-name = "gps-cdcl" 68 | solver-version = 1 69 | -------------------------------------------------------------------------------- /fargate/application/Gopkg.toml: -------------------------------------------------------------------------------- 1 | # Gopkg.toml example 2 | # 3 | # Refer to https://golang.github.io/dep/docs/Gopkg.toml.html 4 | # for detailed Gopkg.toml documentation. 5 | # 6 | # required = ["github.com/user/thing/cmd/thing"] 7 | # ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"] 8 | # 9 | # [[constraint]] 10 | # name = "github.com/user/project" 11 | # version = "1.0.0" 12 | # 13 | # [[constraint]] 14 | # name = "github.com/user/project2" 15 | # branch = "dev" 16 | # source = "github.com/myfork/project2" 17 | # 18 | # [[override]] 19 | # name = "github.com/x/y" 20 | # version = "2.4.0" 21 | # 22 | # [prune] 23 | # non-go = false 24 | # go-tests = true 25 | # unused-packages = true 26 | 27 | 28 | [[constraint]] 29 | name = "github.com/aws/aws-sdk-go-v2" 30 | version = "2.0.0-preview.4" 31 | 32 | [[constraint]] 33 | name = "github.com/caarlos0/env" 34 | version = "3.3.0" 35 | 36 | [prune] 37 | go-tests = true 38 | unused-packages = true 39 | -------------------------------------------------------------------------------- /fargate/application/deploy/cfn-master.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Description: "A fargate application" 3 | 4 | Outputs: 5 | Cluster: 6 | Description: the ECS cluster name 7 | Value: !Ref Cluster 8 | 9 | Metadata: 10 | AWS::CloudFormation::Interface: 11 | ParameterGroups: 12 | - Label: 13 | default: Basic Configurations 14 | Parameters: 15 | - ProjectID 16 | - Label: 17 | default: Application Configurations 18 | Parameters: 19 | - DockerImage 20 | - BasicAuthUsername 21 | - BasicAuthPassword 22 | - ContainerCpu 23 | - ContainerMemory 24 | 25 | Parameters: 26 | ProjectID: 27 | Type: String 28 | 29 | DockerImage: 30 | Type: String 31 | 32 | BasicAuthUsername: 33 | Type: String 34 | Default: "" 35 | 36 | BasicAuthPassword: 37 | Type: String 38 | Default: "" 39 | NoEcho: true 40 | 41 | ContainerCpu: 42 | Type: Number 43 | Description: Amount of CPU for the container 44 | Default: 256 45 | MinValue: 256 46 | MaxValue: 4096 47 | ConstraintDescription: "Value must be between 256 and 4096 - see: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size" 48 | 49 | ContainerMemory: 50 | Type: Number 51 | Description: Amount of memory for the container 52 | Default: 512 53 | MinValue: 512 54 | MaxValue: 30720 55 | ConstraintDescription: "Value must be between 512 and 30720 - see: https://aws.amazon.com/fargate/pricing/" 56 | 57 | Resources: 58 | 59 | # Fargate のコントロールプレインとなる ECS クラスタ 60 | Cluster: 61 | Type: AWS::ECS::Cluster 62 | Properties: 63 | ClusterName: !Ref AWS::StackName 64 | 65 | # アプリケーションログの出力先 66 | CloudWatchLogsGroup: 67 | Type: AWS::Logs::LogGroup 68 | Properties: 69 | RetentionInDays: 1 70 | 71 | # Fargate の起動に必要な権限 72 | ExecutionRole: 73 | Type: AWS::IAM::Role 74 | Properties: 75 | AssumeRolePolicyDocument: 76 | Statement: 77 | - Effect: Allow 78 | Action: sts:AssumeRole 79 | Principal: 80 | Service: ecs-tasks.amazonaws.com 81 | ManagedPolicyArns: 82 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 83 | Path: / 84 | 85 | # Fargate（アプリケーション）に付与する権限 86 | TaskRole: 87 | Type: AWS::IAM::Role 88 | Properties: 89 | AssumeRolePolicyDocument: 90 | Statement: 91 | - Effect: Allow 92 | Action: sts:AssumeRole 93 | Principal: 94 | Service: ecs-tasks.amazonaws.com 95 | ManagedPolicyArns: 96 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 97 | Path: / 98 | 99 | # Fargate（アプリケーション）の定義 100 | FargateTaskDefinition: 101 | Type: AWS::ECS::TaskDefinition 102 | Properties: 103 | Family: !Ref AWS::StackName 104 | ExecutionRoleArn: !GetAtt ExecutionRole.Arn 105 | RequiresCompatibilities: 106 | - FARGATE 107 | ContainerDefinitions: 108 | - Name: app 109 | Image: !Ref DockerImage 110 | Essential: true 111 | PortMappings: 112 | - ContainerPort: 80 113 | Environment: 114 | - Name: PROJECT_ID 115 | Value: !Ref ProjectID 116 | - Name: BASIC_AUTH_USER 117 | Value: !Ref BasicAuthUsername 118 | - Name: BASIC_AUTH_PASS 119 | Value: !Ref BasicAuthPassword 120 | LogConfiguration: 121 | LogDriver: awslogs 122 | Options: 123 | awslogs-region: !Ref AWS::Region 124 | awslogs-group: !Ref CloudWatchLogsGroup 125 | awslogs-stream-prefix: fargate 126 | Cpu: !Ref ContainerCpu 127 | Memory: !Ref ContainerMemory 128 | NetworkMode: awsvpc 129 | TaskRoleArn: !GetAtt TaskRole.Arn 130 | 131 | # Fargate の外部インターフェイスの定義 132 | FargateService: 133 | Type: AWS::ECS::Service 134 | Properties: 135 | Cluster: !Ref Cluster 136 | LaunchType: FARGATE 137 | DesiredCount: 1 138 | TaskDefinition: !Ref FargateTaskDefinition 139 | NetworkConfiguration: 140 | AwsvpcConfiguration: 141 | AssignPublicIp: ENABLED 142 | SecurityGroups: 143 | - Fn::ImportValue: !Sub PublicAccess-${ProjectID} 144 | Subnets: 145 | - Fn::ImportValue: !Sub PublicSubnet1-${ProjectID} 146 | - Fn::ImportValue: !Sub PublicSubnet2-${ProjectID} 147 | -------------------------------------------------------------------------------- /fargate/application/deploy/cfn-release.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Description: "A fargate application" 3 | 4 | Outputs: 5 | Cluster: 6 | Description: the ECS cluster name 7 | Value: !Ref Cluster 8 | 9 | LoadBalancerDNSName: 10 | Value: !GetAtt LoadBalancer.DNSName 11 | 12 | Metadata: 13 | AWS::CloudFormation::Interface: 14 | ParameterGroups: 15 | - Label: 16 | default: Basic Configurations 17 | Parameters: 18 | - ProjectID 19 | - Label: 20 | default: Application Configurations 21 | Parameters: 22 | - DockerImage 23 | - BasicAuthUsername 24 | - BasicAuthPassword 25 | - ContainerCpu 26 | - ContainerMemory 27 | - TaskMinContainerCount 28 | - TaskMaxContainerCount 29 | 30 | Parameters: 31 | ProjectID: 32 | Type: String 33 | 34 | DockerImage: 35 | Type: String 36 | 37 | BasicAuthUsername: 38 | Type: String 39 | Default: "" 40 | 41 | BasicAuthPassword: 42 | Type: String 43 | Default: "" 44 | NoEcho: true 45 | 46 | ContainerCpu: 47 | Type: Number 48 | Description: Amount of CPU for the container 49 | Default: 256 50 | MinValue: 256 51 | MaxValue: 4096 52 | ConstraintDescription: "Value must be between 256 and 4096 - see: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size" 53 | 54 | ContainerMemory: 55 | Type: Number 56 | Description: Amount of memory for the container 57 | Default: 512 58 | MinValue: 512 59 | MaxValue: 30720 60 | ConstraintDescription: "Value must be between 512 and 30720 - see: https://aws.amazon.com/fargate/pricing/" 61 | 62 | # Auto scaling container counts 63 | TaskMinContainerCount: 64 | Type: Number 65 | Description: Minimum number of containers to run for the service 66 | Default: 1 67 | MinValue: 1 68 | ConstraintDescription: Value must be at least one 69 | 70 | TaskMaxContainerCount: 71 | Type: Number 72 | Description: Maximum number of containers to run for the service when auto scaling out 73 | Default: 2 74 | MinValue: 1 75 | ConstraintDescription: Value must be at least one 76 | 77 | # Scaling params 78 | DefaultServiceScaleEvaluationPeriods: 79 | Description: The number of periods over which data is compared to the specified threshold 80 | Type: Number 81 | Default: 2 82 | MinValue: 2 83 | 84 | DefaultServiceCpuScaleOutThreshold: 85 | Type: Number 86 | Description: Average CPU value to trigger auto scaling out 87 | Default: 50 88 | MinValue: 0 89 | MaxValue: 100 90 | ConstraintDescription: Value must be between 0 and 100 91 | 92 | DefaultServiceCpuScaleInThreshold: 93 | Type: Number 94 | Description: Average CPU value to trigger auto scaling in 95 | Default: 25 96 | MinValue: 0 97 | MaxValue: 100 98 | ConstraintDescription: Value must be between 0 and 100 99 | 100 | Resources: 101 | 102 | # Fargate のコントロールプレインとなる ECS クラスタ 103 | Cluster: 104 | Type: AWS::ECS::Cluster 105 | Properties: 106 | ClusterName: !Ref AWS::StackName 107 | 108 | # アプリケーションログの出力先 109 | CloudWatchLogsGroup: 110 | Type: AWS::Logs::LogGroup 111 | Properties: 112 | RetentionInDays: 1 113 | 114 | # Fargate の起動に必要な権限 115 | ExecutionRole: 116 | Type: AWS::IAM::Role 117 | Properties: 118 | AssumeRolePolicyDocument: 119 | Statement: 120 | - Effect: Allow 121 | Action: sts:AssumeRole 122 | Principal: 123 | Service: ecs-tasks.amazonaws.com 124 | ManagedPolicyArns: 125 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 126 | Path: / 127 | 128 | # Fargate（アプリケーション）に付与する権限 129 | TaskRole: 130 | Type: AWS::IAM::Role 131 | Properties: 132 | AssumeRolePolicyDocument: 133 | Statement: 134 | - Effect: Allow 135 | Action: sts:AssumeRole 136 | Principal: 137 | Service: ecs-tasks.amazonaws.com 138 | ManagedPolicyArns: 139 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 140 | Path: / 141 | 142 | # Fargate（アプリケーション）の定義 143 | FargateTaskDefinition: 144 | Type: AWS::ECS::TaskDefinition 145 | Properties: 146 | Family: !Ref AWS::StackName 147 | ExecutionRoleArn: !GetAtt ExecutionRole.Arn 148 | RequiresCompatibilities: 149 | - FARGATE 150 | ContainerDefinitions: 151 | - Name: app 152 | Image: !Ref DockerImage 153 | Essential: true 154 | PortMappings: 155 | - ContainerPort: 80 156 | Environment: 157 | - Name: PROJECT_ID 158 | Value: !Ref ProjectID 159 | - Name: BASIC_AUTH_USER 160 | Value: !Ref BasicAuthUsername 161 | - Name: BASIC_AUTH_PASS 162 | Value: !Ref BasicAuthPassword 163 | LogConfiguration: 164 | LogDriver: awslogs 165 | Options: 166 | awslogs-region: !Ref AWS::Region 167 | awslogs-group: !Ref CloudWatchLogsGroup 168 | awslogs-stream-prefix: fargate 169 | Cpu: !Ref ContainerCpu 170 | Memory: !Ref ContainerMemory 171 | NetworkMode: awsvpc 172 | TaskRoleArn: !GetAtt TaskRole.Arn 173 | 174 | # Fargate の外部インターフェイスの定義 175 | FargateService: 176 | Type: AWS::ECS::Service 177 | Properties: 178 | Cluster: !Ref Cluster 179 | LaunchType: FARGATE 180 | DesiredCount: !Ref TaskMinContainerCount 181 | TaskDefinition: !Ref FargateTaskDefinition 182 | LoadBalancers: 183 | - ContainerName: app 184 | ContainerPort: 80 185 | TargetGroupArn: !Ref TargetGroup 186 | NetworkConfiguration: 187 | AwsvpcConfiguration: 188 | AssignPublicIp: ENABLED 189 | SecurityGroups: 190 | - Fn::ImportValue: !Sub InternalAccess-${ProjectID} 191 | Subnets: 192 | - Fn::ImportValue: !Sub PublicSubnet1-${ProjectID} 193 | - Fn::ImportValue: !Sub PublicSubnet2-${ProjectID} 194 | DependsOn: 195 | - LoadBalancer 196 | - Listener 197 | - TargetGroup 198 | - ListenerRule 199 | 200 | # フロントエンドのロードバランサー 201 | LoadBalancer: 202 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 203 | Properties: 204 | Type: application 205 | Scheme: internet-facing 206 | IpAddressType: ipv4 207 | LoadBalancerAttributes: 208 | - Key: deletion_protection.enabled 209 | Value: false 210 | - Key: idle_timeout.timeout_seconds 211 | Value: 30 212 | Subnets: 213 | - Fn::ImportValue: !Sub PublicSubnet1-${ProjectID} 214 | - Fn::ImportValue: !Sub PublicSubnet2-${ProjectID} 215 | SecurityGroups: 216 | - Fn::ImportValue: !Sub PublicAccess-${ProjectID} 217 | 218 | Listener: 219 | Type: AWS::ElasticLoadBalancingV2::Listener 220 | Properties: 221 | DefaultActions: 222 | - Type: forward 223 | TargetGroupArn: !Ref TargetGroup 224 | LoadBalancerArn: !Ref LoadBalancer 225 | Protocol: HTTP 226 | Port: 80 227 | 228 | TargetGroup: 229 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 230 | Properties: 231 | VpcId: 232 | Fn::ImportValue: !Sub VPC-${ProjectID} 233 | TargetType: ip 234 | Protocol: HTTP 235 | Port: 80 236 | HealthCheckProtocol: HTTP 237 | HealthCheckPath: /health 238 | Matcher: 239 | HttpCode: 200 240 | HealthCheckIntervalSeconds: 10 241 | HealthCheckTimeoutSeconds: 5 242 | HealthyThresholdCount: 2 243 | UnhealthyThresholdCount: 2 244 | TargetGroupAttributes: 245 | - Key: deregistration_delay.timeout_seconds 246 | Value: 30 247 | Tags: 248 | - Key: Stack 249 | Value: !Ref AWS::StackName 250 | 251 | ListenerRule: 252 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 253 | Properties: 254 | Actions: 255 | - Type: forward 256 | TargetGroupArn: !Ref TargetGroup 257 | Conditions: 258 | - Field: path-pattern 259 | Values: [/] 260 | ListenerArn: !Ref Listener 261 | Priority: 1 262 | 263 | # オートスケーリングの設定 264 | AutoScalingRole: 265 | Type: AWS::IAM::Role 266 | Properties: 267 | AssumeRolePolicyDocument: 268 | Statement: 269 | - Effect: Allow 270 | Action: sts:AssumeRole 271 | Principal: 272 | Service: application-autoscaling.amazonaws.com 273 | Policies: 274 | - PolicyName: service-autoscaling 275 | PolicyDocument: 276 | Statement: 277 | - Effect: Allow 278 | Action: 279 | - application-autoscaling:* 280 | - cloudwatch:DescribeAlarms 281 | - cloudwatch:PutMetricAlarm 282 | - ecs:DescribeServices 283 | - ecs:UpdateService 284 | Resource: '*' 285 | Path: / 286 | 287 | ScalingTarget: 288 | Type: AWS::ApplicationAutoScaling::ScalableTarget 289 | Properties: 290 | ServiceNamespace: ecs 291 | ScalableDimension: ecs:service:DesiredCount 292 | ResourceId: !Sub service/${Cluster}/${FargateService.Name} 293 | RoleARN: !GetAtt AutoScalingRole.Arn 294 | MinCapacity: !Ref TaskMinContainerCount 295 | MaxCapacity: !Ref TaskMaxContainerCount 296 | 297 | ScaleOutPolicy: 298 | Type: AWS::ApplicationAutoScaling::ScalingPolicy 299 | Properties: 300 | PolicyName: ScaleOutPolicy 301 | PolicyType: StepScaling 302 | ScalingTargetId: !Ref ScalingTarget 303 | StepScalingPolicyConfiguration: 304 | AdjustmentType: ChangeInCapacity 305 | MetricAggregationType: Average 306 | StepAdjustments: 307 | - ScalingAdjustment: 1 308 | MetricIntervalLowerBound: 0 309 | Cooldown: 60 310 | 311 | ScaleInPolicy: 312 | Type: AWS::ApplicationAutoScaling::ScalingPolicy 313 | Properties: 314 | PolicyName: ScaleInPolicy 315 | PolicyType: StepScaling 316 | ScalingTargetId: !Ref ScalingTarget 317 | StepScalingPolicyConfiguration: 318 | AdjustmentType: ChangeInCapacity 319 | MetricAggregationType: Average 320 | StepAdjustments: 321 | - ScalingAdjustment: -1 322 | MetricIntervalUpperBound: 0 323 | Cooldown: 60 324 | 325 | ScaleOutAlarm: 326 | Type: AWS::CloudWatch::Alarm 327 | Properties: 328 | AlarmDescription: Alarm to add capacity if CPU is high 329 | Namespace: AWS/ECS 330 | MetricName: CPUUtilization 331 | Statistic: Average 332 | Dimensions: 333 | - Name: ClusterName 334 | Value: !Ref Cluster 335 | - Name: ServiceName 336 | Value: !GetAtt FargateService.Name 337 | ComparisonOperator: GreaterThanThreshold 338 | Period: 60 339 | EvaluationPeriods: !Ref DefaultServiceScaleEvaluationPeriods 340 | Threshold: !Ref DefaultServiceCpuScaleOutThreshold 341 | TreatMissingData: notBreaching 342 | AlarmActions: 343 | - !Ref ScaleOutPolicy 344 | DependsOn: 345 | - ScaleOutPolicy 346 | 347 | ScaleInAlarm: 348 | Type: AWS::CloudWatch::Alarm 349 | Properties: 350 | AlarmDescription: Alarm to reduce capacity if container CPU is low 351 | Namespace: AWS/ECS 352 | MetricName: CPUUtilization 353 | Statistic: Average 354 | Dimensions: 355 | - Name: ClusterName 356 | Value: !Ref Cluster 357 | - Name: ServiceName 358 | Value: !GetAtt FargateService.Name 359 | ComparisonOperator: LessThanThreshold 360 | Period: 300 361 | EvaluationPeriods: !Ref DefaultServiceScaleEvaluationPeriods 362 | Threshold: !Ref DefaultServiceCpuScaleInThreshold 363 | TreatMissingData: notBreaching 364 | AlarmActions: 365 | - !Ref ScaleInPolicy 366 | DependsOn: 367 | - ScaleInPolicy 368 | -------------------------------------------------------------------------------- /fargate/application/deploy/edge/buildspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | 3 | phases: 4 | install: 5 | commands: 6 | - apt-get update 7 | - apt-get -y install python-pip 8 | - pip install --upgrade python 9 | - pip install --upgrade awscli 10 | pre_build: 11 | commands: 12 | - aws cloudformation validate-template --template-body fileb://deploy/cfn-master.yaml 13 | - IMAGE_NAME="${REPOSITORY_URI}:$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | head -c 7)" 14 | - BASIC_AUTH='"BasicAuthUsername":"","BasicAuthPassword":""' 15 | - | 16 | if [ -e "deploy/${ENVIRONMENT_NAME}/env.sh" ]; then 17 | $( cat "deploy/${ENVIRONMENT_NAME}/env.sh" ) 18 | fi 19 | - $(aws ecr get-login --no-include-email) 20 | build: 21 | commands: 22 | - echo Build started on $(date) 23 | - docker build --tag ${IMAGE_NAME} . 24 | - docker push ${IMAGE_NAME} 25 | post_build: 26 | commands: 27 | - echo Build completed on $(date) 28 | - printf '{"Parameters":{"ProjectID":"%s","DockerImage":"%s",%s}}' ${PROJECT_ID} ${IMAGE_NAME} ${BASIC_AUTH} > config.json 29 | 30 | artifacts: 31 | files: config.json 32 | -------------------------------------------------------------------------------- /fargate/application/deploy/production/buildspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | 3 | phases: 4 | pre_build: 5 | commands: 6 | - IMAGE_NAME="${REPOSITORY_URI}:$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | head -c 7)" 7 | build: 8 | commands: 9 | # アプリケーションのビルドは個別にせず、検証された staging と同じものを利用する 10 | - echo "do nothing." 11 | post_build: 12 | commands: 13 | # 本番環境は Basic 認証を行わない 14 | - printf '{"Parameters":{"ProjectID":"%s","DockerImage":"%s"}}' ${PROJECT_ID} ${IMAGE_NAME} > config.json 15 | 16 | artifacts: 17 | files: config.json 18 | -------------------------------------------------------------------------------- /fargate/application/deploy/staging/buildspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | 3 | phases: 4 | install: 5 | commands: 6 | - apt-get update 7 | - apt-get -y install python-pip 8 | - pip install --upgrade python 9 | - pip install --upgrade awscli 10 | pre_build: 11 | commands: 12 | - aws cloudformation validate-template --template-body fileb://deploy/cfn-release.yaml 13 | - IMAGE_NAME="${REPOSITORY_URI}:$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | head -c 7)" 14 | - BASIC_AUTH='"BasicAuthUsername":"","BasicAuthPassword":""' 15 | - | 16 | if [ -e "deploy/${ENVIRONMENT_NAME}/env.sh" ]; then 17 | $( cat "deploy/${ENVIRONMENT_NAME}/env.sh" ) 18 | fi 19 | - $(aws ecr get-login --no-include-email) 20 | build: 21 | commands: 22 | - echo Build started on $(date) 23 | - docker build --tag ${IMAGE_NAME} . 24 | - docker push ${IMAGE_NAME} 25 | post_build: 26 | commands: 27 | - echo Build completed on $(date) 28 | - printf '{"Parameters":{"ProjectID":"%s","DockerImage":"%s",%s}}' ${PROJECT_ID} ${IMAGE_NAME} ${BASIC_AUTH} > config.json 29 | 30 | artifacts: 31 | files: config.json 32 | -------------------------------------------------------------------------------- /fargate/application/diff/iam.yaml: -------------------------------------------------------------------------------- 1 | --- cfn-master-old.yaml 2 | +++ cfn-master.yaml 3 | @@ -94,6 +94,7 @@ 4 | Service: ecs-tasks.amazonaws.com 5 | ManagedPolicyArns: 6 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 7 | + - arn:aws:iam::aws:policy/TranslateReadOnly 8 | Path: / 9 | 10 | # Fargate（アプリケーション）の定義 11 | -------------------------------------------------------------------------------- /fargate/application/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "html/template" 6 | "io" 7 | "log" 8 | "net/http" 9 | "strings" 10 | "time" 11 | 12 | "github.com/aws/aws-sdk-go-v2/aws" 13 | "github.com/aws/aws-sdk-go-v2/aws/endpoints" 14 | "github.com/aws/aws-sdk-go-v2/aws/external" 15 | "github.com/aws/aws-sdk-go-v2/service/translate" 16 | "github.com/caarlos0/env" 17 | ) 18 | 19 | type config struct { 20 | BasicAuthUser string `env:"BASIC_AUTH_USER" envDefault:""` 21 | BasicAuthPass string `env:"BASIC_AUTH_PASS" envDefault:""` 22 | Port string `env:"PORT" envDefault:"80"` 23 | } 24 | 25 | var ( 26 | cfg config 27 | svc *translate.Translate 28 | ) 29 | 30 | func main() { 31 | cfg = config{} 32 | env.Parse(&cfg) 33 | 34 | awscfg, err := external.LoadDefaultAWSConfig() 35 | if err != nil { 36 | panic("Unable to load AWS SDK config, " + err.Error()) 37 | } 38 | awscfg.Region = endpoints.UsEast2RegionID 39 | svc = translate.New(awscfg) 40 | 41 | http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { 42 | w.WriteHeader(http.StatusOK) 43 | }) 44 | http.Handle("/", wrap(cfg, logic)) 45 | 46 | log.Printf("[service] listening on %s", cfg.Port) 47 | log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", cfg.Port), nil)) 48 | } 49 | 50 | type custom struct { 51 | io.Writer 52 | http.ResponseWriter 53 | status int 54 | } 55 | 56 | func (r *custom) Write(b []byte) (int, error) { 57 | if r.Header().Get("Content-Type") == "" { 58 | r.Header().Set("Content-Type", http.DetectContentType(b)) 59 | } 60 | return r.Writer.Write(b) 61 | } 62 | 63 | func (r *custom) WriteHeader(status int) { 64 | r.ResponseWriter.WriteHeader(status) 65 | r.status = status 66 | } 67 | 68 | func auth(c config, r *http.Request) bool { 69 | if username, password, ok := r.BasicAuth(); ok { 70 | return username == c.BasicAuthUser && password == c.BasicAuthPass 71 | } 72 | return false 73 | } 74 | 75 | func wrap(c config, f func(w http.ResponseWriter, r *http.Request)) http.Handler { 76 | return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { 77 | if (len(c.BasicAuthUser) > 0) && (len(c.BasicAuthPass) > 0) && !auth(c, r) { 78 | w.Header().Set("WWW-Authenticate", `Basic realm="REALM"`) 79 | http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized) 80 | return 81 | } 82 | proc := time.Now() 83 | addr := r.RemoteAddr 84 | if candidate := r.Header["X-Forwarded-For"]; len(candidate) > 0 { 85 | addr = candidate[0] 86 | } 87 | ioWriter := w.(io.Writer) 88 | writer := &custom{Writer: ioWriter, ResponseWriter: w, status: http.StatusOK} 89 | f(writer, r) 90 | 91 | log.Printf("[%s] %.3f %d %s %s", 92 | addr, time.Now().Sub(proc).Seconds(), 93 | writer.status, r.Method, r.URL) 94 | }) 95 | } 96 | 97 | func logic(w http.ResponseWriter, r *http.Request) { 98 | data := map[string]string{} 99 | 100 | if strings.EqualFold(r.Method, http.MethodPost) { 101 | from := r.FormValue("f") 102 | 103 | res, err := svc.TextRequest(&translate.TextInput{ 104 | SourceLanguageCode: aws.String("ja"), 105 | TargetLanguageCode: aws.String("en"), 106 | Text: aws.String(from), 107 | }).Send() 108 | if err != nil { 109 | http.Error(w, err.Error(), http.StatusInternalServerError) 110 | return 111 | } 112 | data["from"] = from 113 | data["to"] = aws.StringValue(res.TranslatedText) 114 | } 115 | t := template.Must(template.New("html").Parse(` 116 | 117 | 118 | 119 | AWS SDK sample 120 | 121 | 125 | 126 | 127 |
128 | 129 | 130 | 131 |
132 | 133 | `)) 134 | t.Execute(w, data) 135 | } 136 | -------------------------------------------------------------------------------- /fargate/infrastructure/.gitignore: -------------------------------------------------------------------------------- 1 | cfn.yaml 2 | -------------------------------------------------------------------------------- /fargate/infrastructure/diff/orig.yaml: -------------------------------------------------------------------------------- 1 | Transform: AWS::Serverless-2016-10-31 2 | AWSTemplateFormatVersion: '2010-09-09' 3 | Description: "A fargate application with a CI/CD pipeline" 4 | 5 | Outputs: 6 | GitHTTP: 7 | Description: CodeCommit clone URL for HTTPS 8 | Value: !GetAtt CodeRepository.CloneUrlHttp 9 | 10 | CodePipeline: 11 | Value: !Ref CodePipeline 12 | 13 | VPC: 14 | Value: !Ref VPC 15 | Export: 16 | Name: !Sub VPC-${ProjectID} 17 | 18 | PublicSubnet1: 19 | Value: !Ref PublicSubnet1 20 | Export: 21 | Name: !Sub PublicSubnet1-${ProjectID} 22 | 23 | PublicSubnet2: 24 | Value: !Ref PublicSubnet2 25 | Export: 26 | Name: !Sub PublicSubnet2-${ProjectID} 27 | 28 | PublicAccess: 29 | Value: !GetAtt PublicAccessSecurityGroup.GroupId 30 | Export: 31 | Name: !Sub PublicAccess-${ProjectID} 32 | 33 | InternalAccess: 34 | Value: !GetAtt InternalSecurityGroup.GroupId 35 | Export: 36 | Name: !Sub InternalAccess-${ProjectID} 37 | 38 | Metadata: 39 | AWS::CloudFormation::Interface: 40 | ParameterGroups: 41 | - Label: 42 | default: Basic Configurations 43 | Parameters: 44 | - ProjectID 45 | - EdgeStackName 46 | - Label: 47 | default: Pipeline Configurations 48 | Parameters: 49 | - S3BucketName 50 | - DockerBuildImage 51 | - ApprovalEmail 52 | 53 | Parameters: 54 | ProjectID: 55 | Type: String 56 | 57 | S3BucketName: 58 | Type: String 59 | 60 | EdgeStackName: 61 | Type: String 62 | 63 | DockerBuildImage: 64 | Type: String 65 | Default: "aws/codebuild/docker:17.09.0" 66 | Description: Docker Image used by CodeBuild 67 | 68 | ApprovalEmail: 69 | Type: String 70 | Description: Email address to which approval should be sent 71 | AllowedPattern: "([a-zA-Z0-9_\\-\\.]+)@((\\[[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.)|(([a-zA-Z0-9\\-]+\\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\\]?)" 72 | ConstraintDescription: Must be a valid email address. (e.g. name@example.com) 73 | 74 | Globals: 75 | Function: 76 | Runtime: nodejs8.10 77 | MemorySize: 128 78 | Timeout: 30 79 | 80 | Resources: 81 | 82 | DockerRepository: 83 | Type: AWS::ECR::Repository 84 | Properties: 85 | RepositoryName: fargate-handson/api 86 | LifecyclePolicy: 87 | LifecyclePolicyText: | 88 | { 89 | "rules": [ 90 | { 91 | "rulePriority": 1, 92 | "description": "Only keep untagged images for 1 day", 93 | "selection": { 94 | "tagStatus": "untagged", 95 | "countType": "sinceImagePushed", 96 | "countNumber": 1, 97 | "countUnit": "days" 98 | }, 99 | "action": { "type": "expire" } 100 | } 101 | ] 102 | } 103 | 104 | CodeRepository: 105 | Type: AWS::CodeCommit::Repository 106 | Properties: 107 | RepositoryName: fargate-handson 108 | RepositoryDescription: Code repository for Fargate hands-on 109 | 110 | Topic: 111 | Type: AWS::SNS::Topic 112 | Properties: 113 | Subscription: 114 | - Protocol: email 115 | Endpoint: !Ref ApprovalEmail 116 | 117 | NotificationFuncRole: 118 | Type: AWS::IAM::Role 119 | Properties: 120 | AssumeRolePolicyDocument: 121 | Statement: 122 | - Effect: Allow 123 | Action: sts:AssumeRole 124 | Principal: 125 | Service: lambda.amazonaws.com 126 | Policies: 127 | - PolicyName: allow-codepipeline-and-sns 128 | PolicyDocument: 129 | Statement: 130 | - Effect: Allow 131 | Action: 132 | - codepipeline:* 133 | - sns:* 134 | Resource: "*" 135 | ManagedPolicyArns: 136 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 137 | Path: / 138 | 139 | NotificationFunc: 140 | Type: AWS::Serverless::Function 141 | Properties: 142 | FunctionName: fargate-cicd-notification 143 | CodeUri: notification/ 144 | Handler: index.handler 145 | Environment: 146 | Variables: 147 | TOPIC_ARN: !Ref Topic 148 | Role: !GetAtt NotificationFuncRole.Arn 149 | 150 | CodeBuildServiceRole: 151 | Type: AWS::IAM::Role 152 | Properties: 153 | AssumeRolePolicyDocument: 154 | Statement: 155 | - Effect: Allow 156 | Action: sts:AssumeRole 157 | Principal: 158 | Service: codebuild.amazonaws.com 159 | Policies: 160 | - PolicyName: code-build-service 161 | PolicyDocument: 162 | Statement: 163 | - Effect: Allow 164 | Action: 165 | - cloudformation:ValidateTemplate 166 | Resource: "*" 167 | - Effect: Allow 168 | Action: 169 | - logs:CreateLogGroup 170 | - logs:CreateLogStream 171 | - logs:PutLogEvents 172 | - ecr:GetAuthorizationToken 173 | Resource: "*" 174 | - Effect: Allow 175 | Action: 176 | - s3:GetObject 177 | - s3:PutObject 178 | - s3:GetObjectVersion 179 | Resource: !Sub arn:aws:s3:::${S3BucketName}/* 180 | - Effect: Allow 181 | Action: 182 | - ecr:GetDownloadUrlForLayer 183 | - ecr:BatchGetImage 184 | - ecr:BatchCheckLayerAvailability 185 | - ecr:PutImage 186 | - ecr:InitiateLayerUpload 187 | - ecr:UploadLayerPart 188 | - ecr:CompleteLayerUpload 189 | Resource: !Sub arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${DockerRepository} 190 | Path: / 191 | 192 | CodeBuild: 193 | Type: AWS::CodeBuild::Project 194 | Properties: 195 | Name: !Ref AWS::StackName 196 | ServiceRole: !Ref CodeBuildServiceRole 197 | Environment: 198 | ComputeType: BUILD_GENERAL1_SMALL 199 | Type: LINUX_CONTAINER 200 | Image: !Ref DockerBuildImage 201 | EnvironmentVariables: 202 | - Name: PROJECT_ID 203 | Value: !Ref ProjectID 204 | - Name: ENVIRONMENT_NAME 205 | Value: edge 206 | - Name: REPOSITORY_URI 207 | Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${DockerRepository} 208 | Artifacts: 209 | Type: CODEPIPELINE 210 | Source: 211 | Type: CODEPIPELINE 212 | BuildSpec: deploy/edge/buildspec.yml 213 | 214 | CodePipelineServiceRole: 215 | Type: AWS::IAM::Role 216 | Properties: 217 | AssumeRolePolicyDocument: 218 | Statement: 219 | - Effect: Allow 220 | Action: sts:AssumeRole 221 | Principal: 222 | Service: codepipeline.amazonaws.com 223 | Policies: 224 | - PolicyName: code-pipeline-service 225 | PolicyDocument: 226 | Statement: 227 | - Effect: Allow 228 | Action: 229 | - iam:PassRole 230 | - codecommit:ListRepositories 231 | - lambda:List* 232 | - cloudformation:CreateChangeSet 233 | - cloudformation:CreateStack 234 | - cloudformation:CreateUploadBucket 235 | - cloudformation:DeleteStack 236 | - cloudformation:Describe* 237 | - cloudformation:List* 238 | - cloudformation:UpdateStack 239 | - cloudformation:ValidateTemplate 240 | - cloudformation:ExecuteChangeSet 241 | Resource: "*" 242 | - Effect: Allow 243 | Action: 244 | - s3:PutObject 245 | - s3:GetObject 246 | - s3:GetObjectVersion 247 | Resource: !Sub arn:aws:s3:::${S3BucketName}/* 248 | - Effect: Allow 249 | Action: 250 | - codecommit:ListBranches 251 | - codecommit:BatchGetRepositories 252 | - codecommit:Get* 253 | - codecommit:GitPull 254 | - codecommit:UploadArchive 255 | - codecommit:CancelUploadArchive 256 | Resource: 257 | - !Sub arn:aws:codecommit:${AWS::Region}:${AWS::AccountId}:${CodeRepository.Name} 258 | - Effect: Allow 259 | Action: 260 | - codebuild:StartBuild 261 | - codebuild:BatchGetBuilds 262 | Resource: 263 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuild} 264 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuild}:* 265 | - Effect: Allow 266 | Action: 267 | - lambda:* 268 | Resource: 269 | - !GetAtt NotificationFunc.Arn 270 | Path: / 271 | 272 | CloudFormationRole: 273 | Type: AWS::IAM::Role 274 | Properties: 275 | AssumeRolePolicyDocument: 276 | Statement: 277 | - Effect: Allow 278 | Action: sts:AssumeRole 279 | Principal: 280 | Service: cloudformation.amazonaws.com 281 | Policies: 282 | - PolicyName: code-pipeline-service 283 | PolicyDocument: 284 | Statement: 285 | - Effect: Allow 286 | Action: 287 | - iam:* 288 | - ecs:RegisterTaskDefinition 289 | - ecs:DeregisterTaskDefinition 290 | # for stack update / deletion 291 | - ecs:CreateService 292 | - ecs:Delete* 293 | - logs:Delete* 294 | - cloudwatch:Delete* 295 | Resource: "*" 296 | ManagedPolicyArns: 297 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 298 | - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole 299 | - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole 300 | Path: / 301 | 302 | CodePipeline: 303 | Type: AWS::CodePipeline::Pipeline 304 | Properties: 305 | RoleArn: !GetAtt CodePipelineServiceRole.Arn 306 | ArtifactStore: 307 | Type: S3 308 | Location: !Ref S3BucketName 309 | Stages: 310 | - Name: Source 311 | Actions: 312 | - Name: Source 313 | ActionTypeId: 314 | Category: Source 315 | Provider: CodeCommit 316 | Owner: AWS 317 | Version: 1 318 | Configuration: 319 | RepositoryName: !GetAtt CodeRepository.Name 320 | BranchName: master 321 | OutputArtifacts: 322 | - Name: SourceCode 323 | RunOrder: 1 324 | - Name: Build 325 | Actions: 326 | - Name: Build 327 | ActionTypeId: 328 | Category: Build 329 | Provider: CodeBuild 330 | Owner: AWS 331 | Version: 1 332 | Configuration: 333 | ProjectName: !Ref CodeBuild 334 | InputArtifacts: 335 | - Name: SourceCode 336 | OutputArtifacts: 337 | - Name: Artifacts 338 | RunOrder: 1 339 | - Name: Deploy 340 | Actions: 341 | - Name: CreateEdgeChangeSet 342 | ActionTypeId: 343 | Category: Deploy 344 | Provider: CloudFormation 345 | Owner: AWS 346 | Version: 1 347 | Configuration: 348 | StackName: !Ref EdgeStackName 349 | ActionMode: CHANGE_SET_REPLACE 350 | ChangeSetName: ChangeSet 351 | RoleArn: !GetAtt CloudFormationRole.Arn 352 | Capabilities: CAPABILITY_NAMED_IAM 353 | TemplatePath: SourceCode::deploy/cfn-master.yaml 354 | TemplateConfiguration: Artifacts::config.json 355 | InputArtifacts: 356 | - Name: SourceCode 357 | - Name: Artifacts 358 | OutputArtifacts: 359 | - Name: CreatedEdgeChangeSet 360 | RunOrder: 1 361 | - Name: ExecuteEdgeChangeSet 362 | ActionTypeId: 363 | Category: Deploy 364 | Provider: CloudFormation 365 | Owner: AWS 366 | Version: 1 367 | Configuration: 368 | StackName: !Ref EdgeStackName 369 | ActionMode: CHANGE_SET_EXECUTE 370 | ChangeSetName: ChangeSet 371 | InputArtifacts: 372 | - Name: CreatedEdgeChangeSet 373 | OutputArtifacts: 374 | - Name: EdgeDeployed 375 | RunOrder: 2 376 | - Name: EdgeDeployedNotification 377 | ActionTypeId: 378 | Category: Invoke 379 | Provider: Lambda 380 | Owner: AWS 381 | Version: 1 382 | Configuration: 383 | FunctionName: !Ref NotificationFunc 384 | UserParameters: "{\"EnvName\":\"Edge\"}" 385 | InputArtifacts: 386 | - Name: EdgeDeployed 387 | OutputArtifacts: 388 | - Name: EdgeFinished 389 | RunOrder: 3 390 | DependsOn: 391 | - CodeBuild 392 | - CodePipelineServiceRole 393 | 394 | VPC: 395 | Type: AWS::EC2::VPC 396 | Properties: 397 | CidrBlock: 10.0.0.0/16 398 | EnableDnsSupport: true 399 | EnableDnsHostnames: true 400 | Tags: 401 | - Key: Name 402 | Value: Fargate Hands-on 403 | 404 | PublicSubnet1: 405 | Type: AWS::EC2::Subnet 406 | DependsOn: AttachGateway 407 | Properties: 408 | VpcId: !Ref VPC 409 | MapPublicIpOnLaunch: true 410 | AvailabilityZone: !Select [ 0, !GetAZs ] 411 | CidrBlock: 10.0.128.0/18 412 | Tags: 413 | - Key: Name 414 | Value: "Public Subnet 1" 415 | 416 | PublicSubnet2: 417 | Type: AWS::EC2::Subnet 418 | DependsOn: AttachGateway 419 | Properties: 420 | VpcId: !Ref VPC 421 | MapPublicIpOnLaunch: true 422 | AvailabilityZone: !Select [ 1, !GetAZs ] 423 | CidrBlock: 10.0.192.0/18 424 | Tags: 425 | - Key: Name 426 | Value: "Public Subnet 2" 427 | 428 | InternetGateway: 429 | Type: AWS::EC2::InternetGateway 430 | DependsOn: VPC 431 | 432 | AttachGateway: 433 | Type: AWS::EC2::VPCGatewayAttachment 434 | Properties: 435 | VpcId: !Ref VPC 436 | InternetGatewayId: !Ref InternetGateway 437 | 438 | PublicRouteTable: 439 | Type: AWS::EC2::RouteTable 440 | DependsOn: AttachGateway 441 | Properties: 442 | VpcId: !Ref VPC 443 | Tags: 444 | - Key: Name 445 | Value: Public 446 | 447 | PublicRoute: 448 | Type: AWS::EC2::Route 449 | DependsOn: AttachGateway 450 | Properties: 451 | RouteTableId: !Ref PublicRouteTable 452 | GatewayId: !Ref InternetGateway 453 | DestinationCidrBlock: 0.0.0.0/0 454 | 455 | PublicSubnet1RouteTableAssociation: 456 | Type: AWS::EC2::SubnetRouteTableAssociation 457 | DependsOn: AttachGateway 458 | Properties: 459 | SubnetId: !Ref PublicSubnet1 460 | RouteTableId: !Ref PublicRouteTable 461 | 462 | PublicSubnet2RouteTableAssociation: 463 | Type: AWS::EC2::SubnetRouteTableAssociation 464 | DependsOn: AttachGateway 465 | Properties: 466 | SubnetId: !Ref PublicSubnet2 467 | RouteTableId: !Ref PublicRouteTable 468 | 469 | PublicAccessSecurityGroup: 470 | Type: AWS::EC2::SecurityGroup 471 | Properties: 472 | GroupDescription: SecurityGroup for load balancers 473 | VpcId: !Ref VPC 474 | SecurityGroupIngress: 475 | - IpProtocol: tcp 476 | FromPort: 80 477 | ToPort: 80 478 | CidrIp: 0.0.0.0/0 479 | Tags: 480 | - Key: Name 481 | Value: PublicAccess 482 | 483 | InternalSecurityGroup: 484 | Type: AWS::EC2::SecurityGroup 485 | Properties: 486 | GroupDescription: SecurityGroup for internal access 487 | VpcId: !Ref VPC 488 | SecurityGroupIngress: 489 | - CidrIp: 10.0.0.0/16 490 | IpProtocol: -1 491 | Tags: 492 | - Key: Name 493 | Value: InternalAccess 494 | 495 | InternalSecurityGroupIngress: 496 | Type: AWS::EC2::SecurityGroupIngress 497 | Properties: 498 | GroupId: !Ref InternalSecurityGroup 499 | IpProtocol: -1 500 | SourceSecurityGroupId: !Ref InternalSecurityGroup 501 | -------------------------------------------------------------------------------- /fargate/infrastructure/diff/production: -------------------------------------------------------------------------------- 1 | --- sam-staging.yaml 2 | +++ sam.yaml 3 | @@ -47,6 +47,7 @@ 4 | - ProjectID 5 | - EdgeStackName 6 | - StagingStackName 7 | + - ProductionStackName 8 | - Label: 9 | default: Pipeline Configurations 10 | Parameters: 11 | @@ -67,6 +68,9 @@ 12 | StagingStackName: 13 | Type: String 14 | 15 | + ProductionStackName: 16 | + Type: String 17 | + 18 | DockerBuildImage: 19 | Type: String 20 | Default: "aws/codebuild/docker:17.09.0" 21 | @@ -248,6 +252,29 @@ 22 | Type: CODEPIPELINE 23 | BuildSpec: deploy/staging/buildspec.yml 24 | 25 | + # アプリケーションのテストやビルドの定義（本番環境） 26 | + CodeBuildProduction: 27 | + Type: AWS::CodeBuild::Project 28 | + Properties: 29 | + Name: !Sub ${AWS::StackName}-production 30 | + ServiceRole: !Ref CodeBuildServiceRole 31 | + Environment: 32 | + ComputeType: BUILD_GENERAL1_SMALL 33 | + Type: LINUX_CONTAINER 34 | + Image: !Ref DockerBuildImage 35 | + EnvironmentVariables: 36 | + - Name: PROJECT_ID 37 | + Value: !Ref ProjectID 38 | + - Name: ENVIRONMENT_NAME 39 | + Value: production 40 | + - Name: REPOSITORY_URI 41 | + Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${DockerRepository} 42 | + Artifacts: 43 | + Type: CODEPIPELINE 44 | + Source: 45 | + Type: CODEPIPELINE 46 | + BuildSpec: deploy/production/buildspec.yml 47 | + 48 | # 継続的インテグレーション/継続的デプロイのパイプラインに必要な権限 49 | CodePipelineServiceRole: 50 | Type: AWS::IAM::Role 51 | @@ -303,6 +330,8 @@ 52 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuild}:* 53 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuildStaging} 54 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuildStaging}:* 55 | + - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuildProduction} 56 | + - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuildProduction}:* 57 | - Effect: Allow 58 | Action: 59 | - lambda:* 60 | @@ -475,6 +504,19 @@ 61 | OutputArtifacts: 62 | - Name: Artifacts 63 | RunOrder: 1 64 | + - Name: ProdConfig 65 | + ActionTypeId: 66 | + Category: Build 67 | + Provider: CodeBuild 68 | + Owner: AWS 69 | + Version: 1 70 | + Configuration: 71 | + ProjectName: !Ref CodeBuildProduction 72 | + InputArtifacts: 73 | + - Name: SourceCode 74 | + OutputArtifacts: 75 | + - Name: ProdConfig 76 | + RunOrder: 1 77 | - Name: DeployToStaging 78 | Actions: 79 | - Name: CreateStagingChangeSet 80 | @@ -526,8 +568,70 @@ 81 | OutputArtifacts: 82 | - Name: StagingFinished 83 | RunOrder: 3 84 | + - Name: DeployToProduction 85 | + Actions: 86 | + - Name: CreateProductionChangeSet 87 | + ActionTypeId: 88 | + Category: Deploy 89 | + Provider: CloudFormation 90 | + Owner: AWS 91 | + Version: 1 92 | + Configuration: 93 | + StackName: !Ref ProductionStackName 94 | + ActionMode: CHANGE_SET_REPLACE 95 | + ChangeSetName: ChangeSet 96 | + RoleArn: !GetAtt CloudFormationRole.Arn 97 | + Capabilities: CAPABILITY_NAMED_IAM 98 | + TemplatePath: SourceCode::deploy/cfn-release.yaml 99 | + TemplateConfiguration: ProdConfig::config.json 100 | + InputArtifacts: 101 | + - Name: SourceCode 102 | + - Name: ProdConfig 103 | + OutputArtifacts: 104 | + - Name: CreatedProductionChangeSet 105 | + RunOrder: 1 106 | + - Name: ProdApproval 107 | + ActionTypeId: 108 | + Category: Approval 109 | + Provider: Manual 110 | + Owner: AWS 111 | + Version: 1 112 | + Configuration: 113 | + NotificationArn: !Ref Topic 114 | + CustomData: Approve deployment in production. 115 | + RunOrder: 2 116 | + - Name: ExecuteProductionChangeSet 117 | + ActionTypeId: 118 | + Category: Deploy 119 | + Provider: CloudFormation 120 | + Owner: AWS 121 | + Version: 1 122 | + Configuration: 123 | + StackName: !Ref ProductionStackName 124 | + ActionMode: CHANGE_SET_EXECUTE 125 | + ChangeSetName: ChangeSet 126 | + InputArtifacts: 127 | + - Name: CreatedProductionChangeSet 128 | + OutputArtifacts: 129 | + - Name: ProductionDeployed 130 | + RunOrder: 3 131 | + - Name: ProductionDeployedNotification 132 | + ActionTypeId: 133 | + Category: Invoke 134 | + Provider: Lambda 135 | + Owner: AWS 136 | + Version: 1 137 | + Configuration: 138 | + FunctionName: !Ref NotificationFunc 139 | + UserParameters: "{\"EnvName\":\"Production\"}" 140 | + InputArtifacts: 141 | + - Name: ProductionDeployed 142 | + OutputArtifacts: 143 | + - Name: ProductionFinished 144 | + RunOrder: 4 145 | DependsOn: 146 | - CodeBuildStaging 147 | + - CodeBuildProduction 148 | - CodePipelineServiceRole 149 | 150 | # アプリケーションのための VPC 151 | -------------------------------------------------------------------------------- /fargate/infrastructure/diff/staging: -------------------------------------------------------------------------------- 1 | --- sam-edge.yaml 2 | +++ sam.yaml 3 | @@ -10,6 +10,9 @@ 4 | CodePipeline: 5 | Value: !Ref CodePipeline 6 | 7 | + CodePipelineRelease: 8 | + Value: !Ref CodePipelineRelease 9 | + 10 | VPC: 11 | Value: !Ref VPC 12 | Export: 13 | @@ -43,6 +46,7 @@ 14 | Parameters: 15 | - ProjectID 16 | - EdgeStackName 17 | + - StagingStackName 18 | - Label: 19 | default: Pipeline Configurations 20 | Parameters: 21 | @@ -60,6 +64,9 @@ 22 | EdgeStackName: 23 | Type: String 24 | 25 | + StagingStackName: 26 | + Type: String 27 | + 28 | DockerBuildImage: 29 | Type: String 30 | Default: "aws/codebuild/docker:17.09.0" 31 | @@ -218,6 +225,29 @@ 32 | Type: CODEPIPELINE 33 | BuildSpec: deploy/edge/buildspec.yml 34 | 35 | + # アプリケーションのテストやビルドの定義（ステージング環境） 36 | + CodeBuildStaging: 37 | + Type: AWS::CodeBuild::Project 38 | + Properties: 39 | + Name: !Sub ${AWS::StackName}-staging 40 | + ServiceRole: !Ref CodeBuildServiceRole 41 | + Environment: 42 | + ComputeType: BUILD_GENERAL1_SMALL 43 | + Type: LINUX_CONTAINER 44 | + Image: !Ref DockerBuildImage 45 | + EnvironmentVariables: 46 | + - Name: PROJECT_ID 47 | + Value: !Ref ProjectID 48 | + - Name: ENVIRONMENT_NAME 49 | + Value: staging 50 | + - Name: REPOSITORY_URI 51 | + Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${DockerRepository} 52 | + Artifacts: 53 | + Type: CODEPIPELINE 54 | + Source: 55 | + Type: CODEPIPELINE 56 | + BuildSpec: deploy/staging/buildspec.yml 57 | + 58 | # 継続的インテグレーション/継続的デプロイのパイプラインに必要な権限 59 | CodePipelineServiceRole: 60 | Type: AWS::IAM::Role 61 | @@ -271,6 +301,8 @@ 62 | Resource: 63 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuild} 64 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuild}:* 65 | + - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuildStaging} 66 | + - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuildStaging}:* 67 | - Effect: Allow 68 | Action: 69 | - lambda:* 70 | @@ -298,6 +330,9 @@ 71 | - ecs:RegisterTaskDefinition 72 | - ecs:DeregisterTaskDefinition 73 | # for stack update / deletion 74 | + - elasticloadbalancing:ModifyTargetGroup 75 | + - elasticloadbalancing:Delete* 76 | + - application-autoscaling:* 77 | - ecs:CreateService 78 | - ecs:Delete* 79 | - logs:Delete* 80 | @@ -400,6 +435,99 @@ 81 | RunOrder: 3 82 | DependsOn: 83 | - CodeBuild 84 | + - CodePipelineServiceRole 85 | + 86 | + # 継続的インテグレーション/継続的デプロイのパイプライン（ステージング・本番環境） 87 | + CodePipelineRelease: 88 | + Type: AWS::CodePipeline::Pipeline 89 | + Properties: 90 | + RoleArn: !GetAtt CodePipelineServiceRole.Arn 91 | + ArtifactStore: 92 | + Type: S3 93 | + Location: !Ref S3BucketName 94 | + Stages: 95 | + - Name: Source 96 | + Actions: 97 | + - Name: Source 98 | + ActionTypeId: 99 | + Category: Source 100 | + Provider: CodeCommit 101 | + Owner: AWS 102 | + Version: 1 103 | + Configuration: 104 | + RepositoryName: !GetAtt CodeRepository.Name 105 | + BranchName: release 106 | + OutputArtifacts: 107 | + - Name: SourceCode 108 | + RunOrder: 1 109 | + - Name: Build 110 | + Actions: 111 | + - Name: Build 112 | + ActionTypeId: 113 | + Category: Build 114 | + Provider: CodeBuild 115 | + Owner: AWS 116 | + Version: 1 117 | + Configuration: 118 | + ProjectName: !Ref CodeBuildStaging 119 | + InputArtifacts: 120 | + - Name: SourceCode 121 | + OutputArtifacts: 122 | + - Name: Artifacts 123 | + RunOrder: 1 124 | + - Name: DeployToStaging 125 | + Actions: 126 | + - Name: CreateStagingChangeSet 127 | + ActionTypeId: 128 | + Category: Deploy 129 | + Provider: CloudFormation 130 | + Owner: AWS 131 | + Version: 1 132 | + Configuration: 133 | + StackName: !Ref StagingStackName 134 | + ActionMode: CHANGE_SET_REPLACE 135 | + ChangeSetName: ChangeSet 136 | + RoleArn: !GetAtt CloudFormationRole.Arn 137 | + Capabilities: CAPABILITY_NAMED_IAM 138 | + TemplatePath: SourceCode::deploy/cfn-release.yaml 139 | + TemplateConfiguration: Artifacts::config.json 140 | + InputArtifacts: 141 | + - Name: SourceCode 142 | + - Name: Artifacts 143 | + OutputArtifacts: 144 | + - Name: CreatedStagingChangeSet 145 | + RunOrder: 1 146 | + - Name: ExecuteStagingChangeSet 147 | + ActionTypeId: 148 | + Category: Deploy 149 | + Provider: CloudFormation 150 | + Owner: AWS 151 | + Version: 1 152 | + Configuration: 153 | + StackName: !Ref StagingStackName 154 | + ActionMode: CHANGE_SET_EXECUTE 155 | + ChangeSetName: ChangeSet 156 | + InputArtifacts: 157 | + - Name: CreatedStagingChangeSet 158 | + OutputArtifacts: 159 | + - Name: StagingDeployed 160 | + RunOrder: 2 161 | + - Name: StagingDeployedNotification 162 | + ActionTypeId: 163 | + Category: Invoke 164 | + Provider: Lambda 165 | + Owner: AWS 166 | + Version: 1 167 | + Configuration: 168 | + FunctionName: !Ref NotificationFunc 169 | + UserParameters: "{\"EnvName\":\"Staging\"}" 170 | + InputArtifacts: 171 | + - Name: StagingDeployed 172 | + OutputArtifacts: 173 | + - Name: StagingFinished 174 | + RunOrder: 3 175 | + DependsOn: 176 | + - CodeBuildStaging 177 | - CodePipelineServiceRole 178 | 179 | # アプリケーションのための VPC 180 | -------------------------------------------------------------------------------- /fargate/infrastructure/notification/index.js: -------------------------------------------------------------------------------- 1 | var AWS = require('aws-sdk'); 2 | 3 | exports.handler = function(event, context) { 4 | var eventText = JSON.stringify(event, null, 2); 5 | console.log("Received event:", eventText); 6 | 7 | var job = event["CodePipeline.job"], 8 | jobId = job.id, 9 | params = JSON.parse(job.data.actionConfiguration.configuration.UserParameters); 10 | 11 | // Validate parameters passed in UserParameters 12 | if(! params || params.EnvName == '') { 13 | failure('The UserParameters is invalid'); 14 | return; 15 | } 16 | 17 | // Publish to a SNS topic 18 | var sns = new AWS.SNS(); 19 | sns.publish({ 20 | TopicArn: process.env.TOPIC_ARN, 21 | Subject: "A new version was deployed via AWS CodePipeline", 22 | Message: "Successfully deployed a new version to "+params.EnvName+" fargate!" 23 | }, function (err, data) { 24 | if (err) { 25 | failure(err); 26 | return; 27 | } 28 | success("Function Finished."); 29 | }); 30 | 31 | function success(message) { 32 | var api = new AWS.CodePipeline(); 33 | api.putJobSuccessResult({jobId: jobId}, function (err, data) { 34 | if (err) { 35 | context.fail(err); 36 | } else { 37 | context.succeed(message); 38 | } 39 | }); 40 | } 41 | function failure(message) { 42 | var api = new AWS.CodePipeline(), 43 | params = { 44 | jobId: jobId, 45 | failureDetails: { 46 | type: 'JobFailed', 47 | message: JSON.stringify(message), 48 | externalExecutionId: context.invokeid 49 | } 50 | }; 51 | api.putJobFailureResult(params, function (err, data) { 52 | context.fail(message); 53 | }); 54 | } 55 | }; 56 | -------------------------------------------------------------------------------- /fargate/infrastructure/sam.yaml: -------------------------------------------------------------------------------- 1 | Transform: AWS::Serverless-2016-10-31 2 | AWSTemplateFormatVersion: '2010-09-09' 3 | Description: "A fargate application with a CI/CD pipeline" 4 | 5 | Outputs: 6 | GitHTTP: 7 | Description: CodeCommit clone URL for HTTPS 8 | Value: !GetAtt CodeRepository.CloneUrlHttp 9 | 10 | CodePipeline: 11 | Value: !Ref CodePipeline 12 | 13 | VPC: 14 | Value: !Ref VPC 15 | Export: 16 | Name: !Sub VPC-${ProjectID} 17 | 18 | PublicSubnet1: 19 | Value: !Ref PublicSubnet1 20 | Export: 21 | Name: !Sub PublicSubnet1-${ProjectID} 22 | 23 | PublicSubnet2: 24 | Value: !Ref PublicSubnet2 25 | Export: 26 | Name: !Sub PublicSubnet2-${ProjectID} 27 | 28 | PublicAccess: 29 | Value: !GetAtt PublicAccessSecurityGroup.GroupId 30 | Export: 31 | Name: !Sub PublicAccess-${ProjectID} 32 | 33 | InternalAccess: 34 | Value: !GetAtt InternalSecurityGroup.GroupId 35 | Export: 36 | Name: !Sub InternalAccess-${ProjectID} 37 | 38 | Metadata: 39 | AWS::CloudFormation::Interface: 40 | ParameterGroups: 41 | - Label: 42 | default: Basic Configurations 43 | Parameters: 44 | - ProjectID 45 | - EdgeStackName 46 | - Label: 47 | default: Pipeline Configurations 48 | Parameters: 49 | - S3BucketName 50 | - DockerBuildImage 51 | - ApprovalEmail 52 | 53 | Parameters: 54 | ProjectID: 55 | Type: String 56 | 57 | S3BucketName: 58 | Type: String 59 | 60 | EdgeStackName: 61 | Type: String 62 | 63 | DockerBuildImage: 64 | Type: String 65 | Default: "aws/codebuild/docker:17.09.0" 66 | Description: Docker Image used by CodeBuild 67 | 68 | ApprovalEmail: 69 | Type: String 70 | Description: Email address to which approval should be sent 71 | AllowedPattern: "([a-zA-Z0-9_\\-\\.]+)@((\\[[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.)|(([a-zA-Z0-9\\-]+\\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\\]?)" 72 | ConstraintDescription: Must be a valid email address. (e.g. name@example.com) 73 | 74 | Globals: 75 | Function: 76 | Runtime: nodejs8.10 77 | MemorySize: 128 78 | Timeout: 30 79 | 80 | Resources: 81 | 82 | # 成果物である Docker イメージを保持するリポジトリ 83 | DockerRepository: 84 | Type: AWS::ECR::Repository 85 | Properties: 86 | RepositoryName: fargate-handson/api 87 | LifecyclePolicy: 88 | LifecyclePolicyText: | 89 | { 90 | "rules": [ 91 | { 92 | "rulePriority": 1, 93 | "description": "Only keep untagged images for 1 day", 94 | "selection": { 95 | "tagStatus": "untagged", 96 | "countType": "sinceImagePushed", 97 | "countNumber": 1, 98 | "countUnit": "days" 99 | }, 100 | "action": { "type": "expire" } 101 | } 102 | ] 103 | } 104 | 105 | # ソースコードを保持するためのリポジトリ 106 | CodeRepository: 107 | Type: AWS::CodeCommit::Repository 108 | Properties: 109 | RepositoryName: fargate-handson 110 | RepositoryDescription: Code repository for Fargate hands-on 111 | 112 | # パイプラインの完了通知や承認リンクを送信するための通知トピック 113 | Topic: 114 | Type: AWS::SNS::Topic 115 | Properties: 116 | Subscription: 117 | - Protocol: email 118 | Endpoint: !Ref ApprovalEmail 119 | 120 | # 通知 Lambda の動作に必要な権限 121 | NotificationFuncRole: 122 | Type: AWS::IAM::Role 123 | Properties: 124 | AssumeRolePolicyDocument: 125 | Statement: 126 | - Effect: Allow 127 | Action: sts:AssumeRole 128 | Principal: 129 | Service: lambda.amazonaws.com 130 | Policies: 131 | - PolicyName: allow-codepipeline-and-sns 132 | PolicyDocument: 133 | Statement: 134 | - Effect: Allow 135 | Action: 136 | - codepipeline:* 137 | - sns:* 138 | Resource: "*" 139 | ManagedPolicyArns: 140 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 141 | Path: / 142 | 143 | # 通知 Lambda 144 | NotificationFunc: 145 | Type: AWS::Serverless::Function 146 | Properties: 147 | FunctionName: fargate-cicd-notification 148 | CodeUri: notification/ 149 | Handler: index.handler 150 | Environment: 151 | Variables: 152 | TOPIC_ARN: !Ref Topic 153 | Role: !GetAtt NotificationFuncRole.Arn 154 | 155 | # アプリケーションのテストやビルドに必要な権限 156 | CodeBuildServiceRole: 157 | Type: AWS::IAM::Role 158 | Properties: 159 | AssumeRolePolicyDocument: 160 | Statement: 161 | - Effect: Allow 162 | Action: sts:AssumeRole 163 | Principal: 164 | Service: codebuild.amazonaws.com 165 | Policies: 166 | - PolicyName: code-build-service 167 | PolicyDocument: 168 | Statement: 169 | - Effect: Allow 170 | Action: 171 | - cloudformation:ValidateTemplate 172 | Resource: "*" 173 | - Effect: Allow 174 | Action: 175 | - logs:CreateLogGroup 176 | - logs:CreateLogStream 177 | - logs:PutLogEvents 178 | - ecr:GetAuthorizationToken 179 | Resource: "*" 180 | - Effect: Allow 181 | Action: 182 | - s3:GetObject 183 | - s3:PutObject 184 | - s3:GetObjectVersion 185 | Resource: !Sub arn:aws:s3:::${S3BucketName}/* 186 | - Effect: Allow 187 | Action: 188 | - ecr:GetDownloadUrlForLayer 189 | - ecr:BatchGetImage 190 | - ecr:BatchCheckLayerAvailability 191 | - ecr:PutImage 192 | - ecr:InitiateLayerUpload 193 | - ecr:UploadLayerPart 194 | - ecr:CompleteLayerUpload 195 | Resource: !Sub arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${DockerRepository} 196 | Path: / 197 | 198 | # アプリケーションのテストやビルドの定義（開発環境） 199 | CodeBuild: 200 | Type: AWS::CodeBuild::Project 201 | Properties: 202 | Name: !Ref AWS::StackName 203 | ServiceRole: !Ref CodeBuildServiceRole 204 | Environment: 205 | ComputeType: BUILD_GENERAL1_SMALL 206 | Type: LINUX_CONTAINER 207 | Image: !Ref DockerBuildImage 208 | EnvironmentVariables: 209 | - Name: PROJECT_ID 210 | Value: !Ref ProjectID 211 | - Name: ENVIRONMENT_NAME 212 | Value: edge 213 | - Name: REPOSITORY_URI 214 | Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${DockerRepository} 215 | Artifacts: 216 | Type: CODEPIPELINE 217 | Source: 218 | Type: CODEPIPELINE 219 | BuildSpec: deploy/edge/buildspec.yml 220 | 221 | # 継続的インテグレーション/継続的デプロイのパイプラインに必要な権限 222 | CodePipelineServiceRole: 223 | Type: AWS::IAM::Role 224 | Properties: 225 | AssumeRolePolicyDocument: 226 | Statement: 227 | - Effect: Allow 228 | Action: sts:AssumeRole 229 | Principal: 230 | Service: codepipeline.amazonaws.com 231 | Policies: 232 | - PolicyName: code-pipeline-service 233 | PolicyDocument: 234 | Statement: 235 | - Effect: Allow 236 | Action: 237 | - iam:PassRole 238 | - sns:Publish 239 | - codecommit:ListRepositories 240 | - lambda:List* 241 | - cloudformation:CreateChangeSet 242 | - cloudformation:CreateStack 243 | - cloudformation:CreateUploadBucket 244 | - cloudformation:DeleteStack 245 | - cloudformation:Describe* 246 | - cloudformation:List* 247 | - cloudformation:UpdateStack 248 | - cloudformation:ValidateTemplate 249 | - cloudformation:ExecuteChangeSet 250 | Resource: "*" 251 | - Effect: Allow 252 | Action: 253 | - s3:PutObject 254 | - s3:GetObject 255 | - s3:GetObjectVersion 256 | Resource: !Sub arn:aws:s3:::${S3BucketName}/* 257 | - Effect: Allow 258 | Action: 259 | - codecommit:ListBranches 260 | - codecommit:BatchGetRepositories 261 | - codecommit:Get* 262 | - codecommit:GitPull 263 | - codecommit:UploadArchive 264 | - codecommit:CancelUploadArchive 265 | Resource: 266 | - !Sub arn:aws:codecommit:${AWS::Region}:${AWS::AccountId}:${CodeRepository.Name} 267 | - Effect: Allow 268 | Action: 269 | - codebuild:StartBuild 270 | - codebuild:BatchGetBuilds 271 | Resource: 272 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuild} 273 | - !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/${CodeBuild}:* 274 | - Effect: Allow 275 | Action: 276 | - lambda:* 277 | Resource: 278 | - !GetAtt NotificationFunc.Arn 279 | Path: / 280 | 281 | # 継続的デプロイに必要な権限 282 | CloudFormationRole: 283 | Type: AWS::IAM::Role 284 | Properties: 285 | AssumeRolePolicyDocument: 286 | Statement: 287 | - Effect: Allow 288 | Action: sts:AssumeRole 289 | Principal: 290 | Service: cloudformation.amazonaws.com 291 | Policies: 292 | - PolicyName: code-pipeline-service 293 | PolicyDocument: 294 | Statement: 295 | - Effect: Allow 296 | Action: 297 | - iam:* 298 | - ecs:RegisterTaskDefinition 299 | - ecs:DeregisterTaskDefinition 300 | # for stack update / deletion 301 | - ecs:CreateService 302 | - ecs:Delete* 303 | - logs:Delete* 304 | - cloudwatch:Delete* 305 | Resource: "*" 306 | ManagedPolicyArns: 307 | - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy 308 | - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole 309 | - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole 310 | Path: / 311 | 312 | # 継続的インテグレーション/継続的デプロイのパイプライン（開発環境） 313 | CodePipeline: 314 | Type: AWS::CodePipeline::Pipeline 315 | Properties: 316 | RoleArn: !GetAtt CodePipelineServiceRole.Arn 317 | ArtifactStore: 318 | Type: S3 319 | Location: !Ref S3BucketName 320 | Stages: 321 | - Name: Source 322 | Actions: 323 | - Name: Source 324 | ActionTypeId: 325 | Category: Source 326 | Provider: CodeCommit 327 | Owner: AWS 328 | Version: 1 329 | Configuration: 330 | RepositoryName: !GetAtt CodeRepository.Name 331 | BranchName: master 332 | OutputArtifacts: 333 | - Name: SourceCode 334 | RunOrder: 1 335 | - Name: Build 336 | Actions: 337 | - Name: Build 338 | ActionTypeId: 339 | Category: Build 340 | Provider: CodeBuild 341 | Owner: AWS 342 | Version: 1 343 | Configuration: 344 | ProjectName: !Ref CodeBuild 345 | InputArtifacts: 346 | - Name: SourceCode 347 | OutputArtifacts: 348 | - Name: Artifacts 349 | RunOrder: 1 350 | - Name: Deploy 351 | Actions: 352 | - Name: CreateEdgeChangeSet 353 | ActionTypeId: 354 | Category: Deploy 355 | Provider: CloudFormation 356 | Owner: AWS 357 | Version: 1 358 | Configuration: 359 | StackName: !Ref EdgeStackName 360 | ActionMode: CHANGE_SET_REPLACE 361 | ChangeSetName: ChangeSet 362 | RoleArn: !GetAtt CloudFormationRole.Arn 363 | Capabilities: CAPABILITY_NAMED_IAM 364 | TemplatePath: SourceCode::deploy/cfn-master.yaml 365 | TemplateConfiguration: Artifacts::config.json 366 | InputArtifacts: 367 | - Name: SourceCode 368 | - Name: Artifacts 369 | OutputArtifacts: 370 | - Name: CreatedEdgeChangeSet 371 | RunOrder: 1 372 | - Name: ExecuteEdgeChangeSet 373 | ActionTypeId: 374 | Category: Deploy 375 | Provider: CloudFormation 376 | Owner: AWS 377 | Version: 1 378 | Configuration: 379 | StackName: !Ref EdgeStackName 380 | ActionMode: CHANGE_SET_EXECUTE 381 | ChangeSetName: ChangeSet 382 | InputArtifacts: 383 | - Name: CreatedEdgeChangeSet 384 | OutputArtifacts: 385 | - Name: EdgeDeployed 386 | RunOrder: 2 387 | - Name: EdgeDeployedNotification 388 | ActionTypeId: 389 | Category: Invoke 390 | Provider: Lambda 391 | Owner: AWS 392 | Version: 1 393 | Configuration: 394 | FunctionName: !Ref NotificationFunc 395 | UserParameters: "{\"EnvName\":\"Edge\"}" 396 | InputArtifacts: 397 | - Name: EdgeDeployed 398 | OutputArtifacts: 399 | - Name: EdgeFinished 400 | RunOrder: 3 401 | DependsOn: 402 | - CodeBuild 403 | - CodePipelineServiceRole 404 | 405 | # アプリケーションのための VPC 406 | VPC: 407 | Type: AWS::EC2::VPC 408 | Properties: 409 | CidrBlock: 10.0.0.0/16 410 | EnableDnsSupport: true 411 | EnableDnsHostnames: true 412 | Tags: 413 | - Key: Name 414 | Value: Fargate Hands-on 415 | 416 | PublicSubnet1: 417 | Type: AWS::EC2::Subnet 418 | DependsOn: AttachGateway 419 | Properties: 420 | VpcId: !Ref VPC 421 | MapPublicIpOnLaunch: true 422 | AvailabilityZone: !Select [ 0, !GetAZs ] 423 | CidrBlock: 10.0.128.0/18 424 | Tags: 425 | - Key: Name 426 | Value: "Public Subnet 1" 427 | 428 | PublicSubnet2: 429 | Type: AWS::EC2::Subnet 430 | DependsOn: AttachGateway 431 | Properties: 432 | VpcId: !Ref VPC 433 | MapPublicIpOnLaunch: true 434 | AvailabilityZone: !Select [ 1, !GetAZs ] 435 | CidrBlock: 10.0.192.0/18 436 | Tags: 437 | - Key: Name 438 | Value: "Public Subnet 2" 439 | 440 | InternetGateway: 441 | Type: AWS::EC2::InternetGateway 442 | DependsOn: VPC 443 | 444 | AttachGateway: 445 | Type: AWS::EC2::VPCGatewayAttachment 446 | Properties: 447 | VpcId: !Ref VPC 448 | InternetGatewayId: !Ref InternetGateway 449 | 450 | PublicRouteTable: 451 | Type: AWS::EC2::RouteTable 452 | DependsOn: AttachGateway 453 | Properties: 454 | VpcId: !Ref VPC 455 | Tags: 456 | - Key: Name 457 | Value: Public 458 | 459 | PublicRoute: 460 | Type: AWS::EC2::Route 461 | DependsOn: AttachGateway 462 | Properties: 463 | RouteTableId: !Ref PublicRouteTable 464 | GatewayId: !Ref InternetGateway 465 | DestinationCidrBlock: 0.0.0.0/0 466 | 467 | PublicSubnet1RouteTableAssociation: 468 | Type: AWS::EC2::SubnetRouteTableAssociation 469 | DependsOn: AttachGateway 470 | Properties: 471 | SubnetId: !Ref PublicSubnet1 472 | RouteTableId: !Ref PublicRouteTable 473 | 474 | PublicSubnet2RouteTableAssociation: 475 | Type: AWS::EC2::SubnetRouteTableAssociation 476 | DependsOn: AttachGateway 477 | Properties: 478 | SubnetId: !Ref PublicSubnet2 479 | RouteTableId: !Ref PublicRouteTable 480 | 481 | # 一般公開からの接続用セキュリティグループ 482 | PublicAccessSecurityGroup: 483 | Type: AWS::EC2::SecurityGroup 484 | Properties: 485 | GroupDescription: SecurityGroup for load balancers 486 | VpcId: !Ref VPC 487 | SecurityGroupIngress: 488 | - IpProtocol: tcp 489 | FromPort: 80 490 | ToPort: 80 491 | CidrIp: 0.0.0.0/0 492 | Tags: 493 | - Key: Name 494 | Value: PublicAccess 495 | 496 | # VPC 内部からの接続用セキュリティグループ 497 | InternalSecurityGroup: 498 | Type: AWS::EC2::SecurityGroup 499 | Properties: 500 | GroupDescription: SecurityGroup for internal access 501 | VpcId: !Ref VPC 502 | SecurityGroupIngress: 503 | - CidrIp: 10.0.0.0/16 504 | IpProtocol: -1 505 | Tags: 506 | - Key: Name 507 | Value: InternalAccess 508 | 509 | InternalSecurityGroupIngress: 510 | Type: AWS::EC2::SecurityGroupIngress 511 | Properties: 512 | GroupId: !Ref InternalSecurityGroup 513 | IpProtocol: -1 514 | SourceSecurityGroupId: !Ref InternalSecurityGroup 515 | -------------------------------------------------------------------------------- /fargate/notebooks/00-overview.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# ハンズオンの概要\n", 8 | "\n", 9 | "このハンズオンで実施する内容を概観します。\n", 10 | "\n", 11 | "## Jupyter notebook について\n", 12 | "\n", 13 | "[Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/stable/) はブラウザから操作のできる、Python やシェルスクリプトなどの実行環境です。 \n", 14 | "\n", 15 | "**セル**（いま左側にフォーカスがあたっていますよね？この単位がセルです）ごとに \n", 16 | "上方のメニューにある `▶︎| RUN ` でコマンドを実行することができます。\n", 17 | "\n", 18 | "`Shift + Enter` キーでも実行しながら先に進むことができます。 \n", 19 | "試しに何度か押してみましょう！" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "date" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "応答例）\n", 36 | "```text\n", 37 | "Fri Jul 27 12:30:25 UTC 2018\n", 38 | "```" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "ls" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | "応答例）\n", 55 | "```text\n", 56 | "00-overview.ipynb 04-deploy-to-production.ipynb\n", 57 | "01-provision-aws-resources.ipynb 05-teardown-resources.ipynb\n", 58 | "02-develop-with-git.ipynb application\n", 59 | "03-deploy-to-staging.ipynb infrastructure\n", 60 | "```" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "今回利用するコンテンツはすべて起動した Docker イメージの中にあり、 \n", 68 | "AWS CLI 以外の処理は **すべてコンテナの中で完結** しています。 \n", 69 | "ホスト側の環境には影響を与えませんので安心してご利用ください。 \n", 70 | "\n", 71 | "すべてのノートは画面左上の `File` からダウンロードすることもできますし \n", 72 | "`jupyter アイコン` からファイル一覧に戻り、その他のファイル操作をすることもできます。 \n", 73 | "\n", 74 | "（すべてのソースコードは [こちら](https://github.com/jawsug-container/hands-on) で公開されています）" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "## ハンズオンと、そのサンプルケースの流れ\n", 82 | "\n", 83 | "### サンプルケース\n", 84 | "\n", 85 | "このハンズオンは、とあるアプリケーション開発者が成果物を **Fargate で公開する** ことを想定しています。 \n", 86 | "\n", 87 | "ビルド・テスト・デプロイ・リリースは極力自動化されており、以下のプロセスを経ます。\n", 88 | "\n", 89 | "1. **master ブランチに push** するとテストが走り、テストが通れば **edge** 環境にデプロイされる\n", 90 | "\n", 91 | "\n", 92 | "\n", 93 | "2. edge 環境を実際にさわって確認\n", 94 | "3. release ブランチに対してプルリクエストを作成\n", 95 | "4. **release ブランチにマージ** されるとテストが走り、テストが通れば **staging** 環境にデプロイされる\n", 96 | "\n", 97 | "\n", 98 | "\n", 99 | "5. staging 環境を実際にさわって確認\n", 100 | "6. **承認ボタンのクリック** で **production** 環境にリリースされる\n", 101 | "\n", 102 | "\n", 103 | "\n", 104 | "7. production 環境を実際にさわって確認" 105 | ] 106 | }, 107 | { 108 | "cell_type": "markdown", 109 | "metadata": {}, 110 | "source": [ 111 | "### ハンズオンの流れ\n", 112 | "\n", 113 | "サンプルケースの流れに添い、追体験をしていきます。\n", 114 | "\n", 115 | "| ノート名 | |\n", 116 | "| :-- | :-- |\n", 117 | "| 01-provision-aws-resources | 後述のシステム構成を AWS 上にプロビジョニング |\n", 118 | "| 02-develop-with-git | CodeCommit と git を使い、サンプルケースの **1, 2** を体験 |\n", 119 | "| 03-deploy-to-staging | ステージング環境を作りつつ、サンプルケースの **3, 4, 5** を体験 |\n", 120 | "| 04-deploy-to-production | 本番環境を作りつつ、サンプルケースの **6, 7** を体験 |\n", 121 | "| 05-teardown-resources | AWS リソースを掃除 |" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "## システム構成\n", 129 | "\n", 130 | "基本的にはこうです。\n", 131 | "\n", 132 | "- Fargate (on ECS) を使って Web アプリケーションを提供\n", 133 | "- CodeCommit と CodeBuild と CloudFormation を CodePipeline で括った CI/CD パイプラインでデプロイ" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "最終的なシステム構成は以下の通りです。\n", 141 | "\n", 142 | "" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "## このハンズオンに含まれていないもの\n", 150 | "\n", 151 | "通常 CI/CD にはある類のものも、ハンズオンの簡素化のため除外しています。 \n", 152 | "実際に本番稼働させるにあたっては別途十分な検討をしてください。\n", 153 | "\n", 154 | "- 本来デプロイ前後に可能な各種テスト、開発者へのフィードバック\n", 155 | "- ブランチ別のテストや PR に対してのテスト、レギュレーションの設定\n", 156 | "- 各種メトリクスやインジケータのモニタリング、通知、初期対応\n", 157 | "- イシュー・インシデントのトラック、トレース情報の収集の自動化\n", 158 | "\n", 159 | "などなど" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "[次へ: 01-provision-aws-resources.ipynb](01-provision-aws-resources.ipynb)" 167 | ] 168 | } 169 | ], 170 | "metadata": { 171 | "kernelspec": { 172 | "display_name": "Bash", 173 | "language": "bash", 174 | "name": "bash" 175 | }, 176 | "language_info": { 177 | "codemirror_mode": "shell", 178 | "file_extension": ".sh", 179 | "mimetype": "text/x-sh", 180 | "name": "bash" 181 | } 182 | }, 183 | "nbformat": 4, 184 | "nbformat_minor": 2 185 | } 186 | -------------------------------------------------------------------------------- /fargate/notebooks/03-deploy-to-staging.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 3. 冗長構成の Fargate ステージング環境\n", 8 | "\n", 9 | "git の release ブランチへのマージで、ロードバランサを備えた評価用環境が更新される環境を作成します。\n", 10 | "\n", 11 | "## 3.1. パイプラインの改変\n", 12 | "\n", 13 | "インフラ環境更新に必要な設定値をローカルファイルに追記します。" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": null, 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "cat << EOT >> ~/config/.env \n", 23 | "export STAGING_STACK_NAME=\"fargate-handson-staging-env\"\n", 24 | "EOT" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "Edge 環境のリソース定義に [この](/edit/infrastructure/diff/staging) パッチをあてます。 " 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "patch ~/notebook/infrastructure/sam.yaml < ~/notebook/infrastructure/diff/staging" 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "metadata": {}, 46 | "source": [ 47 | "このテンプレートの適用により、以下のような構成へ更新していきます。 \n", 48 | "" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "適用前に、新しい sam.yaml の内容が正しいことを検証します。" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "aws cloudformation validate-template --template-body file://infrastructure/sam.yaml" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "成功時応答例）\n", 72 | "```text\n", 73 | "{\n", 74 | " \"Description\": \"A fargate application with a CI/CD pipeline\",\n", 75 | " \"Parameters\": [\n", 76 | " {\n", 77 | " \"ParameterKey\": \"ProjectID\",\n", 78 | " \"NoEcho\": false\n", 79 | " },\n", 80 | " ..\n", 81 | "```" 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": {}, 87 | "source": [ 88 | "Lambda 関数を含むテンプレートのため、パッケージングします。" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "source ~/config/.env\n", 98 | "\n", 99 | "aws cloudformation package \\\n", 100 | " --template-file infrastructure/sam.yaml \\\n", 101 | " --output-template-file infrastructure/cfn.yaml \\\n", 102 | " --s3-bucket \"${S3_BUCKET_NAME}\" \\\n", 103 | " --s3-prefix \"cloudformation\"" 104 | ] 105 | }, 106 | { 107 | "cell_type": "markdown", 108 | "metadata": {}, 109 | "source": [ 110 | "成功時応答）\n", 111 | "```text\n", 112 | "Uploading to cloudformation/0e64849e597b443d9a1349275098ab43 766 / 766.0 (100.00%)\n", 113 | "Successfully packaged artifacts and wrote output template to file infrastructure/cfn.yaml\n", 114 | "..\n", 115 | "```" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "パッケージングした CloudFormation テンプレートでインフラを更新します。" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "aws cloudformation deploy \\\n", 132 | " --stack-name \"${BASE_STACK_NAME}\" \\\n", 133 | " --template-file infrastructure/cfn.yaml \\\n", 134 | " --parameter-overrides \\\n", 135 | " ProjectID=\"${PROJECT_ID}\" \\\n", 136 | " S3BucketName=\"${S3_BUCKET_NAME}\" \\\n", 137 | " EdgeStackName=\"${EDGE_STACK_NAME}\" \\\n", 138 | " StagingStackName=\"${STAGING_STACK_NAME}\" \\\n", 139 | " ApprovalEmail=\"${GIT_EMAIL_ADDRESS}\" \\\n", 140 | " --capabilities CAPABILITY_IAM" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": {}, 146 | "source": [ 147 | "成功時応答）\n", 148 | "```text\n", 149 | "Waiting for changeset to be created..\n", 150 | "Waiting for stack create/update to complete\n", 151 | "Successfully created/updated stack - fargate-handson-base\n", 152 | "```" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "## 3.2. ステージング環境の Fargate を起動する\n", 160 | "\n", 161 | "[こちら](/edit/application/deploy/cfn-release.yaml) のテンプレートを使い、ステージング環境を作成します。 \n", 162 | "この環境はロードバランサーやオートスケーリングの設定がされ、本番環境を想定した構成になっています。\n", 163 | "\n", 164 | "```\n", 165 | "- CloudWatch Logs ロググループ\n", 166 | "- ECS クラスター\n", 167 | "- ECS タスク定義\n", 168 | "- ECS サービス\n", 169 | "- IAM ロール * 2\n", 170 | "```\n", 171 | "\n", 172 | "TODO: 図を用意する\n", 173 | "\n", 174 | "開発環境 Fargate とステージング・本番環境 Fargate の差分を確認してみます。" 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": {}, 181 | "outputs": [], 182 | "source": [ 183 | "diff ~/notebook/application/deploy/cfn-master.yaml \\\n", 184 | " ~/notebook/application/deploy/cfn-release.yaml" 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "metadata": {}, 190 | "source": [ 191 | "では実際にステージング環境の Fargate をデプロイしてみます。" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": null, 197 | "metadata": {}, 198 | "outputs": [], 199 | "source": [ 200 | "DOCKER_IMAGE=dockercloud/hello-world\n", 201 | "\n", 202 | "aws cloudformation deploy \\\n", 203 | " --stack-name \"${STAGING_STACK_NAME}\" \\\n", 204 | " --template-file application/deploy/cfn-release.yaml \\\n", 205 | " --parameter-overrides \\\n", 206 | " ProjectID=\"${PROJECT_ID}\" \\\n", 207 | " DockerImage=\"${DOCKER_IMAGE}\" \\\n", 208 | " --capabilities CAPABILITY_IAM" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "metadata": {}, 214 | "source": [ 215 | "成功時応答）\n", 216 | "```text\n", 217 | "Waiting for changeset to be created..\n", 218 | "Waiting for stack create/update to complete\n", 219 | "Successfully created/updated stack - fargate-handson-staging-env\n", 220 | "```" 221 | ] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": {}, 226 | "source": [ 227 | "構築が完了したら、ステージング環境の Fargate を確認してみましょう。" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": null, 233 | "metadata": {}, 234 | "outputs": [], 235 | "source": [ 236 | "STAGING_ENDPOINT=\"http://$( aws cloudformation describe-stacks \\\n", 237 | " --stack-name \"${STAGING_STACK_NAME}\" --output text \\\n", 238 | " --query 'Stacks[*].Outputs[?OutputKey==`LoadBalancerDNSName`].OutputValue' )\"\n", 239 | "\n", 240 | "cat << EOT >> ~/config/.env\n", 241 | "export STAGING_ENDPOINT=\"${STAGING_ENDPOINT}\"\n", 242 | "EOT\n", 243 | "echo \"${STAGING_ENDPOINT}\"" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "応答例）\n", 251 | "```text\n", 252 | "http://farga-LoadB-xxx-xxx.ap-northeast-1.elb.amazonaws.com\n", 253 | "```" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": {}, 259 | "source": [ 260 | "ロードバランサー経由で Fargate にアクセスできましたか？ \n", 261 | "オートスケーリングの設定もされており、これで負荷分散する Fargate の完成です。" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "## 3.3. リリースブランチへのマージでステージング環境を更新する\n", 269 | "\n", 270 | "CodeCommit に現在の master ブランチから release ブランチを作り、ステージング環境も更新しましょう。" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": null, 276 | "metadata": {}, 277 | "outputs": [], 278 | "source": [ 279 | "cd ~/notebook/application\n", 280 | "git log -1\n", 281 | "\n", 282 | "aws codecommit create-branch \\\n", 283 | " --repository-name \"fargate-handson\" \\\n", 284 | " --branch-name \"release\" \\\n", 285 | " --commit-id \"$( git log -1 --pretty=%H )\"" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "応答例）\n", 293 | "```text\n", 294 | "commit 2236df05287049512148f204395f0d433c47a382 (HEAD -> master, origin/master)\n", 295 | "Author: foo \n", 296 | "Date: Mon Jul 23 13:24:04 2018 +0000\n", 297 | "\n", 298 | " first commit\n", 299 | "```" 300 | ] 301 | }, 302 | { 303 | "cell_type": "markdown", 304 | "metadata": {}, 305 | "source": [ 306 | "CodePipeline のコンソールから、**リリース版パイプライン** が更新されるのを眺めます。" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": null, 312 | "metadata": {}, 313 | "outputs": [], 314 | "source": [ 315 | "codebuild_project_name=$( aws cloudformation describe-stacks \\\n", 316 | " --stack-name \"${BASE_STACK_NAME}\" --output text \\\n", 317 | " --query 'Stacks[*].Outputs[?OutputKey==`CodePipelineRelease`].OutputValue' )\n", 318 | "codebuild_console=\"https://ap-northeast-1.console.aws.amazon.com/codepipeline/home\"\n", 319 | "echo \"${codebuild_console}?region=${AWS_DEFAULT_REGION}#/view/${codebuild_project_name}\"" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "更新されたら、Fargate の内容が新しくなっていることを確認しましょう。" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": null, 332 | "metadata": {}, 333 | "outputs": [], 334 | "source": [ 335 | "echo \"${STAGING_ENDPOINT}\"" 336 | ] 337 | }, 338 | { 339 | "cell_type": "markdown", 340 | "metadata": {}, 341 | "source": [ 342 | "## 3.4. プルリクエストで環境を更新する\n", 343 | "\n", 344 | "master を更新し、release に対しプルリクを作成し、マージされることでも同様の環境更新がされることを確認します。\n", 345 | "\n", 346 | "### 3.4.1. アプリケーション定義の書き換え、master へ push\n", 347 | "\n", 348 | "- Dockerfile を書き換え、ip-api.com へのリバースプロキシを Fargate で動かします\n", 349 | "- Basic 認証がかかるよう、CodeBuild 経由で cfn.yaml に渡す環境変数をセットします\n", 350 | "\n", 351 | "ここまでを git commit -> push します。" 352 | ] 353 | }, 354 | { 355 | "cell_type": "code", 356 | "execution_count": null, 357 | "metadata": { 358 | "scrolled": false 359 | }, 360 | "outputs": [], 361 | "source": [ 362 | "cd ~/notebook/application\n", 363 | "\n", 364 | "cat << EOF > Dockerfile\n", 365 | "FROM pottava/proxy\n", 366 | "\n", 367 | "ENV APP_PORT=80 \\\\\n", 368 | " PROXY_PATTERNS=\"*=http://ip-api.com/json\" \\\\\n", 369 | " HEALTHCHECK_PATH=/health \\\\\n", 370 | " ACCESS_LOG=true\n", 371 | "EOF\n", 372 | "\n", 373 | "cat << EOT > deploy/edge/env.sh\n", 374 | "export BASIC_AUTH=\"BasicAuthUsername\":\"edge\",\"BasicAuthPassword\":\"fargate\"\n", 375 | "EOT\n", 376 | "chmod +x deploy/edge/env.sh\n", 377 | "cat << EOT > deploy/staging/env.sh\n", 378 | "export BASIC_AUTH=\"BasicAuthUsername\":\"staging\",\"BasicAuthPassword\":\"fargate\"\n", 379 | "EOT\n", 380 | "chmod +x deploy/staging/env.sh\n", 381 | "\n", 382 | "git add .\n", 383 | "git commit -m \"reverse-proxy to ip-api.com\"\n", 384 | "git push" 385 | ] 386 | }, 387 | { 388 | "cell_type": "markdown", 389 | "metadata": {}, 390 | "source": [ 391 | "応答例）\n", 392 | "```text\n", 393 | "[master 628191e] reverse-proxy to ip-api.com\n", 394 | " 2 files changed, 2 insertions(+), 2 deletions(-)\n", 395 | "Counting objects: 7, done.\n", 396 | "..\n", 397 | "```" 398 | ] 399 | }, 400 | { 401 | "cell_type": "markdown", 402 | "metadata": {}, 403 | "source": [ 404 | "開発（Edge）環境のパイプラインが動き、環境が更新されます。 \n", 405 | "内容が正しそうか確認しましょう。" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": null, 411 | "metadata": {}, 412 | "outputs": [], 413 | "source": [ 414 | "codebuild_project_name=$( aws cloudformation describe-stacks \\\n", 415 | " --stack-name \"${BASE_STACK_NAME}\" --output text \\\n", 416 | " --query 'Stacks[*].Outputs[?OutputKey==`CodePipeline`].OutputValue' )\n", 417 | "codebuild_console=\"https://ap-northeast-1.console.aws.amazon.com/codepipeline/home\"\n", 418 | "echo \"${codebuild_console}?region=${AWS_DEFAULT_REGION}#/view/${codebuild_project_name}\"" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "cluster_name=$( aws cloudformation describe-stacks \\\n", 428 | " --stack-name \"${EDGE_STACK_NAME}\" --output text \\\n", 429 | " --query 'Stacks[*].Outputs[?OutputKey==`Cluster`].OutputValue' )\n", 430 | "task_id=$( aws ecs list-tasks --cluster \"${cluster_name}\" \\\n", 431 | " --family \"${EDGE_STACK_NAME}\" | jq -r '.taskArns[0]' )\n", 432 | "eni_id=$( aws ecs describe-tasks --cluster \"${cluster_name}\" --task \"${task_id}\" \\\n", 433 | " | jq '.tasks[0].attachments[0].details[]' \\\n", 434 | " | jq -r 'select( .name | contains(\"networkInterfaceId\")).value' )\n", 435 | "edge_endpoint=\"http://$( aws ec2 describe-network-interfaces \\\n", 436 | " --network-interface-ids ${eni_id} \\\n", 437 | " | jq -r '.NetworkInterfaces[].Association.PublicIp' )\"\n", 438 | "\n", 439 | "echo \"${edge_endpoint}/json\"" 440 | ] 441 | }, 442 | { 443 | "cell_type": "markdown", 444 | "metadata": {}, 445 | "source": [ 446 | "Basic 認証は上記変更の通り、ユーザー名は **edge**、パスワードは **fargate** です。\n", 447 | "\n", 448 | "ブラウザでアクセスすると、以下のような JSON か返ってくるかと思います。 \n", 449 | "応答例）\n", 450 | "```\n", 451 | "{\n", 452 | " as: \"AS16509 Amazon.com, Inc.\",\n", 453 | " countryCode: \"JP\",\n", 454 | " city: \"Tokyo\",\n", 455 | " country: \"Japan\",\n", 456 | " isp: \"Amazon.com\",\n", 457 | " query: \"13.115.178.180\",\n", 458 | " ..\n", 459 | "}\n", 460 | "```" 461 | ] 462 | }, 463 | { 464 | "cell_type": "markdown", 465 | "metadata": {}, 466 | "source": [ 467 | "Edge 環境で動作確認が取れたら、release ブランチに対してプルリクエストを作成します。" 468 | ] 469 | }, 470 | { 471 | "cell_type": "code", 472 | "execution_count": null, 473 | "metadata": {}, 474 | "outputs": [], 475 | "source": [ 476 | "repo=\"fargate-handson\"\n", 477 | "aws codecommit create-pull-request \\\n", 478 | " --title \"My first Pull Request\" \\\n", 479 | " --description \"Please review this immediately!\" \\\n", 480 | " --targets \"repositoryName=${repo},sourceReference=master,destinationReference=release\"" 481 | ] 482 | }, 483 | { 484 | "cell_type": "markdown", 485 | "metadata": {}, 486 | "source": [ 487 | "応答例）\n", 488 | "```text\n", 489 | "{\n", 490 | " \"pullRequest\": {\n", 491 | " \"pullRequestId\": \"1\",\n", 492 | " \"title\": \"My first Pull Request\",\n", 493 | " \"description\": \"Please review this immediately!\",\n", 494 | " \"pullRequestTargets\": [\n", 495 | " {\n", 496 | " \"repositoryName\": \"fargate-handson\",\n", 497 | " \"sourceReference\": \"refs/heads/master\",\n", 498 | " \"destinationReference\": \"refs/heads/release\",\n", 499 | " ..\n", 500 | "}\n", 501 | "```" 502 | ] 503 | }, 504 | { 505 | "cell_type": "markdown", 506 | "metadata": {}, 507 | "source": [ 508 | "残念ながら、孤独に一人で開発中なので、自分でレビューをしてマージしましょう・・ \n", 509 | "https://ap-northeast-1.console.aws.amazon.com/codecommit/home?region=ap-northeast-1#/repository/fargate-handson/pull-requests" 510 | ] 511 | }, 512 | { 513 | "cell_type": "markdown", 514 | "metadata": {}, 515 | "source": [ 516 | "release ブランチにマージされると、ステージング環境へのデプロイが開始されます。" 517 | ] 518 | }, 519 | { 520 | "cell_type": "code", 521 | "execution_count": null, 522 | "metadata": {}, 523 | "outputs": [], 524 | "source": [ 525 | "codebuild_project_name=$( aws cloudformation describe-stacks \\\n", 526 | " --stack-name \"${BASE_STACK_NAME}\" --output text \\\n", 527 | " --query 'Stacks[*].Outputs[?OutputKey==`CodePipelineRelease`].OutputValue' )\n", 528 | "codebuild_console=\"https://ap-northeast-1.console.aws.amazon.com/codepipeline/home\"\n", 529 | "echo \"${codebuild_console}?region=${AWS_DEFAULT_REGION}#/view/${codebuild_project_name}\"" 530 | ] 531 | }, 532 | { 533 | "cell_type": "markdown", 534 | "metadata": {}, 535 | "source": [ 536 | "ステージング環境でも内容が変更されたことを確認してみましょう。" 537 | ] 538 | }, 539 | { 540 | "cell_type": "code", 541 | "execution_count": null, 542 | "metadata": { 543 | "scrolled": true 544 | }, 545 | "outputs": [], 546 | "source": [ 547 | "echo \"${STAGING_ENDPOINT}/json\"" 548 | ] 549 | }, 550 | { 551 | "cell_type": "markdown", 552 | "metadata": {}, 553 | "source": [ 554 | "Basic 認証のユーザー名は **staging**、パスワードは **fargate** です。\n", 555 | "\n", 556 | "開発環境同様、以下のような JSON か返ってくれば OK です！ \n", 557 | "応答例）\n", 558 | "```\n", 559 | "{\n", 560 | " as: \"AS16509 Amazon.com, Inc.\",\n", 561 | " countryCode: \"JP\",\n", 562 | " city: \"Tokyo\",\n", 563 | " country: \"Japan\",\n", 564 | " isp: \"Amazon.com\",\n", 565 | " query: \"13.115.178.180\",\n", 566 | " ..\n", 567 | "}\n", 568 | "```" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "[次へ: 04-deploy-to-production.ipynb](04-deploy-to-production.ipynb)" 576 | ] 577 | } 578 | ], 579 | "metadata": { 580 | "kernelspec": { 581 | "display_name": "Bash", 582 | "language": "bash", 583 | "name": "bash" 584 | }, 585 | "language_info": { 586 | "codemirror_mode": "shell", 587 | "file_extension": ".sh", 588 | "mimetype": "text/x-sh", 589 | "name": "bash" 590 | } 591 | }, 592 | "nbformat": 4, 593 | "nbformat_minor": 2 594 | } 595 | -------------------------------------------------------------------------------- /fargate/notebooks/05-teardown-resources.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 5. AWS リソースの削除\n", 8 | "\n", 9 | "## 5.1. Production 環境のサービス停止\n", 10 | "\n", 11 | "CloudFormation スタックが更新中や停止中の場合削除に失敗します。 \n", 12 | "すべて正常に処理が終了することを確認しながら進めてください！" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "source ~/config/.env\n", 22 | "\n", 23 | "aws cloudformation delete-stack --stack-name \"${PRODUCTION_STACK_NAME}\"\n", 24 | "aws cloudformation wait stack-delete-complete --stack-name \"${PRODUCTION_STACK_NAME}\"" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "## 5.3. Staging 環境のサービス停止" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "aws cloudformation delete-stack --stack-name \"${STAGING_STACK_NAME}\"\n", 41 | "aws cloudformation wait stack-delete-complete --stack-name \"${STAGING_STACK_NAME}\"" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "## 5.3. Edge 環境のサービス停止" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "aws cloudformation delete-stack --stack-name \"${EDGE_STACK_NAME}\"\n", 58 | "aws cloudformation wait stack-delete-complete --stack-name \"${EDGE_STACK_NAME}\"" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "成功時応答）\n", 66 | "```text\n", 67 | "なし\n", 68 | "```" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## 5.4. ECR リポジトリの削除" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": null, 81 | "metadata": {}, 82 | "outputs": [], 83 | "source": [ 84 | "aws ecr delete-repository --repository-name fargate-handson/api --force" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "成功時応答例）\n", 92 | "```text\n", 93 | "{\n", 94 | " \"repository\": {\n", 95 | " \"repositoryArn\": \"arn:aws:ecr:ap-northeast-1:xxx:repository/fargate-handson/api\",\n", 96 | " \"registryId\": \"111111111111\",\n", 97 | " ..\n", 98 | " }\n", 99 | "}\n", 100 | "```" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "## 5.5. VPC や CI/CD パイプラインの削除" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "aws cloudformation delete-stack --stack-name \"${BASE_STACK_NAME}\"\n", 117 | "aws cloudformation wait stack-delete-complete --stack-name \"${BASE_STACK_NAME}\"" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "metadata": {}, 123 | "source": [ 124 | "成功時応答）\n", 125 | "```text\n", 126 | "なし\n", 127 | "```" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "## 5.6. S3 バケットの削除" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "aws s3 rb \"s3://${S3_BUCKET_NAME}\" --force" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "成功時応答）\n", 151 | "```text\n", 152 | "..\n", 153 | "delete: s3://fargate-handson-56333cb5-a2d0-442d-92fb-4e26f9b5a3dc/fargate-handson-base/Artifacts/qncICBP\n", 154 | "delete: s3://fargate-handson-56333cb5-a2d0-442d-92fb-4e26f9b5a3dc/fargate-handson-base/SourceCode/iGJNnsS\n", 155 | "remove_bucket: fargate-handson-56333cb5-a2d0-442d-92fb-4e26f9b5a3dc\n", 156 | "```" 157 | ] 158 | }, 159 | { 160 | "cell_type": "markdown", 161 | "metadata": {}, 162 | "source": [ 163 | "## 5.7. CloudWatch Logs の削除" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": null, 169 | "metadata": { 170 | "scrolled": true 171 | }, 172 | "outputs": [], 173 | "source": [ 174 | "aws logs delete-log-group --log-group-name /aws/codebuild/fargate-handson-base\n", 175 | "aws logs delete-log-group --log-group-name /aws/codebuild/fargate-handson-base-staging\n", 176 | "aws logs delete-log-group --log-group-name /aws/codebuild/fargate-handson-base-production\n", 177 | "aws logs delete-log-group --log-group-name /aws/lambda/fargate-cicd-notification" 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "成功時応答）\n", 185 | "```text\n", 186 | "なし\n", 187 | "```" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "最後までお付き合いいただきありがとうございました。 \n", 195 | "ハンズオンは以上です！" 196 | ] 197 | } 198 | ], 199 | "metadata": { 200 | "kernelspec": { 201 | "display_name": "Bash", 202 | "language": "bash", 203 | "name": "bash" 204 | }, 205 | "language_info": { 206 | "codemirror_mode": "shell", 207 | "file_extension": ".sh", 208 | "mimetype": "text/x-sh", 209 | "name": "bash" 210 | } 211 | }, 212 | "nbformat": 4, 213 | "nbformat_minor": 2 214 | } 215 | -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/app-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/app-1.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/infra-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/infra-1.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/infra-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/infra-2.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/whole-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/whole-arch.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/workflow-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/workflow-1.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/workflow-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/workflow-2.png -------------------------------------------------------------------------------- /fargate/notebooks/notebook-assets/workflow-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jawsug-container/hands-on/367d71fff56c203040280ca742e4d5a4b58211ee/fargate/notebooks/notebook-assets/workflow-3.png -------------------------------------------------------------------------------- /fargate/notebooks_config/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | function usage() { 4 | set -e 5 | cat </dev/null ) 68 | ret="$?" 69 | if [ "${identity}" = "" ]; then 70 | echo "AWS アクセスキーが不正です" 71 | exit "${ret}" 72 | fi 73 | 74 | versions=$( docker version ) 75 | ret="$?" 76 | if [ "${ret}" != "0" ]; then 77 | echo "Docker ホストとの通信に問題があります" 78 | exit "${ret}" 79 | fi 80 | 81 | if [ ! -e /root/config/.env ]; then 82 | cat << EOF > /root/config/.env 83 | export PROJECT_ID=$( python3 -c "import uuid;print(uuid.uuid4())" ) 84 | EOF 85 | fi 86 | # shellcheck disable=SC1091 87 | source /root/config/.env 88 | 89 | cat << EOT 90 | 91 | ProjectID: ${PROJECT_ID} 92 | 93 | [ AWS 環境情報 ] 94 | 95 | Account: $( echo "${identity}" | jq -r '.Account' ) 96 | Region: ${AWS_DEFAULT_REGION} 97 | IAM User: $( echo "${identity}" | jq -r '.Arn' ) 98 | AccessKey: $( echo "${identity}" | jq -r '.UserId' ) 99 | 100 | [ Git 情報 ] 101 | 102 | UserName: ${GIT_USER_NAME} 103 | Email: ${GIT_EMAIL_ADDRESS} 104 | 105 | [ Docker 情報 ] 106 | 107 | ClientVer: $( echo "${versions}" | yq -r '.Client.Version' ) 108 | ServerVer: $( echo "${versions}" | yq -r '.Server.Engine.Version' ) 109 | 110 | EOT 111 | 112 | exec "$@" 113 | -------------------------------------------------------------------------------- /fargate/notebooks_config/jupyter_notebook.py: -------------------------------------------------------------------------------- 1 | # Configuration file for jupyter-notebook. 2 | 3 | #------------------------------------------------------------------------------ 4 | # Application(SingletonConfigurable) configuration 5 | #------------------------------------------------------------------------------ 6 | 7 | ## This is an application. 8 | 9 | ## The date format used by logging formatters for %(asctime)s 10 | #c.Application.log_datefmt = '%Y-%m-%d %H:%M:%S' 11 | 12 | ## The Logging format template 13 | #c.Application.log_format = '[%(name)s]%(highlevel)s %(message)s' 14 | 15 | ## Set the log level by value or name. 16 | #c.Application.log_level = 30 17 | 18 | #------------------------------------------------------------------------------ 19 | # JupyterApp(Application) configuration 20 | #------------------------------------------------------------------------------ 21 | 22 | ## Base class for Jupyter applications 23 | 24 | ## Answer yes to any prompts. 25 | #c.JupyterApp.answer_yes = False 26 | 27 | ## Full path of a config file. 28 | #c.JupyterApp.config_file = '' 29 | 30 | ## Specify a config file to load. 31 | #c.JupyterApp.config_file_name = '' 32 | 33 | ## Generate default config file. 34 | #c.JupyterApp.generate_config = False 35 | 36 | #------------------------------------------------------------------------------ 37 | # NotebookApp(JupyterApp) configuration 38 | #------------------------------------------------------------------------------ 39 | 40 | ## Set the Access-Control-Allow-Credentials: true header 41 | #c.NotebookApp.allow_credentials = False 42 | 43 | ## Set the Access-Control-Allow-Origin header 44 | # 45 | # Use '*' to allow any origin to access your server. 46 | # 47 | # Takes precedence over allow_origin_pat. 48 | #c.NotebookApp.allow_origin = '' 49 | 50 | ## Use a regular expression for the Access-Control-Allow-Origin header 51 | # 52 | # Requests from an origin matching the expression will get replies with: 53 | # 54 | # Access-Control-Allow-Origin: origin 55 | # 56 | # where `origin` is the origin of the request. 57 | # 58 | # Ignored if allow_origin is set. 59 | #c.NotebookApp.allow_origin_pat = '' 60 | 61 | ## Allow password to be changed at login for the notebook server. 62 | # 63 | # While loggin in with a token, the notebook server UI will give the opportunity 64 | # to the user to enter a new password at the same time that will replace the 65 | # token login mechanism. 66 | # 67 | # This can be set to false to prevent changing password from the UI/API. 68 | #c.NotebookApp.allow_password_change = True 69 | 70 | ## Whether to allow the user to run the notebook as root. 71 | #c.NotebookApp.allow_root = False 72 | c.NotebookApp.allow_root = True 73 | 74 | ## DEPRECATED use base_url 75 | #c.NotebookApp.base_project_url = '/' 76 | 77 | ## The base URL for the notebook server. 78 | # 79 | # Leading and trailing slashes can be omitted, and will automatically be added. 80 | #c.NotebookApp.base_url = '/' 81 | 82 | ## Specify what command to use to invoke a web browser when opening the notebook. 83 | # If not specified, the default browser will be determined by the `webbrowser` 84 | # standard library module, which allows setting of the BROWSER environment 85 | # variable to override it. 86 | #c.NotebookApp.browser = '' 87 | 88 | ## The full path to an SSL/TLS certificate file. 89 | #c.NotebookApp.certfile = '' 90 | 91 | ## The full path to a certificate authority certificate for SSL/TLS client 92 | # authentication. 93 | #c.NotebookApp.client_ca = '' 94 | 95 | ## The config manager class to use 96 | #c.NotebookApp.config_manager_class = 'notebook.services.config.manager.ConfigManager' 97 | 98 | ## The notebook manager class to use. 99 | #c.NotebookApp.contents_manager_class = 'notebook.services.contents.largefilemanager.LargeFileManager' 100 | 101 | ## Extra keyword arguments to pass to `set_secure_cookie`. See tornado's 102 | # set_secure_cookie docs for details. 103 | #c.NotebookApp.cookie_options = {} 104 | 105 | ## The random bytes used to secure cookies. By default this is a new random 106 | # number every time you start the Notebook. Set it to a value in a config file 107 | # to enable logins to persist across server sessions. 108 | # 109 | # Note: Cookie secrets should be kept private, do not share config files with 110 | # cookie_secret stored in plaintext (you can read the value from a file). 111 | #c.NotebookApp.cookie_secret = b'' 112 | 113 | ## The file where the cookie secret is stored. 114 | #c.NotebookApp.cookie_secret_file = '' 115 | 116 | ## The default URL to redirect to from `/` 117 | #c.NotebookApp.default_url = '/tree' 118 | 119 | ## Disable cross-site-request-forgery protection 120 | # 121 | # Jupyter notebook 4.3.1 introduces protection from cross-site request 122 | # forgeries, requiring API requests to either: 123 | # 124 | # - originate from pages served by this server (validated with XSRF cookie and 125 | # token), or - authenticate with a token 126 | # 127 | # Some anonymous compute resources still desire the ability to run code, 128 | # completely without authentication. These services can disable all 129 | # authentication and security checks, with the full knowledge of what that 130 | # implies. 131 | #c.NotebookApp.disable_check_xsrf = False 132 | 133 | ## Whether to enable MathJax for typesetting math/TeX 134 | # 135 | # MathJax is the javascript library Jupyter uses to render math/LaTeX. It is 136 | # very large, so you may want to disable it if you have a slow internet 137 | # connection, or for offline use of the notebook. 138 | # 139 | # When disabled, equations etc. will appear as their untransformed TeX source. 140 | #c.NotebookApp.enable_mathjax = True 141 | 142 | ## extra paths to look for Javascript notebook extensions 143 | #c.NotebookApp.extra_nbextensions_path = [] 144 | 145 | ## handlers that should be loaded at higher priority than the default services 146 | #c.NotebookApp.extra_services = [] 147 | 148 | ## Extra paths to search for serving static files. 149 | # 150 | # This allows adding javascript/css to be available from the notebook server 151 | # machine, or overriding individual files in the IPython 152 | #c.NotebookApp.extra_static_paths = [] 153 | 154 | ## Extra paths to search for serving jinja templates. 155 | # 156 | # Can be used to override templates from notebook.templates. 157 | #c.NotebookApp.extra_template_paths = [] 158 | 159 | ## 160 | #c.NotebookApp.file_to_run = '' 161 | 162 | ## Deprecated: Use minified JS file or not, mainly use during dev to avoid JS 163 | # recompilation 164 | #c.NotebookApp.ignore_minified_js = False 165 | 166 | ## (bytes/sec) Maximum rate at which stream output can be sent on iopub before 167 | # they are limited. 168 | #c.NotebookApp.iopub_data_rate_limit = 1000000 169 | 170 | ## (msgs/sec) Maximum rate at which messages can be sent on iopub before they are 171 | # limited. 172 | #c.NotebookApp.iopub_msg_rate_limit = 1000 173 | 174 | ## The IP address the notebook server will listen on. 175 | #c.NotebookApp.ip = 'localhost' 176 | c.NotebookApp.ip = '0.0.0.0' 177 | 178 | ## Supply extra arguments that will be passed to Jinja environment. 179 | #c.NotebookApp.jinja_environment_options = {} 180 | 181 | ## Extra variables to supply to jinja templates when rendering. 182 | #c.NotebookApp.jinja_template_vars = {} 183 | 184 | ## The kernel manager class to use. 185 | #c.NotebookApp.kernel_manager_class = 'notebook.services.kernels.kernelmanager.MappingKernelManager' 186 | 187 | ## The kernel spec manager class to use. Should be a subclass of 188 | # `jupyter_client.kernelspec.KernelSpecManager`. 189 | # 190 | # The Api of KernelSpecManager is provisional and might change without warning 191 | # between this version of Jupyter and the next stable one. 192 | #c.NotebookApp.kernel_spec_manager_class = 'jupyter_client.kernelspec.KernelSpecManager' 193 | 194 | ## The full path to a private key file for usage with SSL/TLS. 195 | #c.NotebookApp.keyfile = '' 196 | 197 | ## The login handler class to use. 198 | #c.NotebookApp.login_handler_class = 'notebook.auth.login.LoginHandler' 199 | 200 | ## The logout handler class to use. 201 | #c.NotebookApp.logout_handler_class = 'notebook.auth.logout.LogoutHandler' 202 | 203 | ## The MathJax.js configuration file that is to be used. 204 | #c.NotebookApp.mathjax_config = 'TeX-AMS-MML_HTMLorMML-full,Safe' 205 | 206 | ## A custom url for MathJax.js. Should be in the form of a case-sensitive url to 207 | # MathJax, for example: /static/components/MathJax/MathJax.js 208 | #c.NotebookApp.mathjax_url = '' 209 | 210 | ## Dict of Python modules to load as notebook server extensions.Entry values can 211 | # be used to enable and disable the loading ofthe extensions. The extensions 212 | # will be loaded in alphabetical order. 213 | #c.NotebookApp.nbserver_extensions = {} 214 | 215 | ## The directory to use for notebooks and kernels. 216 | #c.NotebookApp.notebook_dir = '' 217 | c.NotebookApp.notebook_dir = '/root/notebook' 218 | 219 | ## Whether to open in a browser after starting. The specific browser used is 220 | # platform dependent and determined by the python standard library `webbrowser` 221 | # module, unless it is overridden using the --browser (NotebookApp.browser) 222 | # configuration option. 223 | #c.NotebookApp.open_browser = True 224 | c.NotebookApp.open_browser = False 225 | 226 | ## Hashed password to use for web authentication. 227 | # 228 | # To generate, type in a python/IPython shell: 229 | # 230 | # from notebook.auth import passwd; passwd() 231 | # 232 | # The string should be of the form type:salt:hashed-password. 233 | c.NotebookApp.password = 'sha1:c1ff390e0b6c:5298138a410f2077b46bf45d61536add40ee5e13' 234 | 235 | ## Forces users to use a password for the Notebook server. This is useful in a 236 | # multi user environment, for instance when everybody in the LAN can access each 237 | # other's machine through ssh. 238 | # 239 | # In such a case, server the notebook server on localhost is not secure since 240 | # any user can connect to the notebook server via ssh. 241 | #c.NotebookApp.password_required = False 242 | 243 | ## The port the notebook server will listen on. 244 | c.NotebookApp.port = 8080 245 | 246 | ## The number of additional ports to try if the specified port is not available. 247 | #c.NotebookApp.port_retries = 50 248 | 249 | ## DISABLED: use %pylab or %matplotlib in the notebook to enable matplotlib. 250 | #c.NotebookApp.pylab = 'disabled' 251 | 252 | ## (sec) Time window used to check the message and data rate limits. 253 | #c.NotebookApp.rate_limit_window = 3 254 | 255 | ## Reraise exceptions encountered loading server extensions? 256 | #c.NotebookApp.reraise_server_extension_failures = False 257 | 258 | ## DEPRECATED use the nbserver_extensions dict instead 259 | #c.NotebookApp.server_extensions = [] 260 | 261 | ## The session manager class to use. 262 | #c.NotebookApp.session_manager_class = 'notebook.services.sessions.sessionmanager.SessionManager' 263 | 264 | ## Shut down the server after N seconds with no kernels or terminals running and 265 | # no activity. This can be used together with culling idle kernels 266 | # (MappingKernelManager.cull_idle_timeout) to shutdown the notebook server when 267 | # it's not in use. This is not precisely timed: it may shut down up to a minute 268 | # later. 0 (the default) disables this automatic shutdown. 269 | #c.NotebookApp.shutdown_no_activity_timeout = 0 270 | 271 | ## Supply SSL options for the tornado HTTPServer. See the tornado docs for 272 | # details. 273 | #c.NotebookApp.ssl_options = {} 274 | 275 | ## Supply overrides for terminado. Currently only supports "shell_command". 276 | #c.NotebookApp.terminado_settings = {} 277 | 278 | ## Token used for authenticating first-time connections to the server. 279 | # 280 | # When no password is enabled, the default is to generate a new, random token. 281 | # 282 | # Setting to an empty string disables authentication altogether, which is NOT 283 | # RECOMMENDED. 284 | #c.NotebookApp.token = '' 285 | 286 | ## Supply overrides for the tornado.web.Application that the Jupyter notebook 287 | # uses. 288 | #c.NotebookApp.tornado_settings = {} 289 | 290 | ## Whether to trust or not X-Scheme/X-Forwarded-Proto and X-Real-Ip/X-Forwarded- 291 | # For headerssent by the upstream reverse proxy. Necessary if the proxy handles 292 | # SSL 293 | #c.NotebookApp.trust_xheaders = False 294 | 295 | ## DEPRECATED, use tornado_settings 296 | #c.NotebookApp.webapp_settings = {} 297 | 298 | ## Specify Where to open the notebook on startup. This is the 299 | # `new` argument passed to the standard library method `webbrowser.open`. 300 | # The behaviour is not guaranteed, but depends on browser support. Valid 301 | # values are: 302 | # 2 opens a new tab, 303 | # 1 opens a new window, 304 | # 0 opens in an existing window. 305 | # See the `webbrowser.open` documentation for details. 306 | #c.NotebookApp.webbrowser_open_new = 2 307 | 308 | ## Set the tornado compression options for websocket connections. 309 | # 310 | # This value will be returned from 311 | # :meth:`WebSocketHandler.get_compression_options`. None (default) will disable 312 | # compression. A dict (even an empty one) will enable compression. 313 | # 314 | # See the tornado docs for WebSocketHandler.get_compression_options for details. 315 | #c.NotebookApp.websocket_compression_options = None 316 | 317 | ## The base URL for websockets, if it differs from the HTTP server (hint: it 318 | # almost certainly doesn't). 319 | # 320 | # Should be in the form of an HTTP origin: ws[s]://hostname[:port] 321 | #c.NotebookApp.websocket_url = '' 322 | 323 | #------------------------------------------------------------------------------ 324 | # ConnectionFileMixin(LoggingConfigurable) configuration 325 | #------------------------------------------------------------------------------ 326 | 327 | ## Mixin for configurable classes that work with connection files 328 | 329 | ## JSON file in which to store connection info [default: kernel-.json] 330 | # 331 | # This file will contain the IP, ports, and authentication key needed to connect 332 | # clients to this kernel. By default, this file will be created in the security 333 | # dir of the current profile, but can be specified by absolute path. 334 | #c.ConnectionFileMixin.connection_file = '' 335 | 336 | ## set the control (ROUTER) port [default: random] 337 | #c.ConnectionFileMixin.control_port = 0 338 | 339 | ## set the heartbeat port [default: random] 340 | #c.ConnectionFileMixin.hb_port = 0 341 | 342 | ## set the iopub (PUB) port [default: random] 343 | #c.ConnectionFileMixin.iopub_port = 0 344 | 345 | ## Set the kernel's IP address [default localhost]. If the IP address is 346 | # something other than localhost, then Consoles on other machines will be able 347 | # to connect to the Kernel, so be careful! 348 | #c.ConnectionFileMixin.ip = '' 349 | 350 | ## set the shell (ROUTER) port [default: random] 351 | #c.ConnectionFileMixin.shell_port = 0 352 | 353 | ## set the stdin (ROUTER) port [default: random] 354 | #c.ConnectionFileMixin.stdin_port = 0 355 | 356 | ## 357 | #c.ConnectionFileMixin.transport = 'tcp' 358 | 359 | #------------------------------------------------------------------------------ 360 | # KernelManager(ConnectionFileMixin) configuration 361 | #------------------------------------------------------------------------------ 362 | 363 | ## Manages a single kernel in a subprocess on this host. 364 | # 365 | # This version starts kernels with Popen. 366 | 367 | ## Should we autorestart the kernel if it dies. 368 | #c.KernelManager.autorestart = True 369 | 370 | ## DEPRECATED: Use kernel_name instead. 371 | # 372 | # The Popen Command to launch the kernel. Override this if you have a custom 373 | # kernel. If kernel_cmd is specified in a configuration file, Jupyter does not 374 | # pass any arguments to the kernel, because it cannot make any assumptions about 375 | # the arguments that the kernel understands. In particular, this means that the 376 | # kernel does not receive the option --debug if it given on the Jupyter command 377 | # line. 378 | #c.KernelManager.kernel_cmd = [] 379 | 380 | ## Time to wait for a kernel to terminate before killing it, in seconds. 381 | #c.KernelManager.shutdown_wait_time = 5.0 382 | 383 | #------------------------------------------------------------------------------ 384 | # Session(Configurable) configuration 385 | #------------------------------------------------------------------------------ 386 | 387 | ## Object for handling serialization and sending of messages. 388 | # 389 | # The Session object handles building messages and sending them with ZMQ sockets 390 | # or ZMQStream objects. Objects can communicate with each other over the 391 | # network via Session objects, and only need to work with the dict-based IPython 392 | # message spec. The Session will handle serialization/deserialization, security, 393 | # and metadata. 394 | # 395 | # Sessions support configurable serialization via packer/unpacker traits, and 396 | # signing with HMAC digests via the key/keyfile traits. 397 | # 398 | # Parameters ---------- 399 | # 400 | # debug : bool 401 | # whether to trigger extra debugging statements 402 | # packer/unpacker : str : 'json', 'pickle' or import_string 403 | # importstrings for methods to serialize message parts. If just 404 | # 'json' or 'pickle', predefined JSON and pickle packers will be used. 405 | # Otherwise, the entire importstring must be used. 406 | # 407 | # The functions must accept at least valid JSON input, and output *bytes*. 408 | # 409 | # For example, to use msgpack: 410 | # packer = 'msgpack.packb', unpacker='msgpack.unpackb' 411 | # pack/unpack : callables 412 | # You can also set the pack/unpack callables for serialization directly. 413 | # session : bytes 414 | # the ID of this Session object. The default is to generate a new UUID. 415 | # username : unicode 416 | # username added to message headers. The default is to ask the OS. 417 | # key : bytes 418 | # The key used to initialize an HMAC signature. If unset, messages 419 | # will not be signed or checked. 420 | # keyfile : filepath 421 | # The file containing a key. If this is set, `key` will be initialized 422 | # to the contents of the file. 423 | 424 | ## Threshold (in bytes) beyond which an object's buffer should be extracted to 425 | # avoid pickling. 426 | #c.Session.buffer_threshold = 1024 427 | 428 | ## Whether to check PID to protect against calls after fork. 429 | # 430 | # This check can be disabled if fork-safety is handled elsewhere. 431 | #c.Session.check_pid = True 432 | 433 | ## Threshold (in bytes) beyond which a buffer should be sent without copying. 434 | #c.Session.copy_threshold = 65536 435 | 436 | ## Debug output in the Session 437 | #c.Session.debug = False 438 | 439 | ## The maximum number of digests to remember. 440 | # 441 | # The digest history will be culled when it exceeds this value. 442 | #c.Session.digest_history_size = 65536 443 | 444 | ## The maximum number of items for a container to be introspected for custom 445 | # serialization. Containers larger than this are pickled outright. 446 | #c.Session.item_threshold = 64 447 | 448 | ## execution key, for signing messages. 449 | #c.Session.key = b'' 450 | 451 | ## path to file containing execution key. 452 | #c.Session.keyfile = '' 453 | 454 | ## Metadata dictionary, which serves as the default top-level metadata dict for 455 | # each message. 456 | #c.Session.metadata = {} 457 | 458 | ## The name of the packer for serializing messages. Should be one of 'json', 459 | # 'pickle', or an import name for a custom callable serializer. 460 | #c.Session.packer = 'json' 461 | 462 | ## The UUID identifying this session. 463 | #c.Session.session = '' 464 | 465 | ## The digest scheme used to construct the message signatures. Must have the form 466 | # 'hmac-HASH'. 467 | #c.Session.signature_scheme = 'hmac-sha256' 468 | 469 | ## The name of the unpacker for unserializing messages. Only used with custom 470 | # functions for `packer`. 471 | #c.Session.unpacker = 'json' 472 | 473 | ## Username for the Session. Default is your system username. 474 | #c.Session.username = 'username' 475 | 476 | #------------------------------------------------------------------------------ 477 | # MultiKernelManager(LoggingConfigurable) configuration 478 | #------------------------------------------------------------------------------ 479 | 480 | ## A class for managing multiple kernels. 481 | 482 | ## The name of the default kernel to start 483 | #c.MultiKernelManager.default_kernel_name = 'python3' 484 | 485 | ## The kernel manager class. This is configurable to allow subclassing of the 486 | # KernelManager for customized behavior. 487 | #c.MultiKernelManager.kernel_manager_class = 'jupyter_client.ioloop.IOLoopKernelManager' 488 | 489 | #------------------------------------------------------------------------------ 490 | # MappingKernelManager(MultiKernelManager) configuration 491 | #------------------------------------------------------------------------------ 492 | 493 | ## A KernelManager that handles notebook mapping and HTTP error handling 494 | 495 | ## Whether messages from kernels whose frontends have disconnected should be 496 | # buffered in-memory. 497 | # 498 | # When True (default), messages are buffered and replayed on reconnect, avoiding 499 | # lost messages due to interrupted connectivity. 500 | # 501 | # Disable if long-running kernels will produce too much output while no 502 | # frontends are connected. 503 | #c.MappingKernelManager.buffer_offline_messages = True 504 | 505 | ## Whether to consider culling kernels which are busy. Only effective if 506 | # cull_idle_timeout > 0. 507 | #c.MappingKernelManager.cull_busy = False 508 | 509 | ## Whether to consider culling kernels which have one or more connections. Only 510 | # effective if cull_idle_timeout > 0. 511 | #c.MappingKernelManager.cull_connected = False 512 | 513 | ## Timeout (in seconds) after which a kernel is considered idle and ready to be 514 | # culled. Values of 0 or lower disable culling. Very short timeouts may result 515 | # in kernels being culled for users with poor network connections. 516 | #c.MappingKernelManager.cull_idle_timeout = 0 517 | 518 | ## The interval (in seconds) on which to check for idle kernels exceeding the 519 | # cull timeout value. 520 | #c.MappingKernelManager.cull_interval = 300 521 | 522 | ## 523 | #c.MappingKernelManager.root_dir = '' 524 | 525 | #------------------------------------------------------------------------------ 526 | # ContentsManager(LoggingConfigurable) configuration 527 | #------------------------------------------------------------------------------ 528 | 529 | ## Base class for serving files and directories. 530 | # 531 | # This serves any text or binary file, as well as directories, with special 532 | # handling for JSON notebook documents. 533 | # 534 | # Most APIs take a path argument, which is always an API-style unicode path, and 535 | # always refers to a directory. 536 | # 537 | # - unicode, not url-escaped 538 | # - '/'-separated 539 | # - leading and trailing '/' will be stripped 540 | # - if unspecified, path defaults to '', 541 | # indicating the root path. 542 | 543 | ## Allow access to hidden files 544 | #c.ContentsManager.allow_hidden = False 545 | 546 | ## 547 | #c.ContentsManager.checkpoints = None 548 | 549 | ## 550 | #c.ContentsManager.checkpoints_class = 'notebook.services.contents.checkpoints.Checkpoints' 551 | 552 | ## 553 | #c.ContentsManager.checkpoints_kwargs = {} 554 | 555 | ## handler class to use when serving raw file requests. 556 | # 557 | # Default is a fallback that talks to the ContentsManager API, which may be 558 | # inefficient, especially for large files. 559 | # 560 | # Local files-based ContentsManagers can use a StaticFileHandler subclass, which 561 | # will be much more efficient. 562 | # 563 | # Access to these files should be Authenticated. 564 | #c.ContentsManager.files_handler_class = 'notebook.files.handlers.FilesHandler' 565 | 566 | ## Extra parameters to pass to files_handler_class. 567 | # 568 | # For example, StaticFileHandlers generally expect a `path` argument specifying 569 | # the root directory from which to serve files. 570 | #c.ContentsManager.files_handler_params = {} 571 | 572 | ## Glob patterns to hide in file and directory listings. 573 | #c.ContentsManager.hide_globs = ['__pycache__', '*.pyc', '*.pyo', '.DS_Store', '*.so', '*.dylib', '*~'] 574 | 575 | ## Python callable or importstring thereof 576 | # 577 | # To be called on a contents model prior to save. 578 | # 579 | # This can be used to process the structure, such as removing notebook outputs 580 | # or other side effects that should not be saved. 581 | # 582 | # It will be called as (all arguments passed by keyword):: 583 | # 584 | # hook(path=path, model=model, contents_manager=self) 585 | # 586 | # - model: the model to be saved. Includes file contents. 587 | # Modifying this dict will affect the file that is stored. 588 | # - path: the API path of the save destination 589 | # - contents_manager: this ContentsManager instance 590 | #c.ContentsManager.pre_save_hook = None 591 | 592 | ## 593 | #c.ContentsManager.root_dir = '/' 594 | 595 | ## The base name used when creating untitled directories. 596 | #c.ContentsManager.untitled_directory = 'Untitled Folder' 597 | 598 | ## The base name used when creating untitled files. 599 | #c.ContentsManager.untitled_file = 'untitled' 600 | 601 | ## The base name used when creating untitled notebooks. 602 | #c.ContentsManager.untitled_notebook = 'Untitled' 603 | 604 | #------------------------------------------------------------------------------ 605 | # FileManagerMixin(Configurable) configuration 606 | #------------------------------------------------------------------------------ 607 | 608 | ## Mixin for ContentsAPI classes that interact with the filesystem. 609 | # 610 | # Provides facilities for reading, writing, and copying both notebooks and 611 | # generic files. 612 | # 613 | # Shared by FileContentsManager and FileCheckpoints. 614 | # 615 | # Note ---- Classes using this mixin must provide the following attributes: 616 | # 617 | # root_dir : unicode 618 | # A directory against against which API-style paths are to be resolved. 619 | # 620 | # log : logging.Logger 621 | 622 | ## By default notebooks are saved on disk on a temporary file and then if 623 | # succefully written, it replaces the old ones. This procedure, namely 624 | # 'atomic_writing', causes some bugs on file system whitout operation order 625 | # enforcement (like some networked fs). If set to False, the new notebook is 626 | # written directly on the old one which could fail (eg: full filesystem or quota 627 | # ) 628 | #c.FileManagerMixin.use_atomic_writing = True 629 | 630 | #------------------------------------------------------------------------------ 631 | # FileContentsManager(FileManagerMixin,ContentsManager) configuration 632 | #------------------------------------------------------------------------------ 633 | 634 | ## If True (default), deleting files will send them to the platform's 635 | # trash/recycle bin, where they can be recovered. If False, deleting files 636 | # really deletes them. 637 | #c.FileContentsManager.delete_to_trash = True 638 | 639 | ## Python callable or importstring thereof 640 | # 641 | # to be called on the path of a file just saved. 642 | # 643 | # This can be used to process the file on disk, such as converting the notebook 644 | # to a script or HTML via nbconvert. 645 | # 646 | # It will be called as (all arguments passed by keyword):: 647 | # 648 | # hook(os_path=os_path, model=model, contents_manager=instance) 649 | # 650 | # - path: the filesystem path to the file just written - model: the model 651 | # representing the file - contents_manager: this ContentsManager instance 652 | #c.FileContentsManager.post_save_hook = None 653 | 654 | ## 655 | #c.FileContentsManager.root_dir = '' 656 | 657 | ## DEPRECATED, use post_save_hook. Will be removed in Notebook 5.0 658 | #c.FileContentsManager.save_script = False 659 | 660 | #------------------------------------------------------------------------------ 661 | # NotebookNotary(LoggingConfigurable) configuration 662 | #------------------------------------------------------------------------------ 663 | 664 | ## A class for computing and verifying notebook signatures. 665 | 666 | ## The hashing algorithm used to sign notebooks. 667 | #c.NotebookNotary.algorithm = 'sha256' 668 | 669 | ## The sqlite file in which to store notebook signatures. By default, this will 670 | # be in your Jupyter data directory. You can set it to ':memory:' to disable 671 | # sqlite writing to the filesystem. 672 | #c.NotebookNotary.db_file = '' 673 | 674 | ## The secret key with which notebooks are signed. 675 | #c.NotebookNotary.secret = b'' 676 | 677 | ## The file where the secret key is stored. 678 | #c.NotebookNotary.secret_file = '' 679 | 680 | ## A callable returning the storage backend for notebook signatures. The default 681 | # uses an SQLite database. 682 | #c.NotebookNotary.store_factory = traitlets.Undefined 683 | 684 | #------------------------------------------------------------------------------ 685 | # KernelSpecManager(LoggingConfigurable) configuration 686 | #------------------------------------------------------------------------------ 687 | 688 | ## If there is no Python kernelspec registered and the IPython kernel is 689 | # available, ensure it is added to the spec list. 690 | #c.KernelSpecManager.ensure_native_kernel = True 691 | 692 | ## The kernel spec class. This is configurable to allow subclassing of the 693 | # KernelSpecManager for customized behavior. 694 | #c.KernelSpecManager.kernel_spec_class = 'jupyter_client.kernelspec.KernelSpec' 695 | 696 | ## Whitelist of allowed kernel names. 697 | # 698 | # By default, all installed kernels are allowed. 699 | #c.KernelSpecManager.whitelist = set() 700 | --------------------------------------------------------------------------------