├── .gitattributes ├── .gitignore ├── .vscode └── settings.json ├── Dockerfile ├── Jenkinsfile ├── Makefile ├── README.md ├── app └── web.py ├── cloudformation ├── 01-network-parameters.json ├── 01-network.yml ├── 02-cluster-parameters.json └── 02-cluster.yml ├── img ├── 01-cfStacks.png ├── 02-networkOutputs.png ├── 03-clusterStackInfo.png ├── 04-clusterOutputs.png ├── 05-jenkinsBuildFail.png ├── 06-jenkinsBuildPass.png ├── 07-dockerImageUploaded.png └── 08-applicationRunning.png ├── jenkinsSetup.md ├── requirements.txt ├── run_docker.sh ├── run_kubernetes.sh ├── scripts ├── awsCoUp.sh ├── awsCreate.sh └── awsUpdate.sh ├── steps.md └── tests └── test_web.py /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *__pycache__ 2 | *.coverage 3 | *.pyc 4 | *pytest_cache 5 | -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "python.formatting.provider": "black" 3 | } -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.7.3-stretch 2 | 3 | # Working Directory 4 | WORKDIR /app 5 | 6 | # Copy source code to working directory 7 | COPY . app/web.py /app/ 8 | 9 | # Install packages from requirements.txt 10 | # hadolint ignore=DL3013 11 | RUN pip install --upgrade pip &&\ 12 | pip install --trusted-host pypi.python.org -r requirements.txt 13 | 14 | # Expose port 80 15 | EXPOSE 80 16 | 17 | # Run app.py at container launch 18 | CMD ["python", "web.py"] 19 | -------------------------------------------------------------------------------- /Jenkinsfile: -------------------------------------------------------------------------------- 1 | pipeline { 2 | agent any 3 | stages { 4 | stage('Install requirements') { 5 | steps { 6 | sh 'make install' 7 | } 8 | } 9 | stage('Lint files') { 10 | steps { 11 | sh 'make lint' 12 | } 13 | } 14 | stage('Build and upload to ECR') { 15 | steps { 16 | sh 'rm -f ~/.dockercfg' 17 | sh 'rm -f ~/.docker/config.json' 18 | script { 19 | docker.withRegistry('https://119841056280.dkr.ecr.us-east-1.amazonaws.com', 'ecr:us-east-1:AwsCreds') { 20 | def customImage = docker.build("nano-devops-05:latest") 21 | customImage.push() 22 | } 23 | } 24 | } 25 | } 26 | stage('Deploy to EKS') { 27 | steps { 28 | withCredentials([[ 29 | $class: 'AmazonWebServicesCredentialsBinding', 30 | credentialsId: 'AwsCreds', 31 | accessKeyVariable: 'AWS_ACCESS_KEY_ID', 32 | secretKeyVariable: 'AWS_SECRET_ACCESS_KEY' 33 | ]]) { 34 | sh 'AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} AWS_DEFAULT_REGION=us-east-1 aws ecs update-service --cluster NanoDevops05 --service helloWorld --force-new-deployment' 35 | } 36 | } 37 | } 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | help: ## Show this help. 2 | @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' 3 | 4 | setup: ## Create the virtial environment to run this project 5 | python3 -m venv ~/.nano-devops-05 6 | 7 | env: ## List the environment versions 8 | which python3 9 | python3 --version 10 | which pytest 11 | which pylint 12 | 13 | lint: ## Check the validity of the project files 14 | hadolint Dockerfile 15 | pylint --load-plugins pylint_flask --disable=R,C app/*.py 16 | 17 | test: ## Run the tests for this prject 18 | @cd tests; pytest -vv --cov-report term-missing --cov=web test_*.py 19 | 20 | install: ## Install the required imports for this project 21 | pip install -r requirements.txt 22 | 23 | docker-build: ## Build the docker image and list available docker images 24 | docker build -t maweeks/nano-devops-05 . 25 | docker image ls 26 | 27 | docker-upload: ## Upload the docker image to AWS 28 | $(aws ecr get-login --no-include-email --region us-east-1) 29 | docker tag nano-devops-05:latest 119841056280.dkr.ecr.us-east-1.amazonaws.com/nano-devops-05:latest 30 | docker push 119841056280.dkr.ecr.us-east-1.amazonaws.com/nano-devops-05 31 | 32 | start-api: ## Run the python application locally 33 | python web.py 34 | 35 | all: install lint test 36 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Capstone Project (Cloud DevOps Engineer) 2 | 3 | Capstone project for Udacity Dev-Ops Nanodegree: develop a CI/CD pipeline for micro services applications with either blue/green deployment or rolling deployment. 4 | 5 | ## CI CD Pipeline 6 | 7 | The CI CD pipeline is as follows for this project. For initial set up: 8 | 9 | - Set up jenkins 10 | - Install dependencies for local development 11 | - Create AWS infrastructure 12 | 13 | For application development: 14 | 15 | - Make development change 16 | - Commit to git 17 | - Update AWS stack using `./awsCoUp.sh` commands listed below if required 18 | - Push to repository 19 | - Jenkins build automatically runs based on triggers 20 | - Files are linted 21 | - Docker image build and uploaded to ECR 22 | - ECS tasks updated to run new Docker image using rolling deployment 23 | 24 | ## Running the stuff 25 | 26 | There is a `Makefile` that contains lots of useful commands. 27 | Running `make` will list them, like the below output. 28 | 29 | ```text 30 | help: Show this help. 31 | setup: Create the virtial environment to run this project 32 | env: List the environment versions 33 | lint: Check the validity of the project files 34 | test: Run the tests for this prject 35 | install: Install the required imports for this project 36 | docker-build: Build the docker image and list available docker images 37 | docker-upload: Upload the docker image to AWS 38 | start-api: Run the python application locally 39 | ``` 40 | 41 | ## Creating the infrastructure 42 | 43 | From the base of this repository run the following to create or update the deployment. 44 | 45 | ```bash 46 | aws configure # set to credentials with appropriate access 47 | cd scripts; ./awsCoUp.sh network ../cloudformation/01-network.yml ../cloudformation/01-network-parameters.json; cd .. 48 | # Wait for the network stack to be complete 49 | cd scripts; ./awsCoUp.sh cluster ../cloudformation/02-cluster.yml ../cloudformation/02-cluster-parameters.json; cd .. 50 | ``` 51 | 52 | ## Jenkins set up 53 | 54 | Full jenkins set up details can be found in `jenkinsSetup.md`. 55 | 56 | ## Output 57 | 58 | From running the above the following will be created. 59 | 60 | The application is currently running [here](http://clust-applb-c1p26laf80l1-69580312.us-east-1.elb.amazonaws.com/) is a link to the load balancer. 61 | 62 | AWS Stacks: 63 | 64 | ![CloudFormation stacks](./img/01-cfStacks.png) 65 | ![CloudFormation network stack outputs](./img/02-networkOutputs.png) 66 | ![CloudFormation cluster stack info](./img/03-clusterStackInfo.png) 67 | ![CloudFormation cluster stack outputs](./img/04-clusterOutputs.png) 68 | 69 | Jenkins build when the lint fails: 70 | ![Jenkins lint failing](./img/05-jenkinsBuildFail.png) 71 | 72 | Jenkins build when the lint passes and deploys to ECR: 73 | ![Jenkins lint passing and deploying to AWS](./img/06-jenkinsBuildPass.png) 74 | 75 | ECR with new image uploaded: 76 | ![ECR docker image uploaded](./img/07-dockerImageUploaded.png) 77 | 78 | Hello world application running: 79 | ![Application running in AWS](./img/08-applicationRunning.png) 80 | 81 |

Author

82 | GitHub 83 | -------------------------------------------------------------------------------- /app/web.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, escape, request 2 | from flasgger import Swagger 3 | from sensible.loginit import logger 4 | 5 | log = logger(__name__) 6 | app = Flask(__name__) 7 | Swagger(app) 8 | 9 | 10 | @app.route("/") 11 | def hello(): 12 | name = request.args.get("name", "World") 13 | return "Hello, " + escape(name) + "!" 14 | 15 | 16 | @app.route("/health") 17 | def health(): 18 | return {"isRunning": True} 19 | 20 | 21 | if __name__ == "__main__": # pragma: no cover 22 | log.info("START Flask") 23 | app.debug = True 24 | app.run(host="0.0.0.0", port=80) 25 | log.info("SHUTDOWN Flask") 26 | -------------------------------------------------------------------------------- /cloudformation/01-network-parameters.json: -------------------------------------------------------------------------------- 1 | [ 2 | { "ParameterKey": "EnvironmentName", "ParameterValue": "NanoDevops05" }, 3 | { "ParameterKey": "VpcCIDR", "ParameterValue": "10.0.0.0/16" }, 4 | { "ParameterKey": "PublicSubnet1CIDR", "ParameterValue": "10.0.0.0/24" }, 5 | { "ParameterKey": "PublicSubnet2CIDR", "ParameterValue": "10.0.1.0/24" }, 6 | { "ParameterKey": "PrivateSubnet1CIDR", "ParameterValue": "10.0.2.0/24" }, 7 | { "ParameterKey": "PrivateSubnet2CIDR", "ParameterValue": "10.0.3.0/24" } 8 | ] 9 | -------------------------------------------------------------------------------- /cloudformation/01-network.yml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Description: > 3 | Full network setup for Fargate deployment. 4 | 5 | Parameters: 6 | EnvironmentName: 7 | Description: An environment name that will be prefixed to resource names 8 | Type: String 9 | 10 | VpcCIDR: 11 | Description: Please enter the IP range (CIDR notation) for this VPC 12 | Type: String 13 | Default: 10.0.0.0/16 14 | 15 | PublicSubnet1CIDR: 16 | Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone 17 | Type: String 18 | Default: 10.0.0.0/24 19 | 20 | PublicSubnet2CIDR: 21 | Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone 22 | Type: String 23 | Default: 10.0.1.0/24 24 | 25 | PrivateSubnet1CIDR: 26 | Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone 27 | Type: String 28 | Default: 10.0.2.0/24 29 | 30 | PrivateSubnet2CIDR: 31 | Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone 32 | Type: String 33 | Default: 10.0.3.0/24 34 | 35 | Resources: 36 | VPC: 37 | Type: AWS::EC2::VPC 38 | Properties: 39 | CidrBlock: !Ref VpcCIDR 40 | EnableDnsSupport: true 41 | EnableDnsHostnames: true 42 | Tags: 43 | - Key: Name 44 | Value: !Ref EnvironmentName 45 | 46 | InternetGateway: 47 | Type: AWS::EC2::InternetGateway 48 | Properties: 49 | Tags: 50 | - Key: Name 51 | Value: !Ref EnvironmentName 52 | 53 | InternetGatewayAttachment: 54 | Type: AWS::EC2::VPCGatewayAttachment 55 | Properties: 56 | InternetGatewayId: !Ref InternetGateway 57 | VpcId: !Ref VPC 58 | 59 | PublicSubnet1: 60 | Type: AWS::EC2::Subnet 61 | Properties: 62 | VpcId: !Ref VPC 63 | AvailabilityZone: !Select [0, !GetAZs ''] 64 | CidrBlock: !Ref PublicSubnet1CIDR 65 | MapPublicIpOnLaunch: true 66 | Tags: 67 | - Key: Name 68 | Value: !Sub ${EnvironmentName} Public Subnet (AZ1) 69 | 70 | PublicSubnet2: 71 | Type: AWS::EC2::Subnet 72 | Properties: 73 | VpcId: !Ref VPC 74 | AvailabilityZone: !Select [1, !GetAZs ''] 75 | CidrBlock: !Ref PublicSubnet2CIDR 76 | MapPublicIpOnLaunch: true 77 | Tags: 78 | - Key: Name 79 | Value: !Sub ${EnvironmentName} Public Subnet (AZ2) 80 | 81 | PrivateSubnet1: 82 | Type: AWS::EC2::Subnet 83 | Properties: 84 | VpcId: !Ref VPC 85 | AvailabilityZone: !Select [0, !GetAZs ''] 86 | CidrBlock: !Ref PrivateSubnet1CIDR 87 | MapPublicIpOnLaunch: false 88 | Tags: 89 | - Key: Name 90 | Value: !Sub ${EnvironmentName} Private Subnet (AZ1) 91 | 92 | PrivateSubnet2: 93 | Type: AWS::EC2::Subnet 94 | Properties: 95 | VpcId: !Ref VPC 96 | AvailabilityZone: !Select [1, !GetAZs ''] 97 | CidrBlock: !Ref PrivateSubnet2CIDR 98 | MapPublicIpOnLaunch: false 99 | Tags: 100 | - Key: Name 101 | Value: !Sub ${EnvironmentName} Private Subnet (AZ2) 102 | 103 | NatGateway1EIP: 104 | Type: AWS::EC2::EIP 105 | DependsOn: InternetGatewayAttachment 106 | Properties: 107 | Domain: vpc 108 | 109 | NatGateway2EIP: 110 | Type: AWS::EC2::EIP 111 | DependsOn: InternetGatewayAttachment 112 | Properties: 113 | Domain: vpc 114 | 115 | NatGateway1: 116 | Type: AWS::EC2::NatGateway 117 | Properties: 118 | AllocationId: !GetAtt NatGateway1EIP.AllocationId 119 | SubnetId: !Ref PublicSubnet1 120 | 121 | NatGateway2: 122 | Type: AWS::EC2::NatGateway 123 | Properties: 124 | AllocationId: !GetAtt NatGateway2EIP.AllocationId 125 | SubnetId: !Ref PublicSubnet2 126 | 127 | PublicRouteTable: 128 | Type: AWS::EC2::RouteTable 129 | Properties: 130 | VpcId: !Ref VPC 131 | Tags: 132 | - Key: Name 133 | Value: !Sub ${EnvironmentName} Public Routes 134 | 135 | DefaultPublicRoute: 136 | Type: AWS::EC2::Route 137 | DependsOn: InternetGatewayAttachment 138 | Properties: 139 | RouteTableId: !Ref PublicRouteTable 140 | DestinationCidrBlock: 0.0.0.0/0 141 | GatewayId: !Ref InternetGateway 142 | 143 | PublicSubnet1RouteTableAssociation: 144 | Type: AWS::EC2::SubnetRouteTableAssociation 145 | Properties: 146 | RouteTableId: !Ref PublicRouteTable 147 | SubnetId: !Ref PublicSubnet1 148 | 149 | PublicSubnet2RouteTableAssociation: 150 | Type: AWS::EC2::SubnetRouteTableAssociation 151 | Properties: 152 | RouteTableId: !Ref PublicRouteTable 153 | SubnetId: !Ref PublicSubnet2 154 | 155 | PrivateRouteTable1: 156 | Type: AWS::EC2::RouteTable 157 | Properties: 158 | VpcId: !Ref VPC 159 | Tags: 160 | - Key: Name 161 | Value: !Sub ${EnvironmentName} Private Routes (AZ1) 162 | 163 | DefaultPrivateRoute1: 164 | Type: AWS::EC2::Route 165 | Properties: 166 | RouteTableId: !Ref PrivateRouteTable1 167 | DestinationCidrBlock: 0.0.0.0/0 168 | NatGatewayId: !Ref NatGateway1 169 | 170 | PrivateSubnet1RouteTableAssociation: 171 | Type: AWS::EC2::SubnetRouteTableAssociation 172 | Properties: 173 | RouteTableId: !Ref PrivateRouteTable1 174 | SubnetId: !Ref PrivateSubnet1 175 | 176 | PrivateRouteTable2: 177 | Type: AWS::EC2::RouteTable 178 | Properties: 179 | VpcId: !Ref VPC 180 | Tags: 181 | - Key: Name 182 | Value: !Sub ${EnvironmentName} Private Routes (AZ2) 183 | 184 | DefaultPrivateRoute2: 185 | Type: AWS::EC2::Route 186 | Properties: 187 | RouteTableId: !Ref PrivateRouteTable2 188 | DestinationCidrBlock: 0.0.0.0/0 189 | NatGatewayId: !Ref NatGateway2 190 | 191 | PrivateSubnet2RouteTableAssociation: 192 | Type: AWS::EC2::SubnetRouteTableAssociation 193 | Properties: 194 | RouteTableId: !Ref PrivateRouteTable2 195 | SubnetId: !Ref PrivateSubnet2 196 | 197 | Outputs: 198 | VPC: 199 | Description: A reference to the created VPC 200 | Value: !Ref VPC 201 | Export: 202 | Name: !Sub ${EnvironmentName}-VPCID 203 | 204 | VPCPublicRouteTable: 205 | Description: Public Routing 206 | Value: !Ref PublicRouteTable 207 | Export: 208 | Name: !Sub ${EnvironmentName}-PUB-RT 209 | 210 | VPCPrivateRouteTable1: 211 | Description: Private Routing AZ1 212 | Value: !Ref PrivateRouteTable1 213 | Export: 214 | Name: !Sub ${EnvironmentName}-PRI1-RT 215 | 216 | VPCPrivateRouteTable2: 217 | Description: Private Routing AZ2 218 | Value: !Ref PrivateRouteTable2 219 | Export: 220 | Name: !Sub ${EnvironmentName}-PRI2-RT 221 | 222 | PublicSubnets: 223 | Description: A list of the public subnets 224 | Value: !Join [',', [!Ref PublicSubnet1, !Ref PublicSubnet2]] 225 | Export: 226 | Name: !Sub ${EnvironmentName}-PUB-NETS 227 | 228 | PrivateSubnets: 229 | Description: A list of the private subnets 230 | Value: !Join [',', [!Ref PrivateSubnet1, !Ref PrivateSubnet2]] 231 | Export: 232 | Name: !Sub ${EnvironmentName}-PRIV-NETS 233 | 234 | PublicSubnet1: 235 | Description: A reference to the public subnet in the 1st Availability Zone 236 | Value: !Ref PublicSubnet1 237 | Export: 238 | Name: !Sub ${EnvironmentName}-PUB1-SN 239 | 240 | PublicSubnet2: 241 | Description: A reference to the public subnet in the 2nd Availability Zone 242 | Value: !Ref PublicSubnet2 243 | Export: 244 | Name: !Sub ${EnvironmentName}-PUB2-SN 245 | 246 | PrivateSubnet1: 247 | Description: A reference to the private subnet in the 1st Availability Zone 248 | Value: !Ref PrivateSubnet1 249 | Export: 250 | Name: !Sub ${EnvironmentName}-PRI1-SN 251 | 252 | PrivateSubnet2: 253 | Description: A reference to the private subnet in the 2nd Availability Zone 254 | Value: !Ref PrivateSubnet2 255 | Export: 256 | Name: !Sub ${EnvironmentName}-PRI2-SN 257 | -------------------------------------------------------------------------------- /cloudformation/02-cluster-parameters.json: -------------------------------------------------------------------------------- 1 | [ 2 | { "ParameterKey": "DesiredCount", "ParameterValue": "1" }, 3 | { "ParameterKey": "EnvironmentName", "ParameterValue": "NanoDevops05" }, 4 | { "ParameterKey": "ImageUrl", "ParameterValue": "119841056280.dkr.ecr.us-east-1.amazonaws.com/nano-devops-05:latest" }, 5 | { "ParameterKey": "ServiceName", "ParameterValue": "helloWorld" } 6 | ] 7 | -------------------------------------------------------------------------------- /cloudformation/02-cluster.yml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Description: > 3 | Full cluster setup for Fargate deployment. 4 | 5 | Parameters: 6 | DesiredCount: 7 | Type: Number 8 | Default: 2 9 | Description: How many copies of the service task to run 10 | EnvironmentName: 11 | Description: An environment name that will be prefixed to resource names 12 | Type: String 13 | ImageUrl: 14 | Type: String 15 | Default: 119841056280.dkr.ecr.us-east-1.amazonaws.com/nano-devops-05:latest 16 | Description: Docker image url to be deployed 17 | ServiceName: 18 | Type: String 19 | Default: helloWorld 20 | Description: Name of the service 21 | 22 | Resources: 23 | ContainerSG: 24 | Type: AWS::EC2::SecurityGroup 25 | Properties: 26 | GroupDescription: Access to the Fargate containers 27 | VpcId: 28 | Fn::ImportValue: !Sub '${EnvironmentName}-VPCID' 29 | 30 | AppLBSG: 31 | Type: AWS::EC2::SecurityGroup 32 | Properties: 33 | GroupDescription: Access to the load balancer 34 | VpcId: 35 | Fn::ImportValue: !Sub '${EnvironmentName}-VPCID' 36 | SecurityGroupIngress: 37 | - CidrIp: 0.0.0.0/0 38 | IpProtocol: -1 39 | 40 | EcsSecurityGroupIngressFromPublicALB: 41 | Type: AWS::EC2::SecurityGroupIngress 42 | Properties: 43 | Description: Ingress from the public ALB 44 | GroupId: !Ref ContainerSG 45 | IpProtocol: -1 46 | SourceSecurityGroupId: !Ref AppLBSG 47 | 48 | EcsSecurityGroupIngressFromSelf: 49 | Type: AWS::EC2::SecurityGroupIngress 50 | Properties: 51 | Description: Ingress from other containers in the same security group 52 | GroupId: !Ref ContainerSG 53 | IpProtocol: -1 54 | SourceSecurityGroupId: !Ref ContainerSG 55 | 56 | AppLB: 57 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 58 | Properties: 59 | Scheme: internet-facing 60 | LoadBalancerAttributes: 61 | - Key: idle_timeout.timeout_seconds 62 | Value: '15' 63 | Subnets: 64 | - Fn::ImportValue: !Sub '${EnvironmentName}-PUB1-SN' 65 | - Fn::ImportValue: !Sub '${EnvironmentName}-PUB2-SN' 66 | SecurityGroups: [!Ref AppLBSG] 67 | 68 | AppLBListener: 69 | Type: AWS::ElasticLoadBalancingV2::Listener 70 | DependsOn: 71 | - AppLB 72 | Properties: 73 | DefaultActions: 74 | - TargetGroupArn: !Ref AppTargetGroup 75 | Type: 'forward' 76 | LoadBalancerArn: !Ref AppLB 77 | Port: 80 78 | Protocol: HTTP 79 | 80 | ECSRole: 81 | Type: AWS::IAM::Role 82 | Properties: 83 | AssumeRolePolicyDocument: 84 | Statement: 85 | - Effect: Allow 86 | Principal: 87 | Service: [ecs.amazonaws.com] 88 | Action: ['sts:AssumeRole'] 89 | Path: / 90 | Policies: 91 | - PolicyName: ecs-service 92 | PolicyDocument: 93 | Statement: 94 | - Effect: Allow 95 | Action: 96 | - 'ec2:AttachNetworkInterface' 97 | - 'ec2:CreateNetworkInterface' 98 | - 'ec2:CreateNetworkInterfacePermission' 99 | - 'ec2:DeleteNetworkInterface' 100 | - 'ec2:DeleteNetworkInterfacePermission' 101 | - 'ec2:Describe*' 102 | - 'ec2:DetachNetworkInterface' 103 | - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer' 104 | - 'elasticloadbalancing:DeregisterTargets' 105 | - 'elasticloadbalancing:Describe*' 106 | - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer' 107 | - 'elasticloadbalancing:RegisterTargets' 108 | Resource: '*' 109 | 110 | ECSTaskExecutionRole: 111 | Type: AWS::IAM::Role 112 | Properties: 113 | AssumeRolePolicyDocument: 114 | Statement: 115 | - Effect: Allow 116 | Principal: 117 | Service: [ecs-tasks.amazonaws.com] 118 | Action: ['sts:AssumeRole'] 119 | Path: / 120 | Policies: 121 | - PolicyName: AmazonECSTaskExecutionRolePolicy 122 | PolicyDocument: 123 | Statement: 124 | - Effect: Allow 125 | Action: 126 | - 'ecr:GetAuthorizationToken' 127 | - 'ecr:BatchCheckLayerAvailability' 128 | - 'ecr:GetDownloadUrlForLayer' 129 | - 'ecr:BatchGetImage' 130 | - 'logs:CreateLogStream' 131 | - 'logs:PutLogEvents' 132 | Resource: '*' 133 | 134 | Cluster: 135 | Type: AWS::ECS::Cluster 136 | Properties: 137 | ClusterName: !Ref EnvironmentName 138 | 139 | TaskDefinition: 140 | Type: AWS::ECS::TaskDefinition 141 | Properties: 142 | Family: !Ref ServiceName 143 | Cpu: 256 144 | Memory: 512 145 | NetworkMode: awsvpc 146 | RequiresCompatibilities: 147 | - FARGATE 148 | ExecutionRoleArn: !Ref ECSTaskExecutionRole 149 | ContainerDefinitions: 150 | - Name: !Ref ServiceName 151 | Cpu: 256 152 | Memory: 512 153 | Image: !Ref ImageUrl 154 | PortMappings: 155 | - ContainerPort: 80 156 | 157 | Service: 158 | Type: AWS::ECS::Service 159 | DependsOn: LoadBalancerRule 160 | Properties: 161 | ServiceName: !Ref ServiceName 162 | Cluster: !Ref Cluster 163 | LaunchType: FARGATE 164 | DeploymentConfiguration: 165 | MaximumPercent: 200 166 | MinimumHealthyPercent: 100 167 | DesiredCount: !Ref DesiredCount 168 | NetworkConfiguration: 169 | AwsvpcConfiguration: 170 | AssignPublicIp: ENABLED 171 | SecurityGroups: 172 | - !Ref ContainerSG 173 | Subnets: 174 | - Fn::ImportValue: !Sub '${EnvironmentName}-PRI1-SN' 175 | - Fn::ImportValue: !Sub '${EnvironmentName}-PRI2-SN' 176 | TaskDefinition: !Ref TaskDefinition 177 | LoadBalancers: 178 | - ContainerName: !Ref ServiceName 179 | ContainerPort: 80 180 | TargetGroupArn: !Ref AppTargetGroup 181 | 182 | AppTargetGroup: 183 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 184 | Properties: 185 | HealthCheckIntervalSeconds: 6 186 | HealthCheckPath: / 187 | HealthCheckProtocol: HTTP 188 | HealthCheckTimeoutSeconds: 5 189 | HealthyThresholdCount: 2 190 | TargetType: ip 191 | Name: !Ref ServiceName 192 | Port: 80 193 | Protocol: HTTP 194 | UnhealthyThresholdCount: 2 195 | VpcId: 196 | Fn::ImportValue: !Sub '${EnvironmentName}-VPCID' 197 | 198 | LoadBalancerRule: 199 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 200 | Properties: 201 | Actions: 202 | - TargetGroupArn: !Ref AppTargetGroup 203 | Type: 'forward' 204 | Conditions: 205 | - Field: path-pattern 206 | Values: ['*'] 207 | ListenerArn: !Ref AppLBListener 208 | Priority: 1 209 | 210 | Outputs: 211 | AppLBURL: 212 | Description: URL of public web app 213 | Value: !Sub 'http://${AppLB.DNSName}' 214 | Export: 215 | Name: !Sub '${EnvironmentName}-AppLB-URL' 216 | -------------------------------------------------------------------------------- /img/01-cfStacks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/01-cfStacks.png -------------------------------------------------------------------------------- /img/02-networkOutputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/02-networkOutputs.png -------------------------------------------------------------------------------- /img/03-clusterStackInfo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/03-clusterStackInfo.png -------------------------------------------------------------------------------- /img/04-clusterOutputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/04-clusterOutputs.png -------------------------------------------------------------------------------- /img/05-jenkinsBuildFail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/05-jenkinsBuildFail.png -------------------------------------------------------------------------------- /img/06-jenkinsBuildPass.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/06-jenkinsBuildPass.png -------------------------------------------------------------------------------- /img/07-dockerImageUploaded.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/07-dockerImageUploaded.png -------------------------------------------------------------------------------- /img/08-applicationRunning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aiwithqasim/Capstone-Project-Devops/d3948477cd92d7df418e9831652052beb2fd73f2/img/08-applicationRunning.png -------------------------------------------------------------------------------- /jenkinsSetup.md: -------------------------------------------------------------------------------- 1 | # Jenkins Set Up 2 | 3 | ## Install Jenkins and dependencies 4 | 5 | Install Jenkins on new EC2 Ubuntu 18 box: 6 | 7 | ```bash 8 | sudo apt-get update 9 | sudo apt install -y default-jdk 10 | wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add - 11 | sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list' 12 | sudo apt-get update 13 | sudo apt-get install -y jenkins 14 | sudo cat /var/lib/jenkins/secrets/initialAdminPassword 15 | wget -O /bin/hadolint https://github.com/hadolint/hadolint/releases/download/v1.17.5/hadolint-Linux-x86_64 16 | sudo wget -O /bin/hadolint https://github.com/hadolint/hadolint/releases/download/v1.17.5/hadolint-Linux-x86_64 17 | sudo chmod +x /bin/hadolint 18 | sudo apt-get install build-essential 19 | apt install python-pip 20 | sudo apt install python-pip 21 | pylint 22 | sudo apt install pylint 23 | docker 24 | sudo apt install docker.io 25 | sudo usermod -a -G docker jenkins 26 | sudo service jenkins restart 27 | ``` 28 | 29 | ## Plugins 30 | 31 | Install the following plugins: 32 | 33 | - Amazon ECR 34 | - Blue Ocean 35 | - Blue Ocean Executor Info 36 | - Blue Ocean Pipeline Editor 37 | - Cloudbees AWS Credentials 38 | - Config API for Blue Ocean 39 | - Display URL for Blue Ocean 40 | - Docker 41 | - Events API for Blue Ocean 42 | - Git Pipeline for Blue Ocean 43 | - GitHub Pipeline for Blue Ocean 44 | - Pipeline: AWS Steps 45 | - Pipeline implementation for Blue Ocean 46 | 47 | ## Further Jenkins set up 48 | 49 | After the above add the aws credentials to jenkins credential store and connect to source control. 50 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | escape 2 | flasgger 3 | Flask 4 | pylint 5 | pylint-flask 6 | request 7 | sensible -------------------------------------------------------------------------------- /run_docker.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | ## Complete the following steps to get Docker running locally 4 | 5 | # Build image and add a descriptive tag 6 | docker build -t maweeks/nano-devops-05 . 7 | 8 | # List docker images 9 | docker image ls 10 | 11 | # Run flask app 12 | docker run -p 8000:80 maweeks/nano-devops-05 13 | -------------------------------------------------------------------------------- /run_kubernetes.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | ## Complete the following steps to get kubernetes pods running locally 4 | 5 | # This is your Docker ID/path 6 | dockerpath=maweeks/nano-devops-05 7 | 8 | # Run the Docker Hub container with kubernetes 9 | kubectl run nano-devops-05\ 10 | --generator=run-pod/v1\ 11 | --image=$dockerpath\ 12 | --port=80 --labels app=nano-devops-05 13 | 14 | # List kubernetes pods 15 | kubectl get pod 16 | 17 | # Forward the container port to a host 18 | kubectl port-forward nano-devops-05 8000:80 19 | -------------------------------------------------------------------------------- /scripts/awsCoUp.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if ./awsCreate.sh $1 $2 $3 ; then 4 | echo "Create succeeded" 5 | elif ./awsUpdate.sh $1 $2 $3 ; then 6 | echo "Update succeeded" 7 | else 8 | echo "CoUp failed" 9 | fi -------------------------------------------------------------------------------- /scripts/awsCreate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | aws cloudformation create-stack \ 4 | --stack-name $1 \ 5 | --template-body file://$2 \ 6 | --parameters file://$3 \ 7 | --region=us-east-1 \ 8 | --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM 9 | -------------------------------------------------------------------------------- /scripts/awsUpdate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | aws cloudformation update-stack \ 4 | --stack-name $1 \ 5 | --template-body file://$2 \ 6 | --parameters file://$3 \ 7 | --region=us-east-1 \ 8 | --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM 9 | -------------------------------------------------------------------------------- /steps.md: -------------------------------------------------------------------------------- 1 | # Steps in Completing Your Project 2 | 3 | ## Step 1: Propose and Scope the Project 4 | 5 | - Plan what your pipeline will look like. 6 | 7 | - Checkout 8 | - Lint 9 | - Build 10 | - Push 11 | - Deploy (rolling) 12 | 13 | - Decide which options you will include in your Continuous Integration phase. 14 | - Use Jenkins. 15 | - Pick a deployment type - either rolling deployment or blue/green deployment. 16 | - For the Docker application you can either use an application which you come up with, or use an open-source application pulled from the Internet, or if you have no idea, you can use an Nginx “Hello World, my name is (student name)” application. 17 | 18 | ## Step 2: Use Jenkins, and implement blue/green or rolling deployment 19 | 20 | - Create your Jenkins master box with either Jenkins and install the plugins you will need. 21 | - Set up your environment to which you will deploy code. 22 | 23 | ## Step 3: Pick AWS Kubernetes as a Service, or build your own Kubernetes cluster 24 | 25 | - Use Ansible or CloudFormation to build your “infrastructure”; i.e., the Kubernetes Cluster. 26 | - It should create the EC2 instances (if you are building your own), set the correct networking settings, and deploy software to these instances. 27 | - As a final step, the Kubernetes cluster will need to be initialized. The Kubernetes cluster initialization can either be done by hand, or with Ansible/Cloudformation at the student’s discretion. 28 | 29 | ## Step 4: Build your pipeline 30 | 31 | - Construct your pipeline in your GitHub repository. 32 | - Set up all the steps that your pipeline will include. 33 | - Configure a deployment pipeline. 34 | - Include your Dockerfile/source code in the Git repository. 35 | - Include with your Linting step both a failed Linting screenshot and a successful Linting screenshot to show the Linter working properly. 36 | 37 | ## Step 5: Test your pipeline 38 | 39 | - Perform builds on your pipeline. 40 | - Verify that your pipeline works as you designed it. 41 | - Take a screenshot of the Jenkins pipeline showing deployment and a screenshot of your AWS EC2 page showing the newly created (for blue/green) or modified (for rolling) instances. Make sure you name your instances differently between blue and green deployments. 42 | -------------------------------------------------------------------------------- /tests/test_web.py: -------------------------------------------------------------------------------- 1 | """pytest tests for library""" 2 | 3 | import base64 4 | import json 5 | import pytest 6 | import sys 7 | 8 | sys.path.append("../app") 9 | 10 | from web import app 11 | 12 | 13 | @pytest.fixture 14 | def client(): 15 | """Generic Flask application fixture""" 16 | 17 | app.testing = True 18 | return app.test_client() 19 | 20 | 21 | def test_root(client): 22 | """Tests that a redirect is in place for root""" 23 | 24 | # A GET request should return a 200 25 | res_get = client.get("/") 26 | assert res_get.status_code == 200 27 | 28 | 29 | def test_health(client): 30 | """Tests that the health endpoint is working""" 31 | 32 | # A GET request should return a 200 33 | res_get = client.get("/health") 34 | assert res_get.status_code == 200 35 | assert res_get.json == {"isRunning": True} 36 | --------------------------------------------------------------------------------