8 |
9 | 
10 |
11 | In this workshop, we will explore...
12 |
--------------------------------------------------------------------------------
/content/appendix/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Appendix"
3 | chapter: true
4 | weight: 65
5 | ---
6 |
7 | # Appendix
8 |
--------------------------------------------------------------------------------
/content/appendix/servicediscovery.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Adding service discovery to EC2 backends"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 5
5 | ---
6 |
7 | The following SSM Document lets you register / deregister EC2 instances belonging to an Auto Scaling Group in Cloud Map service using servicectl.
8 |
9 | * Execute the following script to create the SSM document
10 |
11 | ```bash
12 | cat <<-"EOF" > /tmp/configure_cloudmap.yml
13 | ---
14 | schemaVersion: '2.2'
15 | description: Configure Cloud Map
16 | parameters:
17 | serviceId:
18 | type: String
19 | instancePort:
20 | type: String
21 | mainSteps:
22 | - action: aws:runShellScript
23 | name: configureCloudMap
24 | inputs:
25 | runCommand:
26 | - |
27 | #! /bin/bash
28 |
29 | EC2_METADATA=http://169.254.169.254/latest
30 | REGION=$(curl -s $EC2_METADATA/dynamic/instance-identity/document | jq -r '.region')
31 | INSTANCE_ID=$(curl -s $EC2_METADATA/meta-data/instance-id);
32 | INSTANCE_IP=$(curl -s $EC2_METADATA/latest/meta-data/local-ipv4);
33 |
34 | cat > /etc/init.d/cloudmap-register <<-EOF
35 | #! /bin/bash -ex
36 | aws servicediscovery register-instance \
37 | --region $REGION \
38 | --service-id {{serviceId}} \
39 | --instance-id $INSTANCE_ID \
40 | --attributes AWS_INSTANCE_IPV4=$INSTANCE_IP,AWS_INSTANCE_PORT={{instancePort}}
41 | exit 0
42 | EOF
43 | chmod a+x /etc/init.d/cloudmap-register
44 |
45 | cat > /etc/init.d/cloudmap-deregister <<-EOF
46 | #! /bin/bash -ex
47 | aws servicediscovery deregister-instance \
48 | --region $REGION \
49 | --service-id {{serviceid}} \
50 | --instance-id $INSTANCE_ID
51 | exit 0
52 | EOF
53 | chmod a+x /etc/init.d/cloudmap-deregister
54 |
55 | cat > /usr/lib/systemd/system/cloudmap.service <<-EOF
56 | [Unit]
57 | Description=Run CloudMap service
58 | Requires=network-online.target network.target
59 | DefaultDependencies=no
60 | Before=shutdown.target reboot.target halt.target
61 |
62 | [Service]
63 | Type=oneshot
64 | KillMode=none
65 | RemainAfterExit=yes
66 | ExecStart=/etc/init.d/cloudmap-register
67 | ExecStop=/etc/init.d/cloudmap-deregister
68 |
69 | [Install]
70 | WantedBy=multi-user.target
71 | EOF
72 |
73 | systemctl enable cloudmap.service
74 | systemctl start cloudmap.service
75 | EOF
76 | aws ssm create-document \
77 | --name appmesh-workshop-ConfigureCloudMap \
78 | --document-format YAML \
79 | --content file://configure-cloudmap.yml \
80 | --document-type Command
81 | ```
--------------------------------------------------------------------------------
/content/authors.md.disabled:
--------------------------------------------------------------------------------
1 | ---
2 | title: Credits
3 | disableToc: true
4 | ---
5 |
6 | ### Thanks to our wonderful contributors for making Open Source a better place!
7 |
8 | {{% ghcontributors "https://api.github.com/repos/brentley/appmeshworkshop/contributors?per_page=1000" %}}
--------------------------------------------------------------------------------
/content/cleanup/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup"
3 | chapter: true
4 | weight: 55
5 | ---
6 |
7 | # Cleanup
8 | 
9 |
--------------------------------------------------------------------------------
/content/cleanup/appmesh.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the mesh"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 40
5 | ---
6 |
7 | * Delete the virtual services.
8 |
9 | ```bash
10 | # Delete app mesh virtual services #
11 | aws appmesh list-virtual-services \
12 | --mesh-name appmesh-workshop | \
13 | jq -r ' .virtualServices[] | [.virtualServiceName] | @tsv ' | \
14 | while IFS=$'\t' read -r virtualServiceName; do
15 | aws appmesh delete-virtual-service \
16 | --mesh-name appmesh-workshop \
17 | --virtual-service-name $virtualServiceName
18 | done
19 | ```
20 |
21 | * Delete the virtual routers.
22 |
23 | ```bash
24 | # Delete app mesh virtual routers #
25 | aws appmesh list-virtual-routers \
26 | --mesh-name appmesh-workshop | \
27 | jq -r ' .virtualRouters[] | [.virtualRouterName] | @tsv ' | \
28 | while IFS=$'\t' read -r virtualRouterName; do
29 | aws appmesh list-routes \
30 | --mesh-name appmesh-workshop \
31 | --virtual-router-name $virtualRouterName | \
32 | jq -r ' .routes[] | [ .routeName] | @tsv ' | \
33 | while IFS=$'\t' read -r routeName; do
34 | aws appmesh delete-route \
35 | --mesh appmesh-workshop \
36 | --virtual-router-name $virtualRouterName \
37 | --route-name $routeName
38 | done
39 | aws appmesh delete-virtual-router \
40 | --mesh-name appmesh-workshop \
41 | --virtual-router-name $virtualRouterName
42 | done
43 | ```
44 |
45 | * Delete the virtual nodes.
46 |
47 | ```bash
48 | # Delete app mesh virtual nodes #
49 | aws appmesh list-virtual-nodes \
50 | --mesh-name appmesh-workshop | \
51 | jq -r ' .virtualNodes[] | [.virtualNodeName] | @tsv ' | \
52 | while IFS=$'\t' read -r virtualNodeName; do
53 | aws appmesh delete-virtual-node \
54 | --mesh-name appmesh-workshop \
55 | --virtual-node-name $virtualNodeName
56 | done
57 | ```
58 |
59 | * Delete the mesh.
60 |
61 | ```bash
62 | # Delete app mesh mesh #
63 | aws appmesh delete-mesh \
64 | --mesh-name appmesh-workshop
65 | ```
66 |
--------------------------------------------------------------------------------
/content/cleanup/cfn.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Delete the baseline template"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 70
5 | ---
6 |
7 | * Delete the infrastructure deployed by the baseline template.
8 |
9 | ```bash
10 | # Define variables #
11 | STACK_NAME=$(jq < cfn-output.json -r '.StackName');
12 | # Delete cloudformation stack #
13 | aws cloudformation delete-stack \
14 | --stack-name $STACK_NAME
15 | aws cloudformation wait stack-delete-complete \
16 | --stack-name $STACK_NAME
17 | ```
--------------------------------------------------------------------------------
/content/cleanup/cloudmap.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the namespace"
3 | date: 2018-08-07T13:37:53-07:00
4 | weight: 20
5 | ---
6 |
7 | Deregister the instances from the Cloud Map service discovery.
8 |
9 | ```bash
10 | NAMESPACE=$(aws servicediscovery list-namespaces | \
11 | jq -r ' .Namespaces[] |
12 | select ( .Properties.HttpProperties.HttpName == "appmeshworkshop.pvt.local" ) | .Id ');
13 | SERVICE_ID=$(aws servicediscovery list-services --filters Name="NAMESPACE_ID",Values=$NAMESPACE,Condition="EQ" | jq -r ' .Services[] | [ .Id ] | @tsv ' )
14 | aws servicediscovery list-instances --service-id $SERVICE_ID | jq -r ' .Instances[] | [ .Id ] | @tsv ' |\
15 | while IFS=$'\t' read -r instanceId; do
16 | aws servicediscovery deregister-instance --service-id $SERVICE_ID --instance-id $instanceId
17 | done
18 | ```
19 |
20 | Delete the services in the namespace.
21 |
22 | ```bash
23 | # Define variables #
24 | NAMESPACE=$(aws servicediscovery list-namespaces | \
25 | jq -r ' .Namespaces[] |
26 | select ( .Properties.HttpProperties.HttpName == "appmeshworkshop.pvt.local" ) | .Id ');
27 | # Delete cloud map services #
28 | aws servicediscovery list-services \
29 | --filters Name="NAMESPACE_ID",Values=$NAMESPACE,Condition="EQ" | \
30 | jq -r ' .Services[] | [ .Id ] | @tsv ' | \
31 | while IFS=$'\t' read -r serviceId; do
32 | aws servicediscovery delete-service \
33 | --id $serviceId
34 | done
35 | ```
36 |
37 | Delete the namespace.
38 |
39 | ```bash
40 | # Define variables #
41 | NAMESPACE=$(aws servicediscovery list-namespaces | \
42 | jq -r ' .Namespaces[] |
43 | select ( .Properties.HttpProperties.HttpName == "appmeshworkshop.pvt.local" ) | .Id ');
44 | # Delete cloud map namespace #
45 | OPERATION_ID=$(aws servicediscovery delete-namespace \
46 | --id $NAMESPACE | \
47 | jq -r ' .OperationId ')
48 | _operation_status() {
49 | aws servicediscovery get-operation \
50 | --operation-id $OPERATION_ID | \
51 | jq -r '.Operation.Status '
52 | }
53 | until [ $(_operation_status) != "PENDING" ]; do
54 | echo "Namespace is deleting ..."
55 | sleep 10s
56 | if [ $(_operation_status) == "SUCCESS" ]; then
57 | echo "Namespace deleted"
58 | break
59 | fi
60 | done
61 | ```
62 |
--------------------------------------------------------------------------------
/content/cleanup/ecr.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the ECR repositories"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 50
5 | ---
6 |
7 | * Delete the container images.
8 |
9 | ```bash
10 | # Define variables #
11 | CRYSTAL_ECR_REPO=$(jq < cfn-output.json -r '.CrystalEcrRepo' | cut -d'/' -f2)
12 | NODEJS_ECR_REPO=$(jq < cfn-output.json -r '.NodeJSEcrRepo' | cut -d'/' -f2)
13 | # Delete ecr images #
14 | aws ecr list-images \
15 | --repository-name $CRYSTAL_ECR_REPO | \
16 | jq -r ' .imageIds[] | [ .imageDigest ] | @tsv ' | \
17 | while IFS=$'\t' read -r imageDigest; do
18 | aws ecr batch-delete-image \
19 | --repository-name $CRYSTAL_ECR_REPO \
20 | --image-ids imageDigest=$imageDigest
21 | done
22 | aws ecr list-images \
23 | --repository-name $NODEJS_ECR_REPO | \
24 | jq -r ' .imageIds[] | [ .imageDigest ] | @tsv ' | \
25 | while IFS=$'\t' read -r imageDigest; do
26 | aws ecr batch-delete-image \
27 | --repository-name $NODEJS_ECR_REPO \
28 | --image-ids imageDigest=$imageDigest
29 | done
30 | ```
--------------------------------------------------------------------------------
/content/cleanup/ecs.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the ECS cluster"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 10
5 | ---
6 |
7 | * Delete the services running in your cluster.
8 |
9 | ```bash
10 | # Define variables #
11 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
12 | # Delete ecs services #
13 | aws ecs list-services \
14 | --cluster $CLUSTER_NAME | \
15 | jq -r ' .serviceArns[] | [.] | @tsv ' | \
16 | while IFS=$'\t' read -r serviceArn; do
17 | aws ecs delete-service \
18 | --cluster $CLUSTER_NAME \
19 | --service $serviceArn \
20 | --force
21 | done
22 | ```
23 |
24 | * Stop all running tasks.
25 |
26 | ```bash
27 | # Define variables #
28 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
29 | # Stop ecs tasks #
30 | aws ecs list-tasks \
31 | --cluster $CLUSTER_NAME | \
32 | jq -r ' .taskArns[] | [.] | @tsv' | \
33 | while IFS=$'\t' read -r taskArn; do
34 | aws ecs stop-task --cluster $CLUSTER_NAME --task $taskArn;
35 | done
36 | ```
37 |
38 | * Delete the cluster.
39 |
40 | ```bash
41 | # Define variables #
42 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
43 | # Delete the ecs cluster #
44 | aws ecs delete-cluster --cluster $CLUSTER_NAME
45 | ```
46 |
47 | * De-register all task definitions.
48 |
49 | ```bash
50 | # Define variables #
51 | TASK_DEF_FAMILY=$(jq < cfn-output.json -r '.CrystalTaskDefinition' | \
52 | cut -d'/' -f2 | cut -d':' -f1)
53 | # Delete ecs task definitions #
54 | aws ecs list-task-definitions \
55 | --family-prefix $TASK_DEF_FAMILY | \
56 | jq -r ' .taskDefinitionArns[] | [.] | @tsv' | \
57 | while IFS=$'\t' read -r taskDefinitionArn; do
58 | aws ecs deregister-task-definition --task-definition $taskDefinitionArn;
59 | done
60 | ```
--------------------------------------------------------------------------------
/content/cleanup/iam_role.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the admin IAM Role"
3 | chapter: false
4 | weight: 100
5 | ---
6 |
7 | After deleting the Cloud9 instance, you will be able to delete de IAM Role with Administrator access previously created.
8 |
9 | - Go to the [IAM console](https://console.aws.amazon.com/iam/home#/roles)
10 | - Select the Role named **AppMesh-Workshop-Admin** and click in **Delete role**.
--------------------------------------------------------------------------------
/content/cleanup/kubernetes_assets.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup Kubernetes components"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 15
5 | ---
6 |
7 | Delete the app namespace along with its AWS resources
8 |
9 | ```bash
10 | # Delete the virtual service
11 | kubectl delete ns appmesh-workshop-ns
12 | ```
13 |
14 | Delete the cluster
15 |
16 | ```bash
17 | eksctl delete cluster appmesh-workshop
18 | ```
19 |
--------------------------------------------------------------------------------
/content/cleanup/log_groups.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Delete the CW log groups"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 80
5 | ---
6 |
7 | * Delete the Log Groups from CloudWatch logs.
8 |
9 | ```bash
10 | # Delete CloudWatch log groups
11 | aws logs delete-log-group --log-group-name appmesh-workshop-crystal-envoy
12 | aws logs delete-log-group --log-group-name appmesh-workshop-frontend-envoy
13 | aws logs delete-log-group --log-group-name /aws/containerinsights/appmesh-workshop/application
14 | aws logs delete-log-group --log-group-name /aws/containerinsights/appmesh-workshop/dataplane
15 | aws logs delete-log-group --log-group-name /aws/containerinsights/appmesh-workshop/host
16 | aws logs delete-log-group --log-group-name /aws/containerinsights/appmesh-workshop/performance
17 | aws logs delete-log-group --log-group-name /aws/ecs/containerinsights/cluster-appmesh-workshop/performance
18 | ```
--------------------------------------------------------------------------------
/content/cleanup/route53.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the Route53 Hosted Zone"
3 | date: 2018-08-07T12:37:34-07:00
4 | weight: 60
5 | ---
6 |
7 | * Delete the Route53 Hosted Zone.
8 |
9 | ```bash
10 | # Define variables #
11 | HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name \
12 | --dns-name appmeshworkshop.hosted.local \
13 | --max-items 1 | \
14 | jq -r ' .HostedZones | first | .Id');
15 | CRYSTAL_RECORD_SET=$(aws route53 list-resource-record-sets --hosted-zone-id=$HOSTED_ZONE_ID | \
16 | jq -r '.ResourceRecordSets[] | select (.Name == "crystal.appmeshworkshop.hosted.local.")');
17 | NODEJS_RECORD_SET=$(aws route53 list-resource-record-sets --hosted-zone-id=$HOSTED_ZONE_ID | \
18 | jq -r '.ResourceRecordSets[] | select (.Name == "nodejs.appmeshworkshop.hosted.local.")');
19 |
20 | # Create temaplate file
21 | cat <<-EOF > /tmp/delete_r53.json
22 | {
23 | "Comment": "DELETE crystal.appmeshworkshop.hosted.local and nodejs.appmeshworkshop.hosted.local",
24 | "Changes": [
25 | {
26 | "Action": "DELETE",
27 | "ResourceRecordSet": $CRYSTAL_RECORD_SET
28 | },
29 | {
30 | "Action": "DELETE",
31 | "ResourceRecordSet": $NODEJS_RECORD_SET
32 | }
33 | ]
34 | }
35 | EOF
36 |
37 | # Delete hosted zone
38 | aws route53 change-resource-record-sets \
39 | --hosted-zone-id $HOSTED_ZONE_ID \
40 | --change-batch file:///tmp/delete_r53.json
41 |
42 | aws route53 delete-hosted-zone \
43 | --id $HOSTED_ZONE_ID
44 | ```
--------------------------------------------------------------------------------
/content/cleanup/ssm.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the SSM resources"
3 | date: 2018-08-07T13:37:53-07:00
4 | weight: 30
5 | ---
6 |
7 | * Delete the SSM association.
8 |
9 | ```bash
10 | # Define variables #
11 | ASSOCIATION=$(aws ssm list-associations \
12 | --association-filter key=AssociationName,value=appmesh-workshop-state | \
13 | jq -r ' .Associations | first | .AssociationId')
14 | # Delete ssm association #
15 | aws ssm delete-association \
16 | --association-id $ASSOCIATION
17 | ```
18 |
19 | * Delete the SSM document.
20 |
21 | ```bash
22 | # Delete ssm document #
23 | aws ssm delete-document \
24 | --name appmesh-workshop-installenvoy
25 | ```
--------------------------------------------------------------------------------
/content/cleanup/workspace.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cleanup the Workspace"
3 | chapter: false
4 | weight: 90
5 | ---
6 |
7 | Since we no longer need the Cloud9 instance to have Administrator access
8 | to our account, we can delete the workspace we created:
9 |
10 | - Go to your [Cloud9 Environment](https://console.aws.amazon.com/cloud9/home)
11 | - Select the environment named **appmesh-workshop** and pick **delete**
12 |
--------------------------------------------------------------------------------
/content/conclusion/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Conclusion"
3 | chapter: true
4 | weight: 60
5 | ---
6 |
7 | # Conclusion
8 |
--------------------------------------------------------------------------------
/content/conclusion/conclusion.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "What Have We Accomplished"
3 | chapter: false
4 | weight: 1
5 | ---
6 |
7 | We have:
8 |
9 | - Configured App Mesh in services that run in different compute platforms, specifically EC2, EKS and Fargate.
10 | - Added monitoring and logging capabilities using CloudWatch.
11 | - Added distributed tracing capabilities using X-Ray.
12 | - Switched from dedicated load balancing to service discovery.
13 | - Implemented traffic routing scenarios to support deployment strategies, and handle failures.
14 |
15 | Comments, ideas, issues around App Mesh? Please submit them on the [App Mesh Public Roadmap](https://github.com/aws/aws-app-mesh-roadmap/issues)
16 |
--------------------------------------------------------------------------------
/content/conclusion/survey.md:
--------------------------------------------------------------------------------
1 | +++
2 | title = "Let us know what you think!"
3 | chapter = false
4 | weight = 60
5 | +++
6 |
7 | - Please take our survey!
8 | {{% surveymonkey %}}
9 |
--------------------------------------------------------------------------------
/content/deployment/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Deployment Strategies"
3 | chapter: true
4 | weight: 40
5 | ---
6 |
7 | # Roll out updates to the example microservices
8 |
9 | 
10 |
11 | **Canary release** is a software development strategy in which a new version of a service (as well as other software) is deployed as a canary release for testing purposes, and the base version remains deployed as a production release for normal operations.
12 |
13 | In this chapter, let's create a new version of the **Crystal backend** application and use the App Mesh to control the Canary deployment process where we will start sending only a small part of the traffic to the new version. If all goes well, then we will continue to shift more traffic to the new version until it is serving 100% of all requests.
14 |
15 |
16 |
17 | {{% children showhidden="false" %}}
18 |
--------------------------------------------------------------------------------
/content/deployment/updatecrystal.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update the Crystal code"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 5
5 | ---
6 |
7 | In a canary release deployment, total traffic is separated at random into a production release and a canary release with a pre-configured ratio. Typically, the canary release receives a small percentage of the traffic and the production release takes up the rest. The updated service features are only visible to the traffic through the canary. You can adjust the canary traffic percentage to optimize test coverage or performance.
8 |
9 | Remember, in App Mesh, every version of a service is ultimately backed by actual running code somewhere (Fargate tasks in the case of Crystal), so each service will have it's own virtual node representation in the mesh that provides this conduit.
10 |
11 | Additionaly, there is the physical deployment of the application itself to a compute environment. Both Crystal deployments will run on ECS using the Fargate launch type. Our goal is to test with a portion of traffic going to the new version, ultimately increasing to 100% of traffic.
12 |
13 | * Create the following patch in your **ecsdemo-crystal** project.
14 |
15 | ```bash
16 | cat <<-'EOF' > ~/environment/ecsdemo-crystal/add_time_ms.patch
17 | From beb964253b921a0a34ed45bac0ab052667523441 Mon Sep 17 00:00:00 2001
18 | From:
19 | Date:
20 | Subject: [PATCH 1/2] add canary hash
21 |
22 | ---
23 | code_hash.txt | 2 +-
24 | 1 file changed, 1 insertion(+), 1 deletion(-)
25 |
26 | diff --git a/code_hash.txt b/code_hash.txt
27 | index a7c041f..dbd1857 100644
28 | --- a/code_hash.txt
29 | +++ b/code_hash.txt
30 | @@ -1 +1 @@
31 | -NOHASH
32 | +CANARY
33 | --
34 | 2.22.0
35 |
36 |
37 | From 23a8640575f0fcc5b5fac4936d51a5dff656c281 Mon Sep 17 00:00:00 2001
38 | From:
39 | Date:
40 | Subject: [PATCH 2/2] add time ms
41 |
42 | ---
43 | src/server.cr | 4 +++-
44 | 1 file changed, 3 insertions(+), 1 deletion(-)
45 |
46 | diff --git a/src/server.cr b/src/server.cr
47 | index 2fad0ea..b08ae32 100644
48 | --- a/src/server.cr
49 | +++ b/src/server.cr
50 | @@ -1,5 +1,6 @@
51 | require "logger"
52 | require "http/server"
53 | +require "time"
54 |
55 | log = Logger.new(STDOUT)
56 | log.level = Logger::DEBUG
57 | @@ -24,8 +25,9 @@ server = HTTP::Server.new(
58 | HTTP::CompressHandler.new,
59 | ]) do |context|
60 | if context.request.path == "/crystal" || context.request.path == "/crystal/"
61 | + epoch_ms = Time.now.epoch_ms
62 | context.response.content_type = "text/plain"
63 | - context.response.print "Crystal backend: Hello! from #{az_message} commit #{code_hash}"
64 | + context.response.print "Crystal backend: Hello! from #{az_message} commit #{code_hash} at #{epoch_ms}"
65 | elsif context.request.path == "/health"
66 | context.response.content_type = "text/plain"
67 | context.response.print "Healthy!"
68 | --
69 | 2.22.0
70 |
71 |
72 | EOF
73 | ```
74 |
75 | * First, take a look at what changes are in the patch.
76 |
77 | ```bash
78 | git apply --stat ecsdemo-crystal/add_time_ms.patch
79 | ```
80 |
81 | * Run **git apply** to apply the patch.
82 |
83 | ```bash
84 | git -C ~/environment/ecsdemo-crystal apply add_time_ms.patch
85 | ```
86 |
87 | * Build the container
88 |
89 | ```bash
90 | CRYSTAL_ECR_REPO=$(jq < cfn-output.json -r '.CrystalEcrRepo')
91 |
92 | $(aws ecr get-login --no-include-email)
93 |
94 | docker build -t crystal-service ecsdemo-crystal
95 | docker tag crystal-service:latest $CRYSTAL_ECR_REPO:epoch
96 | docker push $CRYSTAL_ECR_REPO:epoch
97 | ```
98 |
--------------------------------------------------------------------------------
/content/deployment/virtualnode.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a new virtual node"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 |
7 | * Create a new virtual node for our Crystal backend. This time, the virtual node will reference our epoch version.
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "serviceDiscovery": {
14 | "awsCloudMap": {
15 | "namespaceName": "appmeshworkshop.pvt.local",
16 | "serviceName": "crystal",
17 | "attributes": [
18 | {
19 | "key": "ECS_TASK_SET_EXTERNAL_ID",
20 | "value": "epoch-task-set"
21 | }
22 | ]
23 | }
24 | },
25 | "logging": {
26 | "accessLog": {
27 | "file": {
28 | "path": "/dev/stdout"
29 | }
30 | }
31 | },
32 | "listeners": [
33 | {
34 | "healthCheck": {
35 | "healthyThreshold": 3,
36 | "intervalMillis": 10000,
37 | "path": "/health",
38 | "port": 3000,
39 | "protocol": "http",
40 | "timeoutMillis": 5000,
41 | "unhealthyThreshold": 3
42 | },
43 | "portMapping": { "port": 3000, "protocol": "http" }
44 | }
45 | ]
46 | }
47 | EOF
48 | ); \
49 | # Create app mesh virtual node #
50 | aws appmesh create-virtual-node \
51 | --mesh-name appmesh-workshop \
52 | --virtual-node-name crystal-sd-epoch \
53 | --spec "$SPEC"
54 | ```
--------------------------------------------------------------------------------
/content/deployment/virtualroutes.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create traffic routes"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 40
5 | ---
6 |
7 | We will try sending 33% of the traffic over to our canary. If our monitoring indicates that the service is healthy, we could start granualy increasing the load with automated rollouts (and rollback if issues are indicated), but we're keeping things simple for the workshop. In order to do so, we will create a second route in the existing virtual router. The new route will have a higher priority (lower number) and the same match condition as the existing one. The net effect is the new route will be evaluated and selected over the existing one.
8 |
9 |
10 | * Start shifting traffic to your canary virtual node. Traffic will be distributed between the crystal-sd-vanilla and crystal-sd-epoch virtual nodes at a 2:1 ratio respectively.
11 |
12 | ```bash
13 | # Define variables #
14 | SPEC=$(cat <<-EOF
15 | {
16 | "httpRoute": {
17 | "action": {
18 | "weightedTargets": [
19 | {
20 | "virtualNode": "crystal-sd-vanilla",
21 | "weight": 2
22 | },
23 | {
24 | "virtualNode": "crystal-sd-epoch",
25 | "weight": 1
26 | }
27 | ]
28 | },
29 | "match": {
30 | "prefix": "/"
31 | }
32 | },
33 | "priority": 5
34 | }
35 | EOF
36 | ); \
37 | # Create app mesh route #
38 | aws appmesh create-route \
39 | --mesh-name appmesh-workshop \
40 | --virtual-router-name crystal-router \
41 | --route-name crystal-random-route \
42 | --spec "$SPEC"
43 | ```
44 |
--------------------------------------------------------------------------------
/content/failures/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Handling Failures"
3 | chapter: true
4 | weight: 50
5 | ---
6 |
7 | # Monitoring the Example Microservices
8 |
9 | 
10 |
11 | Planning to handling transient faults is a common practice in a microservices architecture. Retry of a request can help overcome short lived network blips or short term interruptions on the server services due to redeployments like Service Unavailable (HTTP Error code 503), Gateway Timeout (HTTP Error code 504).
12 |
13 | Since our application makes recurring use of network calls to other services, we will add the ability to retry connections between services using AWS App Mesh based on HTTP error codes.
14 |
15 | Let’s see how we can set retry duration and attempts within route configurations. Currently, when a request is made, our frontend Ruby service creates a header to denote the current time and send that request to the Crystal backend service.
16 |
17 | In this chapter, the first thing we will do is to change the Crystal backend application code so the backend application will start sending `503` responses to some of the Frontend requests if the time of the original request (passed through the header) and the current time is less than 1 second.
18 |
19 | Initially without retry in place this error will be thrown consistently. After we introduce retries, we will begin to see the time of the original request and the current time will eventually be greater than 1 second (depending on the retry policy configuration you want to set but the example configuration will work once the route is updated during the walkthrough). At this point the Crystal backend service will return a 200 and send back the response.
20 |
21 | Follow these steps to implement the error responses in the Crystal backend application and the Retry logic in App Mesh:
22 |
23 | {{% children showhidden="false" %}}
24 |
--------------------------------------------------------------------------------
/content/failures/curljson.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Check results"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 30
5 | ---
6 |
7 | * Run the following script to check if you are always getting responses from Crystal. You can compare these results with the random results from the web interface.
8 |
9 | ```bash
10 | # Define variables #
11 | URL=$(jq < cfn-output.json -r '.ExternalLoadBalancerDNS');
12 | # Execute curl #
13 | for ((i=1;i<=15;i++)); do
14 | curl --location --silent --header "canary_fleet: true" $URL/json | jq ' .';
15 | sleep 2s
16 | done
17 | ```
18 |
19 | * Alternatively, you can run the following script to compare your previous results using the command line. Notice, we are just omitting the **canary_fleet** header.
20 |
21 | ```bash
22 | # Define variables #
23 | URL=$(jq < cfn-output.json -r '.ExternalLoadBalancerDNS');
24 | # Execute curl #
25 | for ((i=1;i<=15;i++)); do
26 | curl --location --silent $URL/json | jq ' .';
27 | sleep 2s
28 | done
29 | ```
30 |
--------------------------------------------------------------------------------
/content/failures/updatecrystal.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update the Crystal code"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 5
5 | ---
6 |
7 | The first thing we need to do is to change the Crystal backend application code in order to introduce the 503 error responses. To do so, create the following patch in your **ecsdemo-crystal** project.
8 |
9 | ```bash
10 | cat <<-'EOF' > ~/environment/ecsdemo-crystal/add_server_error.patch
11 | From 48f89f671650e1827fbac5f7bb3c8ff3a06d9c73 Mon Sep 17 00:00:00 2001
12 | From:
13 | Date:
14 | Subject: [PATCH] add service unavailable error if req < 1sec
15 |
16 | ---
17 | src/server.cr | 11 +++++++++--
18 | 1 file changed, 9 insertions(+), 2 deletions(-)
19 |
20 | diff --git a/src/server.cr b/src/server.cr
21 | index b08ae32..b6875b5 100644
22 | --- a/src/server.cr
23 | +++ b/src/server.cr
24 | @@ -26,8 +26,15 @@ server = HTTP::Server.new(
25 | ]) do |context|
26 | if context.request.path == "/crystal" || context.request.path == "/crystal/"
27 | epoch_ms = Time.now.epoch_ms
28 | - context.response.content_type = "text/plain"
29 | - context.response.print "Crystal backend: Hello! from #{az_message} commit #{code_hash} at #{epoch_ms}"
30 | + req_epoch_ms = context.request.headers["epoch_ms"].to_i64
31 | + if (epoch_ms - req_epoch_ms) > 1000
32 | + context.response.content_type = "text/plain"
33 | + context.response.print "Crystal backend: Hello! from #{az_message} commit #{code_hash} at #{epoch_ms}"
34 | + else
35 | + context.response.status_code = 503
36 | + context.response.headers["Content-Type"] = "text/plain"
37 | + context.response.puts "The server cannot handle the request"
38 | + end
39 | elsif context.request.path == "/health"
40 | context.response.content_type = "text/plain"
41 | context.response.print "Healthy!"
42 | --
43 | 2.22.0
44 |
45 |
46 | EOF
47 | ```
48 |
49 | First, take a look at what changes are in the patch. Note that now the application will start sending the 503 error messages.
50 |
51 | ```bash
52 | git apply --stat ecsdemo-crystal/add_server_error.patch
53 | ```
54 |
55 | Run **git apply** to apply the patch.
56 |
57 | ```bash
58 | git -C ~/environment/ecsdemo-crystal apply add_server_error.patch
59 | ```
60 |
61 | And build the container with the newest version of the Crystal application:
62 |
63 | ```bash
64 | CRYSTAL_ECR_REPO=$(jq < cfn-output.json -r '.CrystalEcrRepo')
65 |
66 | $(aws ecr get-login --no-include-email)
67 |
68 | docker build -t crystal-service ecsdemo-crystal
69 | docker tag crystal-service:latest $CRYSTAL_ECR_REPO:error
70 | docker push $CRYSTAL_ECR_REPO:error
71 | ```
72 |
--------------------------------------------------------------------------------
/content/failures/virtualnode.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a new virtual node"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 |
7 | * Create a new virtual node for our Crystal backend. This time, the virtual node will reference our error version.
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "serviceDiscovery": {
14 | "awsCloudMap": {
15 | "namespaceName": "appmeshworkshop.pvt.local",
16 | "serviceName": "crystal",
17 | "attributes": [
18 | {
19 | "key": "ECS_TASK_SET_EXTERNAL_ID",
20 | "value": "error-task-set"
21 | }
22 | ]
23 | }
24 | },
25 | "logging": {
26 | "accessLog": {
27 | "file": {
28 | "path": "/dev/stdout"
29 | }
30 | }
31 | },
32 | "listeners": [
33 | {
34 | "healthCheck": {
35 | "healthyThreshold": 3,
36 | "intervalMillis": 10000,
37 | "path": "/health",
38 | "port": 3000,
39 | "protocol": "http",
40 | "timeoutMillis": 5000,
41 | "unhealthyThreshold": 3
42 | },
43 | "portMapping": { "port": 3000, "protocol": "http" }
44 | }
45 | ]
46 | }
47 | EOF
48 | ); \
49 | # Create app mesh virtual node #
50 | aws appmesh create-virtual-node \
51 | --mesh-name appmesh-workshop \
52 | --virtual-node-name crystal-sd-error \
53 | --spec "$SPEC"
54 | ```
--------------------------------------------------------------------------------
/content/failures/virtualroutes.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create traffic routes"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 20
5 | ---
6 |
7 | Since we know that the Crystal application will start replying requests with `503` error messages, we need to to add a retry policy in the route, so that if a request to crystal-sd-error resulted in any server error status codes as “500 or 503”, a retry attempt would be made with a max limit of 10 retries:
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "httpRoute": {
14 | "action": {
15 | "weightedTargets": [
16 | {
17 | "virtualNode": "crystal-sd-error",
18 | "weight": 1
19 | }
20 | ]
21 | },
22 | "match": {
23 | "prefix": "/",
24 | "headers": [
25 | {
26 | "name": "canary_fleet",
27 | "match": {
28 | "exact": "true"
29 | }
30 | }
31 | ]
32 | },
33 | "retryPolicy": {
34 | "maxRetries": 10,
35 | "perRetryTimeout": {
36 | "unit": "s",
37 | "value": 2
38 | },
39 | "httpRetryEvents": [
40 | "server-error"
41 | ]
42 | }
43 | },
44 | "priority": 1
45 | }
46 | EOF
47 | ); \
48 | # Create app mesh route #
49 | aws appmesh update-route \
50 | --mesh-name appmesh-workshop \
51 | --virtual-router-name crystal-router \
52 | --route-name crystal-header-route \
53 | --spec "$SPEC"
54 | ```
55 |
56 | * Update your `crystal-random-route` to route to `crystal-sd-error` instead of `crystal-sd-epoch`. This will let you check the default traffic behavior (without the presence of a header).
57 |
58 | ```bash
59 | # Define variables #
60 | SPEC=$(cat <<-EOF
61 | {
62 | "httpRoute": {
63 | "action": {
64 | "weightedTargets": [
65 | {
66 | "virtualNode": "crystal-sd-vanilla",
67 | "weight": 2
68 | },
69 | {
70 | "virtualNode": "crystal-sd-error",
71 | "weight": 1
72 | }
73 | ]
74 | },
75 | "match": {
76 | "prefix": "/"
77 | }
78 | },
79 | "priority": 5
80 | }
81 | EOF
82 | ); \
83 | # Create app mesh route #
84 | aws appmesh update-route \
85 | --mesh-name appmesh-workshop \
86 | --virtual-router-name crystal-router \
87 | --route-name crystal-random-route \
88 | --spec "$SPEC"
89 | ```
90 |
91 | * We no longer need the `epoch-task-set` task set. Let's scale it to 0%.
92 |
93 | ```bash
94 | # Define variables #
95 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
96 | SERVICE_ARN=$(aws ecs list-services --cluster $CLUSTER_NAME | \
97 | jq -r ' .serviceArns[] | select( . | contains("sd"))' | tail -1)
98 | TASK_SET_ARN=$(aws ecs describe-services --cluster $CLUSTER_NAME --service $SERVICE_ARN | \
99 | jq -r ' .services | first | .taskSets[] | select( .externalId == "epoch-task-set") | .taskSetArn')
100 | # Update ecs task set #
101 | aws ecs update-task-set \
102 | --service $SERVICE_ARN \
103 | --cluster $CLUSTER_NAME \
104 | --task-set $TASK_SET_ARN \
105 | --scale value=0,unit=PERCENT
106 | ```
--------------------------------------------------------------------------------
/content/introduction/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Introduction"
3 | weight: 5
4 | chapter: true
5 | draft: false
6 | ---
7 |
8 | # Introduction to AWS App Mesh
9 |
10 | A walkthrough of basic App Mesh concepts.
11 |
12 | 
13 |
14 | Welcome to the AWS App Mesh Workshop!
15 |
16 | The intent of this workshop is to educate users about the features and usage of AWS App Mesh.
17 |
18 | Background in ECS, EKS, Docker, and container workflows are not required, but they are recommended.
19 |
20 | This chapter will introduce you to the basic workings of App Mesh, laying the foundation for the hands-on portion of the workshop.
21 |
22 | Specifically, we will walk you through the following topics:
23 |
24 | {{% children showhidden="false" %}}
25 |
--------------------------------------------------------------------------------
/content/introduction/appmesh_benefits.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "App Mesh benefits"
3 | date: 2018-10-03T10:15:55-07:00
4 | draft: false
5 | weight: 50
6 | ---
7 |
8 | #### End-to-end visibility
9 | App Mesh captures metrics, logs, and traces from all of your applications. You can combine and export this data to Amazon CloudWatch, AWS X-Ray, and compatible AWS partner and community tools for monitoring and tracing. This lets you quickly identify and isolate issues with any service to optimize your entire application.
10 |
11 | #### Ensure high availability
12 | App Mesh gives you controls to configure how traffic flows between your services. You can easily implement custom traffic routing rules to ensure every service is highly available during deployments, after failures, and as your application scales.
13 |
14 | #### Streamline operations
15 | App Mesh deploys and configures a proxy that manages all communications traffic to and from your services. This removes the need to configure communication protocols for each service, write custom code, or implement libraries to operate your application.
16 |
17 | #### Enhance any application
18 | You can use App Mesh with services running on any compute service such as AWS Fargate, Amazon ECS, Amazon EKS, and Amazon EC2. App Mesh can monitor and control communications for monoliths running on EC2, teams running containerized applications, orchestration systems, or VPCs as a single application without any code changes.
19 |
20 | #### End-to-end Encryption
21 | App Mesh gives you the ability to encrypt traffic between services using AWS Certificate Manager (ACM) or customer-provided certificates. This helps you to achieve the security and compliance requirements while building enterprise grade mesh architectures.
22 |
--------------------------------------------------------------------------------
/content/introduction/appmesh_components.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "App Mesh components"
3 | date: 2018-10-03T10:15:55-07:00
4 | draft: false
5 | weight: 40
6 | ---
7 |
8 | 
9 |
10 | App Mesh is made up of the following components:
11 |
12 | **Service mesh** – A service mesh is a logical boundary for network traffic between the services that reside within it.
13 |
14 | **Virtual services** – A virtual service is an abstraction of a real service that is provided by a virtual node directly or indirectly by means of a virtual router.
15 |
16 | **Virtual nodes** – A virtual node acts as a logical pointer to a particular task group, such as an ECS service or a Kubernetes deployment. When you create a virtual node, you must specify the service discovery name for your task group.
17 |
18 | **Envoy proxy** – The Envoy proxy configures your microservice task group to use the App Mesh service mesh traffic rules that you set up for your virtual routers and virtual nodes. You add the Envoy container to your task group after you have created your virtual nodes, virtual routers, routes, and virtual services.
19 |
20 | **Virtual routers** – The virtual router handles traffic for one or more virtual services within your mesh.
21 |
22 | **Routes** – A route is associated with a virtual router, and it directs traffic that matches a service name prefix to one or more virtual nodes.
23 |
--------------------------------------------------------------------------------
/content/introduction/how_it_works.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "How it works"
3 | date: 2018-10-03T10:15:55-07:00
4 | draft: false
5 | weight: 60
6 | ---
7 | #### Before App Mesh
8 |
9 | Communications and monitoring are manually configured for every service.
10 |
11 | 
12 |
13 |
14 | #### After App Mesh
15 |
16 | App Mesh configures communications and monitoring for all services.
17 |
18 | 
19 |
--------------------------------------------------------------------------------
/content/introduction/what_is_appmesh.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "What is App Mesh"
3 | date: 2018-10-03T10:14:46-07:00
4 | draft: false
5 | weight: 30
6 | ---
7 |
8 | 
9 |
10 | AWS App Mesh is a service mesh that provides application-level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. App Mesh standardizes how your services communicate, giving you end-to-end visibility and ensuring high-availability for your applications.
11 |
12 | Modern applications are typically composed of multiple services. Each service may be built using multiple types of compute infrastructure such as Amazon EC2, Amazon EKS and AWS Fargate. As the number of services grow within an application, it becomes difficult to pinpoint the exact location of errors, re-route traffic after failures, and safely deploy code changes. Previously, this has required you to build monitoring and control logic directly into your code and redeploy your service every time there are changes.
13 |
14 | AWS App Mesh makes it easy to run services by providing consistent visibility and network traffic controls for services built across multiple types of compute infrastructure. App Mesh removes the need to update application code to change how monitoring data is collected or traffic is routed between services. App Mesh configures each service to export monitoring data and implements consistent communications control logic across your application. This makes it easy to quickly pinpoint the exact location of errors and automatically re-route network traffic when there are failures or when code changes need to be deployed.
15 |
16 | You can use App Mesh with AWS Fargate, Amazon EC2, Amazon ECS, Amazon EKS, and Kubernetes running on AWS, to better run your application at scale. App Mesh uses the open source Envoy proxy, making it compatible with a wide range of AWS partner and open source tools.
17 |
18 |
19 | More information on what App Mesh is all about can be found on the official [App Mesh documentation page](https://docs.aws.amazon.com/app-mesh/latest/userguide/what-is-app-mesh.html).
20 |
--------------------------------------------------------------------------------
/content/keybase.txt:
--------------------------------------------------------------------------------
1 | ==================================================================
2 | https://keybase.io/brentley
3 | --------------------------------------------------------------------
4 |
5 | I hereby claim:
6 |
7 | * I am an admin of https://appmeshworkshop.com
8 | * I am brentley (https://keybase.io/brentley) on keybase.
9 | * I have a public key ASCJfE-C_wdx8-OrWdzPVkji7xZL8GRMOzec2cjEsV6YmAo
10 |
11 | To do so, I am signing this object:
12 |
13 | {
14 | "body": {
15 | "key": {
16 | "eldest_kid": "01200f24307cb6e4c5b0ab8a20b5759f3767c3bc0f197a1f180e9313389a91464fa80a",
17 | "host": "keybase.io",
18 | "kid": "0120897c4f82ff0771f3e3ab59dccf5648e2ef164bf0644c3b379cd9c8c4b15e98980a",
19 | "uid": "a22c231f34f5a746964dbf7a7219fc19",
20 | "username": "brentley"
21 | },
22 | "merkle_root": {
23 | "ctime": 1591035646,
24 | "hash": "43cd73bf4e1e9d39a96264373a46b25cbdcfd92bd73d7577e12a499ae6d8093a805e84940654b67a422c4707369147e070c63586d1c2468788cab462344330b4",
25 | "hash_meta": "fe7a999cb9935532f29ea612ea7f5f85844ff950430c46195ca4902f8669c931",
26 | "seqno": 16481022
27 | },
28 | "service": {
29 | "entropy": "A/Fnc0s/IxyMBBFd8nDldMFe",
30 | "hostname": "appmeshworkshop.com",
31 | "protocol": "https:"
32 | },
33 | "type": "web_service_binding",
34 | "version": 2
35 | },
36 | "client": {
37 | "name": "keybase.io go client",
38 | "version": "5.5.0"
39 | },
40 | "ctime": 1591035754,
41 | "expire_in": 504576000,
42 | "prev": "12ba416b6d9f85fdb7ccf3ce207c30fa056f6c75cc96a9d1af6a95d8f0c5d543",
43 | "seqno": 56,
44 | "tag": "signature"
45 | }
46 |
47 | which yields the signature:
48 |
49 | hKRib2R5hqhkZXRhY2hlZMOpaGFzaF90eXBlCqNrZXnEIwEgiXxPgv8HcfPjq1ncz1ZI4u8WS/BkTDs3nNnIxLFemJgKp3BheWxvYWTESpcCOMQgErpBa22fhf23zPPOIHww+gVvbHXMlqnRr2qV2PDF1UPEIHCy4bUqyhz8Y8adKwVdD9eFtbHU/Hpc7A1aZDDMPlthAgHCo3NpZ8RA7WbLO6yj/ABaUNeoCgPeNGpwXG+2h+PVCLOz1wjMHlP09hLNdDhXg/Wni66PG9zRUKGKTx0kTn5T+aBd4dHLD6hzaWdfdHlwZSCkaGFzaIKkdHlwZQildmFsdWXEIEFLq2Q+T7sDDQyn2Dmbm8RaJ2w6qwF3t6lvwTWTW/tbo3RhZ80CAqd2ZXJzaW9uAQ==
50 |
51 | And finally, I am proving ownership of this host by posting or
52 | appending to this document.
53 |
54 | View my publicly-auditable identity here: https://keybase.io/brentley
55 |
56 | ==================================================================
57 |
--------------------------------------------------------------------------------
/content/mesh_crystal/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Mesh the Crystal Service"
3 | chapter: true
4 | weight: 15
5 | ---
6 |
7 | # Mesh the Backend Crystal Service
8 |
9 | 
10 |
11 | At this point, you should have the application up and running in your lab environment with the EC2 instances serving the frontend with ECS and EKS services managing the backend tasks.
12 |
13 | In this chapter, our goal is to create the Mesh and edit your ECS backend deployment in order to have the Envoy containers running and intercepting the network traffic from your ECS tasks.
14 |
15 | To accomplish this, let's run the following steps:
16 |
17 | {{% children showhidden="false" %}}
18 |
--------------------------------------------------------------------------------
/content/mesh_crystal/createmesh.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create the service mesh"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 5
5 | ---
6 |
7 | **AWS App Mesh** is a service mesh based on the **Envoy proxy**. It standardizes how microservices communicate, giving you end-to-end visibility and helping to ensure high-availability for your applications.
8 |
9 | Throughout this workshop we will use AWS App Mesh, to gain consistent visibility and network traffic controls on our microservices running in **Amazon EC2, AWS Fargate and Amazon EKS**.
10 |
11 | A service mesh is composed of two parts, a control plane and a data plane. AWS App Mesh is a managed control plane so you as a user don't need to install or manage any servers to run it. The data plane for App Mesh is the open source Envoy proxy. It is your responsability to add the Envoy proxy as a side car to each microservice you want to expose and manage via App Mesh.
12 |
13 | In the next few sections, you will perform 2 broad set of tasks:
14 |
15 | * You will use the AWS CLI to create and configure the App Mesh resources needed to represent the microservices available in your environment like virtual services and virtual nodes.
16 |
17 | * You will add a docker container running the Envoy proxy to each compute environment where you have microservices running. AWS App Mesh supports EC2, ECS and EKS and will show you how to add Envoy in each of these environments, without modifying your microservice source code. The process for enabling the Envoy proxy is different for each compute environment. In the following sections you will perform the changes needed to enable each compute environment.
18 |
19 | A key feature of Envoy worth mentioning is that it exposes a set of APIs. These APIs are leveraged by AWS App Mesh to dynamically configure Envoy's routing logic, freeing developers from the tedious task of manually updating config files.
20 |
21 | Now every time you launch an Envoy enabled microservice, the Envoy proxy will contact the AWS App Mesh Management API to subscribe to resource information for Listeners, Clusters, Routes, and Endpoints. The connection from the Envoy proxy to the AWS App Mesh management API endpoint is held open, which allows AWS App Mesh to stream updates to the Envoy proxy as users introduce changes to either of the resources listed above.
22 |
23 | Let's start by creating the service mesh. A service mesh is a logical boundary for network traffic between the services that reside within it.
24 |
25 | ```bash
26 | # Create mesh #
27 | aws appmesh create-mesh \
28 | --mesh-name appmesh-workshop \
29 | --spec egressFilter={type=DROP_ALL}
30 | ```
31 |
--------------------------------------------------------------------------------
/content/mesh_crystal/virtualservice.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create the virtual service"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 |
7 | In the previous section, we created the service mesh. Lets now add a representation of the Crystal ECS Service. We will do so by creating the following two artifacts in App Mesh, a virtual node and a virtual service.
8 |
9 | A **virtual service** is an abstraction of a real service that is provided by a **virtual node** in the mesh. Virtual nodes act as a logical pointer to a particular task group, such as an EC2 Auto Scaling Group, an Amazon ECS service or a Kubernetes deployment. Remember our Crystal service is running as a Amazon ECS (Fargate) service.
10 |
11 | Virtual nodes have friendly names. We will name this virtual node **crystal-lb-vanilla**. Inside a Virtual nodes you will also define the service discovery mechanism for your service. It support both DNS and Cloud Map based service discovery. We will start by using DNS and most specifically we will leverage the DNS name given to the internal ALB that is already fronting our ECS Service.
12 |
13 | The Crystal service is running as a service on ECS Fargate and is reachable via an internal Application Load Balancer. Lets use the DNS name given to it by AWS as part of the CloudFormation template run in section Start the Workshop.
14 |
15 | * Start by creating the virtual node.
16 |
17 | ```bash
18 | # Define variables #
19 | INT_LOAD_BALANCER=$(jq < cfn-output.json -r '.InternalLoadBalancerDNS');
20 | SPEC=$(cat <<-EOF
21 | {
22 | "serviceDiscovery": {
23 | "dns": {
24 | "hostname": "$INT_LOAD_BALANCER"
25 | }
26 | },
27 | "logging": {
28 | "accessLog": {
29 | "file": {
30 | "path": "/dev/stdout"
31 | }
32 | }
33 | },
34 | "listeners": [
35 | {
36 | "healthCheck": {
37 | "healthyThreshold": 3,
38 | "intervalMillis": 10000,
39 | "path": "/health",
40 | "port": 3000,
41 | "protocol": "http",
42 | "timeoutMillis": 5000,
43 | "unhealthyThreshold": 3
44 | },
45 | "portMapping": { "port": 3000, "protocol": "http" }
46 | }
47 | ]
48 | }
49 | EOF
50 | ); \
51 | # Create app mesh virual node #
52 | aws appmesh create-virtual-node \
53 | --mesh-name appmesh-workshop \
54 | --virtual-node-name crystal-lb-vanilla \
55 | --spec "$SPEC"
56 | ```
57 |
58 | Now that we have our virtual node in place, we are ready to create the virtual service.
59 |
60 | We will supply two important pieces of information to the definition of the virtual service. First, we will select the virtual node named **crystal-lb-vanilla** created above as the provider of the virtual service. Second, we will give it a service name. The name of a service is a FQDN and is the name used by clients interested in contacting the service. In our example, the Ruby Frontend will issue HTTP requests to **crystal.appmeshworkshop.hosted.local** in order to interact with the Crystal service.
61 |
62 | In Route53, we have already created a private hosted zone named **appmeshworkshop.hosted.local** and inside, an A ALIAS record named **crystal** with value the FQDN of the internal ALB.
63 |
64 | * Create the virtual service.
65 |
66 | ```bash
67 | # Define variables #
68 | SPEC=$(cat <<-EOF
69 | {
70 | "provider": {
71 | "virtualNode": {
72 | "virtualNodeName": "crystal-lb-vanilla"
73 | }
74 | }
75 | }
76 | EOF
77 | ); \
78 | # Create app mesh virtual service #
79 | aws appmesh create-virtual-service \
80 | --mesh-name appmesh-workshop \
81 | --virtual-service-name crystal.appmeshworkshop.hosted.local \
82 | --spec "$SPEC"
83 | ```
84 |
--------------------------------------------------------------------------------
/content/mesh_frontend/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Mesh the Frontend Service"
3 | chapter: true
4 | weight: 21
5 | ---
6 |
7 | # Mesh the Frontend Ruby Service
8 |
9 | 
10 |
11 | Now that we already have the App Mesh taking care of our backend network traffic, it's time to put our frontend application inside the Mesh.
12 |
13 | In this chapter we will install and configure the Envoy proxy into our EC2 instances that are running the Frontend application by using the [AWS Systems Manager (SSM)](https://aws.amazon.com/systems-manager/).
14 |
15 | To do so, follow these steps:
16 |
17 | {{% children showhidden="false" %}}
18 |
--------------------------------------------------------------------------------
/content/mesh_frontend/virtualservice.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create the virtual service"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 |
7 | * Like you did previously, start by creating a virtual node.
8 |
9 | ```bash
10 | # Define variables #
11 | EXT_LOAD_BALANCER=$(jq < cfn-output.json -r '.ExternalLoadBalancerDNS');
12 | SPEC=$(cat <<-EOF
13 | {
14 | "serviceDiscovery": {
15 | "dns": {
16 | "hostname": "$EXT_LOAD_BALANCER"
17 | }
18 | },
19 | "backends": [
20 | {
21 | "virtualService": {
22 | "virtualServiceName": "crystal.appmeshworkshop.hosted.local"
23 | }
24 | },
25 | {
26 | "virtualService": {
27 | "virtualServiceName": "nodejs.appmeshworkshop.hosted.local"
28 | }
29 | }
30 | ],
31 | "logging": {
32 | "accessLog": {
33 | "file": {
34 | "path": "/dev/stdout"
35 | }
36 | }
37 | },
38 | "listeners": [
39 | {
40 | "healthCheck": {
41 | "healthyThreshold": 3,
42 | "intervalMillis": 10000,
43 | "path": "/health",
44 | "port": 3000,
45 | "protocol": "http",
46 | "timeoutMillis": 5000,
47 | "unhealthyThreshold": 3
48 | },
49 | "portMapping": { "port": 3000, "protocol": "http" }
50 | }
51 | ]
52 | }
53 | EOF
54 | ); \
55 | # Create app mesh virtual node #
56 | aws appmesh create-virtual-node \
57 | --mesh-name appmesh-workshop \
58 | --virtual-node-name frontend \
59 | --spec "$SPEC"
60 | ```
61 |
62 | * Create the virtual service.
63 |
64 | ```bash
65 | # Define variables #
66 | SPEC=$(cat <<-EOF
67 | {
68 | "provider": {
69 | "virtualNode": {
70 | "virtualNodeName": "frontend"
71 | }
72 | }
73 | }
74 | EOF
75 | ); \
76 | # Create app mesh virtual service #
77 | aws appmesh create-virtual-service \
78 | --mesh-name appmesh-workshop \
79 | --virtual-service-name frontend.appmeshworkshop.hosted.local \
80 | --spec "$SPEC"
81 | ```
--------------------------------------------------------------------------------
/content/mesh_nodejs/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Mesh the NodeJS Service"
3 | chapter: true
4 | weight: 20
5 | ---
6 |
7 | # Mesh the NodeJS Service running on EKS
8 |
9 | 
10 |
11 | Now that App Mesh is taking care of our Crystal backend network traffic, the next step is to put our NodeJS backend application inside the Mesh.
12 |
13 | In this chapter we will install AWS App Mesh Controller For Kubernetes and inject the Envoy proxy into our EKS pods.
14 |
15 | To do so, follow these steps:
16 |
17 | {{% children showhidden="false" %}}
18 |
--------------------------------------------------------------------------------
/content/mesh_nodejs/install_appmesh_controller.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Install AWS App Mesh Controller For K8s"
3 | date: 2018-08-07T08:30:11-07:00
4 | weight: 10
5 | ---
6 |
7 | [AWS App Mesh Controller For K8s](https://github.com/aws/aws-app-mesh-controller-for-k8s) manages App Mesh resources within your Kubernetes clusters. The controller is accompanied by Custom Resource Definitions (CRDs) that allow you to define App Mesh components such as Meshes and VirtualNodes using the Kubernetes API just as you define native Kubernetes objects such as Deployments and Services. These custom resources map to App Mesh API objects which the controller manages for you. The controller watches these custom resources for changes and reflects those changes into the App Mesh API.
8 |
9 | The controller is installed via Helm Chart. Follow [this link](https://helm.sh/docs/intro/install/) to install the latest version of Helm. Once done, you should see a version of 3.0 or higher.
10 |
11 | ```bash
12 | helm version --short
13 | ```
14 |
15 | Now, add the EKS Charts repo to Helm.
16 |
17 | ```bash
18 | helm repo add eks https://aws.github.io/eks-charts
19 |
20 | helm repo list | grep eks-charts
21 | ```
22 |
23 | Create the `appmesh-system` namespace and attach IAM Policies for AWS App Mesh and AWS Cloud Map access.
24 |
25 | {{% notice tip %}}
26 | If you are new to the `IAM Roles for Service Accounts (IRSA)` concept, [Click here](/beginner/110_irsa/) for me information.
27 | {{% /notice %}}
28 |
29 | ```bash
30 | kubectl create ns appmesh-system
31 |
32 | # Create your OIDC identity provider for the cluster
33 | eksctl utils associate-iam-oidc-provider \
34 | --cluster appmesh-workshop \
35 | --approve
36 |
37 | # Create an IAM role for the appmesh-controller service account
38 | eksctl create iamserviceaccount \
39 | --cluster appmesh-workshop \
40 | --namespace appmesh-system \
41 | --name appmesh-controller \
42 | --attach-policy-arn arn:aws:iam::aws:policy/AWSCloudMapFullAccess,arn:aws:iam::aws:policy/AWSAppMeshFullAccess \
43 | --override-existing-serviceaccounts \
44 | --approve
45 | ```
46 |
47 | Now install App Mesh Controller into the `appmesh-system` namespace using the project's Helm chart, specifying the service account you just created.
48 |
49 | ```bash
50 | helm upgrade -i appmesh-controller eks/appmesh-controller \
51 | --namespace appmesh-system \
52 | --set region=${AWS_REGION} \
53 | --set serviceAccount.create=false \
54 | --set serviceAccount.name=appmesh-controller
55 | ```
56 |
57 | To verify the installation was successful, list the objects in the `appmesh-system` namespace and ensure the `appmesh-controller` pod instance is in a `Running` state before moving on.
58 |
59 | ```bash
60 | kubectl -n appmesh-system get all
61 | ```
62 |
63 | You can also see that the App Mesh Custom Resource Definitions were installed.
64 |
65 | ```bash
66 | kubectl get crds | grep appmesh
67 | ```
68 |
69 |
--------------------------------------------------------------------------------
/content/mesh_nodejs/restart_pods.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Inject the Envoy proxy"
3 | date: 2018-08-07T08:30:11-07:00
4 | weight: 15
5 | ---
6 | Now that our objects are in place, we have one remaining step to add our `nodejs` application service to the mesh. We've labeled the namespace so that all new pods will have their Envoy sidecar containers automatically injected, but our pods are already up and running. To inject the sidecar proxies, we'll need to restart those pods.
7 |
8 | To do so, run the following command:
9 |
10 | ```bash
11 | kubectl -n appmesh-workshop-ns rollout restart deployment nodejs-app
12 | ```
13 |
14 | This command will restart all the pods present in the `appmesh-workshop-ns` namespace. You can monitor the restart by watching the pods roll out.
15 |
16 | ```bash
17 | kubectl -n appmesh-workshop-ns get pods -w
18 | ```
19 |
20 | Once you see all pods in a `Running` state, verify that the envoy proxy containers were added by checking the number of container instances in each pod. You should now see `2/2` for each pod.
21 |
22 | To dig a little deeper, get the running containers from one of the new pods. Note that your pod names will differ, and you can choose any of your running pods.
23 |
24 | ```bash
25 | POD=$(kubectl -n appmesh-workshop-ns get pods -o jsonpath='{.items[0].metadata.name}')
26 | kubectl -n appmesh-workshop-ns get pods ${POD} -o jsonpath='{.spec.containers[*].name}'; echo
27 | ```
28 |
29 | You will see `envoy` listed as one of the two containers, which means that sidecar injection in the controller is working and that Envoy is running inside your NodeJS pods.
30 |
31 | Now, it's time to test and see if the responses are being sent by the Envoy proxy. Let's access one of the EC2 instances serving the frontend again:
32 |
33 | ```bash
34 | AUTO_SCALING_GROUP=$(jq < cfn-output.json -r '.RubyAutoScalingGroupName');
35 | TARGET_EC2=$(aws ec2 describe-instances \
36 | --filters Name=tag:aws:autoscaling:groupName,Values=$AUTO_SCALING_GROUP | \
37 | jq -r ' .Reservations | first | .Instances | first | .InstanceId')
38 | aws ssm start-session --target $TARGET_EC2
39 | ```
40 |
41 | And curl the NodeJS microservice:
42 |
43 | ```bash
44 | curl -v http://nodejs.appmeshworkshop.hosted.local:3000/
45 | ```
46 |
47 | You should see a response coming from the NodeJS microservice running in the EKS cluster. In this response, look for `envoy` in the `server` parameter:
48 |
49 | ```text
50 | * Trying 10.0.102.194...
51 | * TCP_NODELAY set
52 | * Connected to nodejs.appmeshworkshop.hosted.local (10.0.102.194) port 3000 (#0)
53 | > GET / HTTP/1.1
54 | > Host: nodejs.appmeshworkshop.hosted.local:3000
55 | > User-Agent: curl/7.61.1
56 | > Accept: */*
57 | >
58 | < HTTP/1.1 200 OK
59 | < x-powered-by: Express
60 | < content-type: text/plain; charset=utf-8
61 | < content-length: 64
62 | < etag: W/"40-fCLGgbgasYs+i/eJyifWO2rSi40"
63 | < date: Wed, 01 Jul 2020 00:31:28 GMT
64 | < x-envoy-upstream-service-time: 4
65 | < server: envoy
66 | <
67 | Node.js backend: Hello! from 10.0.102.41 in AZ-c commit 219f52d
68 | * Connection #0 to host nodejs.appmeshworkshop.hosted.local left intact
69 | ```
70 |
71 | This means that the responses from the NodeJS application are passing in the Envoy proxy.
72 |
73 | * Terminate the session.
74 |
75 | ```bash
76 | exit
77 | ```
78 |
--------------------------------------------------------------------------------
/content/monitoring/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Monitoring & Logging"
3 | chapter: true
4 | weight: 25
5 | ---
6 |
7 | # Add Monitoring and Logging Capabilities
8 |
9 | 
10 |
11 | One of the main benefits of implementing App Mesh is the ability to have *visibility* around your microservices and it's communications.
12 |
13 | AWS App Mesh allows you to integrate the logs generated by the Envoy proxies running in your infrastructure with [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/).
14 |
15 | In this chapter, let's change the infrastructure so we can:
16 |
17 | * Use [CloudWatch Container Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights.html) to get more information about your deployments in ECS.
18 | * Ship logs from the Envoy proxies running in the ECS backend service to [CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html).
19 |
20 |
21 | {{% children showhidden="false" %}}
22 |
--------------------------------------------------------------------------------
/content/monitoring/containerinsights_nodejs.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Setup Container Insights for NodeJS App"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 21
5 | ---
6 |
7 | Let's now enable container insights for the NodeJS App running in the EKS cluster. Let's deploy the Container Insights in the cluster using the following command:
8 |
9 | ```bash
10 | # Set environment variables
11 | CLUSTER_NAME="$(echo $C9_PROJECT | sed 's/^Project-//' | tr 'A-Z' 'a-z')"
12 | AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)
13 |
14 | # Deploy Container Insights
15 | curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/master/k8s-yaml-templates/quickstart/cwagent-fluentd-quickstart.yaml | \
16 | sed "s/{{cluster_name}}/$CLUSTER_NAME/;s/{{region_name}}/$AWS_REGION/" | kubectl apply -f -
17 | ```
18 |
19 | Now that we have the Container Insights properly deployed in the cluster, let's change the NodeJS virtual node, so it will start sending logs to `stdout`:
20 |
21 | ```bash
22 | # Set environment variable
23 | NODEJS_LB_URL=$(kubectl get service nodejs-app-service -n appmesh-workshop-ns -o json | jq -r '.status.loadBalancer.ingress[].hostname')
24 |
25 | # Update virtual node file
26 | cat < ~/environment/eks-scripts/virtual-node.yml
27 | apiVersion: appmesh.k8s.aws/v1beta2
28 | kind: VirtualNode
29 | metadata:
30 | name: nodejs-app
31 | namespace: appmesh-workshop-ns
32 | spec:
33 | podSelector:
34 | matchLabels:
35 | app: nodejs-app
36 | listeners:
37 | - portMapping:
38 | port: 3000
39 | protocol: http
40 | serviceDiscovery:
41 | dns:
42 | hostname: $NODEJS_LB_URL
43 | logging:
44 | accessLog:
45 | file:
46 | path: /dev/stdout
47 | EOF
48 |
49 | # Apply the configuration
50 | kubectl apply -f ~/environment/eks-scripts/virtual-node.yml
51 | ```
52 |
53 | And finally, restart the pods, so it will start sending data to CloudWatch:
54 |
55 | ```bash
56 | # Restart pods
57 | kubectl -n appmesh-workshop-ns rollout restart deployment nodejs-app
58 | ```
59 |
60 | At this moment you should be able to see data in the CloudWatch logs interface by accessing [this url](http://console.aws.amazon.com/cloudwatch/home#logs:prefix=/aws/containerinsights/appmesh-workshop/).
61 |
62 | 
63 |
64 | If you navigate to the `/aws/containerinsights/appmesh-workshop/application` will will be able to see all the logs from the `nodejs-app` pod, including the Envoy proxy container:
65 |
66 | 
67 |
--------------------------------------------------------------------------------
/content/monitoring/enableinsights.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Enable CloudWatch Container Insights"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 5
5 | ---
6 |
7 | CloudWatch Container Insights is a fully managed service that collects, aggregates, and summarizes Amazon ECS metrics and logs. The CloudWatch Container Insights dashboard gives you access to the following information:
8 |
9 | * CPU and memory utilization
10 | * Task and service counts
11 | * Read/write storage
12 | * Network Rx/Tx
13 | * Container instance counts for clusters, services, and tasks
14 |
15 | To enable container insights, execute the following script
16 |
17 | ```bash
18 | # Define variables #
19 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
20 | # Update ecs cluster settings #
21 | aws ecs update-cluster-settings \
22 | --cluster $CLUSTER_NAME \
23 | --settings name=containerInsights,value=enabled
24 | ```
25 |
26 | Read and follow the instructions provided at [Introducing Container Insights for Amazon ECS](https://aws.amazon.com/blogs/mt/introducing-container-insights-for-amazon-ecs/) to access metrics collected by CloudWatch Container Insights.
--------------------------------------------------------------------------------
/content/monitoring/loggroup.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Review CloudWatch log groups"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 20
5 | ---
6 |
7 | * Access CloudWatch Logs to check the existence of the **appmesh-workshop-frontrend-envoy** and **appmesh-workshop-crystal-envoy** log groups.
8 |
9 | 
10 |
11 | * Take a few minutes to review the logging details produced by the Envoy container.
12 |
13 | 
14 |
15 | * Let's query the log group using CloudWatch Insights. Select CloudWatch Insights.
16 |
17 | 
18 |
19 | * Select the **appmesh-workshop-crystal-envoy** log group.
20 |
21 | 
22 |
23 | * Replace the exiting query with the following one.
24 |
25 | ```
26 | parse @message '[*] "* * *" * * * * * * "*" "*" "*" "*" "*"' as startTime, method, envoyOriginalPath, protocol, responseCode, responseFlags, bytesReceived, bytesSent, duration, upstreamServiceTime, xForwardedFor, userAgent, requestId, host, upstreamHost
27 | | sort startTime desc
28 | | filter envoyOriginalPath like /^(?!\s*$).+/
29 | | stats count(responseCode) by envoyOriginalPath, responseCode
30 | ```
31 |
32 | * This query will parse the Envoy access logs, and count the number of response codes per request path. Run the query to get your results.
33 |
34 | 
--------------------------------------------------------------------------------
/content/monitoring/monitoringcrystal.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Enable CloudWatch for Crystal"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 10
5 | ---
6 |
7 | You will create a new task definition that adds logging to the Envoy container. Once you have enabled it, you will access CloudWatch Logs to consume the logs produced by the Envoy proxy. By default, Envoy will produce application and access logs intermingled in the same CloudWatch Log file. Make sure you configured your Virtual Node representing your application to send logs to /dev/stdout
8 |
9 |
10 | * Register a new task definition to add logging to the Envoy container.
11 |
12 | ```bash
13 | # Define variables #
14 | TASK_DEF_ARN=$(aws ecs list-task-definitions | \
15 | jq -r ' .taskDefinitionArns[] | select( . | contains("crystal"))' | tail -1)
16 | TASK_DEF_OLD=$(aws ecs describe-task-definition --task-definition $TASK_DEF_ARN);
17 | TASK_DEF_NEW=$(echo $TASK_DEF_OLD \
18 | | jq ' .taskDefinition' \
19 | | jq --arg AWS_REGION $AWS_REGION ' .containerDefinitions |= map(
20 | if .name == "envoy" then . +=
21 | {
22 | "logConfiguration": {
23 | "logDriver": "awslogs",
24 | "options": {
25 | "awslogs-create-group": "true",
26 | "awslogs-region": $AWS_REGION,
27 | "awslogs-group": "appmesh-workshop-crystal-envoy",
28 | "awslogs-stream-prefix": "fargate"
29 | }
30 | }
31 | }
32 | else . end) ' \
33 | | jq ' del(.status, .compatibilities, .taskDefinitionArn, .requiresAttributes, .revision, .registeredBy, .registeredAt) '
34 | ); \
35 | TASK_DEF_FAMILY=$(echo $TASK_DEF_ARN | cut -d"/" -f2 | cut -d":" -f1);
36 | echo $TASK_DEF_NEW > /tmp/$TASK_DEF_FAMILY.json &&
37 | # Register ecs task definition #
38 | aws ecs register-task-definition \
39 | --cli-input-json file:///tmp/$TASK_DEF_FAMILY.json
40 | ```
41 |
42 | * Update the service.
43 |
44 | ```bash
45 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
46 | TASK_DEF_ARN=$(aws ecs list-task-definitions | \
47 | jq -r ' .taskDefinitionArns[] | select( . | contains("crystal"))' | tail -1)
48 | aws ecs update-service \
49 | --cluster $CLUSTER_NAME \
50 | --service crystal-service-lb \
51 | --task-definition "$(echo $TASK_DEF_ARN)"
52 | ```
53 |
54 | * Wait for the service tasks to be in a running state.
55 |
56 | ```bash
57 | # Define variables #
58 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
59 | TASK_DEF_ARN=$(aws ecs list-task-definitions | \
60 | jq -r ' .taskDefinitionArns[] | select( . | contains("crystal"))' | tail -1);
61 | # Get task state #
62 | _list_tasks() {
63 | aws ecs list-tasks \
64 | --cluster $CLUSTER_NAME \
65 | --service crystal-service-lb | \
66 | jq -r ' .taskArns | @text' | \
67 | while read taskArns; do
68 | aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $taskArns;
69 | done | \
70 | jq -r --arg TASK_DEF_ARN $TASK_DEF_ARN \
71 | ' [.tasks[] | select( (.taskDefinitionArn == $TASK_DEF_ARN)
72 | and (.lastStatus == "RUNNING" ))] | length'
73 | }
74 | until [ $(_list_tasks) == "3" ]; do
75 | echo "Tasks are starting ..."
76 | sleep 10s
77 | if [ $(_list_tasks) == "3" ]; then
78 | echo "Tasks started"
79 | break
80 | fi
81 | done
82 | ```
--------------------------------------------------------------------------------
/content/more_resources.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "More Resources"
3 | disableToc: true
4 | ---
5 |
6 | Discover more AWS resources for building and running your application on AWS:
7 |
8 | #### More Workshops
9 |
10 | * [Amazon ECS Workshop](https://ecsworkshop.com) - Learn how to use Stelligent Mu to deploy a microservice architecture that runs in AWS Fargate
11 | * [Amazon Lightsail Workshop](https://lightsailworkshop.com) - If you are getting started with the cloud and looking for a way to run an extremely low cost environment Lightsail is perfect. Learn how to deploy to Amazon Lightsail with this workshop.
12 |
13 | #### Tools for AWS Fargate and Amazon ECS
14 |
15 | * [Containers on AWS](https://containersonaws.com/) - Learn common best-practices for running containers on AWS
16 | - [fargate](http://somanymachines.com/fargate/) - Command line tool for interacting with AWS Fargate. With just a single command you can build, push, and launch your container in Fargate, orchestrated by ECS.
17 | - [Terraform](https://thecode.pub/easy-deploy-your-docker-applications-to-aws-using-ecs-and-fargate-a988a1cc842f) - Use Terraform to deploy your docker containers in Fargate
18 | - [Wonqa](https://www.npmjs.com/package/wonqa) is a tool for spinning up disposable QA environments in AWS Fargate, with SSL enabled by Let's Encrypt. More details about Wonqa on the [Wonder Engineering blog](https://medium.com/wonder-engineering/on-demand-qa-environments-with-aws-fargate-c23b41f15a0c)
19 | - [coldbrew](https://github.com/coldbrewcloud/coldbrew-cli) - Fantastic tool that provisions ECS infrastructure, builds and deploys your container, and connects your services to an application load balancer automatically. Has a great developer experience for day to day use
20 | - [mu](https://github.com/stelligent/mu) - Automates everything relating to ECS devops and CI/CD. This framework lets you write a simple metadata file and it constructs all the infrastructure you need so that you can deploy to ECS by simply pushing to your Git repo.
21 |
22 | #### Courses
23 |
24 | - [Microservices with Docker, Flask, and React](https://testdriven.io/) - Learn how to build, test, and deploy microservices powered by Docker, Flask, React Amazon ECS!
25 |
26 |
--------------------------------------------------------------------------------
/content/policyrouting/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Header Based Routing"
3 | chapter: true
4 | weight: 45
5 | ---
6 |
7 | # Routing to specific microservices
8 |
9 | 
10 |
11 | AWS App Mesh allows you to implement advanced routing capabilities between your microservices. One of the ways to achieve this is to working with **Header based routing**.
12 |
13 | Using header-based routing, you can create patterns such as session persistence (sticky sessions) or an enhanced experience using "state". HTTP header-based routing enables using HTTP header information as a basis to determine how to route a request. This might be a standard header, like Accept or Cookie, or it might be a custom header, like my-own-header-key-value.
14 |
15 | Header-based routing can also be used to enable use cases such as A/B testing (e.g.: custom headers using any string), canary or blue/green deployments, delivering different pages or user experiences based on categories of devices. (e.g.: using regex in header), handling traffic from different browsers differently (e.g. using user-agent) or configuring access restrictions based on IP address or CDN. (e.g. using X-Forwarded-for).
16 |
17 | In this step, let's configure a route that checks if the header `canary_fleet` is equals true. If so, the requests will be send to the `crystal-sd-epoch` version of the Crystal backend service.
18 |
19 |
20 |
21 | {{% children showhidden="false" %}}
22 |
--------------------------------------------------------------------------------
/content/policyrouting/curljson.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Check results"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 |
7 | * Run the following script to check if you are consistently getting a canary response from Crystal. You can compare these results with the random results from the web interface.
8 |
9 | ```bash
10 | # Define variables #
11 | URL=$(jq < cfn-output.json -r '.ExternalLoadBalancerDNS');
12 | # Execute curl #
13 | for ((i=1;i<=15;i++)); do
14 | curl --location --silent --header "canary_fleet: true" $URL/json | jq ' .';
15 | sleep 2s
16 | done
17 | ```
18 |
19 | * Alternatively, you can run the following script to compare your previous results using the command line. Notice, we are just omitting the **canary_fleet** header.
20 |
21 | ```bash
22 | # Define variables #
23 | URL=$(jq < cfn-output.json -r '.ExternalLoadBalancerDNS');
24 | # Execute curl #
25 | for ((i=1;i<=15;i++)); do
26 | curl --location --silent $URL/json | jq ' .';
27 | sleep 2s
28 | done
29 | ```
--------------------------------------------------------------------------------
/content/policyrouting/virtualroutes.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create traffic routes"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 5
5 | ---
6 |
7 | * Instead of randomly distributing traffic using weights among virtual nodes, direct the traffic to our canary only if the request header canary_fleet equals true.
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "httpRoute": {
14 | "action": {
15 | "weightedTargets": [
16 | {
17 | "virtualNode": "crystal-sd-epoch",
18 | "weight": 1
19 | }
20 | ]
21 | },
22 | "match": {
23 | "prefix": "/",
24 | "headers": [
25 | {
26 | "name": "canary_fleet",
27 | "match": {
28 | "exact": "true"
29 | }
30 | }
31 | ]
32 | }
33 | },
34 | "priority": 1
35 | }
36 | EOF
37 | ); \
38 | # Create app mesh route #
39 | aws appmesh create-route \
40 | --mesh-name appmesh-workshop \
41 | --virtual-router-name crystal-router \
42 | --route-name crystal-header-route \
43 | --spec "$SPEC"
44 | ```
--------------------------------------------------------------------------------
/content/prerequisites/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Start the Workshop..."
3 | chapter: true
4 | weight: 10
5 | ---
6 |
7 | # Getting Started
8 | To start the workshop, follow one of the following depending on whether you are...
9 |
10 | * ...[running the workshop on your own](self_paced/), or
11 | * ...[attending an AWS hosted event](aws_event/)
12 |
13 | Once you have completed with either setup, continue with [**Install the required tools**](/prerequisites/installtools/)
14 |
--------------------------------------------------------------------------------
/content/prerequisites/aws_event/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "...at an AWS event"
3 | chapter: true
4 | weight: 10
5 | ---
6 |
7 | ### Running the workshop at an AWS Event
8 |
9 | {{% notice warning %}}
10 | Only complete this section if you are at an AWS hosted event (such as re:Invent,
11 | Kubecon, Immersion Day, or any other event hosted by an AWS employee). If you
12 | are running the workshop on your own, go to:
13 |
14 | [Start the workshop on your own](../self_paced/).
15 | {{% /notice %}}
16 |
17 | {{% children %}}
18 |
--------------------------------------------------------------------------------
/content/prerequisites/aws_event/portal.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "AWS Workshop Portal"
3 | chapter: false
4 | weight: 20
5 | ---
6 |
7 | ### Login to AWS Workshop Portal
8 |
9 | This workshop creates an AWS acccount and a Cloud9 environment. You will need the **Participant Hash** provided upon entry, and your email address to track your unique session.
10 |
11 | Connect to the portal by clicking the button or browsing to [https://dashboard.eventengine.run/](https://dashboard.eventengine.run/). The following screen shows up.
12 |
13 | 
14 |
15 | Enter the provided hash in the text box. The button on the bottom right corner changes to **Accept Terms & Login**. Click on that button to continue.
16 |
17 | 
18 |
19 | Click on **AWS Console** on dashboard.
20 |
21 | 
22 |
23 | Take the defaults and click on **Open AWS Console**. This will open AWS Console in a new browser tab.
24 |
25 |
26 | {{% notice info %}}
27 | If you are running this workshop at an AWS Event, someone might have warmed your account for you. This means that you might already have the baseline infrastructure created and ready. Follow the next step to confirm if this is the case.
28 | {{% /notice %}}
29 |
30 | After logging in to your AWS account, click on the following link to access the Cloud9 console:
31 |
32 | https://console.aws.amazon.com/cloud9/home
33 |
34 | When in the Cloud9 console, check if there is a Cloud9 environment with the name `Project-appmesh-workshop` created. If so, click on `Open IDE` button to access the Cloud9 instance and continue the workshop from the chapter [**Update IAM Settings for your workspace**](/prerequisites/workspaceiam/)
35 |
36 | If the Cloud9 environment is not present, head straight to [**Create a Workspace**](/prerequisites/workspace/).
--------------------------------------------------------------------------------
/content/prerequisites/awscli.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update to the latest AWS CLI"
3 | chapter: false
4 | weight: 45
5 | draft: true
6 | comment: default install now includes aws-cli/1.15.83
7 | ---
8 |
9 | {{% notice tip %}}
10 | For this workshop, please ignore warnings about the version of pip being used.
11 | {{% /notice %}}
12 |
13 | 1. Run the following command to view the current version of aws-cli:
14 | ```
15 | aws --version
16 | ```
17 |
18 | 1. Update to the latest version:
19 | ```
20 | pip install --user --upgrade awscli
21 | ```
22 |
23 | 1. Confirm you have a newer version:
24 | ```
25 | aws --version
26 | ```
27 |
--------------------------------------------------------------------------------
/content/prerequisites/clone.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Clone the service repos"
3 | chapter: false
4 | weight: 20
5 | ---
6 |
7 | * Download the microservices artifacts.
8 |
9 | ```bash
10 | # clone the github repositories
11 | cd ~/environment
12 | git clone https://github.com/brentley/ecsdemo-frontend.git
13 | git clone https://github.com/brentley/ecsdemo-nodejs.git
14 | git clone https://github.com/brentley/ecsdemo-crystal.git
15 | ```
--------------------------------------------------------------------------------
/content/prerequisites/deploycfn.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Deploy the baseline stack"
3 | chapter: false
4 | weight: 21
5 | ---
6 |
7 | * Download the CloudFormation template:
8 |
9 | ```bash
10 | cd ~/environment
11 | curl -s https://raw.githubusercontent.com/brentley/appmeshworkshop/master/templates/appmesh-baseline.yml -o appmesh-baseline.yml
12 | ```
13 |
14 | * Deploy the CloudFormation stack:
15 |
16 | ```bash
17 | # Define environment variable
18 | IAM_ROLE=$(curl -s 169.254.169.254/latest/meta-data/iam/info | \
19 | jq -r '.InstanceProfileArn' | cut -d'/' -f2)
20 |
21 | #Check if the template is already deployed. If not, deploy it
22 | CFN_TEMPLATE=$(aws cloudformation list-stacks | jq -c '.StackSummaries[].StackName | select( . == "appmesh-workshop" )')
23 |
24 | if [ -z "$CFN_TEMPLATE" ]
25 | then
26 | echo "Deploying Cloudformation Template"
27 | aws cloudformation deploy \
28 | --template-file appmesh-baseline.yml \
29 | --stack-name appmesh-workshop \
30 | --capabilities CAPABILITY_IAM \
31 | --parameter-overrides Cloud9IAMRole=$IAM_ROLE
32 | else
33 | echo "Template already deployed. Go ahead to the next chapter."
34 | fi
35 | ```
36 |
37 | ___
38 |
39 | The CloudFormation template will launch the following:
40 |
41 | - VPC with private and public subnets - including routes, NAT Gateways and an Internet Gateway
42 | - VPC Endpoints to privately connect your VPC to AWS services
43 | - An ECS cluster with no EC2 resources because we're using Fargate
44 | - ECR repositories for your container images
45 | - A Launch Template and an Auto Scaling Group for your EC2 based services
46 | - Two Application Load Balancers to front internal and external services
47 | - A Private Hosted Zone for service discovery
48 |
49 | This is the detailed application architecture:
50 |
51 | 
52 |
--------------------------------------------------------------------------------
/content/prerequisites/ec2instance.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Attach the IAM role to your Workspace"
3 | chapter: false
4 | weight: 14
5 | ---
6 |
7 | 1. Follow [this deep link to find your Cloud9 EC2 instance](https://console.aws.amazon.com/ec2/v2/home?#Instances:tag:Name=aws-cloud9-AppMesh-Workshop;sort=desc:launchTime)
8 | 1. Select the instance, then choose **Actions / Security / Modify IAM Role**
9 | 
10 | 1. Choose **AppMesh-Workshop-Admin** from the **IAM Role** drop down, and select **Save**
11 | 
12 |
--------------------------------------------------------------------------------
/content/prerequisites/gitclone.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Clone the Workshop Repository"
3 | chapter: false
4 | weight: 65
5 | draft: true
6 | comment: disabled since eksctl doesn't require it
7 | ---
8 |
9 | We have a repository of tools to help with the remaining workshop. Clone the
10 | repository to your local workspace with this command:
11 | ```
12 | cd ~/environment
13 | git clone https://github.com/mandusm/howto-launch-eks-workshop.git
14 | ```
15 |
--------------------------------------------------------------------------------
/content/prerequisites/iamrole.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create an IAM role for your workspace"
3 | chapter: false
4 | weight: 13
5 | ---
6 |
7 | {{% notice info %}}
8 | Starting from here, when you see command to be entered such as below, you will enter these commands into Cloud9 IDE. You can use the **Copy to clipboard** feature (right hand upper corner) to simply copy and paste into Cloud9. In order to paste, you can use Ctrl + V for Windows or Command + V for Mac.
9 | {{% /notice %}}
10 |
11 | 1. Follow [this deep link to create an IAM role with Administrator access.](https://console.aws.amazon.com/iam/home#/roles$new?step=review&commonUseCase=EC2%2BEC2&selectedUseCase=EC2&policies=arn:aws:iam::aws:policy%2FAdministratorAccess)
12 | 1. Confirm that **AWS service** and **EC2** are selected, then click **Next** to view permissions.
13 | 1. Confirm that **AdministratorAccess** is checked, then click **Next** to review.
14 | 1. Enter **AppMesh-Workshop-Admin** for the Name, and select **Create Role**
15 | 
16 |
--------------------------------------------------------------------------------
/content/prerequisites/installtools.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Install the required tools"
3 | chapter: false
4 | weight: 16
5 | ---
6 |
7 | {{% notice info %}}
8 | Starting from here, when you see command to be entered such as below, you will enter these commands into Cloud9 IDE. You can use the **Copy to clipboard** feature (right hand upper corner) to simply copy and paste into Cloud9. In order to paste, you can use Ctrl + V for Windows or Command + V for Mac.
9 | {{% /notice %}}
10 |
11 | * Before deploying the baseline stack, let's install the required tools (kubectl, jq, gettext and the latest awscli version) to you Cloud9 environment. To do so, start creating an install script with the following commands:
12 |
13 | ```bash
14 | # create a folder for the scripts
15 | mkdir ~/environment/scripts
16 |
17 | # tools script
18 | cat > ~/environment/scripts/install-tools <<-"EOF"
19 |
20 | #!/bin/bash -ex
21 |
22 | sudo yum install -y jq gettext bash-completion
23 |
24 | sudo curl --silent --location "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm" -o "session-manager-plugin.rpm"
25 | sudo yum install -y session-manager-plugin.rpm
26 |
27 | sudo curl --silent --location -o /usr/local/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/v1.16.8/bin/linux/amd64/kubectl
28 | sudo chmod +x /usr/local/bin/kubectl
29 | echo 'source <(kubectl completion bash)' >>~/.bashrc
30 | source ~/.bashrc
31 |
32 | curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
33 | sudo mv -v /tmp/eksctl /usr/local/bin
34 |
35 | if ! [ -x "$(command -v jq)" ] || ! [ -x "$(command -v envsubst)" ] || ! [ -x "$(command -v kubectl)" ] || ! [ -x "$(command -v eksctl)" ] || ! [ -x "$(command -v ssm-cli)" ]; then
36 | echo 'ERROR: tools not installed.' >&2
37 | exit 1
38 | fi
39 |
40 | pip install awscli --upgrade --user
41 |
42 | EOF
43 |
44 | chmod +x ~/environment/scripts/install-tools
45 | ```
46 |
47 | * Now, run it with the following command:
48 |
49 | ```bash
50 | ~/environment/scripts/install-tools
51 | ```
52 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "...on your own"
3 | chapter: true
4 | weight: 11
5 | ---
6 |
7 | ### Running the workshop on your own
8 |
9 |
10 | {{% notice warning %}}
11 | Only complete this section if you are running the workshop on your own. If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, etc), go to [Start the workshop at an AWS event](../aws_event/).
12 | {{% /notice %}}
13 |
14 | {{% children %}}
15 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/account.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create an AWS account"
3 | chapter: false
4 | weight: 1
5 | ---
6 |
7 | {{% notice warning %}}
8 | Your account must have the ability to create new IAM roles and scope other IAM permissions.
9 | {{% /notice %}}
10 |
11 | 1. If you don't already have an AWS account with Administrator access: [create
12 | one now by clicking here](https://aws.amazon.com/getting-started/)
13 |
14 | 1. Once you have an AWS account, ensure you are following the remaining workshop steps
15 | as an IAM user with administrator access to the AWS account:
16 | [Create a new IAM user to use for the workshop](https://console.aws.amazon.com/iam/home?#/users$new)
17 |
18 | 1. Enter the user details:
19 | 
20 |
21 | 1. Attach the AdministratorAccess IAM Policy:
22 | 
23 |
24 | 1. Click to create the new user:
25 | 
26 |
27 | 1. Take note of the login URL and save:
28 | 
29 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/ap-southeast-1.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Singapore"
3 | chapter: false
4 | disableToc: true
5 | hidden: true
6 | ---
7 |
8 | Create a Cloud9 Environment: [https://ap-southeast-1.console.aws.amazon.com/cloud9/home?region=ap-southeast-1](https://ap-southeast-1.console.aws.amazon.com/cloud9/home?region=ap-southeast-1)
9 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/eu-west-1.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Ireland"
3 | chapter: false
4 | disableToc: true
5 | hidden: true
6 | ---
7 |
8 | Create a Cloud9 Environment: [https://eu-west-1.console.aws.amazon.com/cloud9/home?region=eu-west-1](https://eu-west-1.console.aws.amazon.com/cloud9/home?region=eu-west-1)
9 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/us-east-2.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Ohio"
3 | chapter: false
4 | disableToc: true
5 | hidden: true
6 | ---
7 |
8 | Create a Cloud9 Environment: [https://us-east-2.console.aws.amazon.com/cloud9/home?region=us-east-2](https://us-east-2.console.aws.amazon.com/cloud9/home?region=us-east-2)
9 |
--------------------------------------------------------------------------------
/content/prerequisites/self_paced/us-west-2.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Oregon"
3 | chapter: false
4 | disableToc: true
5 | hidden: true
6 | ---
7 |
8 | Create a Cloud9 Environment: [https://us-west-2.console.aws.amazon.com/cloud9/home?region=us-west-2](https://us-west-2.console.aws.amazon.com/cloud9/home?region=us-west-2)
9 |
--------------------------------------------------------------------------------
/content/prerequisites/sshkey.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Retrieve the SSH key"
3 | chapter: false
4 | weight: 22
5 | draft: false
6 | ---
7 |
8 | Run the following command to retrieve the SSH Key and store it in Cloud9. This key will be used on the ec2 and worker node instances to allow ssh access if necessary.
9 |
10 | ```bash
11 | # Retrieve private key
12 | aws ssm get-parameter \
13 | --name /appmeshworkshop/keypair/id_rsa \
14 | --with-decryption | jq .Parameter.Value --raw-output > ~/.ssh/id_rsa
15 |
16 | # Set appropriate permission on private key
17 | chmod 600 ~/.ssh/id_rsa
18 |
19 | # Store public key separately from private key
20 | ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub
21 | ```
22 |
--------------------------------------------------------------------------------
/content/prerequisites/workspace.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a workspace"
3 | chapter: false
4 | weight: 12
5 | ---
6 |
7 | {{% notice warning %}}
8 | The Cloud9 workspace should be built by an IAM user with Administrator privileges,
9 | not the root account user. Please ensure you are logged in as an IAM user, not the root
10 | account user.
11 | {{% /notice %}}
12 |
13 | {{% notice info %}}
14 | This workshop was designed to run in the **Oregon (us-west-2)** region. **Please don't
15 | run in any other region.** Future versions of this workshop will expand region availability,
16 | and this message will be removed.
17 | {{% /notice %}}
18 |
19 | {{% notice tip %}}
20 | Ad blockers, javascript disablers, and tracking blockers should be disabled for
21 | the cloud9 domain, or connecting to the workspace might be impacted.
22 | Cloud9 requires third-party-cookies. You can whitelist the [specific domains]( https://docs.aws.amazon.com/cloud9/latest/user-guide/troubleshooting.html#troubleshooting-env-loading).
23 | {{% /notice %}}
24 |
25 | ### Launch Cloud9:
26 | Create a Cloud9 Environment: [https://us-west-2.console.aws.amazon.com/cloud9/home?region=us-west-2](https://us-west-2.console.aws.amazon.com/cloud9/home?region=us-west-2)
27 |
28 | {{% notice warning %}}
29 | Make sure you are naming your Cloud9 environment `AppMesh-Workshop`, otherwise things will break later.
30 | {{% /notice %}}
31 |
32 | - Select **Create environment**
33 | - Name it **AppMesh-Workshop**, and take all other defaults
34 | - When it comes up, customize the environment by closing the **welcome tab**
35 | and **lower work area**, and opening a new **terminal** tab in the main work area:
36 | 
37 |
38 | - Your workspace should now look like this:
39 | 
40 |
41 | - If you like this theme, you can choose it yourself by selecting **View / Themes / Solarized / Solarized Dark**
42 | in the Cloud9 workspace menu.
43 |
--------------------------------------------------------------------------------
/content/prerequisites/workspaceiam.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update IAM settings for your workspace"
3 | chapter: false
4 | weight: 15
5 | ---
6 |
7 | {{% notice info %}}
8 | Cloud9 normally manages IAM credentials dynamically. This isn't currently compatible with
9 | the EKS IAM authentication, so we will disable it and rely on the IAM role instead.
10 | {{% /notice %}}
11 |
12 | - Return to your workspace and click the sprocket, or launch a new tab to open the Preferences tab
13 | - Select **AWS SETTINGS**
14 | - Turn off **AWS managed temporary credentials**
15 | - Close the Preferences tab
16 | 
17 |
18 | To ensure temporary credentials aren't already in place we will also remove
19 | any existing credentials file:
20 |
21 | ```bash
22 | rm -vf ${HOME}/.aws/credentials
23 | ```
24 |
25 | We should configure our aws cli with our current region as default:
26 |
27 | ```bash
28 | export ACCOUNT_ID=$(aws sts get-caller-identity --output text --query Account)
29 | export AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | \
30 | grep region | cut -d\" -f4)
31 |
32 | echo "export ACCOUNT_ID=${ACCOUNT_ID}" >> ~/.bash_profile
33 | echo "export AWS_REGION=${AWS_REGION}" >> ~/.bash_profile
34 | aws configure set default.region ${AWS_REGION}
35 | aws configure get default.region
36 | ```
37 |
38 | ### Validate the IAM role
39 |
40 | Use the [GetCallerIdentity](https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html) CLI command to validate that the Cloud9 IDE is using the correct IAM role.
41 |
42 | ```bash
43 | aws sts get-caller-identity
44 | ```
45 |
46 |
53 |
54 | The output assumed-role name should contain:
55 |
56 | ```
57 | TeamRole
58 | or
59 | AppMesh-Workshop-Admin
60 | ```
61 |
62 | #### VALID
63 |
64 | If the _Arn_ contains the role name from above and an Instance ID, you may proceed.
65 |
66 | ```output
67 | {
68 | "Account": "123456789012",
69 | "UserId": "AROA1SAMPLEAWSIAMROLE:i-01234567890abcdef",
70 | "Arn": "arn:aws:sts::123456789012:assumed-role/TeamRole/i-01234567890abcdef"
71 | }
72 | ```
73 |
74 | ```output
75 | {
76 | "Account": "123456789012",
77 | "UserId": "AROA1SAMPLEAWSIAMROLE:i-01234567890abcdef",
78 | "Arn": "arn:aws:sts::123456789012:assumed-role/AppMesh-Workshop-Admin/i-01234567890abcdef"
79 | }
80 | ```
81 |
82 |
83 | #### INVALID
84 |
85 | If the _Arn_ contains `MasterRole`, or does not match the role name output above, **DO NOT PROCEED**. Go back and confirm the steps on this page.
86 |
87 | ```output
88 | {
89 | "Account": "123456789012",
90 | "UserId": "AROA1SAMPLEAWSIAMROLE:i-01234567890abcdef",
91 | "Arn": "arn:aws:sts::123456789012:assumed-role/AppMesh-Workshop-Admin/MasterRole"
92 | }
93 | ```
94 |
--------------------------------------------------------------------------------
/content/servicediscovery/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Cloud Map Service Discovery"
3 | chapter: true
4 | weight: 35
5 | ---
6 |
7 | # Use Cloud Map based service discovery
8 |
9 | 
10 |
11 | Until now, the way our frontend application running in the EC2 instances was talking to the backend Crystal service running in the ECS Cluster and Nodejs backend running in Amazon EKS was by using two dedicated Load Balancers.
12 |
13 | Part of the transition to microservices and modern architectures involves having dynamic, autoscaling, and robust services that can respond quickly to failures and changing loads. A modern architectural best practice is to loosely couple these services by allowing them to specify their own dependencies. Compared to dedicated load balancing, service discovery (client side load balancing) can help improve resiliency, and convenience in dynamic and large microservice environments.
14 |
15 | [AWS Cloud Map](https://aws.amazon.com/cloud-map/) is a cloud resource discovery service. Cloud Map enables you to name your application resources with custom names, and it automatically updates the locations of these dynamically changing resources.
16 |
17 | The objective of this chapter is to change the way the Frontend application talks to the Crystal and NodeJS Backend application by configuring the integration between AWS App Mesh and AWS Cloud Map.
18 |
19 |
20 | {{% children showhidden="false" %}}
21 |
--------------------------------------------------------------------------------
/content/servicediscovery/delete_nodejs_lb.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Remove NodeJS Load Balancer"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 90
5 | ---
6 |
7 | Now that the frontend ec2 instances are talking to the NodeJS backend service using the Cloud Map service discovery integration, we can delete the Load Balancer that is fronting this application. To do so, let's delete the `nodejs-app-service`
8 |
9 | ```bash
10 | kubectl delete service nodejs-app-service -n appmesh-workshop-ns
11 | ```
12 |
13 | And now, changing the Route53 hosted zone in order to point it to the new domain name:
14 |
15 | ```bash
16 | # Define variables
17 | HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name \
18 | --dns-name appmeshworkshop.hosted.local \
19 | --max-items 1 | \
20 | jq -r ' .HostedZones | first | .Id');
21 | RECORD_SET=$(aws route53 list-resource-record-sets --hosted-zone-id=$HOSTED_ZONE_ID | \
22 | jq -r '.ResourceRecordSets[] | select (.Name == "nodejs.appmeshworkshop.hosted.local.")');
23 | cat <<-EOF > /tmp/update_nodejs_r53.json
24 | {
25 | "Comment": "UPDATE nodejs.appmeshworkshop.hosted.local",
26 | "Changes": [
27 | {
28 | "Action": "DELETE",
29 | "ResourceRecordSet": $RECORD_SET
30 | },
31 | {
32 | "Action": "CREATE",
33 | "ResourceRecordSet": {
34 | "Name": "nodejs.appmeshworkshop.hosted.local",
35 | "Type": "CNAME",
36 | "TTL": 300,
37 | "ResourceRecords": [
38 | { "Value": "nodejs.appmeshworkshop.pvt.local." }
39 | ]
40 | }
41 | }
42 | ]
43 | }
44 | EOF
45 | # Change route53 record set
46 | aws route53 change-resource-record-sets \
47 | --hosted-zone-id $HOSTED_ZONE_ID \
48 | --change-batch file:///tmp/update_nodejs_r53.json
49 |
50 | ```
51 |
52 | Finally, we need to change the security group in the Kubernetes worker nodes in order to allow traffic from the Frontend instances to the port 3000 of the worker nodes:
53 |
54 | ```bash
55 | # Set environment variables
56 | WORKER_NODE_DNS=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
57 | WORKER_NODES_SG_ID=$(aws ec2 describe-instances --filter "Name=private-dns-name,Values=$WORKER_NODE_DNS" | \
58 | jq -r .Reservations[].Instances[].SecurityGroups[0].GroupId)
59 | AUTO_SCALING_GROUP=$(jq < cfn-output.json -r '.RubyAutoScalingGroupName');
60 | FRONTEND_SG_ID=$(aws ec2 describe-instances --filters Name=tag:aws:autoscaling:groupName,Values=$AUTO_SCALING_GROUP | \
61 | jq -r '.Reservations[0].Instances[0].NetworkInterfaces[0].Groups[0].GroupId')
62 |
63 | # Update kubernetes worker nodes sg
64 | aws ec2 authorize-security-group-ingress --group-id $WORKER_NODES_SG_ID --protocol tcp --port 3000 --source-group $FRONTEND_SG_ID
65 | ```
66 |
67 |
68 |
69 | You can confirm that the Load Balancer is not there anymore by acessing the [EC2 console](http://console.aws.amazon.com/ec2/home) and clicking in `Load balancers` on the left side menu. You will notice that at this moment you just have the public load balancer that is fronting your EC2 instances that are serving the Ruby frontend application.
--------------------------------------------------------------------------------
/content/servicediscovery/nodejs_servicediscovery.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "NodeJS app with Cloud Map"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 80
5 | ---
6 |
7 | In order to leverage the integration between the NodeJS app running in the EKS cluster and the AWS Cloud Map service, we will be using the ExternalDNS Kubernetes connector.
8 |
9 | Since we are using the `AWS App Mesh Controller For K8s` and it is already integrated with Cloud Map, let's change the `Virtual Node` configuration, so it can register the Pods IP addresses to the Cloud Map:
10 |
11 | ```bash
12 | # Update the Virtual Node yaml file
13 | cat < ~/environment/eks-scripts/virtual-node.yml
14 | apiVersion: appmesh.k8s.aws/v1beta2
15 | kind: VirtualNode
16 | metadata:
17 | name: nodejs-app
18 | namespace: appmesh-workshop-ns
19 | spec:
20 | podSelector:
21 | matchLabels:
22 | app: nodejs-app
23 | listeners:
24 | - portMapping:
25 | port: 3000
26 | protocol: http
27 | serviceDiscovery:
28 | awsCloudMap:
29 | namespaceName: appmeshworkshop.pvt.local
30 | serviceName: nodejs
31 | EOF
32 |
33 | # Apply the configuration
34 | kubectl apply -f ~/environment/eks-scripts/virtual-node.yml
35 | ```
36 |
37 | After applying these changes, your pods IP Addresses will be added to the `appmeshworkshop.pvt.local` domain name, previously created in the Cloud Map. You can check that everything is working properly by using the AWS cli command bellow:
38 |
39 | ```bash
40 | aws servicediscovery discover-instances \
41 | --namespace appmeshworkshop.pvt.local \
42 | --service-name nodejs
43 | ```
44 |
45 | You can compare the IP addresses presented in the `Instance ID` fields with your Pods IP Addresses presented in the `IP` field of the following command:
46 |
47 | ```bash
48 | kubectl get pods -n appmesh-workshop-ns -o wide
49 | ```
50 |
51 | Or you can access the [Cloud Map console](https://console.aws.amazon.com/cloudmap/home), click in the `appmeshworkshop.pvt.local` domain name and then in the `nodejs` service name to see that your pods are registered under `Service instances`:
52 |
53 |
54 | 
55 |
--------------------------------------------------------------------------------
/content/servicediscovery/remove_internal_lb.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Remove internal load balancer"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 70
5 | ---
6 |
7 | Now that we could see that the Crystal backend service is working without using the Load Balancer, we can shift 100% of the traffic to it and delete the old service and the internal LB.
8 |
9 | To do so, let's start shifting 100% of the traffic to the virtual node with the Cloud Map integration:
10 |
11 | ```bash
12 | # Define variables #
13 | SPEC=$(cat <<-EOF
14 | {
15 | "httpRoute": {
16 | "action": {
17 | "weightedTargets": [
18 | {
19 | "virtualNode": "crystal-sd-vanilla",
20 | "weight": 1
21 | }
22 | ]
23 | },
24 | "match": {
25 | "prefix": "/"
26 | }
27 | },
28 | "priority": 10
29 | }
30 | EOF
31 | ); \
32 | # Update app mesh route #
33 | aws appmesh update-route \
34 | --mesh-name appmesh-workshop \
35 | --virtual-router-name crystal-router \
36 | --route-name crystal-traffic-route \
37 | --spec "$SPEC"
38 | ```
39 |
40 | After redirecting all the traffic to the new virtual node with the service discovery attributes, let's delete the old ECS service and the internal Load Balancer:
41 |
42 | ```bash
43 | # Define variables
44 | CLUSTER_NAME=$(jq < cfn-output.json -r '.EcsClusterName');
45 | INTERNAL_LB_ARN=$(jq < cfn-output.json -r '.InternalLoadBalancerArn');
46 | # Delete ecs service
47 | aws ecs delete-service \
48 | --cluster $CLUSTER_NAME \
49 | --service crystal-service-lb \
50 | --force
51 | # Delete load lalancer
52 | aws elbv2 delete-load-balancer \
53 | --load-balancer-arn $INTERNAL_LB_ARN
54 | ```
55 |
56 | * Update the CNAME record in Route53 to crystal.appmeshworkshop.pvt.local
57 |
58 | ```bash
59 | # Define variables
60 | HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name \
61 | --dns-name appmeshworkshop.hosted.local \
62 | --max-items 1 | \
63 | jq -r ' .HostedZones | first | .Id');
64 | RECORD_SET=$(aws route53 list-resource-record-sets --hosted-zone-id=$HOSTED_ZONE_ID | \
65 | jq -r '.ResourceRecordSets[] | select (.Name == "crystal.appmeshworkshop.hosted.local.")');
66 | cat <<-EOF > /tmp/update_r53.json
67 | {
68 | "Comment": "UPDATE crystal.appmeshworkshop.hosted.local",
69 | "Changes": [
70 | {
71 | "Action": "DELETE",
72 | "ResourceRecordSet": $RECORD_SET
73 | },
74 | {
75 | "Action": "CREATE",
76 | "ResourceRecordSet": {
77 | "Name": "crystal.appmeshworkshop.hosted.local",
78 | "Type": "CNAME",
79 | "TTL": 300,
80 | "ResourceRecords": [
81 | { "Value": "crystal.appmeshworkshop.pvt.local." }
82 | ]
83 | }
84 | }
85 | ]
86 | }
87 | EOF
88 | # Change route53 record set
89 | aws route53 change-resource-record-sets \
90 | --hosted-zone-id $HOSTED_ZONE_ID \
91 | --change-batch file:///tmp/update_r53.json
92 | ```
93 |
94 | You can now go ahead and test your application again to make sure everything is still working.
95 |
--------------------------------------------------------------------------------
/content/servicediscovery/servicediscovery.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a discovery service"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 10
5 | ---
6 | The Crystal backend service operates behind an internal (dedicated) load balancer. We will now configure it to use Amazon ECS Service Discovery. Service discovery uses AWS Cloud Map API actions to manage HTTP and DNS namespaces for Amazon ECS services.
7 |
8 | Given the dependencies between Cloud Map, ECS and App Mesh, you will proceed in the following order to enable Service Discovery:
9 |
10 | 1. We will start off by configuring a namespace and a service in Cloud Map.
11 | 2. We will then create the App Mesh resources needed to represent the new version of our ECS-based Crystal service.
12 | 3. Finally we will create a new service in ECS for the Crystal backend.
13 |
14 | * Let's create a namespace in Cloud Map to hold the service. Name it **appmeshworkshop.pvt.local**
15 |
16 | ```bash
17 | # Define variables #
18 | VPC_ID=$(jq < cfn-output.json -r '.VpcId');
19 | # Create cloud map namespace #
20 | OPERATION_ID=$(aws servicediscovery create-private-dns-namespace \
21 | --name appmeshworkshop.pvt.local \
22 | --description 'App Mesh Workshop private DNS namespace' \
23 | --vpc $VPC_ID | \
24 | jq -r ' .OperationId ')
25 | _operation_status() {
26 | aws servicediscovery get-operation \
27 | --operation-id $OPERATION_ID | \
28 | jq -r '.Operation.Status '
29 | }
30 | until [ $(_operation_status) != "PENDING" ]; do
31 | echo "Namespace is creating ..."
32 | sleep 10s
33 | if [ $(_operation_status) == "SUCCESS" ]; then
34 | echo "Namespace created"
35 | break
36 | fi
37 | done
38 | ```
39 |
40 | * Create a service inside the namespace created in the step above. Name it **crystal**. The service's FQDN becomes **crystal.appmeshworkshop.pvt.local**
41 |
42 | ```bash
43 | # Define variables #
44 | NAMESPACE=$(aws servicediscovery list-namespaces | \
45 | jq -r ' .Namespaces[] |
46 | select ( .Properties.HttpProperties.HttpName == "appmeshworkshop.pvt.local" ) | .Id ');
47 | # Create cloud map service #
48 | aws servicediscovery create-service \
49 | --name crystal \
50 | --description 'Discovery service for the Crystal service' \
51 | --namespace-id $NAMESPACE \
52 | --dns-config 'RoutingPolicy=MULTIVALUE,DnsRecords=[{Type=A,TTL=300}]' \
53 | --health-check-custom-config FailureThreshold=1
54 | ```
55 |
56 | Go back to the AWS Admin console and locate the Cloud Map service. Expand the left hand side section and click on the namespace **appmeshworkshop.pvt.local**. Take a look at the service definition.
57 |
58 | We are ready to start using the service we just defined in Cloud Map. Let's leverage ECS integration with Cloud Map to configure it.
59 |
--------------------------------------------------------------------------------
/content/servicediscovery/virtualnode.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a new virtual node"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 20
5 | ---
6 |
7 | * Create a second virtual node for the Crystal backend, and declare Cloud Map as the service discovery mechanism (instead of DNS).
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "serviceDiscovery": {
14 | "awsCloudMap": {
15 | "namespaceName": "appmeshworkshop.pvt.local",
16 | "serviceName": "crystal",
17 | "attributes": [
18 | {
19 | "key": "ECS_TASK_SET_EXTERNAL_ID",
20 | "value": "vanilla-task-set"
21 | }
22 | ]
23 | }
24 | },
25 | "logging": {
26 | "accessLog": {
27 | "file": {
28 | "path": "/dev/stdout"
29 | }
30 | }
31 | },
32 | "listeners": [
33 | {
34 | "healthCheck": {
35 | "healthyThreshold": 3,
36 | "intervalMillis": 10000,
37 | "path": "/health",
38 | "port": 3000,
39 | "protocol": "http",
40 | "timeoutMillis": 5000,
41 | "unhealthyThreshold": 3
42 | },
43 | "portMapping": { "port": 3000, "protocol": "http" }
44 | }
45 | ]
46 | }
47 | EOF
48 | ); \
49 | # Create app mesh virtual node #
50 | aws appmesh create-virtual-node \
51 | --mesh-name appmesh-workshop \
52 | --virtual-node-name crystal-sd-vanilla \
53 | --spec "$SPEC"
54 | ```
55 |
--------------------------------------------------------------------------------
/content/servicediscovery/virtualrouter.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Create a virtual router"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 30
5 | ---
6 |
7 | Virtual routers handle traffic for one or more virtual services within your mesh.
8 | We will create a virtual router and associate routes to direct incoming requests to the different virtual node destinations we have for the Crystal backend.
9 |
10 | * Begin by creating the virtual router.
11 |
12 | ```bash
13 | # Define variables #
14 | SPEC=$(cat <<-EOF
15 | {
16 | "listeners": [
17 | {
18 | "portMapping": { "port": 3000, "protocol": "http" }
19 | }
20 | ]
21 | }
22 | EOF
23 | ); \
24 | # Create app mesh virtual router #
25 | aws appmesh create-virtual-router \
26 | --mesh-name appmesh-workshop \
27 | --virtual-router-name crystal-router \
28 | --spec "$SPEC"
29 | ```
30 |
31 | * Create a route to shift the traffic to the new virtual node. For now, all traffic will be sent to the crystal-lb-vanilla virtual node. Once the crystal-sd-vanilla virutal node is fully operational, we will distribute between them at a 2:1 ratio (i.e., the crystal-sd-vanilla node will receive two thirds of the traffic).
32 |
33 | ```bash
34 | # Define variables #
35 | SPEC=$(cat <<-EOF
36 | {
37 | "httpRoute": {
38 | "action": {
39 | "weightedTargets": [
40 | {
41 | "virtualNode": "crystal-lb-vanilla",
42 | "weight": 1
43 | },
44 | {
45 | "virtualNode": "crystal-sd-vanilla",
46 | "weight": 0
47 | }
48 | ]
49 | },
50 | "match": {
51 | "prefix": "/"
52 | }
53 | },
54 | "priority": 10
55 | }
56 | EOF
57 | ); \
58 | # Create app mesh route #
59 | aws appmesh create-route \
60 | --mesh-name appmesh-workshop \
61 | --virtual-router-name crystal-router \
62 | --route-name crystal-traffic-route \
63 | --spec "$SPEC"
64 | ```
65 |
--------------------------------------------------------------------------------
/content/servicediscovery/virtualroutes.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update traffic route"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 60
5 | ---
6 |
7 | * We are ready to start shifting traffic to crystal-sd-vanilla. We only need to update the weight assigned to the crystal-sd-virtual node (from 0 to 2).
8 |
9 | ```bash
10 | # Define variables #
11 | SPEC=$(cat <<-EOF
12 | {
13 | "httpRoute": {
14 | "action": {
15 | "weightedTargets": [
16 | {
17 | "virtualNode": "crystal-lb-vanilla",
18 | "weight": 1
19 | },
20 | {
21 | "virtualNode": "crystal-sd-vanilla",
22 | "weight": 2
23 | }
24 | ]
25 | },
26 | "match": {
27 | "prefix": "/"
28 | }
29 | },
30 | "priority": 10
31 | }
32 | EOF
33 | ); \
34 | # Create app mesh route #
35 | aws appmesh update-route \
36 | --mesh-name appmesh-workshop \
37 | --virtual-router-name crystal-router \
38 | --route-name crystal-traffic-route \
39 | --spec "$SPEC"
40 | ```
--------------------------------------------------------------------------------
/content/servicediscovery/virtualservice.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Update the virtual service"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 40
5 | ---
6 | Virtual Services require a provider and you can specify either a Virtual Node or a Virtual Router as the provider of any given Virtual Service. The main difference between the two is that Virtual Routers allow to split traffic among multiple endpoints. If you plan to support A/B testing or Canary releases of new micro services version, then go with Virtual Routers. So far we have been using a Virtual node as the provider of the Crystal backend. Now we will switch to a Virtual Router as we want to gradually move traffic over the just created ECS service. Should things go wrong, can can simply adjust the traffic shaping percentages in our routes to move 100% of the incoming traffic to the old service version.
7 |
8 |
9 | * Update the virtual service to use the virtual router provider you just created.
10 |
11 | ```bash
12 | # Define variables #
13 | SPEC=$(cat <<-EOF
14 | {
15 | "provider": {
16 | "virtualRouter": {
17 | "virtualRouterName": "crystal-router"
18 | }
19 | }
20 | }
21 | EOF
22 | ); \
23 | # Create app mesh virtual service #
24 | aws appmesh update-virtual-service \
25 | --mesh-name appmesh-workshop \
26 | --virtual-service-name crystal.appmeshworkshop.hosted.local \
27 | --spec "$SPEC"
28 | ```
29 |
--------------------------------------------------------------------------------
/content/virtualgateway/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Ingress with Virtual Gateway"
3 | chapter: true
4 | weight: 38
5 | ---
6 |
7 | # Adding Ingress with Virtual Gateway
8 |
9 | In this section of workshop you will add an Ingress so that clients external to the mesh can interact with the (virtual) services that were already brought into the mesh, as is the case of the nodeJS EKS-based service.
10 |
11 | So what is an Ingress you may ask? Well, it’s a set of components that provide external client access to the services that reside inside the mesh.
12 |
13 | As you may recall, we have a frontend running on EC2 with 2 dependencies, one of them on ECS and another one on EKS.
14 |
15 | So under this arrangement there is no need for an Ingress for, say, the EKS backed service because its clients also reside inside the mesh (the app running on EC2). But what if you had an external client (sitting outside the mesh, in the same VPC) for instance a curler client that needs to access the Virtual Service represented by our EKS service? Ingress are the way to enable such communications.
16 |
17 | AppMesh offers a construct called Virtual Gateway, that provides this ingress functionality. You can read more about it [here](https://docs.aws.amazon.com/app-mesh/latest/userguide/virtual_gateways.html). The newer versions of the AppMesh controllers provide CDR for Virtual Gateways and Virtual Routes. So you can create a VG as you would with any other AppMesh construct, by leveraging the kubectl tool.
18 |
19 | So what happens when you create such component? The AppMesh controller will automatically deploy an AWS NLB inside your VPC and you get to define whether the NLB will be internal only or externally available (internet facing). Additionally, the controller will go ahead and create a new K8S deployment for the Envoy containers that will be the target of all the traffic that the NLB receives from its clients. Based on routing rules (path or header based at the time of this writing) that you define, the fleet of envoys that receive the traffic from the NLB will further route those requests to the corresponding Virtual Service inside the mesh.
20 |
21 | Here is a diagram of the new architecture with the Virtual gateway in place.
22 |
23 | 
24 |
25 | Let’s get started!
26 |
27 | {{% children showhidden="false" %}}
--------------------------------------------------------------------------------
/content/virtualgateway/crystal-ingress.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Virtual Gateway Ingress to Crystal service"
3 | date: 2018-09-18T17:39:30-05:00
4 | weight: 20
5 | ---
6 |
7 | As we can see during the workshop, the AWS App Mesh services enable you to add services to the mesh, whether they are running on EKS, ECS or EC2. So, what we are going to do now is to add the Crystal service, which is running on the ECS cluster, enabling access from outside the mesh to both App Mesh services inside de cluster.
8 |
9 | We just need to add a gateway route to the Crystal service running en ECS.
10 |
11 |
12 | ```bash
13 | VIRTUAL_GATEWAY=$(aws appmesh list-virtual-gateways --mesh-name appmesh-workshop | jq -r ".virtualGateways[].virtualGatewayName")
14 | VIRTUAL_SERVICE=$(aws appmesh list-virtual-services --mesh-name appmesh-workshop | jq -r ' .virtualServices[] | select( .virtualServiceName | contains("crystal")).virtualServiceName')
15 |
16 | # Create gatweay route spec json
17 | cat <<-EOF > ~/environment/eks-scripts/gateway-route-ecs-spec.json
18 | {
19 | "httpRoute": {
20 | "action": {
21 | "target": {
22 | "virtualService": {
23 | "virtualServiceName": "$VIRTUAL_SERVICE"
24 | }
25 | }
26 | },
27 | "match": {
28 | "prefix": "/ecs"
29 | }
30 | }
31 | }
32 | EOF
33 |
34 | aws appmesh create-gateway-route --gateway-route-name gateway-route-ecs --mesh-name appmesh-workshop --spec file://~/environment/eks-scripts/gateway-route-ecs-spec.json --virtual-gateway-name $VIRTUAL_GATEWAY
35 | ```
36 |
37 | Let’s test our new route.
38 |
39 | Connect to the EC2 instance outside the mesh in order to test reachability from external requests to the mesh using the virtual gateway ingress
40 |
41 | ```
42 | EXTERNAL_EC2=$(aws ec2 describe-instances --filters Name=tag:Usage,Values=ExternalEC2Instance | jq -r '.Reservations[].Instances[].InstanceId')
43 |
44 | aws ssm start-session --target $EXTERNAL_EC2
45 | ```
46 |
47 | And finally, let’s get the FQDN of the NLB that was created as part of the K8s Service of type LoadBalancer.
48 |
49 | ```
50 | NLB_ENDPOINT=$(kubectl get service -n appmesh-workshop-ns -o json | jq -r ".items[].status.loadBalancer.ingress[].hostname")
51 | ```
52 |
53 | Let’s try to connect
54 | ```
55 | curl -v $NLB_ENDPOINT/ecs/crystal
56 | ```
57 |
58 | You should now see a response similar to the one below
59 |
60 | 
61 |
--------------------------------------------------------------------------------
/content/x-ray/_index.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Distributed Tracing with X-Ray"
3 | chapter: true
4 | weight: 30
5 | ---
6 |
7 | # Add End-to-End Tracing Capabilities
8 |
9 | 
10 |
11 | [AWS X-Ray](https://aws.amazon.com/xray/) integrates with AWS App Mesh to manage Envoy proxies for microservices. App Mesh provides a version of Envoy that you can configure to send trace data to the X-Ray daemon running in a container of the same task or pod. X-Ray supports tracing with the following App Mesh compatible services:
12 |
13 | * Amazon Elastic Container Service (Amazon ECS)
14 | * Amazon Elastic Kubernetes Service (Amazon EKS)
15 | * Amazon Elastic Compute Cloud (Amazon EC2)
16 |
17 | Let's now enable the X-Ray integration for the Envoy proxies:
18 |
19 | {{% children showhidden="false" %}}
20 |
--------------------------------------------------------------------------------
/content/x-ray/reviewtraces.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Review traces"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 20
5 | ---
6 |
7 | Now access the [X-Ray console](https://console.aws.amazon.com/xray/home). Once in the X-Ray console, select **Service Map** from the left hand side menu. Wait a few seconds for the service map to render.
8 |
9 | You will see something like this:
10 |
11 | 
12 |
13 | Whenever you select a node or edge on an AWS X-Ray service map, the X-Ray console shows a latency distribution histogram. Latency is the amount of time between when a request starts and when it completes. A histogram shows a distribution of latencies. It shows duration on the x-axis, and the percentage of requests that match each duration on the y-axis.
14 |
15 | 
16 |
--------------------------------------------------------------------------------
/content/x-ray/xray_nodejs.md:
--------------------------------------------------------------------------------
1 | ---
2 | title: "Enable X-Ray for NodeJS App"
3 | date: 2018-09-18T16:01:14-05:00
4 | weight: 11
5 | ---
6 |
7 | Let's now enable the X-Ray integration to the NodeJS app currently running on the EKS cluster.
8 |
9 | The first step is to change the configuration of the App Mesh controller in order to automatically add the X-Ray container to new pods and configure the Envoy proxy containers to send data to them:
10 |
11 | ```bash
12 | helm upgrade -i appmesh-controller eks/appmesh-controller \
13 | --namespace appmesh-system \
14 | --set region=${AWS_REGION} \
15 | --set serviceAccount.create=false \
16 | --set serviceAccount.name=appmesh-controller \
17 | --set tracing.enabled=true \
18 | --set tracing.provider=x-ray
19 | ```
20 |
21 | Now, let's restart our pods again in order to force the injection of the X-Ray container:
22 |
23 | ```bash
24 | kubectl -n appmesh-workshop-ns rollout restart deployment nodejs-app
25 | ```
26 |
27 | Monitor the restarts and move on once the new pods are all in a `Running` state.
28 |
29 | ```bash
30 | kubectl -n appmesh-workshop-ns get pods -w
31 | ```
32 |
33 | Let's now check if the X-Ray container was injected in your pod:
34 |
35 | ```bash
36 | POD=$(kubectl -n appmesh-workshop-ns get pods -o jsonpath='{.items[0].metadata.name}')
37 | kubectl -n appmesh-workshop-ns get pods ${POD} -o jsonpath='{.spec.containers[*].name}'; echo
38 | ```
39 |
40 | You should see a `xray-daemon` container in the pod along with the app and Envoy containers.
41 |
42 |
--------------------------------------------------------------------------------
/layouts/404.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | {{ partial "meta.html" . }} {{ partial "favicon.html" . }} {{ .Scratch.Add "title" "" }}{{ if eq .Site.Data.titles .Title }}{{ .Scratch.Set "title" (index .Site.Data.titles .Title).title }}{{ else }}{{ .Scratch.Set "title" .Title}}{{end}}
6 | {{ .Scratch.Get "title" }}
7 |
8 | {{ $assetBusting := not .Site.Params.disableAssetsBusting }}
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 | {{with .Site.Params.themeVariant}}
17 |
18 | {{end}}
19 |
34 | {{ partial "custom-header.html" . }}
35 |
36 |
37 |
38 |
39 |
40 |
41 |
42 |
43 |