├── .github ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── ISSUE_TEMPLATE.md ├── PULL_REQUEST_TEMPLATE.md └── workflows │ └── main.yaml ├── .gitignore ├── .mdlrc ├── 00-dev-environment └── README.md ├── 01-cloudformation └── README.md ├── 02-s3 └── README.md ├── 03-iam └── README.md ├── 04-vpcs └── README.md ├── 05-ec2 └── README.md ├── 06-auto-scaling └── README.md ├── 07-load-balancing ├── README.md └── asg_example.yaml ├── 08-cloudwatch-logs ├── 8.1.2.yml └── README.md ├── 09-lambda └── README.md ├── 10-kms └── README.md ├── 11-parameter-store └── README.md ├── 12-codepipeline ├── README.md └── bucket.yaml ├── 13-ECS ├── README.md └── starter.yml ├── 14-Jenkins ├── README.md ├── base-params.json ├── base.yaml └── my-app │ ├── pom.xml │ └── src │ ├── main │ └── java │ │ └── com │ │ └── stelligent │ │ └── app │ │ └── App.java │ └── test │ └── java │ └── com │ └── stelligent │ └── app │ └── AppTest.java ├── 15-Kubernetes ├── README.md ├── ecr │ └── ecr.yaml ├── eksctl │ └── cluster.yaml └── sample_app │ ├── .gitignore │ ├── Dockerfile │ ├── README.md │ ├── package-lock.json │ ├── package.json │ ├── public │ ├── favicon.ico │ ├── favicon.ico.org │ ├── index.html │ ├── logo192.png │ ├── logo512.png │ ├── manifest.json │ └── robots.txt │ └── src │ ├── App.css │ ├── App.js │ ├── App.test.js │ ├── index.css │ ├── index.js │ ├── logo.svg │ ├── platformone.png │ ├── serviceWorker.js │ ├── setupTests.js │ └── stelligent.jpg ├── 16-SAM ├── README.md ├── codeDeployResources.yml ├── create-table.json └── lifecycleHook.js ├── 17-Terraform └── README.md ├── 18-step-functions └── README.md ├── LICENSE ├── MVE.md ├── README.md ├── WORKFLOW.md ├── images ├── create-new-repo.png ├── generated-repo.png ├── logo.png ├── pr-example.png └── use-this-template.png └── templates ├── NN-topic-name └── README.md ├── README.md └── topic-name └── README.md /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | - Using welcoming and inclusive language 18 | - Being respectful of differing viewpoints and experiences 19 | - Gracefully accepting constructive criticism 20 | - Focusing on what is best for the community 21 | - Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | - The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | - Trolling, insulting/derogatory comments, and personal or political attacks 28 | - Public or private harassment 29 | - Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | - Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at admin@stelligent.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the 71 | [Contributor Covenant Homepage](https://www.contributor-covenant.org), 72 | [version 1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct.html). 73 | 74 | [homepage]: https://www.contributor-covenant.org 75 | 76 | For answers to common questions about this code of conduct, see 77 | [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq) -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Stelligent Nu 2 | 3 | :100::boom: 4 | 5 | The following is a set of guidelines for contributing to Stelligent Nu 6 | 7 | ## Table Of Contents 8 | 9 | 10 | 11 | - [Code of Conduct](#code-of-conduct) 12 | - [How Can I Contribute?](#how-can-i-contribute) 13 | - [Reporting Bugs](#reporting-bugs) 14 | - [Suggesting Enhancements](#suggesting-enhancements) 15 | - [Your First Code Contribution](#your-first-code-contribution) 16 | - [Pull Requests](#pull-requests) 17 | - [Style Guides](#style-guides) 18 | - [Git Commit Messages](#git-commit-messages) 19 | - [Additional Notes](#additional-notes) 20 | - [Issue and Pull Request Labels](#issue-and-pull-request-labels) 21 | 22 | 23 | 24 | ## Code of Conduct 25 | 26 | This project and everyone participating in it is governed by the 27 | [Code of Conduct](.github/CODE_OF_CONDUCT.md). By participating, you are 28 | expected to uphold this code. 29 | 30 | ## How To Contribute 31 | 32 | ### Reporting Bugs 33 | 34 | This section guides you through submitting a bug report for Stelligent Nu. 35 | Following these guidelines helps maintainers and the community understand your 36 | report :pencil:, reproduce the behavior :computer: :computer:, and find related 37 | reports :mag_right:. 38 | 39 | Before creating bug reports, please check 40 | [this list](#before-submitting-a-bug-report) as you might find out that you 41 | don't need to create one. When you are creating a bug report, please 42 | [include as many details as possible](#how-do-i-submit-a-good-bug-report). 43 | Fill out [the required template](.github/ISSUE_TEMPLATE.md), the information 44 | it asks for helps us resolve issues faster. 45 | 46 | > __Note:__ If you find a __Closed__ issue that seems like it is the same thing 47 | > that you're experiencing, open a new issue and include a link to the original 48 | > issue in the body of your new one. 49 | 50 | #### Before Submitting A Bug Report 51 | 52 | - __Perform a [cursory search](https://github.com/stelligent/stelligent-nu/issues?utf8=%E2%9C%93&q=is%3Aissueu)__ 53 | to see if the problem has already been reported. If it has, 54 | __and the issue is still open__, add a comment to the existing issue instead 55 | of opening a new one. 56 | 57 | #### Submitting A (Good) Bug Report 58 | 59 | Bugs are tracked as [GitHub issues](https://guides.github.com/features/issues/). 60 | Create an issue and provide the following information by filling in 61 | [the template](.github/ISSUE_TEMPLATE.md). 62 | 63 | Explain the problem and include additional details to help maintainers 64 | reproduce the problem: 65 | 66 | - __Use a clear and descriptive title__ for the issue to identify the problem. 67 | - __Describe the exact steps which reproduce the problem__ in as many details 68 | as possible. For example, start by explaining how you started Stelligent Nu, 69 | e.g. which topic and lab you are on, or how you started Stelligent Nu 70 | otherwise. When listing steps, 71 | __don't just say what you did, but explain howyou did it__. For example, if 72 | you moved the cursor to the end of a line, explain if you used the mouse, or 73 | a keyboard shortcut or an Stelligent Nu question, and if so which one? 74 | - __Provide specific examples to demonstrate the steps__. Include links to 75 | files or GitHub projects, or copy/pasteable snippets, which you use in those 76 | examples. If you're providing snippets in the issue, use 77 | [Markdown code blocks](https://help.github.com/articles/markdown-basics/#multiple-lines). 78 | - __Describe the behavior you observed after following the steps__ and point 79 | out what exactly is the problem with that behavior. 80 | - __Explain which behavior you expected to see instead and why.__ 81 | 82 | Provide more context by answering these questions: 83 | 84 | - __Did the problem start happening recently__ (e.g. after completing a lab on 85 | Stelligent Nu) or was this always a problem? 86 | - __Can you reliably reproduce the issue?__ If not, provide details about how 87 | often the problem happens and under which conditions it normally happens. 88 | 89 | Include details about your configuration and environment: 90 | 91 | - __What's the name and version of the OS you're using__? 92 | 93 | ### Suggesting Enhancements 94 | 95 | This section guides you through submitting an enhancement suggestion for 96 | Stelligent Nu, including completely new features and minor improvements to 97 | existing functionality. Following these guidelines helps maintainers and the 98 | community understand your suggestion :pencil: and find related suggestions 99 | :mag_right:. 100 | 101 | Before creating enhancement suggestions, please check [this list](#before-submitting-an-enhancement-suggestion) 102 | as you might find out that you don't need to create one. When you are creating 103 | an enhancement suggestion, please [include as many details as possible](#how-do-i-submit-a-good-enhancement-suggestion). 104 | Fill in [the template](.github/ISSUE_TEMPLATE.md), including the steps that you 105 | imagine you would take if the feature you're requesting existed. 106 | 107 | #### Before Submitting An Enhancement Suggestion 108 | 109 | - __Perform a [cursory search](https://github.com/stelligent/stelligent-nu/issues?utf8=%E2%9C%93&q=is%3Aissue)__ 110 | to see if the enhancement has already been suggested. If it has, add a 111 | comment to the existing issue instead of opening a new one. 112 | 113 | #### Submitting A (Good) Enhancement Suggestion 114 | 115 | Enhancement suggestions are tracked as [GitHub issues](https://guides.github.com/features/issues/). 116 | Create an issue and provide the following information: 117 | 118 | - __Use a clear and descriptive title__ for the issue to identify the 119 | suggestion. 120 | - __Provide a step-by-step description of the suggested enhancement__ in as 121 | many details as possible. 122 | - __Provide specific examples to demonstrate the steps__. Include 123 | copy/pasteable snippets which you use in those examples, as 124 | [Markdown code blocks](https://help.github.com/articles/markdown-basics/#multiple-lines). 125 | - __Describe the current behavior__ and 126 | __explain which behavior you expected to see instead__ and why. 127 | - __Explain why this enhancement would be useful__ to most Stelligent Nu users. 128 | - __Specify the name and version of the OS you're using.__ 129 | 130 | ### Your First Code Contribution 131 | 132 | Unsure where to begin contributing to Stelligent Nu? You can start by looking 133 | through these `help-wanted` issues: 134 | 135 | - [Help wanted issues][help-wanted] - issues which should only require a few 136 | lines of code, and a test or two. 137 | 138 | Both issue lists are sorted by total number of comments. While not perfect, 139 | number of comments is a reasonable proxy for impact a given change will have. 140 | 141 | ### Pull Requests 142 | 143 | The process described here has several goals: 144 | 145 | - Maintain Stelligent Nu's quality 146 | - Fix problems that are important to users 147 | - Engage the community in working toward the best possible Stelligent Nu 148 | Coursework 149 | - Enable a sustainable system for Stelligent Nu's maintainers to review 150 | contributions 151 | 152 | Please follow these steps to have your contribution considered by the 153 | maintainers: 154 | 155 | 1. Follow all instructions in [the template](.github/PULL_REQUEST_TEMPLATE.md) 156 | 1. Follow the [styleguides](#styleguides) 157 | 1. Ensure you have linted your pull request with the [markdownlint gem](https://github.com/markdownlint/markdownlint). 158 | 1. After you submit your pull request, verify that all [status checks](https://help.github.com/articles/about-status-checks/) 159 | are passing. 160 | 161 | > _What if the status checks are failing?_ 162 | > 163 | > If a status check is failing, and you believe that the failure is unrelated to 164 | > your change, please leave a comment on the pull request explaining why you 165 | > believe the failure is unrelated. A maintainer will re-run the status check for 166 | > you. If we conclude that the failure was a false positive, then we will open an 167 | > issue to track that problem with our status check suite. 168 | 169 | While the prerequisites above must be satisfied prior to having your pull 170 | request reviewed, the reviewer(s) may ask you to complete additional design 171 | work, tests, or other changes before your pull request can be ultimately 172 | accepted. 173 | 174 | ## Style Guides 175 | 176 | ### Git Commit Messages 177 | 178 | - Use the present tense ("Add feature" not "Added feature") 179 | - Use the imperative mood ("Move cursor to..." not "Moves cursor to...") 180 | - Limit the first line to 72 characters or less 181 | - Reference issues and pull requests liberally after the first line 182 | - Consider starting the commit message with an applicable emoji: 183 | - :art: `:art:` when improving the format/structure of the code 184 | - :racehorse: `:racehorse:` when improving performance 185 | - :non-potable_water: `:non-potable_water:` when plugging memory leaks 186 | - :memo: `:memo:` when writing docs 187 | - :penguin: `:penguin:` when fixing something on Linux 188 | - :apple: `:apple:` when fixing something on macOS 189 | - :checkered_flag: `:checkered_flag:` when fixing something on Windows 190 | - :bug: `:bug:` when fixing a bug 191 | - :fire: `:fire:` when removing code or files 192 | - :green_heart: `:green_heart:` when fixing the CI build 193 | - :white_check_mark: `:white_check_mark:` when adding tests 194 | - :lock: `:lock:` when dealing with security 195 | - :arrow_up: `:arrow_up:` when upgrading dependencies 196 | - :arrow_down: `:arrow_down:` when downgrading dependencies 197 | - :shirt: `:shirt:` when removing linter warnings 198 | 199 | ## Additional Notes 200 | 201 | ### Issue and Pull Request Labels 202 | 203 | This section lists the labels we use to help us track and manage issues and 204 | pull requests. 205 | 206 | The labels are loosely grouped by their purpose, but it's not required that 207 | every issue have a label from every group or that an issue can't have more than 208 | one label from the same group. 209 | 210 | Please open an issue on `stelligent/stelligent-nu` if you have suggestions for 211 | new labels, and if you notice some labels are missing on some repositories, 212 | then please open an issue on that repository. 213 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 8 | 9 | # Prerequisites 10 | 11 | Check that your issue isn't [already filed](https://github.com/stelligent/stelligent-nu/issues?utf8=%E2%9C%93&q=is%3Aissue) 12 | 13 | ## Lab and Topic 14 | 15 | [Lab 2.2.2] 16 | 17 | ## Description 18 | 19 | [Description of the issue] 20 | 21 | ## Steps to Reproduce 22 | 23 | 1. [First Step] 24 | 1. [Second Step] 25 | 1. [and so on...] 26 | 27 | __Expected behavior:__ [What you expect to happen] 28 | 29 | __Actual behavior:__ [What actually happens] 30 | 31 | __Reproduces how often:__ [What percentage of the time does it reproduce?] 32 | 33 | ## Additional Information 34 | 35 | Any additional information, configuration or data that might be necessary to 36 | reproduce the issue. 37 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | # Requirements for Contributing a Bug Fix 2 | 3 | - Fill out the template below. Any pull request that does not include enough 4 | information to be reviewed in a timely manner may be closed at the maintainers' 5 | discretion. 6 | 7 | ## Identify the Bug 8 | 9 | 20 | 21 | ## Description of the Change 22 | 23 | 32 | 33 | ## Alternate Designs 34 | 35 | 37 | 38 | ## Possible Drawbacks 39 | 40 | 41 | 42 | ## Verification Process 43 | 44 | 51 | 52 | ## Release Notes 53 | 54 | 71 | -------------------------------------------------------------------------------- /.github/workflows/main.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: master 3 | on: 4 | pull_request: 5 | branches: 6 | - master 7 | 8 | jobs: 9 | linting: 10 | runs-on: ubuntu-latest 11 | 12 | steps: 13 | - name: Set Up ruby 2.7 14 | uses: actions/setup-ruby@v1 15 | with: 16 | ruby-version: "2.7" 17 | 18 | - name: Install dependencies 19 | run: gem install mdl 20 | 21 | - name: Checkout the code 22 | uses: actions/checkout@v1 23 | 24 | - name: Lint 25 | run: mdl $GITHUB_WORKSPACE 26 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .*.swp 3 | .*.swo 4 | *.pyc 5 | __pycache__ 6 | .pytest_cache 7 | 8 | .vscode 9 | .vscode/* 10 | *.code-workspace 11 | 12 | .ruby-version 13 | *.pem 14 | -------------------------------------------------------------------------------- /.mdlrc: -------------------------------------------------------------------------------- 1 | verbose false 2 | -------------------------------------------------------------------------------- /00-dev-environment/README.md: -------------------------------------------------------------------------------- 1 | # Topic 0: Development Environment 2 | 3 | 4 | 5 | - [Topic 0: Development Environment](#topic-0-development-environment) 6 | - [Conventions](#conventions) 7 | - [Lesson 0.1: Setting up your development environment](#lesson-01-setting-up-your-development-environment) 8 | - [Principle 0.1](#principle-01) 9 | - [Practice 0.1](#practice-01) 10 | - [Lab 0.1.1: AWS Access Keys](#lab-011-aws-access-keys) 11 | - [Option 1: Getting Credentials via STS command](#option-1-getting-credentials-via-sts-command) 12 | - [Exercise 0.1.1: MFA Script](#exercise-011-mfa-script) 13 | - [Question 0.1.1: 1](#question-011-1) 14 | - [Question 0.1.1: 2](#question-011-2) 15 | - [Option 2: Using AWS Vault to automatically handle your temporary tokens](#option-2-using-aws-vault-to-automatically-handle-your-temporary-tokens) 16 | - [Lab 0.1.2: GitHub](#lab-012-github) 17 | - [Lab 0.1.3: Cloud9 Environment](#lab-013-cloud9-environment) 18 | - [Lab 0.1.4: Clone Repository](#lab-014-clone-repository) 19 | - [Retrospective 0.1](#retrospective-01) 20 | - [Question: Environments](#question-environments) 21 | - [Task](#task) 22 | 23 | 24 | 25 | ## Conventions 26 | 27 | - Do NOT store AWS credentials in code repositories. 28 | - [Managing access keys](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html) 29 | - [Configuration and credentials files](https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html) 30 | 31 | - DO use the [AWS documentation and user guides](https://aws.amazon.com/documentation/). 32 | 33 | - DO use --debug and --event switches on aws cli commands to view the 34 | behind-the-curtain actions. 35 | 36 | ## Lesson 0.1: Setting up your development environment 37 | 38 | ### Principle 0.1 39 | 40 | *A reliable, repeatable process to create your development environment 41 | leads to a solution others can contribute to.* 42 | 43 | The following labs will prepare you to work through the Stelligent 44 | new hire training. During the training you 45 | will be using AWS access keys for programmatic access to an AWS account, 46 | GitHub code repositories, and, optionally, AWS Cloud9 for your development environment. 47 | 48 | ### Practice 0.1 49 | 50 | #### Lab 0.1.1: AWS Access Keys 51 | 52 | Save your AWS access key and secret key to a private credentials file on your laptop 53 | before enabling MFA. You won't be able to retrieve your token if the access 54 | key and secret key are not added to your credentials file. You will need to 55 | [install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html) 56 | and then run some simple commands to confirm access using your keys: 57 | 58 | - [list-buckets](https://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html) 59 | - [describe-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html) 60 | 61 | Remember that the combination of an access key and a secret key are 62 | the same as user credentials and should not be given out or stored 63 | in a public location like a GitHub repository, nor should they be 64 | transmitted in an insecure manner like unencrypted email. 65 | _Never_ commit credentials to a git repo. 66 | 67 | - [Enable MFA](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable_virtual.html) 68 | for your AWS account. With MFA enabled you will need to generate temporary 69 | credentials using your MFA device and STS. You can do this in a few 70 | ways. The first way would be to request the token from STS via the CLI. 71 | 72 | ##### Option 1: Getting Credentials via STS command 73 | 74 | ##### (in the AWS console > [user] > 75 | 76 | ##### Summary page under "Security credentials" tab 77 | 78 | ##### use the value for the "Assigned MFA device") 79 | 80 | ##### The token-code is your MFA validation code 81 | 82 | ```shell 83 | aws sts get-session-token \ 84 | --serial-number arn:aws:iam::324320755747:mfa/USERNAME \ 85 | --token-code 123456 \ 86 | / 87 | ``` 88 | 89 | This will return json containing the temporarily credentials. 90 | (**WARNING: special characters 91 | in the 'SecretAccessKey' may not work for Windoze machines) 92 | 93 | ```shell 94 | "Credentials": { 95 | "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", 96 | "SessionToken": "AQoDYXdzEJr...", 97 | "Expiration": "2018-10-11T10:09:50Z", 98 | "AccessKeyId": "ASIAIOSFODNN7EXAMPLE", 99 | } 100 | } 101 | ``` 102 | 103 | Using these temporary credentials you will need to edit your `.aws/credentials` 104 | file and change the security configuration for the profile to be used. 105 | 106 | ```shell 107 | [temp] 108 | output = json 109 | region = us-east-1 110 | aws_access_key_id = AKIAIOSFODNN7EXAMPLE 111 | aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY 112 | aws_session_token = AQoDYXdzEJr... 113 | ``` 114 | 115 | Now set AWS_PROFILE to temp in your environment variables: 116 | 117 | ```shell 118 | export AWS_ACCESS_KEY_ID= 119 | export AWS_SECRET_ACCESS_KEY= 120 | export AWS_DEFAULT_REGION= 121 | ``` 122 | 123 | or set the --profile flag to temp when 124 | running awscli: 125 | 126 | ```shell 127 | aws s3 ls --profile temp 128 | ``` 129 | 130 | You should be able to access any of the resources you're 131 | authorized to in the labs account. These tokens will last approximately 132 | 12 hours before needing to be reset using the process above. 133 | 134 | ##### Exercise 0.1.1: MFA Script 135 | 136 | 1. Create a script to automate the gathering and assigning of the temporary 137 | AWS MFA credentials from Option 1. 138 | 1. Try to reduce the amount of manual input as much as possible. 139 | 140 | ##### Question 0.1.1: 1 141 | 142 | What method did you use to store the aws credentials? What are some other 143 | options? 144 | 145 | ###### Question 0.1.1: 2 146 | 147 | Which AWS environment variable cannot be set in order to run the 148 | `aws sts get-session-token` command? 149 | 150 | ##### Option 2: Using AWS Vault to automatically handle your temporary tokens 151 | 152 | If you would rather not have to manually get STS tokens and add them to your 153 | credentials file you can use [aws-vault](https://github.com/99designs/aws-vault). 154 | 155 | Configure the profile that you would like to use using this command: 156 | `aws-vault add MY_PROFILE` 157 | 158 | If you accidentally setup aws-vault with a password that you forgot, 159 | you can remove its database (`rm -rf Library/Keychains/aws-vault.keychain-db`) 160 | and start over. 161 | 162 | To use aws-vault for temporary credential management is simple. You need to add 163 | the arn of your mfa token to your profiles config in `~/.aws/config` like so: 164 | 165 | ```shell 166 | [profile MY_PROFILE] 167 | output = json 168 | region = us-east-1 169 | mfa_serial = arn:aws:iam::324320755747:mfa/USERNAME 170 | ``` 171 | 172 | Now you can execute any command using aws-vault. Example: 173 | `aws-vault exec MY_PROFILE -- aws s3 ls` 174 | 175 | You want to set an alias in your .bashrc or .zshrc to something like this: 176 | `alias aws-myprofile="aws-vault exec MY_PROFILE -- aws"` 177 | 178 | #### Lab 0.1.2: GitHub 179 | 180 | 1. Create a new repository from the [Stelligent-U repository template](https://github.com/stelligent/stelligent-u/generate) 181 | 1. Select the owner of the repository 182 | 1. Name the new private repository 183 | 1. Generate ssh keys and 184 | test access. Use [this GitHub guide](https://help.github.com/articles/connecting-to-github-with-ssh/) 185 | to get access and clone the private repo to your laptop. 186 | 187 | #### Lab 0.1.3: Cloud9 Environment (Optional) 188 | 189 | AWS Cloud 9 is a resource to quickly re-create a stock development 190 | environment accessible from almost anywhere in a browser. You can use 191 | the Cloud9 environment to work on the remainder of the Stelligent-U 192 | labs, but it is not required. 193 | 194 | Create a new Cloud 9 environment in the region of your preference 195 | (ie: `us-east-1`) in the AWS account using the [AWS user guide](https://docs.aws.amazon.com/cloud9/latest/user-guide/welcome.html). 196 | Be sure to note the different options when setting up the environment, 197 | but use the default options for this lab. In Cloud9, run the same AWS 198 | CLI commands as you did in [lab 0.1.1](#lab-011-aws-access-keys): 199 | 200 | - `list-buckets` 201 | - `describe-instances` 202 | 203 | #### Lab 0.1.4: Clone Repository 204 | 205 | Work with your GitHub repo in your development environment. Start 206 | at [step 3 of this user guide](https://docs.aws.amazon.com/cloud9/latest/user-guide/sample-github.html#sample-github-install-git) 207 | to configure the Cloud9 instance to work with GitHub. 208 | 209 | 1. Install git in your development environment. 210 | 211 | 1. Clone the repository to your Cloud9 environment. To gain 212 | access to the repository try using either 213 | [token authentication](https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/) 214 | or an [SSH key](https://help.github.com/articles/connecting-to-github-with-ssh/). 215 | 216 | ### Retrospective 0.1 217 | 218 | #### Question: Environments 219 | 220 | _Running the two commands in [lab 0.1.1](#lab-011-aws-access-keys) and 221 | [lab 0.1.3](#lab-013-cloud9-environment) should have shown the same 222 | results. What does this tell you about the access the keys give you on 223 | your laptop and the access you have in the Cloud9 environment? What 224 | other methods are there to provide this level of access without using 225 | keys?_ 226 | 227 | #### Task 228 | 229 | Configure Cloud9 to work with the programming language you will be using 230 | to complete the training work. In your Cloud9 Editor, perform the following: 231 | 232 | - Please choose either Python, Ruby, Go or NodeJS 233 | - Create a new branch off master 234 | - In the 00-dev-environment directory, develop a small "hello world!" 235 | application in that language 236 | - Add the files to your new branch, commit them, and push the branch up to 237 | your forked repository 238 | - Create a pull request from your branch to the master branch within your forked 239 | repository and merge after reviewing 240 | -------------------------------------------------------------------------------- /01-cloudformation/README.md: -------------------------------------------------------------------------------- 1 | # Topic 1: CloudFormation 2 | 3 | 4 | 5 | - [Topic 1: CloudFormation](#topic-1-cloudformation) 6 | - [Conventions](#conventions) 7 | - [Lesson 1.1: Introduction to CloudFormation](#lesson-11-introduction-to-cloudformation) 8 | - [Principle 1.1](#principle-11) 9 | - [Practice 1.1](#practice-11) 10 | - [Lab 1.1.1: CloudFormation Template Requirements](#lab-111-cloudformation-template-requirements) 11 | - [Lab 1.1.2: Stack Parameters](#lab-112-stack-parameters) 12 | - [Lab 1.1.3: Pseudo-Parameters](#lab-113-pseudo-parameters) 13 | - [Lab 1.1.4: Using Conditions](#lab-114-using-conditions) 14 | - [Lab 1.1.5: Termination Protection; Clean up](#lab-115-termination-protection-clean-up) 15 | - [Retrospective 1.1](#retrospective-11) 16 | - [Question: Why YAML](#question-why-yaml) 17 | - [Question: Protecting Resources](#question-protecting-resources) 18 | - [Task: String Substitution](#task-string-substitution) 19 | - [Lesson 1.2: Integration with Other AWS Resources](#lesson-12-integration-with-other-aws-resources) 20 | - [Principle 1.2](#principle-12) 21 | - [Practice 1.2](#practice-12) 22 | - [Lab 1.2.1: Cross-Referencing Resources within a Template](#lab-121-cross-referencing-resources-within-a-template) 23 | - [Lab 1.2.2: Exposing Resource Details via Exports](#lab-122-exposing-resource-details-via-exports) 24 | - [Lab 1.2.3: Importing another Stack's Exports](#lab-123-importing-another-stacks-exports) 25 | - [Lab 1.2.4: Import/Export Dependencies](#lab-124-importexport-dependencies) 26 | - [Retrospective 1.2](#retrospective-12) 27 | - [Task: Policy Tester](#task-policy-tester) 28 | - [Task: SSM Parameter Store](#task-ssm-parameter-store) 29 | - [Lesson 1.3: Portability & Staying DRY](#lesson-13-portability--staying-dry) 30 | - [Principle 1.3](#principle-13) 31 | - [Practice 1.3](#practice-13) 32 | - [Lab 1.3.1: Scripts and Configuration](#lab-131-scripts-and-configuration) 33 | - [Lab 1.3.2: Coding with AWS SDKs](#lab-132-coding-with-aws-sdks) 34 | - [Lab 1.3.3: Enhancing the Code](#lab-133-enhancing-the-code) 35 | - [Retrospective 1.3](#retrospective-13) 36 | - [Question: Portability](#question-portability) 37 | - [Task: DRYer Code](#task-dryer-code) 38 | - [Additional Reading](#additional-reading) 39 | 40 | 41 | 42 | ## Conventions 43 | 44 | - All CloudFormation templates should be 45 | [written in YAML](https://getopentest.org/reference/yaml-primer.html) 46 | 47 | - Do NOT copy and paste CloudFormation templates from the Internet at large 48 | 49 | - DO use the [CloudFormation documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html) 50 | 51 | - DO utilize every link in this document; note how the AWS documentation is 52 | laid out 53 | 54 | - DO use the [AWS CLI for CloudFormation](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/index.html#) 55 | (NOT the Console) unless otherwise specified. 56 | 57 | ## Lesson 1.1: Introduction to CloudFormation 58 | 59 | ### Principle 1.1 60 | 61 | AWS CloudFormation (CFN) is the preferred way we create AWS resources at Stelligent 62 | 63 | ### Practice 1.1 64 | 65 | A CFN Template is essentially a set of instructions for creating AWS 66 | resources, which includes practically everything that can be created in 67 | AWS. At its simplest, the service accepts a Template (a YAML-based 68 | blueprint describing the resources you want to create or update) and 69 | creates a Stack (a set of resources created using a single template). 70 | The resulting Stacks represent groups of resources whose life-cycles are 71 | inherently linked. 72 | 73 | Read through [Template Anatomy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html) 74 | and get familiar with the basic parts of a CloudFormation template. 75 | 76 | #### Lab 1.1.1: CloudFormation Template Requirements 77 | 78 | Create the *most minimal CFN template possible* that can be used to 79 | create an AWS Simple Storage Service (S3) Bucket. 80 | 81 | - Always write your CloudFormation [templates in YAML](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-formats.html). 82 | 83 | - Launch a Stack by [using the AWS CLI tool](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/create-stack.html) 84 | to run the template. Use your preferred region. 85 | 86 | - Note the output provided by creating the Stack. 87 | 88 | - Though *functionally* unnecessary, the Description (i.e. its *purpose*) 89 | element documents your code's *intent*, so provide one. The Description 90 | key-value pair should be at the _root level_ of your template. If you place 91 | it under the definition of a resource, AWS will allow the template's creation 92 | but your description will not populate anything. See 93 | [here](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html) 94 | for a useful guide to the anatomy of a template as well as 95 | [YAML terminology](https://yaml.org/spec/1.2/spec.html#id2759768). 96 | 97 | - Commit the template to your Github repository under the 01-cloudformation 98 | folder. 99 | 100 | #### Lab 1.1.2: Stack Parameters 101 | 102 | Update the same template by adding a CloudFormation 103 | [Parameter](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html) 104 | to the stack and use the parameter's value as the name of the S3 bucket. 105 | 106 | - Put your parameter into a separate JSON file and pass that file to the CLI. 107 | 108 | - Update your stack. 109 | 110 | - Add the template changes and new parameter file to your Github repo. 111 | 112 | #### Lab 1.1.3: Pseudo-Parameters 113 | 114 | Update the same template by prefixing the name of the bucket with the 115 | Account ID in which it is being created, no matter which account you're 116 | running the template from (i.e., using 117 | [pseudo-parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html)). 118 | 119 | - Use built-in CFN string functions to combine the two strings for the Bucket name. 120 | 121 | - Do not hard code the Account ID. Do not use an additional parameter to 122 | provide the Account ID value. 123 | 124 | - Update the stack. 125 | 126 | - Commit the changes to your Github repo. 127 | 128 | #### Lab 1.1.4: Using Conditions 129 | 130 | Update the same template one final time. This time, use a CloudFormation 131 | [Condition](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html) 132 | to add a prefix to the name of the bucket. When the current execution 133 | region is your preferred region, prefix the bucket name with the 134 | Account ID. When executing in all other regions, use the region 135 | name. 136 | 137 | - Update the stack that you originally deployed. 138 | 139 | - Create a new stack _with the same stack name_, but this time 140 | deploying to some region other than your preferred region. 141 | 142 | - Commit the changes to your Github repo. 143 | 144 | #### Lab 1.1.5: Termination Protection; Clean up 145 | 146 | - Before deleting this lesson's Stacks, apply 147 | [Termination Protection](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-protect-stacks.html) 148 | to one of them. 149 | 150 | - Try to delete the Stack using the AWS CLI. What happens? 151 | 152 | - Remove termination protection and try again. 153 | 154 | - List the S3 buckets in both regions once this lesson's Stacks have been 155 | deleted to ensure their removal. 156 | 157 | ### Retrospective 1.1 158 | 159 | #### Question: Why YAML 160 | 161 | _Why do we prefer the YAML format for CFN templates?_ 162 | 163 | #### Question: Protecting Resources 164 | 165 | _What else can you do to prevent resources in a stack from being deleted?_ 166 | 167 | See [DeletionPolicy](https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-accidental-updates/). 168 | 169 | _How is that different from applying Termination Protection?_ 170 | 171 | #### Task: String Substitution 172 | 173 | Demonstrate 2 ways to code string combination/substitution using 174 | built-in CFN functions. 175 | 176 | ## Lesson 1.2: Integration with Other AWS Resources 177 | 178 | ### Principle 1.2 179 | 180 | CloudFormation integrates well with the rest of the AWS ecosystem 181 | 182 | ### Practice 1.2 183 | 184 | A CFN template's resources can reference: each other's attributes, 185 | resource attributes exported from other Stacks in the same region, and 186 | Systems Manager Parameter Store values in the same region. This provides 187 | a way to have resources build on each other to create your AWS 188 | ecosystem. 189 | 190 | #### Lab 1.2.1: Cross-Referencing Resources within a Template 191 | 192 | Create a CFN template that describes two resources: an IAM User, and an 193 | IAM Managed Policy that controls that user. 194 | 195 | - The policy should allow access solely to 'Read' actions against all 196 | S3 Buckets (including listing buckets and downloading individual bucket contents) 197 | 198 | - Attach the policy to the user via the template. 199 | 200 | - Use a CFN Parameter to set the user's name 201 | 202 | - Create the Stack. 203 | 204 | #### Lab 1.2.2: Exposing Resource Details via Exports 205 | 206 | Update the template by adding a CFN Output that exports the Managed 207 | Policy's Amazon Resource Name ([ARN](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html)). 208 | 209 | - Update the Stack. 210 | 211 | - [List all the Stack Exports](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/list-exports.html) 212 | in that Stack's region. 213 | 214 | #### Lab 1.2.3: Importing another Stack's Exports 215 | 216 | Create a *new* CFN template that describes an IAM User and applies to it 217 | the Managed Policy ARN created by and exported from the previous Stack. 218 | 219 | - Create this new Stack. 220 | 221 | - [List all the Stack Imports](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/list-imports.html) 222 | in that stack's region. 223 | 224 | #### Lab 1.2.4: Import/Export Dependencies 225 | 226 | Delete your CFN stacks in the same order you created them in. Did you 227 | succeed? If not, describe how you would _identify_ the problem, and 228 | resolve it yourself. 229 | 230 | ### Retrospective 1.2 231 | 232 | #### Task: Policy Tester 233 | 234 | Show how to use the IAM policy tester to demonstrate that the user 235 | cannot perform 'Put' actions on any S3 buckets. 236 | 237 | #### Task: SSM Parameter Store 238 | 239 | Using the AWS Console, create a Systems Manager Parameter Store 240 | parameter in the same region as the first Stack, and provide a value for 241 | that parameter. Modify the first Stack's template so that it utilizes 242 | this Parameter Store parameter value as the IAM User's name. Update the 243 | first stack. Finally, tear it down. 244 | 245 | ## Lesson 1.3: Portability & Staying DRY 246 | 247 | ### Principle 1.3 248 | 249 | _CloudFormation templates should be portable, supporting 250 | [Don't Repeat Yourself](http://wiki.c2.com/?DontRepeatYourself) (DRY) 251 | practices._ 252 | 253 | ### Practice 1.3 254 | 255 | Portability refers to the ability of code (whether it's a script or an 256 | entire application) to work in multiple execution environments. This is 257 | achieved most often by removing hard coded configuration elements and 258 | providing an environment-specific configuration file. For CFN templates, 259 | portability is best provided by parameterizing the template (refer to 260 | [AWS CloudFormation Best Practices](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/best-practices.html#reuse) 261 | for a more thorough list of recommendations for improving your use of 262 | CloudFormation). Some lab exercises have already demonstrated 263 | portability (_can you point out where?_) and this lesson will focus 264 | on it specifically. 265 | 266 | #### Lab 1.3.1: Scripts and Configuration 267 | 268 | Create a single script that re-uses one CloudFormation template to 269 | deploy _a single S3 bucket_. 270 | 271 | - Use shell scripting (bash or PowerShell) to create a Stack in each 272 | of the [4 American regions](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html), 273 | using a looping construct to run the template the proper number of times. 274 | 275 | - Use an external JSON or YAML configuration file to maintain the target 276 | deployment region parameters. Consider using `jq` or `yq` to parse this file. 277 | 278 | - Each bucket name should be of the format 279 | "_current-Region_-_current-Account_-_friendly-name_" 280 | where the "_friendly-name_" value is parameterized in the CFN template 281 | but has a default value. 282 | 283 | #### Lab 1.3.2: Coding with AWS SDKs 284 | 285 | Repeat the exercise in the previous lab, with two modifications: 286 | 287 | - Use only a programming language 288 | ([Python](https://aws.amazon.com/developers/getting-started/python/), 289 | [Ruby](https://aws.amazon.com/developers/getting-started/ruby/) 290 | or [Javascript - i.e. NodeJS](https://aws.amazon.com/developers/getting-started/nodejs/)) 291 | and the corresponding SDK to repeat exactly what was done in that lab. 292 | 293 | - Extend the region targets (i.e. modify your configuration file) to 294 | include another US region. 295 | 296 | Also adhere to these criteria: 297 | 298 | - The code must support updating existing stacks and creating new 299 | ones. This can be tricky as some SDKs require that you use a 300 | 'try/catch' construct to determine the existence of a stack. 301 | (Using rescue-oriented structures for decision logic is generally 302 | considered a programming anti-pattern.) 303 | 304 | - Use only a single shell command to execute your code script. 305 | 306 | #### Lab 1.3.3: Enhancing the Code 307 | 308 | Add code that provides for the deletion of your CFN stacks using the 309 | same configuration list, and then delete the stacks using that new 310 | functionality. Query S3 to ensure that the buckets have been deleted. 311 | 312 | - Commit your changes to your latest branch. 313 | 314 | ### Retrospective 1.3 315 | 316 | #### Question: Portability 317 | 318 | _Can you list 4 features of CloudFormation that help make a CFN template 319 | portable code?_ 320 | 321 | #### Task: DRYer Code 322 | 323 | How reusable is your SDK-orchestration code? Did you share a single 324 | method to load the configuration file for both stack creation/updating 325 | (Lab 1.3.2) and deletion (Lab 1.3.3)? Did you separate the methods for 326 | finding existing stacks from the methods that create or update those stacks? 327 | 328 | If not, refactor your Python, Ruby or NodeJS scripts to work in the 329 | manner described. 330 | 331 | ## Additional Reading 332 | 333 | Related topics to extend your knowledge about CloudFormation: 334 | 335 | - Using [Stack Policies](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html) 336 | to apply permissions to modify a stack 337 | 338 | - Using [StackSets](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/what-is-cfnstacksets.html) 339 | to deploy a CloudFormation stack simultaneously across an array of 340 | AWS Account and Regions 341 | -------------------------------------------------------------------------------- /03-iam/README.md: -------------------------------------------------------------------------------- 1 | # Topic 3: Identity and Access Management (IAM) 2 | 3 | 4 | 5 | - [Topic 3: Identity and Access Management (IAM)](#topic-3-identity-and-access-management-iam) 6 | - [Conventions](#conventions) 7 | - [Lesson 3.1: Introduction to Identity and Access Management](#lesson-31-introduction-to-identity-and-access-management) 8 | - [Principle 3.1](#principle-31) 9 | - [Practice 3.1](#practice-31) 10 | - [Lab 3.1.1: IAM Role](#lab-311-iam-role) 11 | - [Lab 3.1.2: Customer Managed Policy](#lab-312-customer-managed-policy) 12 | - [Lab 3.1.3: Customer Managed Policy Re-Use](#lab-313-customer-managed-policy-re-use) 13 | - [Lab 3.1.4: AWS-Managed Policies](#lab-314-aws-managed-policies) 14 | - [Lab 3.1.5: Policy Simulator](#lab-315-policy-simulator) 15 | - [Lab 3.1.6: Clean Up](#lab-316-clean-up) 16 | - [Retrospective 3.1](#retrospective-31) 17 | - [Question: Stack Outputs](#question-stack-outputs) 18 | - [Task: Stack Outputs](#task-stack-outputs) 19 | - [Lesson 3.2: Trust Relationships & Assuming Roles](#lesson-32-trust-relationships--assuming-roles) 20 | - [Principle 3.2](#principle-32) 21 | - [Practice 3.2](#practice-32) 22 | - [Lab 3.2.1: Trust Policy](#lab-321-trust-policy) 23 | - [Lab 3.2.2: Explore the assumed role](#lab-322-explore-the-assumed-role) 24 | - [Lab 3.2.3: Add privileges to the role](#lab-323-add-privileges-to-the-role) 25 | - [Lab 3.2.4: Clean up](#lab-324-clean-up) 26 | - [Retrospective 3.2](#retrospective-32) 27 | - [Question: Inline vs Customer Managed Policies](#question-inline-vs-customer-managed-policies) 28 | - [Question: Role Assumption](#question-role-assumption) 29 | - [Lesson 3.3: Fine-Grained Controls With Policies](#lesson-33-fine-grained-controls-with-policies) 30 | - [Principle 3.3](#principle-33) 31 | - [Practice 3.3](#practice-33) 32 | - [Lab 3.3.1: Unrestricted access to a service](#lab-331-unrestricted-access-to-a-service) 33 | - [Lab 3.3.2: Resource restrictions](#lab-332-resource-restrictions) 34 | - [Lab 3.3.3: Conditional restrictions](#lab-333-conditional-restrictions) 35 | - [Retrospective](#retrospective) 36 | - [Question: Positive and Negative Tests](#question-positive-and-negative-tests) 37 | - [Task: Positive and Negative Tests](#task-positive-and-negative-tests) 38 | - [Question: Limiting Uploads](#question-limiting-uploads) 39 | - [Task: Limiting Uploads](#task-limiting-uploads) 40 | - [Further Reading](#further-reading) 41 | 42 | 43 | 44 | ## Conventions 45 | 46 | - All CloudFormation templates should be written in YAML 47 | 48 | - Do NOT copy and paste CloudFormation templates from the Internet at 49 | large 50 | 51 | - DO use the [CloudFormation documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html) 52 | 53 | - DO utilize every link in this document; note how the AWS 54 | documentation is laid out 55 | 56 | - DO use the AWS CLI for 57 | [CloudFormation](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/index.html#) 58 | and 59 | [IAM](https://docs.aws.amazon.com/cli/latest/reference/iam/index.html) 60 | (NOT the Console) unless otherwise specified. 61 | 62 | ## Lesson 3.1: Introduction to Identity and Access Management 63 | 64 | ### Principle 3.1 65 | 66 | *Identity and Access Management (IAM) is the **authentication and authorization service** 67 | used to control access to virtually everything in AWS.* 68 | 69 | ### Practice 3.1 70 | 71 | IAM consists of a set of services and resources that allow individuals 72 | (and services) to authenticate with AWS and then authorizes those 73 | entities to perform specific activities with specific services. Like 74 | most authentication/authorization systems, IAM deals with the *concepts* 75 | of users, groups and permissions, but not necessarily those precise 76 | *entities*. 77 | 78 | #### Lab 3.1.1: IAM Role 79 | 80 | Create a CFN template that specifies an IAM Role. 81 | 82 | - Provide values only for required attributes. 83 | 84 | - Using inline Policies, give the Role read-only access to all IAM 85 | resources. 86 | 87 | - Create the Stack. 88 | 89 | - Use the [awscli](https://docs.aws.amazon.com/cli/latest/reference/iam/index.html) 90 | to query the IAM service twice: 91 | 92 | - List all the Roles 93 | - Describe the specific Role your Stack created. 94 | 95 | #### Lab 3.1.2: Customer Managed Policy 96 | 97 | Update the template and the corresponding Stack to make the IAM Role's 98 | inline policy more generally usable: 99 | 100 | - Convert the IAM Role's inline Policies array to a separate 101 | [customer managed policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies) 102 | resource. 103 | 104 | - Attach the new resource to the IAM Role. 105 | 106 | - Update the Stack using the modified template. 107 | 108 | #### Lab 3.1.3: Customer Managed Policy Re-Use 109 | 110 | Update the template further to demonstrate reuse of the customer managed 111 | policy: 112 | 113 | - Add another IAM Role. 114 | 115 | - Attach the customer managed policy resource to the new role. 116 | 117 | - Be sure that you're not referencing an AWS managed policy in the 118 | role. 119 | 120 | - Add/Update the Description of the customer managed policy to 121 | indicate the re-use of the policy. 122 | 123 | - Update the Stack. *Did the stack update work?* 124 | 125 | - Query the stack to determine its state. 126 | - If the stack update was not successful, 127 | [troubleshoot and determine why](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html#update-replacement). 128 | 129 | #### Lab 3.1.4: AWS-Managed Policies 130 | 131 | Replace the customer managed policy with 132 | [AWS managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies). 133 | 134 | - To both roles, replace the customer managed policy reference with 135 | the corresponding AWS managed policy granting Read permissions to 136 | the IAM service. 137 | 138 | - To the second role, add an additional AWS managed policy to grant 139 | Read permissions to the EC2 service. 140 | 141 | - Update the stack. 142 | 143 | #### Lab 3.1.5: Policy Simulator 144 | 145 | Read about the [AWS Policy Simulator](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html) 146 | tool and practice using it. 147 | 148 | - Using the two roles in your stack, simulate the ability of each role 149 | to perform the following actions (using the AWS CLI): 150 | 151 | - `iam:CreateRole` 152 | - `iam:ListRoles` 153 | - `iam:SimulatePrincipalPolicy` 154 | - `ec2:DescribeImages` 155 | - `ec2:RunInstances` 156 | - `ec2:DescribeSecurityGroups` 157 | 158 | #### Lab 3.1.6: Clean Up 159 | 160 | Clean up after yourself by deleting the stack. 161 | 162 | ### Retrospective 3.1 163 | 164 | #### Question: Stack Outputs 165 | 166 | _In Lab 3.1.5, you had to determine the Amazon resource Names (ARN) of the 167 | stack's two roles in order to pass those values to the CLI function. You 168 | probably used the AWS web console to get the ARN for each role. What 169 | could you have done to your CFN template to make that unnecessary?_ 170 | 171 | #### Task: Stack Outputs 172 | 173 | Institute that change from the Question above. Recreate the stack as per 174 | Lab 3.1.5, and demonstrate how to retrieve the ARNs. 175 | 176 | ## Lesson 3.2: Trust Relationships & Assuming Roles 177 | 178 | ### Principle 3.2 179 | 180 | *AWS service roles and other IAM principals can assume customer created 181 | roles, enabling a principle-of-least-privilege of permissions for AWS 182 | services and applications.* 183 | 184 | ### Practice 3.2 185 | 186 | An IAM Role has two kinds of policies. The first we've worked with 187 | already and this policy type (whether inline or managed) describes 188 | permissions the role has. The second is a trust policy, describing which 189 | AWS principles (services, roles and users) are allowed to masquerade as 190 | that role. 191 | 192 | For example, an AWS Lambda Function requires an execution role that 193 | defines the permissions the function will have when it executes. To 194 | provide those permissions, the role must trust the AWS Lambda service to 195 | assume it, and this trust must be granted explicitly by the role. 196 | 197 | In these labs, you will use your IAM User to assume roles in order to 198 | explore policies and permissions. 199 | 200 | #### Lab 3.2.1: Trust Policy 201 | 202 | Create a CFN template that creates an IAM Role and makes it possible for 203 | your User to assume that role. 204 | 205 | - The role should reference the AWS managed policy ReadOnlyAccess. 206 | 207 | - Add a trust relationship to the role that enables your specific IAM 208 | user to assume that role. 209 | 210 | - Create the stack. 211 | 212 | - Using the AWS CLI, assume that new role. If this fails, take note of 213 | the error you receive, diagnose the issue and fix it. 214 | 215 | *Hint: Instead of setting up a new profile in your \~/.aws/credentials 216 | file, use [aws sts assume-role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html#using-temp-creds-sdk-cli). 217 | It's a valuable mechanism you'll use often through the API, and it's good to 218 | know how to do it from the CLI as well.* 219 | 220 | #### Lab 3.2.2: Explore the assumed role 221 | 222 | Test the capabilities of this new Role. 223 | 224 | - Using the AWS CLI, assume that updated role and list the S3 buckets 225 | in the us-east-1 region. 226 | 227 | - Acting as this role, try to create an S3 bucket using the AWS CLI. 228 | 229 | - Did it succeed? It should not have! 230 | - If it succeeded, troubleshoot how Read access allowed the role 231 | to create a bucket. 232 | 233 | #### Lab 3.2.3: Add privileges to the role 234 | 235 | Update the CFN template to give this role the ability to upload to S3 236 | buckets. 237 | 238 | - Create an S3 bucket. 239 | 240 | - Using either an inline policy or an AWS managed policy, provide the 241 | role with S3 full access 242 | 243 | - Update the stack. 244 | 245 | - Assuming this role again, try to upload a text file to the bucket. 246 | 247 | - If it failed, troubleshoot the error iteratively until the role is 248 | able to upload a file to the bucket. 249 | 250 | #### Lab 3.2.4: Clean up 251 | 252 | Clean up. Take the actions necessary to delete the stack. 253 | 254 | ### Retrospective 3.2 255 | 256 | #### Question: Inline vs Customer Managed Policies 257 | 258 | _In the context of an AWS User or Role, what is the difference between 259 | an inline policy and a customer managed policy? What are the differences 260 | between a customer managed policy and an AWS managed policy?_ 261 | 262 | #### Question: Role Assumption 263 | 264 | _When assuming a role, are the permissions of the initial principal 265 | mixed with those of the role being assumed? 266 | Describe how that could easily be demonstrated with both a 267 | [positive and negative testing](https://www.guru99.com/positive-vs-negative-testing.html) 268 | approach._ 269 | 270 | ## Lesson 3.3: Fine-Grained Controls With Policies 271 | 272 | ### Principle 3.3 273 | 274 | *AWS policies can provide fine-grained access control to specific 275 | resources using specific conditions.* 276 | 277 | ### Practice 3.3 278 | 279 | So far we have only provided service-level IAM policy controls, but IAM 280 | policies generally should be more specific than that. For example, a 281 | service role for an application will generally only need read/write 282 | access to those specific resources that the application uses, and even 283 | then that resource might only be accessible under certain conditions. 284 | [Actions, Resources, and Condition Keys for AWS Services](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_actions-resources-contextkeys.html) 285 | introduces the topic. We'll be exploring Resource restrictions and 286 | Condition keys in this lesson. 287 | 288 | Keep in mind that not all resource types support resource-level 289 | restrictions. See the Resource-level permissions information in 290 | [AWS Services That Work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) 291 | for details. 292 | 293 | #### Lab 3.3.1: Unrestricted access to a service 294 | 295 | Create a CFN template that generates two S3 buckets and a Role, and 296 | demonstrate you have full access to each bucket with this new role. 297 | 298 | - Code a Role your User can assume with a customer managed policy that 299 | allows full access to the S3 service. 300 | 301 | - Create the stack. 302 | 303 | - As your User: 304 | 305 | - list the contents of your 2 new buckets 306 | - upload a file to each new bucket 307 | 308 | - Assume the new role and repeat those two checks as that role. 309 | 310 | #### Lab 3.3.2: Resource restrictions 311 | 312 | Add a resource restriction to the role's policy that limits full access 313 | to the S3 service for just one of the two buckets and allows only 314 | read-only access to the other. 315 | 316 | - Update the stack. 317 | 318 | - Assume the new role and perform these steps as that role: 319 | 320 | - List the contents of your 2 new buckets. 321 | - Upload a file to each new bucket. 322 | 323 | *Were there any errors? If so, take note of them.* 324 | 325 | *What were the results you expected, based on the role's policy?* 326 | 327 | #### Lab 3.3.3: Conditional restrictions 328 | 329 | Add a conditional restriction to the role's policy. Provide a condition 330 | that grants list access only to objects that start with "lebowski/". 331 | 332 | - Update the stack. 333 | 334 | - Assume the new role and perform the remaining directives as that 335 | role. 336 | 337 | - Try to list a file in the root of the available bucket 338 | 339 | - If it *worked*, fix your policy and update the stack until this 340 | fails. 341 | 342 | - Try to list that same file but now with the proper object key 343 | prefix. 344 | 345 | - If it *doesn't work*, troubleshoot why and fix either the role's 346 | policy or the list command syntax until you are able to 347 | list a file. 348 | 349 | ### Retrospective 350 | 351 | #### Question: Positive and Negative Tests 352 | 353 | _Were the tests you ran for resource- and condition-specific 354 | restrictions exhaustive? Did you consider additional [[positive and/or negative 355 | tests]](https://smartbear.com/learn/automated-testing/negative-testing/) 356 | that could be automated in order to confirm the permissions for the 357 | Role?_ 358 | 359 | #### Task: Positive and Negative Tests 360 | 361 | Code at least one new positive and one new negative test. 362 | 363 | #### Question: Limiting Uploads 364 | 365 | _Is it possible to limit uploads of objects with a specific prefix (e.g. 366 | starting with "lebowski/") to an S3 bucket using IAM conditions? If not, how else 367 | could this be accomplished?_ 368 | 369 | #### Task: Limiting Uploads 370 | 371 | Research and review the best method to limit uploads with a specific prefix to 372 | an S3 bucket. 373 | 374 | ## Further Reading 375 | 376 | - Read through the [IAM Best Practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) 377 | and be sure you're familiar with the ideas there. 378 | -------------------------------------------------------------------------------- /04-vpcs/README.md: -------------------------------------------------------------------------------- 1 | # Topic 4: Virtual Private Clouds (VPCs) 2 | 3 | 4 | 5 | - [Topic 4: Virtual Private Clouds (VPCs)](#topic-4-virtual-private-clouds-vpcs) 6 | - [Lesson 4.1: Creating Your Own VPC](#lesson-41-creating-your-own-vpc) 7 | - [Principle 4.1](#principle-41) 8 | - [Practice 4.1](#practice-41) 9 | - [Lab 4.1.1: New VPC with "Private" Subnet](#lab-411-new-vpc-with-private-subnet) 10 | - [Lab 4.1.2: Internet Gateway](#lab-412-internet-gateway) 11 | - [Lab 4.1.3: EC2 Key Pair](#lab-413-ec2-key-pair) 12 | - [Lab 4.1.4: Test Instance](#lab-414-test-instance) 13 | - [Question: Post Launch](#question-post-launch) 14 | - [Question: Verify Connectivity](#question-verify-connectivity) 15 | - [Lab 4.1.5: Security Group](#lab-415-security-group) 16 | - [Question: Connectivity](#question-connectivity) 17 | - [Lab 4.1.6: Elastic IP](#lab-416-elastic-ip) 18 | - [Question: Ping](#question-ping) 19 | - [Question: SSH](#question-ssh) 20 | - [Question: Traffic](#question-traffic) 21 | - [Lab 4.1.7: NAT Gateway](#lab-417-nat-gateway) 22 | - [Question: Access](#question-access) 23 | - [Question: Egress](#question-egress) 24 | - [Question: Deleting the Gateway](#question-deleting-the-gateway) 25 | - [Question: Recreating the Gateway](#question-recreating-the-gateway) 26 | - [Lab 4.1.8: Network ACL](#lab-418-network-acl) 27 | - [Question: EC2 Connection](#question-ec2-connection) 28 | - [Retrospective 4.1](#retrospective-41) 29 | - [Lesson 4.2: Integration with VPCs](#lesson-42-integration-with-vpcs) 30 | - [Principle 4.2](#principle-42) 31 | - [Practice 4.2](#practice-42) 32 | - [Lab 4.2.1: VPC Peering](#lab-421-vpc-peering) 33 | - [Lab 4.2.2: EC2 across VPCs](#lab-422-ec2-across-vpcs) 34 | - [Question: Public to Private](#question-public-to-private) 35 | - [Question: Private to Public](#question-private-to-public) 36 | - [Lab 4.2.3: VPC Endpoint Gateway to S3](#lab-423-vpc-endpoint-gateway-to-s3) 37 | - [Retrospective 4.2](#retrospective-42) 38 | - [Question: Corporate Networks](#question-corporate-networks) 39 | - [Further Reading](#further-reading) 40 | 41 | 42 | 43 | ## Lesson 4.1: Creating Your Own VPC 44 | 45 | ### Principle 4.1 46 | 47 | *VPCs provide isolated environments for running all of your AWS 48 | services. Non-default VPCs are a critical component of any safe 49 | architecture.* 50 | 51 | ### Practice 4.1 52 | 53 | This section walks you through the steps to create a new VPC. On every 54 | engagement, you'll be working in VPCs created by us or the client. Never 55 | use EC2 Classic or the default VPC. 56 | 57 | This is a complicated set of labs. If you get stuck, take a look at the 58 | example template in the 59 | [AWS::EC2::VPCPeering](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpcpeeringconnection.html) 60 | doc. It gives you a lot of info, but you can use it to see how resources 61 | are tied together. The AWS docs also provide a 62 | [VPC template sample](https://s3.amazonaws.com/cloudformation-templates-us-east-1/vpc_single_instance_in_subnet.template) 63 | that may be useful. 64 | 65 | #### Lab 4.1.1: New VPC with "Private" Subnet 66 | 67 | Launch a new VPC via your AWS account, specifying a region that will be 68 | used throughout these lessons. 69 | 70 | - Use a [CloudFormation YAML template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html). 71 | 72 | - Assign it a /16 CIDR block in [private IP space](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html#VPC_Sizing), 73 | and provide that block as a stack parameter in a [[separate parameters.json 74 | file]](https://aws.amazon.com/blogs/devops/passing-parameters-to-cloudformation-stacks-with-the-aws-cli-and-powershell). 75 | 76 | - Create an EC2 subnet resource within your CIDR block that has a /24 77 | netmask. 78 | 79 | - Provide the VPC ID and subnet ID as stack outputs. 80 | 81 | - Tag all your new resources with: 82 | 83 | - the key "user" and your AWS user name; 84 | - "stelligent-u-lesson" and this lesson number; 85 | - "stelligent-u-lab" and this lab number. 86 | 87 | - Don't use dedicated tenancy (it's needlessly expensive). 88 | 89 | #### Lab 4.1.2: Internet Gateway 90 | 91 | Update your template to allow traffic [to and from instances](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html) 92 | on your "private" subnet. 93 | 94 | - Add an Internet gateway 95 | [resource](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-internetgateway.html). 96 | 97 | - Attach the gateway to your VPC. 98 | 99 | - Create a route table for the VPC, add the gateway to it, attach it 100 | to your subnet. 101 | 102 | We can't call your subnet "private" any more. Now that it has an 103 | Internet Gateway, it can get traffic directly from the public Internet. 104 | 105 | #### Lab 4.1.3: EC2 Key Pair 106 | 107 | [Create an EC2 key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair) 108 | that you'll use to ssh to a test instance created in later labs. Use the 109 | AWS CLI. 110 | 111 | - Save the output as a .pem file in your project directory. 112 | 113 | - Be sure to create it in the same region you'll be doing your labs. 114 | 115 | #### Lab 4.1.4: Test Instance 116 | 117 | Launch an EC2 instance into your VPC. 118 | 119 | - Create another CFN template that specifies an EC2 instance. 120 | 121 | - For the subnet and VPC, reference the outputs from your VPC stack. 122 | 123 | - Use the latest Amazon Linux AMI. 124 | 125 | - Create a new parameter file for this template. Include the EC2 AMI 126 | ID, a T2 instance type, and the name of your key pair. 127 | 128 | - Provide the instance ID and private IP address as stack outputs. 129 | 130 | - Use the same tags you put on your VPC. 131 | 132 | ##### Question: Post Launch 133 | 134 | _After you launch your new stack, can you ssh to the instance?_ 135 | 136 | ##### Question: Verify Connectivity 137 | 138 | _Is there a way that you can verify Internet connectivity from the instance 139 | without ssh'ing to it?_ 140 | 141 | #### Lab 4.1.5: Security Group 142 | 143 | Add a security group to your EC2 stack: 144 | 145 | - Allow ICMP (for ping) and ssh traffic into your instance. 146 | 147 | ##### Question: Connectivity 148 | 149 | _Can you ssh to your instance yet?_ 150 | 151 | #### Lab 4.1.6: Elastic IP 152 | 153 | Add an Elastic IP to your EC2 stack: 154 | 155 | - Attach it to your EC2 resource. 156 | 157 | - Provide the public IP as a stack output. 158 | 159 | Your EC2 was already on a network with an IGW, and now we've fully 160 | exposed it to the Internet by giving it a public IP address that's 161 | reachable from anywhere outside your VPC. 162 | 163 | ##### Question: Ping 164 | 165 | _Can you ping your instance now?_ 166 | 167 | ##### Question: SSH 168 | 169 | _Can you ssh into your instance now?_ 170 | 171 | ##### Question: Traffic 172 | 173 | _If you can ssh, can you send any traffic (e.g. curl) out to the Internet?_ 174 | 175 | At this point, you've made your public EC2 instance an [ssh bastion](https://docs.aws.amazon.com/quickstart/latest/linux-bastion/architecture.html). 176 | We'll make use of that to explore your network below. 177 | 178 | #### Lab 4.1.7: NAT Gateway 179 | 180 | Update your VPC template/stack by adding a [NAT gateway](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html). 181 | 182 | - Attach your NAT GW to the subnet you created earlier. 183 | 184 | - Provision and attach a new Elastic IP for the NAT gateway. 185 | 186 | We need a private instance to explore some of the concepts below. Let's 187 | add a new subnet and put a new EC2 instance on it. Add them to your 188 | existing instance stack. 189 | 190 | - The new subnet must have a unique netblock. 191 | 192 | - The NAT gateway should be the default route for the new subnet. 193 | 194 | - Aside from the subnet association, configure this instance just like 195 | the first one. 196 | 197 | - This instance will not have an Elastic IP. 198 | 199 | ##### Question: Access 200 | 201 | _Can you find a way to ssh to this instance?_ 202 | 203 | ##### Question: Egress 204 | 205 | _If you can ssh to it, can you send traffic out?_ 206 | 207 | ##### Question: Deleting the Gateway 208 | 209 | _If you delete the NAT gateway, what happens to the ssh session on your private 210 | instance?_ 211 | 212 | ##### Question: Recreating the Gateway 213 | 214 | _If you recreate the NAT gateway and detach the Elastic IP from the public EC2 215 | instance, can you still reach the instance from the outside?_ 216 | 217 | Test it out with the AWS console. 218 | 219 | #### Lab 4.1.8: Network ACL 220 | 221 | Add Network ACLs to your VPC stack. 222 | 223 | First, add one on the public subnet: 224 | 225 | - It applies to all traffic (0.0.0.0/0). 226 | 227 | - Only allows ssh traffic from your IP address. 228 | 229 | - Allows egress traffic to anything. 230 | 231 | ##### Question: EC2 Connection 232 | 233 | _Can you still reach your EC2 instances?_ 234 | 235 | Add another ACL to your private subnet: 236 | 237 | - Only allow traffic from the public subnet. 238 | 239 | - Allow only ssh, ping, and HTTP. 240 | 241 | - Allow all ports for egress traffic, but restrict replies to the 242 | public subnet. 243 | 244 | _Verify again that you can reach your instance._ 245 | 246 | ### Retrospective 4.1 247 | 248 | For more information, read the [AWS Documentation on VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) 249 | 250 | ## Lesson 4.2: Integration with VPCs 251 | 252 | ### Principle 4.2 253 | 254 | *VPCs are most useful when connected to external resources: other VPCs, 255 | other AWS services, and corporate networks.* 256 | 257 | ### Practice 4.2 258 | 259 | VPCs provide important isolation for your resources. Often, though, they 260 | need to be connected to other services to poke holes through those walls 261 | of isolation. 262 | 263 | #### Lab 4.2.1: VPC Peering 264 | 265 | Copy the VPC template you created earlier and modify it to launch a 266 | private VPC in another region. 267 | 268 | - Add a new [CIDR block](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html#VPC_Sizing) 269 | for this VPC that doesn't overlap with the original one. 270 | 271 | - Don't attach an Internet gateway or NAT gateway to the new VPC. The 272 | new VPC will be private-only. 273 | 274 | - Update both VPC stacks to accept the netblock of the peering VPC as 275 | a parameter, so that you can... 276 | 277 | - add network ACLs in each VPC that allow all traffic in from the 278 | other VPC, and allow all traffic out from the source VPC. 279 | 280 | Create a separate stack that will create a 281 | [peering](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-basics.html) 282 | link between the 2 VPCs. 283 | 284 | - Create a [VPC Peering Connection](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpcpeeringconnection.html) 285 | from one to the other. 286 | 287 | - Add a route in each VPC that sends traffic for the other VPC's CIDR 288 | to that VPC. 289 | 290 | - The VPC IDs should be passed as stack parameters. 291 | 292 | #### Lab 4.2.2: EC2 across VPCs 293 | 294 | Create a new EC2 template similar to your original one, but without an 295 | Elastic IP. 296 | 297 | - Launch it in your new private VPC. 298 | 299 | ##### Question: Public to Private 300 | 301 | _Can you ping this instance from the public instance you created earlier?_ 302 | 303 | ##### Question: Private to Public 304 | 305 | _Can you ping your public instance from this private instance? Which IPs are 306 | reachable, the public instance's private IP or its public IP, or both?_ 307 | 308 | Use traceroute to see where traffic flows to both the public and private IPs. 309 | 310 | #### Lab 4.2.3: VPC Endpoint Gateway to S3 311 | 312 | VPC endpoints are something you'll see in practically all of our client 313 | engagements. It's really useful to know about them, but we realize the 314 | entire VPC topic is more time-consuming than most. 315 | 316 | Create a [VPC endpoint](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpcendpoint.html) 317 | connection from your private VPC to S3. 318 | 319 | - Add the VPC endpoint gateway to your private VPC's CFN template. 320 | Pass the S3 bucket name as a parameter so it can be included in 321 | the policy. 322 | 323 | - Rework your access controls a bit to accommodate using a VPC 324 | endpoint: 325 | 326 | - Change the egress NACL rules on the subnet where the endpoint is 327 | attached so that they allow all traffic (see "Network ACL rules" in 328 | [Troubleshoot Issues Connecting to S3 from VPC Endpoints](https://aws.amazon.com/premiumsupport/knowledge-center/connect-s3-vpc-endpoint/). 329 | 330 | - In the bucket's [policy document](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-s3.html#vpc-endpoints-policies-s3), 331 | grant access from your VPC and endpoint. 332 | 333 | - In the endpoint policy, grant access to the bucket you created in the S3 lesson. 334 | 335 | After you update the stack, make sure you can reach the bucket from the 336 | instance in your private VPC. 337 | 338 | _Note: Try this out, but don't get stalled out here.If you're not 339 | making good progress after a few hours, even with the help of others, 340 | document where you're at and what's not working for you, then move on. 341 | Even though this is a valuable foundation, we have more important things for 342 | you to learn._ 343 | 344 | ### Retrospective 4.2 345 | 346 | #### Question: Corporate Networks 347 | 348 | _How would you integrate your VPC with a corporate network?_ 349 | 350 | ## Further Reading 351 | 352 | - [VPN](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpn-connections.html) 353 | connections provide a way to connect to customer-premise networks. 354 | 355 | - [VPC Endpoints](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html) 356 | provide a way to connect VPC privately to many more Amazon 357 | services, protecting any of that service traffic from traversing 358 | the open Internet. 359 | 360 | - [Amazon VPC-to-Amazon VPC Connectivity Options](https://docs.aws.amazon.com/aws-technical-content/latest/aws-vpc-connectivity-options/amazon-vpc-to-amazon-vpc-connectivity-options.html) 361 | describes many more options and design patterns for using VPCs. 362 | 363 | - Jellili Adebello did a Sharing is Caring presentation about 364 | [Multiple VPC deployments with a pipeline](https://github.com/stelligent/multi-vpc-pipeline) 365 | on 2018-10-12. 366 | -------------------------------------------------------------------------------- /07-load-balancing/README.md: -------------------------------------------------------------------------------- 1 | # Topic 7: Load Balancers 2 | 3 | 4 | 5 | - [Topic 7: Load Balancers](#topic-7-load-balancers) 6 | - [Lesson 7.1: Introduction to Load Balancers](#lesson-71-introduction-to-load-balancers) 7 | - [Principle 7.1](#principle-71) 8 | - [Practice 7.1](#practice-71) 9 | - [Lab 7.1.1: ASG Basics](#lab-711-asg-basics) 10 | - [Question: Listeners and Target Groups](#question-listeners-and-target-groups) 11 | - [Lab 7.1.2: Health Checks](#lab-712-health-checks) 12 | - [Question: Health Checks](#question-health-checks) 13 | - [Question: ASG Behavior](#question-asg-behavior) 14 | - [Lab 7.1.3: Secure Sockets](#lab-713-secure-sockets) 15 | - [Question: SSL Policy](#question-ssl-policy) 16 | - [Question: Certificate Management](#question-certificate-management) 17 | - [Lab 7.1.4: Cleanup](#lab-714-cleanup) 18 | - [Retrospective 7.1](#retrospective-71) 19 | - [Further reading](#further-reading) 20 | 21 | 22 | 23 | ## Lesson 7.1: Introduction to Load Balancers 24 | 25 | ### Principle 7.1 26 | 27 | *Application Load Balancers are the best general-purpose service for 28 | distributing web traffic to many servers in multiple availability 29 | zones.* 30 | 31 | This section demonstrates the setup, management, and configuration you 32 | may run into in on an engagement concerning Load Balancers. While this 33 | is not an exhaustive look at LBs, this will give you the fundamentals to 34 | know where to start. 35 | 36 | ### Practice 7.1 37 | 38 | This section will get you familiar with the basic setup of an 39 | Application Load Balancer (ALB) and practical configurations you're 40 | likely to encounter in enterprises. 41 | 42 | #### Lab 7.1.1: ASG Basics 43 | 44 | In Topic 6, you created Auto Scaling Groups (ASGs) with a number of 45 | instances. In this lab, we'll take an ASG with instances that serve a 46 | specific web page on port 80 with HTTP and balance traffic across the 47 | many instances with an ALB. 48 | 49 | - Working from the [ASG Template](https://github.com/stelligent/stelligent-u/blob/master/07-load-balancing/asg_example.yaml), 50 | associate a [target group](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticloadbalancingv2-targetgroup.html) 51 | with the autoscaling group, giving it a health check on `/index.html`. 52 | - Use Amazon Linux 2 AMI. 53 | 54 | - Create an internet-facing ALB 55 | 56 | - Create an ALB Listener that references the previously created target 57 | group and ALB. 58 | 59 | - Update the AutoScalingGroup to use 'ELB' for HealthCheckType and 30 60 | for HealthCheckGracePeriod 61 | 62 | - Once created, go to the endpoint associated with the Load Balancer. 63 | 64 | ##### Question: Listeners and Target Groups 65 | 66 | _What is the benefit of breaking up the load balancer into specific listeners 67 | and target groups?_ 68 | 69 | #### Lab 7.1.2: Health Checks 70 | 71 | Now, let's update our health check to see what happens when things go 72 | haywire! 73 | 74 | - Modify the target group: 75 | 76 | - Update the health check value to be `/BADindex.html` 77 | - Change the interval to be 20 78 | - Change the healthy threshold to 2 79 | - Change the unhealthy threshold to 3 80 | - Create a target group attribute with key 81 | `deregistration_delay.timeout_seconds`, value 20 82 | 83 | - Wait about two minutes after the stack completes. 84 | 85 | - Go to your load balancer endpoint. 86 | 87 | ##### Question: Health Checks 88 | 89 | _What can be controlled with the interval/healthy threshold/unhealthy threshold 90 | settings?_ 91 | 92 | ##### Question: ASG Behavior 93 | 94 | _What's happening to the instances in the ASG? How do you know?_ 95 | 96 | #### Lab 7.1.3: Secure Sockets 97 | 98 | Let's fix that bad health check endpoint and add an https listener. 99 | 100 | - First, fix your health check and verify everything is working 101 | smoothly. 102 | 103 | - [Create a self-signed certificate locally](https://www.ibm.com/support/knowledgecenter/SSMNED_5.0.0/com.ibm.apic.cmc.doc/task_apionprem_gernerate_self_signed_openSSL.html) 104 | 105 | - Via the aws acm CLI or AWS Certificate Manager console, import your 106 | newly created certificate, make note of its ARN. 107 | 108 | - Add a new listener to your previously created load balancer using 109 | HTTPS on port 443 and referencing your newly uploaded certificate. 110 | 111 | - Let's be extra secure and specify a [security policy](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies) 112 | on that listener which requires Forward Secrecy (has FS in its 113 | name). 114 | 115 | - Visit your ALB endpoint, add the security exception and enjoy your 116 | encrypted session. 117 | 118 | ##### Question: SSL Policy 119 | 120 | _What is the trade off of going with a more secure SSL policy?_ 121 | 122 | ##### Question: Certificate Management 123 | 124 | _We imported a local certificate into ACM, what other options do you have? How 125 | do those processes work?_ 126 | 127 | #### Lab 7.1.4: Cleanup 128 | 129 | - [Load balancers are expensive](https://aws.amazon.com/elasticloadbalancing/pricing/) 130 | so delete your stack. 131 | 132 | - Delete your imported self-signed cert. 133 | 134 | ### Retrospective 7.1 135 | 136 | Discuss with your mentor: *What are some of the common cloud architectures 137 | where you would want to implement an ALB?* 138 | 139 | ## Further reading 140 | 141 | - [Run Containerized Microservices with Amazon ECS and Application Load Balancer](https://aws.amazon.com/blogs/compute/microservice-delivery-with-amazon-ecs-and-application-load-balancers/) 142 | 143 | - [Target Groups and Sticky Sessions](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html) 144 | 145 | - [Using path-based routing.](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/tutorial-load-balancer-routing.html) 146 | -------------------------------------------------------------------------------- /07-load-balancing/asg_example.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Resources: 3 | WebServerAsg: 4 | Type: 'AWS::AutoScaling::AutoScalingGroup' 5 | Properties: 6 | AutoScalingGroupName: 'AutomationBoxes' 7 | VPCZoneIdentifier: 8 | - 'REPLACE_WITH_PUBLIC_SUBNET_IN_AZ1' # Find public subnets within the default VPC or 9 | - 'REPLACE_WITH_PUBLIC_SUBNET_IN_AZ2' 10 | DesiredCapacity: '3' 11 | HealthCheckType: 'ELB' 12 | HealthCheckGracePeriod: 30 13 | LaunchConfigurationName: !Ref WebServersLC 14 | MaxSize: '3' 15 | MinSize: '3' 16 | 17 | WebServersLC: 18 | Type: 'AWS::AutoScaling::LaunchConfiguration' 19 | Properties: 20 | ImageId: 'REPLACE_WITH_AMAZON_LINUXV1_AMI' 21 | InstanceType: 't2.micro' 22 | LaunchConfigurationName: 'SimpleWebServerLC' 23 | SecurityGroups: 24 | - 'REPLACE' 25 | UserData: 26 | Fn::Base64: 27 | Fn::Sub: | 28 | #!/bin/bash -xe 29 | 30 | yum install -y aws-cfn-bootstrap 31 | 32 | # Install the files and packages from the metadata 33 | /opt/aws/bin/cfn-init -v \ 34 | --stack ${AWS::StackName} \ 35 | --resource WebServersLC \ 36 | --configsets All \ 37 | --region ${AWS::Region} 38 | 39 | # Signal the status from cfn-init 40 | /opt/aws/bin/cfn-signal -e $? \ 41 | --stack ${AWS::StackName} \ 42 | --resource WebServersLC \ 43 | --region ${AWS::Region} 44 | 45 | Metadata: 46 | 'AWS::CloudFormation::Init': 47 | configSets: 48 | All: 49 | - ConfigureStelligentProject 50 | ConfigureStelligentProject: 51 | packages: 52 | yum: 53 | nginx: [] 54 | files: 55 | /usr/share/nginx/html/index.html: 56 | content: '

Automation for the People

' 57 | mode: '000644' 58 | owner: root 59 | group: root 60 | services: 61 | sysvinit: 62 | nginx: 63 | enabled: 'true' 64 | ensureRunning: 'true' 65 | -------------------------------------------------------------------------------- /08-cloudwatch-logs/8.1.2.yml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: "2010-09-09" 2 | Parameters: 3 | UbuntuAMI: 4 | Type: "AWS::SSM::Parameter::Value" 5 | Default: '/aws/service/canonical/ubuntu/server/20.04/stable/current/amd64/hvm/ebs-gp2/ami-id' 6 | VPCID: 7 | Type: 'AWS::EC2::VPC::Id' 8 | SubnetID: 9 | Type: 'AWS::EC2::Subnet::Id' 10 | Resources: 11 | LaunchTemplate: 12 | Type: AWS::EC2::LaunchTemplate 13 | Properties: 14 | LaunchTemplateName: EC2LaunchTemplate 15 | LaunchTemplateData: 16 | InstanceType: t3.nano 17 | ImageId: !Ref UbuntuAMI 18 | IamInstanceProfile: 19 | Name: !Ref InstanceProfile 20 | NetworkInterfaces: 21 | - DeviceIndex: 0 22 | AssociatePublicIpAddress: True 23 | SubnetId: !Ref SubnetID 24 | UserData: 25 | Fn::Base64: !Sub | 26 | #!/bin/bash -xe 27 | /usr/bin/apt-get update -y 28 | /usr/bin/apt-get install -y python3-pip 29 | python3 -m pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-2.0-9.tar.gz 30 | /usr/local/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2 --configsets default --region ${AWS::Region} 31 | /usr/local/bin/cfn-signal --exit-code $? --stack ${AWS::StackName} --resource EC2 --region ${AWS::Region} 32 | EC2: 33 | Type: AWS::EC2::Instance 34 | Properties: 35 | LaunchTemplate: 36 | LaunchTemplateId: !Ref LaunchTemplate 37 | Version: "1" 38 | CreationPolicy: 39 | ResourceSignal: 40 | Timeout: PT5M 41 | Count: '1' 42 | Metadata: 43 | AWS::CloudFormation::Init: 44 | configSets: 45 | default: 46 | - install 47 | - config 48 | - launch 49 | install: 50 | sources: 51 | /etc/cloudwatchagent: https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip 52 | commands: 53 | installCloudwatch: 54 | command: ./install.sh 55 | cwd: /etc/cloudwatchagent 56 | config: 57 | files: 58 | /opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json: 59 | # Add your configuration file here 60 | launch: 61 | commands: 62 | launchCloudwatch: 63 | command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json -s 64 | InstanceProfile: 65 | Type: AWS::IAM::InstanceProfile 66 | Properties: 67 | Roles: 68 | - !Ref RoleCloudWatch 69 | RoleCloudWatch: 70 | Type: AWS::IAM::Role 71 | Properties: 72 | ManagedPolicyArns: 73 | - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy 74 | - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM 75 | AssumeRolePolicyDocument: 76 | Version: "2012-10-17" 77 | Statement: 78 | - 79 | Effect: "Allow" 80 | Principal: 81 | Service: 82 | - "ec2.amazonaws.com" 83 | Action: 84 | - "sts:AssumeRole" 85 | SecurityGroup: 86 | Type: AWS::EC2::SecurityGroup 87 | Properties: 88 | GroupDescription: 'Stelligent-U 08 CloudWatch' 89 | VpcId: !Ref VPCID 90 | -------------------------------------------------------------------------------- /08-cloudwatch-logs/README.md: -------------------------------------------------------------------------------- 1 | # Topic 8: CloudWatch 2 | 3 | 4 | 5 | - [Topic 8: CloudWatch](#topic-8-cloudwatch) 6 | - [Conventions](#conventions) 7 | - [Lesson 8.1: CloudWatch Logs storage and retrieval](#lesson-81-cloudwatch-logs-storage-and-retrieval) 8 | - [Principle 8.1](#principle-81) 9 | - [Practice 8.1](#practice-81) 10 | - [Lab 8.1.1: Log groups and streams](#lab-811-log-groups-and-streams) 11 | - [Lab 8.1.2: The CloudWatch agent](#lab-812-the-cloudwatch-agent) 12 | - [Lab 8.1.3: 3rd party tool awslogs](#lab-813-3rd-party-tool-awslogs) 13 | - [Lab 8.1.4: CloudWatch logs lifecycle](#lab-814-cloudwatch-logs-lifecycle) 14 | - [Lab 8.1.5: Clean up](#lab-815-clean-up) 15 | - [Retrospective 8.1](#retrospective-81) 16 | - [Lesson 8.2: CloudWatch Logs with CloudTrail events](#lesson-82-cloudwatch-logs-with-cloudtrail-events) 17 | - [Principle 8.2](#principle-82) 18 | - [Practice 8.2](#practice-82) 19 | - [Lab 8.2.1: CloudWatch and CloudTrail resources](#lab-821-cloudwatch-and-cloudtrail-resources) 20 | - [Lab 8.2.2: Logging AWS infrastructure changes](#lab-822-logging-aws-infrastructure-changes) 21 | - [Lab 8.2.3: Clean up](#lab-823-clean-up) 22 | - [Retrospective 8.2](#retrospective-82) 23 | - [Question](#question) 24 | - [Task](#task) 25 | 26 | 27 | 28 | ## Conventions 29 | 30 | - DO review CloudFormation documentation to see if a property is 31 | required when creating a resource. 32 | 33 | ## Lesson 8.1: CloudWatch Logs storage and retrieval 34 | 35 | ### Principle 8.1 36 | 37 | CloudWatch Logs are the best way to securely and reliably store text 38 | logs from application services and AWS resources (EC2, Lambda, 39 | CodePipeline, etc) over time. 40 | 41 | ### Practice 8.1 42 | 43 | This section shows you how to configure CloudWatch to monitor and store 44 | logs for AWS resources, as well as how to retrieve and review those logs 45 | using the AWS CLI and a utility called "awslogs". 46 | 47 | #### Lab 8.1.1: Log groups and streams 48 | 49 | A log group is an arbitrary collection of similar logs, using whatever 50 | definition of "similar" you want. A log stream is a uniquely 51 | identifiable flow of data into that group. Use the AWS CLI to create a 52 | log group and log stream: 53 | 54 | - Name the log group based on your username: *first.last*.c9logs 55 | 56 | - Name the log stream named c9.training in your log group. 57 | 58 | When you're done, list the log groups and the log streams to confirm 59 | they exist. 60 | 61 | #### Lab 8.1.2: The CloudWatch agent 62 | 63 | The CloudWatch agent is the standard tool for sending log data to 64 | CloudWatch Logs. We've provided a stack template for you in your *clone* 65 | of the 66 | [stelligent-u](https://github.com/stelligent/stelligent-u) 67 | repo: 68 | 69 | - [Documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-on-first-instance.html) 70 | for installing the Cloud Watch agent, for reference. 71 | The example template installs the agent. 72 | 73 | - We need to generate a Cloud Watch configuration file to be included 74 | in your Cloud Formation Template. The simplest way to approach this 75 | is to start an EC2 instance with the Cloud Watch agent installed and 76 | use the wizard it provides. For the example Cloud Formation template 77 | the wizard is in 78 | `/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard` 79 | You will need to add references to the log streams defined in 8.1.1 80 | [Documentation on generating the template file](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file.html) 81 | for reference. 82 | 83 | - The wizard will prompt you to use `collectd`, but we do not recommend this 84 | as it can cause the agent to fail to start 85 | 86 | - Modify the template mappings to reference your 87 | own VPC ID's and Subnet ID generated in other lessons, 88 | or provide appropriate code in the resources section. 89 | 90 | - Once you have added the Cloud Watch configuration to your Cloud Formation template, 91 | delete the running stack, and relaunch. 92 | 93 | - Use the AWS CLI to display the log events for your group and stream from 8.1.1. 94 | 95 | > *Note:* logs may take several minutes to appear. 96 | 97 | #### Lab 8.1.3: 3rd party tool awslogs 98 | 99 | [awslogs](https://github.com/jorgebastida/awslogs) is a 100 | publicly-available Python tool that you can use to read CloudWatch logs. 101 | It's especially convenient for tailing the log streams, showing you data 102 | as it arrives. 103 | 104 | - Install the awslogs client on your running EC2 instance. 105 | 106 | - Use it to watch logs as they are put into your log group. 107 | 108 | - Use awslogs to get logs from your group from the last 5 minutes, 109 | last 20 minutes and last hour. 110 | 111 | #### Lab 8.1.4: CloudWatch logs lifecycle 112 | 113 | Any time you're logging information, it's important to consider the 114 | lifecycle of the logs. 115 | 116 | - Use the AWS CLI to [set the retention policy](https://docs.aws.amazon.com/cli/latest/reference/logs/put-retention-policy.htm) 117 | of your log group to 60 days. 118 | 119 | - Use the CLI to review the policy in your log group. 120 | 121 | - Set the retention policy to the maximum allowed time, and review the 122 | change again to double-check. 123 | 124 | #### Lab 8.1.5: Clean up 125 | 126 | You can tear down your EC2 stack at this point. 127 | 128 | Use the AWS CLI to remove the log group and log stream you created 129 | earlier. 130 | 131 | You'll need [jorgebastida/awslogs](https://github.com/jorgebastida/awslogs) 132 | in Lesson 8.2.1, so now's a good time to install it on your laptop. You may 133 | find that it's handy for client engagements and future lab work as well. 134 | 135 | ### Retrospective 8.1 136 | 137 | *Log retention is an important issue that can affect many parts of a 138 | company's business. It's helpful to know what CloudWatch Log's service 139 | limitations are.* 140 | 141 | - What are the minimum and maximum retention times? 142 | 143 | - Instead of keeping data in CW Logs forever, can you do anything else 144 | with them? What might a useful lifecycle for logs look like? 145 | 146 | ## Lesson 8.2: CloudWatch Logs with CloudTrail events 147 | 148 | ### Principle 8.2 149 | 150 | *CloudWatch Logs let you monitor AWS API changes via CloudTrail logged 151 | events.* 152 | 153 | ### Practice 8.2 154 | 155 | This section demonstrates CloudWatch's ability to send alerts based on 156 | changes to AWS resources made via the API changes, identified through 157 | CloudTrail events. This is useful for many reasons. For example, you may 158 | want to understand what changes are being made to AWS resources and 159 | decide if they are appropriate. Notifications or automated corrective 160 | action can be configured when inappropriate changes are being made. 161 | 162 | #### Lab 8.2.1: CloudWatch and CloudTrail resources 163 | 164 | Let's switch from the awscli to CloudFormation. Create a template that 165 | provides the following in a single stack: 166 | 167 | - A new CloudWatch Logs log group 168 | 169 | - An S3 bucket for CloudTrail to publish logs. 170 | 171 | - A CloudTrail trail that uses the CloudWatch log group. 172 | 173 | #### Lab 8.2.2: Logging AWS infrastructure changes 174 | 175 | Now that you have your logging infrastructure, create a separate stack 176 | for the resources that will use it: 177 | 178 | - Create an S3 bucket or any other AWS resource of your choice. 179 | 180 | - Add tags that mark it with your AWS username, and identify it as a 181 | stelligent-u resource with this topic and lab number. 182 | 183 | - Use awslogs client utility to review the logs from the new group. 184 | You should see the activity from creating and changing the 185 | resource. 186 | 187 | - Delete the CloudFormation stack and resources. 188 | 189 | - Use the awslogs utility again to view those changes. 190 | 191 | #### Lab 8.2.3: Clean up 192 | 193 | - Delete any stacks that you made for this topic. 194 | 195 | - Make sure you keep all of the CloudFormation templates from this 196 | lesson in your GitHub repo. 197 | 198 | ### Retrospective 8.2 199 | 200 | #### Question 201 | 202 | _What type of events might be important to track in an AWS account? If 203 | you were automating mitigating actions for the events, what might they 204 | be and what AWS resource(s) would you use?_ 205 | 206 | #### Task 207 | 208 | Dig out the CloudFormation template you used to create the CloudTrail 209 | trail in lab 8.2.1. Add a CloudWatch event, SNS topic and SNS 210 | subscription that will email you when any changes to EC2 instances are 211 | made. Test this mechanism by creating and modifying new EC2 instances. 212 | Make sure to clean up the CloudFormation stacks and any other resources 213 | when you are done. 214 | -------------------------------------------------------------------------------- /09-lambda/README.md: -------------------------------------------------------------------------------- 1 | # Topic 9: Lambda 2 | 3 | 4 | 5 | - [Topic 9: Lambda](#topic-9-lambda) 6 | - [Conventions](#conventions) 7 | - [Lesson 9.1: Lambda is a fully-managed compute resource](#lesson-91-lambda-is-a-fully-managed-compute-resource) 8 | - [Principle 9.1](#principle-91) 9 | - [Practice 9.1](#practice-91) 10 | - [Lab 9.1.1: Simple Lambda function](#lab-911-simple-lambda-function) 11 | - [Lab 9.1.2: Lambda behind API Gateway](#lab-912-lambda-behind-api-gateway) 12 | - [Lab 9.1.3: Lambda & CloudFormation with awscli](#lab-913-lambda--cloudformation-with-awscli) 13 | - [Retrospective 9.1](#retrospective-91) 14 | - [Task](#task) 15 | - [Lesson 9.2: Lambda and other AWS resources](#lesson-92-lambda-and-other-aws-resources) 16 | - [Principle 9.2](#principle-92) 17 | - [Practice 9.2](#practice-92) 18 | - [Lab 9.2.1: Lambda with DynamoDB](#lab-921-lambda-with-dynamodb) 19 | - [Lab 9.2.2: Lambda via CloudWatch Rules](#lab-922-lambda-via-cloudwatch-rules) 20 | - [Lab 9.2.3: Query data with Lambda and API Gateway](#lab-923-query-data-with-lambda-and-api-gateway) 21 | - [Retrospective 9.2](#retrospective-92) 22 | - [Question](#question) 23 | - [Further Reading](#further-reading) 24 | 25 | 26 | 27 | ## Conventions 28 | 29 | - DO use the [AWS Lambda documentation](https://aws.amazon.com/documentation/lambda/), 30 | [DynamoDB documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html), 31 | and [ApiGateway documentation.](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-method.html) 32 | 33 | - DO continuously commit all your templates and code to the topic 34 | folder in your GitHub repo. 35 | 36 | ## Lesson 9.1: Lambda is a fully-managed compute resource 37 | 38 | ### Principle 9.1 39 | 40 | *As a fully managed compute resource, Lambda can reduce all the time and 41 | effort to configure and maintain virtual servers.* 42 | 43 | Lambda gives you a fully-managed compute resource running in AWS that 44 | allows you to run your code on demand with very little configuration 45 | needed and no maintenance required. AWS also provides robust testing 46 | tools via Cloud9, and methods of packaging and deploying your code 47 | easily. Lambda can be paired with API gateway to quickly create 48 | microservices. 49 | 50 | API Gateway is a fully managed service that makes it easy to create, 51 | publish, maintain, monitor, and secure APIs at any scale. You can create 52 | an API that acts as a "front door" for applications to access data or 53 | functionality from your back-end services, such as workloads running on 54 | EC2, code running on AWS Lambda, or any web application. 55 | 56 | ### Practice 9.1 57 | 58 | #### Lab 9.1.1: Simple Lambda function 59 | 60 | Create and test a simple AWS Lambda function using the Lambda console. 61 | 62 | - Use the wizard to create a new Lambda using your choice of language. 63 | 64 | - Update the lambda to return "Hello AWS!" and use the "Test" tool to 65 | run a test. 66 | 67 | - Review the options you have for testing and running Lambdas. 68 | 69 | - When you're done, delete the Lambda. 70 | 71 | #### Lab 9.1.2: Lambda behind API Gateway 72 | 73 | Using API gateway to run a Lambda function. 74 | 75 | - Use CloudFormation to create the same lambda as you did in lab 1. 76 | Use the in-line code feature to write the same "Hello AWS!" 77 | function. 78 | 79 | - Add an AWS API gateway to your template. Configure the gateway so 80 | that it will call the lambda function. You will need to implement: 81 | 82 | - `AWS::ApiGateway::Method` 83 | - `AWS::ApiGateway::RestApi` 84 | - `AWS::ApiGateway::Deployment` 85 | - Lambda execution role (`AWS::IAM::Role`) with an AssumeRole policy 86 | - Appropriate Lambda invoke permissions (`AWS::Lambda::Permission`) 87 | 88 | - Use the AWS CLI to call the API gateway which will call your Lambda 89 | function. 90 | 91 | - Lambdas can take a payload like JSON as input. Rewrite the function 92 | to take a JSON payload and simply return the payload, or an item 93 | from the payload. 94 | 95 | #### Lab 9.1.3: Lambda & CloudFormation with awscli 96 | 97 | Use the AWS CLI to create Lambda functions: 98 | 99 | - Using the template you created in lab 2, move the in-line code to a 100 | separate file and update the Lambda resource to reference the 101 | handler. 102 | 103 | - Use the "aws cloudformation package" and "\... deploy" commands to 104 | create the CloudFormation stack. Note: The "package" command will 105 | need an S3 bucket to temporarily store the deployment package. 106 | 107 | - Use the API gateway to make a test call to the lambda to confirm 108 | it's working. 109 | 110 | ### Retrospective 9.1 111 | 112 | #### Task 113 | 114 | Review other methods of creating microservices using API Gateway and 115 | Lambda such as [Chalice](https://github.com/aws/chalice) (Python), 116 | [Claudia.js](https://claudiajs.com/tutorials/index.html) (Node) 117 | and [Aegis](https://github.com/tmaiaroto/aegis) (Go). 118 | Understand how you can use [AWS SAM](https://github.com/awslabs/serverless-application-model) 119 | to test and deploy Lambdas. 120 | 121 | ## Lesson 9.2: Lambda and other AWS resources 122 | 123 | ### Principle 9.2 124 | 125 | *Lambda can interact with other AWS services when using the appropriate 126 | execution policies.* 127 | 128 | Coupling Lambda with native AWS services allows you to create powerful 129 | code very quickly. Lambda can even be used with CloudWatch events to 130 | perform complex log analysis or react with automated mitigation 131 | functionality. 132 | 133 | ### Practice 9.2 134 | 135 | #### Lab 9.2.1: Lambda with DynamoDB 136 | 137 | As a simple example, let's extend your Lambda function to write data to 138 | a table in DynamoDB: 139 | 140 | - Start with the template and code you created in lab 2 141 | 142 | - Add a DynamoDB table with several attributes of your choice 143 | 144 | - Update the Lambda code to take input based on the attributes and 145 | insert new items into the DynamoDB table. 146 | 147 | Test the code using an API call as you've done before. Confirm that the 148 | call is inserting the item in the table. 149 | 150 | #### Lab 9.2.2: Lambda via CloudWatch Rules 151 | 152 | CloudWatch rules can be used to call Lambda functions based on events. 153 | 154 | - Add a CloudWatch rule to the template which targets your Lambda 155 | function when the S3 PutObject operation is called. If a trail 156 | doesn't exist for this, you may need to create one. 157 | 158 | - Modify your Lambda handler to log some of the event data to the 159 | DynamoDB. 160 | 161 | - Create an S3 bucket and test that the Lambda logs event data to the 162 | DB. 163 | 164 | #### Lab 9.2.3: Query data with Lambda and API Gateway 165 | 166 | Write another Lambda function that will query the DynamoDB table: 167 | 168 | - The function should take a pattern (for instance, a bucket name) and 169 | return all events for that pattern. 170 | 171 | - Add an API gateway that calls the Lambda to query the data. 172 | 173 | ### Retrospective 9.2 174 | 175 | #### Question 176 | 177 | *Can you think of practical ways an organization can use Lambda in 178 | reaction to AWS resource changes?* 179 | 180 | ## Further Reading 181 | 182 | - Read about [Capital One's Cloud Custodian project](https://stelligent.com/2017/05/15/cloud-custodian-cleans-up-your-cloud-clutter/) 183 | and see how it uses AWS Lambda. 184 | -------------------------------------------------------------------------------- /10-kms/README.md: -------------------------------------------------------------------------------- 1 | # Topic 10: Key Management Service (KMS) 2 | 3 | 4 | 5 | - [Topic 10: Key Management Service (KMS)](#topic-10-key-management-service-kms) 6 | - [Lesson 10.1: Introduction to KMS](#lesson-101-introduction-to-kms) 7 | - [Principle 10.1](#principle-101) 8 | - [Practice 10.1](#practice-101) 9 | - [Lab 10.1.1: Create a KMS CMK](#lab-1011-create-a-kms-cmk) 10 | - [Lab 10.1.2 : Create a KMS Alias](#lab-1012--create-a-kms-alias) 11 | - [Lab 10.1.3: Encrypt a text file with your KMS CMK](#lab-1013-encrypt-a-text-file-with-your-kms-cmk) 12 | - [Lab 10.1.4: Decrypt a ciphertext file](#lab-1014-decrypt-a-ciphertext-file) 13 | - [Retrospective 10.1](#retrospective-101) 14 | - [Question: Decrypting the Ciphertext File](#question-decrypting-the-ciphertext-file) 15 | - [Question: KMS Alias](#question-kms-alias) 16 | - [Lesson 10.2: Implementation of KMS Keys in S3](#lesson-102-implementation-of-kms-keys-in-s3) 17 | - [Principle 10.2](#principle-102) 18 | - [Practice 10.2](#practice-102) 19 | - [Lab 10.2.1: Client Side Encryption of S3 Object](#lab-1021-client-side-encryption-of-s3-object) 20 | - [Lab 10.2.2: Delete your CMK](#lab-1022-delete-your-cmk) 21 | - [Question: CMK](#question-cmk) 22 | - [Retrospective 10.2](#retrospective-102) 23 | 24 | 25 | 26 | ## Lesson 10.1: Introduction to KMS 27 | 28 | ### Principle 10.1 29 | 30 | *AWS Key Management Service (AWS KMS) is a managed service that makes it 31 | easy for you to create and control the encryption keys used to encrypt 32 | your data.* 33 | 34 | ### Practice 10.1 35 | 36 | Rather than storing the encryption keys ourselves, Amazon securely 37 | stores them and provides the ability to disperse and use keys to others 38 | via IAM. The following labs will introduce you to the fundamental resources in 39 | KMS: KMS Customer Master Keys (CMKs) and KMS Aliases. 40 | 41 | #### Lab 10.1.1: Create a KMS CMK 42 | 43 | Create a CFN Template that 44 | [creates a CMK key in KMS](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kms-key.html): 45 | 46 | - For a key policy, set your IAM user as Key Administrator and as a Key User 47 | 48 | #### Lab 10.1.2 : Create a KMS Alias 49 | 50 | Update your CFN template to add a KMS Alias with a snazzy name. 51 | Associate your CMK with this alias. 52 | 53 | #### Lab 10.1.3: Encrypt a text file with your KMS CMK 54 | 55 | Use the AWS KMS CLI to encrypt a plaintext file with a secret message 56 | (maybe that combo to the safe, or your luggage password). Send your file 57 | to a colleague with administrator access. 58 | 59 | #### Lab 10.1.4: Decrypt a ciphertext file 60 | 61 | Use the KMS CLI to now decrypt a ciphertext file. 62 | 63 | ### Retrospective 10.1 64 | 65 | #### Question: Decrypting the Ciphertext File 66 | 67 | _For decrypting the ciphertext file, why didn't you have to specify a key? How 68 | did you have permission to decrypt?_ 69 | 70 | #### Question: KMS Alias 71 | 72 | _Why is it beneficial to use a KMS Alias?_ 73 | 74 | ## Lesson 10.2: Implementation of KMS Keys in S3 75 | 76 | ### Principle 10.2 77 | 78 | For AWS S3, you've seen in a previous lab about setting a KMS key to be 79 | used with Server Side Encryption. Additionally, a KMS key can be used 80 | for client side encryption of S3 assets. Client side encryption adds an 81 | extra layer of security by encrypting information before transmission to 82 | S3. 83 | 84 | ### Practice 10.2 85 | 86 | Amazon has integrated KMS into many of their services (check out the 87 | full list: 88 | [https://docs.aws.amazon.com/kms/latest/developerguide/service-integration.html](https://docs.aws.amazon.com/kms/latest/developerguide/service-integration.html) 89 | ). In the following labs, we'll take your existing CMK and begin using 90 | them for a practical purposes: client side encryption of S3 objects. 91 | 92 | Ruby SDK for Encryption client: 93 | [https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/EncryptionV2/Client.html](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/EncryptionV2/Client.html) 94 | 95 | #### Lab 10.2.1: Client Side Encryption of S3 Object 96 | 97 | Use the ruby-sdk to create a script that will: 98 | 99 | - Encrypt a local plaintext file and upload to S3 100 | 101 | - Read back the encrypted ciphertext from the uploaded file 102 | 103 | - Pull down and decrypt the file, saving as another name. 104 | 105 | #### Lab 10.2.2: Delete your CMK 106 | 107 | Delete your KMS CFN Stack. 108 | 109 | ##### Question: CMK 110 | 111 | _What happened to your CMK? Why?_ 112 | 113 | ### Retrospective 10.2 114 | 115 | Check out the code for [stelligent/crossing](https://github.com/stelligent/crossing) 116 | and [stelligent/keystore](https://github.com/stelligent/keystore) 117 | on GitHub for tools that simplify using KMS encrypted resources. 118 | -------------------------------------------------------------------------------- /11-parameter-store/README.md: -------------------------------------------------------------------------------- 1 | # Topic 11: Parameter Store 2 | 3 | 4 | 5 | - [Topic 11: Parameter Store](#topic-11-parameter-store) 6 | - [Guidance](#guidance) 7 | - [Lesson 11.1: Introduction to Parameter Store](#lesson-111-introduction-to-parameter-store) 8 | - [Principle 11.1](#principle-111) 9 | - [Practice 11.1](#practice-111) 10 | - [Lab 11.1.1: Storing data](#lab-1111-storing-data) 11 | - [Lab 11.1.2: Reading data](#lab-1112-reading-data) 12 | - [Question: Finding Parameter Resources](#question-finding-parameter-resources) 13 | - [Lab 11.1.3: Integration with CloudFormation](#lab-1113-integration-with-cloudformation) 14 | - [Lab 11.1.4: Secure Strings](#lab-1114-secure-strings) 15 | - [Retrospective 11.1](#retrospective-111) 16 | - [Question 1](#question-1) 17 | - [Question 2](#question-2) 18 | - [Further Reading](#further-reading) 19 | 20 | 21 | 22 | ## Guidance 23 | 24 | - Explore the official docs! See the the Parameter Store [User Guide](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html), 25 | [API Reference](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameters.html), 26 | [CLI Reference](https://docs.aws.amazon.com/cli/latest/reference/ssm/index.html), 27 | and 28 | [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 29 | docs. 30 | 31 | - Avoid using other sites like stackoverflow.com for answers \-- part 32 | of the skill set you're building is finding answers straight from 33 | the source, AWS. 34 | 35 | - Explore your curiosity. Try to understand why things work the way 36 | they do. Read more of the documentation than just what you need to 37 | find the answers. 38 | 39 | ## Lesson 11.1: Introduction to Parameter Store 40 | 41 | ### Principle 11.1 42 | 43 | *Parameter Store is a key-value store that is compelling and powerful in 44 | its simplicity.* 45 | 46 | ### Practice 11.1 47 | 48 | Let's start out by creating a few parameters and finding ways we can 49 | read them. In the labs below, keep a few things in mind: 50 | 51 | - Parameter Store keys are naturally split by `/` into a hierarchy (but 52 | [it's not necessary](https://aws.amazon.com/blogs/mt/organize-parameters-by-hierarchy-tags-or-amazon-cloudwatch-events-with-amazon-ec2-systems-manager-parameter-store/). 53 | You can use this to namespace your keys. 54 | 55 | - Because we generally work in shared accounts, namespacing is important. 56 | 57 | - Parameter Store data is unique per region, but not per availability 58 | zone, and it would be easy to make your keys overlap with somebody 59 | else's if you're all working in the same region in the same time 60 | period. 61 | 62 | - To establish a convention where we avoid overlapping with other's 63 | work, prefix your keys with 64 | `/*your-aws-username*/stelligent-u/lab11/`. 65 | 66 | #### Lab 11.1.1: Storing data 67 | 68 | Create a new CloudFormation stack that can store a handful 69 | of data points for an engineer in Parameter Store(e.g., Name, Title, Address, 70 | etc.). Make it generic, so that each data point is passed as a Stack parameter, 71 | as if the stack template might be used to store info about each engineer. 72 | 73 | - Create a sub-tree of 74 | [AWS::SSM::Parameter](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 75 | resources that will store information on a single engineer. 76 | 77 | - Every Parameter Store value type can be a string. 78 | 79 | - At the top of the sub-tree, use a name for the engineer as your key 80 | value. See the [Name constraints](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_PutParameter.html#systemsmanager-PutParameter-request-Name) 81 | in the PutParameter API docs for limitations. Try to enforce some 82 | of those limitations in your stack's [parameter properties](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html), 83 | too. (E.g. set an AllowedPattern and MaxLength. Can you set any 84 | other limitations?). 85 | 86 | - Store all other keys under that. 87 | 88 | - Store the engineer's team name under "team". 89 | 90 | - Store their timezone under "timezone". 91 | 92 | - Store their home state as a 2-letter prefix under "state". 93 | 94 | - Store their start date under "start-date". 95 | 96 | Launch the stack with a parameter file that sets information for 97 | yourself. 98 | 99 | #### Lab 11.1.2: Reading data 100 | 101 | Look at ways to read your parameter data. 102 | 103 | - You can look it up in the 104 | [console](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Parameters:) 105 | 106 | - You can read it with the API using 107 | [GetParameter](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameter.html), 108 | [GetParameters](https://boto3.readthedocs.io/en/latest/reference/services/ssm.html#SSM.Client.get_parameters) 109 | and 110 | [GetParametersByPath](https://boto3.readthedocs.io/en/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path) 111 | 112 | - You can invoke those same API queries through the 113 | [aws ssm](https://docs.aws.amazon.com/cli/latest/reference/ssm/index.html) 114 | CLI 115 | 116 | Use all 3 methods to read your parameters individually and to fetch the 117 | entire subtree with a single query. 118 | 119 | ##### Question: Finding Parameter Resources 120 | 121 | _When you look at your stack in the CloudFormation console, can you find 122 | the values of your parameter resources there?_ 123 | 124 | #### Lab 11.1.3: Integration with CloudFormation 125 | 126 | CloudFormation can use Parameter Store keys and values as 127 | [stack parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types). 128 | This gives us an incredibly convenient way to maintain stack parameters 129 | without the hassle of updating JSON files and keeping them in sync 130 | through S3 or git repositories. 131 | 132 | Make a copy of the original EC2 stack from the Load Balancing topic. 133 | Instead of creating a web server with a page that says "Automation for 134 | the People", make one that describes an engineer. 135 | 136 | - One parameter should be an "AWS::SSM::Parameter::Name" that will 137 | reference the top-level key of your hierarchy as set in your stack 138 | from lab 1. 139 | 140 | - Accept an "AWS::SSM::Parameter::Value" type for each of the Param 141 | Store values that you set in your stack from lab 1. 142 | 143 | - Show the value of each of those stack parameters in the web page 144 | that nginx serves. 145 | 146 | When you launch your stack, make sure the web page shows the values of 147 | all the keys that you set in lab 1. 148 | 149 | As follow-up to the question above about finding the values of your 150 | parameter resources, take note of a comment in the SSM Parameter Types 151 | document linked in the previous paragraph: 152 | 153 | > You can see the resolved values for SSM parameters on the stack's 154 | > Parameters tab in the console, or by running 155 | > [describe-stacks](http://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stacks.html) 156 | > or 157 | > [describe-change-set](http://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-change-set.html). 158 | > These are the values that are currently used in the stack definition 159 | > for the corresponding Systems Manager parameter keys. Note that these 160 | > values are set when the stack is created or updated, so they might 161 | > differ from the latest values in Parameter Store. 162 | 163 | #### Lab 11.1.4: Secure Strings 164 | 165 | One of the features that makes Parameter Store *so* compelling is the 166 | way it easily [lets you store secrets](https://docs.aws.amazon.com/kms/latest/developerguide/services-parameter-store.html). 167 | When you set a key to a value, you simply reference a KMS key and mark 168 | the value as a secret. Anybody with access to the key can read it. As we 169 | said above, the simplicity of this service is part of what makes it so 170 | powerful. 171 | 172 | Store the engineer's middle name as a secret, then reference it in your 173 | stack from lab 1. 174 | 175 | First, use the awscli to store the middle name of an engineer in the 176 | hierarchy you created earlier. 177 | 178 | - The middle name should be a 179 | [Secure String](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-about.html#sysman-paramstore-securestring). 180 | 181 | - Store the info under the key "middle-name" within the given 182 | engineer's hierarchy. 183 | 184 | - "middle-name" should be encrypted with a 185 | [KMS key](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html), 186 | [generating a new one](https://docs.aws.amazon.com/cli/latest/reference/kms/create-key.html) 187 | if you have to. 188 | 189 | Then, in your template, look up the value of "middle-name" and add it to 190 | the web page served by nginx. 191 | 192 | ### Retrospective 11.1 193 | 194 | #### Question 1 195 | 196 | Read [Using Dynamic References to Specify Template Values](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html). 197 | Why can't you use that feature directly to read "middle-name" and show it 198 | in your web page? 199 | 200 | #### Question 2 201 | 202 | Can you use Secure String as an `AWS::SSM::Parameter::Value` type in a 203 | CloudFormation stack? 204 | 205 | ## Further Reading 206 | 207 | - Amazon recently [introduced](https://aws.amazon.com/blogs/aws/aws-secrets-manager-store-distribute-and-rotate-credentials-securely/) 208 | [Secrets Manager](https://aws.amazon.com/secrets-manager/). It 209 | integrates Parameter Store, Lambda, and a few other services into 210 | a product that makes password maintenance and rotation easier to do. 211 | Take a look at the [FAQ](https://aws.amazon.com/secrets-manager/faqs/) 212 | for a succinct summary of the service. 213 | 214 | - [The Right Way to Store Secrets Using Parameter Store](https://aws.amazon.com/blogs/mt/the-right-way-to-store-secrets-using-parameter-store/), 215 | in the AWS Tools Blog, covers Parameter Store usage nicely, and 216 | introduces the IAM permissions that are required to managed it as well. 217 | -------------------------------------------------------------------------------- /12-codepipeline/README.md: -------------------------------------------------------------------------------- 1 | # Topic 12: CodePipeline 2 | 3 | 4 | 5 | - [Topic 12: CodePipeline](#topic-12-codepipeline) 6 | - [Conventions](#conventions) 7 | - [Lesson 12.1: Introduction to CI/CD in AWS](#lesson-121-introduction-to-cicd-in-aws) 8 | - [Principle 12.1](#principle-121) 9 | - [Practice 12.1](#practice-121) 10 | - [Lab 12.1.1](#lab-1211) 11 | - [Lab 12.1.2](#lab-1212) 12 | - [Retrospective 12.1](#retrospective-121) 13 | - [Question: CloudFormation Template](#question-cloudformation-template) 14 | - [Question: Pipeline Template](#question-pipeline-template) 15 | - [Task](#task) 16 | - [Lesson 12.2: Pipelines Support Infrastructure as Code](#lesson-122-pipelines-support-infrastructure-as-code) 17 | - [Principle 12.2](#principle-122) 18 | - [Practice 12.2](#practice-122) 19 | - [Lab 12.2.1](#lab-1221) 20 | - [Lab 12.2.2](#lab-1222) 21 | - [Lab 12.2.3](#lab-1223) 22 | - [Retrospective 12.2](#retrospective-122) 23 | - [Further Reading](#further-reading) 24 | 25 | 26 | 27 | ## Conventions 28 | 29 | - All CloudFormation templates should be written in YAML. 30 | 31 | - Do NOT copy and paste CloudFormation templates from the Internet at large. 32 | 33 | - DO use the [CloudFormation documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html) 34 | 35 | - DO utilize every link in this document; note how the AWS documentation is 36 | laid out. 37 | 38 | - DO use the [AWS CLI for CloudFormation](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/index.html#) 39 | (NOT the Console) unless otherwise specified. 40 | 41 | ## Lesson 12.1: Introduction to CI/CD in AWS 42 | 43 | ### Principle 12.1 44 | 45 | *Pipelines are how Stelligent delivers infrastructure, software, custom 46 | environments and processes, and just about everything else.* 47 | 48 | ### Practice 12.1 49 | 50 | The CI/CD landscape is full of rich and full-featured tools, and no two 51 | organizations seem to use the same tool-set. As a result, there 52 | will be some learning curve on every project, and here we'll introduce 53 | some of the AWS-native tools. CodePipeline integrates well with 54 | CloudFormation, Lambda and other AWS resources. CodeBuild provides an 55 | extension of innate CodePipeline capabilities by providing customizable 56 | execution environments to perform custom, codified actions. 57 | 58 | This innate tool-set has its weaknesses as well, such as integration 59 | with only GitHub and S3 as 'sources' of code (and perhaps a sub-par 60 | organization of the documentation). However, *the ability and relative 61 | ease with which we can codify and version a pipeline, all without a CI 62 | server, is a very desirable advantage*. 63 | 64 | *HINT:* Organizing your references to critical parts of AWS 65 | documentation will probably be helpful. 66 | 67 | [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline.html) 68 | 69 | #### Lab 12.1.1 70 | 71 | Code a CFN template that generates a 72 | [CodePipeline Pipeline](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-codepipeline-pipeline.html) 73 | and its minimal set of requisite resources: 74 | 75 | - Add an S3 bucket for maintaining Pipeline execution state 76 | 77 | - Add an IAM [execution role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) 78 | that trusts the CodePipeline service and provides sufficient 79 | permissions to [deploy cloudformation](https://docs.aws.amazon.com/codepipeline/latest/userguide/how-to-custom-role.html). 80 | 81 | - Add an IAM execution role that trusts CloudFormation to create an S3 82 | bucket. 83 | 84 | - Add the CodePipeline Pipeline resource itself, encoding 85 | - a 'Source' stage to retrieve the code from GitHub 86 | - Create a [GitHub personal access token](https://github.com/settings/tokens) 87 | - Poll the repository for changes to the master branch 88 | - a 'Deploy' stage with an action that executes 89 | [this CFN template](https://github.com/stelligent/stelligent-u/blob/master/12-codepipeline/bucket.yaml) 90 | and creates an S3 bucket 91 | 92 | Deploy the Pipeline stack via the AWS CLI; its creation should 93 | automatically initiate the first Pipeline execution. 94 | 95 | #### Lab 12.1.2 96 | 97 | Add two new stages to your pipeline, utilizing CodeBuild: 98 | 99 | - A 'Build' stage with a single action, encoding commands to validate 100 | the 'application' CFN template using the AWS CLI 101 | 102 | - follows immediately after the Source stage 103 | 104 | - A 'Test' stage, again with a single action, that validates the 105 | 'application' stack is in a healthy state 106 | 107 | - Follows immediately after the 'Deploy' stage 108 | 109 | Make sure you understand how CodeBuild projects work prior to coding 110 | them, including the CFN resources you need: 111 | 112 | - an IAM execution role that trusts the CodeBuild service and provides 113 | sufficient permissions to perform the actions required by the 114 | actions in the 'Build' and 'Test' stages 115 | 116 | - the CodeBuild resources themselves, including build specs 117 | 118 | ### Retrospective 12.1 119 | 120 | #### Question: CloudFormation Template 121 | 122 | _Is executing a CloudFormation template a legitimate example of an 123 | "application"? Provide an explanation._ 124 | 125 | #### Question: Pipeline Template 126 | 127 | _Is your Pipeline template portable? Update and re-create your Pipeline if 128 | you hard-coded any of the following:_ 129 | 130 | - the name of the 'application' stack 131 | 132 | - the repository name, the branch to track, or personal access token 133 | 134 | - the S3 bucket name 135 | 136 | - anything else that might enhance portability 137 | 138 | #### Task 139 | 140 | Delete your pipeline stack and leave your bucket stack alone. Once the 141 | pipeline stack is gone try to delete the bucket stack. What happens? You 142 | likely won't be able to delete it because of missing roles. The order 143 | you delete stacks and which stack your define resources in can become a 144 | dependency web. To delete your bucket stack you can recreate your 145 | pipeline stack and use the created roles to recreate the missing role 146 | for your orphaned bucket stack. To help avoid this in the future it's a 147 | good idea to define your roles in a separate stack and use the outputs 148 | in your other stacks. 149 | 150 | ## Lesson 12.2: Pipelines Support Infrastructure as Code 151 | 152 | ### Principle 12.2 153 | 154 | *All AWS resources used by an application are part of that application's 155 | infrastructure and should be versioned code, just like the application 156 | itself, and delivered through automation pipelines.* 157 | 158 | ### Practice 12.2 159 | 160 | Well-architected software applications built in the cloud should take 161 | advantage of existing services wherever possible, to reduce the level of 162 | effort required to code and maintain the business logic that makes the 163 | application valuable to its consumers. Although the term 164 | [infrastructure as code](https://en.wikipedia.org/wiki/Infrastructure_as_Code) 165 | was originally coined for virtual servers, in AWS it should be used to 166 | describe all the AWS resources utilized by an application or application 167 | stack. 168 | 169 | Pipelines that once supported building and deploying applications can 170 | now easily build and deploy the entirety of an application stack, from 171 | networking elements and services that make an application 172 | highly-available to storage and monitoring resources that provide a 173 | functional and secure environment in which the application operates. 174 | 175 | These labs will assume we're building an inventory application that 176 | allows us to track stock of bicycle parts. We won't actually create that 177 | application, but we will create the DynamoDB table that will serve as 178 | the inventory. 179 | 180 | #### Lab 12.2.1 181 | 182 | Create an application infrastructure CFN template that generates the 183 | following. For the Table, keep it as simple as possible - add ID as the 184 | primary partition key 185 | 186 | - a DynamoDB table, as described 187 | 188 | - a Role that uses an AWS managed policy that will allow a fictional 189 | application the enough access to the DynamoDB service to read and 190 | write records to the table 191 | 192 | - You can use AmazonDynamoDBReadOnlyAccess or any other policy 193 | since we won't actually access our database 194 | 195 | Use the template: 196 | 197 | - Create a stack using this template to ensure it functions as 198 | expected 199 | 200 | - Delete this stack once the template is working. 201 | 202 | #### Lab 12.2.2 203 | 204 | Create a pipeline that leverages CodePipeline and CodeBuild to deploy 205 | the application stack (the table you just designed), with these 206 | requirements: 207 | 208 | - the pipeline should use a [CloudFormation ChangeSet](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-changesets.html) 209 | to either deploy or update the application stack 210 | 211 | - the *first* time the pipeline is executed, it should create the 212 | stack using a 'CREATE' type of ChangeSet 213 | 214 | - each *subsequent* pipeline execution should update the stack, again 215 | using an 'UPDATE' type of ChangeSet 216 | 217 | The Pipeline should therefore have a Source stage and a Deploy stage, 218 | and the Deploy stage should consist of two actions: 219 | 220 | - creating the ChangeSet 221 | 222 | - this is like a dry-run, which allows us to examine *what would 223 | have happened* if we had simply created or updated the stack 224 | 225 | - executing the ChangeSet 226 | 227 | - this *actually applies the changes* that were laid out when the 228 | ChangeSet was created 229 | 230 | #### Lab 12.2.3 231 | 232 | The previous Lab used a combination of *creating* and then *executing* a 233 | ChangeSet to generate a DynamoDB table for our hypothetical application. 234 | Why did we bother? We could have just created the stack in a single step 235 | and been done! 236 | 237 | Database data can be a vital resource and we want to protect it from 238 | accidental deletion[^1]. Once a DynamoDB table is created via 239 | CloudFormation, a template or parameter value change *that renames the 240 | table* will actually cause the table to be dropped and recreated. If you 241 | have data in the table, it will be lost as well. Let's exercise a simple 242 | precaution to prevent this by taking advantage of a built-in 243 | CodePipeline feature: an Approval action. 244 | 245 | - Add an Approval action between the 'creation' and 'execution' of the 246 | ChangeSet in the Pipeline's Deploy stage 247 | 248 | - this pauses the pipeline, allowing you to query and analyze the 249 | ChangeSet 250 | 251 | - If the ChangeSet wants to drop and recreate your DynamoDB table, 252 | reject the continuation of the pipeline 253 | 254 | - Otherwise, approve the continuation of the pipeline, allowing the 255 | pipeline to continue creating or updating the application stack. 256 | 257 | ### Retrospective 12.2 258 | 259 | Commit a change to the master branch of your repository to 260 | demonstrate how using ChangeSets and the Pipeline Approval features can 261 | help guard against dropping and recreating the table. 262 | 263 | ## Further Reading 264 | 265 | - Validating the CFN template right before deploying the stack isn't 266 | really that helpful since stack creation/update will validate it 267 | as well. Using a tool like [cfn_nag](https://github.com/stelligent/cfn_nag) 268 | would be a better test of your CFN template. 269 | 270 | - Manual approval of a change set can also 271 | [generate a notification](https://docs.aws.amazon.com/codepipeline/latest/userguide/approvals.html) 272 | to stakeholders so that users know when to review the application. 273 | 274 | [^1]: For the record, there are other ways of protecting AWS resources 275 | from being accidentally deleted, generally involving being very 276 | selective with access permissions. This exercise demonstrates 277 | another mechanism that can be more generally applicable. 278 | -------------------------------------------------------------------------------- /12-codepipeline/bucket.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | BucketNameParameter: 4 | Type: String 5 | Description: S3 Bucket Name 6 | Resources: 7 | S3Bucket: 8 | Type: AWS::S3::Bucket 9 | Properties: 10 | BucketName: !Ref BucketNameParameter 11 | DeletionPolicy: Delete 12 | -------------------------------------------------------------------------------- /13-ECS/README.md: -------------------------------------------------------------------------------- 1 | # Topic 13: Elastic Container Service (ECS) 2 | 3 | 4 | 5 | - [Topic 13: Elastic Container Service (ECS)](#topic-13-elastic-container-service-ecs) 6 | - [Guidance](#guidance) 7 | - [Lesson 13.1: Elastic Container Registry - ECR](#lesson-131-elastic-container-registry---ecr) 8 | - [Principle 13.1](#principle-131) 9 | - [Practice 13.1](#practice-131) 10 | - [Lab 13.1.1 Create a Repository](#lab-1311-create-a-repository) 11 | - [Lab 13.1.2 Create a life-cycle policy that expires images after a month](#lab-1312-create-a-life-cycle-policy-that-expires-images-after-a-month) 12 | - [Lab 13.1.3 Create a life-cycle policy that keeps only one untagged image](#lab-1313-create-a-life-cycle-policy-that-keeps-only-one-untagged-image) 13 | - [Lab 13.1.4 Use `ecr get-login` to authorize your Docker CLI](#lab-1314-use-ecr-get-login-to-authorize-your-docker-cli) 14 | - [Lab 13.1.5 Use `ecr get-authorization-token` to authorize your Docker CLI](#lab-1315-use-ecr-get-authorization-token-to-authorize-your-docker-cli) 15 | - [Lab 13.1.6 Push the latest image of NGINX from DockerHub to your repository](#lab-1316-push-the-latest-image-of-nginx-from-dockerhub-to-your-repository) 16 | - [Retrospective 13.1](#retrospective-131) 17 | - [Question: Image Encryption](#question-image-encryption) 18 | - [Question: Image Mirroring](#question-image-mirroring) 19 | - [Question: Repository Policies](#question-repository-policies) 20 | - [Question: Repository Authentication](#question-repository-authentication) 21 | - [Lesson 13.2: Elastic Container Service - Classic ECS](#lesson-132-elastic-container-service---classic-ecs) 22 | - [Principle 13.2](#principle-132) 23 | - [Practice 13.2](#practice-132) 24 | - [Lab 13.2.1 Create an ECS Cluster](#lab-1321-create-an-ecs-cluster) 25 | - [Lab 13.2.2 Add EC2 capacity to your Cluster](#lab-1322-add-ec2-capacity-to-your-cluster) 26 | - [Lab 13.2.3 Create a Task Definition with your NGINX container](#lab-1323-create-a-task-definition-with-your-nginx-container) 27 | - [Lab 13.2.4 Create a Service that runs your Task Definition](#lab-1324-create-a-service-that-runs-your-task-definition) 28 | - [Lab 13.2.5 Deploy your stack and verify your configuration](#lab-1325-deploy-your-stack-and-verify-your-configuration) 29 | - [Retrospective 13.2](#retrospective-132) 30 | - [Task: Enable CloudWatch logging](#task-enable-cloudwatch-logging) 31 | - [Task: Cleanup your ECS service](#task-cleanup-your-ecs-service) 32 | - [Question: Clusters with non-default names](#question-clusters-with-non-default-names) 33 | - [Question: ECS with custom AMIs](#question-ecs-with-custom-amis) 34 | - [Lesson 13.3: Elastic Container Service - Fargate ECS](#lesson-133-elastic-container-service---fargate-ecs) 35 | - [Principle 13.3](#principle-133) 36 | - [Practice 13.3](#practice-133) 37 | - [Lab 13.3.1 Create an ECS Cluster](#lab-1331-create-an-ecs-cluster) 38 | - [Lab 13.3.2 Create a Task Definition with two of your NGINX containers](#lab-1332-create-a-task-definition-with-two-of-your-nginx-containers) 39 | - [Lab 13.3.3 Create a Service that runs your Task Definition](#lab-1333-create-a-service-that-runs-your-task-definition) 40 | - [Lab 13.3.4 Deploy your stack and verify your configuration](#lab-1334-deploy-your-stack-and-verify-your-configuration) 41 | - [Retrospective 13.3](#retrospective-133) 42 | - [Question: EC2 capacity](#question-ec2-capacity) 43 | - [Question: Fargate volume/bind mounts](#question-fargate-volumebind-mounts) 44 | - [Question: Fargate Networking](#question-fargate-networking) 45 | - [Question: Fargate vs. Classic](#question-fargate-vs-classic) 46 | - [Question: Scaling vertically and horizontally](#question-scaling-vertically-and-horizontally) 47 | - [Task: Cleanup](#task-cleanup) 48 | - [Further Reading](#further-reading) 49 | 50 | ## Guidance 51 | 52 | - Explore ECS developer guides: [on AWS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) 53 | - If at any point you are confused with the terminology of ECS, keep the 54 | following diagram in mind: 55 | 56 | ```text 57 | +-----------------------------------------------------------+ 58 | | +----------------------+ +----------------------+ | 59 | | | +--------------------------------------------------+ | | 60 | | | | +------+ +------+ | | +------+ +------+ | | | 61 | | | | | | | | | | | | | | | | | 62 | | | | | Task | | Task | | | | Task | | Task | | | | 63 | | | | | | | | | | | | | | | | | 64 | | | | +------+ +------+ | | +------+ +------+ | | | 65 | | | | | Service | | | | 66 | | | +--------------------------------------------------+ | | 67 | | | Container | | Container | | 68 | | | Instance | | Instance | | 69 | | +----------------------+ +----------------------+ | 70 | | | 71 | | ECS Cluster | 72 | +-----------------------------------------------------------+ 73 | ``` 74 | 75 | - For lessons 2 and 3, a [starter](starter.yml) CloudFormation stack is 76 | provided for you. This stack lacks all resources under `AWS::ECS::*` 77 | namespace and relevant resources required in adding _EC2 capacity_ to an 78 | ECS cluster. You are more than welcome to write everything from scratch 79 | or use CDK and/or similar tools. This template is just provided for your 80 | convenience. 81 | 82 | ## Lesson 13.1: Elastic Container Registry - ECR 83 | 84 | ### Principle 13.1 85 | 86 | *ECR is the preferred way to store, update, and deploy Docker containers on AWS.* 87 | 88 | ### Practice 13.1 89 | 90 | This section walks you through setting up a basic ECR repository which houses the 91 | latest NGINX container image. The repository and its images will be used in the 92 | future parts of this course. 93 | 94 | #### Lab 13.1.1 Create a Repository 95 | 96 | Create a new ECR docker repository. 97 | 98 | - Use the [AWS::ECR::Repository](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecr-repository.html) 99 | resource type. 100 | - You will use this repository throughout this course, so export it from your 101 | CloudFormation stack with an [output](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html). 102 | 103 | #### Lab 13.1.2 Create a life-cycle policy that expires images after a month 104 | 105 | Modify your repository to include this life-cycle policy. 106 | 107 | #### Lab 13.1.3 Create a life-cycle policy that keeps only one untagged image 108 | 109 | Modify your repository to include this life-cycle policy. 110 | 111 | #### Lab 13.1.4 Use `ecr get-login` to authorize your Docker CLI 112 | 113 | Docker CLI needs to have access to your ECR repository before you can push to and 114 | pull images from it. Take a quick glance at [ECR CLI commands](https://docs.aws.amazon.com/cli/latest/reference/ecr/index.html#cli-aws-ecr): 115 | 116 | - Use `docker logout` to clear your previous sessions. 117 | - Try `docker pull`ing the image you exported earlier from your stack. It 118 | should fail. 119 | - Use `ecr get-login` to login to your ECR repository. 120 | - Try `docker pull`ing the image you exported earlier from your stack. It 121 | should succeed, can you explain what happened? 122 | 123 | #### Lab 13.1.5 Use `ecr get-authorization-token` to authorize your Docker CLI 124 | 125 | Take a quick look at the documentation for [this lab](https://docs.aws.amazon.com/cli/latest/reference/ecr/get-authorization-token.html): 126 | 127 | - Use `docker logout` to clear your previous sessions. 128 | - Try `docker pull`ing the image you exported earlier from your stack. It 129 | should fail. 130 | - Use `ecr get-authorization-token` to obtain a base64 encoded authorization 131 | token for Docker CLI. 132 | - Base64 decode the token and `docker login` with it. Use `AWS` as username 133 | for Docker CLI. 134 | - Try `docker pull`ing the image you exported earlier from your stack. It 135 | should succeed, can you explain what happened? 136 | 137 | #### Lab 13.1.6 Push the latest image of NGINX from DockerHub to your repository 138 | 139 | Pull the latest [NGINX](https://hub.docker.com/_/nginx) image and push it to 140 | your ECR repository 141 | 142 | - Verify you pushed successfully by `docker run`ing off of your ECR repo. 143 | - One liner for running a container and auto-removing it upon exit: 144 | `docker run --rm -it `. 145 | 146 | ### Retrospective 13.1 147 | 148 | #### Question: Image Encryption 149 | 150 | _We did not talk about encryption of images in this lab. How do you make sure 151 | images are encrypted at rest and while in-transit when using ECR?_ 152 | 153 | #### Question: Image Mirroring 154 | 155 | _When pushing public images from Docker Hub to ECR, images can get out of date 156 | compared to their version on Docker Hub. Think about AWS native solutions to 157 | mirror a public Docker Hub image on ECR._ 158 | 159 | #### Question: Repository Policies 160 | 161 | _In order to follow *the least privileges* principal, it is a common ask to set 162 | up repository policies for an ECR repository. What are some conditions you can 163 | impose on accessing a repository?_ 164 | 165 | #### Question: Repository Authentication 166 | 167 | _In this lesson, you were presented with two authentication methods for your 168 | Docker CLI. Explain the differences and why would you choose one over the other._ 169 | 170 | ## Lesson 13.2: Elastic Container Service - Classic ECS 171 | 172 | ### Principle 13.2 173 | 174 | _ECS (Classic ECS or EC2 backed ECS) is a way to orchestrate, provision and run 175 | containers on AWS._ 176 | 177 | ### Practice 13.2 178 | 179 | This section walks you through creating an ECS deployment which serves a basic 180 | _nginx_ web server container. This is a long and complicated sequence of labs. 181 | At the end of the lab, you should have a basic grasp on how to run containers on 182 | AWS through ECS. 183 | 184 | #### Lab 13.2.1 Create an ECS Cluster 185 | 186 | Create an ECS cluster for your service: 187 | 188 | - Use the provided [starter](starter.yml) CloudFormation template (`starter.yml`). 189 | - Use the [AWS::ECS::Cluster](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-cluster.html) 190 | resource type. 191 | - Tag your cluster with your Stelligent labs account name. 192 | 193 | #### Lab 13.2.2 Add EC2 capacity to your Cluster 194 | 195 | Modify your cluster to have _EC2 capacity_: 196 | 197 | - Use the [AWS::AutoScaling::LaunchConfiguration](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-launchconfig.html) 198 | resource type. 199 | - Set your Container Instances to be of type _t2.micro_. 200 | - Setup Container Instance security groups to allow outgoing port 80 traffic. 201 | - Use Amazon Linux 2 for your AMI. 202 | 203 | #### Lab 13.2.3 Create a Task Definition with your NGINX container 204 | 205 | Create a task definition that includes one instance of your NGINX container from 206 | the previous lesson: 207 | 208 | - Expose port 80 in the task definition 209 | - Set _CPU_ to 256 and _Memory_ to 512 210 | 211 | #### Lab 13.2.4 Create a Service that runs your Task Definition 212 | 213 | Create a Service that runs your task: 214 | 215 | - Use the [AWS::ECS::Service](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-service.html) 216 | resource type. 217 | - Set _DesiredCount_ to 1 218 | 219 | #### Lab 13.2.5 Deploy your stack and verify your configuration 220 | 221 | Deploy your finished CloudFormation stack and verify it works by exporting stack's 222 | public load balancer DNS address through a CloudFormation Output. 223 | 224 | ### Retrospective 13.2 225 | 226 | #### Task: Enable CloudWatch logging 227 | 228 | Now that your container is running, you need to have access to its logs. Go back 229 | and configure your containers to log to CloudWatch. What steps do you need to take 230 | to enable this feature? 231 | 232 | #### Task: Cleanup your ECS service 233 | 234 | Remove your entire ECS service, its associated ELB, auto-scaler, task definition, 235 | service and cluster. Do not remove the ECR repository yet! You will need it in the 236 | next lesson. 237 | 238 | #### Question: Clusters with non-default names 239 | 240 | _If you don't name your Cluster, it will be named `default` and no further 241 | configuration is needed in your EC2 instances. By referring to the documentation 242 | for [ECS agents](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-install.html), 243 | explain the steps required to associate EC2 Container Instances with a non-default 244 | cluster._ 245 | 246 | #### Question: ECS with custom AMIs 247 | 248 | _You can run containers on your own AMIs. Can you explain how is this possible?_ 249 | 250 | ## Lesson 13.3: Elastic Container Service - Fargate ECS 251 | 252 | ### Principle 13.3 253 | 254 | _Fargate is a managed and serverless abstraction over ECS. Fargate is the preferred 255 | way to run containers on AWS_ 256 | 257 | ### Practice 13.3 258 | 259 | This section introduces you to ECS Fargate, a serverless approach to running 260 | containers. You will deploy the same service as the previous lesson on Fargate. 261 | 262 | #### Lab 13.3.1 Create an ECS Cluster 263 | 264 | Create an ECS cluster for your service: 265 | 266 | - Use the provided [starter](starter.yml) CloudFormation template (`starter.yml`). 267 | - Use the [AWS::ECS::Cluster](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-cluster.html) 268 | resource type. 269 | - Tag your cluster with your Stelligent labs account name. 270 | 271 | #### Lab 13.3.2 Create a Task Definition with two of your NGINX containers 272 | 273 | Create a task definition that includes an instance of your NGINX container from 274 | the previous lessons: 275 | 276 | - Expose port 80 in the task definition 277 | - Set _CPU_ to 256 vCPU and _Memory_ to 512 278 | - Set _RequiresCompatibilities_ to _FARGATE_ 279 | - Set networking mode to _awsvpc_ 280 | 281 | #### Lab 13.3.3 Create a Service that runs your Task Definition 282 | 283 | Create a Service that runs your task. 284 | 285 | - Use the [AWS::ECS::Service](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-service.html) 286 | resource type. 287 | - Set _DesiredCount_ to 1 288 | 289 | #### Lab 13.3.4 Deploy your stack and verify your configuration 290 | 291 | Deploy your finished CloudFormation stack and verify it works by exporting stack's 292 | public load balancer DNS address through a CloudFormation Output. 293 | 294 | ### Retrospective 13.3 295 | 296 | #### Question: EC2 capacity 297 | 298 | _You did not need to specify an EC2 capacity in this lab, why?_ 299 | 300 | #### Question: Fargate volume/bind mounts 301 | 302 | _Fargate does not support volume and bind mounts. Why?_ 303 | 304 | #### Question: Fargate Networking 305 | 306 | _[Container linking](https://docs.docker.com/network/links/) is a deprecated feature 307 | of Docker and it is still widely used! This is currently supported on ECS, but not 308 | on Fargate. Can you explain how a similar networking setup can be achieved on Fargate 309 | without container linking?_ 310 | 311 | #### Question: Fargate vs. Classic 312 | 313 | _Under what circumstances could you _not_ use Fargate and why? Would ECS be capable 314 | of handling these situations?_ 315 | 316 | #### Question: Scaling vertically and horizontally 317 | 318 | _In terms of ECS application scaling, horizontal scaling means having more machines 319 | and scaling vertically means having more instances of the same application on the 320 | same machine. Can you explain which one of these are possible on Fargate and which 321 | ones are possible on ECS and why? Be precise in explaining the differences._ 322 | 323 | #### Task: Cleanup 324 | 325 | Cleanup after yourself! Remove all the resources you created during completion of 326 | the above labs (including the ECR repository). 327 | 328 | ## Further Reading 329 | 330 | - Make sure you understand how [Fargate networking](https://aws.amazon.com/blogs/compute/task-networking-in-aws-fargate/) 331 | works. Fargate is a sandbox, you need to know its ins and outs as it does not 332 | provide granular configuration options. 333 | - You can share your ECR images cross accounts. See an example 334 | [repository policy here](https://docs.aws.amazon.com/AmazonECR/latest/userguide/RepositoryPolicyExamples.html#IAM_allow_other_accounts). 335 | - Fargate has its own official CLI that abstracts a lot of CloudFormation details. 336 | [Check it out here](https://github.com/awslabs/fargatecli). 337 | -------------------------------------------------------------------------------- /13-ECS/starter.yml: -------------------------------------------------------------------------------- 1 | Resources: 2 | MyVpc: 3 | Type: AWS::EC2::VPC 4 | Properties: 5 | CidrBlock: 10.0.0.0/16 6 | EnableDnsHostnames: true 7 | EnableDnsSupport: true 8 | InstanceTenancy: default 9 | MyVpcPublicSubnet1Subnet: 10 | Type: AWS::EC2::Subnet 11 | Properties: 12 | CidrBlock: 10.0.0.0/18 13 | VpcId: 14 | Ref: MyVpc 15 | AvailabilityZone: 16 | Fn::Select: 17 | - 0 18 | - Fn::GetAZs: "" 19 | MapPublicIpOnLaunch: true 20 | MyVpcPublicSubnet1RouteTable: 21 | Type: AWS::EC2::RouteTable 22 | Properties: 23 | VpcId: 24 | Ref: MyVpc 25 | MyVpcPublicSubnet1RouteTableAssociation: 26 | Type: AWS::EC2::SubnetRouteTableAssociation 27 | Properties: 28 | RouteTableId: 29 | Ref: MyVpcPublicSubnet1RouteTable 30 | SubnetId: 31 | Ref: MyVpcPublicSubnet1Subnet 32 | MyVpcPublicSubnet1DefaultRoute: 33 | Type: AWS::EC2::Route 34 | Properties: 35 | RouteTableId: 36 | Ref: MyVpcPublicSubnet1RouteTable 37 | DestinationCidrBlock: 0.0.0.0/0 38 | GatewayId: 39 | Ref: MyVpcIGW 40 | DependsOn: 41 | - MyVpcVPCGW 42 | MyVpcPublicSubnet1EIP: 43 | Type: AWS::EC2::EIP 44 | Properties: 45 | Domain: vpc 46 | MyVpcPublicSubnet1NATGatewayAD3400C1: 47 | Type: AWS::EC2::NatGateway 48 | Properties: 49 | AllocationId: 50 | Fn::GetAtt: 51 | - MyVpcPublicSubnet1EIP 52 | - AllocationId 53 | SubnetId: 54 | Ref: MyVpcPublicSubnet1Subnet 55 | MyVpcPublicSubnet2Subnet: 56 | Type: AWS::EC2::Subnet 57 | Properties: 58 | CidrBlock: 10.0.64.0/18 59 | VpcId: 60 | Ref: MyVpc 61 | AvailabilityZone: 62 | Fn::Select: 63 | - 1 64 | - Fn::GetAZs: "" 65 | MapPublicIpOnLaunch: true 66 | MyVpcPublicSubnet2RouteTable: 67 | Type: AWS::EC2::RouteTable 68 | Properties: 69 | VpcId: 70 | Ref: MyVpc 71 | MyVpcPublicSubnet2RouteTableAssociation: 72 | Type: AWS::EC2::SubnetRouteTableAssociation 73 | Properties: 74 | RouteTableId: 75 | Ref: MyVpcPublicSubnet2RouteTable 76 | SubnetId: 77 | Ref: MyVpcPublicSubnet2Subnet 78 | MyVpcPublicSubnet2DefaultRoute: 79 | Type: AWS::EC2::Route 80 | Properties: 81 | RouteTableId: 82 | Ref: MyVpcPublicSubnet2RouteTable 83 | DestinationCidrBlock: 0.0.0.0/0 84 | GatewayId: 85 | Ref: MyVpcIGW 86 | DependsOn: 87 | - MyVpcVPCGW 88 | MyVpcPublicSubnet2EIP: 89 | Type: AWS::EC2::EIP 90 | Properties: 91 | Domain: vpc 92 | MyVpcPublicSubnet2NATGateway: 93 | Type: AWS::EC2::NatGateway 94 | Properties: 95 | AllocationId: 96 | Fn::GetAtt: 97 | - MyVpcPublicSubnet2EIP 98 | - AllocationId 99 | SubnetId: 100 | Ref: MyVpcPublicSubnet2Subnet 101 | MyVpcPrivateSubnet1: 102 | Type: AWS::EC2::Subnet 103 | Properties: 104 | CidrBlock: 10.0.128.0/18 105 | VpcId: 106 | Ref: MyVpc 107 | AvailabilityZone: 108 | Fn::Select: 109 | - 0 110 | - Fn::GetAZs: "" 111 | MapPublicIpOnLaunch: false 112 | MyVpcPrivateSubnet1RouteTable: 113 | Type: AWS::EC2::RouteTable 114 | Properties: 115 | VpcId: 116 | Ref: MyVpc 117 | MyVpcPrivateSubnet1RouteTableAssociation: 118 | Type: AWS::EC2::SubnetRouteTableAssociation 119 | Properties: 120 | RouteTableId: 121 | Ref: MyVpcPrivateSubnet1RouteTable 122 | SubnetId: 123 | Ref: MyVpcPrivateSubnet1 124 | MyVpcPrivateSubnet1DefaultRoute: 125 | Type: AWS::EC2::Route 126 | Properties: 127 | RouteTableId: 128 | Ref: MyVpcPrivateSubnet1RouteTable 129 | DestinationCidrBlock: 0.0.0.0/0 130 | NatGatewayId: 131 | Ref: MyVpcPublicSubnet1NATGatewayAD3400C1 132 | MyVpcPrivateSubnet2: 133 | Type: AWS::EC2::Subnet 134 | Properties: 135 | CidrBlock: 10.0.192.0/18 136 | VpcId: 137 | Ref: MyVpc 138 | AvailabilityZone: 139 | Fn::Select: 140 | - 1 141 | - Fn::GetAZs: "" 142 | MapPublicIpOnLaunch: false 143 | MyVpcPrivateSubnet2RouteTable: 144 | Type: AWS::EC2::RouteTable 145 | Properties: 146 | VpcId: 147 | Ref: MyVpc 148 | MyVpcPrivateSubnet2RouteTableAssociation: 149 | Type: AWS::EC2::SubnetRouteTableAssociation 150 | Properties: 151 | RouteTableId: 152 | Ref: MyVpcPrivateSubnet2RouteTable 153 | SubnetId: 154 | Ref: MyVpcPrivateSubnet2 155 | MyVpcPrivateSubnet2DefaultRoute: 156 | Type: AWS::EC2::Route 157 | Properties: 158 | RouteTableId: 159 | Ref: MyVpcPrivateSubnet2RouteTable 160 | DestinationCidrBlock: 0.0.0.0/0 161 | NatGatewayId: 162 | Ref: MyVpcPublicSubnet2NATGateway 163 | MyVpcIGW: 164 | Type: AWS::EC2::InternetGateway 165 | MyVpcVPCGW: 166 | Type: AWS::EC2::VPCGatewayAttachment 167 | Properties: 168 | VpcId: 169 | Ref: MyVpc 170 | InternetGatewayId: 171 | Ref: MyVpcIGW 172 | MyAppLB: 173 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 174 | Properties: 175 | Scheme: internet-facing 176 | SecurityGroups: 177 | - Fn::GetAtt: 178 | - MyAppLBSecurityGroup 179 | - GroupId 180 | Subnets: 181 | - Ref: MyVpcPublicSubnet1Subnet 182 | - Ref: MyVpcPublicSubnet2Subnet 183 | Type: application 184 | DependsOn: 185 | - MyVpcPublicSubnet1DefaultRoute 186 | - MyVpcPublicSubnet2DefaultRoute 187 | MyAppLBSecurityGroup: 188 | Type: AWS::EC2::SecurityGroup 189 | Properties: 190 | GroupDescription: Security Group for ELB ECSMyApp 191 | SecurityGroupIngress: 192 | - CidrIp: 0.0.0.0/0 193 | Description: Allow from anyone on port 80 194 | FromPort: 80 195 | IpProtocol: tcp 196 | ToPort: 80 197 | VpcId: 198 | Ref: MyVpc 199 | MyAppLBSecurityGroupEgress: 200 | Type: AWS::EC2::SecurityGroupEgress 201 | Properties: 202 | GroupId: 203 | Fn::GetAtt: 204 | - MyAppLBSecurityGroup 205 | - GroupId 206 | IpProtocol: tcp 207 | Description: Load balancer to target 208 | DestinationSecurityGroupId: 209 | Fn::GetAtt: 210 | - MyAppServiceSecurityGroup 211 | - GroupId 212 | FromPort: 80 213 | ToPort: 80 214 | MyAppLBPublicListener: 215 | Type: AWS::ElasticLoadBalancingV2::Listener 216 | Properties: 217 | DefaultActions: 218 | - TargetGroupArn: 219 | Ref: MyAppLBPublicListenerECSGroup 220 | Type: forward 221 | LoadBalancerArn: 222 | Ref: MyAppLB 223 | Port: 80 224 | Protocol: HTTP 225 | MyAppLBPublicListenerECSGroup: 226 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 227 | Properties: 228 | Port: 80 229 | Protocol: HTTP 230 | TargetType: ip 231 | VpcId: 232 | Ref: MyVpc 233 | MyAppTaskDefTaskRole: 234 | Type: AWS::IAM::Role 235 | Properties: 236 | AssumeRolePolicyDocument: 237 | Statement: 238 | - Action: sts:AssumeRole 239 | Effect: Allow 240 | Principal: 241 | Service: ecs-tasks.amazonaws.com 242 | Version: "2012-10-17" 243 | MyAppTaskDefwebLogGroup: 244 | Type: AWS::Logs::LogGroup 245 | UpdateReplacePolicy: Retain 246 | DeletionPolicy: Retain 247 | MyAppTaskDefExecutionRole: 248 | Type: AWS::IAM::Role 249 | Properties: 250 | AssumeRolePolicyDocument: 251 | Statement: 252 | - Action: sts:AssumeRole 253 | Effect: Allow 254 | Principal: 255 | Service: ecs-tasks.amazonaws.com 256 | Version: "2012-10-17" 257 | MyAppTaskDefExecutionRoleDefaultPolicy: 258 | Type: AWS::IAM::Policy 259 | Properties: 260 | PolicyDocument: 261 | Statement: 262 | - Action: 263 | - logs:CreateLogStream 264 | - logs:PutLogEvents 265 | Effect: Allow 266 | Resource: 267 | Fn::GetAtt: 268 | - MyAppTaskDefwebLogGroup 269 | - Arn 270 | Version: "2012-10-17" 271 | PolicyName: MyAppTaskDefExecutionRoleDefaultPolicy 272 | Roles: 273 | - Ref: MyAppTaskDefExecutionRole 274 | MyAppServiceSecurityGroup: 275 | Type: AWS::EC2::SecurityGroup 276 | Properties: 277 | GroupDescription: ECS/MyApp/Service/SecurityGroup 278 | SecurityGroupEgress: 279 | - CidrIp: 0.0.0.0/0 280 | Description: Allow all outbound traffic by default 281 | IpProtocol: "-1" 282 | VpcId: 283 | Ref: MyVpc 284 | MyAppServiceSecurityGroupIngress: 285 | Type: AWS::EC2::SecurityGroupIngress 286 | Properties: 287 | IpProtocol: tcp 288 | Description: Load balancer to target 289 | FromPort: 80 290 | GroupId: 291 | Fn::GetAtt: 292 | - MyAppServiceSecurityGroup 293 | - GroupId 294 | SourceSecurityGroupId: 295 | Fn::GetAtt: 296 | - MyAppLBSecurityGroup 297 | - GroupId 298 | ToPort: 80 299 | -------------------------------------------------------------------------------- /14-Jenkins/base-params.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "ParameterKey": "PrivateIp", 4 | "ParameterValue": "10.0.0.0" 5 | }, 6 | { 7 | "ParameterKey": "VPCMask", 8 | "ParameterValue": "16" 9 | }, 10 | { 11 | "ParameterKey": "IAMUserOwnerOfResources", 12 | "ParameterValue": "test.user" 13 | }, 14 | { 15 | "ParameterKey": "EC2InstanceAmi", 16 | "ParameterValue": "ami-0bce08e823ed38bdd" 17 | }, 18 | { 19 | "ParameterKey": "EC2InstanceType", 20 | "ParameterValue": "t2.micro" 21 | }, 22 | { 23 | "ParameterKey": "EC2KeyName", 24 | "ParameterValue": "test-user-key" 25 | }, 26 | { 27 | "ParameterKey": "IAMUserOwnerIp", 28 | "ParameterValue": "XX.XXX.XXX.XXX" 29 | } 30 | ] -------------------------------------------------------------------------------- /14-Jenkins/base.yaml: -------------------------------------------------------------------------------- 1 | Description: Jenkins Continuous Learning 2 | Parameters: 3 | PrivateIp: 4 | Type: String 5 | Description: Private Ip used for the VPC 6 | Default: 10.0.0.0 7 | VPCMask: 8 | Type: String 9 | Description: Mask used for the VPC 10 | Default: 16 11 | IAMUserOwnerOfResources: 12 | Type: String 13 | Description: IAM user that is owner of this CFN stack 14 | EC2InstanceAmi: 15 | Type: AWS::EC2::Image::Id 16 | Description: EC2 Instance AMI 17 | Default: ami-014534a11156c1a0d 18 | EC2KeyName: 19 | Type: String 20 | Description: Name of the EC2 keys used for ssh 21 | Default: 22 | IAMUserOwnerIp: 23 | Type: String 24 | Description: IP of the Iam user owner 25 | Default: 26 | EC2InstanceType: 27 | Type: String 28 | Default: 29 | Resources: 30 | MyVpc: 31 | Type: AWS::EC2::VPC 32 | Properties: 33 | CidrBlock: !Join ['/', [!Ref PrivateIp, !Ref VPCMask ] ] 34 | Tags: &ResourceTags 35 | - 36 | Key: user 37 | Value: !Ref IAMUserOwnerOfResources 38 | - 39 | Key: stack 40 | Value: !Ref "AWS::StackId" 41 | MySubnet: 42 | Type: AWS::EC2::Subnet 43 | Properties: 44 | VpcId: !Ref MyVpc 45 | CidrBlock: !Join ['/', [!Ref PrivateIp, '25' ] ] 46 | Tags: 47 | - 48 | Key: user 49 | Value: !Ref IAMUserOwnerOfResources 50 | - 51 | Key: stack 52 | Value: !Ref "AWS::StackId" 53 | MyInternetGateway: 54 | Type: AWS::EC2::InternetGateway 55 | AttachInternetGatewayToVpc: 56 | Type: AWS::EC2::VPCGatewayAttachment 57 | Properties: 58 | InternetGatewayId: !Ref MyInternetGateway 59 | VpcId: !Ref MyVpc 60 | MyRouteTable: 61 | Type: AWS::EC2::RouteTable 62 | Properties: 63 | VpcId: !Ref MyVpc 64 | SubnetRouteTableAssociation: 65 | Type: AWS::EC2::SubnetRouteTableAssociation 66 | Properties: 67 | RouteTableId: !Ref MyRouteTable 68 | SubnetId: !Ref MySubnet 69 | MyRoute: 70 | Type: AWS::EC2::Route 71 | Properties: 72 | RouteTableId: !Ref MyRouteTable 73 | GatewayId: !Ref MyInternetGateway 74 | DestinationCidrBlock: '0.0.0.0/0' 75 | MyEC2SecurityGroup: 76 | Type: AWS::EC2::SecurityGroup 77 | Properties: 78 | GroupDescription: Allow ICMP/ping & SSH access 79 | GroupName: !Join ['-', [!Ref 'AWS::StackName', SecurityGroup] ] 80 | VpcId: !Ref MyVpc 81 | SecurityGroupIngress: 82 | - 83 | FromPort: 22 84 | ToPort: 22 85 | IpProtocol: tcp 86 | CidrIp: 0.0.0.0/0 87 | - 88 | FromPort: -1 89 | ToPort: -1 90 | IpProtocol: icmp 91 | CidrIp: 0.0.0.0/0 92 | MyEC2InstanceIp: 93 | Type: AWS::EC2::EIP 94 | Properties: 95 | Domain: vpc 96 | InstanceId: !Ref MyEC2Instance 97 | MyEC2Instance: 98 | Type: AWS::EC2::Instance 99 | Properties: 100 | SubnetId: !Ref MySubnet 101 | KeyName: !Ref EC2KeyName 102 | ImageId: !Ref EC2InstanceAmi 103 | InstanceType: !Ref EC2InstanceType 104 | SecurityGroupIds: 105 | - !Ref MyEC2SecurityGroup 106 | Tags: 107 | - 108 | Key: user 109 | Value: !Ref IAMUserOwnerOfResources 110 | - 111 | Key: stack 112 | Value: !Ref "AWS::StackId" 113 | MyNetworkAcl: 114 | Type: AWS::EC2::NetworkAcl 115 | Properties: 116 | VpcId: !Ref MyVpc 117 | MyNetworkEntryMySubnetIngressAllTraffic: 118 | Type: AWS::EC2::NetworkAclEntry 119 | Properties: 120 | NetworkAclId: !Ref MyNetworkAcl 121 | RuleNumber: 100 122 | RuleAction: deny 123 | Protocol: -1 124 | Egress: True 125 | CidrBlock: 0.0.0.0/0 126 | MyNetworkEntryMySubnetIngressIamOwner: 127 | Type: AWS::EC2::NetworkAclEntry 128 | Properties: 129 | NetworkAclId: !Ref MyNetworkAcl 130 | RuleNumber: 101 131 | RuleAction: allow 132 | Protocol: 6 133 | CidrBlock: !Join ['/', [!Ref IAMUserOwnerIp, '32' ] ] 134 | PortRange: 135 | From: 22 136 | To: 22 137 | MyNetworkEntryMySubnetEgressIamOwner: 138 | Type: AWS::EC2::NetworkAclEntry 139 | Properties: 140 | NetworkAclId: !Ref MyNetworkAcl 141 | RuleNumber: 101 142 | RuleAction: allow 143 | Protocol: -1 144 | Egress: True 145 | CidrBlock: !Join ['/', [!Ref IAMUserOwnerIp, '32' ] ] 146 | MyNetworkEntryMySecondSubnetIngressAllTraffic: 147 | Type: AWS::EC2::NetworkAclEntry 148 | Properties: 149 | NetworkAclId: !Ref MyNetworkAcl 150 | RuleNumber: 200 151 | RuleAction: allow 152 | Protocol: -1 153 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 154 | MyNetworkEntryMySecondSubnetIngressAllSsh: 155 | Type: AWS::EC2::NetworkAclEntry 156 | Properties: 157 | NetworkAclId: !Ref MyNetworkAcl 158 | RuleNumber: 201 159 | RuleAction: allow 160 | Protocol: 6 161 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 162 | PortRange: 163 | From: 22 164 | To: 22 165 | MyNetworkEntryMySecondSubnetIngressAllPing: 166 | Type: AWS::EC2::NetworkAclEntry 167 | Properties: 168 | NetworkAclId: !Ref MyNetworkAcl 169 | RuleNumber: 202 170 | RuleAction: allow 171 | Protocol: 6 172 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 173 | Icmp: 174 | Code: -1 175 | Type: -1 176 | PortRange: 177 | From: 178 | To: 179 | MyNetworkEntryMySecondSubnetIngressAllHttp: 180 | Type: AWS::EC2::NetworkAclEntry 181 | Properties: 182 | NetworkAclId: !Ref MyNetworkAcl 183 | RuleNumber: 203 184 | RuleAction: allow 185 | Protocol: 6 186 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 187 | PortRange: 188 | From: 80 189 | To: 80 190 | MyNetworkEntryMySecondSubnetEgressAllTraffic: 191 | Type: AWS::EC2::NetworkAclEntry 192 | Properties: 193 | NetworkAclId: !Ref MyNetworkAcl 194 | RuleNumber: 204 195 | RuleAction: allow 196 | Egress: True 197 | Protocol: -1 198 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 199 | PortRange: 200 | From: 201 | To: 202 | MySecondSubnet: 203 | Type: AWS::EC2::Subnet 204 | Properties: 205 | VpcId: !Ref MyVpc 206 | CidrBlock: !Join ['/', ["10.0.0.128", '25' ] ] 207 | Tags: 208 | - 209 | Key: user 210 | Value: !Ref IAMUserOwnerOfResources 211 | - 212 | Key: stack 213 | Value: !Ref "AWS::StackId" 214 | MySecondRouteTable: 215 | Type: AWS::EC2::RouteTable 216 | Properties: 217 | VpcId: !Ref MyVpc 218 | SecondSubnetRouteTableAssociation: 219 | Type: AWS::EC2::SubnetRouteTableAssociation 220 | Properties: 221 | RouteTableId: !Ref MySecondRouteTable 222 | SubnetId: !Ref MySecondSubnet 223 | MySecondRoute: 224 | Type: AWS::EC2::Route 225 | Properties: 226 | RouteTableId: !Ref MySecondRouteTable 227 | NatGatewayId: !Ref MyNatGateway 228 | DestinationCidrBlock: '0.0.0.0/0' 229 | MyNatGatewayInstanceIp: 230 | Type: AWS::EC2::EIP 231 | Properties: 232 | Domain: vpc 233 | MyNatGateway: 234 | Type: AWS::EC2::NatGateway 235 | Properties: 236 | AllocationId: !GetAtt [ MyNatGatewayInstanceIp, AllocationId ] 237 | SubnetId: !Ref MySubnet 238 | MySecondEC2Instance: 239 | Type: AWS::EC2::Instance 240 | Properties: 241 | SubnetId: !Ref MySecondSubnet 242 | KeyName: !Ref EC2KeyName 243 | ImageId: !Ref EC2InstanceAmi 244 | SecurityGroupIds: 245 | - !Ref MyEC2SecurityGroup 246 | Tags: 247 | - 248 | Key: user 249 | Value: !Ref IAMUserOwnerOfResources 250 | - 251 | Key: stack 252 | Value: !Ref "AWS::StackId" 253 | Outputs: 254 | MyVpcOutput: 255 | Description: Id of MyVpc 256 | Value: !Ref MyVpc 257 | MySubnetOutput: 258 | Description: Id of MyVpc 259 | Value: !Ref MySubnet 260 | MyEC2InstanceOutput: 261 | Description: Id of MyEC2Instance 262 | Value: !Ref MyEC2Instance 263 | MyEC2InstancePrivateIpOutput: 264 | Description: Private Ip address of MyEC2Instance 265 | Value: !GetAtt [MyEC2Instance, PrivateIp] 266 | MyEC2InstancePublicIpOutput: 267 | Description: Public Ip address of MyEC2Instance 268 | Value: !GetAtt [MyEC2Instance, PublicIp] 269 | -------------------------------------------------------------------------------- /14-Jenkins/my-app/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 4.0.0 6 | 7 | com.stelligent.app 8 | my-app 9 | 1.0-SNAPSHOT 10 | 11 | my-app 12 | 13 | http://www.example.com 14 | 15 | 16 | UTF-8 17 | 1.7 18 | 1.7 19 | 20 | 21 | 22 | 23 | junit 24 | junit 25 | 4.13.1 26 | test 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | maven-clean-plugin 36 | 3.1.0 37 | 38 | 39 | 40 | maven-resources-plugin 41 | 3.0.2 42 | 43 | 44 | maven-compiler-plugin 45 | 3.8.0 46 | 47 | 48 | maven-surefire-plugin 49 | 2.22.1 50 | 51 | 52 | maven-jar-plugin 53 | 3.0.2 54 | 55 | 56 | maven-install-plugin 57 | 2.5.2 58 | 59 | 60 | maven-deploy-plugin 61 | 2.8.2 62 | 63 | 64 | 65 | maven-site-plugin 66 | 3.7.1 67 | 68 | 69 | maven-project-info-reports-plugin 70 | 3.0.0 71 | 72 | 73 | 74 | 75 | 76 | -------------------------------------------------------------------------------- /14-Jenkins/my-app/src/main/java/com/stelligent/app/App.java: -------------------------------------------------------------------------------- 1 | package com.stelligent.app; 2 | 3 | /** 4 | * Hello world! 5 | * 6 | */ 7 | public class App 8 | { 9 | public static void main( String[] args ) 10 | { 11 | System.out.println( "Hello World!" ); 12 | } 13 | } 14 | -------------------------------------------------------------------------------- /14-Jenkins/my-app/src/test/java/com/stelligent/app/AppTest.java: -------------------------------------------------------------------------------- 1 | package com.stelligent.app; 2 | 3 | import static org.junit.Assert.assertTrue; 4 | 5 | import org.junit.Test; 6 | 7 | /** 8 | * Unit test for simple App. 9 | */ 10 | public class AppTest 11 | { 12 | /** 13 | * Rigorous Test :-) 14 | */ 15 | @Test 16 | public void shouldAnswerWithTrue() 17 | { 18 | assertTrue( true ); 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /15-Kubernetes/ecr/ecr.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: "2010-09-09" 2 | Description: "Kubernetes ECR Repository" 3 | 4 | Parameters: 5 | Prefix: 6 | Type: String 7 | Description: ECR Repostory Prefix 8 | 9 | Resources: 10 | 11 | KubernetesECR: 12 | Type: AWS::ECR::Repository 13 | Properties: 14 | RepositoryName: !Sub "${Prefix}-kubernetes" 15 | RepositoryPolicyText: 16 | Version: "2012-10-17" 17 | Statement: 18 | - 19 | Sid: AllowPushPull 20 | Effect: Allow 21 | Principal: 22 | AWS: 23 | - !Sub "arn:aws:iam::${AWS::AccountId}:root" 24 | Action: 25 | - "ecr:GetDownloadUrlForLayer" 26 | - "ecr:BatchGetImage" 27 | - "ecr:BatchCheckLayerAvailability" 28 | - "ecr:PutImage" 29 | - "ecr:InitiateLayerUpload" 30 | - "ecr:UploadLayerPart" 31 | - "ecr:CompleteLayerUpload" 32 | - "ecr:GetAuthorizationToken" 33 | 34 | Outputs: 35 | ECRRepoName: 36 | Value: !Ref KubernetesECR 37 | Description: ECR Repository Name -------------------------------------------------------------------------------- /15-Kubernetes/eksctl/cluster.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: eksctl.io/v1alpha5 2 | kind: ClusterConfig 3 | 4 | metadata: 5 | name: stelligent-u-cluster 6 | region: us-east-1 7 | # availabilityZones: ["us-east-1a", "us-east-1b"] 8 | 9 | managedNodeGroups: 10 | - name: ng-1 11 | instanceType: t3.small 12 | desiredCapacity: 2 13 | - name: ng-2 14 | instanceType: t3.small 15 | desiredCapacity: 2 16 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/.gitignore: -------------------------------------------------------------------------------- 1 | build/ 2 | node_modules/ 3 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM registry.access.redhat.com/ubi8/ubi-minimal 2 | FROM node:12.0-stretch-slim AS builder 3 | 4 | ENV APP=/var/www 5 | 6 | # Create app directory 7 | RUN mkdir -p $APP 8 | WORKDIR $APP 9 | 10 | # Install app dependencies 11 | COPY package*.json $APP/ 12 | RUN npm install --production 13 | 14 | COPY . $APP 15 | RUN npm run-script build 16 | 17 | EXPOSE 8080 18 | CMD ["npm", "run", "start"] 19 | 20 | 21 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/README.md: -------------------------------------------------------------------------------- 1 | # Kubernetes Sample React JS App 2 | 3 | ## Steps to Build 4 | 5 | 1. Run `docker build .` 6 | 1. Run 7 | 8 | ```bash 9 | docker run -p 8080:3000 \ 10 | --env REACT_APP_BG_COLOR= 11 | ``` 12 | 13 | 1. Navigate your browser to `http://localhost:8080` 14 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "platformone", 3 | "version": "0.1.0", 4 | "private": true, 5 | "dependencies": { 6 | "@testing-library/jest-dom": "^4.2.4", 7 | "@testing-library/react": "^9.3.2", 8 | "@testing-library/user-event": "^7.1.2", 9 | "react": "^16.13.0", 10 | "react-dom": "^16.13.0", 11 | "react-scripts": "3.4.0" 12 | }, 13 | "scripts": { 14 | "start": "react-scripts start", 15 | "build": "react-scripts build", 16 | "test": "react-scripts test", 17 | "eject": "react-scripts eject" 18 | }, 19 | "eslintConfig": { 20 | "extends": "react-app" 21 | }, 22 | "browserslist": { 23 | "production": [ 24 | ">0.2%", 25 | "not dead", 26 | "not op_mini all" 27 | ], 28 | "development": [ 29 | "last 1 chrome version", 30 | "last 1 firefox version", 31 | "last 1 safari version" 32 | ] 33 | } 34 | } 35 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/public/favicon.ico -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/favicon.ico.org: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/public/favicon.ico.org -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 12 | 13 | 17 | 18 | 27 | Platform One 28 | 29 | 30 | 31 |
32 | 42 | 43 | 44 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/logo192.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/public/logo192.png -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/logo512.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/public/logo512.png -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/manifest.json: -------------------------------------------------------------------------------- 1 | { 2 | "short_name": "React App", 3 | "name": "Create React App Sample", 4 | "icons": [ 5 | { 6 | "src": "favicon.ico", 7 | "sizes": "64x64 32x32 24x24 16x16", 8 | "type": "image/x-icon" 9 | }, 10 | { 11 | "src": "logo192.png", 12 | "type": "image/png", 13 | "sizes": "192x192" 14 | }, 15 | { 16 | "src": "logo512.png", 17 | "type": "image/png", 18 | "sizes": "512x512" 19 | } 20 | ], 21 | "start_url": ".", 22 | "display": "standalone", 23 | "theme_color": "#000000", 24 | "background_color": "#ffffff" 25 | } 26 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/public/robots.txt: -------------------------------------------------------------------------------- 1 | # https://www.robotstxt.org/robotstxt.html 2 | User-agent: * 3 | Disallow: 4 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/App.css: -------------------------------------------------------------------------------- 1 | .App { 2 | text-align: center; 3 | } 4 | 5 | .App-logo { 6 | height: 40vmin; 7 | pointer-events: none; 8 | } 9 | 10 | @media (prefers-reduced-motion: no-preference) { 11 | .App-logo1 { 12 | animation: App-logo-spin infinite 20s linear; 13 | } 14 | } 15 | 16 | .App-header { 17 | background-color: #282c34; 18 | min-height: 10vh; 19 | display: flex; 20 | flex-direction: column; 21 | align-items: center; 22 | justify-content: center; 23 | font-size: calc(10px + 2vmin); 24 | color: white; 25 | } 26 | 27 | .App-link { 28 | color: #61dafb; 29 | } 30 | 31 | @keyframes App-logo-spin { 32 | from { 33 | transform: rotate(0deg); 34 | } 35 | to { 36 | transform: rotate(360deg); 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/App.js: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import logo from './stelligent.jpg'; 3 | import './App.css'; 4 | 5 | function App() { 6 | const bgColor = process.env.REACT_APP_BG_COLOR || 'white' 7 | return ( 8 |
9 |
10 | Stelligent 11 |
12 |
13 | logo 14 |

15 |
We're here to chew bubble gum and kick butt...and we're all out of bubble gum...
16 |
17 |
18 | ); 19 | } 20 | 21 | export default App; 22 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/App.test.js: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import { render } from '@testing-library/react'; 3 | import App from './App'; 4 | 5 | test('renders learn react link', () => { 6 | const { getByText } = render(); 7 | const linkElement = getByText(/Office/i); 8 | expect(linkElement).toBeInTheDocument(); 9 | }); 10 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/index.css: -------------------------------------------------------------------------------- 1 | body { 2 | margin: 0; 3 | font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 4 | 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 5 | sans-serif; 6 | -webkit-font-smoothing: antialiased; 7 | -moz-osx-font-smoothing: grayscale; 8 | } 9 | 10 | code { 11 | font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', 12 | monospace; 13 | } 14 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/index.js: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import ReactDOM from 'react-dom'; 3 | import './index.css'; 4 | import App from './App'; 5 | import * as serviceWorker from './serviceWorker'; 6 | 7 | ReactDOM.render(, document.getElementById('root')); 8 | 9 | // If you want your app to work offline and load faster, you can change 10 | // unregister() to register() below. Note this comes with some pitfalls. 11 | // Learn more about service workers: https://bit.ly/CRA-PWA 12 | serviceWorker.unregister(); 13 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/logo.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/platformone.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/src/platformone.png -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/serviceWorker.js: -------------------------------------------------------------------------------- 1 | // This optional code is used to register a service worker. 2 | // register() is not called by default. 3 | 4 | // This lets the app load faster on subsequent visits in production, and gives 5 | // it offline capabilities. However, it also means that developers (and users) 6 | // will only see deployed updates on subsequent visits to a page, after all the 7 | // existing tabs open on the page have been closed, since previously cached 8 | // resources are updated in the background. 9 | 10 | // To learn more about the benefits of this model and instructions on how to 11 | // opt-in, read https://bit.ly/CRA-PWA 12 | 13 | const isLocalhost = Boolean( 14 | window.location.hostname === 'localhost' || 15 | // [::1] is the IPv6 localhost address. 16 | window.location.hostname === '[::1]' || 17 | // 127.0.0.0/8 are considered localhost for IPv4. 18 | window.location.hostname.match( 19 | /^127(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}$/ 20 | ) 21 | ); 22 | 23 | export function register(config) { 24 | if (process.env.NODE_ENV === 'production' && 'serviceWorker' in navigator) { 25 | // The URL constructor is available in all browsers that support SW. 26 | const publicUrl = new URL(process.env.PUBLIC_URL, window.location.href); 27 | if (publicUrl.origin !== window.location.origin) { 28 | // Our service worker won't work if PUBLIC_URL is on a different origin 29 | // from what our page is served on. This might happen if a CDN is used to 30 | // serve assets; see https://github.com/facebook/create-react-app/issues/2374 31 | return; 32 | } 33 | 34 | window.addEventListener('load', () => { 35 | const swUrl = `${process.env.PUBLIC_URL}/service-worker.js`; 36 | 37 | if (isLocalhost) { 38 | // This is running on localhost. Let's check if a service worker still exists or not. 39 | checkValidServiceWorker(swUrl, config); 40 | 41 | // Add some additional logging to localhost, pointing developers to the 42 | // service worker/PWA documentation. 43 | navigator.serviceWorker.ready.then(() => { 44 | console.log( 45 | 'This web app is being served cache-first by a service ' + 46 | 'worker. To learn more, visit https://bit.ly/CRA-PWA' 47 | ); 48 | }); 49 | } else { 50 | // Is not localhost. Just register service worker 51 | registerValidSW(swUrl, config); 52 | } 53 | }); 54 | } 55 | } 56 | 57 | function registerValidSW(swUrl, config) { 58 | navigator.serviceWorker 59 | .register(swUrl) 60 | .then(registration => { 61 | registration.onupdatefound = () => { 62 | const installingWorker = registration.installing; 63 | if (installingWorker == null) { 64 | return; 65 | } 66 | installingWorker.onstatechange = () => { 67 | if (installingWorker.state === 'installed') { 68 | if (navigator.serviceWorker.controller) { 69 | // At this point, the updated precached content has been fetched, 70 | // but the previous service worker will still serve the older 71 | // content until all client tabs are closed. 72 | console.log( 73 | 'New content is available and will be used when all ' + 74 | 'tabs for this page are closed. See https://bit.ly/CRA-PWA.' 75 | ); 76 | 77 | // Execute callback 78 | if (config && config.onUpdate) { 79 | config.onUpdate(registration); 80 | } 81 | } else { 82 | // At this point, everything has been precached. 83 | // It's the perfect time to display a 84 | // "Content is cached for offline use." message. 85 | console.log('Content is cached for offline use.'); 86 | 87 | // Execute callback 88 | if (config && config.onSuccess) { 89 | config.onSuccess(registration); 90 | } 91 | } 92 | } 93 | }; 94 | }; 95 | }) 96 | .catch(error => { 97 | console.error('Error during service worker registration:', error); 98 | }); 99 | } 100 | 101 | function checkValidServiceWorker(swUrl, config) { 102 | // Check if the service worker can be found. If it can't reload the page. 103 | fetch(swUrl, { 104 | headers: { 'Service-Worker': 'script' } 105 | }) 106 | .then(response => { 107 | // Ensure service worker exists, and that we really are getting a JS file. 108 | const contentType = response.headers.get('content-type'); 109 | if ( 110 | response.status === 404 || 111 | (contentType != null && contentType.indexOf('javascript') === -1) 112 | ) { 113 | // No service worker found. Probably a different app. Reload the page. 114 | navigator.serviceWorker.ready.then(registration => { 115 | registration.unregister().then(() => { 116 | window.location.reload(); 117 | }); 118 | }); 119 | } else { 120 | // Service worker found. Proceed as normal. 121 | registerValidSW(swUrl, config); 122 | } 123 | }) 124 | .catch(() => { 125 | console.log( 126 | 'No internet connection found. App is running in offline mode.' 127 | ); 128 | }); 129 | } 130 | 131 | export function unregister() { 132 | if ('serviceWorker' in navigator) { 133 | navigator.serviceWorker.ready 134 | .then(registration => { 135 | registration.unregister(); 136 | }) 137 | .catch(error => { 138 | console.error(error.message); 139 | }); 140 | } 141 | } 142 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/setupTests.js: -------------------------------------------------------------------------------- 1 | // jest-dom adds custom jest matchers for asserting on DOM nodes. 2 | // allows you to do things like: 3 | // expect(element).toHaveTextContent(/react/i) 4 | // learn more: https://github.com/testing-library/jest-dom 5 | import '@testing-library/jest-dom/extend-expect'; 6 | -------------------------------------------------------------------------------- /15-Kubernetes/sample_app/src/stelligent.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/15-Kubernetes/sample_app/src/stelligent.jpg -------------------------------------------------------------------------------- /16-SAM/codeDeployResources.yml: -------------------------------------------------------------------------------- 1 | beforeAllowTraffic: 2 | Type: AWS::Serverless::Function 3 | Properties: 4 | Handler: lifecycleHook.handler 5 | Policies: 6 | - Version: "2012-10-17" 7 | Statement: 8 | - Effect: "Allow" 9 | Action: 10 | - "codedeploy:PutLifecycleEventHookExecutionStatus" 11 | Resource: 12 | !Sub 'arn:aws:codedeploy:${AWS::Region}:${AWS::AccountId}:deploymentgroup:${ServerlessDeploymentApplication}/*' 13 | - Version: "2012-10-17" 14 | Statement: 15 | - Effect: "Allow" 16 | Action: 17 | - "lambda:InvokeFunction" 18 | Resource: !Ref .Version 19 | Runtime: nodejs8.10 20 | FunctionName: 'CodeDeployHook_beforeAllowTraffic' 21 | DeploymentPreference: 22 | Enabled: false 23 | Timeout: 5 24 | Environment: 25 | Variables: 26 | NewVersion: !Ref .Version 27 | 28 | afterAllowTraffic: 29 | Type: AWS::Serverless::Function 30 | Properties: 31 | Handler: lifecycleHook.handler 32 | Policies: 33 | - Version: "2012-10-17" 34 | Statement: 35 | - Effect: "Allow" 36 | Action: 37 | - "codedeploy:PutLifecycleEventHookExecutionStatus" 38 | Resource: 39 | !Sub 'arn:aws:codedeploy:${AWS::Region}:${AWS::AccountId}:deploymentgroup:${ServerlessDeploymentApplication}/*' 40 | - Version: "2012-10-17" 41 | Statement: 42 | - Effect: "Allow" 43 | Action: 44 | - "lambda:InvokeFunction" 45 | Resource: !Ref .Version 46 | Runtime: nodejs8.10 47 | FunctionName: 'CodeDeployHook_afterAllowTraffic' 48 | DeploymentPreference: 49 | Enabled: false 50 | Timeout: 5 51 | Environment: 52 | Variables: 53 | NewVersion: !Ref .Version 54 | -------------------------------------------------------------------------------- /16-SAM/create-table.json: -------------------------------------------------------------------------------- 1 | { 2 | "TableName": "", 3 | "KeySchema": [ 4 | { "AttributeName": "id", "KeyType": "HASH" } 5 | ], 6 | "AttributeDefinitions": [ 7 | { "AttributeName": "id", "AttributeType": "S" } 8 | ], 9 | "ProvisionedThroughput": { 10 | "ReadCapacityUnits": 1, 11 | "WriteCapacityUnits": 1 12 | } 13 | } 14 | -------------------------------------------------------------------------------- /16-SAM/lifecycleHook.js: -------------------------------------------------------------------------------- 1 | 'use strict'; 2 | 3 | const aws = require('aws-sdk'); 4 | const codedeploy = new aws.CodeDeploy({apiVersion: '2014-10-06'}); 5 | 6 | exports.handler = (event, context, callback) => { 7 | //Read the DeploymentId from the event payload. 8 | var deploymentId = event.DeploymentId; 9 | 10 | //Read the LifecycleEventHookExecutionId from the event payload 11 | var lifecycleEventHookExecutionId = event.LifecycleEventHookExecutionId; 12 | 13 | /* 14 | Enter validation tests here. 15 | */ 16 | 17 | // Prepare the validation test results with the deploymentId and 18 | // the lifecycleEventHookExecutionId for AWS CodeDeploy. 19 | var params = { 20 | deploymentId: deploymentId, 21 | lifecycleEventHookExecutionId: lifecycleEventHookExecutionId, 22 | status: 'Succeeded' // status can be 'Succeeded' or 'Failed' 23 | }; 24 | 25 | // Pass AWS CodeDeploy the prepared validation test results. 26 | codedeploy.putLifecycleEventHookExecutionStatus(params, function(err, data) { 27 | if (err) { 28 | // Validation failed. 29 | callback('Validation test failed'); 30 | } else { 31 | // Validation succeeded. 32 | callback(null, 'Validation test succeeded'); 33 | } 34 | }); 35 | }; 36 | -------------------------------------------------------------------------------- /18-step-functions/README.md: -------------------------------------------------------------------------------- 1 | # Topic 18: Step Functions 2 | 3 | 4 | 5 | - [Topic 18: Step Functions](#topic-18-step-functions) 6 | - [Guidance](#guidance) 7 | - [Lesson 18.1: Introduction to Step Functions](#lesson-181-introduction-to-step-functions) 8 | - [Principle 18.1](#principle-181) 9 | - [Practice 18.1](#practice-181) 10 | - [Lab 18.1.1: Create a State Machine in the AWS Console](#lab-1811-create-a-state-machine-in-the-aws-console) 11 | - [Question: Pass State](#question-pass-state) 12 | - [Question: States](#question-states) 13 | - [Question: Input and Output](#question-input-and-output) 14 | - [Retrospective 18.1](#retrospective-181) 15 | - [Question: Workflows](#question-workflows) 16 | - [Lesson 18.2: Lambda State Machine](#lesson-182-lambda-state-machine) 17 | - [Principle 18.2](#principle-182) 18 | - [Practice 18.2](#practice-182) 19 | - [Lab 18.2.1: Create a Lambda State Machine with CloudFormation](#lab-1821-create-a-lambda-state-machine-with-cloudformation) 20 | - [Question: Activity State Machine](#question-activity-state-machine) 21 | - [Retrospective 18.2](#retrospective-182) 22 | - [Question: Service Integrations](#question-service-integrations) 23 | - [Lesson 18.3: Step Functions and Events](#lesson-183-step-functions-and-events) 24 | - [Principle 18.3](#principle-183) 25 | - [Practice 18.3](#practice-183) 26 | - [Lab 18.3.1: Start an Execution in Response to a S3 Event](#lab-1831-start-an-execution-in-response-to-a-s3-event) 27 | - [Retrospective 18.3](#retrospective-183) 28 | - [Question: Other Events](#question-other-events) 29 | - [Further Reading](#further-reading) 30 | 31 | 32 | 33 | ## Guidance 34 | 35 | - Explore the official docs! See the AWS Step Functions 36 | [Developer Guide](https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html), 37 | [API Reference](https://docs.aws.amazon.com/step-functions/latest/apireference/Welcome.html), 38 | and 39 | [CLI Reference](https://docs.aws.amazon.com/cli/latest/reference/stepfunctions/index.html) 40 | docs. 41 | 42 | - Avoid using other sites like stackoverflow.com for answers \-- part 43 | of the skill set you're building is finding answers straight from 44 | the source, AWS. 45 | 46 | - Explore your curiosity. Try to understand why things work the way 47 | they do. Read more of the documentation than just what you need to 48 | find the answers. 49 | 50 | ## Lesson 18.1: Introduction to Step Functions 51 | 52 | ### Principle 18.1 53 | 54 | *AWS Step Functions is a web service that enables you to coordinate the 55 | components of distributed applications and microservices using visual 56 | workflows.* 57 | 58 | ### Practice 18.1 59 | 60 | Step Functions provides a reliable way to coordinate components and step 61 | through the functions of your application. They are based on the concepts 62 | of tasks and state machines, which are defined using the JSON-based 63 | [Amazon States Language](https://states-language.net/spec.html). The Step 64 | Functions console displays a graphical view of your state machine's 65 | structure. This provides a way to visually check your state machine's 66 | logic and monitor executions. 67 | 68 | #### Lab 18.1.1: Create a State Machine in the AWS Console 69 | 70 | Create a simple State Machine using the AWS console. 71 | 72 | - Use the default "Author with code snippets" for _Define state machine_. 73 | 74 | - Use the default "Standard" for _Type_. 75 | 76 | - Update the code snippet to accept an input JSON property named _executioner_ 77 | and output the following JSON object: 78 | { 79 | "executioner": "_Your Name_", 80 | "message": "Hello AWS!" 81 | } 82 | 83 | - Give your state machine an appropriate name. 84 | 85 | - Make a note of the new IAM role being created. You will need this when 86 | cleaning up the lab resources. 87 | 88 | - Review the results and look at the step details after executing your 89 | state machine. Does the output look as expected? 90 | 91 | - When you're done, delete the state machine and the IAM role. 92 | 93 | ##### Question: Pass State 94 | 95 | _Describe the "Pass" state and what it does._ 96 | 97 | ##### Question: States 98 | 99 | _What are some of the other states you can use in a state machine?_ 100 | 101 | ##### Question: Input and Output 102 | 103 | _Describe how input and output are handled and manipulated in the step 104 | functions._ 105 | 106 | ### Retrospective 18.1 107 | 108 | #### Question: Workflows 109 | 110 | _When choosing a [workflow](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html) 111 | when is it recommended to use an Express workflow versus the Standard 112 | workflow?_ 113 | 114 | ## Lesson 18.2: Lambda State Machine 115 | 116 | ### Principle 18.2 117 | 118 | *AWS Step Functions state machine can use an AWS Lambda function to 119 | implement a Task state.* 120 | 121 | ### Practice 18.2 122 | 123 | AWS Lambdas are well suited for implementing Task states, because 124 | Lambda functions are stateless (they have a predictable input-output 125 | relationship), easy to write, and don't require deploying code to a 126 | server instance. 127 | 128 | #### Lab 18.2.1: Create a Lambda State Machine with CloudFormation 129 | 130 | Create a CFN template that creates a state machine, a lambda, and the 131 | required IAM roles for those services when the stack is created: 132 | 133 | - Lambda should return "Hello AWS!". 134 | 135 | - The state machine should use the Lambda function to implement a task 136 | state. 137 | 138 | - State machine execution should output "Hello AWS!" 139 | 140 | ##### Question: Activity State Machine 141 | 142 | _What are some use cases for using an 143 | [Activity State Machine](https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-creating-activity-state-machine.html) 144 | instead of the Lambda State Machine?_ 145 | 146 | ### Retrospective 18.2 147 | 148 | #### Question: Service Integrations 149 | 150 | _What are some other supported AWS service integrations for Step Functions?_ 151 | 152 | ## Lesson 18.3: Step Functions and Events 153 | 154 | ### Principle 18.3 155 | 156 | *You can use Amazon CloudWatch Events to execute an AWS Step Functions 157 | state machine in response to an event or on a schedule.* 158 | 159 | ### Practice 18.3 160 | 161 | CloudWatch Events, such as uploading a new object to a S3 bucket, can be 162 | used to trigger the execution of a state machine. This type of function 163 | could be useful if you needed to perform operations on files that are 164 | added to a bucket. This is also only a single example of the many types of 165 | events that can trigger a state machine. 166 | 167 | #### Lab 18.3.1: Start an Execution in Response to a S3 Event 168 | 169 | Using the CFN template from the previous lab, make the following changes: 170 | 171 | - Create a new S3 bucket. 172 | 173 | - Create a new trail in CloudTrail that logs the events that occur in 174 | your new S3 bucket. 175 | 176 | - Create a new CloudWatch rule to trigger the state machine when a file is 177 | uploaded to your new S3 bucket. 178 | 179 | - Update the Lambda to return the name of the file uploaded. 180 | 181 | - After the stack is updated, upload a file to your new S3 bucket and 182 | then review the execution log for your state machine execution. 183 | 184 | - When you're done, delete the stack and all resources you created for 185 | these labs. 186 | 187 | ### Retrospective 18.3 188 | 189 | #### Question: Other Events 190 | 191 | _What are other types of events that can be used to trigger a state 192 | machine execution?_ 193 | 194 | ## Further Reading 195 | 196 | - Read about [SAM integration](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-lambda.html) 197 | for testing your step functions locally. 198 | 199 | - Learn more about [handling error conditions](https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-handling-error-conditions.html) 200 | in a state machine. 201 | 202 | - Review [Step Functions Best Practices](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-best-practices.html) 203 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2020 Mphasis Stelligent 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /MVE.md: -------------------------------------------------------------------------------- 1 | # Stelligent's Minimum Viable Engineer 2 | 3 | The goal of Stelligent U is to prepare all of our engineers for their first 4 | engagements as Stelligent consultants through practical, hands-on experience. 5 | Any topic or principal presented here is part of our technical knowledge 6 | baseline: this is what we consider a _Minimum Viable Engineer_. 7 | 8 | What do we mean by that? 9 | 10 | Stelligent is a consulting company and our core expertise is DevOps automation 11 | in AWS. Let's start by applying the phrase "minimum viable engineer" at face 12 | value. We're talking about a software professional with the baseline amount of 13 | skill we're willing to put in front of a client. We can expand on that, though. 14 | We wanted to establish a number of key characteristics that build towards some 15 | core ideas. What are the best components of who we are? How can we ensure that 16 | our clients consistently experience excellence? How can we maintain our core 17 | ideals over the long term? These questions led us to some essential 18 | characteristics we thought we’d build into our program: 19 | 20 | 1. A course of material that can be improved and updated over time. Putting all 21 | of our training into a clearly-defined program lets us iterate on it. Ideally, 22 | that means we’re always making it better, too. Let’s apply continuous 23 | improvement everywhere we can! 24 | 25 | 1. Readily reproducible. Any of our experienced engineers should be able to 26 | teach the program. Although not everyone is naturally a good teacher, we’ve 27 | found that the same traits and skills make both good teachers and good 28 | consultants, and they’re the same qualities that we hire for. 29 | 30 | 1. A consistent onboarding journey. We want to make sure everyone gets the best 31 | new-hire experience we can give them, and consistency of delivery is a 32 | fundamental part of that. This carries forward into consistency with our 33 | clients, as well, and builds a reliable foundation of excellence for the work 34 | we provide to companies. 35 | 36 | 1. Reproducing our technical and cultural DNA. If we want to keep our core 37 | identity over time, there are some fundamentals we want to ensure everybody 38 | has: strength in AWS, skill with common DevOps tooling, and insight into our 39 | foundational DevOps principles & techniques. 40 | 41 | 1. A clear technical threshold. As much as we believe in the people we’ve just 42 | hired, they’re unproven. Which leads us to... 43 | 44 | 1. A promise of quality to the sales, management, and engineering staff. The 45 | program gives individuals a chance to prove to themselves and others that 46 | they meet our bar before we ask them to do the same in front of clients. 47 | 48 | To get all of that value, we pored over our common technological challenges and 49 | sought deep within our engineer hearts to find all the core ideas and skills 50 | that meant the most to us. We selected twelve essential AWS technologies that we 51 | wanted every one of our engineers to have multifaceted experience with. Nobody 52 | leaves our training program before they've got their hands dirty with 53 | CloudFormation, S3, IAM, VPCs, EC2, Auto Scaling, Load Balancing, CloudWatch 54 | Logs, Lambda, KMS, Parameter Store, and CodePipeline. If they've done 55 | everything, they'll have diverse experience with each of those, exercising 56 | CloudFormation, the awscli, and perhaps the API in each AWS service. 57 | 58 | Reducing AWS to a set of essentials is not easy, and we left a lot of important 59 | technologies out. We just couldn't fit it all in, though, and this is what we 60 | decided we absolutely couldn't spare. As it is, the program takes a number of 61 | weeks to work through. We've continued to add new modules to the initial core 62 | dozen. Our engineers come back to them as "continuous learning modules" when 63 | they're ready to learn a new technology. 64 | 65 | ## Graduating from Stelligent U 66 | 67 | This hands-on technical curriculum is part of our new engineer onboarding 68 | program. Because of that, we apply the notion of "graduating" from Stelligent U. 69 | Graduating symbolizes a number of important things to us, and it marks a solid 70 | transition point for any new engineer. 71 | 72 | At the simplest level, providing a curriculum and asking new hires to complete 73 | it gives them a goal. Once it's completed, our new engineers can mark a valuable 74 | moment of success here. We all get to celebrate with them, and we love doing it 75 | publicly at our weekly all-hands, "Sharing is Caring", so that we can all give 76 | them some well-deserved kudos for completing the hard work. It's also a nice 77 | feel-good moment that helps reintroduce a new face to the group shortly after 78 | they were welcomed on their first week. 79 | 80 | Achieving that goal does a few more valuable things. It builds the engineer's 81 | confidence that they can succeed in their new environment. Because it's 82 | something we've established as our core technical identity, it also helps them 83 | internalize that they really do fit in here. It should work against that little 84 | voice behind imposter syndrome. It's not easy making the transition from 85 | internal engineering to consulting engineering, and it can be scary. If you can 86 | finish Stelligent U, you're absolutely qualified to be an engineer here and 87 | speak for the company when you talk to clients! 88 | 89 | Graduating establishes another important thing: it communicates internally, to 90 | the rest of the company, that the engineer is ready to speak for us. The folks 91 | on the sales team know with confidence they've got another qualified engineer 92 | ready to go. The managers know they have a strong performer who can hit the 93 | ground running on their projects. The other engineers know the new person will 94 | be able to hold up their end if they get teamed up. Honestly, we don't really 95 | even think about it later. We know the people here are qualified -- day to day, 96 | it's a safe assumption that our coworkers are going to be awesome. 97 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | # Welcome to Stelligent U 4 | 5 | Welcome to the technical side of Stelligent University, 6 | [Stelligent's](https://stelligent.com) onboarding program for engineers. This 7 | repo includes a series of learning modules designed to give cloud engineers 8 | practical experience working with AWS and related technologies. 9 | 10 | The goal of this material is to prepare all of our engineers for their first 11 | engagements as Stelligent consultants. Any topic or principal presented here 12 | is part of our technical knowledge baseline: this is what we consider a 13 | "Minimum Viable Engineer". There are a _lot_ of other ideas and technologies 14 | that you'll come across and need to know in your work. We can't fit it all in, 15 | though, and these are the essentials. 16 | 17 | For further information, please see: 18 | 19 | * [MVE.md](MVE.md): what we mean by "Minimum Viable Engineer" 20 | * [WORKFLOW.md](WORKFLOW.md): how we suggest you use Stelligent U 21 | 22 | ## Audience 23 | 24 | This is a series of lessons originally written for new Engineers at 25 | Stelligent. The goal of this material is to prepare all of our engineers for 26 | their first engagements as Stelligent engineers. Any topic or principal 27 | presented here is part of our technical knowledge baseline: this is what we 28 | consider a _Minimum Viable Engineer_. There are a _lot_ of other ideas and 29 | technologies that you'll come across and need to know in your career. 30 | We can't fit it all in, though, and these are the essentials. 31 | 32 | ## Materials 33 | 34 | The core of Stelligent U is modules 1-12, the series that makes up our baseline 35 | definition of a Minimum Viable Engineer. Other modules are also available that 36 | provide hands-on experience with tech beyond the baseline. We want to add more 37 | and more to those "continuous learning" modules as time goes by. 38 | 39 | You'll find a handful of ways we present each topic. Most of the technical 40 | exercises are labs, where we want you to gain a surface level of exposure to 41 | a variety of AWS services and their most common or compelling features. 42 | 43 | Labs within a lesson build on each other. Many lessons require the experience 44 | of previous lessons. Unless you're completely blocked, work through them in 45 | order. 46 | 47 | When we provide materials for a lab -- e.g. CloudFormation templates or 48 | policy files -- start with those and add to them as requested. 49 | 50 | Some topics also include retrospectives. These aren't always focused so much 51 | on a technical exercise. Our goal here is usually to get you to think more 52 | broadly about the technology at hand. 53 | 54 | We also provide further reading for each topic. Find time to explore some of 55 | these materials more deeply. Pursue your curiosity. There are many excellent 56 | resources out there, and we particularly want you to learn more where the topics 57 | match up with your interests. 58 | -------------------------------------------------------------------------------- /WORKFLOW.md: -------------------------------------------------------------------------------- 1 | # Working with Stelligent U 2 | 3 | Our program isn't a step by step "how-to". The modules generally don't say much 4 | on how to accomplish specific tasks; instead they walk you through goals in a 5 | sensible order, provide insight on unique challenges, and give pointers to the 6 | solutions. They aim to teach you how to learn AWS, constantly encouraging you to 7 | use Amazon's docs directly. Learning from the source of truth is always the 8 | best, and with AWS docs, it is certainly a skill unto itself. As the poet 9 | Basho once said, "Do not seek to follow in the footsteps of the wise. Seek what 10 | they sought." 11 | 12 | At Stelligent, we set aside four weeks for new engineers to onboard and go 13 | through the core modules. While we pair new engineers up with a mentor who has 14 | gone through the course and is more knowledgeable and experienced with AWS, the 15 | majority of the material is completed by individuals. This course was designed 16 | to be complementary to other studies and coursework and to fill a niche we felt 17 | was not available elsewhere. 18 | 19 | Digging deeper is strongly encouraged. If you want to explore StackSets in the 20 | Cloudformation module, cross account role assumptions in the IAM module, or use 21 | an API Gateway in conjunction with the Lambda module, please do explore that 22 | frontier. We built Stelligent U to focus on learning how to learn, and deeper, 23 | unguided exploration is one of the best ways to do so. 24 | 25 | ## Getting Started: Duplicate this Repo 26 | 27 | 1. Click the "Use this template" button near the top of our repo page to create 28 | your repository: 29 | 30 | ![use-this-template](images/use-this-template.png) 31 | 32 | 1. For _Owner_ on the page that comes up, be sure to choose your repo space 33 | rather than Stelligent's. 34 | 35 | ![create new repo](images/create-new-repo.png) 36 | 37 | ## Working on Lessons 38 | 39 | Each subdirectory is its own topic. They're numbered so that you can progress 40 | through them in a logical order. Later lessons will build on previous ones. 41 | 42 | Within each topic's README, you will find a series of lessons and labs. 43 | As you go through them, save any code you write: scripts, templates, policy 44 | documents, etc. Add them to your copy of this repo. Answer questions inline 45 | using the Markdown quote prefix ("> ") to identify your own additions to the 46 | README files. 47 | 48 | ## General Tips 49 | 50 | - Use a Markdown previewer and linter so formatting your answers is easier. 51 | - Atom: 52 | - [Lint](https://atom.io/packages/linter-markdown) 53 | - [Preview](https://atom.io/packages/markdown-it-preview) 54 | - Sublime Text: 55 | - [Lint](https://packagecontrol.io/packages/SublimeLinter-contrib-markdownlint) 56 | - [Preview](https://packagecontrol.io/packages/MarkdownPreview) 57 | - VSCode: 58 | - [Lint](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) 59 | - Markdown Preview is built in 60 | - macOS: 61 | - Brett Terpstra's excellent [Marked 2](https://marked2app.com/) provides both 62 | 63 | - [cfn-lint](https://github.com/aws-cloudformation/cfn-python-lint) is very 64 | handy for lessons that involve CloudFormation. 65 | - [Atom](https://atom.io/packages/atom-cfn-lint) 66 | - [Sublime Text](https://packagecontrol.io/packages/SublimeLinter-contrib-cloudformation) 67 | - [VSCode](https://marketplace.visualstudio.com/items?itemName=kddejong.vscode-cfn-lint) 68 | 69 | - When creating AWS resources for these lessons be mindful of how they are 70 | named. If you're working within an AWS account that's shared with others, 71 | include your name or some other unique identifier to avoid colliding with 72 | others who may be working on the same module. 73 | 74 | - Stelligent's sandbox account is "cleaned" every weekend and most resources 75 | are automatically deleted. If you have a solution like 76 | [aws-nuke](https://github.com/rebuy-de/aws-nuke), 77 | [cloud-custodian](https://cloudcustodian.io/), or 78 | [DivvyCloud](https://divvycloud.com/) running in your account, 79 | it can be great for infrastructure-as-code and pipeline deployment discipline, 80 | but it can also be really frustrating if you're not ready for that. Just be 81 | sure you're aware of the automation running and account for it in your 82 | deployment workflow. Most importantly, keep everything in code and be sure 83 | to push it from your dev computer to your upstream repo often! 84 | 85 | ## Finishing a Lesson 86 | 87 | When you're ready to review a lesson with your mentor, create a branch, push 88 | your changes to it, and create a pull request to the `master` branch of the 89 | repository you created from the template. 90 | 91 | (NOTE: please _do not_ open a PR back to the `stelligent/stelligent-u` repo.) 92 | 93 | ![pr example](images/pr-example.png) 94 | -------------------------------------------------------------------------------- /images/create-new-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/images/create-new-repo.png -------------------------------------------------------------------------------- /images/generated-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/images/generated-repo.png -------------------------------------------------------------------------------- /images/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/images/logo.png -------------------------------------------------------------------------------- /images/pr-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/images/pr-example.png -------------------------------------------------------------------------------- /images/use-this-template.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/stelligent/stelligent-u/51429ffb733a8c9b2b789ff2b9315dd8594bcf54/images/use-this-template.png -------------------------------------------------------------------------------- /templates/NN-topic-name/README.md: -------------------------------------------------------------------------------- 1 | # Topic NN: Topic Name 2 | 3 | 4 | 5 | - [Guidance](#guidance) 6 | - [Lesson NN.1: Introduction to Topic name](#lesson-NN1-introduction-to-topic-name) 7 | - [Principle NN.1](#principle-NN1) 8 | - [Practice NN.1](#practice-NN1) 9 | - [Lab NN.1.1: Exercise Title](#lab-NN11-exercise-title) 10 | - [Question: Question Title](#question-question-title) 11 | - [Lab NN.1.2: Exercise Title](#lab-NN12-exercise-title) 12 | - [Question: Another Question](#question-another-question) 13 | - [Lab NN.1.3: Exercise Title](#lab-NN13-exercise-title) 14 | - [Retrospective NN.1](#retrospective-NN1) 15 | - [Question: Last Question](#last-question) 16 | - [Further Reading](#further-reading) 17 | 18 | 19 | 20 | ## Guidance 21 | 22 | - Explore the official docs! See the the Topic Name 23 | [User Guide](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html), 24 | [API Reference](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameters.html), 25 | [CLI Reference](https://docs.aws.amazon.com/cli/latest/reference/ssm/index.html), 26 | and 27 | [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 28 | docs. 29 | 30 | - Avoid using other sites like stackoverflow.com for answers \-- part 31 | of the skill set you're building is finding answers straight from 32 | the source, AWS. 33 | 34 | - Explore your curiosity. Try to understand why things work the way 35 | they do. Read more of the documentation than just what you need to 36 | find the answers. 37 | 38 | ## Lesson NN.1: Introduction to Topic Name 39 | 40 | ### Principle NN.1 41 | 42 | *Write a single, short, memorable concept about the following lessons 43 | here.* 44 | 45 | ### Practice NN.1 46 | 47 | *Give an overview of how the following labs will reinforce the principle. 48 | It shouldn't be more than a paragraph or two.* 49 | 50 | *When writing labs for the principle, keep each one brief and tightly focused. 51 | As the student moves through them, they should build knowledge once discrete 52 | piece at a time. By the time they're done with all the labs for the principle, 53 | they should have enough fullness in their understanding that they'll feel 54 | confident knowing where to learn more, as well as knowing that they can 55 | take on a project with the given technologies and learn what else they need 56 | to know with reasonable efficiency.* 57 | 58 | #### Lab NN.1.1: Exercise Title 59 | 60 | *Each lab should illustrate one key aspect of the topic at hand. 61 | Don't put too much in each lab, so that the code resulting from 62 | a lab isn't burdensome for the mentor and student to review together.* 63 | 64 | ##### Question: Question Title 65 | 66 | *At the end of each lab, consider asking 1 or more simple questions that 67 | help reinforce the aspect of the principle that was just exercised.* 68 | 69 | #### Lab NN.1.2: Exercise Title 70 | 71 | ... 72 | 73 | ##### Question: Another Question 74 | 75 | ... 76 | 77 | #### Lab NN.1.3: Exercise Title 78 | 79 | ... 80 | 81 | ### Retrospective 82 | 83 | *The retrospective is designed to create a conversation point for 84 | further discussion between the mentor and mentee. The questions 85 | here should provide good food for thought. They should challenge 86 | the way the student solved the previous problems, seeking to ensure 87 | they understand why they got the results they did. Consider asking 88 | questions that play off common assumptions or call attention to 89 | nuance in the solutions that might not have been obvious but is 90 | valuable to know.* 91 | 92 | #### Last Question 93 | 94 | ... 95 | 96 | ## Further Reading 97 | 98 | - *Link to AWS blogs and other relevant sources that encourage 99 | the reader to explore their curiousity.* 100 | -------------------------------------------------------------------------------- /templates/README.md: -------------------------------------------------------------------------------- 1 | This directory holds templates to simplify creating new content. 2 | -------------------------------------------------------------------------------- /templates/topic-name/README.md: -------------------------------------------------------------------------------- 1 | # Topic: Topic Name 2 | 3 | 4 | 5 | - [Guidance](#guidance) 6 | - [Lesson 1: Introduction to Topic name](#lesson-1-introduction-to-topic-name) 7 | - [Principle 1](#principle-1) 8 | - [Practice 1](#practice-1) 9 | - [Lab 1.1: Storing Data](#lab-11-storing-data) 10 | - [Lab 1.2: Reading Data](#lab-12-reading-data) 11 | - [Question: CloudFormation Console](#question-cloudformation-console) 12 | - [Lab 1.3: Integration with CloudFormation](#lab-13-integration-with-cloudformation) 13 | - [Retrospective](#retrospective) 14 | - [Question 1](#question-1) 15 | - [Question 2](#question-2) 16 | - [Further Reading](#further-reading) 17 | 18 | 19 | 20 | ## Guidance 21 | 22 | - Explore the official docs! See the the Topic Name 23 | [User Guide](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html), 24 | [API Reference](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameters.html), 25 | [CLI Reference](https://docs.aws.amazon.com/cli/latest/reference/ssm/index.html), 26 | and 27 | [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 28 | docs. 29 | 30 | - Avoid using other sites like stackoverflow.com for answers \-- part 31 | of the skill set you're building is finding answers straight from 32 | the source, AWS. 33 | 34 | - Explore your curiosity. Try to understand why things work the way 35 | they do. Read more of the documentation than just what you need to 36 | find the answers. 37 | 38 | ## Lesson 1: Introduction to Topic Name 39 | 40 | ### Principle 1 41 | 42 | *Parameter Store is a key-value store that is compelling and powerful in 43 | its simplicity.* 44 | 45 | ### Practice 1 46 | 47 | Let's start out by creating a few parameters and finding ways we can 48 | read them. In the labs below, keep a few things in mind: 49 | 50 | - Parameter Store keys are naturally split by `/` into a hierarchy (but 51 | [it's not necessary](https://aws.amazon.com/blogs/mt/organize-parameters-by-hierarchy-tags-or-amazon-cloudwatch-events-with-amazon-ec2-systems-manager-parameter-store/). 52 | You can use this to namespace your keys. 53 | 54 | - Because we work in a shared account, namespacing is important. 55 | 56 | - Parameter Store data is unique per region, but not per availability 57 | zone, and it would be easy to make your keys overlap with somebody 58 | else's if you're all working in the same region in the same time 59 | period. 60 | 61 | - To establish a convention where we avoid overlapping with other's 62 | work, prefix your keys with 63 | `/*your-aws-username*/onboarding/lab11/`. 64 | 65 | #### Lab 1.1: Storing Data 66 | 67 | Using the data in [For The People](https://forthepeople.stelligent.com) as 68 | inspiration, create a new CloudFormation stack that can store a handful 69 | of data points for a Stelligent engineer in Parameter Store. Make it 70 | generic, so that each data point is passed as a Stack parameter, as if 71 | the stack template might be used to store info about each of our 72 | engineers. 73 | 74 | - Create a sub-tree of 75 | [AWS::SSM::Parameter](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 76 | resources that will store information on a single engineer. 77 | - Every Parameter Store value type can be a string. 78 | - At the top of the sub-tree, use a name for the engineer as your key 79 | value. See the [Name constraints](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_PutParameter.html#systemsmanager-PutParameter-request-Name) 80 | in the PutParameter API docs for limitations. Try to enforce some 81 | of those limitations in your stack's [parameter properties](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html), 82 | too. (E.g. set an AllowedPattern and MaxLength. Can you set any 83 | other limitations?). 84 | - Store all other keys under that. 85 | - Store the engineer's customer/team name under "team". 86 | - Store their timezone under "timezone". 87 | - Store their home state as a 2-letter prefix under "state". 88 | - Store their start date under "start-date". 89 | 90 | Launch the stack with a parameter file that sets information for 91 | yourself. 92 | 93 | #### Lab 1.2: Reading Data 94 | 95 | Look at ways to read your parameter data. 96 | 97 | - You can look it up in the 98 | [console](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Parameters:) 99 | 100 | - You can read it with the API using 101 | [GetParameter](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameter.html), 102 | [GetParameters](https://boto3.readthedocs.io/en/latest/reference/services/ssm.html#SSM.Client.get_parameters) 103 | and 104 | [GetParametersByPath](https://boto3.readthedocs.io/en/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path) 105 | 106 | - You can invoke those same API queries through the 107 | [aws ssm](https://docs.aws.amazon.com/cli/latest/reference/ssm/index.html) 108 | CLI 109 | 110 | Use all 3 methods to read your parameters individually and to fetch the 111 | entire subtree with a single query. 112 | 113 | ##### Question: CloudFormation Console 114 | 115 | _When you look at your stack in the CloudFormation console, can you find 116 | the values of your parameter resources there?_ 117 | 118 | #### Lab 1.3: Integration with CloudFormation 119 | 120 | CloudFormation can use Parameter Store keys and values as 121 | [stack parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types). 122 | This gives us an incredibly convenient way to maintain stack parameters 123 | without the hassle of updating JSON files and keeping them in sync 124 | through S3 or git repositories. 125 | 126 | Make a copy of the original EC2 stack from the Load Balancing topic. 127 | Instead of creating a web server with a page that says "Automation for 128 | the People", make one that describes an engineer. 129 | 130 | - One parameter should be an "AWS::SSM::Parameter::Name" that will 131 | reference the top-level key of your hierarchy as set in your stack 132 | from lab 1. 133 | 134 | - Accept an "AWS::SSM::Parameter::Value" type for each of the Param 135 | Store values that you set in your stack from lab 1. 136 | 137 | - Show the value of each of those stack parameters in the web page 138 | that nginx serves. 139 | 140 | When you launch your stack, make sure the web page shows the values of 141 | all the keys that you set in lab 1. 142 | 143 | As follow-up to the question above about finding the values of your 144 | parameter resources, take note of a comment in the SSM Parameter Types 145 | document linked in the previous paragraph: 146 | 147 | > You can see the resolved values for SSM parameters on the stack's 148 | > Parameters tab in the console, or by running 149 | > [describe-stacks](http://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stacks.html) 150 | > or 151 | > [describe-change-set](http://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-change-set.html). 152 | > These are the values that are currently used in the stack definition 153 | > for the corresponding Systems Manager parameter keys. Note that these 154 | > values are set when the stack is created or updated, so they might 155 | > differ from the latest values in Parameter Store. 156 | 157 | ### Retrospective 158 | 159 | ... 160 | 161 | #### Question 1 162 | 163 | _Read [Using Dynamic References to Specify Template Values](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html). 164 | Why can't you use that feature directly to read "middle-name" and show 165 | it in your web page?_ 166 | 167 | #### Question 2 168 | 169 | _Can you use Secure String as an `AWS::SSM::Parameter::Value` type in a 170 | CloudFormation stack?_ 171 | 172 | ## Further Reading 173 | 174 | - Amazon recently [introduced](https://aws.amazon.com/blogs/aws/aws-secrets-manager-store-distribute-and-rotate-credentials-securely/) 175 | [Secrets Manager](https://aws.amazon.com/secrets-manager/). It 176 | integrates Parameter Store, Lambda, and a few other services into 177 | a product that makes password maintenance and rotation easier to do. 178 | Take a look at the [FAQ](https://aws.amazon.com/secrets-manager/faqs/) 179 | for a succinct summary of the service. 180 | 181 | - [The Right Way to Store Secrets Using Parameter Store](https://aws.amazon.com/blogs/mt/the-right-way-to-store-secrets-using-parameter-store/), 182 | in the AWS Tools Blog, covers Parameter Store usage nicely, and 183 | introduces the IAM permissions that are required to managed it as well. 184 | --------------------------------------------------------------------------------