└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Open CDK Guide 2 | 3 | # Table of Contents 4 | 5 | 6 | 7 | - [Purpose](#purpose) 8 | * [Why This Guide?](#why-this-guide) 9 | * [Contributions](#contributions) 10 | * [Credits](#credits) 11 | * [Legend](#legend) 12 | - [Why CDK?](#why-cdk) 13 | - [Tips and Best Practices](#tips-and-best-practices) 14 | * [Constructs](#constructs) 15 | * [Stacks](#stacks) 16 | * [Structure](#structure) 17 | * [Naming](#naming) 18 | * [Tagging](#tagging) 19 | * [Config](#config) 20 | * [Patterns](#patterns) 21 | * [Tools and Libraries](#tools-and-libraries) 22 | * [Deployments](#deployments) 23 | * [Limitations](#limitations) 24 | * [Get Help](#get-help) 25 | * [Further Reading](#further-reading) 26 | - [Legal](#legal) 27 | * [Disclaimer](#disclaimer) 28 | * [License](#license) 29 | 30 | 31 | 32 | --- 33 | # Purpose 34 | 35 | ## Why This Guide? 36 | 37 | The [AWS CloudDevelopment Kit](https://github.com/aws/aws-cdk) (CDK) is a framework built on top of CloudFormation that makes it delightful for users to manage AWS [Infrastructure as Code](https://en.wikipedia.org/wiki/Infrastructure_as_code) (IaC). When everything is going right, the CDK will make you feel like a devops wizard. That being said, the cloud is complicated, CloudFormation coverage of AWS is incomplete, and the CDK itself (and IaC in general) is still a young framework with little in the way of established best practices. 38 | 39 | This guide is an opinionated set of tips and best practices for working with the CDK. It is meant to be a living document, updated on an ongoing basis by the community as the CDK and practices around it mature. 40 | 41 | This guide assumes that you are familiar with CDK concepts or have gone through the [getting started documentation](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html) 42 | 43 | Before using the guide, please read the [license](#license) and [disclaimer](#disclaimer). 44 | 45 | ## Contributions 46 | 47 | Trying to keep up with AWS is a [Sisyphean](https://en.wikipedia.org/wiki/Sisyphus) task and this guide is far from complete. If you have something to add, please submit a pull request to our [github repo](https://github.com/kevinslin/open-cdk) and help define conventions of the CDK. 48 | 49 | ## Credits 50 | 51 | This guide was started by [Kevin S Lin](https://kevinslin.com), an early adopter of the CDK. It is heavily inspired in format and layout from the [og-aws guide](https://github.com/open-guides/og-aws). 52 | 53 | Below are a list of awesome folks who have helped out on the project: 54 | - [James Baker](https://github.com/nzspambot) 55 | - [Forrest C. Shields II](https://github.com/fshields) 56 | 57 | ## Legend 58 | - 🔸 A gotcha, limitation, or quirk 59 | - 🚧 Areas where correction or improvement are needed 60 | - 🔦 Hard to find feature 61 | - 🚪 Third party library or service solutions 62 | 63 | --- 64 | 65 | # Why CDK? 66 | 67 | Before diving into best practices, a question that naturally arises when considering a new IaC in 2019 is why. AWS already has CloudFormation and there are many third party solutions (eg. Terraform, Troposphere, etc). This section exists to highlight reasons for considering. 68 | 69 | - true Infrastructure as **code** 70 | - no configs or config like languages, use actual code to create infrastructure 71 | - full power of programming language at your disposal including conditionals, loops, string interpolation, etc. 72 | - see [here](https://kevinslin.com/aws/cdk_all_the_things/#just-code) for some examples 73 | 74 | - type checking for infrastructure code 75 | - aws service properties are mapped to explicit types and results in compile time error for wrong properties 76 | 77 | - superior cloudformation tooling 78 | - cdk deploy combines `create-changeset`, `execute-changeset`, `delete-changeset`, and live tailing of `describe-stack-events` 79 | - cdk diff creates a diff that actually show line item IAM and AWS resource changes (see [here](https://kevinslin.com/aws/cdk_all_the_things/#updating-a-service) for example) 80 | 81 | - CDK lets you do things that are impossible to do in CloudFormation without relying on multiple stacks/multiple deployments with flags/ hard coding resource names/creating your own custom resources 82 | - eg1. create an s3 bucket notification that triggers a lambda. see entire code below 83 | - for why this is hard using cloudformation, see this [post](https://read.acloud.guru/cloudformation-is-an-infrastructure-graph-management-service-and-needs-to-act-more-like-it-fa234e567c82) 84 | ```typescript 85 | const bucket = new Bucket(this, 'fooBucket'); 86 | 87 | const fn = new lambda.Function(this, 'fooFunc', { 88 | runtime: lambda.Runtime.NODEJS_10_X, 89 | handler: 'lambda.handler', 90 | code: lambda.Code.asset('functions/fooFunc'), 91 | timeout: Duration.seconds(60) 92 | }); 93 | 94 | bucket.addEventNotification( 95 | EventType.OBJECT_CREATED, 96 | new s3n.LambdaDestination(fn) 97 | ); 98 | ``` 99 | - eg2. create a new cert using CertificateManager and [automatically validate it with route53](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-certificatemanager-readme.html#automatic-dns-validated-certificates-using-route53) 100 | 101 | - for features that CDK doesn't support, CDK offers many [escape hatches](https://docs.aws.amazon.com/cdk/latest/guide/cfn_layer.html) 102 | 103 | - even if you don't plan on using CDK, the library is a superb reference for cloudformation documentation 104 | - all generated constructs come with property descriptions and links back to the original cloudformation documentation 105 | - eg. [definition for s3 bucket](https://gist.github.com/kevinslin/aca4eb60f01735545ed30a4fe7668178) 106 | 107 | --- 108 | 109 | # Tips and Best Practices 110 | 111 | ## Constructs 112 | - CDK has three types of [constructs](https://docs.aws.amazon.com/cdk/latest/guide/constructs.html) 113 | - Low Level constructs which are automatically generated from and map 1:1 to all CloudFormation resources and properties - they have a `CfnXXX` prefix 114 | - High Level constructs that initialize CFN resources with sane defaults and provide a high level interface to said resource 115 | - CDK pattern constructs which stitch together multiple resources together 116 | - eg. [LambdaRestApi](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-apigateway.LambdaRestApi.html) which creates an api gateway that's backed by Lambda 117 | 118 | - high level constructs 119 | - 🔸 read the small print in high level constructs, sometimes the default deployment is overkill for small projects 120 | - eg. [default vpc](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.Vpc.html) will create a NAT gateway in all AZs of the given region 121 | - a NAT gateway is $0.045/h and an additional $0.045 per GB processed 122 | - creating a default VPC in us-west-2 which has 4 AZs would result in a monthly charge of $0.045 * 24 * 30 * 4 = $129.60 from idle NAT gateways alone 123 | - 🔸some constructs take raw strings to initialize (eg. `ServicePrinciple`, `ManagedPolicy`) 124 | - these constructs are not checked and errors in the name will only show up in deploy time 125 | - 🚪[cdk-constants](https://github.com/kevinslin/cdk-constants) is a collection of constants to mitigate this exact issue 126 | - most constructs allow importing existing resources via a `{construct}.from[Name|Attr|Name|...}` pattern 127 | ```typescript 128 | let importedBucket = Bucket.fromBucketName(scope, "fooBucket") 129 | ``` 130 | - 🔸 constructs imported in this manner are not managed by the CDK but their properties can be read and used by the CDK (eg. existing S3 bucket to be used to store codebuild artifacts) 131 | - 🔸constructs imported in this manner have the signature of `I{Construct}` (eg. an imported bucket has the signature of `IBucket`) 132 | - can be an issue if your functions expect concrete construct types as IBucket is not a valid Bucket 133 | - 🔦 cross service integrations are packaged together into separate packages 134 | - you will generally find integrations with a `-targets` or `-actions` suffix 135 | - see [aws-route53-targets](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-route53-targets-readme.html), [aws-codepipeline-actions](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-codepipeline-actions-readme.html) 136 | - even though the CDK is now generally available, many of the constructs in the library are not. check the module documentation for status of specific modules (eg. [DocumentDB is still experimental]( https://docs.aws.amazon.com/cdk/api/latest/docs/aws-docdb-readme.html )) 137 | 138 | - low level constructs 139 | - working with CFN resources is just like working with cloudformation, except instead of json/yaml, you get to write type checked code 140 | - use [cloudformation reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html) to find property values 141 | - querying cloudtrail logs is great to find additional IAM permissions necessary to launch a service 142 | - 🚪[former2](https://former2.com/) is a tool created by the brilliant [Ian Mckay](https://github.com/iann0036) that can generate CDK/cloudformation/terraform from your existing infrastructure 143 | 144 | ## Stacks 145 | 146 | - a [stack](https://docs.aws.amazon.com/cdk/latest/guide/stacks.html) is the basic unit of deployment in the CDK 147 | - stacks are composable - one stack can consist of multiple stacks 148 | - stacks can share values - one stack can consume resources created in another stack (behind the scenes, CDK generates cloudformation [exports](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-exports.html) and [Fn::ImportValue](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html) to pass around values) 149 | - a useful mental model of thinking about stacks is to separate them as `outer` and `inner` 150 | - outer stacks describe the stack purpose (eg. service stack) 151 | - inner stack are a component that makes up the outer stack (eg. monitoring) 152 | - common outer stacks 153 | - foundation: vpc, security groups, etc 154 | - tools: ci/cd, automation, opsworks, etc 155 | - service|app: your user facing service or app 156 | - business|analytics: aws budgets, quicksignt, etc 157 | - security: config rules, acm certs, etc 158 | - ml|data pipeline: sagemaker, athena, etc 159 | - common inner stacks 160 | - for applications 161 | - frontend 162 | - backend 163 | - database 164 | - for services 165 | - control plane 166 | - data plane 167 | - for any all stacks 168 | - logs and monitoring 169 | - automation 170 | 171 | ## Structure 172 | - split the `bin` directory so you have one command per app 173 | - split the `lib` by application 174 | - create a `functions` directory if you plan on importing [lambda functions](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html) via code assets 175 | ``` 176 | - bin/ 177 | - create-serviceA.ts 178 | - create-serviceB.ts 179 | - lib 180 | - serviceA/ 181 | - serviceB/ 182 | - common/ 183 | - functions/ 184 | - func1/ 185 | - func2 186 | ``` 187 | 188 | ## Naming 189 | - have a consistent naming scheme for stack ids 190 | - use a fully qualified naming scheme 191 | - eg. {org}-{app}-{stage} 192 | - this value will be prepended to all constructs created inside the stack 193 | ```typescript 194 | const app = new cdk.App(); 195 | const stage = app.node.tryGetContext('stage') 196 | const infra = new InfraStack(app, `fooOrg-fooApp-prod`, {...}) 197 | // a bucket defined inside InfraStack (eg. new Bucket(this, userImages)) 198 | // will have the following name -> fooOrg-fooApp-dev-userImages87437600-kke5zcq506k7 199 | ``` 200 | - use simple ids for constructs 201 | - stick to camelCase with no spaces, punctuation or underscores 202 | - every AWS service has different naming restrictions so its best to use a compatible default naming scheme 203 | - resist temptation to name resources directly 204 | - cdk/cloudformation will generate a unique name based on the construct `id` 205 | - explicitly defining a name will cause operations that require `Replacement` to fail 206 | - eg. changing primary key of dynamo table requires a replacement. if you explicitly defined a name, you'll get an error like the following `CloudFormation cannot update a stack when a custom-named resource requires replacing. Rename testTable and update the stack again.` 207 | - if you do need to give resources a human readable name, use tags instead 208 | - many aws services will use the value of the `Name` tag as the resource name 209 | 210 | ## Tagging 211 | - tag your stacks - CDK makes this [really easy](https://docs.aws.amazon.com/cdk/latest/guide/tagging.html) and tags are applied recursively throughout the stack 212 | - use constants to ensure consistent tagging 213 | ```typescript 214 | import {TAGS} from './tags' 215 | const infra = new InfraStack(app, "fooOrg-fooApp-dev", {...}) 216 | let {KEYS: K, VALUES: V} = TAGS 217 | Tag.add(infra, K.APP, V.APP.FOO_APP) 218 | Tag.add(infra, K.STAGE, V.STAGE.DEV) 219 | ``` 220 | - 🔸tag your low level constructs - stack tags don't apply to low level constructs so you'll have to manually tag them 221 | - 🔸not all resources can be tagged because it is not supported in cloudformation - you'll need to use the `awscli` or create a custom resource 222 | - eg. [LogGroup](https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/77) can't be tagged via cloudformation 223 | 224 | ## Config 225 | - use cdk.json to store config values 226 | - for secrets, store the SSM|secretsmanager identifier 227 | - 🔸to create SSM|secretmanager values, you will still need to bootstrap by using the cli 228 | - scope config values by stage 229 | ```yaml 230 | "context": { 231 | "dev": { 232 | "account": 1234567, 233 | "region": "us-west-2", 234 | # ... 235 | }, 236 | "prod": { 237 | # ... 238 | } 239 | } 240 | ``` 241 | - use a helper function to retrieve values 242 | ```typescript 243 | function getEnv(stage, key) { 244 | scope.node.tryGetContext(stage)[key] 245 | } 246 | ``` 247 | - avoid use CloudFormation [parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html) 248 | - parameters are resolved during deployed time which prevents the CDK from validating values 249 | - use context variables instead since the values can be validated before deployment 250 | 251 | - 🔸creating the same stack with different context parameters will result in different cloudformation template 252 | - can be confusing to those coming from CloudFormation parameters which will update the existing stack instead of creating a new one 253 | 254 | ## Patterns 255 | 256 | - 🚧 creating default constructs 257 | - as a company, you might want to enforce certain defaults and policies when creating constructs 258 | - consider using composition, inheritance and factories. see discussion [here](https://github.com/aws/aws-cdk/issues/3235) 259 | - factories example: creates s3 bucket with overridable defaults 260 | ```typescript 261 | export function createBucket({scope, id, bucketProps = {}, accessLogBucket}: { 262 | scope: Construct|App, 263 | id: string, 264 | bucketProps?: BucketProps 265 | accessLogBucket?: IBucket 266 | }) { 267 | bucketProps = _.defaults({}, bucketProps, { 268 | encryption: BucketEncryption.KMS_MANAGED, 269 | blockPublicAccess: BlockPublicAccess.BLOCK_ALL 270 | }) 271 | const bucket = new Bucket(scope, id, bucketProps) 272 | 273 | if (!_.isUndefined(accessLogBucket)) { 274 | let logFilePrefix = `${id}/` 275 | addS3AccessLogs({srcBucket: bucket, destBucket: accessLogBucket, logFilePrefix}) 276 | } 277 | return bucket 278 | } 279 | ``` 280 | 281 | ## Tools and Libraries 282 | 283 | Collection of tools and libraries that make it easier to work with CDK 284 | 285 | - constructs and construct libraries 286 | - [aws-cdk-examples](https://github.com/aws-samples/aws-cdk-examples): official AWS repository for example CDK constructs 287 | - [cdk-constants](https://github.com/kevinslin/cdk-constants): collection of useful constants for the cdk 288 | - [punchcard](https://github.com/sam-goodwin/punchcard): combine infra and runtime code 289 | 290 | - tools 291 | - [aws-vault](https://github.com/99designs/aws-vault): A vault for securely storing and accessing AWS credentials in development environments. Supports switching between profiles, MFA access, assuming roles and more 292 | - [former2](https://former2.com/): tool created by the brilliant [Ian Mckay](https://github.com/iann0036) that can generate CDK/cloudformation/terraform from your existing infrastructure 293 | - [AWSConsoleRecorder](https://github.com/iann0036/AWSConsoleRecorder): tool created by the brilliant [Ian Mckay](https://github.com/iann0036) that can generate CDK/cloudformation/terraform/cli/boto3 (and more) from actions performed on the console. 294 | 295 | ## Deployments 296 | - when checking in CR for CDK code, include output of `cdk diff` in the code review 297 | - 🔸 CDK diff will exit with error code of 1 if diff is detected. this is [expected behavior](https://github.com/aws/aws-cdk/issues/1440) 298 | - its a good idea to version control the CloudFormation template in addition to the cdk code 299 | - can compare against past templates to get diff since a particular commit 300 | - easy to grep cloudformation to preview current state of infra 301 | - 🚧 deploy CDK changes as part of CI/CD pipeline (same benefits as putting code in a pipeline - eg. reduce manual actions, better monitoring and better IAM isolation) 302 | 303 | ## Limitations 304 | - aws cdk doesn't [support everything](https://github.com/aws/aws-cdk/issues/1656) that `awscli` does specifically 305 | - issue: CDK CLI will not read your region from your `default` profile 306 | - workaround: set region in `cdk.json` 307 | - issue: CDK CLI does not support MFA authentication 308 | - workaround: 🚪 use [aws-vault](https://github.com/99designs/aws-vault) to generate session tokens 309 | - issue: CDK CLI does not support `credential_source` 310 | - workaround: use `source_profile` 311 | - issue: Cannot have a profile named "default" in the config file 312 | - workaround: do not have `[default]` or `[profile default]` in config 313 | - cdk diff won't show detailed construct properties when a construct is first created 314 | - use `cdk synthesize` to verify template properties before doing deployments that involve creating new constructs 315 | - as far as features, CDK is built on top of cloudformation which means anything that cloudformation doesn't support is also not supported with the CDK 316 | - high level constructs don't always support all the options of the cloudformation resource they are modifying 317 | - in this case, either add a property override or use the low level construct directly 318 | - don't forget to also submit an [issue](https://github.com/aws/aws-cdk/issues/new/choose) on github 319 | 320 | ## Get Help 321 | - refer to the [api reference](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html) 322 | - figure out if the feature is a hole in cdk or one in cloudformation 323 | - [cloudformation open road map issues](https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues) 324 | - [cdk issues](https://github.com/aws/aws-cdk/issues) 325 | - update to latest cdk - releases are frequent and they're constantly adding new things 326 | - bring it up on the [cdk gitter](https://gitter.im/awslabs/aws-cdk) (this is actively monitored by CDK folks - I've had [issues](https://github.com/aws/aws-cdk/issues/3467) that I brought up be patched within a few hours) 327 | - checkout the [cdk code](https://github.com/aws/aws-cdk) and see the tests 328 | - cdk has great test coverage and they have great examples here 329 | - `/packages/@aws-cdk/aws-codepipeline/test/*` 330 | - check out the [CDK official examples](https://github.com/aws-samples/aws-cdk-examples) 331 | 332 | ## Further Reading 333 | - [awesome-cdk](https://github.com/eladb/awesome-cdk): collection of CDK resources by one of the devs that created it 334 | - [CDK All The Things](https://kevinslin.com/aws/cdk_all_the_things/): article covering general impressions of CDK and features 335 | 336 | # Legal 337 | 338 | ## Disclaimer 339 | The authors and contributors to this content cannot guarantee the validity of the information found here. Please make sure that you understand that the information provided here is being provided freely, and that no kind of agreement or contract is created between you and any persons associated with this content or project. The authors and contributors do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions in the information contained in, associated with, or linked from this content, whether such errors or omissions result from negligence, accident, or any other cause. 340 | 341 | ## License 342 | 343 | [![Creative Commons License](https://i.creativecommons.org/l/by-sa/4.0/88x31.png)](http://creativecommons.org/licenses/by-sa/4.0/) 344 | 345 | This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/). 346 | --------------------------------------------------------------------------------