├── lib └── some.js ├── .gitignore ├── bin ├── test ├── deploy-app └── deploy-api ├── index.js ├── .buildkite ├── upload-pipeline └── pipeline.yml ├── .eslintrc ├── test ├── .eslintrc ├── lib │ └── some-test.js └── index-test.js ├── auto ├── test └── ci ├── deployment ├── app_config.yml └── cloudformation.yml ├── README.md ├── docker-compose.yml ├── Dockerfile ├── package.json └── ARTICLE.md /lib/some.js: -------------------------------------------------------------------------------- 1 | module.exports = () => 'hello world'; 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | coverage/ 2 | dist/ 3 | docs/api.yml 4 | node_modules/ 5 | -------------------------------------------------------------------------------- /bin/test: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | cd $(dirname $0)/.. 4 | 5 | yarn install 6 | yarn run test 7 | -------------------------------------------------------------------------------- /index.js: -------------------------------------------------------------------------------- 1 | exports.handler = (event, context, callback) => { 2 | callback(null, 'hello world'); 3 | }; 4 | -------------------------------------------------------------------------------- /.buildkite/upload-pipeline: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | buildkite-agent pipeline upload < $(dirname $0)/pipeline.yml 4 | -------------------------------------------------------------------------------- /.eslintrc: -------------------------------------------------------------------------------- 1 | { 2 | "extends": "airbnb", 3 | "rules": { 4 | "comma-dangle": ["error", "never"] 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /.buildkite/pipeline.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - label: ':pray: Execute Tests' 3 | command: auto/test 4 | - label: ':clap: Deploy' 5 | command: auto/ci 6 | -------------------------------------------------------------------------------- /bin/deploy-app: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | cd $(dirname $0)/.. 4 | 5 | rm -rf dist/ 6 | mkdir dist/ 7 | cp -r index.js lib package.json yarn.lock dist/ 8 | cd dist/ 9 | yarn install --production 10 | -------------------------------------------------------------------------------- /test/.eslintrc: -------------------------------------------------------------------------------- 1 | { 2 | "env": { 3 | "mocha": true 4 | }, 5 | "rules": { 6 | "import/no-extraneous-dependencies": ["error", { "devDependencies": true }], 7 | "no-magic-numbers": "off" 8 | } 9 | } 10 | -------------------------------------------------------------------------------- /auto/test: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | APP_NAME=my-app 4 | 5 | cd $(dirname $0)/.. 6 | 7 | trap "docker-compose down --volumes --remove-orphans" 0 8 | docker-compose build ${APP_NAME} 9 | docker-compose run --rm ${APP_NAME} /app/bin/test 10 | -------------------------------------------------------------------------------- /test/lib/some-test.js: -------------------------------------------------------------------------------- 1 | const chai = require('chai'); 2 | const some = require('../../lib/some'); 3 | 4 | const expect = chai.expect; 5 | 6 | describe('some', () => { 7 | it('should do something', () => { 8 | expect(some()).to.equal('hello world'); 9 | }); 10 | }); 11 | -------------------------------------------------------------------------------- /deployment/app_config.yml: -------------------------------------------------------------------------------- 1 | app_project_name: my-project 2 | app_name: my-app 3 | app_region: ap-southeast-2 4 | lambda_build_dir: dist 5 | 6 | template_params: 7 | PLambdaName: 'my-lambda' 8 | PLambdaRuntime: nodejs4.3 9 | PLambdaHandler: index.handler 10 | PLambdaS3Key: '{{ lambda_zip_for_s3 }}' 11 | PLambdaS3Bucket: '{{ lambda_bucket }}' 12 | -------------------------------------------------------------------------------- /test/index-test.js: -------------------------------------------------------------------------------- 1 | const chai = require('chai'); 2 | const index = require('../index').handler; 3 | 4 | const expect = chai.expect; 5 | 6 | describe('index', () => { 7 | it('should succeed', (done) => { 8 | index(undefined, undefined, (error, result) => { 9 | expect(result).to.equal('hello world'); 10 | done(error, result); 11 | }); 12 | }); 13 | }); 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Serverless REST App on AWS 2 | 3 | This is a stencil for deploying an AWS API Gateway with a backing lambda using Swagger for both AWS CloudFormation as well as providing API documentation. 4 | 5 | Besides this repo, you need to set up an S3 bucket into which `./bin/deploy-api` can upload the api.yml into. Download Swagger UI, alter index.html to point to this file (e.g. `/files/api.yml`) and upload it into the same S3 bucket. 6 | 7 | See the corresponding article for further information. 8 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2' 2 | 3 | services: 4 | new-homes-bff: 5 | build: . 6 | volumes: 7 | - .:/app 8 | working_dir : /app 9 | environment: 10 | - BUILD_NUMBER=${BUILDKITE_BUILD_NUMBER} 11 | 12 | ansible: 13 | image: corporate-registry/ansible:latest 14 | volumes: 15 | - .:/app 16 | - ../ansible:/ansible 17 | environment: 18 | - AWS_ACCESS_KEY_ID 19 | - AWS_SECRET_ACCESS_KEY 20 | - AWS_SECURITY_TOKEN 21 | - BUILD_NUMBER=${BUILDKITE_BUILD_NUMBER} 22 | -------------------------------------------------------------------------------- /bin/deploy-api: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | cd $(dirname $0)/.. 4 | 5 | rm -rf docs/ 6 | mkdir docs/ 7 | # Grep 100k lines **A**fter and 100k lines **B**efore the markers, 8 | # except for those lines themselves and un-indent them 9 | # to extract the swagger from cloudformation.yml 10 | grep -A100000 SWAGGER_START deployment/cloudformation.yml \ 11 | | grep -B100000 SWAGGER_END \ 12 | | egrep -v "SWAGGER_START|SWAGGER_END" \ 13 | | sed 's/^ //' \ 14 | > docs/api.yml 15 | 16 | aws s3 cp docs/api.yml s3://${BUCKET_NAME}/files/api.yml 17 | -------------------------------------------------------------------------------- /auto/ci: -------------------------------------------------------------------------------- 1 | #!/bin/bash -eu 2 | 3 | APP_NAME=my-app 4 | ANSIBLE_NAME=ansible 5 | 6 | # Setup 7 | cd $(dirname $0)/.. 8 | APP_DIR=$(pwd) 9 | ANSIBLE_DIR="${APP_DIR}/../ansible" 10 | trap "docker-compose down --volumes --remove-orphans" 0 11 | 12 | # Build 13 | docker-compose build ${APP_NAME} 14 | docker-compose run --rm ${APP_NAME} /app/bin/deploy-app 15 | 16 | # Run Ansible 17 | rm -rf ${ANSIBLE_DIR} 18 | git clone git@git.realestate.com.au:devlob-media/ansible.git ${ANSIBLE_DIR} 19 | docker-compose run --rm ${ANSIBLE_NAME} "SOME PARAMS" 20 | 21 | # Upload Swagger file 22 | docker-compose run --rm ${APP_NAME} /app/bin/deploy-api 23 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:latest 2 | 3 | # Config 4 | ENV USER_NAME devlob 5 | ENV NODE_VERSION 4.3.2 6 | 7 | # Setup dependencies 8 | RUN apt-get update && apt-get install -y curl git python-pip 9 | 10 | RUN groupadd $USER_NAME && useradd -m -g $USER_NAME $USER_NAME 11 | USER $USER_NAME 12 | 13 | # Install AWS CLI tools 14 | RUN pip install awscli 15 | ENV PATH $PATH:/home/$USER_NAME/.local/bin/ 16 | 17 | # Install AWS Lambda node version 18 | ENV NVM_DIR /home/$USER_NAME/.nvm 19 | ENV NODE_PATH $NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules 20 | ENV PATH $PATH:$NVM_DIR/versions/node/v$NODE_VERSION/bin 21 | RUN git clone https://github.com/creationix/nvm.git "$NVM_DIR" && \ 22 | cd "$NVM_DIR" && \ 23 | git checkout `git describe --abbrev=0 --tags --match "v[0-9]*" origin` && \ 24 | . "$NVM_DIR/nvm.sh" && \ 25 | nvm install $NODE_VERSION && \ 26 | npm install -g yarn 27 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "my-app", 3 | "version": "1.0.0", 4 | "description": "Do cool stuff", 5 | "main": "index.js", 6 | "repository": "git://...", 7 | "author": "David Christ { 164 | const message = event.querystring.input; 165 | callback(null, { output: message }); 166 | } 167 | ``` 168 | 169 | This function can now be ZIPped and placed into the aforementioned location in S3. This step can be performed by a continuous integration system. Later on we will see how that event object came to include the query string. 170 | 171 | # AWS API Gateway REST API 172 | 173 | API Gateway operates as an HTTP endpoint that can, amongst others, be an event source to a Lambda. There are various ways to configure endpoint resources (aka URL paths), but the one we are interested in here is passing it a swagger file. This can be expressed in a CloudFormation Resource. 174 | 175 | ```yaml 176 | ApiGatewayRestApi: 177 | Type: AWS::ApiGateway::RestApi 178 | Properties: 179 | Name: MyApi 180 | Description: My AWS API Gateway config 181 | Body: 182 | # INSERT swagger.yml content here 183 | ``` 184 | 185 | In addition to defining the REST endpoints, naturally we need to tell AWS which Lambda to invoke for a given endpoint. We can do so by adding the proprietary `x-amazon-apigateway-integration` field to our swagger template. Counter-intuitively in that section we have to specify `httpMethod` as POST, sine API Gateway talks to Lambda through POST requests, regardless of what the incoming request to API Gateway was. The section `requestTemplates` allows us to specify how the incoming request parameters get transformed and passed to the Lambda. In this example we just wrap everything (including the query string) in a JSON object. That object is what gets passed as the event object into the Lambda as seen in the code above. As our service might be used as a backend (or backend for frontend) for an app hosted at a different location, we need to allow cross-origin resource sharing (CORS). This is seen in the various `Access-Control-*` fields across this Resource. Lastly, we again have to attach an IAM Role to allow our REST API. This time to allow it to invoke it's backing Lambda. 186 | 187 | ```yaml 188 | ApiGatewayRestApi: 189 | Type: AWS::ApiGateway::RestApi 190 | Properties: 191 | Name: MyApi 192 | Description: My AWS API Gateway config 193 | Body: 194 | ### SWAGGER_START !!! DO NOT ALTER THIS LINE !!! 195 | swagger: '2.0' 196 | info: 197 | title: My API 198 | description: My AWS API Gateway config 199 | version: '1.0.0' 200 | schemes: 201 | - https 202 | basePath: /api/v1 203 | produces: 204 | - application/json 205 | definitions: 206 | Listing: 207 | type: object 208 | properties: 209 | id: 210 | type: string 211 | description: Listing ID 212 | title: 213 | type: string 214 | description: Title of the listing. 215 | paths: 216 | /listings: 217 | get: 218 | summary: Get Project Profiles 219 | description: | 220 | This endpoint returns information about listings 221 | with a specific state, surburb and post code. 222 | parameters: 223 | - name: postcode 224 | in: query 225 | description: postcode 226 | required: false 227 | type: string 228 | responses: 229 | '200': 230 | description: Project profiles 231 | headers: 232 | Access-Control-Allow-Headers: 233 | type: "string" 234 | Access-Control-Allow-Methods: 235 | type: "string" 236 | Access-Control-Allow-Origin: 237 | type: "string" 238 | schema: 239 | $ref: '#/definitions/Listing' 240 | ### SWAGGER_END !!! DO NOT ALTER THIS LINE !!! 241 | x-amazon-apigateway-integration: 242 | type: aws 243 | responses: 244 | default: 245 | statusCode: '200' 246 | responseParameters: 247 | method.response.header.Access-Control-Allow-Headers : "'Content-Type'" 248 | method.response.header.Access-Control-Allow-Methods : "'*'" 249 | method.response.header.Access-Control-Allow-Origin : "'*'" 250 | # Yes, indeed it needs to be POST! 251 | httpMethod: POST 252 | credentials: !GetAtt ApiGatewayIamRole.Arn 253 | requestTemplates: 254 | application/json: '#set($allParams = $input.params()) { #foreach($type in $allParams.keySet()) #set($params = $allParams.get($type)) "$type" : { #foreach($paramName in $params.keySet()) "$paramName" : "$util.escapeJavaScript($params.get($paramName))" #if($foreach.hasNext),#end #end } #if($foreach.hasNext),#end #end }' 255 | uri: !Join 256 | - '' 257 | - 258 | - 'arn:aws:apigateway:' 259 | - !Ref 'AWS::Region' 260 | - ':lambda:path/2015-03-31/functions/arn:aws:lambda:' 261 | - !Ref 'AWS::Region' 262 | - ':' 263 | - !Ref 'AWS::AccountId' 264 | - ':function:' 265 | - !Ref PLambdaName 266 | - '/invocations' 267 | options: 268 | summary: CORS support 269 | description: Enable CORS by returning correct headers 270 | consumes: 271 | - application/json 272 | produces: 273 | - application/json 274 | tags: 275 | - CORS 276 | x-amazon-apigateway-integration: 277 | type: mock 278 | requestTemplates: 279 | application/json: | 280 | { 281 | "statusCode" : 200 282 | } 283 | responses: 284 | "default": 285 | statusCode: "200" 286 | responseParameters: 287 | method.response.header.Access-Control-Allow-Headers : "'Content-Type'" 288 | method.response.header.Access-Control-Allow-Methods : "'*'" 289 | method.response.header.Access-Control-Allow-Origin : "'*'" 290 | responseTemplates: 291 | application/json: | 292 | {} 293 | responses: 294 | '200': 295 | description: Default response for CORS method 296 | headers: 297 | Access-Control-Allow-Headers: 298 | type: "string" 299 | Access-Control-Allow-Methods: 300 | type: "string" 301 | Access-Control-Allow-Origin: 302 | type: "string" 303 | x-amazon-apigateway-request-validators: 304 | params-only: 305 | validateRequestBody: false 306 | validateRequestParameters: true 307 | x-amazon-apigateway-request-validator : params-only 308 | 309 | ApiGatewayIamRole: 310 | Properties: 311 | AssumeRolePolicyDocument: 312 | Statement: 313 | - Action: 314 | - sts:AssumeRole 315 | Effect: Allow 316 | Principal: 317 | Service: 318 | - apigateway.amazonaws.com 319 | Version: '2012-10-17' 320 | Path: / 321 | Policies: 322 | - PolicyDocument: 323 | Statement: 324 | - Action: 325 | - lambda:InvokeFunction 326 | - iam:PassRole 327 | Effect: Allow 328 | Resource: '*' 329 | PolicyName: PermitApiGateway 330 | Type: AWS::IAM::Role 331 | ``` 332 | 333 | # AWS API Gateway Deployment and Stage 334 | 335 | The "logistics" behind API Gateway requires us to define a Deployment and a Stage. It is easy to think of the Stage as the web server (e.g. Nginx or Apache), as it defines things like log level or throttling. A Deployment is a snapshot of a REST API and fixes or overrides default settings of a Stage. Lastly, an Account gives us permissions to write our logs to CloudWatch. The Stage-Deployment-mechanism allows for multiple deployments for different lifecycle stages but is beyond the concerns of this article. We only define one Deployment and Stage for our REST API. 336 | 337 | ```yaml 338 | ApiGatewayDeployment: 339 | Type: AWS::ApiGateway::Deployment 340 | Properties: 341 | RestApiId: 342 | Ref: ApiGatewayRestApi 343 | 344 | ApiGatewayStage: 345 | Type: AWS::ApiGateway::Stage 346 | Properties: 347 | StageName: latest 348 | Description: latest stage 349 | RestApiId: 350 | Ref: ApiGatewayRestApi 351 | DeploymentId: 352 | Ref: ApiGatewayDeployment 353 | MethodSettings: 354 | - LoggingLevel: INFO 355 | HttpMethod: "*" 356 | DataTraceEnabled: true 357 | ResourcePath: "/*" 358 | CachingEnabled: true 359 | CacheTtlInSeconds: 60 360 | MetricsEnabled: true 361 | 362 | Account: 363 | Type: "AWS::ApiGateway::Account" 364 | Properties: 365 | CloudWatchRoleArn: !GetAtt ApiGatewayIamRole.Arn 366 | ``` 367 | 368 | Currently, in the setup with Swagger, the stage is not fully created by AWS through CloudFormation, so we have to execute this step manually in the web console. After everything has been deployed, go to APIs > MyApi (*AWS-API-ID*) > Resources > / (*AWS-Resource-ID*), choose Actions > Deploy API select your Stage and click Deploy. 369 | 370 | # Continuous Integration / Deployment 371 | 372 | Now, this part of the puzzle is arguably the one that is the most dependent on your company's way of delivering things. This example roughly outlines the way of doing it at ours and for brevity leaves out some details like different deployment environments (e.g. production and staging). The deployment process can be broken down into these parts: 373 | 374 | * A CI environment (we use BuildKite with docker). 375 | * A process to build delivery artifacts (we use a dev docker container). 376 | * A process that consumes artifacts and CloudFormation and deploys to AWS (we use an ops container running Ansible). 377 | 378 | In order to build the Lambda, the dev container has Node.js in the same version as available on AWS installed. Additionally, it has the AWS CLI tools to upload the swagger.yml file into an S3 bucket, since the ops container only deploys build artifacts to Lambda (or EC2, …). Yarn runs test and build scripts. Creating the swagger file is done quick and dirty by `grep`ping it out of the CloudFormation file. That file is than copied into the desired bucket. The ops container, amongst others, uploads the Lambda artifact into S3 and runs the CloudFormation template and parameter files to deploy everything into AWS. 379 | 380 | # Source 381 | 382 | The code on [GitHub](https://github.com/dctr/aws-api-gateway-lambda-and-swagger) contains: 383 | 384 | * A nice README.md :-) 385 | * Application logic in the usual Node.js way (index.js, packagage.json, lib/, test/, …) 386 | * Continuous integration related files 387 | * .buildkite/ contains a pipeline uploaded to BuildKite 388 | * auto/ contains the scripts executed on BuildKite (e.g. to build containers) 389 | * Dockerfile and docker-compose.yml to set up the containers 390 | * bin/ contains the scripts executed inside the containers 391 | * CloudFormation files in deployment/ 392 | * cloudformation.yml is the template file, comprised of snippets mostly generic to all our projects 393 | * app_config.yml contains parameters that are most likely to change across projects 394 | 395 | Not included is: 396 | 397 | * SwaggerUI, which can be downloaded as ZIP from their homepage. 398 | 399 | # Conclusion 400 | 401 | This article gives a good starting point to deploy a serverless service, which integrates documentation as a first class citizen. The service can be useful as standalone REST API or as a backend for frontend. Thanks to CORS settings, the consuming service needs no fancy magic to make it seem that API and app are hosted on the same domain. 402 | --------------------------------------------------------------------------------