├── .github └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── architecture.png ├── cloudformation ├── main.yaml ├── sftp_idp.yaml └── sftp_server.yaml └── src ├── authorizor └── lambda.js └── datagen └── lambda_function.py /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/aws-samples/transfer-for-sftp-logical-directories/issues), or [recently closed](https://github.com/aws-samples/transfer-for-sftp-logical-directories/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/transfer-for-sftp-logical-directories/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](https://github.com/aws-samples/transfer-for-sftp-logical-directories/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Transfer for SFTP - Logical Directory demonstration 2 | 3 | This sample code demonstrates how to configure per-user logical directory configurations as part of a custom identity provider for use with the 4 | AWS Transfer for SFTP service. To see the sample in action you can deploy the code into your AWS account now using one of the buttons below or you 5 | can clone this repository and explore the code for yourself. 6 | 7 | This repository contains the following files: 8 | ``` 9 | ├── README.md 10 | ├── cloudformation 11 | │ ├── main.yaml <-- CloudFormation to deploy S3 buckets, an SFTP server, and identity provider 12 | │ ├── sftp_idp.yaml <-- CloudFormation to provision custom identity provider 13 | │ └── sftp_server.yaml <-- CloudFormation to provision Transfer for SFTP server 14 | └── src 15 | ├── authorizor 16 | │ └── lambda.js <-- custom identity provider, responsible for configuring logical directories 17 | └── datagen 18 | └── lambda_function.py <-- custom resource to generate sample data, part of cloudformation 19 | ``` 20 | 21 | ## License Summary 22 | 23 | This sample code is made available under the MIT-0 license. See the LICENSE file. 24 | 25 | ## Getting started 26 | 27 | 1. To see the demonstration in action deploy the custom identity provider using the CloudFormation provided or deploy the CloudFormation into your AWS account now using the buttons below. 28 | 29 | | Region | Launch Template | 30 | |---|:---:| 31 | | Ireland (eu-west-1) | [![Launch SFTP Demo (Ireland)](https://github.com/awslabs/ami-builder-packer/raw/master/images/deploy-to-aws.png "Launch SFTP Demo in Ireland")](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?stackName=logical-directory-demo&templateURL=https://s3.amazonaws.com/transfer-for-sftp-logical-directories-eu-west-1/logical-directories.yaml) | 32 | | London (eu-west-2) | [![Launch SFTP Demo (London)](https://github.com/awslabs/ami-builder-packer/raw/master/images/deploy-to-aws.png "Launch SFTP Demo in London")](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?stackName=logical-directory-demo&templateURL=https://s3.amazonaws.com/transfer-for-sftp-logical-directories-eu-west-2/logical-directories.yaml) | 33 | | N. Virginia (us-east-1) | [![Launch SFTP Demo (N. Virginia)](https://github.com/awslabs/ami-builder-packer/raw/master/images/deploy-to-aws.png "Launch SFTP Demo in Virginia")](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?stackName=logical-directory-demo&templateURL=https://s3.amazonaws.com/transfer-for-sftp-logical-directories-us-east-1/logical-directories.yaml) | 34 | | Oregon (us-west-2) | [![Launch SFTP Demo (Oregon)](https://github.com/awslabs/ami-builder-packer/raw/master/images/deploy-to-aws.png "Launch SFTP Demo in Oregon")](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?stackName=logical-directory-demo&templateURL=https://s3.amazonaws.com/transfer-for-sftp-logical-directories-us-west-2/logical-directories.yaml) | 35 | | Sydney (ap-southeast-2) | [![Launch SFTP Demo (Sydney)](https://github.com/awslabs/ami-builder-packer/raw/master/images/deploy-to-aws.png "Launch SFTP Demo in Sydney")](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?stackName=logical-directory-demo&templateURL=https://s3.amazonaws.com/transfer-for-sftp-logical-directories-ap-southeast-2/logical-directories.yaml) | 36 | 37 | **Or using the AWS CLI...** 38 | 39 | ```bash 40 | $ aws cloudformation package --template-file cloudformation/main.yaml --s3-bucket YOUR-CLOUDFORMATION-BUCKET-NAME --output-template-file packaged.yaml 41 | $ aws cloudformation deploy --template-file packaged.yaml --stack-name sftp-demo --capabilities CAPABILITY_IAM --parameter-overrides StackId=logical-demo 42 | ``` 43 | 44 | 2. Once the CloudFormation template has finished deploying visit the [AWS Transfer for SFTP console](https://console.aws.amazon.com/transfer/home) and open the details page for your new SFTP server. Make a note of the `Endpoint` URI. 45 | 46 | 2. Then using your preferred SFTP client open a connection to the SFTP server: 47 | 48 | ```bash 49 | sftp alice@s-123abcca122e420c8.server.transfer.eu-west-2.amazonaws.com 50 | ``` 51 | 52 | The custom identity provider is configured for the following users: 53 | 54 | | Username | Password | 55 | |---|:---:| 56 | | alice | Password01 | 57 | | bryan | Password02 | 58 | 59 | Once logged in you should see one of the following directory structures: 60 | > Alice 61 | ``` 62 | / 63 | ├── public 64 | │ └── research --> s3://public-research 65 | │ └── global 66 | └── subscribed 67 | ├── 2018 68 | │ └── indices --> s3://subscriptions/historical/2018/indices 69 | └── 2019 70 | ├── indices --> s3://subscriptions/historical/2019/indices 71 | └── equities --> s3://subscriptions/historical/2019/equities 72 | ``` 73 | 74 | > Bryan 75 | ``` 76 | / 77 | ├── public 78 | │ └── research --> s3://public-research 79 | │ └── global 80 | └── subscribed 81 | ├── 2018 82 | │ ├── equities --> s3://subscriptions/historical/2018/equities 83 | │ └── indices --> s3://subscriptions/historical/2018/indices 84 | └── 2019 85 | ├── credit --> s3://subscriptions/historical/2019/credit 86 | └── equities --> s3://subscriptions/historical/2019/equities 87 | ``` 88 | 89 | > Please note the actual S3 bucket names will be specific to your CloudFormation deployment. 90 | 91 | You can log into the SFTP server and observe the user experience for both Alice and Bryan. Please also experiment with the custom identity provider by accessing its source code from the AWS Lambda console. Try giving Alice and Bryan permissions to write files as well as read files, try mapping additional buckets to other parts of the file system, experiment with different permutations to align the identity provider more with your use case. 92 | 93 | ## About the CloudFormation 94 | There are two stacks deployed as part of an over-arching stack defined by main.yaml. This top level stack creates 2 Amazon S3 buckets for use by the SFTP server and then creates 2 nested stacks. The first nested stack creates a Lambda function fronted by an API Gateway, this will act as a custom identity provider for our SFTP server. The next nested stack will provision the Transfer for SFTP server leveraging the custom IDP. 95 | 96 | The CloudFormation templates create an architecture which is made up of four key components. As shown in the diagram below they are the AWS Transfer for SFTP server, Amazon API Gateway, AWS Lambda, and two S3 buckets for the data repository. 97 | 98 | ![AWS Transfer for SFTP Demonstration Architecture](architecture.png "AWS Transfer for SFTP Logical Directories Demonstration Architecture") 99 | 100 | When a user opens a connection to AWS SFTP, the service will forward the user’s credentials to a custom identity provider which is fronted by an API method implemented using Amazon API Gateway. The API Gateway passes the user credentials and server details to an AWS Lambda function to authenticate and authorize the user. 101 | 102 | The AWS Lambda function will attempt to authenticate the user and if authentication is successful, it will return a JSON object describing the user’s SFTP session. The JSON object includes the IAM role and an optional scope-down policy that governs the user’s access to one or more S3 buckets. It also describes the folder structure the user will see and how those folders map to S3 bucket paths. The JSON document is returned to the AWS SFTP server which then maintains a session with the user, handling SFTP commands such as get, ls, and put. 103 | 104 | 105 | ## About the custom IDP 106 | The identity provider is a Lambda function which is responsible for determining if the user is first authenticated and then how the user should be authorized. After authentication the Lambda function should respond with an IAM policy and virtual mapping to be used by the Transfer for SFTP service. This IAM policy is a scope down policy meant to grant user-specific permissions as a subset of the role granted to the user. 107 | 108 | All Lambda functions have an entry-point, called a “Handler,” which is where execution of the Lambda function begins. Below is the code for our handler: 109 | 110 | ```javascript 111 | exports.handler = (event, context, callback) => { 112 | console.log("Event:", JSON.stringify(event)); 113 | 114 | var response = {}; 115 | 116 | if (authenticated(event.username, event.password)) { 117 | var scopeDownPolicy = getScopeDownPolicy(event.username); 118 | var directoryMapping = getDirectoryMapping(event.username); 119 | 120 | response = { 121 | Role: userRoleArn, 122 | Policy: JSON.stringify(scopeDownPolicy), 123 | HomeDirectoryType: "LOGICAL", 124 | HomeDirectoryDetails: JSON.stringify(directoryMapping) 125 | }; 126 | } 127 | 128 | console.log("Returning ", JSON.stringify(response)); 129 | callback(null, response); 130 | }; 131 | ``` 132 | 133 | This function does four important things: 134 | 1. Authenticates the SFTP user 135 | 2. Generates a scope-down policy to define user access to the S3 bucket 136 | 3. Generates a logical directory mapping - these are the folders the user will see in their SFTP browser 137 | 4. Builds a response that will be returned to the AWS SFTP service (the caller of the API method) 138 | 139 | Let’s now look at each of these in more depth. 140 | 141 | ### Authentication 142 | The event object passed to the Lambda function holds data that was passed to the API Gateway method by the AWS SFTP server. This data includes the username and password from the SFTP client that is trying to connect to the SFTP server. To authenticate the user, we pass the username and password to the “authenticate” function. 143 | 144 | ```javascript 145 | function authenticated(username, password) { 146 | if (username in userDb) { 147 | var userRecord = userDb[username]; 148 | 149 | if (password == userRecord.password) { 150 | return true; 151 | } 152 | } 153 | 154 | return false; 155 | } 156 | ``` 157 | 158 | This function simply verifies that the specified username is in our “user database” (i.e. userDb) and then compares the password. If the password matches, then the user is authenticated, otherwise authentication fails. 159 | 160 | We kept our authentication code extremely simple for the sake of demonstration. You will want to replace this code with a call to Amazon Cognito, Microsoft Active Directory, an LDAP server, or even a 3rd-party authentication provider before going into production. 161 | 162 | ### User Database 163 | For our example code, the user database is a simple data structure that defines the password, scope-down policy, and directory mappings for the current users: Alice and Bryan. 164 | 165 | ```javascript 166 | var public_bucket = process.env.PUBLIC_BUCKET; 167 | var subscription_bucket = process.env.SUBSCRIBE_BUCKET; 168 | var userRoleArn = process.env.USER_ROLE; 169 | var sftpServerId = process.env.SERVER_ID; 170 | 171 | var userDb = { 172 | "alice": { 173 | "password": "Password01", 174 | "policy": { 175 | "Version": "2012-10-17", 176 | "Statement": [ 177 | { 178 | "Sid": "AllowListingOfFolder", 179 | "Action": [ 180 | "s3:ListBucket" 181 | ], 182 | "Effect": "Allow", 183 | "Resource": [ 184 | "arn:aws:s3:::" + public_bucket, 185 | "arn:aws:s3:::" + subscription_bucket 186 | ] 187 | }, 188 | { 189 | "Sid": "AllowObjectAccess", 190 | "Effect": "Allow", 191 | "Action": [ 192 | "s3:GetObject", 193 | "s3:GetObjectVersion" 194 | ], 195 | "Resource": [ 196 | "arn:aws:s3:::" + public_bucket + "/global/*", 197 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/indices/*", 198 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/indices/*", 199 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/equities/*" 200 | ] 201 | } 202 | ] 203 | }, 204 | "directoryMap": [ 205 | { 206 | "Entry": "/public/research", 207 | "Target": "/"+ public_bucket 208 | }, 209 | { 210 | "Entry": "/subscribed/2018/indices", 211 | "Target": "/"+ subscription_bucket + "/historical/2018/indices" 212 | }, 213 | { 214 | "Entry": "/subscribed/2019/indices", 215 | "Target": "/"+ subscription_bucket + "/historical/2019/indices" 216 | }, 217 | { 218 | "Entry": "/subscribed/2019/equities", 219 | "Target": "/"+ subscription_bucket + "/historical/2019/equities" 220 | } 221 | ] 222 | }, 223 | "bryan": { 224 | "password": "Password02", 225 | "policy": { 226 | "Version": "2012-10-17", 227 | "Statement": [ 228 | { 229 | "Sid": "AllowListingOfFolder", 230 | "Action": [ 231 | "s3:ListBucket" 232 | ], 233 | "Effect": "Allow", 234 | "Resource": [ 235 | "arn:aws:s3:::" + public_bucket, 236 | "arn:aws:s3:::" + subscription_bucket 237 | ] 238 | }, 239 | { 240 | "Sid": "AllowObjectAccess", 241 | "Effect": "Allow", 242 | "Action": [ 243 | "s3:GetObject", 244 | "s3:GetObjectVersion" 245 | ], 246 | "Resource": [ 247 | "arn:aws:s3:::" + public_bucket + "/global/*", 248 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/indices/*", 249 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/equities/*", 250 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/credit/*", 251 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/equities/*" 252 | ] 253 | } 254 | ] 255 | }, 256 | "directoryMap": [ 257 | { 258 | "Entry": "/public/research", 259 | "Target": "/"+ public_bucket 260 | }, 261 | { 262 | "Entry": "/subscribed/2018/indices", 263 | "Target": "/"+ subscription_bucket + "/historical/2018/indices" 264 | }, 265 | { 266 | "Entry": "/subscribed/2018/equities", 267 | "Target": "/"+ subscription_bucket + "/historical/2018/equities" 268 | }, 269 | { 270 | "Entry": "/subscribed/2019/credit", 271 | "Target": "/"+ subscription_bucket + "/historical/2019/credit" 272 | }, 273 | { 274 | "Entry": "/subscribed/2019/equities", 275 | "Target": "/"+ subscription_bucket + "/historical/2019/equities" 276 | } 277 | ] 278 | } 279 | }; 280 | ``` 281 | 282 | While our hard-coded user database is fine for testing purposes, where there are only a few users, it would be cumbersome to maintain for many users in a production environment. To build a scalable solution, you will want to look at using an Amazon Relational Database Service (RDS), Amazon DynamoDB table, or some other scalable data structure for managing data about your users. 283 | 284 | ### Scope-down policies 285 | 286 | AWS SFTP uses both an IAM role and an optional scope-down policy to control user access to the S3 buckets. The scope-down policy gives you granular control to define which S3 bucket paths a user can access and which paths are explicitly denied. In our user database, we only define “Allow” entries in our scope-down policies, but these can easily be extended to include “Deny” entries as well. 287 | 288 | Because the scope-down policy is hard-coded into the user database, we can simply return it directly in the getScopeDownPolicy function: 289 | 290 | ```javascript 291 | function getScopeDownPolicy(username) { 292 | var userRecord = userDb[username]; 293 | return userRecord.policy; 294 | } 295 | ``` 296 | 297 | ### Logical directory entries 298 | 299 | Building the folder structure that you want for your users is easily done using logical directories. Each logical directory is an object with two fields: Entry and Target. The Entry is the name of the folder that the user will see and the Target is the S3 folder path. 300 | 301 | Like the scope-down policy, the directory mappings are hard-coded into the user database and can be returned directly in the getDirectoryMapping function: 302 | 303 | ```javascript 304 | function getDirectoryMapping(username) { 305 | var userRecord = userDb[username]; 306 | return userRecord.directoryMap; 307 | } 308 | ``` 309 | 310 | ### Response Data 311 | With the user successfully authenticated and the scope-down policy and logical directories defined, we can build a response object to return to the AWS SFTP server. 312 | 313 | ```javascript 314 | response = { 315 | Role: userRoleArn, 316 | Policy: JSON.stringify(scopeDownPolicy), 317 | HomeDirectoryType: "LOGICAL", 318 | HomeDirectoryDetails: JSON.stringify(directoryMapping) 319 | }; 320 | ``` 321 | 322 | The response object requires the ARN for an IAM role that will define access to the S3 buckets for the user. In our example, the role is created as part of the CloudFormation stack. We also include the scope-down policy and the logical directory entries, both of which are formatted as JSON strings. 323 | 324 | The response object will be returned to the AWS SFTP server and the directory mapping specified in HomeDirectoryDetails will be used to construct the folders shown to the authenticated user. 325 | -------------------------------------------------------------------------------- /architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/transfer-for-sftp-logical-directories/d3be6d3098fa5b0c7e4fcec62c918b79f3239fea/architecture.png -------------------------------------------------------------------------------- /cloudformation/main.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy of 5 | # this software and associated documentation files (the "Software"), to deal in 6 | # the Software without restriction, including without limitation the rights to 7 | # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 8 | # the Software, and to permit persons to whom the Software is furnished to do so. 9 | 10 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 11 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 12 | # FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 13 | # COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 14 | # IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 15 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 16 | 17 | AWSTemplateFormatVersion: 2010-09-09 18 | Description: CloudFormation template to deploy Transfer for SFTP with a custom authorizor. 19 | 20 | Parameters: 21 | StackId: 22 | Description: Stack identifier 23 | Type: String 24 | 25 | Resources: 26 | PublicBucket: 27 | Type: AWS::S3::Bucket 28 | Properties: 29 | AccessControl: Private 30 | BucketName: !Sub 31 | - ${StackId}-public-bucket 32 | - StackId: !Ref StackId 33 | 34 | SubscribedBucket: 35 | Type: AWS::S3::Bucket 36 | Properties: 37 | AccessControl: Private 38 | BucketName: !Sub 39 | - ${StackId}-subscribe-bucket 40 | - StackId: !Ref StackId 41 | 42 | SampleData: 43 | Type: "Custom::SampleData" 44 | Properties: 45 | ServiceToken: !GetAtt SampleDataLambda.Arn 46 | DependsOn: 47 | - PublicBucket 48 | - SubscribedBucket 49 | 50 | SampleDataLambdaRole: 51 | Type: "AWS::IAM::Role" 52 | Properties: 53 | AssumeRolePolicyDocument: 54 | Statement: 55 | - Action: 56 | - "sts:AssumeRole" 57 | Effect: "Allow" 58 | Principal: 59 | Service: 60 | - lambda.amazonaws.com 61 | Version: "2012-10-17" 62 | ManagedPolicyArns: 63 | - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 64 | Policies: 65 | - PolicyName: SampleDataBucketFullAccess 66 | PolicyDocument: 67 | Statement: 68 | - Action: 69 | - "s3:*" 70 | Effect: "Allow" 71 | Resource: 72 | - !GetAtt SubscribedBucket.Arn 73 | - !GetAtt PublicBucket.Arn 74 | - !Sub "${SubscribedBucket.Arn}/*" 75 | - !Sub "${PublicBucket.Arn}/*" 76 | Version: "2012-10-17" 77 | 78 | SampleDataLambda: 79 | Type: "AWS::Lambda::Function" 80 | Properties: 81 | Code: ../src/datagen 82 | Description: "A function to create sample data in the associated S3 buckets" 83 | Handler: lambda_function.handler 84 | Role: !GetAtt SampleDataLambdaRole.Arn 85 | Runtime: "python3.7" 86 | Environment: 87 | Variables: 88 | public_research: !Ref PublicBucket 89 | subscriptions: !Ref SubscribedBucket 90 | 91 | SftpIdp: 92 | Type: AWS::CloudFormation::Stack 93 | Properties: 94 | TemplateURL: sftp_idp.yaml 95 | Parameters: 96 | PublicBucket: !Ref PublicBucket 97 | SubscribedBucket: !Ref SubscribedBucket 98 | 99 | SftpServer: 100 | Type: AWS::CloudFormation::Stack 101 | Properties: 102 | TemplateURL: sftp_server.yaml 103 | Parameters: 104 | IdpInvocationRole: !GetAtt 105 | - SftpIdp 106 | - Outputs.TransferIdentityProviderInvocationRole 107 | IdpUrl: !GetAtt 108 | - SftpIdp 109 | - Outputs.TransferIdentityProviderUrl 110 | 111 | Outputs: 112 | SftpServerId: 113 | Description: SFTP Server ID 114 | Value: !GetAtt "SftpServer.Outputs.SftpServerId" 115 | SftpIdpLambda: 116 | Description: SFTP IDP Lambda handler 117 | Value: !GetAtt "SftpIdp.Outputs.LambdaName" 118 | -------------------------------------------------------------------------------- /cloudformation/sftp_idp.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy of 5 | # this software and associated documentation files (the "Software"), to deal in 6 | # the Software without restriction, including without limitation the rights to 7 | # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 8 | # the Software, and to permit persons to whom the Software is furnished to do so. 9 | 10 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 11 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 12 | # FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 13 | # COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 14 | # IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 15 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 16 | 17 | "AWSTemplateFormatVersion": "2010-09-09" 18 | 19 | Parameters: 20 | PublicBucket: 21 | Description: S3 bucket for SFTP user home directories 22 | Type: String 23 | 24 | SubscribedBucket: 25 | Description: S3 bucket for shared usage across users 26 | Type: String 27 | 28 | "Outputs": 29 | "StackArn": 30 | "Value": 31 | "Ref": "AWS::StackId" 32 | TransferIdentityProviderInvocationRole: 33 | "Description": "IAM Role to pass to transfer createServer call as part of optional IdentityProviderDetails" 34 | "Value": 35 | "Fn::Sub": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/${TransferIdentityProviderRole}" 36 | TransferIdentityProviderUrl: 37 | "Description": "URL to pass to transfer createServer call as part of optional IdentityProviderDetails" 38 | "Value": 39 | "Fn::Join": 40 | - "" 41 | - - "https://" 42 | - "Ref": "CustomIdentityProviderApi" 43 | - ".execute-api." 44 | - "Ref": "AWS::Region" 45 | - ".amazonaws.com/" 46 | - "Ref": "ApiStage" 47 | LambdaName: 48 | Description: Name of the Lambda function handling AuthN and AuthZ 49 | Value: !Ref GetUserConfigLambda 50 | 51 | "Resources": 52 | "ApiCloudWatchLogsRole": 53 | "Type": "AWS::IAM::Role" 54 | "Properties": 55 | "AssumeRolePolicyDocument": 56 | "Statement": 57 | - "Action": 58 | - "sts:AssumeRole" 59 | "Effect": "Allow" 60 | "Principal": 61 | "Service": 62 | - "apigateway.amazonaws.com" 63 | "Version": "2012-10-17" 64 | "Policies": 65 | - "PolicyDocument": 66 | "Statement": 67 | - "Action": 68 | - "logs:CreateLogGroup" 69 | - "logs:CreateLogStream" 70 | - "logs:DescribeLogGroups" 71 | - "logs:DescribeLogStreams" 72 | - "logs:PutLogEvents" 73 | - "logs:GetLogEvents" 74 | - "logs:FilterLogEvents" 75 | "Effect": "Allow" 76 | "Resource": "*" 77 | "Version": "2012-10-17" 78 | "PolicyName": "ApiGatewayLogsPolicy" 79 | 80 | "ApiDeployment": 81 | "Type": "AWS::ApiGateway::Deployment" 82 | "DependsOn": 83 | - "GetUserConfigRequest" 84 | "Properties": 85 | "RestApiId": 86 | "Ref": "CustomIdentityProviderApi" 87 | "StageName": "dummystagefordeployment" 88 | 89 | "ApiLoggingAccount": 90 | "Type": "AWS::ApiGateway::Account" 91 | "Properties": 92 | "CloudWatchRoleArn": 93 | "Fn::GetAtt": "ApiCloudWatchLogsRole.Arn" 94 | 95 | "ApiStage": 96 | "Type": "AWS::ApiGateway::Stage" 97 | "DependsOn": 98 | - "ApiLoggingAccount" 99 | "Properties": 100 | "DeploymentId": 101 | "Ref": "ApiDeployment" 102 | "MethodSettings": 103 | - "DataTraceEnabled": !!bool "true" 104 | "HttpMethod": "*" 105 | "LoggingLevel": "INFO" 106 | "ResourcePath": "/*" 107 | "RestApiId": 108 | "Ref": "CustomIdentityProviderApi" 109 | "StageName": "prod" 110 | 111 | "CustomIdentityProviderApi": 112 | "Type": "AWS::ApiGateway::RestApi" 113 | "Properties": 114 | "Description": "API used for Greeting requests" 115 | "FailOnWarnings": !!bool "true" 116 | "Name": "Transfer Bring Your Own Auth template API" 117 | 118 | TransferUserRole: 119 | Type: AWS::IAM::Role 120 | Properties: 121 | Path: '/' 122 | AssumeRolePolicyDocument: 123 | Version: "2012-10-17" 124 | Statement: 125 | - 126 | Effect: "Allow" 127 | Principal: 128 | Service: 129 | - transfer.amazonaws.com 130 | Action: 131 | - "sts:AssumeRole" 132 | Policies: 133 | - 134 | PolicyName: "root" 135 | PolicyDocument: 136 | Version: "2012-10-17" 137 | Statement: 138 | - 139 | Effect: "Allow" 140 | Action: "s3:*" 141 | Resource: "*" 142 | 143 | "GetUserConfigLambda": 144 | "Type": "AWS::Lambda::Function" 145 | "Properties": 146 | Code: ../src/authorizor 147 | "Description": "A function to provide IAM roles and policies for given user and serverId." 148 | "Handler": lambda.handler 149 | "Role": 150 | "Fn::GetAtt": "LambdaExecutionRole.Arn" 151 | "Runtime": "nodejs12.x" 152 | Environment: 153 | Variables: 154 | PUBLIC_BUCKET: !Ref PublicBucket 155 | SUBSCRIBE_BUCKET: !Ref SubscribedBucket 156 | USER_ROLE: !GetAtt TransferUserRole.Arn 157 | 158 | "GetUserConfigLambdaPermission": 159 | "Type": "AWS::Lambda::Permission" 160 | "Properties": 161 | "Action": "lambda:invokeFunction" 162 | "FunctionName": 163 | "Fn::GetAtt": "GetUserConfigLambda.Arn" 164 | "Principal": "apigateway.amazonaws.com" 165 | "SourceArn": 166 | "Fn::Join": 167 | - "" 168 | - - "arn:aws:execute-api:" 169 | - "Ref": "AWS::Region" 170 | - ":" 171 | - "Ref": "AWS::AccountId" 172 | - ":" 173 | - "Ref": "CustomIdentityProviderApi" 174 | - "/*" 175 | 176 | "GetUserConfigRequest": 177 | "Type": "AWS::ApiGateway::Method" 178 | "DependsOn": "GetUserConfigLambdaPermission" 179 | "Properties": 180 | "AuthorizationType": "AWS_IAM" 181 | "HttpMethod": "GET" 182 | "Integration": 183 | "IntegrationHttpMethod": "POST" 184 | "IntegrationResponses": 185 | - "StatusCode": "200" 186 | "RequestTemplates": 187 | "application/json": "{\n \"username\": \"$input.params('username')\",\n \"password\": \"$input.params('Password')\",\n \"serverId\": \"$input.params('serverId')\"\n}\n" 188 | "Type": "AWS" 189 | "Uri": 190 | "Fn::Join": 191 | - "" 192 | - - "arn:aws:apigateway:" 193 | - "Ref": "AWS::Region" 194 | - ":lambda:path/2015-03-31/functions/" 195 | - "Fn::GetAtt": 196 | - "GetUserConfigLambda" 197 | - "Arn" 198 | - "/invocations" 199 | "MethodResponses": 200 | - "ResponseModels": 201 | "application/json": "UserConfigResponseModel" 202 | "StatusCode": "200" 203 | "RequestParameters": 204 | "method.request.header.Password": !!bool "false" 205 | "method.request.querystring.serverId": !!bool "true" 206 | "method.request.querystring.username": !!bool "true" 207 | "ResourceId": 208 | "Ref": "GetUserConfigResource" 209 | "RestApiId": 210 | "Ref": "CustomIdentityProviderApi" 211 | 212 | "GetUserConfigResource": 213 | "Type": "AWS::ApiGateway::Resource" 214 | "Properties": 215 | "ParentId": 216 | "Ref": "UserNameResource" 217 | "PathPart": "config" 218 | "RestApiId": 219 | "Ref": "CustomIdentityProviderApi" 220 | 221 | "GetUserConfigResponseModel": 222 | "Type": "AWS::ApiGateway::Model" 223 | "Properties": 224 | "ContentType": "application/json" 225 | "Description": "API reponse for GetUserConfig" 226 | "Name": "UserConfigResponseModel" 227 | "RestApiId": 228 | "Ref": "CustomIdentityProviderApi" 229 | "Schema": 230 | "$schema": "http://json-schema.org/draft-04/schema#" 231 | "properties": 232 | "HomeDirectory": 233 | "type": "string" 234 | "Policy": 235 | "type": "string" 236 | "PublicKeys": 237 | "items": 238 | "type": "string" 239 | "type": "array" 240 | "Role": 241 | "type": "string" 242 | "title": "UserUserConfig" 243 | "type": "object" 244 | 245 | "LambdaExecutionRole": 246 | "Type": "AWS::IAM::Role" 247 | "Properties": 248 | "AssumeRolePolicyDocument": 249 | "Statement": 250 | - "Action": 251 | - "sts:AssumeRole" 252 | "Effect": "Allow" 253 | "Principal": 254 | "Service": 255 | - "lambda.amazonaws.com" 256 | "Version": "2012-10-17" 257 | "ManagedPolicyArns": 258 | - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 259 | 260 | "ServerIdResource": 261 | "Type": "AWS::ApiGateway::Resource" 262 | "Properties": 263 | "ParentId": 264 | "Ref": "ServersResource" 265 | "PathPart": "{serverId}" 266 | "RestApiId": 267 | "Ref": "CustomIdentityProviderApi" 268 | 269 | "ServersResource": 270 | "Type": "AWS::ApiGateway::Resource" 271 | "Properties": 272 | "ParentId": 273 | "Fn::GetAtt": 274 | - "CustomIdentityProviderApi" 275 | - "RootResourceId" 276 | "PathPart": "servers" 277 | "RestApiId": 278 | "Ref": "CustomIdentityProviderApi" 279 | 280 | "TransferIdentityProviderRole": 281 | "Type": "AWS::IAM::Role" 282 | "Properties": 283 | "AssumeRolePolicyDocument": 284 | "Statement": 285 | - "Action": 286 | - "sts:AssumeRole" 287 | "Effect": "Allow" 288 | "Principal": 289 | "Service": "transfer.amazonaws.com" 290 | "Version": "2012-10-17" 291 | "Policies": 292 | - "PolicyDocument": 293 | "Statement": 294 | - "Action": 295 | - "execute-api:Invoke" 296 | "Effect": "Allow" 297 | "Resource": 298 | "Fn::Sub": "arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${CustomIdentityProviderApi}/prod/GET/*" 299 | "Version": "2012-10-17" 300 | "PolicyName": "TransferCanInvokeThisApi" 301 | - "PolicyDocument": 302 | "Statement": 303 | - "Action": 304 | - "apigateway:GET" 305 | "Effect": "Allow" 306 | "Resource": "*" 307 | "Version": "2012-10-17" 308 | "PolicyName": "TransferCanReadThisApi" 309 | 310 | "UserNameResource": 311 | "Type": "AWS::ApiGateway::Resource" 312 | "Properties": 313 | "ParentId": 314 | "Ref": "UsersResource" 315 | "PathPart": "{username}" 316 | "RestApiId": 317 | "Ref": "CustomIdentityProviderApi" 318 | 319 | "UsersResource": 320 | "Type": "AWS::ApiGateway::Resource" 321 | "Properties": 322 | "ParentId": 323 | "Ref": "ServerIdResource" 324 | "PathPart": "users" 325 | "RestApiId": 326 | "Ref": "CustomIdentityProviderApi" 327 | ... 328 | -------------------------------------------------------------------------------- /cloudformation/sftp_server.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy of 5 | # this software and associated documentation files (the "Software"), to deal in 6 | # the Software without restriction, including without limitation the rights to 7 | # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 8 | # the Software, and to permit persons to whom the Software is furnished to do so. 9 | 10 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 11 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 12 | # FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 13 | # COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 14 | # IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 15 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 16 | 17 | AWSTemplateFormatVersion: 2010-09-09 18 | Description: Create a Transfer for SFTP server 19 | 20 | Parameters: 21 | IdpInvocationRole: 22 | Description: Role for invoking the IDP endpoint 23 | Type: String 24 | IdpUrl: 25 | Description: Url for invoking the IDP endpoint 26 | Type: String 27 | 28 | Resources: 29 | SftpLoggingRole: 30 | Type: AWS::IAM::Role 31 | Properties: 32 | Path: "/" 33 | AssumeRolePolicyDocument: 34 | Version: "2012-10-17" 35 | Statement: 36 | - 37 | Effect: "Allow" 38 | Principal: 39 | Service: 40 | - transfer.amazonaws.com 41 | Action: 42 | - "sts:AssumeRole" 43 | Policies: 44 | - 45 | PolicyName: "LoggerPolicy" 46 | PolicyDocument: 47 | Version: "2012-10-17" 48 | Statement: 49 | - 50 | Effect: "Allow" 51 | Action: 52 | - "logs:CreateLogGroup" 53 | - "logs:CreateLogStream" 54 | - "logs:PutLogEvents" 55 | Resource: "*" 56 | 57 | SftpServer: 58 | Type: AWS::Transfer::Server 59 | Properties: 60 | LoggingRole: !GetAtt SftpLoggingRole.Arn 61 | EndpointType: PUBLIC 62 | IdentityProviderType: API_GATEWAY 63 | IdentityProviderDetails: 64 | InvocationRole: !Ref IdpInvocationRole 65 | Url: !Ref IdpUrl 66 | 67 | Outputs: 68 | SftpServerId: 69 | Description: SFTP Server ID 70 | Value: !GetAtt "SftpServer.ServerId" -------------------------------------------------------------------------------- /src/authorizor/lambda.js: -------------------------------------------------------------------------------- 1 | // Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | // this software and associated documentation files (the "Software"), to deal in 5 | // the Software without restriction, including without limitation the rights to 6 | // use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | // the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | // FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | // COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | // IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | // CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | 'use strict'; 17 | 18 | /** 19 | * The following lambda performs 2 functions. The first authorizes a user based on 20 | * a very simple in-memory database (see `userDb` below). The second function is to 21 | * craft a scope down policy for the specific user to restrict what the user can 22 | * access via SFTP. The scope down policy is overlayed atop the more generic user 23 | * role which is passed to the Lambda as an environment variable. The scope down 24 | * policy will restrict users to their particular home directory and give them read 25 | * access to those allowDirectories to which they've subscribed. 26 | */ 27 | 28 | var __version__ = '0.1'; 29 | 30 | // GetUserConfig Lambda 31 | var public_bucket = process.env.PUBLIC_BUCKET; 32 | var subscription_bucket = process.env.SUBSCRIBE_BUCKET; 33 | var userRoleArn = process.env.USER_ROLE; 34 | var sftpServerId = process.env.SERVER_ID; 35 | 36 | var userDb = { 37 | "alice": { 38 | "password": "Password01", 39 | "policy": { 40 | "Version": "2012-10-17", 41 | "Statement": [ 42 | { 43 | "Sid": "AllowListingOfFolder", 44 | "Action": [ 45 | "s3:ListBucket" 46 | ], 47 | "Effect": "Allow", 48 | "Resource": [ 49 | "arn:aws:s3:::" + public_bucket, 50 | "arn:aws:s3:::" + subscription_bucket 51 | ] 52 | }, 53 | { 54 | "Sid": "AllowObjectAccess", 55 | "Effect": "Allow", 56 | "Action": [ 57 | "s3:GetObject", 58 | "s3:GetObjectVersion" 59 | ], 60 | "Resource": [ 61 | "arn:aws:s3:::" + public_bucket + "/global/*", 62 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/indices/*", 63 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/indices/*", 64 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/equities/*" 65 | ] 66 | } 67 | ] 68 | }, 69 | "directoryMap": [ 70 | { 71 | "Entry": "/public/research", 72 | "Target": "/"+ public_bucket 73 | }, 74 | { 75 | "Entry": "/subscribed/2018/indices", 76 | "Target": "/"+ subscription_bucket + "/historical/2018/indices" 77 | }, 78 | { 79 | "Entry": "/subscribed/2019/indices", 80 | "Target": "/"+ subscription_bucket + "/historical/2019/indices" 81 | }, 82 | { 83 | "Entry": "/subscribed/2019/equities", 84 | "Target": "/"+ subscription_bucket + "/historical/2019/equities" 85 | } 86 | ] 87 | }, 88 | "bryan": { 89 | "password": "Password02", 90 | "policy": { 91 | "Version": "2012-10-17", 92 | "Statement": [ 93 | { 94 | "Sid": "AllowListingOfFolder", 95 | "Action": [ 96 | "s3:ListBucket" 97 | ], 98 | "Effect": "Allow", 99 | "Resource": [ 100 | "arn:aws:s3:::" + public_bucket, 101 | "arn:aws:s3:::" + subscription_bucket 102 | ] 103 | }, 104 | { 105 | "Sid": "AllowObjectAccess", 106 | "Effect": "Allow", 107 | "Action": [ 108 | "s3:GetObject", 109 | "s3:GetObjectVersion" 110 | ], 111 | "Resource": [ 112 | "arn:aws:s3:::" + public_bucket + "/global/*", 113 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/indices/*", 114 | "arn:aws:s3:::" + subscription_bucket + "/historical/2018/equities/*", 115 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/credit/*", 116 | "arn:aws:s3:::" + subscription_bucket + "/historical/2019/equities/*" 117 | ] 118 | } 119 | ] 120 | }, 121 | "directoryMap": [ 122 | { 123 | "Entry": "/public/research", 124 | "Target": "/"+ public_bucket 125 | }, 126 | { 127 | "Entry": "/subscribed/2018/indices", 128 | "Target": "/"+ subscription_bucket + "/historical/2018/indices" 129 | }, 130 | { 131 | "Entry": "/subscribed/2018/equities", 132 | "Target": "/"+ subscription_bucket + "/historical/2018/equities" 133 | }, 134 | { 135 | "Entry": "/subscribed/2019/credit", 136 | "Target": "/"+ subscription_bucket + "/historical/2019/credit" 137 | }, 138 | { 139 | "Entry": "/subscribed/2019/equities", 140 | "Target": "/"+ subscription_bucket + "/historical/2019/equities" 141 | } 142 | ] 143 | } 144 | }; 145 | 146 | function authenticated(username, password) { 147 | if (username in userDb) { 148 | var userRecord = userDb[username]; 149 | 150 | if (password == userRecord.password) { 151 | return true; 152 | } 153 | } 154 | 155 | return false; 156 | } 157 | 158 | function getDirectoryMapping(username) { 159 | var userRecord = userDb[username]; 160 | return userRecord.directoryMap; 161 | } 162 | 163 | function getScopeDownPolicy(username) { 164 | var userRecord = userDb[username]; 165 | return userRecord.policy; 166 | } 167 | 168 | exports.handler = (event, context, callback) => { 169 | console.log("Event:", JSON.stringify(event)); 170 | 171 | var response = {}; 172 | 173 | if (authenticated(event.username, event.password)) { 174 | var scopeDownPolicy = getScopeDownPolicy(event.username); 175 | var directoryMapping = getDirectoryMapping(event.username); 176 | 177 | response = { 178 | Role: userRoleArn, 179 | Policy: JSON.stringify(scopeDownPolicy), 180 | HomeDirectoryType: "LOGICAL", 181 | HomeDirectoryDetails: JSON.stringify(directoryMapping) 182 | }; 183 | } 184 | 185 | console.log("Returning ", JSON.stringify(response)); 186 | callback(null, response); 187 | }; 188 | -------------------------------------------------------------------------------- /src/datagen/lambda_function.py: -------------------------------------------------------------------------------- 1 | # Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | # Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | # this software and associated documentation files (the "Software"), to deal in 5 | # the Software without restriction, including without limitation the rights to 6 | # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | # the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | # FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | # COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | # IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | """ 17 | Custom resource handler for the AWS SFTP Logical Directories blog post. 18 | This code will generate a folder structure in two S3 buckets specific to the 19 | needs of the blog post. Much of the helper code was pulled directory from the 20 | the custom-resource-helper code at https://github.com/aws-cloudformation/custom-resource-helper. 21 | We did not use the CfnResource code deliberately so we could keep the Lambda 22 | function lightweight. 23 | """ 24 | 25 | import logging 26 | import os 27 | import boto3 28 | import json 29 | import time 30 | import random 31 | import string 32 | 33 | from botocore.vendored import requests 34 | 35 | logger = logging.getLogger(__name__) 36 | logger.setLevel('DEBUG') 37 | 38 | ACTION_CREATE = "Create" 39 | ACTION_UPDATE = "Update" 40 | ACTION_DELETE = "Delete" 41 | 42 | STATUS_SUCCESS = "SUCCESS" 43 | STATUS_FAILED = "FAILED" 44 | 45 | def _send_response(response_url, response_body, put=requests.put): 46 | try: 47 | json_response_body = json.dumps(response_body) 48 | except Exception as e: 49 | msg = "Failed to convert response to json: {}".format(str(e)) 50 | logger.error(msg, exc_info=True) 51 | response_body = {'Status': 'FAILED', 'Data': {}, 'Reason': msg} 52 | json_response_body = json.dumps(response_body) 53 | logger.debug("CFN response URL: {}".format(response_url)) 54 | logger.debug(json_response_body) 55 | headers = {'content-type': '', 'content-length': str(len(json_response_body))} 56 | while True: 57 | try: 58 | response = put(response_url, data=json_response_body, headers=headers) 59 | logger.info("CloudFormation returned status code: {}".format(response.reason)) 60 | break 61 | except Exception as e: 62 | logger.error("Unexpected failure sending response to CloudFormation {}".format(e), exc_info=True) 63 | time.sleep(5) 64 | 65 | def _rand_string(l): 66 | return ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(l)) 67 | 68 | def _gen_physical_resource_id(event): 69 | if "PhysicalResourceId" in event.keys(): 70 | logger.info("PhysicalResourceId present in event, Using that for response") 71 | return event['PhysicalResourceId'] 72 | else: 73 | logger.info("No physical resource id returned, generating one...") 74 | return event['StackId'].split('/')[1] + '_' + event['LogicalResourceId'] + '_' + _rand_string(8) 75 | 76 | def _process(event, context, action): 77 | 78 | public_research_bucket = os.environ['public_research'] 79 | subscriptions_bucket = os.environ['subscriptions'] 80 | 81 | files = [ 82 | [public_research_bucket, 'global/document1.txt'], 83 | [public_research_bucket, 'global/northamer/document1-northamer.txt'], 84 | [public_research_bucket, 'global/northamer/document2-northamer.txt'], 85 | [public_research_bucket, 'global/southamer/document1-southamer.txt'], 86 | [subscriptions_bucket, 'historical/2018/indices/index1-2018.txt'], 87 | [subscriptions_bucket, 'historical/2018/indices/index2-2018.txt'], 88 | [subscriptions_bucket, 'historical/2018/equities/equity1-2018.txt'], 89 | [subscriptions_bucket, 'historical/2019/credit/credit1-2019.txt'], 90 | [subscriptions_bucket, 'historical/2019/equities/equity1-2019.txt'], 91 | [subscriptions_bucket, 'historical/2019/equities/equity2-2019.txt'], 92 | [subscriptions_bucket, 'historical/2019/indices/index1-2019.txt'], 93 | [subscriptions_bucket, 'historical/2019/indices/index2-2019.txt'], 94 | [subscriptions_bucket, 'historical/2019/indices/index3-2019.txt'], 95 | ] 96 | 97 | s3 = boto3.client('s3') 98 | 99 | if action == ACTION_CREATE: 100 | file_text = "Test data generated by CloudFormation stack %s. These objects will be automatically deleted on stack cleanup." % event['StackId'] 101 | for bucket, key in files: 102 | print ("Putting data to s3://{}/{}".format (bucket, key)) 103 | s3.put_object(Bucket=bucket, Key=key, Body=file_text) 104 | elif action == ACTION_DELETE: 105 | for bucket, key in files: 106 | s3.delete_object(Bucket=bucket, Key=key) 107 | 108 | response_body = { 109 | 'Status': STATUS_SUCCESS, 110 | 'PhysicalResourceId': _gen_physical_resource_id(event), 111 | 'StackId': event['StackId'], 112 | 'RequestId': event['RequestId'], 113 | 'LogicalResourceId': event['LogicalResourceId'], 114 | 'Reason': "", 115 | 'Data': {}, 116 | } 117 | 118 | _send_response(event['ResponseURL'], response_body) 119 | 120 | def handler(event, context): 121 | logger.debug(event) 122 | try: 123 | _process(event, context, event['RequestType']) 124 | except Exception as e: 125 | msg = "Failed to process custom resource event: {}".format(str(e)) 126 | logger.error(msg, exc_info=True) 127 | response_body = {'Status': STATUS_FAILED, 'Data': {}, 'Reason': msg} 128 | _send_response(event['ResponseURL'], response_body) 129 | --------------------------------------------------------------------------------