├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── app.py ├── cdk.json ├── docs └── diagram_arch.png ├── lambda ├── detect_faces.py └── write_results.py ├── rekognition_processing_app ├── __init__.py └── rekognition_processing_app_stack.py ├── requirements.txt └── setup.py /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 4 | software and associated documentation files (the "Software"), to deal in the Software 5 | without restriction, including without limitation the rights to use, copy, modify, 6 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 7 | permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 10 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 11 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 12 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 13 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 14 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazon Rekognition Large Scale Processing 2 | 3 | The objective of this article is to carry out a detailed review of strategies and a solution that we can incorporate into high demand applications in the use of the Amazon Rekognition API (specifically DetectFaces) that require temporarily exceeding the service quota without incurring errors on the side. of the application, we will see it from the point of view of Architecture, as well as recommendations on handling errors in API calls. 4 | 5 | This code sample is discussed in detail in this AWS [Blog Post](https://aws.amazon.com/es/blogs/aws-spanish/escalando-aplicaciones-de-reconocimiento-de-imagenes-con-amazon-rekognition/). 6 | 7 | ### Architecture 8 | 9 | ![Architecture Diagram](docs/diagram_arch.png) 10 | 11 | ### Usage 12 | 13 | #### Prerequisites 14 | 15 | - An AWS Account 16 | - AWS CDK for Python 17 | - AWS CLI configured with access creadentials 18 | - Python 3.6+ 19 | - IAM Privileges to deploy the components of the architecture 20 | 21 | #### Deployment 22 | 23 | 1 - Clone the code in this repository. 24 | 25 | 2 - Proceed with the following commands in the Terminal. 26 | 27 | ``` 28 | npm install -g aws-cdk 29 | cd amazon-rekognition-large-scale-processing 30 | python3 -m venv .env 31 | source .env/bin/activate 32 | pip install -r requirements.txt 33 | cdk deploy 34 | ``` 35 | 36 | 3 - Check the correct deployment: The result from the last section is the AWS S3 bucket in wich to upload the images, the results of the image processing will be available in the AWS DynamoDB Table ```detect_faces_results``` 37 | 38 | #### Cleanup 39 | 40 | In order to delete all the components deployed by this solution and avoid additional charges: 41 | 42 | 1 - Proceed with the following commands in the Terminal. 43 | 44 | ``` 45 | cd amazon-rekognition-large-scale-processing 46 | cdk destroy # This comand will delete all the deployed resources 47 | ``` 48 | 49 | 2 - Delete the AWS DynamoDB Table ```detect_faces_results```: Navigate to AWS DynamoDB service in the AWS Console > Look for the results table and proceed to delete the table. 50 | 51 | 3 - Delete the Amazon S3 Bucket: Navigate to the Amazon S3 service in the AWS Console > Search the bucket name from the output of the deployment process and proceed to delete the bucket. 52 | 53 | ### Making changes to the code and customization 54 | 55 | The [contributing guidelines](CONTRIBUTING.md) contains some instructions about how to run the front-end locally and make changes to the back-end stack. 56 | 57 | ## Contributing 58 | 59 | Contributions are more than welcome. Please read the [code of conduct](CODE_OF_CONDUCT.md) and the [contributing guidelines](CONTRIBUTING.md). 60 | 61 | ## Licensing 62 | 63 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 64 | 65 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 66 | 67 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """ 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | SPDX-License-Identifier: MIT-0 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 7 | software and associated documentation files (the "Software"), to deal in the Software 8 | without restriction, including without limitation the rights to use, copy, modify, 9 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 10 | permit persons to whom the Software is furnished to do so. 11 | 12 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 13 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 14 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 15 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 16 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 17 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 18 | """ 19 | 20 | from aws_cdk import core 21 | 22 | from rekognition_processing_app.rekognition_processing_app_stack import RekognitionProcessingAppStack 23 | 24 | 25 | app = core.App() 26 | RekognitionProcessingAppStack(app, "rekognition-processing-app", env={'region': 'us-east-1'}) 27 | 28 | app.synth() 29 | -------------------------------------------------------------------------------- /cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "context": { 4 | "@aws-cdk/core:enableStackNameDuplicates": "true", 5 | "aws-cdk:enableDiffNoFail": "true", 6 | "@aws-cdk/core:stackRelativeExports": "true" 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /docs/diagram_arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-rekognition-large-scale-processing/a09228e275b9995474051bd008010215ffc1bfb9/docs/diagram_arch.png -------------------------------------------------------------------------------- /lambda/detect_faces.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | SPDX-License-Identifier: MIT-0 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 6 | software and associated documentation files (the "Software"), to deal in the Software 7 | without restriction, including without limitation the rights to use, copy, modify, 8 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 9 | permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 12 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 13 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 14 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 15 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 16 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | """ 18 | 19 | import json 20 | import logging 21 | import boto3 22 | from botocore.exceptions import ClientError 23 | import os 24 | from time import sleep 25 | from random import randint 26 | 27 | # Environ Variables 28 | SQS_RESPONSE_QUEUE = os.environ['SQS_RESPONSE_QUEUE'] 29 | MAX_RETRIES = 3 30 | 31 | # Boto 3 Resources / Clients 32 | rekognition_client = boto3.client('rekognition', region_name='us-east-1') 33 | sqs_resource = boto3.resource('sqs') 34 | 35 | 36 | def detect_faces_rekognition(s3_bucket_name, s3_object_key, attributes='ALL'): 37 | """ 38 | Detect Rekognition faces with Exponential backoff + jitter and max retries 39 | :param s3_bucket_name: str that contains the bucket name 40 | :param s3_object_key: str that contains the object key name 41 | :param attributes: str used to retrieve all Amazon Rekognition labels on DetectFaces API 42 | :return: Image Response dict / Error dict 43 | """ 44 | 45 | retry_rekgonition = True 46 | num_retries = 0 47 | 48 | while retry_rekgonition and num_retries <= MAX_RETRIES: 49 | 50 | try: 51 | 52 | print(f'Detecting {s3_bucket_name}/{s3_object_key}') 53 | 54 | image_response = rekognition_client.detect_faces(Image={ 55 | 'S3Object': { 56 | 'Bucket': s3_bucket_name, 57 | 'Name': s3_object_key 58 | } 59 | }, 60 | Attributes=[attributes] 61 | ) 62 | 63 | print(image_response) 64 | 65 | # Check if the call to Rekognition returned any value 66 | if image_response: 67 | # Add the object key in the message to send to Write Queue 68 | image_response['s3_object_key'] = s3_object_key 69 | 70 | # Send Mesage to SQS Queue 71 | send_response_sqs(image_response) 72 | retry_rekgonition = False 73 | 74 | return image_response 75 | 76 | except ClientError as error: 77 | 78 | if num_retries == MAX_RETRIES: 79 | raise error 80 | 81 | if error.__class__.__name__ == 'ThrottlingException' or\ 82 | error.__class__.__name__ == 'ProvisionedThroughputExceededException': 83 | 84 | num_retries += 1 85 | wait_time = 0.10 * (2 ** num_retries) 86 | rand_jitter = randint(200, 1000) / 1000 87 | sleep(wait_time + rand_jitter) 88 | 89 | elif error.__class__.__name__ == 'LimitExceededException': 90 | print(error.__class__.__name__) 91 | print(error.response) 92 | retry_rekgonition = False 93 | return error.response 94 | 95 | else: 96 | print(error.response) 97 | retry_rekgonition = False 98 | return error.response 99 | 100 | 101 | def send_response_sqs(message_body): 102 | """ 103 | Sends the response from Amazon Rekognition DetectFaces to a SQS queue 104 | :param message_body: Message to send to SQS queue dict 105 | :return: 106 | """ 107 | # Get the queue 108 | queue = sqs_resource.get_queue_by_name(QueueName=SQS_RESPONSE_QUEUE) 109 | 110 | # Send a new message 111 | try: 112 | response = queue.send_message(MessageBody=json.dumps(message_body)) 113 | message_id = response.get('MessageId') 114 | 115 | print(f'Sent Message {message_id}') 116 | 117 | except ClientError as error: 118 | print(error.response) 119 | return error.response 120 | 121 | 122 | def lambda_handler(event, context): 123 | """ 124 | This function is call on a SQS Queue event, takes the message of the queue, parses the Amazon S3 PutObject event 125 | message, then calls Amazon Rekognition DetectFaces API, finally sends a message to the Write Results SQS Queue 126 | :param event: S3 PutObject Event dict 127 | :param context: Context dict 128 | :return: Amazon Rekognition DetectFaces Response Dict 129 | """ 130 | 131 | print(event) 132 | print(event['Records'][0]['body'][0]) 133 | message_body = json.loads(event['Records'][0]['body']) 134 | print(message_body) 135 | 136 | s3_bucket_name = message_body['Records'][0]['s3']['bucket']['name'] 137 | s3_object_key = message_body['Records'][0]['s3']['object']['key'] 138 | 139 | print(f'Bucket = {s3_bucket_name}') 140 | print(f'Object Key = {s3_object_key}') 141 | 142 | return detect_faces_rekognition(s3_bucket_name, s3_object_key) 143 | -------------------------------------------------------------------------------- /lambda/write_results.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | SPDX-License-Identifier: MIT-0 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 6 | software and associated documentation files (the "Software"), to deal in the Software 7 | without restriction, including without limitation the rights to use, copy, modify, 8 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 9 | permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 12 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 13 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 14 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 15 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 16 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | """ 18 | 19 | import json 20 | import boto3 21 | from botocore.exceptions import ClientError 22 | import os 23 | from decimal import Decimal 24 | 25 | # Environ Variables 26 | TABLE_NAME = os.environ['TABLE_NAME'] 27 | # DynamoDB Resource 28 | dynamodb_resource = boto3.resource('dynamodb', region_name='us-east-1') 29 | 30 | 31 | def put_item_dynamodb(item): 32 | """ 33 | Call DynamoDB API PutItem 34 | :param item: Dictionary of id, FaceDetails and ResponseMetdata 35 | :return: DynamoDB PutItem response dict 36 | """ 37 | dynamodb_table = dynamodb_resource.Table(TABLE_NAME) 38 | 39 | try: 40 | put_item_response = dynamodb_table.put_item(Item=item) 41 | return put_item_response 42 | 43 | except ClientError as error: 44 | return error.response 45 | 46 | 47 | def parse_message_emotions(message): 48 | """ 49 | Parse the original SQS message to a DynamoDB compatible dict 50 | :param message: SQS Message Dict 51 | :return: Parsed dict to insert into DynamoDB Table 52 | """ 53 | face_details = message['FaceDetails'][0] 54 | response_metadata = message['ResponseMetadata'] 55 | # photo_id = str(uuid1()).split('-')[0] 56 | photo_id = message['s3_object_key'] 57 | 58 | item = {'id': photo_id, 59 | 'FaceDetails': face_details, 60 | 'ResponseMetadata': response_metadata} 61 | 62 | # for emotion in face_emotions: 63 | # emotion_type = emotion['Type'] 64 | # emotion_type = emotion_type.lower() 65 | # emotion_confidence = emotion['Confidence'] 66 | # item[emotion_type] = Decimal(str(emotion_confidence)) 67 | # print(f'{emotion_type} = {emotion_confidence}') 68 | 69 | # Parse the Float values in the message to Decimals 70 | ddb_item = json.loads(json.dumps(item), parse_float=Decimal) 71 | 72 | return ddb_item 73 | 74 | 75 | def lambda_handler(event, context): 76 | """ 77 | This function is call on a SQS Queue event, takes the message of the queue, parses the message and inserts a item 78 | into dynamoDB 79 | :param event: SQS Message dict 80 | :param context: Context dict 81 | :return: DynamoDB Response PutItem dict 82 | """ 83 | print(event) 84 | print(event['Records'][0]['body'][0]) 85 | message_body = json.loads(event['Records'][0]['body']) 86 | print(message_body) 87 | 88 | item = parse_message_emotions(message_body) 89 | return put_item_dynamodb(item) 90 | -------------------------------------------------------------------------------- /rekognition_processing_app/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-rekognition-large-scale-processing/a09228e275b9995474051bd008010215ffc1bfb9/rekognition_processing_app/__init__.py -------------------------------------------------------------------------------- /rekognition_processing_app/rekognition_processing_app_stack.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | SPDX-License-Identifier: MIT-0 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 6 | software and associated documentation files (the "Software"), to deal in the Software 7 | without restriction, including without limitation the rights to use, copy, modify, 8 | merge, publish, distribute, sublicense, and/or sell copies of the Software, and to 9 | permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 12 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A 13 | PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 14 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 15 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 16 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | """ 18 | 19 | from aws_cdk import ( 20 | aws_sqs as sqs, 21 | aws_s3 as s3, 22 | aws_lambda as _lambda, 23 | aws_lambda_event_sources as _lambda_events, 24 | aws_s3_notifications as s3n, 25 | aws_iam as iam, 26 | aws_dynamodb as dynamodb, 27 | core 28 | ) 29 | 30 | 31 | class RekognitionProcessingAppStack(core.Stack): 32 | 33 | def __init__(self, scope: core.Construct, id: str, **kwargs) -> None: 34 | super().__init__(scope, id, **kwargs) 35 | 36 | # Create image bucket 37 | image_bucket = s3.Bucket(self, 'inbound_image_s3_bucket') 38 | 39 | # Create the image processing queue 40 | image_process_queue = sqs.Queue( 41 | self, "image_process_queue", 42 | visibility_timeout=core.Duration.seconds(300), 43 | retention_period=core.Duration.days(1) 44 | ) 45 | 46 | # Create the image response queue 47 | response_queue = sqs.Queue( 48 | self, "results_queue", 49 | visibility_timeout=core.Duration.seconds(300), 50 | retention_period=core.Duration.days(1) 51 | ) 52 | 53 | # Set the put object notification to the SQS Queue 54 | image_bucket.add_event_notification(event=s3.EventType.OBJECT_CREATED_PUT, 55 | dest=s3n.SqsDestination(image_process_queue)) 56 | 57 | # Define the AWS Lambda to call Amazon Rekognition DetectFaces 58 | detect_faces_lambda = _lambda.Function(self, 'detect_faces', 59 | runtime=_lambda.Runtime.PYTHON_3_7, 60 | handler='detect_faces.lambda_handler', 61 | code=_lambda.Code.asset('./lambda'), 62 | timeout=core.Duration.seconds(30), 63 | environment={'SQS_RESPONSE_QUEUE': response_queue.queue_name}, 64 | reserved_concurrent_executions=50 65 | ) 66 | 67 | # Set SQS image_process_queue Queue as event source for detect_faces_lambda 68 | detect_faces_lambda.add_event_source(_lambda_events.SqsEventSource(image_process_queue, 69 | batch_size=1)) 70 | 71 | # Allow response queue messages from lambda 72 | response_queue.grant_send_messages(detect_faces_lambda) 73 | 74 | # Allow lambda to call Rekognition by adding a IAM Policy Statement 75 | detect_faces_lambda.add_to_role_policy(iam.PolicyStatement(actions=['rekognition:*'], 76 | resources=['*'])) 77 | # Allow lambda to read from S3 78 | image_bucket.grant_read(detect_faces_lambda) 79 | 80 | # Define the DynamoDB Table 81 | results_table = dynamodb.Table(self, 'detect_faces_results', 82 | table_name='detect_faces_results', 83 | partition_key=dynamodb.Attribute(name='id', type=dynamodb.AttributeType.STRING), 84 | read_capacity=200, 85 | write_capacity=200 86 | ) 87 | 88 | # Define the AWS Lambda to write results into DyanamoDB results_table 89 | write_results_lambda = _lambda.Function(self, 'write_results', 90 | runtime=_lambda.Runtime.PYTHON_3_7, 91 | handler='write_results.lambda_handler', 92 | code=_lambda.Code.asset('./lambda'), 93 | timeout=core.Duration.seconds(30), 94 | environment={'TABLE_NAME': results_table.table_name} 95 | ) 96 | 97 | # Set SQS response_queue Queue as event source for write_results_lambda results_table 98 | write_results_lambda.add_event_source(_lambda_events.SqsEventSource(response_queue, 99 | batch_size=1)) 100 | 101 | # Allow AWS Lambda write_results_lambda to Write to Dynamodb 102 | results_table.grant_write_data(write_results_lambda) 103 | 104 | # Allow AWS Lambda write_results_lambda to read messages from the SQS response_queue Queue 105 | response_queue.grant_consume_messages(write_results_lambda) 106 | 107 | # Output to Amazon S3 Image Bucket 108 | core.CfnOutput(self, 'cdk_output', 109 | value=image_bucket.bucket_name, 110 | description='Input Amazon S3 Image Bucket') -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | -e . 2 | pytest -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | 3 | with open("README.md") as fp: 4 | long_description = fp.read() 5 | 6 | setuptools.setup( 7 | name="rekognition_processing_app", 8 | version="0.0.1", 9 | 10 | description="A sample CDK Python app", 11 | long_description=long_description, 12 | long_description_content_type="text/markdown", 13 | 14 | author="author", 15 | 16 | package_dir={"": "rekognition_processing_app"}, 17 | packages=setuptools.find_packages(where="rekognition_processing_app"), 18 | 19 | install_requires=[ 20 | "aws-cdk.aws-s3", 21 | "aws-cdk.aws-s3-notifications", 22 | "aws-cdk.aws-lambda", 23 | "aws-cdk.aws-lambda-event-sources", 24 | "aws-cdk.aws-sqs", 25 | "aws-cdk.aws-dynamodb", 26 | "aws-cdk.aws-iam" 27 | ], 28 | 29 | python_requires=">=3.6", 30 | 31 | classifiers=[ 32 | "Development Status :: 4 - Beta", 33 | 34 | "Intended Audience :: Developers", 35 | 36 | "License :: OSI Approved :: Apache Software License", 37 | 38 | "Programming Language :: JavaScript", 39 | "Programming Language :: Python :: 3 :: Only", 40 | "Programming Language :: Python :: 3.6", 41 | "Programming Language :: Python :: 3.7", 42 | "Programming Language :: Python :: 3.8", 43 | 44 | "Topic :: Software Development :: Code Generators", 45 | "Topic :: Utilities", 46 | 47 | "Typing :: Typed", 48 | ], 49 | ) 50 | --------------------------------------------------------------------------------