├── LICENSE ├── NOTICE.txt ├── Notebooks ├── AmazonML_Demo.ipynb ├── Comprehend_Demo.ipynb ├── FashionMNIST_MXNet_Demo.ipynb ├── Lex_CreateBot_Demo.ipynb ├── Lex_Demo.ipynb ├── PollyPSE.xml ├── Polly_Demo.ipynb ├── Rekognition_Demo.ipynb ├── large.wav ├── mocha.wav ├── no.wav ├── small.wav └── special.wav ├── README.md └── serverless_predictions.MD /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /NOTICE.txt: -------------------------------------------------------------------------------- 1 | [aws-ai-bootcamp-labs] 2 | Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved. -------------------------------------------------------------------------------- /Notebooks/AmazonML_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Binary Classification with Amazon Machine Learning (Learning from Disaster)" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "***\n", 15 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 16 | "\n", 17 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 18 | "\n", 19 | "http://aws.amazon.com/apache2.0/\n", 20 | "\n", 21 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 22 | "***" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "Description: \n", 30 | "This demo is based on the popular [\"Titanic: Machine Learning from Disaster\"](https://www.kaggle.com/c/titanic) Kaggle competition. I highly recommend creating an account and taking the full challenge as described on the competition page. \n", 31 | "\n", 32 | "From the Kaggle website: \n", 33 | "The sinking of the RMS Titanic is one of the most infamous shipwrecks in history. On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This sensational tragedy shocked the international community and led to better safety regulations for ships.\n", 34 | "\n", 35 | "One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.\n", 36 | "\n", 37 | "We are using the sample dataset provided of passengers on board the ship to training a model with Amazon Machine Learning to predict survival chances with the real time predictions endpoint.\n", 38 | "\n", 39 | "Helpful resources:\n", 40 | "[Amazon Machine Learning – Make Data-Driven Decisions at Scale](https://aws.amazon.com/blogs/aws/amazon-machine-learning-make-data-driven-decisions-at-scale/) \n", 41 | "[Building a Binary Classification Model with Amazon Machine Learning and Amazon Redshift](https://blogs.aws.amazon.com/bigdata/post/TxGVITXN9DT5V6/Building-a-Binary-Classification-Model-with-Amazon-Machine-Learning-and-Amazon-R)\n", 42 | "\n", 43 | "Comments:\n", 44 | "This is meant to be an asynchronous demo where one quickly walks through the deployment sections, runs all cells above the real time prediction section and gets back to the results after about 20 minutes.\n", 45 | "\n", 46 | "Estimated cost: The maximum cost for a single run would be 0.42$. The Data Analysis and Model Building cost depends on the size of the input data, the number of attributes within it, and the number and types of transformations applied. As this is very small scale demo the cost should be even lower.\n", 47 | "The cost of real time predictions is be negligible.\n", 48 | "\n", 49 | "** Prerequisites: **\n", 50 | "\n", 51 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you need to apply the following minimum managed policies to your user or role:\n", 52 | "\n", 53 | "* AmazonMachineLearningFullAccess \n", 54 | "* AmazonS3FullAccess \n", 55 | "\n", 56 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope fof this workshop.\n", 57 | "\n", 58 | "\n", 59 | "Demo Author: Stas Vonholsky" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": null, 65 | "metadata": {}, 66 | "outputs": [], 67 | "source": [ 68 | "import random, os, string, urllib, json, time, sys\n", 69 | "from boto3.session import Session\n", 70 | "\n", 71 | "# Generate session prefix\n", 72 | "demo_pre = \"ipydemo-\"\n", 73 | "N = 4\n", 74 | "random_pre = ''.join(random.SystemRandom().choice(string.ascii_lowercase) for _ in range(N)) + \"-\"\n", 75 | "resource_pre = demo_pre + random_pre\n", 76 | "print (\"\\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\")\n", 77 | "print (\"Demo session ID: \" + resource_pre[:-1])\n", 78 | "print (\"@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\")\n", 79 | "\n", 80 | "# Fetch relevant metadata\n", 81 | "work_dir = \"resources/\"\n", 82 | "meta_data_pre = \"http://169.254.169.254/latest/meta-data/\"\n", 83 | "\n", 84 | "region = 'eu-west-1'\n", 85 | " \n", 86 | "session = Session(region_name=region)\n", 87 | "s3 = session.client('s3')\n", 88 | "ml = session.client('machinelearning')\n", 89 | "\n", 90 | "#File paths\n", 91 | "train_data = \"train.csv\"" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "#Create bucket and upload training data\n", 101 | "s3_bucket = resource_pre + 'aml-bucket'\n", 102 | "s3.create_bucket(Bucket=s3_bucket,CreateBucketConfiguration={'LocationConstraint': region},)\n", 103 | "print (\"Created S3 bucket: \" + s3_bucket)\n", 104 | "\n", 105 | "path_to_train_data = work_dir + train_data\n", 106 | "!aws s3 cp $path_to_train_data s3://$s3_bucket\n", 107 | "\n", 108 | "#Grant read permissions to AML on Bucket\n", 109 | "s3_bucket_policy = \"\"\"{\n", 110 | " \"Version\": \"2012-10-17\",\n", 111 | " \n", 112 | " \"Statement\": [\n", 113 | " {\n", 114 | " \"Sid\": \"AmazonML_s3:ListBucket\",\n", 115 | " \"Effect\": \"Allow\",\n", 116 | " \"Principal\": {\n", 117 | " \"Service\": \"machinelearning.amazonaws.com\"\n", 118 | " },\n", 119 | " \"Action\": \"s3:ListBucket\",\n", 120 | " \"Resource\": \"arn:aws:s3:::\"\"\"+ s3_bucket +\"\"\"\"\n", 121 | " },\n", 122 | " {\n", 123 | " \"Sid\": \"AmazonML_s3:GetObject:PutObject\",\n", 124 | " \"Effect\": \"Allow\",\n", 125 | " \"Principal\": {\n", 126 | " \"Service\": \"machinelearning.amazonaws.com\"\n", 127 | " },\n", 128 | " \"Action\": [\"s3:GetObject\",\"s3:PutObject\"],\n", 129 | " \"Resource\": \"arn:aws:s3:::\"\"\"+ s3_bucket +\"\"\"/*\"\n", 130 | " }\n", 131 | " ]\n", 132 | "}\"\"\"\n", 133 | "trust_policy = json.loads(s3_bucket_policy)\n", 134 | "trust_policy = json.dumps(s3_bucket_policy) \n", 135 | " \n", 136 | "response = s3.put_bucket_policy(\n", 137 | " Bucket = s3_bucket,\n", 138 | " Policy = s3_bucket_policy\n", 139 | ")\n", 140 | "\n", 141 | "print (\"Bucket policy set\")" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": null, 147 | "metadata": {}, 148 | "outputs": [], 149 | "source": [ 150 | "#Sample of the data:\n", 151 | "sample = \"\"\"\n", 152 | "PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked\n", 153 | "1,0,3,\"Braund, Mr. Owen Harris\",male,22,1,0,A/5 21171,7.25,,S\n", 154 | "2,1,1,\"Cumings, Mrs. John Bradley (Florence Briggs Thayer)\",female,38,1,0,PC 17599,71.2833,C85,C\n", 155 | "3,1,3,\"Heikkinen, Miss. Laina\",female,26,0,0,STON/O2. 3101282,7.925,,S\n", 156 | "4,1,1,\"Futrelle, Mrs. Jacques Heath (Lily May Peel)\",female,35,1,0,113803,53.1,C123,S\"\"\"\n", 157 | " \n", 158 | "data_schema = \"\"\"{\n", 159 | " \"version\": \"1.0\",\n", 160 | " \"recordAnnotationFieldName\": \"PassengerId\",\n", 161 | " \"targetFieldName\": \"Survived\",\n", 162 | " \"dataFormat\": \"CSV\",\n", 163 | " \"dataFileContainsHeader\": true,\n", 164 | " \"attributes\": [ \n", 165 | " { \"fieldName\": \"PassengerId\", \"fieldType\": \"CATEGORICAL\" },\n", 166 | " { \"fieldName\": \"Survived\", \"fieldType\": \"BINARY\"},\n", 167 | " { \"fieldName\": \"Pclass\", \"fieldType\": \"CATEGORICAL\"},\n", 168 | " { \"fieldName\": \"Name\", \"fieldType\": \"TEXT\"},\n", 169 | " { \"fieldName\": \"Sex\", \"fieldType\": \"CATEGORICAL\" },\n", 170 | " { \"fieldName\": \"Age\", \"fieldType\": \"NUMERIC\"},\n", 171 | " { \"fieldName\": \"SibSp\", \"fieldType\": \"NUMERIC\" },\n", 172 | " { \"fieldName\": \"Parch\", \"fieldType\": \"NUMERIC\" }, \n", 173 | " { \"fieldName\": \"Ticket\", \"fieldType\": \"TEXT\"},\n", 174 | " { \"fieldName\": \"Fare\", \"fieldType\": \"NUMERIC\"},\n", 175 | " { \"fieldName\": \"Cabin\", \"fieldType\": \"CATEGORICAL\"},\n", 176 | " { \"fieldName\": \"Embarked\", \"fieldType\": \"CATEGORICAL\" }\n", 177 | " ]\n", 178 | "}\"\"\"\n", 179 | "\n", 180 | "#We are using the pre-spliting the data into 2 parts, one is used for training the model \n", 181 | "#and the other for the evaluation (test). From more information on splitting data sets, see:\n", 182 | "#http://docs.aws.amazon.com/machine-learning/latest/dg/splitting-types.html\n", 183 | "\n", 184 | "#Training data\n", 185 | "train_data_split = \"\"\"{\"splitting\": {\"percentBegin\": 0, \"percentEnd\": 70}}\"\"\"\n", 186 | "train_data_source_id = resource_pre + 'train-data-titanic-survival'\n", 187 | "response = ml.create_data_source_from_s3(\n", 188 | " DataSourceId = train_data_source_id,\n", 189 | " DataSourceName = train_data_source_id,\n", 190 | " DataSpec={\n", 191 | " 'DataLocationS3': 's3://' + s3_bucket + \"/\" + train_data,\n", 192 | " 'DataSchema': data_schema,\n", 193 | " 'DataRearrangement' : train_data_split\n", 194 | " },\n", 195 | " ComputeStatistics = True\n", 196 | ")\n", 197 | "#Evaluation data\n", 198 | "evaluation_data_split = \"\"\"{\"splitting\": {\"percentBegin\": 71, \"percentEnd\": 100}}\"\"\"\n", 199 | "evaluation_data_source_id = resource_pre + 'evaluate-data-titanic-survival'\n", 200 | "response = ml.create_data_source_from_s3(\n", 201 | " DataSourceId = evaluation_data_source_id,\n", 202 | " DataSourceName = evaluation_data_source_id,\n", 203 | " DataSpec={\n", 204 | " 'DataLocationS3': 's3://' + s3_bucket + \"/\" + train_data,\n", 205 | " 'DataSchema': data_schema,\n", 206 | " 'DataRearrangement' : evaluation_data_split\n", 207 | " },\n", 208 | " ComputeStatistics = True\n", 209 | ")\n", 210 | "\n", 211 | "print (\"Creating datasource and computing stats.. This will take a couple of minutes\")\n", 212 | "sys.stdout.flush()\n", 213 | "waiter = ml.get_waiter('data_source_available')\n", 214 | "waiter.wait(FilterVariable='Name', EQ=train_data_source_id, Limit=1)\n", 215 | "waiter = ml.get_waiter('data_source_available')\n", 216 | "waiter.wait(FilterVariable='Name', EQ=evaluation_data_source_id, Limit=1)\n", 217 | "print (\"Datasource created!\")\n" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": null, 223 | "metadata": {}, 224 | "outputs": [], 225 | "source": [ 226 | "model_id = resource_pre + 'ml-model-titanic-survival'\n", 227 | "\n", 228 | "# Creating the model\n", 229 | "response = ml.create_ml_model(\n", 230 | " MLModelId=model_id,\n", 231 | " MLModelName=model_id,\n", 232 | " MLModelType='BINARY',\n", 233 | " TrainingDataSourceId=train_data_source_id\n", 234 | ")\n", 235 | "print (\"Generating model.. This will take a couple of minutes\")\n", 236 | "sys.stdout.flush()\n", 237 | "waiter = ml.get_waiter('ml_model_available')\n", 238 | "waiter.wait(FilterVariable='Name',EQ=model_id,Limit=1)\n", 239 | "print (\"Model created!\")\n" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": {}, 246 | "outputs": [], 247 | "source": [ 248 | "\n", 249 | "# Creating an evaluation of the ML model\n", 250 | "evaluation_id = resource_pre + 'evaluation-titanic-survival'\n", 251 | "response = ml.create_evaluation(\n", 252 | " EvaluationId = evaluation_id,\n", 253 | " EvaluationName = evaluation_id,\n", 254 | " MLModelId = model_id,\n", 255 | " EvaluationDataSourceId = evaluation_data_source_id\n", 256 | ")\n", 257 | "\n", 258 | "print (\"Evaluating model.. This will take a couple of minutes\")\n", 259 | "sys.stdout.flush()\n", 260 | "waiter = ml.get_waiter('evaluation_available')\n", 261 | "waiter.wait(FilterVariable='Name',EQ=evaluation_id,Limit=1)\n", 262 | "print (\"Evaluation complete!\")\n" 263 | ] 264 | }, 265 | { 266 | "cell_type": "code", 267 | "execution_count": null, 268 | "metadata": {}, 269 | "outputs": [], 270 | "source": [ 271 | "response = ml.describe_evaluations(\n", 272 | " FilterVariable='Name',\n", 273 | " EQ=evaluation_id\n", 274 | ")\n", 275 | "print (\"Out-of-the-box performance (Binary AUC):\")\n", 276 | "print (response['Results'][0]['PerformanceMetrics']['Properties']['BinaryAUC'])\n" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "If you are building a web service that predicts your chances of survival on the titanic, what would you keep a very close eye on? \n", 284 | "(a) true positives \n", 285 | "(b) true negatives \n", 286 | "(c) false positives \n", 287 | "(d) false negatives \n", 288 | "\n", 289 | "Explore the performance of the model and optionally edit score threshold:" 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": {}, 295 | "source": [ 296 | "" 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": null, 302 | "metadata": {}, 303 | "outputs": [], 304 | "source": [ 305 | "# Enabling real time predictions\n", 306 | "for i in range(6):\n", 307 | " response = ml.create_realtime_endpoint(MLModelId=model_id)\n", 308 | " print (response['RealtimeEndpointInfo']['EndpointStatus'])\n", 309 | " if response['RealtimeEndpointInfo']['EndpointStatus'] == 'READY':\n", 310 | " break\n", 311 | " time.sleep(20)\n", 312 | "predict_endpoint = response['RealtimeEndpointInfo']['EndpointUrl']\n", 313 | "time.sleep(20) #Waiting for endpoint to become active\n", 314 | "print (\"Realtime prediction endpoint: \" + predict_endpoint)\n" 315 | ] 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": null, 320 | "metadata": {}, 321 | "outputs": [], 322 | "source": [ 323 | "# Simple predict function\n", 324 | "def predict(passenger):\n", 325 | " response = ml.predict(\n", 326 | " MLModelId=model_id,\n", 327 | " Record=passenger,\n", 328 | " PredictEndpoint=predict_endpoint\n", 329 | " )\n", 330 | " print (\"Passenger ID: \" + passenger['PassengerId'])\n", 331 | " print (\"Predicted Label: \" + str(response['Prediction']['predictedLabel']))\n", 332 | " print (\"Predicted Score: \" + str(response['Prediction']['predictedScores']))\n", 333 | " print (\"Predictive Model Type: \" + str(response['Prediction']['details']['PredictiveModelType']))\n", 334 | " print (\"Algorithm: \" + str(response['Prediction']['details']['Algorithm']))\n", 335 | " print (\"\")\n", 336 | "\n", 337 | "passenger1 = {\"PassengerId\":\"1\", \"Pclass\":\"3\", \"Name\":\"Jack\", \"Sex\":\"male\",\"Age\":\"20\",\"Fare\":\"10\"}\n", 338 | "predict(passenger1)\n", 339 | "\n", 340 | "# Let's change the sex to male, but change the class, name, age and fare and add amount of siblings onboard.\n", 341 | "passenger2 = {\"PassengerId\":\"3\", \"Pclass\":\"1\", \"Name\":\"Dr. Jack\", \"Sex\":\"male\",\"Age\":\"60\",\"Fare\":\"60\",\"SibSp\":\"3\"}\n", 342 | "predict(passenger2)\n", 343 | "#The predicted score should be much higher.\n", 344 | "\n", 345 | "# Let's change the sex to female and class to first class\n", 346 | "passenger3 = {\"PassengerId\":\"2\", \"Pclass\":\"1\", \"Name\":\"Jacklin\", \"Sex\":\"female\",\"Age\":\"30\",\"Fare\":\"10\"}\n", 347 | "predict(passenger3)\n" 348 | ] 349 | }, 350 | { 351 | "cell_type": "markdown", 352 | "metadata": {}, 353 | "source": [ 354 | "Instead of manipulating the values above try out the UI real time predictions:\n", 355 | "\n", 356 | "" 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": null, 362 | "metadata": {}, 363 | "outputs": [], 364 | "source": [ 365 | "######################################################################################################\n", 366 | "### Clean up environment\n", 367 | "######################################################################################################\n", 368 | "\n", 369 | "response = s3.delete_object(Bucket=s3_bucket,Key=train_data)\n", 370 | "response = s3.delete_bucket(Bucket=s3_bucket)\n", 371 | "print (\"Deleted S3 bucket: \" + s3_bucket)\n", 372 | "\n", 373 | "response = ml.delete_realtime_endpoint(MLModelId=model_id)\n", 374 | "response = ml.delete_evaluation(EvaluationId=evaluation_id)\n", 375 | "response = ml.delete_ml_model(MLModelId=model_id)\n", 376 | "print (\"Deleted Realtime prediction endpoint, evaluation and ML model\")\n", 377 | "\n", 378 | "response = ml.delete_data_source(DataSourceId=train_data_source_id)\n", 379 | "response = ml.delete_data_source(DataSourceId=evaluation_data_source_id)\n", 380 | "print (\"Deleted Data sources\")\n" 381 | ] 382 | }, 383 | { 384 | "cell_type": "code", 385 | "execution_count": null, 386 | "metadata": { 387 | "collapsed": true 388 | }, 389 | "outputs": [], 390 | "source": [] 391 | } 392 | ], 393 | "metadata": { 394 | "kernelspec": { 395 | "display_name": "Python 3", 396 | "language": "python", 397 | "name": "python3" 398 | }, 399 | "language_info": { 400 | "codemirror_mode": { 401 | "name": "ipython", 402 | "version": 3 403 | }, 404 | "file_extension": ".py", 405 | "mimetype": "text/x-python", 406 | "name": "python", 407 | "nbconvert_exporter": "python", 408 | "pygments_lexer": "ipython3", 409 | "version": "3.6.0" 410 | } 411 | }, 412 | "nbformat": 4, 413 | "nbformat_minor": 1 414 | } 415 | -------------------------------------------------------------------------------- /Notebooks/Comprehend_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Amazon Comprehend Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "***\n", 15 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 16 | "\n", 17 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 18 | "\n", 19 | "http://aws.amazon.com/apache2.0/\n", 20 | "\n", 21 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 22 | "***\n", 23 | "\n", 24 | "### Prerequisites:\n", 25 | "\n", 26 | "#### Identity and Acces Management\n", 27 | "\n", 28 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you should apply the following managed policy to your user or role:\n", 29 | "\n", 30 | " ComprehendReadOnly\n", 31 | "\n", 32 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope for this workshop." 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "import boto3\n", 42 | "import gzip\n", 43 | "import json\n", 44 | "import csv\n", 45 | "from pprint import pprint\n", 46 | "\n", 47 | "comprehend = boto3.client('comprehend', region_name='eu-west-1')" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "# download review dataset\n", 57 | "\n", 58 | "!curl -O http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/reviews_Amazon_Instant_Video_5.json.gz" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": {}, 65 | "outputs": [], 66 | "source": [ 67 | "filename = 'reviews_Amazon_Instant_Video_5.json.gz'\n", 68 | "f = gzip.open(filename, 'r') \n", 69 | "out = [] \n", 70 | "x = 50 # only process the first 50 entries \n", 71 | "for line in f: \n", 72 | " x -= 1\n", 73 | " if x == 0:\n", 74 | " break\n", 75 | " review = json.loads(line)\n", 76 | " # get sentiment for reviewText\n", 77 | " reviewText = review['reviewText']\n", 78 | " if len(reviewText) > 5000: # only supporting up to 5000 Bytes, skipping entry\n", 79 | " print ('Skipping: %s' % reviewText)\n", 80 | " else:\n", 81 | " textSentiment = comprehend.detect_sentiment(\n", 82 | " Text=reviewText,\n", 83 | " LanguageCode='en'\n", 84 | " )\n", 85 | "\n", 86 | " out.append([review['reviewText'],review['asin'],textSentiment['Sentiment'],textSentiment['SentimentScore']['Positive'],textSentiment['SentimentScore']['Negative'],textSentiment['SentimentScore']['Neutral'],textSentiment['SentimentScore']['Mixed']]) \n", 87 | "\n" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "pprint (out)" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": null, 102 | "metadata": {}, 103 | "outputs": [], 104 | "source": [ 105 | "with open('sentiment.csv', 'w') as csvfile:\n", 106 | " linewriter = csv.writer(csvfile, delimiter=';',quotechar='|', quoting=csv.QUOTE_MINIMAL)\n", 107 | " linewriter.writerow (['review','asin','Sentiment','Positive','Negative','Neutral','Mixed'])\n", 108 | " for all in out:\n", 109 | " linewriter.writerow(all)" 110 | ] 111 | } 112 | ], 113 | "metadata": { 114 | "kernelspec": { 115 | "display_name": "Python 2", 116 | "language": "python", 117 | "name": "python2" 118 | }, 119 | "language_info": { 120 | "codemirror_mode": { 121 | "name": "ipython", 122 | "version": 2 123 | }, 124 | "file_extension": ".py", 125 | "mimetype": "text/x-python", 126 | "name": "python", 127 | "nbconvert_exporter": "python", 128 | "pygments_lexer": "ipython2", 129 | "version": "2.7.15" 130 | } 131 | }, 132 | "nbformat": 4, 133 | "nbformat_minor": 2 134 | } 135 | -------------------------------------------------------------------------------- /Notebooks/FashionMNIST_MXNet_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Building your first Artificial Neural Network with AWS\n", 8 | "\n", 9 | "#### Predicting fashion type using Zalando's Fasion-MNIST dataset (https://github.com/zalandoresearch/fashion-mnist)" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "***\n", 17 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 18 | "\n", 19 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 20 | "\n", 21 | "http://aws.amazon.com/apache2.0/\n", 22 | "\n", 23 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 24 | "***" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "### Load dependencies" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "metadata": { 38 | "collapsed": true 39 | }, 40 | "outputs": [], 41 | "source": [ 42 | "import mxnet as mx\n", 43 | "import mxnet.notebook.callback\n", 44 | "import numpy as np\n", 45 | "import os\n", 46 | "import urllib\n", 47 | "import gzip\n", 48 | "import struct\n", 49 | "import math\n", 50 | "import cv2\n", 51 | "import scipy.misc\n", 52 | "import matplotlib.image as mpimg\n", 53 | "import matplotlib.pyplot as plt" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "### Prepare training and test datasets" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": null, 66 | "metadata": { 67 | "collapsed": true 68 | }, 69 | "outputs": [], 70 | "source": [ 71 | "def download_data(url, force_download=True): \n", 72 | " fname = url.split(\"/\")[-1]\n", 73 | " if force_download or not os.path.exists(fname):\n", 74 | " urllib.urlretrieve(url, fname)\n", 75 | " return fname\n", 76 | "\n", 77 | "def to4d(img):\n", 78 | " return img.reshape(img.shape[0], 1, 28, 28).astype(np.float32)/255\n", 79 | "\n", 80 | "def read_data(label, image):\n", 81 | " base_url = 'https://github.com/zalandoresearch/fashion-mnist/raw/master/data/fashion/'\n", 82 | " with gzip.open(download_data(base_url+label, os.path.join('data',label))) as flbl:\n", 83 | " magic, num = struct.unpack(\">II\", flbl.read(8))\n", 84 | " label = np.fromstring(flbl.read(), dtype=np.int8)\n", 85 | " with gzip.open(download_data(base_url+image, os.path.join('data',image)), 'rb') as fimg:\n", 86 | " magic, num, rows, cols = struct.unpack(\">IIII\", fimg.read(16))\n", 87 | " image = np.fromstring(fimg.read(), dtype=np.uint8).reshape(len(label), rows, cols)\n", 88 | " return (label, image)\n", 89 | "\n", 90 | "batch_size = 100\n", 91 | "(train_lbl, train_img) = read_data('train-labels-idx1-ubyte.gz', 'train-images-idx3-ubyte.gz')\n", 92 | "(val_lbl, val_img) = read_data('t10k-labels-idx1-ubyte.gz', 't10k-images-idx3-ubyte.gz')\n", 93 | "train_data_iter = mx.io.NDArrayIter(data={'fashion_data': to4d(train_img)}, label= {'fashion_item_label': train_lbl}, batch_size=100, shuffle=True)\n", 94 | "test_data_iter = mx.io.NDArrayIter(data={'fashion_data': to4d(val_img)}, label= {'fashion_item_label': val_lbl}, batch_size=100)" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "### Display example training data" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": { 108 | "collapsed": true 109 | }, 110 | "outputs": [], 111 | "source": [ 112 | "for i in range(10):\n", 113 | " plt.subplot(1,10,i+1)\n", 114 | " dsp_img= cv2.bitwise_not(train_img[i])\n", 115 | " plt.imshow(dsp_img, cmap='Greys_r')\n", 116 | " plt.axis('off')\n", 117 | "\n", 118 | "plt.show()\n", 119 | "print('label: %s' % (train_lbl[0:10],))\n" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": null, 125 | "metadata": { 126 | "collapsed": true 127 | }, 128 | "outputs": [], 129 | "source": [ 130 | "# Zalando fashion labels https://github.com/zalandoresearch/fashion-mnist\n", 131 | "\n", 132 | "fashion_labels=['T-shirt/top',\n", 133 | "'Trouser',\n", 134 | "'Pullover',\n", 135 | "'Dress',\n", 136 | "'Coat',\n", 137 | "'Sandal',\n", 138 | "'Shirt',\n", 139 | "'Sneaker',\n", 140 | "'Bag',\n", 141 | "'Ankle boot']" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "### Build MXNet model" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "collapsed": true 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "fashion_item_label = mx.symbol.Variable('fashion_item_label')\n", 160 | "# input\n", 161 | "data = mx.symbol.Variable('fashion_data')\n", 162 | "# Flatten the data from 4-D shape into 2-D (batch_size, num_channel*width*height)\n", 163 | "data = mx.sym.flatten(data=data, name='flatten')\n", 164 | "\n", 165 | "# 1st fully-connected layer + activation function\n", 166 | "fc1 = mx.sym.FullyConnected(data=data, num_hidden=128)\n", 167 | "act1 = mx.sym.Activation(data=fc1, act_type=\"relu\")\n", 168 | "# 2nd fully-connected layer + activation function\n", 169 | "fc2 = mx.sym.FullyConnected(data=act1, num_hidden = 64)\n", 170 | "act2 = mx.sym.Activation(data=fc2, act_type=\"relu\")\n", 171 | "# 3rd fully connected layer (MNIST uses 10 classes)\n", 172 | "fc3 = mx.sym.FullyConnected(data=act2, num_hidden=10)\n", 173 | "# softmax with cross entropy loss\n", 174 | "mlp = mx.sym.SoftmaxOutput(data = fc3, label = fashion_item_label, name='softmax')\n", 175 | "\n", 176 | "mx.viz.plot_network(mlp)" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "### Train the model and commit checkpoints" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": null, 189 | "metadata": { 190 | "collapsed": true 191 | }, 192 | "outputs": [], 193 | "source": [ 194 | "import logging\n", 195 | "\n", 196 | "logging.basicConfig(level=logging.INFO)\n", 197 | "logging.getLogger().setLevel(logging.INFO)\n", 198 | "\n", 199 | "ctx = mx.gpu()\n", 200 | "\n", 201 | "mod = mx.mod.Module(symbol=mlp, data_names=['fashion_data'], label_names=['fashion_item_label'], context=ctx, logger=logging)\n", 202 | "mod.bind(data_shapes=train_data_iter.provide_data, label_shapes=train_data_iter.provide_label)\n", 203 | "mod.init_params(initializer=mx.init.Xavier(magnitude=2.))\n", 204 | " \n", 205 | "mod.fit(train_data_iter, # train data\n", 206 | " eval_data=test_data_iter, # validation data\n", 207 | " optimizer='sgd', # use SGD to train\n", 208 | " optimizer_params={'learning_rate' : 0.1}, # use fixed learning rate\n", 209 | " eval_metric=mx.metric.Accuracy(), # report accuracy during training\n", 210 | " num_epoch=10, # train for at most 10 dataset passes\n", 211 | " epoch_end_callback = mx.callback.do_checkpoint('fashion_mnist')) " 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "### Run predictions for 10 example elements" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": { 225 | "collapsed": true 226 | }, 227 | "outputs": [], 228 | "source": [ 229 | "pred_data_iter = mx.io.NDArrayIter(data={'fashion_data': to4d(val_img)[0:100]}, batch_size=100)\n", 230 | "pred_digits = mod.predict(eval_data=pred_data_iter).asnumpy()\n", 231 | "\n", 232 | "%matplotlib inline\n", 233 | "import matplotlib.pyplot as plt\n", 234 | "for i in range(10):\n", 235 | " plt.subplot(1,10,i+1)\n", 236 | " plt.imshow(val_img[i + 10], cmap='Greys')\n", 237 | " plt.axis('off')\n", 238 | "plt.show()\n", 239 | "\n", 240 | "for x in range(10, 20):\n", 241 | " print(\"Predicted fashion label for image %s is %s \" % (x, np.where(pred_digits[x,0:10] == pred_digits[x,0:10].max())[0]))" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "### Downloading images for prediction from amazon.com" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": null, 254 | "metadata": { 255 | "collapsed": true 256 | }, 257 | "outputs": [], 258 | "source": [ 259 | "!wget -O predict1.jpg https://images-na.ssl-images-amazon.com/images/I/81OaXwn1x4L._UX679_.jpg\n", 260 | "!wget -O predict2.jpg https://images-eu.ssl-images-amazon.com/images/I/31TcgNHsbIL._AC_UL260_SR200,260_.jpg\n", 261 | "!wget -O predict3.jpg https://images-eu.ssl-images-amazon.com/images/I/41hWhZBIc3L._AC_UL260_SR200,260_.jpg\n" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "### Load model from checkpoint for prediction" 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": null, 274 | "metadata": { 275 | "collapsed": true 276 | }, 277 | "outputs": [], 278 | "source": [ 279 | "prediction_model_check_point = 10\n", 280 | "prediction_model_prefix = 'fashion_mnist'\n", 281 | "prediction_sym, arg_params, aux_params = mx.model.load_checkpoint(prediction_model_prefix, prediction_model_check_point)\n", 282 | "prediction_model = mx.mod.Module(symbol=prediction_sym, data_names=['fashion_data'], label_names=['fashion_item_label'])\n", 283 | "prediction_model.bind(for_training=False, data_shapes=[('fashion_data', (1,1,28,28))])\n", 284 | "prediction_model.set_params(arg_params=arg_params, aux_params=aux_params, allow_missing=True)\n", 285 | "\n", 286 | "# define prediction function\n", 287 | "def predict_fashion(img):\n", 288 | " # format data to run prediction\n", 289 | " array = np.full((1, 28, 28), img, dtype=np.float32)\n", 290 | " array.shape\n", 291 | " pred_data_iter = mx.io.NDArrayIter(data={'fashion_data': to4d(array)}, batch_size=1)\n", 292 | " \n", 293 | " pred_digits = prediction_model.predict(eval_data=pred_data_iter).asnumpy()\n", 294 | "\n", 295 | " label = (np.where(pred_digits[0] == pred_digits[0].max())[0])\n", 296 | " \n", 297 | " print(\"Predicted fashion label for image is %s (%s) \" % (label,fashion_labels[label[0]]))\n" 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "### Predict labels for downloaded images" 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": null, 310 | "metadata": { 311 | "collapsed": true 312 | }, 313 | "outputs": [], 314 | "source": [ 315 | "for i in xrange(3):\n", 316 | " img = mpimg.imread('predict'+str(i+1)+'.jpg')\n", 317 | " plt.imshow(img)\n", 318 | " plt.axis('off')\n", 319 | " plt.show() \n", 320 | " # get colours in line with train data\n", 321 | " img = cv2.bitwise_not(img)\n", 322 | " img= np.array (np.mean(img, -1))\n", 323 | "\n", 324 | " # resize image\n", 325 | " img = scipy.misc.imresize(img, (28, 28))\n", 326 | "\n", 327 | "\n", 328 | " predict_fashion(img)" 329 | ] 330 | } 331 | ], 332 | "metadata": { 333 | "kernelspec": { 334 | "display_name": "Python 3", 335 | "language": "python", 336 | "name": "python3" 337 | }, 338 | "language_info": { 339 | "codemirror_mode": { 340 | "name": "ipython", 341 | "version": 3 342 | }, 343 | "file_extension": ".py", 344 | "mimetype": "text/x-python", 345 | "name": "python", 346 | "nbconvert_exporter": "python", 347 | "pygments_lexer": "ipython3", 348 | "version": "3.6.0" 349 | } 350 | }, 351 | "nbformat": 4, 352 | "nbformat_minor": 2 353 | } 354 | -------------------------------------------------------------------------------- /Notebooks/Lex_CreateBot_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Amazon Lex Create CoffeeBot" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "***\n", 15 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 16 | "\n", 17 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 18 | "\n", 19 | "http://aws.amazon.com/apache2.0/\n", 20 | "\n", 21 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 22 | "***\n", 23 | "\n", 24 | "### Prerequisites:\n", 25 | "\n", 26 | "#### Identity and Acces Management\n", 27 | "\n", 28 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you should apply the following managed policy to your user or role:\n", 29 | "\n", 30 | " AmazonLexFullAccess\n", 31 | "\n", 32 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope for this workshop.\n", 33 | "\n", 34 | "#### Coffee Bot\n", 35 | "\n", 36 | "Use the code examples provided in this notebook to create the fictional Coffee Bot as documented in [this Github repository](https://github.com/awslabs/amz-ai-building-better-bots/blob/master/README.md).\n" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import boto3\n", 46 | "import IPython\n", 47 | "import base64\n", 48 | "import time\n", 49 | "from pprint import pprint\n", 50 | "\n", 51 | "aws_region = 'eu-west-1'\n", 52 | "\n", 53 | "bot = boto3.client('lex-models', region_name=aws_region)\n", 54 | "OUTPUT_BLACKLIST = (\"ResponseMetadata\")" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": {}, 61 | "outputs": [], 62 | "source": [ 63 | "slot_types = {\n", 64 | " 'cafeBeverageType':{'values': {'mocha','latte machiato','cappucino','hot chocolate','frappucino'},\n", 65 | " 'resolution': 'TOP_RESOLUTION'},\n", 66 | " 'cafeBeverageSize': {'values': {'small','medium','large'},\n", 67 | " 'resolution': 'TOP_RESOLUTION'},\n", 68 | " 'cafeBeverageStrength':{'values': {'single','double','triple','quadruple'},\n", 69 | " 'resolution': 'TOP_RESOLUTION'},\n", 70 | " 'cafeCreamerType': {'values': {'skim','half and half','almond','whole'},\n", 71 | " 'resolution': 'TOP_RESOLUTION'}}\n", 72 | "\n", 73 | "intent_name = 'cafeOrderBeverage'\n", 74 | "bot_name = 'CoffeeBot'" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "### create custom slot types" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": null, 87 | "metadata": {}, 88 | "outputs": [], 89 | "source": [ 90 | "for type_name in slot_types:\n", 91 | " all_types = bot.get_slot_types(\n", 92 | " nameContains=type_name\n", 93 | " )\n", 94 | " # only proceed if slot type doesn't exist yet \n", 95 | " if len(all_types['slotTypes'])== 0:\n", 96 | "\n", 97 | " enumerations = []\n", 98 | " values = slot_types[type_name]['values']\n", 99 | " for val in values:\n", 100 | " enumerations.append({'value': val})\n", 101 | "\n", 102 | " response = bot.put_slot_type(\n", 103 | " name=type_name,\n", 104 | " enumerationValues=enumerations,\n", 105 | " valueSelectionStrategy=slot_types[type_name]['resolution']\n", 106 | " )\n", 107 | " if response['ResponseMetadata']['HTTPStatusCode'] ==200:\n", 108 | " print (\"%s slot type created, checksum: %s\" %(type_name,response['checksum']))\n", 109 | " else:\n", 110 | " pprint (response)\n", 111 | " else:\n", 112 | " print (\"%s already exists, skipping slot type\" %type_name)" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "### create order beverage intent" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": null, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "response = bot.put_intent(\n", 129 | " name=intent_name,\n", 130 | " description='Handles coffee order requests',\n", 131 | " slots=[\n", 132 | " {\n", 133 | " 'name': 'BeverageType',\n", 134 | " 'slotConstraint': 'Required',\n", 135 | " 'slotType': 'cafeBeverageType',\n", 136 | " 'slotTypeVersion': '$LATEST',\n", 137 | " 'valueElicitationPrompt': {\n", 138 | " 'messages': [\n", 139 | " {\n", 140 | " 'contentType': 'PlainText',\n", 141 | " 'content': 'What type of coffee would you like'\n", 142 | " },\n", 143 | " {\n", 144 | " 'contentType': 'PlainText',\n", 145 | " 'content': 'What type of coffee do you fancy?'\n", 146 | " },\n", 147 | " ],\n", 148 | " 'maxAttempts': 3\n", 149 | " },\n", 150 | " 'priority': 1,\n", 151 | " 'sampleUtterances': [\n", 152 | " 'I would like to have a {BeverageType} please.','A {BeverageType} please.'\n", 153 | " ]\n", 154 | " },\n", 155 | " {\n", 156 | " 'name': 'BeverageSize',\n", 157 | " 'slotConstraint': 'Required',\n", 158 | " 'slotType': 'cafeBeverageSize',\n", 159 | " 'slotTypeVersion': '$LATEST',\n", 160 | " 'valueElicitationPrompt': {\n", 161 | " 'messages': [\n", 162 | " {\n", 163 | " 'contentType': 'PlainText',\n", 164 | " 'content': 'What size would you like, small, medium or large?'\n", 165 | " }\n", 166 | " ],\n", 167 | " 'maxAttempts': 3\n", 168 | " },\n", 169 | " 'priority': 2,\n", 170 | " 'sampleUtterances': [\n", 171 | " 'I have a {BeverageSize} one please.','{BeverageSize} please.'\n", 172 | " ]\n", 173 | " },\n", 174 | " {\n", 175 | " 'name': 'BeverageStrength',\n", 176 | " 'slotConstraint': 'Optional',\n", 177 | " 'slotType': 'cafeBeverageStrength',\n", 178 | " 'slotTypeVersion': '$LATEST',\n", 179 | " 'valueElicitationPrompt': {\n", 180 | " 'messages': [\n", 181 | " {\n", 182 | " 'contentType': 'PlainText',\n", 183 | " 'content': 'How many shots do you want?'\n", 184 | " }\n", 185 | " ],\n", 186 | " 'maxAttempts': 3\n", 187 | " },\n", 188 | " 'priority': 3,\n", 189 | " 'sampleUtterances': [\n", 190 | " '{BeverageStrength} shot please.'\n", 191 | " ]\n", 192 | " },\n", 193 | " {\n", 194 | " 'name': 'Creamer',\n", 195 | " 'slotConstraint': 'Optional',\n", 196 | " 'slotType': 'cafeCreamerType',\n", 197 | " 'slotTypeVersion': '$LATEST',\n", 198 | " 'valueElicitationPrompt': {\n", 199 | " 'messages': [\n", 200 | " {\n", 201 | " 'contentType': 'PlainText',\n", 202 | " 'content': 'What milk do you like?'\n", 203 | " }\n", 204 | " ],\n", 205 | " 'maxAttempts': 3\n", 206 | " },\n", 207 | " 'priority': 4\n", 208 | " },\n", 209 | " ],\n", 210 | " sampleUtterances=[\n", 211 | " 'I would like to order a coffee',\n", 212 | " 'I would like to order a {BeverageType}',\n", 213 | " 'Can I order a {BeverageType} please',\n", 214 | " 'Can I get a {BeverageSize} {Creamer} {BeverageStrength} {BeverageType}',\n", 215 | " 'Can I get a {BeverageType}',\n", 216 | " 'I would like a {BeverageSize} {BeverageType}'\n", 217 | " ],\n", 218 | " followUpPrompt={\n", 219 | " 'prompt': {\n", 220 | " 'messages': [\n", 221 | " {\n", 222 | " 'contentType': 'PlainText',\n", 223 | " 'content': 'Sure. Is there anything else I can do for you?'\n", 224 | " }\n", 225 | " ],\n", 226 | " 'maxAttempts': 2\n", 227 | " },\n", 228 | " 'rejectionStatement': {\n", 229 | " 'messages': [\n", 230 | " {\n", 231 | " 'contentType': 'PlainText',\n", 232 | " 'content': 'Your {BeverageSize} {BeverageType} will be ready for pick-up soon.'\n", 233 | " },\n", 234 | " {\n", 235 | " 'contentType': 'PlainText',\n", 236 | " 'content': 'We will have your {BeverageSize} {BeverageType} ready for pick-up soon.'\n", 237 | " },\n", 238 | " ]\n", 239 | " }\n", 240 | " },\n", 241 | " fulfillmentActivity={\n", 242 | " 'type': 'ReturnIntent'\n", 243 | " }\n", 244 | ")\n", 245 | "\n", 246 | "if response['ResponseMetadata']['HTTPStatusCode'] ==200:\n", 247 | " print ('Intent created, checksum: %s' %response['checksum'])\n", 248 | "else:\n", 249 | " pprint(response)" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "### create coffee bot" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": null, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "response = bot.put_bot(\n", 266 | " name=bot_name,\n", 267 | " description='Coffee order bot for Amazon Lex demo',\n", 268 | " intents=[\n", 269 | " {\n", 270 | " 'intentName': intent_name,\n", 271 | " 'intentVersion': '$LATEST'\n", 272 | " },\n", 273 | " ],\n", 274 | " clarificationPrompt={\n", 275 | " 'messages': [\n", 276 | " {\n", 277 | " 'contentType': 'PlainText',\n", 278 | " 'content': \"Sorry, can you please repeat that?\"\n", 279 | " },\n", 280 | " {\n", 281 | " 'contentType': 'PlainText',\n", 282 | " 'content': \"Sorry, but I didn't understand that. Could you try again, please?\"\n", 283 | " },\n", 284 | " ],\n", 285 | " 'maxAttempts': 2\n", 286 | " },\n", 287 | " abortStatement={\n", 288 | " 'messages': [\n", 289 | " {\n", 290 | " 'contentType': 'PlainText',\n", 291 | " 'content': 'Sorry, I could not understand. Goodbye.'\n", 292 | " },\n", 293 | " ]\n", 294 | " },\n", 295 | " idleSessionTTLInSeconds=300,\n", 296 | " voiceId='Salli',\n", 297 | " processBehavior='BUILD',\n", 298 | " locale='en-US',\n", 299 | " childDirected=False\n", 300 | ")\n", 301 | "\n", 302 | "if response['ResponseMetadata']['HTTPStatusCode'] ==200:\n", 303 | " print ('Bot created, checksum: %s' %response['checksum'])\n", 304 | " \n", 305 | " print ('Console URL: https://eu-west-1.console.aws.amazon.com/lex/home?region=eu-west-1#bot-editor:bot=%s' %bot_name)\n", 306 | "else:\n", 307 | " pprint (response)" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": {}, 313 | "source": [ 314 | "### open coffee bot in the AWS console\n", 315 | "\n", 316 | "Open [Amazon Lex console](https://eu-west-1.console.aws.amazon.com/lex/home?region=eu-west-1#bots:) and wait for the build to complete before testing. " 317 | ] 318 | }, 319 | { 320 | "cell_type": "markdown", 321 | "metadata": {}, 322 | "source": [ 323 | "### backup code for debugging and clean-up" 324 | ] 325 | }, 326 | { 327 | "cell_type": "code", 328 | "execution_count": null, 329 | "metadata": {}, 330 | "outputs": [], 331 | "source": [ 332 | "# get checksums for resource updated\n", 333 | "response = bot.get_intent(\n", 334 | " name=intent_name,\n", 335 | " version='$LATEST')\n", 336 | "pprint ('Intent cheksum: %s' %response['checksum'])\n", 337 | "response = bot.get_bot(\n", 338 | " name=bot_name,\n", 339 | " versionOrAlias='$LATEST')\n", 340 | "pprint ('Bot cheksum: %s' %response['checksum'])" 341 | ] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "execution_count": null, 346 | "metadata": {}, 347 | "outputs": [], 348 | "source": [ 349 | "# delete resources\n", 350 | "\n", 351 | "def delete_bot():\n", 352 | " response = bot.delete_bot(\n", 353 | " name=bot_name)\n", 354 | " if response['ResponseMetadata']['HTTPStatusCode'] ==204:\n", 355 | " print (\"%s deleted\" %bot_name)\n", 356 | " else:\n", 357 | " pprint (response)\n", 358 | "\n", 359 | " time.sleep(3) \n", 360 | "\n", 361 | " response = bot.delete_intent(\n", 362 | " name=intent_name)\n", 363 | " if response['ResponseMetadata']['HTTPStatusCode'] ==204:\n", 364 | " print (\"%s intent deleted\" %intent_name)\n", 365 | " else:\n", 366 | " pprint (response)\n", 367 | "\n", 368 | " for type_name in slot_types:\n", 369 | " time.sleep(3) \n", 370 | " response = bot.delete_slot_type(\n", 371 | " name=type_name\n", 372 | " )\n", 373 | " if response['ResponseMetadata']['HTTPStatusCode'] ==204:\n", 374 | " print (\"%s slot type deleted\" %type_name)\n", 375 | " else:\n", 376 | " pprint (response)\n", 377 | " \n", 378 | "#delete_bot() " 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": null, 384 | "metadata": {}, 385 | "outputs": [], 386 | "source": [] 387 | } 388 | ], 389 | "metadata": { 390 | "kernelspec": { 391 | "display_name": "Python 3", 392 | "language": "python", 393 | "name": "python3" 394 | }, 395 | "language_info": { 396 | "codemirror_mode": { 397 | "name": "ipython", 398 | "version": 3 399 | }, 400 | "file_extension": ".py", 401 | "mimetype": "text/x-python", 402 | "name": "python", 403 | "nbconvert_exporter": "python", 404 | "pygments_lexer": "ipython3", 405 | "version": "3.6.0" 406 | } 407 | }, 408 | "nbformat": 4, 409 | "nbformat_minor": 2 410 | } 411 | -------------------------------------------------------------------------------- /Notebooks/Lex_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Amazon Lex Demo" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "***\n", 15 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 16 | "\n", 17 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 18 | "\n", 19 | "http://aws.amazon.com/apache2.0/\n", 20 | "\n", 21 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 22 | "***\n", 23 | "\n", 24 | "### Prerequisites:\n", 25 | "\n", 26 | "#### Identity and Acces Management\n", 27 | "\n", 28 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you should apply the following managed policy to your user or role:\n", 29 | "\n", 30 | " AmazonLexFullAccess\n", 31 | "\n", 32 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope for this workshop.\n", 33 | "\n", 34 | "#### Coffee Bot\n", 35 | "\n", 36 | "This demo makes use of a fictional Coffee Bot. Instructions for the creation of the bot can be found at https://github.com/awslabs/amz-ai-building-better-bots/blob/master/README.md. There is no need to follow the steps for the integration with Mobile Hub. \n", 37 | "\n", 38 | "Alternatively, use the code examples provided in the [Amazon Lex Create Bot Demo Notebook](./Lex_CreateBot_Demo.ipynb) to create the chat bot skeleton using code.\n" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "import boto3\n", 48 | "import IPython\n", 49 | "import base64\n", 50 | "from pprint import pprint\n", 51 | "\n", 52 | "bot_name = 'CoffeeBot'\n", 53 | "bot_alias = '$LATEST'\n", 54 | "\n", 55 | "lex = boto3.client('lex-runtime', region_name='eu-west-1')\n", 56 | "OUTPUT_BLACKLIST = (\"ResponseMetadata\")\n", 57 | "\n", 58 | "def print_response(response):\n", 59 | " for i in response:\n", 60 | " if i not in OUTPUT_BLACKLIST:\n", 61 | " if i == 'slots':\n", 62 | " print ('slots:')\n", 63 | " for s in response[i]:\n", 64 | " print (\"-\", s,\":\", response[i][s])\n", 65 | " else: \n", 66 | " print (i,\":\", response[i])" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "### User Chris is ordering a coffee" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "file = \"mocha.wav\"\n", 83 | "IPython.display.Audio(file)" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": null, 89 | "metadata": {}, 90 | "outputs": [], 91 | "source": [ 92 | "message = open(file, \"rb\")\n", 93 | "\n", 94 | "response = lex.post_content(\n", 95 | " botName=bot_name,\n", 96 | " botAlias=bot_alias,\n", 97 | " userId='Chris',\n", 98 | " contentType='audio/x-l16',\n", 99 | " accept='audio/*',\n", 100 | " inputStream=message\n", 101 | ")\n", 102 | " \n", 103 | "print_response(response)\n", 104 | " \n", 105 | "outfile = \"lexresponse.mp3\"\n", 106 | "data = response['audioStream'].read()\n", 107 | "\n", 108 | "with open(outfile,'wb') as f:\n", 109 | " f.write(data)\n", 110 | "IPython.display.Audio(outfile) " 111 | ] 112 | }, 113 | { 114 | "cell_type": "markdown", 115 | "metadata": {}, 116 | "source": [ 117 | "### User Bob has some special requirements" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": null, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "file = \"special.wav\"\n", 127 | "IPython.display.Audio(file)" 128 | ] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "execution_count": null, 133 | "metadata": {}, 134 | "outputs": [], 135 | "source": [ 136 | "message = open(file, \"rb\")\n", 137 | "\n", 138 | "response = lex.post_content(\n", 139 | " botName=bot_name,\n", 140 | " botAlias=bot_alias,\n", 141 | " userId='Bob',\n", 142 | " contentType='audio/x-l16',\n", 143 | " accept='audio/*',\n", 144 | " inputStream=message\n", 145 | ")\n", 146 | "\n", 147 | "print_response(response)\n", 148 | " \n", 149 | "outfile = \"lexresponse.mp3\"\n", 150 | "data = response['audioStream'].read()\n", 151 | "\n", 152 | "with open(outfile,'wb') as f:\n", 153 | " f.write(data)\n", 154 | "IPython.display.Audio(outfile) " 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "file = \"no.wav\"\n", 164 | "message = open(file, \"rb\")\n", 165 | "\n", 166 | "response = lex.post_content(\n", 167 | " botName=bot_name,\n", 168 | " botAlias=bot_alias,\n", 169 | " userId='Bob',\n", 170 | " contentType='audio/x-l16',\n", 171 | " accept='audio/*',\n", 172 | " inputStream=message\n", 173 | ")\n", 174 | "\n", 175 | "print_response(response) \n", 176 | " \n", 177 | "outfile = \"lexresponse.mp3\"\n", 178 | "data = response['audioStream'].read()\n", 179 | "\n", 180 | "with open(outfile,'wb') as f:\n", 181 | " f.write(data)\n", 182 | "IPython.display.Audio(outfile) " 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "### Chris has made up his mind on the size coffee he likes" 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": null, 195 | "metadata": {}, 196 | "outputs": [], 197 | "source": [ 198 | "file = \"large.wav\"\n", 199 | "IPython.display.Audio(file)" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "message = open(file, \"rb\")\n", 209 | "\n", 210 | "response = lex.post_content(\n", 211 | " botName=bot_name,\n", 212 | " botAlias=bot_alias,\n", 213 | " userId='Chris',\n", 214 | " contentType='audio/x-l16',\n", 215 | " accept='audio/*',\n", 216 | " inputStream=message\n", 217 | ")\n", 218 | "\n", 219 | "for i in response:\n", 220 | " if i not in OUTPUT_BLACKLIST:\n", 221 | " if i == 'slots':\n", 222 | " print ('slots:')\n", 223 | " for s in response[i]:\n", 224 | " print (\"-\", s,\":\", response[i][s])\n", 225 | " else: \n", 226 | " print (i,\":\", response[i])\n", 227 | " \n", 228 | "outfile = \"lexresponse.mp3\"\n", 229 | "data = response['audioStream'].read() \n", 230 | "\n", 231 | "with open(outfile,'wb') as f:\n", 232 | " f.write(data)\n", 233 | "IPython.display.Audio(outfile) " 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": null, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "file = \"no.wav\"\n", 243 | "message = open(file, \"rb\")\n", 244 | "\n", 245 | "response = lex.post_content(\n", 246 | " botName=bot_name,\n", 247 | " botAlias=bot_alias,\n", 248 | " userId='Chris',\n", 249 | " contentType='audio/x-l16',\n", 250 | " accept='audio/*',\n", 251 | " inputStream=message\n", 252 | ")\n", 253 | "\n", 254 | "print_response(response) \n", 255 | " \n", 256 | "outfile = \"lexresponse.mp3\"\n", 257 | "data = response['audioStream'].read()\n", 258 | "\n", 259 | "with open(outfile,'wb') as f:\n", 260 | " f.write(data)\n", 261 | "IPython.display.Audio(outfile) " 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "execution_count": null, 267 | "metadata": {}, 268 | "outputs": [], 269 | "source": [] 270 | } 271 | ], 272 | "metadata": { 273 | "kernelspec": { 274 | "display_name": "Python 3", 275 | "language": "python", 276 | "name": "python3" 277 | }, 278 | "language_info": { 279 | "codemirror_mode": { 280 | "name": "ipython", 281 | "version": 3 282 | }, 283 | "file_extension": ".py", 284 | "mimetype": "text/x-python", 285 | "name": "python", 286 | "nbconvert_exporter": "python", 287 | "pygments_lexer": "ipython3", 288 | "version": "3.6.0" 289 | } 290 | }, 291 | "nbformat": 4, 292 | "nbformat_minor": 2 293 | } 294 | -------------------------------------------------------------------------------- /Notebooks/PollyPSE.xml: -------------------------------------------------------------------------------- 1 | 2 | 8 | HeHelium̯ 9 | LiLithium 10 | BeBeryllium 11 | NeNeon 12 | NaSodium 13 | MgMagnesium 14 | AlAluminium 15 | SiSilicon 16 | ClChlorine 17 | ArArgon 18 | CaCalcium 19 | ScScandium 20 | TiTitanium 21 | CrChromium 22 | MnManganese 23 | FeIron 24 | CoCobalt 25 | NiNickel 26 | CuCopper 27 | ZnZinc 28 | 29 | -------------------------------------------------------------------------------- /Notebooks/Polly_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Amazon Polly Demo\n", 8 | "\n", 9 | "### Convert text to speech with Amazon Polly" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "***\n", 17 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 18 | "\n", 19 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 20 | "\n", 21 | "http://aws.amazon.com/apache2.0/\n", 22 | "\n", 23 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 24 | "***" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "**Prerequisites:**\n", 32 | "\n", 33 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you need to apply the following minimum managed policies to your user or role:\n", 34 | "\n", 35 | "* AmazonPollyFullAccess \n", 36 | "\n", 37 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope fof this workshop." 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "import boto3\n", 47 | "import IPython\n", 48 | "from pprint import pprint\n", 49 | "\n", 50 | "polly = boto3.client('polly', region_name='eu-west-1')" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "response = polly.synthesize_speech(\n", 60 | " Text=\"It is great to see you today!\",\n", 61 | " TextType=\"text\",\n", 62 | " OutputFormat=\"mp3\", \n", 63 | " VoiceId=\"Emma\")\n", 64 | "\n", 65 | "pprint (response)\n", 66 | " \n", 67 | "outfile = \"pollyresponse.mp3\"\n", 68 | "data = response['AudioStream'].read()\n", 69 | "\n", 70 | "with open(outfile,'wb') as f:\n", 71 | " f.write(data)\n", 72 | "IPython.display.Audio(outfile) " 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": null, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "response = polly.synthesize_speech(\n", 82 | " Text='I am fine, thank you. \\\n", 83 | " What can I do for you?',\n", 84 | " TextType=\"ssml\",\n", 85 | " OutputFormat=\"mp3\", \n", 86 | " VoiceId=\"Emma\")\n", 87 | " \n", 88 | "outfile = \"pollyresponse.mp3\"\n", 89 | "data = response['AudioStream'].read()\n", 90 | "\n", 91 | "with open(outfile,'wb') as f:\n", 92 | " f.write(data)\n", 93 | "IPython.display.Audio(outfile) " 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "# read (r\\id) vs read (r\\Ed)\n", 103 | "response = polly.synthesize_speech(\n", 104 | " Text=\"Iread this book\",\n", 105 | " TextType=\"ssml\",\n", 106 | " OutputFormat=\"mp3\", \n", 107 | " VoiceId=\"Emma\")\n", 108 | " \n", 109 | "outfile = \"pollyresponse.mp3\"\n", 110 | "data = response['AudioStream'].read()\n", 111 | "\n", 112 | "with open(outfile,'wb') as f:\n", 113 | " f.write(data)\n", 114 | "IPython.display.Audio(outfile) " 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": null, 120 | "metadata": {}, 121 | "outputs": [], 122 | "source": [ 123 | "response = polly.synthesize_speech(\n", 124 | " Text='My favorite chemical element is Al, \\\n", 125 | " but Al prefers Mg.',\n", 126 | " TextType=\"ssml\",\n", 127 | " OutputFormat=\"mp3\", \n", 128 | " VoiceId=\"Brian\")\n", 129 | " \n", 130 | "outfile = \"pollyresponse.mp3\"\n", 131 | "data = response['AudioStream'].read()\n", 132 | "\n", 133 | "with open(outfile,'wb') as f:\n", 134 | " f.write(data)\n", 135 | "IPython.display.Audio(outfile) " 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": null, 141 | "metadata": {}, 142 | "outputs": [], 143 | "source": [ 144 | "response = polly.synthesize_speech(\n", 145 | " Text='My favorite chemical element is Mg',\n", 146 | " TextType=\"text\",\n", 147 | " OutputFormat=\"mp3\", \n", 148 | " VoiceId=\"Brian\",\n", 149 | " LexiconNames=[\"PollyPSE\"]\n", 150 | " )\n", 151 | " \n", 152 | "outfile = \"pollyresponse.mp3\"\n", 153 | "data = response['AudioStream'].read()\n", 154 | "\n", 155 | "with open(outfile,'wb') as f:\n", 156 | " f.write(data)\n", 157 | "IPython.display.Audio(outfile) " 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "response = polly.get_lexicon(\n", 167 | " Name=\"PollyPSE\")\n", 168 | "\n", 169 | "xmlret = response['Lexicon']['Content']\n", 170 | " \n", 171 | "print (xmlret)" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "execution_count": null, 177 | "metadata": {}, 178 | "outputs": [], 179 | "source": [ 180 | "response = polly.synthesize_speech(\n", 181 | " Text=\"Beim sächsisch besiegen die weichen die harten.\",\n", 182 | " TextType=\"ssml\",\n", 183 | " OutputFormat=\"mp3\", \n", 184 | " VoiceId=\"Hans\"\n", 185 | " )\n", 186 | " \n", 187 | "outfile = \"pollyresponse.mp3\"\n", 188 | "data = response['AudioStream'].read()\n", 189 | "\n", 190 | "with open(outfile,'wb') as f:\n", 191 | " f.write(data)\n", 192 | "IPython.display.Audio(outfile) " 193 | ] 194 | } 195 | ], 196 | "metadata": { 197 | "kernelspec": { 198 | "display_name": "Python 3", 199 | "language": "python", 200 | "name": "python3" 201 | }, 202 | "language_info": { 203 | "codemirror_mode": { 204 | "name": "ipython", 205 | "version": 3 206 | }, 207 | "file_extension": ".py", 208 | "mimetype": "text/x-python", 209 | "name": "python", 210 | "nbconvert_exporter": "python", 211 | "pygments_lexer": "ipython3", 212 | "version": "3.6.0" 213 | } 214 | }, 215 | "nbformat": 4, 216 | "nbformat_minor": 2 217 | } 218 | -------------------------------------------------------------------------------- /Notebooks/Rekognition_Demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "collapsed": true 7 | }, 8 | "source": [ 9 | "# Amazon Rekognition Demo" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "***\n", 17 | "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 18 | "\n", 19 | "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n", 20 | "\n", 21 | "http://aws.amazon.com/apache2.0/\n", 22 | "\n", 23 | "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", 24 | "***" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "**Prerequisites:**\n", 32 | "\n", 33 | "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you need to apply the following minimum managed policies to your user or role:\n", 34 | "\n", 35 | "* AmazonRekognitionFullAccess\n", 36 | "* AmazonDynamoDBFullAccess\n", 37 | "* AmazonS3FullAccess\n", 38 | "\n", 39 | "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope fof this workshop.\n", 40 | "\n" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 1, 46 | "metadata": {}, 47 | "outputs": [ 48 | { 49 | "name": "stdout", 50 | "output_type": "stream", 51 | "text": [ 52 | "\n", 53 | "An error occurred (ResourceInUseException) when calling the CreateTable operation: Table already exists: rekognition_actors\n", 54 | "mkdir: tmp: File exists\n" 55 | ] 56 | } 57 | ], 58 | "source": [ 59 | "# initialise Notebook\n", 60 | "import boto3\n", 61 | "from IPython.display import HTML, display\n", 62 | "from PIL import Image, ImageDraw, ImageFont\n", 63 | "from io import BytesIO\n", 64 | "import time\n", 65 | "\n", 66 | "# key variable initiation\n", 67 | "rekognition = boto3.client('rekognition', region_name='eu-west-1')\n", 68 | "dynamodb = boto3.resource('dynamodb', region_name='eu-west-1')\n", 69 | "s3 = boto3.client('s3')\n", 70 | "\n", 71 | "# blue, green, red, grey\n", 72 | "colors = ((220,220,220),(242,168,73),(76,182,252),(52,194,123))\n", 73 | "\n", 74 | "# helper function to lookup names in dynamoDB\n", 75 | "actors = dynamodb.Table('rekognition_actors')\n", 76 | "def get_actor_name(actorId):\n", 77 | " response = actors.get_item(\n", 78 | " Key={\n", 79 | " 'actorId': actorId\n", 80 | " },\n", 81 | " AttributesToGet=['actorFullName'] \n", 82 | " )\n", 83 | " \n", 84 | " if 'Item' in response:\n", 85 | " return (response['Item']['actorFullName'])\n", 86 | " else:\n", 87 | " return ('no match found in actors lookup')\n", 88 | " \n", 89 | "# create Amazon DynamoDB table \n", 90 | "!aws dynamodb create-table --table-name rekognition_actors \\\n", 91 | "--attribute-definitions AttributeName=actorId,AttributeType=S \\\n", 92 | "--key-schema AttributeName=actorId,KeyType=HASH \\\n", 93 | "--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 \\\n", 94 | "--region eu-west-1 \n", 95 | "\n", 96 | "# create temporary directory\n", 97 | "!mkdir tmp" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "## Object and scene detection\n", 105 | "\n", 106 | "This features allows you to detect thousands of objects, scenes and concepts in your images and provide labels describing the objects detected.\n", 107 | "\n", 108 | "This example presents an image to the object and scene detection endpoint to determine the lables for the image. " 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 3, 114 | "metadata": {}, 115 | "outputs": [ 116 | { 117 | "data": { 118 | "text/html": [ 119 | "
Blossom93.37197875976562
Flora93.37197875976562
Flower93.37197875976562
Lily93.37197875976562
Plant93.37197875976562
Pond Lily93.37197875976562
Soil54.79717254638672
Art54.27285385131836
Flower Arrangement54.27285385131836
Ikebana54.27285385131836
Jar54.27285385131836
Ornament54.27285385131836
Pottery54.27285385131836
Vase54.27285385131836
Crocus52.84606170654297
Anthurium50.84963607788086
Outdoors50.73622512817383
" 120 | ], 121 | "text/plain": [ 122 | "" 123 | ] 124 | }, 125 | "metadata": {}, 126 | "output_type": "display_data" 127 | } 128 | ], 129 | "source": [ 130 | "bucket = 'mlrekognitiondemo'\n", 131 | "key = 'images/P1010138.JPG'\n", 132 | "#key = 'images/P4050012.JPG'\n", 133 | "\n", 134 | "# Download image from Amazon S3 for display\n", 135 | "image_location = 'tmp/image'\n", 136 | "s3.download_file(bucket, key, image_location)\n", 137 | "image_bin=Image.open(image_location)\n", 138 | "\n", 139 | "# retrieve labels from Amazon Rekognition service\n", 140 | "ret = rekognition.detect_labels(\n", 141 | " Image={\n", 142 | " \"S3Object\": {\n", 143 | " \"Bucket\": bucket,\n", 144 | " \"Name\" : key,\n", 145 | " }\n", 146 | " },\n", 147 | " #MaxLabels=10,\n", 148 | " # MinConfidence=85\n", 149 | ")\n", 150 | "\n", 151 | "inner = \"\"\n", 152 | "for i in ret['Labels']:\n", 153 | " inner += \"\"\n", 154 | "display(HTML(\"
\" + i['Name'] + \"\" + str(i['Confidence']) + \"
\\\n", 155 | " \"+inner+\"
\")) " 156 | ] 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "## Facial analysis\n", 163 | "\n", 164 | "Using Amazon DetectFaces API, you can detect faces in an image and key facial characteristics.\n", 165 | "\n", 166 | "As faces are detected on an image you get a number of attributes about the face:\n", 167 | "\n", 168 | "\t- gender\n", 169 | "\t- age\n", 170 | "\t- sentiment of the person\n", 171 | " - coordinates of key features\n", 172 | " - image quality\n", 173 | " - etc.\n" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": null, 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [ 182 | "bucket = 'mlrekognitiondemo'\n", 183 | "key = 'images/Werner_Vogels.jpeg'\n", 184 | "OUTPUT_BLACKLIST = (\"BoundingBox\",\"Landmarks\",\"Pose\",\"Confidence\")\n", 185 | "\n", 186 | "# detect faces using Amazon Rekognition detect faces API\n", 187 | "ret = rekognition.detect_faces(\n", 188 | " Image={\n", 189 | " \"S3Object\": {\n", 190 | " \"Bucket\": bucket,\n", 191 | " \"Name\" : key,\n", 192 | " }\n", 193 | " },\n", 194 | " Attributes=['ALL'],\n", 195 | ")\n", 196 | "\n", 197 | "# Output image and labels\n", 198 | "\n", 199 | "image_location = 'tmp/image'\n", 200 | "s3.download_file(bucket, key, image_location)\n", 201 | "\n", 202 | "inner = \"\"\n", 203 | "for line in ret['FaceDetails']: \n", 204 | " for label in line.keys():\n", 205 | " if not label in OUTPUT_BLACKLIST:\n", 206 | " if not type(line[label]) is list:\n", 207 | " inner += \"\"\n", 208 | " elif type(line[label]) is list: \n", 209 | " inner += \"\"\n", 210 | " for entities in line[label]:\n", 211 | " inner += \"\"\n", 212 | "\n", 213 | "display(HTML(\"
\" + label + \"\" + str(line[label]) + \"
\" + label + \":\" + '' + \"
\" + '' + \"\" + str(entities) + \"
\\\n", 214 | " \"+inner+\"
\")) \n" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": null, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "# Detect multiple faces within images\n", 224 | "\n", 225 | "bucket = 'mlrekognitiondemo'\n", 226 | "key = 'images/faces.png'\n", 227 | "\n", 228 | "# retrieve face boxes from Rekognition service\n", 229 | "\n", 230 | "ret = rekognition.detect_faces(\n", 231 | " Image={\n", 232 | " \"S3Object\": {\n", 233 | " \"Bucket\": bucket,\n", 234 | " \"Name\" : key,\n", 235 | " }\n", 236 | " },\n", 237 | " Attributes=['DEFAULT'],\n", 238 | ")\n", 239 | "\n", 240 | "boxes = []\n", 241 | "\n", 242 | "# Output labels\n", 243 | "\n", 244 | "faces = ret['FaceDetails']\n", 245 | "for face in faces:\n", 246 | " boxes.append (face['BoundingBox'])\n", 247 | "\n", 248 | "# donwload image from S3\n", 249 | "image_location ='tmp/image'\n", 250 | "s3.download_file(bucket, key, image_location)\n", 251 | "\n", 252 | "image_out = Image.open(\"tmp/image\")\n", 253 | "\n", 254 | "# apply face boxes to image \n", 255 | "draw = ImageDraw.Draw(image_out)\n", 256 | "width, height = image_out.size\n", 257 | "col = 0\n", 258 | "line= 10\n", 259 | "inner = \"\"\n", 260 | "for box in boxes:\n", 261 | " inner += \"\"\n", 268 | " \n", 269 | " for l in range(line):\n", 270 | " draw.rectangle((x1-l,y1-l,x2+l,y2+l),outline=colors[col])\n", 271 | " col += 1\n", 272 | "\n", 273 | "image_out.save('tmp/image',format=\"PNG\")\n", 274 | " \n", 275 | "#display(image_out)\n", 276 | "\n", 277 | "display(HTML(\"
Face \" + str(col+1) + \":\"\n", 262 | " x1 = int(box['Left'] * width)\n", 263 | " y1 = int(box['Top'] * height)\n", 264 | " x2 = int(box['Left'] * width + box['Width'] * width)\n", 265 | " y2 = int(box['Top'] * height + box['Height'] * height)\n", 266 | " for label in box.keys():\n", 267 | " inner += \"
\" + label + \"\" + str(box[label]) + \"
\\\n", 278 | " \"+inner+\"
\"))\n" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": { 284 | "collapsed": true 285 | }, 286 | "source": [ 287 | "## Face comparison\n", 288 | "\n", 289 | "Given two images with faces, Rekognition will compare the largest face from the source image and find similarity with faces found in the target image. \n" 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": null, 295 | "metadata": {}, 296 | "outputs": [], 297 | "source": [ 298 | "# retrieve and resize images for display\n", 299 | "\n", 300 | "bucket = 'mlrekognitiondemo'\n", 301 | "key1 = 'images/coty.jpg'\n", 302 | "key2 = 'images/Werner_Vogels.jpeg'\n", 303 | "\n", 304 | "s3.download_file(bucket, key1, 'tmp/image1')\n", 305 | "s3.download_file(bucket, key2, 'tmp/image2')\n", 306 | "\n", 307 | "# Compare faces using Rekognition\n", 308 | "\n", 309 | "ret = rekognition.compare_faces(\n", 310 | " SourceImage={\n", 311 | " \"S3Object\": {\n", 312 | " \"Bucket\": bucket,\n", 313 | " \"Name\" : key2,\n", 314 | " }\n", 315 | " \n", 316 | " },\n", 317 | " TargetImage={\n", 318 | " \"S3Object\": {\n", 319 | " \"Bucket\": bucket,\n", 320 | " \"Name\" : key1,\n", 321 | " } \n", 322 | " }\n", 323 | ")\n", 324 | "\n", 325 | "# And highlight match on the image\n", 326 | "\n", 327 | "image_out = Image.open(\"tmp/image1\")\n", 328 | "draw = ImageDraw.Draw(image_out)\n", 329 | "\n", 330 | "# get image size\n", 331 | "width, height = image_out.size\n", 332 | "\n", 333 | "col = 2\n", 334 | "line= 5\n", 335 | "\n", 336 | "box= ret['FaceMatches'][0]['Face']['BoundingBox']\n", 337 | "x1 = int(box['Left'] * width)\n", 338 | "y1 = int(box['Top'] * height)\n", 339 | "x2 = int(box['Left'] * width + box['Width'] * width)\n", 340 | "y2 = int(box['Top'] * height + box['Height'] * height)\n", 341 | " \n", 342 | "for l in range(line):\n", 343 | " draw.rectangle((x1-l,y1-l,x2+l,y2+l),outline=colors[col])\n", 344 | " \n", 345 | "image_out.save('tmp/image',format=\"PNG\") \n", 346 | "\n", 347 | "inner = \"\"\n", 348 | "inner += \"\"\n", 350 | "for label in box.keys():\n", 351 | " inner += \"\"\n", 352 | "\n", 353 | "display(HTML(\"
Face match:\"\n", 349 | "inner += \"
Similarity: \" + str(ret['FaceMatches'][0]['Similarity']) + \"
\" + label + \"\" + str(box[label]) + \"
\"+inner+\"
\"))\n", 354 | "\n" 355 | ] 356 | }, 357 | { 358 | "cell_type": "markdown", 359 | "metadata": { 360 | "collapsed": true 361 | }, 362 | "source": [ 363 | "## Face recognition\n", 364 | "\n", 365 | "With FaceRekognition, you can search your image collection for similar faces by storing faces, using the IndexFaces API operation. \n", 366 | "You can then use the SearchFaces operation to return high-confidence matches. A face collection is an index of faces that you own and manage.\n", 367 | "\n", 368 | "\n" 369 | ] 370 | }, 371 | { 372 | "cell_type": "code", 373 | "execution_count": null, 374 | "metadata": {}, 375 | "outputs": [], 376 | "source": [ 377 | "%%bash\n", 378 | "aws rekognition create-collection --collection-id actors --region eu-west-1" 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": null, 384 | "metadata": {}, 385 | "outputs": [], 386 | "source": [ 387 | "# load face definitions into index\n", 388 | "\n", 389 | "faces = [\n", 390 | "{\"objectId\":\"images/z5oj3dys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},\n", 391 | "{\"objectId\":\"images/z6od3gys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},\n", 392 | "{\"objectId\":\"images/z7ad3fys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"}, \n", 393 | "{\"objectId\":\"images/z5kj4dss.png\",\"actorFullName\":\"Alexandra Maria Lara\"}, \n", 394 | "{\"objectId\":\"images/z6hj7fss.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"}, \n", 395 | "{\"objectId\":\"images/z6hd7hsl.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"}, \n", 396 | "{\"objectId\":\"images/i3bvrk3c.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"},\n", 397 | "{\"objectId\":\"images/idbvyk3c.png\",\"actorFullName\":\"Matthias Schweighoefer\"},\n", 398 | "{\"objectId\":\"images/i2bvrg7d.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"},\n", 399 | "{\"objectId\":\"images/d5bgrg6d.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"}, \n", 400 | "{\"objectId\":\"images/d2sgrg8a.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"}, \n", 401 | "{\"objectId\":\"images/hq1rszje.jpg\",\"actorFullName\":\"Franz Hagn\"},\n", 402 | "{\"objectId\":\"images/hq2r5yje.png\",\"actorFullName\":\"Franz Hagn\"},\n", 403 | "{\"objectId\":\"images/06e3ykz8.jpg\",\"actorFullName\":\"Katrin Bauerfeind\"}, \n", 404 | "{\"objectId\":\"images/hgx13oet.jpg\",\"actorFullName\":\"Tom Beck\"},\n", 405 | "{\"objectId\":\"images/06e3gkf8.jpg\",\"actorFullName\":\"Tony Garrn\"},\n", 406 | "{\"objectId\":\"images/evd9sm37.png\",\"actorFullName\":\"Karoline Herfurth\"},\n", 407 | "{\"objectId\":\"images/evd2am47.png\",\"actorFullName\":\"Karoline Herfurth\"}, \n", 408 | "{\"objectId\":\"images/eva8sm27.jpg\",\"actorFullName\":\"Karoline Herfurth\"}, \n", 409 | "]\n", 410 | "\n", 411 | "s3_image_bucket = 'mlrekognitiondemo'\n", 412 | "face_table = dynamodb.Table('rekognition_actors')\n", 413 | "\n", 414 | "# function to write reference table in DynamoDB\n", 415 | "def populate_lookup_table(actorFullName,actorId):\n", 416 | " print ('Populating DynamoDB index for ' + actorFullName + ' as ' + actorId)\n", 417 | " response = face_table.put_item(\n", 418 | " Item={\n", 419 | " 'actorId': actorId,\n", 420 | " 'actorFullName': actorFullName,\n", 421 | " }\n", 422 | " )\n", 423 | " return response\n", 424 | " \n", 425 | "def populate_rekognition_collection(objectId):\n", 426 | " ret = rekognition.index_faces(\n", 427 | " CollectionId=rekognition_face_index,\n", 428 | " Image={\n", 429 | " 'S3Object': {\n", 430 | " 'Bucket': s3_image_bucket,\n", 431 | " 'Name': objectId\n", 432 | " }\n", 433 | " }\n", 434 | " )\n", 435 | " if ret['ResponseMetadata']['HTTPStatusCode'] == 200:\n", 436 | " return ret['FaceRecords'][0]['Face']['FaceId']\n", 437 | " else:\n", 438 | " return False\n", 439 | " \n", 440 | "\n", 441 | "# function to populate faces into index\n", 442 | "def populate_index(all_faces,rekognition_face_index):\n", 443 | " for f in all_faces:\n", 444 | " ret = populate_rekognition_collection(f['objectId'])\n", 445 | " if ret == False:\n", 446 | " print ('some error')\n", 447 | " else:\n", 448 | " ret = populate_lookup_table(f['actorFullName'],ret)\n", 449 | " \n", 450 | " \n", 451 | " print ('done')\n", 452 | "\n", 453 | "\n", 454 | "\n", 455 | " \n", 456 | "rekognition_face_index = 'actors' \n", 457 | "populate_index(faces,rekognition_face_index)\n" 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": null, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [ 466 | "\n", 467 | "sourceurl = 'http://www.presseportal.de/pm/8337/3336775'\n", 468 | "source = 'obs/Amazon.de/www.stephan-rabold.com'\n", 469 | "imgurl = 'http://cache.pressmailing.net/thumbnail/story_big/eb0a2593-b7a8-4f3f-8b31-5581004c3426/drehstart-fuer-you-are-wanted-der-ersten-deutschen-amazon-originals-serie-von-und-mit-matthias-schwe'\n", 470 | "\n", 471 | "bucket = 'mlrekognitiondemo'\n", 472 | "key1 = 'images/yawcast'\n", 473 | "\n", 474 | "# open image\n", 475 | "s3.download_file(bucket, key1, 'tmp/image')\n", 476 | "image_bin=Image.open('tmp/image')\n", 477 | "\n", 478 | "# resize image for display\n", 479 | "image_out=Image.open('tmp/image')\n", 480 | "size = 900,600\n", 481 | "image_out.thumbnail(size, Image.ANTIALIAS)\n", 482 | "\n", 483 | "# add resized image to image to stream and file for later processing\n", 484 | "stream = BytesIO()\n", 485 | "image_bin.save(stream,format=\"JPEG\")\n", 486 | "image_binary = stream.getvalue()\n", 487 | "image_bin.save('tmp/image',format=\"JPEG\")\n", 488 | "\n", 489 | "# Detect face boxes using Rekognition\n", 490 | "\n", 491 | "def face_detection(image_input):\n", 492 | " response = rekognition.detect_faces(\n", 493 | " Image={\n", 494 | " 'Bytes':image_input\n", 495 | " } \n", 496 | " )\n", 497 | " all_faces=response['FaceDetails']\n", 498 | " \n", 499 | " # initialise list object \n", 500 | " boxes = []\n", 501 | " \n", 502 | " # populate list for each face\n", 503 | " for face in all_faces:\n", 504 | " boxes.append (face['BoundingBox'])\n", 505 | " return boxes\n", 506 | "\n", 507 | "boxes = face_detection(image_binary)\n", 508 | "\n", 509 | "# And highlight them on the image\n", 510 | "\n", 511 | "image_out = Image.open(\"tmp/image\")\n", 512 | "draw = ImageDraw.Draw(image_out)\n", 513 | "\n", 514 | "# get image size\n", 515 | "width, height = image_out.size\n", 516 | " \n", 517 | "# apply face boxes to image and store coordinates for later processing\n", 518 | "def get_coordinates(face_boxes, image_width, image_height):\n", 519 | " # initialise list object \n", 520 | " coordinates = [] \n", 521 | " for box in face_boxes:\n", 522 | " x1 = int(box['Left'] * image_width)-5\n", 523 | " y1 = int(box['Top'] * image_height)-5\n", 524 | " x2 = int(box['Left'] * image_width + box['Width'] * image_width)+5\n", 525 | " y2 = int(box['Top'] * image_height + box['Height'] * image_height)+5\n", 526 | " if x1 < 0 : x1=0\n", 527 | " if y1 < 0 : y1=0\n", 528 | " if x2 < 0 : x2=image_width\n", 529 | " if y2 < 0 : y2=image_height \n", 530 | " \n", 531 | " coordinates.append((x1,y1,x2,y2))\n", 532 | " \n", 533 | " return coordinates\n", 534 | " \n", 535 | "\n", 536 | "coordinates = get_coordinates(boxes,width,height)\n", 537 | " \n", 538 | "col = 0\n", 539 | "line= 5 \n", 540 | "for xy in coordinates: \n", 541 | " for l in range(line):\n", 542 | " draw.rectangle((xy[0]-l,xy[1]-l,xy[2]+l,xy[3]+l),outline=colors[col])\n", 543 | " col += 1\n", 544 | " \n", 545 | "size = 900,600\n", 546 | "image_out.thumbnail(size, Image.ANTIALIAS) \n", 547 | "\n", 548 | "display(image_out)\n" 549 | ] 550 | }, 551 | { 552 | "cell_type": "code", 553 | "execution_count": null, 554 | "metadata": {}, 555 | "outputs": [], 556 | "source": [ 557 | "# Search faces in index\n", 558 | "\n", 559 | "def get_face_data(image_binary):\n", 560 | " \n", 561 | " try:\n", 562 | " response = rekognition.search_faces_by_image(\n", 563 | " CollectionId='actors',\n", 564 | " Image={\n", 565 | " 'Bytes':image_binary\n", 566 | " } \n", 567 | " )\n", 568 | " \n", 569 | " if len(response['FaceMatches']) > 0:\n", 570 | " for o in response['FaceMatches']:\n", 571 | " print (o['Face']['FaceId'],o['Face']['Confidence'],get_actor_name(o['Face']['FaceId']))\n", 572 | " return response['FaceMatches'][0]['Face']['FaceId'],response['FaceMatches'][0]['Face']['Confidence']\n", 573 | " else:\n", 574 | " return ('no match detected',0)\n", 575 | " except Exception as e:\n", 576 | " #print (e)\n", 577 | " return ('no face detected',0)\n", 578 | " \n", 579 | " \n", 580 | "# initialise list object \n", 581 | "ret = [] \n", 582 | "\n", 583 | "# iterate through list of boxes and detect individual face\n", 584 | "for box in coordinates:\n", 585 | " \n", 586 | " image_crop = image_bin.crop(box)\n", 587 | " display (image_crop)\n", 588 | " \n", 589 | " # add cropped image to temporary stream\n", 590 | " stream2 = BytesIO() \n", 591 | " image_crop.save(stream2,format=\"JPEG\")\n", 592 | " image_region_binary = stream2.getvalue() \n", 593 | " stream2.close()\n", 594 | " \n", 595 | " try:\n", 596 | " # get external ID name of cropped image\n", 597 | " retdata=get_face_data(image_region_binary),box\n", 598 | " ret.append (retdata) \n", 599 | " except Exception as e:\n", 600 | " print (e, box)\n", 601 | " ret.append ((('no face',0),box))\n", 602 | " pass\n" 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": null, 608 | "metadata": {}, 609 | "outputs": [], 610 | "source": [ 611 | "# Output matches on image\n", 612 | "\n", 613 | "font = ImageFont.load_default()\n", 614 | "img_out = Image.open(\"tmp/image\")\n", 615 | "draw = ImageDraw.Draw(img_out)\n", 616 | "\n", 617 | "font = ImageFont.truetype(\"/Library/Fonts/Arial.ttf\", 22)\n", 618 | "\n", 619 | "col = 0\n", 620 | "xpos = 0\n", 621 | "ypos = 30\n", 622 | "line = 5\n", 623 | "for f in ret:\n", 624 | " \n", 625 | " actor_name=get_actor_name(f[0][0]) \n", 626 | " xpos = len(actor_name)*2.5\n", 627 | " draw.text((f[1][0]-xpos,f[1][1]-ypos),actor_name,colors[col],font=font)\n", 628 | " for l in range(line):\n", 629 | " draw.rectangle((f[1][0]-l,f[1][1]-l,f[1][2]+l,f[1][3]+l),outline=colors[col])\n", 630 | " col += 1\n", 631 | " \n", 632 | "display(img_out)" 633 | ] 634 | }, 635 | { 636 | "cell_type": "markdown", 637 | "metadata": {}, 638 | "source": [ 639 | "## Celebrity detection\n", 640 | "\n", 641 | "Amazon Rekognition can recognize thousands of celebrities in a wide range of categories, such as entertainment and media, sports, business, and politics." 642 | ] 643 | }, 644 | { 645 | "cell_type": "code", 646 | "execution_count": null, 647 | "metadata": {}, 648 | "outputs": [], 649 | "source": [ 650 | "%%bash\n", 651 | "\n", 652 | "aws rekognition recognize-celebrities \\\n", 653 | "--image '{\"S3Object\":{\"Bucket\":\"mlrekognitiondemo\",\"Name\":\"images/yawcast\"}}' --region eu-west-1 \\\n", 654 | "--query 'CelebrityFaces[*].{Name:Name,Confidence:MatchConfidence,Details:Urls,Face:Face.BoundingBox}' " 655 | ] 656 | }, 657 | { 658 | "cell_type": "markdown", 659 | "metadata": {}, 660 | "source": [ 661 | "## Content moderation\n", 662 | "\n", 663 | "Provides image moderation to suggestive or explicit content that may not be appropriate for your audience based on two category and eight child category labels." 664 | ] 665 | }, 666 | { 667 | "cell_type": "code", 668 | "execution_count": null, 669 | "metadata": {}, 670 | "outputs": [], 671 | "source": [ 672 | "bucket = 'mlrekognitiondemo'\n", 673 | "key = 'images/david-full-front.jpg'\n", 674 | "key = 'images/david-pants.jpg'\n", 675 | "\n", 676 | "# retrieve labels from Rekognition service\n", 677 | "ret = rekognition.detect_moderation_labels(\n", 678 | " Image={\n", 679 | " \"S3Object\": {\n", 680 | " \"Bucket\": bucket,\n", 681 | " \"Name\" : key,\n", 682 | " }\n", 683 | " }\n", 684 | ")\n", 685 | "\n", 686 | "# Output image and labels\n", 687 | "image_location = 'tmp/image'\n", 688 | "s3.download_file(bucket, key, image_location)\n", 689 | "\n", 690 | "inner = \"\"\n", 691 | "for i in ret['ModerationLabels']:\n", 692 | " if i['ParentName'] is '': \n", 693 | " inner += \"\"\n", 694 | " else:\n", 695 | " inner += \"\"\n", 696 | "display(HTML(\"
\" + i['Name'] + \" (parent)\" + str(i['Confidence']) + \"
\" + i['Name'] + \"\" + str(i['Confidence']) + \"
\\\n", 697 | " \"+inner+\"
\")) " 698 | ] 699 | }, 700 | { 701 | "cell_type": "markdown", 702 | "metadata": {}, 703 | "source": [ 704 | "## Text detection\n", 705 | "\n", 706 | "Detect text in images and convert it into machine-readable text.\n" 707 | ] 708 | }, 709 | { 710 | "cell_type": "code", 711 | "execution_count": 4, 712 | "metadata": { 713 | "scrolled": false 714 | }, 715 | "outputs": [ 716 | { 717 | "data": { 718 | "text/html": [ 719 | "
TextConfidence %TypeIdParent Id
IT'S97.4773178100586LINE0
MONDAY92.79122924804688LINE1
but keep94.46160125732422LINE2
Smiling99.04523468017578LINE3
IT'S97.4773178100586WORD40
MONDAY92.79122924804688WORD51
but96.57958984375WORD62
keep92.34361267089844WORD72
Smiling99.04523468017578WORD83
" 720 | ], 721 | "text/plain": [ 722 | "" 723 | ] 724 | }, 725 | "metadata": {}, 726 | "output_type": "display_data" 727 | } 728 | ], 729 | "source": [ 730 | "bucket = 'console-sample-images-dub'\n", 731 | "key = 'coffee_monday.jpg'\n", 732 | "#key = 'license_plate.jpg'\n", 733 | "\n", 734 | "# retrieve labels from Rekognition service\n", 735 | "ret = rekognition.detect_text(\n", 736 | " Image={\n", 737 | " \"S3Object\": {\n", 738 | " \"Bucket\": bucket,\n", 739 | " \"Name\" : key,\n", 740 | " }\n", 741 | " }\n", 742 | ")\n", 743 | "\n", 744 | "# Output image and labels\n", 745 | "image_location = 'tmp/image'\n", 746 | "s3.download_file(bucket, key, image_location)\n", 747 | "\n", 748 | "\n", 749 | "inner = \"\"\n", 750 | "for i in ret['TextDetections']:\n", 751 | " if not 'ParentId' in i :\n", 752 | " inner += \"\"\n", 753 | " else:\n", 754 | " inner += \"\"\n", 755 | "display(HTML(\"
TextConfidence %TypeIdParent Id
\" + i['DetectedText'] + \"\" + str(i['Confidence']) + \"\" + str(i['Type']) + \"\" + str(i['Id']) + \"
\" + i['DetectedText'] + \"\" + str(i['Confidence']) + \"\" + str(i['Type']) + \"\" + str(i['Id']) + \"\" + str(i['ParentId']) +\"
\\\n", 756 | " \"+inner+\"
\")) " 757 | ] 758 | }, 759 | { 760 | "cell_type": "markdown", 761 | "metadata": {}, 762 | "source": [ 763 | "https://eu-west-1.console.aws.amazon.com/rekognition/home?region=eu-west-1#/usage-metrics" 764 | ] 765 | }, 766 | { 767 | "cell_type": "markdown", 768 | "metadata": { 769 | "collapsed": true 770 | }, 771 | "source": [ 772 | "all the dependencies that need to be installed:\n", 773 | "\n", 774 | "pip3 install boto3\n", 775 | "\n", 776 | "pip3 install pillow\n" 777 | ] 778 | } 779 | ], 780 | "metadata": { 781 | "kernelspec": { 782 | "display_name": "Python 3", 783 | "language": "python", 784 | "name": "python3" 785 | }, 786 | "language_info": { 787 | "codemirror_mode": { 788 | "name": "ipython", 789 | "version": 3 790 | }, 791 | "file_extension": ".py", 792 | "mimetype": "text/x-python", 793 | "name": "python", 794 | "nbconvert_exporter": "python", 795 | "pygments_lexer": "ipython3", 796 | "version": "3.6.0" 797 | } 798 | }, 799 | "nbformat": 4, 800 | "nbformat_minor": 2 801 | } 802 | -------------------------------------------------------------------------------- /Notebooks/large.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-ai-bootcamp-labs/520ca3ab38a6dc85e856e1c07eb510264161e269/Notebooks/large.wav -------------------------------------------------------------------------------- /Notebooks/mocha.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-ai-bootcamp-labs/520ca3ab38a6dc85e856e1c07eb510264161e269/Notebooks/mocha.wav -------------------------------------------------------------------------------- /Notebooks/no.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-ai-bootcamp-labs/520ca3ab38a6dc85e856e1c07eb510264161e269/Notebooks/no.wav -------------------------------------------------------------------------------- /Notebooks/small.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-ai-bootcamp-labs/520ca3ab38a6dc85e856e1c07eb510264161e269/Notebooks/small.wav -------------------------------------------------------------------------------- /Notebooks/special.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-ai-bootcamp-labs/520ca3ab38a6dc85e856e1c07eb510264161e269/Notebooks/special.wav -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS-AI-Bootcamp-Labs 2 | This library has a collection of Notebooks and code examples for AWS AI Bootcamps. 3 | 4 | ### Content 5 | 6 | [MNIST MXNet Demo Notebook](Notebooks/FashionMNIST_MXNet_Demo.ipynb) 7 | 8 | [Amazon Lex Demo Notebook](Notebooks/Lex_Demo.ipynb) 9 | 10 | [Amazon Polly Demo Notebook](Notebooks/Polly_Demo.ipynb) 11 | 12 | [Amazon Rekognition Demo Notebook](Notebooks/Rekognition_Demo.ipynb) 13 | 14 | [Amazon Comprehend Demo Notebook](Notebooks/Comprehend_Demo.ipynb) 15 | 16 | [Amazon Machine Learning Demo Notebook](Notebooks/AmazonML_Demo.ipynb) 17 | 18 | [Serverless Predictions at Scale](serverless_predictions.MD) 19 | 20 | ### Launch EC2 instance using the deep learning AMI and open fashion MNIST MXNet demo 21 | 22 | 1. Create EC2 IAM role for the workshop as described [here](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#create-iam-role). We will apply permission policies as documented in each notebook 23 | 2. Launch EC2 Instance using the Ubuntu deep learning AMI in eu-west-1, Ireland (p2.xlarge - $0.972/hour) http://amzn.to/2j3FdOZ 24 | 3. Connect via SSH and tunnel port 8888: 25 | * Linux, Mac: 26 | - `ssh -i user.pem -L 8888:localhost:8888 ubuntu@ec2-ip-ip-ip-ip.region.compute.amazonaws.com` 27 | * Windows: 28 | - Follow the instructions [here](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html) to download PuTTY and to convert your private key 29 | - Host Name: `ubuntu@ec2-ip-ip-ip-ip.region.compute.amazonaws.com` 30 | - Expand Connection and choose Auth, select your .ppk file 31 | - Expand Connection > SSH, choose Tunnels, specify Source Port: `8888`, Destination: `localhost:8888` 32 | - Choose Add and Open 33 | 4. Clone aws-ai-bootcamp-labs github repository `git clone https://github.com/awslabs/aws-ai-bootcamp-labs` 34 | 5. Start jupyter notebook: `nohup jupyter notebook &` 35 | 6. `tail nohup.out` to get the login token 36 | * look for `http://localhost:8888/?token=` 37 | 7. Open demo notebook (FashionMNIST_MXNet_Demo.ipynb) 38 | * Select Kernel > Change kernel > Python 2 39 | 8. Follow steps in notebook 40 | -------------------------------------------------------------------------------- /serverless_predictions.MD: -------------------------------------------------------------------------------- 1 | # Serverless Predictions at Scale 2 | 3 | ## Set up the AWS CodeStar project 4 | 5 | 1. Navigate to the AWS CodeStar service console at https://console.aws.amazon.com/codestar/home#/projects 6 | 2. Click the **Create a new project** tile to launch the quickstart wizard. This takes you to the template selection screen in which you select to build a **Python**-based **Web Service** using **AWS Lambda**. 7 | 3. The next screen asks you to provide details for the project being created. In the "Project name" field enter ***mxnet-resnet***. For the purpose of this lab we will rely on AWS CodeCommit to store the project's source code. Note, that the project name used before is also used in the "Repository name" field. Click the **Next** button to continue with the setup. 8 | 4. Take time to review the project details and the main components of the AWS CodePipeline and resources AWS CodeStar provisions on your behalf. Make sure to provide AWS CodeStar with permissions to provision and administer the underlying AWS resources, i.e. keep the checkbox selected before your continue by clicking the **Create Project** button. 9 | 5. As AWS CodeStar provisions your resources in the background follow the instructions to set up your code editor and tools. Select **Command line tools** and click the **Next** button. Choose your operating system, connection method, set up Git, generate the credentials and eventually clone the repository to your development environment. 10 | 11 | ## Adjust CodeStarWorker-mxnet-resnet-CloudFormation IAM role to be able to put a bucket policy on the S3 model store 12 | 13 | 1. Navigate to the Identity and Access Management console at https://console.aws.amazon.com/iam/home#/roles 14 | 2. Search and filter for the role ***CodeStarWorker-mxnet-resnet-CloudFormation*** 15 | 3. Click on the role name ***CodeStarWorker-mxnet-resnet-CloudFormation*** and review the inline policy attached to the role. 16 | 4. Over the course of the next couple of steps we are going to create an S3 bucket for storing model files. In order to govern these resources using CodeStar it is necessary to adjust the default policy. Click **Edit Policy** and add the following block to the “Statement” part inline policy. Make sure to replace the `${AWS::Region}` with the acronym of the region that you are using, e.g. eu-central-1, and the `${AWS::AccountId}` with your AWS Account ID, e.g. `123456789012`. Click **Save** to persist your changes to the inline policy. 17 | 18 | ```json 19 | { 20 | "Action": [ 21 | "s3:PutBucketPolicy" 22 | ], 23 | "Resource": [ 24 | "arn:aws:s3:::mxnet-resnet-${AWS::Region}-${AWS::AccountId}-models" 25 | ], 26 | "Effect": "Allow" 27 | } 28 | ``` 29 | 30 | ## Add S3 bucket to host pre-trained models 31 | 32 | 1. Open the ***template.yml*** file and add a S3 bucket resource. This bucket will later be used to store the pre-trained models. Our Lambda function will later download the models from this defined location to the function execution environment. Commit and push your changes to the CodeCommit repository. After you have pushed your changes CodePipeline picks up your changes and will create an S3 bucket which can be used to host pre-trained models. 33 | 34 | ```yaml 35 | AWSTemplateFormatVersion: 2010-09-09 36 | Transform: 37 | - AWS::Serverless-2016-10-31 38 | - AWS::CodeStar 39 | 40 | Parameters: 41 | ProjectId: 42 | Type: String 43 | Description: CodeStar projectId used to associate new resources to team members 44 | 45 | Resources: 46 | HelloWorld: 47 | Type: AWS::Serverless::Function 48 | Properties: 49 | Handler: index.handler 50 | Runtime: python2.7 51 | Role: 52 | Fn::ImportValue: 53 | !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']] 54 | Events: 55 | GetEvent: 56 | Type: Api 57 | Properties: 58 | Path: / 59 | Method: get 60 | PostEvent: 61 | Type: Api 62 | Properties: 63 | Path: / 64 | Method: post 65 | 66 | ModelStoreBucket: 67 | Type: "AWS::S3::Bucket" 68 | Properties: 69 | BucketName: 70 | !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', !Ref 'AWS::AccountId', 'models']] 71 | ModelStoreBucketPolicy: 72 | Type: "AWS::S3::BucketPolicy" 73 | Properties: 74 | Bucket: 75 | !Ref ModelStoreBucket 76 | PolicyDocument: 77 | Statement: 78 | - Effect: "Allow" 79 | Action: 80 | - "s3:GetObject" 81 | Resource: 82 | Fn::Join: ["", ["arn:aws:s3:::", !Ref ModelStoreBucket, "/*"]] 83 | Principal: 84 | AWS: 85 | - Fn::ImportValue: !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']] 86 | 87 | Outputs: 88 | ModelStoreBucket: 89 | Description: A reference to S3 bucket to use as model store. 90 | Value: !Ref ModelStoreBucket 91 | ``` 92 | 93 | ## Download the pre-trained models and copy them to the S3 model store bucket 94 | 95 | 1. Download the pre-trained model files from http://data.mxnet.io/models site. The ResNet-18 files can be found in the imagenet/resnet/18-layers subfolder. The resnet-18-symbol.json file contains the model definition of the pre-trained model. The resnet-18-0000.params file is a binary containing the parameters. Also, make sure to fetch the text file containing the labels from http://data.mxnet.io/models/imagenet/resnet/synset.txt. 96 | 2. Copy the files to the S3 bucket used as model store. 97 | 98 | The following code outlines CLI commands that can be used to automate the above steps on macOS/Linux. **Note: Make sure to configure the AWS CLI before executing those commands, e.g. set the region to the region where you created the CodeStar project.** 99 | 100 | ```bash 101 | # create temp. directory 102 | mkdir -p /tmp/resnet18 103 | # enter temp. directory 104 | pushd $_ 105 | # download pre-trained model files 106 | wget "http://data.mxnet.io/models/imagenet/resnet/synset.txt" -O resnet-18-synset.txt 107 | wget "http://data.mxnet.io/models/imagenet/resnet/18-layers/resnet-18-0000.params" -O resnet-18-0000.params 108 | wget "http://data.mxnet.io/models/imagenet/resnet/18-layers/resnet-18-symbol.json" -O resnet-18-symbol.json 109 | # assemble name of S3 bucket to use as model store 110 | region=$(aws configure get region) 111 | account=$(aws sts get-caller-identity --query 'Account' --output text) 112 | bucket="mxnet-resnet-${region}-${account}-models" 113 | # sync files to model store bucket 114 | aws s3 sync . s3://${bucket} 115 | # leave temp. directory 116 | popd 117 | ``` 118 | 119 | ## Add the Lambda function code to the repository 120 | 121 | 1. Create a file named ***lambda_function.py*** based on the source code given below. Add the file to the repository, push and commit the changes to the repository. 122 | 123 | ```python 124 | import os 125 | import boto3 126 | import json 127 | import tempfile 128 | import urllib2 129 | import mxnet as mx 130 | import numpy as np 131 | import cv2 132 | 133 | # get model store config from environment 134 | f_bucket = os.environ['MODEL_STORE_BUCKET_NAME'] 135 | f_prefix = os.environ['MODEL_STORE_PREFIX'] 136 | f_params = os.environ['MODEL_PARAMS_FILE_NAME'] 137 | f_symbol = os.environ['MODEL_SYMBOL_FILE_NAME'] 138 | f_synset = os.environ['MODEL_SYNSET_FILE_NAME'] 139 | 140 | prediction_model_check_point = 0 141 | prediction_model_prefix = '/tmp/resnet-18' 142 | 143 | s3 = boto3.resource('s3') 144 | s3_client = boto3.client('s3') 145 | 146 | # load pre-trained model files to temporary files 147 | s3_client.download_file(f_bucket, f_prefix + f_params, '/tmp/' + f_params) 148 | s3_client.download_file(f_bucket, f_prefix + f_symbol, '/tmp/' + f_symbol) 149 | s3_client.download_file(f_bucket, f_prefix + f_synset, '/tmp/' + f_synset) 150 | 151 | with open('/tmp/' + f_synset, 'r') as f: 152 | synsets = [l.rstrip() for l in f] 153 | 154 | def lambda_handler(event, context): 155 | print("Received event.") 156 | if event['httpMethod'] == 'GET': 157 | url = event['queryStringParameters']['url'] 158 | else: 159 | return None 160 | 161 | print("Determined URL = " + url) 162 | prediction_sym, arg_params, aux_params = mx.model.load_checkpoint(prediction_model_prefix, prediction_model_check_point) 163 | print("Loaded model checkpoint from file.") 164 | prediction_model = mx.mod.Module(symbol=prediction_sym, label_names=['softmax_label']) 165 | print("Created model.") 166 | prediction_model.bind(for_training=False, data_shapes=[('data', (1, 3, 224, 224))], label_shapes=[('softmax_label', (1,))]) 167 | print("Bound the symbols to construct executors.") 168 | prediction_model.set_params(arg_params=arg_params, aux_params=aux_params, allow_missing=True) 169 | print("Assigned parameter and aux state values.") 170 | 171 | req = urllib2.urlopen(url) 172 | arr = np.asarray(bytearray(req.read()), dtype=np.uint8) 173 | cv2_img = cv2.imdecode(arr, -1) 174 | img = cv2.cvtColor(cv2_img, cv2.COLOR_BGR2RGB) 175 | if img is None: 176 | return None 177 | 178 | print("Loaded image.") 179 | img = cv2.resize(img, (224, 224)) 180 | img = np.swapaxes(img, 0, 2) 181 | img = np.swapaxes(img, 1, 2) 182 | img = img[np.newaxis, :] 183 | print("Prepared image for model forward.") 184 | print(img.shape) 185 | 186 | pred_data_iter = mx.io.NDArrayIter(data={'data': img}, batch_size=1) 187 | predictions = prediction_model.predict(eval_data=pred_data_iter).asnumpy() 188 | 189 | # sort predictions based on their probability 190 | predictions = np.squeeze(predictions) 191 | predictions_idxs = np.argsort(predictions) 192 | top_predictions_idxs = predictions_idxs[::-1][0:5] 193 | print("Indices of top predictions %s" % top_predictions_idxs) 194 | 195 | out = '{ "predictions": [' 196 | for i in top_predictions_idxs: 197 | out += '{ "probability" : "%f", "class" : "%s" },' % (predictions[i], synsets[i]) 198 | out = out[:-1] 199 | out += "] }" 200 | return out 201 | 202 | def lambda_proxy_handler(event, context): 203 | body = lambda_handler(event, context); 204 | if body is None: 205 | body = {} 206 | 207 | resp = {} 208 | resp['statusCode'] = 200 209 | resp['headers'] = {} 210 | resp['headers']['Content-Type'] = 'application/json' 211 | resp['body'] = body 212 | return resp 213 | ``` 214 | 215 | ## Adjust the build specification to package the function code 216 | 217 | 1. Navigate to the AWS CodeStar service console at https://console.aws.amazon.com/codestar/home#/projects 218 | 2. Locate the ***mxnet-resnet** *project and open the dashboard 219 | 3. From the dashboard click the **Build** on the left navigator and you will be taken to the overview of the ***mxnet-resnet*** build project in the AWS CodeBuild console. Click the **Edit project** button to adjust the project build settings. 220 | 4. On the build settings page update the image to be used by clicking the **Update image** button. Select **Specify a Docker image**, choose **Linux** as the "Environment type", **Other** as the "Custom image type" and ***amazonlinux:latest*** as the "Custom image ID". Click the **Update** button to apply the changes. 221 | 5. Edit the ***buildspec.yml*** file to update the build specification. The specification installs dependencies to correctly package a ZIP containing the dependencies and execution code for our function code. Make sure to push and commit the changes to the repository. 222 | 223 | ```yaml 224 | version: 0.2 225 | 226 | phases: 227 | install: 228 | commands: 229 | # install Python 2.7 230 | - yum install -y python27 231 | - yum install -y python27-virtualenv 232 | # install AWS CLI 233 | - yum install -y aws-cli 234 | # install zip utility 235 | - yum install -y zip 236 | # install libgomp 237 | - yum install -y libgomp 238 | # set up and configure virtual environment 239 | - mkdir -p $HOME/env/mxnet 240 | - cd $HOME/env/mxnet 241 | - virtualenv . 242 | - source bin/activate 243 | # upgrade pip 244 | - pip install --upgrade pip 245 | # install dependencies 246 | - "pip install mxnet==0.11.0 --only-binary=:all:" 247 | - "pip install opencv-python --only-binary=:all:" 248 | build: 249 | commands: 250 | # create distribution directory for Lambda package 251 | - mkdir -p $VIRTUAL_ENV/dist 252 | # copy function code to distribution directory 253 | - cp -rf $CODEBUILD_SRC_DIR/*.py $VIRTUAL_ENV/dist 254 | # copy dependencies to distribution directory 255 | - cp -rf $VIRTUAL_ENV/lib/python2.7/site-packages/* $VIRTUAL_ENV/dist 256 | - cp -rf $VIRTUAL_ENV/lib64/python2.7/site-packages/* $VIRTUAL_ENV/dist 257 | - cp /usr/lib64/libgomp.so.1 $VIRTUAL_ENV/dist 258 | # minimize Lambda package size and zip 259 | - cd $VIRTUAL_ENV/dist 260 | - rm -rf pip* 261 | - rm -rf setuptools* 262 | - zip -r9 $CODEBUILD_SRC_DIR/mxnet-lambda.zip . 263 | - du -sh $CODEBUILD_SRC_DIR/mxnet-lambda.zip 264 | post_build: 265 | commands: 266 | # package local artifcats 267 | - cd $CODEBUILD_SRC_DIR 268 | - aws cloudformation package --template template.yml --s3-bucket $S3_BUCKET --output-template template-export.yml 269 | artifacts: 270 | type: zip 271 | files: 272 | - template-export.yml 273 | ``` 274 | 275 | 276 | 277 | ## Adjust CodeStarWorker-mxnet-resnet-Lambda IAM role to be able to download the pre-trained models from S3 278 | 279 | 1. Navigate to the Identity and Access Management console at https://console.aws.amazon.com/iam/home#/roles 280 | 2. Search and filter for the role ***CodeStarWorker-mxnet-resnet-Lambda*** 281 | 3. Click on the role name ***CodeStarWorker-mxnet-resnet-Lambda*** and review the attached policies 282 | 4. Click **Attach policy**, filter for *AmazonS3ReadOnlyAccess*, select the checkbox and click **Attach Policy** 283 | 284 | ## Adjust SAM template to include API endpoints for prediction 285 | 286 | 1. Open the ***template.yml*** file and add a function definition to serve predictions based on the pre-trained models. 287 | 288 | ```yaml 289 | AWSTemplateFormatVersion: 2010-09-09 290 | Transform: 291 | - AWS::Serverless-2016-10-31 292 | - AWS::CodeStar 293 | 294 | Parameters: 295 | ProjectId: 296 | Type: String 297 | Description: CodeStar projectId used to associate new resources to team members 298 | 299 | Resources: 300 | HelloWorld: 301 | Type: AWS::Serverless::Function 302 | Properties: 303 | Handler: index.handler 304 | Runtime: python2.7 305 | Role: 306 | Fn::ImportValue: 307 | !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']] 308 | Events: 309 | GetEvent: 310 | Type: Api 311 | Properties: 312 | Path: / 313 | Method: get 314 | PostEvent: 315 | Type: Api 316 | Properties: 317 | Path: / 318 | Method: post 319 | Predict: 320 | Type: AWS::Serverless::Function 321 | Properties: 322 | Handler: lambda_function.lambda_proxy_handler 323 | Runtime: python2.7 324 | CodeUri: ./mxnet-lambda.zip 325 | MemorySize: 1024 326 | Timeout: 120 327 | Role: 328 | Fn::ImportValue: 329 | !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']] 330 | Events: 331 | GetEvent: 332 | Type: Api 333 | Properties: 334 | Path: /predict 335 | Method: get 336 | Environment: 337 | Variables: 338 | MODEL_STORE_BUCKET_NAME: !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', !Ref "AWS::AccountId", 'models']] 339 | MODEL_STORE_PREFIX: "" 340 | MODEL_PARAMS_FILE_NAME: "resnet-18-0000.params" 341 | MODEL_SYMBOL_FILE_NAME: "resnet-18-symbol.json" 342 | MODEL_SYNSET_FILE_NAME: "resnet-18-synset.txt" 343 | 344 | ModelStoreBucket: 345 | Type: "AWS::S3::Bucket" 346 | Properties: 347 | BucketName: 348 | !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', !Ref "AWS::AccountId", 'models']] 349 | ModelStoreBucketPolicy: 350 | Type: "AWS::S3::BucketPolicy" 351 | Properties: 352 | Bucket: 353 | !Ref ModelStoreBucket 354 | PolicyDocument: 355 | Statement: 356 | - Effect: "Allow" 357 | Action: 358 | - "s3:GetObject" 359 | Resource: 360 | Fn::Join: ["", ["arn:aws:s3:::", !Ref ModelStoreBucket, "/*"]] 361 | Principal: 362 | AWS: 363 | - Fn::ImportValue: !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']] 364 | 365 | Outputs: 366 | ModelStoreBucket: 367 | Description: A reference to the S3 bucket to use as model store. 368 | Value: !Ref ModelStoreBucket 369 | ``` 370 | 371 | ## Call the endpoint 372 | 373 | 1. Navigate to the AWS CodeStar service console at https://console.aws.amazon.com/codestar/home#/projects 374 | 2. Locate the ***mxnet-resnet*** project and open the dashboard 375 | 3. Identify the **Application endpoints** tile and click the link. 376 | 4. Manipulate the URL from /Prod to /Prod/predict?url=https://images-na.ssl-images-amazon.com/images/G/01/img15/pet-products/small-tiles/23695_pets_vertical_store_dogs_small_tile_8._CB312176604_.jpg 377 | 5. Open the URL to the image which was used for the prediction, e.g. https://images-na.ssl-images-amazon.com/images/G/01/img15/pet-products/small-tiles/23695_pets_vertical_store_dogs_small_tile_8._CB312176604_.jpg 378 | --------------------------------------------------------------------------------