├── Dockerfile ├── README.md ├── guide.md ├── keras_to_tf.py ├── lambda_function.py ├── media ├── image1.png ├── image10.png ├── image11.png ├── image12.png ├── image13.png ├── image14.png ├── image15.png ├── image2.png ├── image3.png ├── image4.png ├── image5.png ├── image6.png ├── image7.png ├── image8.png └── image9.png └── notebook-model.ipynb /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM public.ecr.aws/lambda/python:3.7 2 | 3 | RUN pip3 install --upgrade pip 4 | 5 | RUN pip3 install keras_image_helper --no-cache-dir 6 | RUN pip3 install https://raw.githubusercontent.com/alexeygrigorev/serverless-deep-learning/master/tflite/tflite_runtime-2.2.0-cp37-cp37m-linux_x86_64.whl --no-cache-dir 7 | 8 | COPY clothing-model-v4.tflite clothing-model-v4.tflite 9 | COPY lambda_function.py lambda_function.py 10 | 11 | CMD [ "lambda_function.lambda_handler" ] -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # AWS Lambda + Docker 3 | 4 | The code for the workshop I made on deploying Keras models with AWS Lambda and Docker 5 | 6 | * Detailed guide: [guide.md](guide.md) 7 | 8 | Want to talk about it? Join [DataTalks.Club](https://datatalks.club) 9 | 10 | 11 | ## Running it 12 | 13 | Download the model: 14 | 15 | ```bash 16 | wget https://github.com/alexeygrigorev/mlbookcamp-code/releases/download/chapter7-model/xception_v4_large_08_0.894.h5 17 | ``` 18 | 19 | Convert the model to tf-lite: 20 | 21 | ```bash 22 | python keras_to_tf.py 23 | ``` 24 | 25 | Build the image: 26 | 27 | (Important: don't change the workdir in Dockerfile) 28 | 29 | ``` 30 | docker build -t tf-lite-lambda . 31 | ``` 32 | 33 | Run it: 34 | 35 | ```bash 36 | docker run --rm \ 37 | -p 8080:8080 \ 38 | tf-lite-lambda 39 | ``` 40 | 41 | Test: 42 | 43 | ```bash 44 | URL="http://localhost:8080/2015-03-31/functions/function/invocations" 45 | 46 | REQUEST='{ 47 | "url": "http://bit.ly/mlbookcamp-pants" 48 | }' 49 | 50 | curl -X POST \ 51 | -H "Content-Type: application/json" \ 52 | --data "${REQUEST}" \ 53 | "${URL}" | jq 54 | ``` 55 | 56 | Response: 57 | 58 | 59 | ```json 60 | { 61 | "dress": -1.8682900667190552, 62 | "hat": -4.7612457275390625, 63 | "longsleeve": -2.3169822692871094, 64 | "outwear": -1.062570571899414, 65 | "pants": 9.88715648651123, 66 | "shirt": -2.8124303817749023, 67 | "shoes": -3.66628360748291, 68 | "shorts": 3.2003610134124756, 69 | "skirt": -2.6023387908935547, 70 | "t-shirt": -4.835044860839844 71 | } 72 | ``` 73 | 74 | 75 | Create a registry in ECR: 76 | 77 | ``` 78 | aws ecr create-repository --repository-name lambda-images 79 | ``` 80 | 81 | Push to ECR 82 | 83 | ``` 84 | ACCOUNT=XXXXXXXXXXXX 85 | 86 | docker tag tf-lite-lambda ${ACCOUNT}.dkr.ecr.eu-west-1.amazonaws.com/lambda-images:tf-lite-lambda 87 | 88 | $(aws ecr get-login --no-include-email) 89 | 90 | docker push ${ACCOUNT}.dkr.ecr.eu-west-1.amazonaws.com/lambda-images:tf-lite-lambda 91 | ``` 92 | 93 | Now use the URI of the image to create a Lambda function. Give it enough RAM (1024MB is good). 94 | 95 | -------------------------------------------------------------------------------- /guide.md: -------------------------------------------------------------------------------- 1 | # Deploying models with AWS Lambda + Docker 2 | 3 | We’ll use AWS Lambda and Docker to deploy a Keras model 4 | 5 | - You can find all the code here: 6 | [https://github.com/alexeygrigorev/aws-lambda-docker](https://github.com/alexeygrigorev/aws-lambda-docker) 7 | - This tutorial is based on 8 | [https://github.com/alexeygrigorev/serverless-deep-learning](https://github.com/alexeygrigorev/serverless-deep-learning) 9 | and 10 | [https://github.com/alexeygrigorev/aws-lambda-model-deployment-workshop](https://github.com/alexeygrigorev/aws-lambda-model-deployment-workshop) 11 | - We will deploy a model for predicting the types of clothes 12 | (trained here: 13 | [https://github.com/alexeygrigorev/mlbookcamp-code/blob/master/chapter-07-neural-nets/07-neural-nets-train.ipynb](https://github.com/alexeygrigorev/mlbookcamp-code/blob/master/chapter-07-neural-nets/07-neural-nets-train.ipynb)) 14 | 15 | Plan: 16 | 17 | - Convert the model from Keras to TF Lite 18 | - Prepare the code for lambda 19 | - Package everything into a Docker image 20 | - Push the image to ECR 21 | - Create the lambda function 22 | - Create an API Gateway 23 | 24 | **Prerequisites** 25 | 26 | - You need to have Python 3.7 (or Python 3.8). The easiest way to 27 | install it — use Anaconda 28 | ([https://www.anaconda.com/products/individual](https://www.anaconda.com/products/individual)) 29 | - Install TensorFlow (pip install tensorflow should be sufficient) 30 | - Make sure you have Docker 31 | - You need to have an account in AWS and AWS CLI installed and 32 | configured 33 | 34 | ## Preparing the model 35 | 36 | Suppose we already trained a model using Keras. Now we want to serve it 37 | with AWS Lambda. We need to convert the model to TF-lite format 38 | 39 | Get the model: 40 | 41 | ```bash 42 | wget https://github.com/alexeygrigorev/mlbookcamp-code/releases/download/chapter7-model/xception_v4_large_08_0.894.h5 43 | ``` 44 | 45 | Create a python script “keras_to_tf.py”. Start with the imports: 46 | 47 | ```python 48 | import numpy as np 49 | import tensorflow as tf 50 | from tensorflow import keras 51 | ``` 52 | 53 | Load the model: 54 | 55 | ```python 56 | model = keras.models.load_model('xception_v4_large_08_0.894.h5') 57 | ``` 58 | 59 | Convert it to TF-Lite: 60 | 61 | ```python 62 | converter = tf.lite.TFLiteConverter.from_keras_model(model) 63 | 64 | tflite_model = converter.convert() 65 | 66 | with tf.io.gfile.GFile('clothing-model-v4.tflite', 'wb') as f: 67 | f.write(tflite_model) 68 | ``` 69 | 70 | ## Preprocessing functions 71 | 72 | To apply the model, we need to do the following steps: 73 | 74 | - Get the image (as a PIL Image) 75 | - Prepare the image (resize, etc) 76 | - Convert the image to a tensor, apply the pre-processing function 77 | (normalization, etc) 78 | - Put the tensor in the model, get the predictions and post-process 79 | the predictions 80 | 81 | In Keras, the logic for doing most of these operations is in the 82 | keras-preprocessing module. We can’t use this module inside AWS Lambda 83 | (it’s too heavy). 84 | 85 | There’s a library called [“keras_image_helper”](https://github.com/alexeygrigorev/keras-image-helper) that has this logic 86 | 87 | ```bash 88 | pip install keras_image_helper 89 | ``` 90 | 91 | Let’s use it: 92 | 93 | ```python 94 | from keras_image_helper import create_preprocessor 95 | 96 | preprocessor = create_preprocessor('xception', target_size=(299, 299)) 97 | 98 | image_url = 'http://bit.ly/mlbookcamp-pants' 99 | X = preprocessor.from_url(image_url) 100 | ``` 101 | 102 | Now let’s use this code for getting the predictions\! 103 | 104 | ## Loading the model 105 | 106 | To use the model, we first need to load it with TF lite: 107 | 108 | ```python 109 | # import tensorflow.lite as tflite # if testing locally 110 | import tflite_runtime.interpreter as tflite 111 | 112 | model_local_path = 'clothing-model-v4.tflite' 113 | 114 | interpreter = tflite.Interpreter(model_path=model_local_path) 115 | interpreter.allocate_tensors() 116 | 117 | input_details = interpreter.get_input_details() 118 | input_index = input_details[0]['index'] 119 | 120 | output_details = interpreter.get_output_details() 121 | output_index = output_details[0]['index'] 122 | ``` 123 | 124 | Now we can use it: 125 | 126 | ```python 127 | interpreter.set_tensor(input_index, X) 128 | interpreter.invoke() 129 | 130 | preds = interpreter.get_tensor(output_index) 131 | ``` 132 | 133 | The preds array contains the predictions 134 | 135 | 136 | ## Code for Lambda 137 | 138 | Each lambda function should have an entrypoint. Let’s create it: 139 | 140 | ```python 141 | def lambda_handler(event, context): 142 | url = event['url'] 143 | X = preprocessor.from_url(url) 144 | preds = predict(X) 145 | results = decode_predictions(preds) 146 | return results 147 | ``` 148 | 149 | The `predict` function is just the code from the previous sections put 150 | together 151 | 152 | ```python 153 | def predict(X): 154 | interpreter.set_tensor(input_index, X) 155 | interpreter.invoke() 156 | 157 | preds = interpreter.get_tensor(output_index) 158 | return preds[0] 159 | ``` 160 | 161 | 162 | The `decode_prediction` function turn the raw output into the final 163 | result: 164 | 165 | ```python 166 | labels = [ 167 | 'dress', 168 | 'hat', 169 | 'longsleeve', 170 | 'outwear', 171 | 'pants', 172 | 'shirt', 173 | 'shoes', 174 | 'shorts', 175 | 'skirt', 176 | 't-shirt' 177 | ] 178 | 179 | def decode_predictions(pred): 180 | result = {c: float(p) for c, p in zip(labels, pred)} 181 | return result 182 | ``` 183 | 184 | We put all this code in `lambda_function.py` (see [the full example](lambda_function.py)). 185 | 186 | ## Preparing the Docker image 187 | 188 | Now we need to prepare a Docker image with all the dependencies. Let’s 189 | create it: 190 | 191 | ```docker 192 | FROM public.ecr.aws/lambda/python:3.7 193 | 194 | RUN pip3 install --upgrade pip 195 | 196 | RUN pip3 install keras_image_helper --no-cache-dir 197 | RUN pip3 install https://raw.githubusercontent.com/alexeygrigorev/serverless-deep-learning/master/tflite/tflite_runtime-2.2.0-cp37-cp37m-linux_x86_64.whl --no-cache-dir 198 | 199 | COPY clothing-model-v4.tflite clothing-model-v4.tflite 200 | COPY lambda_function.py lambda_function.py 201 | 202 | CMD [ "lambda_function.lambda_handler" ] 203 | ``` 204 | 205 | 206 | Note that we use a version of TF-Lite compiled for AWS Lambda. (Use 207 | instructions from 208 | [here](https://github.com/alexeygrigorev/serverless-deep-learning) 209 | to compile it yourself for other versions of Python) 210 | 211 | Let’s build this image: 212 | 213 | ```bash 214 | docker build -t tf-lite-lambda . 215 | ``` 216 | 217 | Next, we need to check that the lambda function works. 218 | 219 | Let’s run the image: 220 | 221 | ```bash 222 | docker run --rm \ 223 | -p 8080:8080 \ 224 | tf-lite-lambda 225 | ``` 226 | 227 | And test it with curl: 228 | 229 | ```bash 230 | URL="http://localhost:8080/2015-03-31/functions/function/invocations" 231 | 232 | REQUEST='{ 233 | "url": "http://bit.ly/mlbookcamp-pants" 234 | }' 235 | 236 | curl -X POST \ 237 | -H "Content-Type: application/json" \ 238 | --data "${REQUEST}" \ 239 | "${URL}" | jq 240 | ``` 241 | 242 | We should see the predictions: 243 | 244 | ```json 245 | { 246 | "dress": -1.8682900667190552, 247 | "hat": -4.7612457275390625, 248 | "longsleeve": -2.3169822692871094, 249 | "outwear": -1.062570571899414, 250 | "pants": 9.88715648651123, 251 | "shirt": -2.8124303817749023, 252 | "shoes": -3.66628360748291, 253 | "shorts": 3.2003610134124756, 254 | "skirt": -2.6023387908935547, 255 | "t-shirt": -4.835044860839844 256 | } 257 | ``` 258 | 259 | Now create an ECR: 260 | 261 | ```bash 262 | aws ecr create-repository --repository-name lambda-images 263 | ``` 264 | 265 | And push the image there: 266 | 267 | ```bash 268 | ACCOUNT=XXXXXXXXXXXX 269 | 270 | docker tag tf-lite-lambda ${ACCOUNT}.dkr.ecr.eu-west-1.amazonaws.com/lambda-images:tf-lite-lambda 271 | 272 | $(aws ecr get-login --no-include-email) 273 | 274 | docker push ${ACCOUNT}.dkr.ecr.eu-west-1.amazonaws.com/lambda-images:tf-lite-lambda 275 | ``` 276 | 277 | It’s pushed! Now we can use it to create a lambda 278 | 279 | ## Creating the Lambda function 280 | 281 | Create a lambda function. Go to services, select “Lambda”. Click “Create 282 | function”. Select “Container image”. 283 | 284 | ![](media/image1.png) 285 | 286 | Fill in the details. Container image URI should be the image we created 287 | earlier and pushed to ECR: 288 | “\.dkr.ecr.eu-west-1.amazonaws.com/lambda-images:tf-lite-lambda”. 289 | 290 | You can also use “Browse images” to find it. 291 | 292 | ![](media/image5.png) 293 | 294 | In execution role, choose “Create a new role with basic Lambda 295 | permissions” 296 | 297 | ![](media/image14.png) 298 | 299 | Click “create function”. Now we have a function\! 300 | 301 | ## Using the lambda function 302 | 303 | Go to the Lambda function. Adjust the basic settings. Click edit: 304 | 305 | ![](media/image4.png) 306 | 307 | Give it 512MB or 1024MB of RAM and set timeout to 30 sec: 308 | 309 | ![](media/image12.png) 310 | 311 | Save it. 312 | 313 | Next, create a test with this request: 314 | 315 | ```json 316 | { 317 | "url": "http://bit.ly/mlbookcamp-pants" 318 | } 319 | ``` 320 | 321 | ![](media/image10.png) 322 | 323 | Save and test it: click the “test” button. 324 | 325 | You should see “Execution results: succeeded”: 326 | 327 | ![](media/image3.png) 328 | 329 | To be able to use it from outside, we need to create an API. We do it 330 | with API Gateway. 331 | 332 | ## Creating the API Gateway 333 | 334 | Go to services ⇒ API Gateway 335 | 336 | Create a new HTTP API: 337 | 338 | ![](media/image7.png) 339 | 340 | Call it “image-classification” 341 | 342 | ![](media/image15.png) 343 | 344 | Then, create a resource “predict”, and create a method POST in this 345 | resource: 346 | 347 | ![](media/image13.png) 348 | 349 | Select “Lambda” and enter the details of your lambda function: 350 | 351 | ![](media/image9.png)Make sure you don’t select “proxy integration” — 352 | this box should remain unchecked. 353 | 354 | Now you should see the integration: 355 | 356 | ![](media/image2.png) 357 | 358 | To test it, click on “test” and put this request to request body: 359 | 360 | ```json 361 | { 362 | "url": "http://bit.ly/mlbookcamp-pants" 363 | } 364 | ``` 365 | 366 | You should get the response: 367 | 368 | ![](media/image11.png) 369 | 370 | To use it, we need to deploy the API. Click on “Deploy API” from 371 | Actions. 372 | 373 | ![](media/image8.png) 374 | 375 | Create a new stage “test”: 376 | 377 | ![](media/image6.png) 378 | 379 | And get the url in from the “Invoke URL” field. For us, it’s 380 | "https://l7avw579a1.execute-api.eu-west-1.amazonaws.com/test" 381 | 382 | Now we can test it from the terminal: 383 | 384 | ```bash 385 | URL="https://l7avw579a1.execute-api.eu-west-1.amazonaws.com/test" 386 | 387 | REQUEST='{ 388 | "url": "http://bit.ly/mlbookcamp-pants" 389 | }' 390 | 391 | curl -X POST \ 392 | -H "Content-Type: application/json" \ 393 | --data "${REQUEST}" \ 394 | "${URL}"/predict | jq 395 | ``` 396 | 397 | 398 | The response: 399 | 400 | ```json 401 | { 402 | "dress": -1.8682900667190552, 403 | "hat": -4.7612457275390625, 404 | "longsleeve": -2.3169822692871094, 405 | "outwear": -1.062570571899414, 406 | "pants": 9.88715648651123, 407 | "shirt": -2.8124303817749023, 408 | "shoes": -3.66628360748291, 409 | "shorts": 3.2003610134124756, 410 | "skirt": -2.6023387908935547, 411 | "t-shirt": -4.835044860839844 412 | } 413 | ``` 414 | 415 | Now it’s working! 416 | -------------------------------------------------------------------------------- /keras_to_tf.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from tensorflow import keras 3 | 4 | model = keras.models.load_model('xception_v4_large_08_0.894.h5') 5 | 6 | converter = tf.lite.TFLiteConverter.from_keras_model(model) 7 | 8 | tflite_model = converter.convert() 9 | 10 | with tf.io.gfile.GFile('clothing-model-v4.tflite', 'wb') as f: 11 | f.write(tflite_model) -------------------------------------------------------------------------------- /lambda_function.py: -------------------------------------------------------------------------------- 1 | import tflite_runtime.interpreter as tflite 2 | from keras_image_helper import create_preprocessor 3 | 4 | 5 | preprocessor = create_preprocessor('xception', target_size=(299, 299)) 6 | 7 | 8 | interpreter = tflite.Interpreter(model_path='clothing-model-v4.tflite') 9 | interpreter.allocate_tensors() 10 | 11 | input_details = interpreter.get_input_details() 12 | input_index = input_details[0]['index'] 13 | 14 | output_details = interpreter.get_output_details() 15 | output_index = output_details[0]['index'] 16 | 17 | 18 | def predict(X): 19 | interpreter.set_tensor(input_index, X) 20 | interpreter.invoke() 21 | 22 | preds = interpreter.get_tensor(output_index) 23 | return preds[0] 24 | 25 | 26 | labels = [ 27 | 'dress', 28 | 'hat', 29 | 'longsleeve', 30 | 'outwear', 31 | 'pants', 32 | 'shirt', 33 | 'shoes', 34 | 'shorts', 35 | 'skirt', 36 | 't-shirt' 37 | ] 38 | 39 | def decode_predictions(pred): 40 | result = {c: float(p) for c, p in zip(labels, pred)} 41 | return result 42 | 43 | 44 | def lambda_handler(event, context): 45 | url = event['url'] 46 | X = preprocessor.from_url(url) 47 | preds = predict(X) 48 | results = decode_predictions(preds) 49 | return results -------------------------------------------------------------------------------- /media/image1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image1.png -------------------------------------------------------------------------------- /media/image10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image10.png -------------------------------------------------------------------------------- /media/image11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image11.png -------------------------------------------------------------------------------- /media/image12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image12.png -------------------------------------------------------------------------------- /media/image13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image13.png -------------------------------------------------------------------------------- /media/image14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image14.png -------------------------------------------------------------------------------- /media/image15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image15.png -------------------------------------------------------------------------------- /media/image2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image2.png -------------------------------------------------------------------------------- /media/image3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image3.png -------------------------------------------------------------------------------- /media/image4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image4.png -------------------------------------------------------------------------------- /media/image5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image5.png -------------------------------------------------------------------------------- /media/image6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image6.png -------------------------------------------------------------------------------- /media/image7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image7.png -------------------------------------------------------------------------------- /media/image8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image8.png -------------------------------------------------------------------------------- /media/image9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeygrigorev/aws-lambda-docker/a04951739114a816c0ad1685290ff490acef59dd/media/image9.png --------------------------------------------------------------------------------