├── src └── lambda-poc │ ├── __init__.py │ └── lambda_function.py ├── .tool-versions ├── simple-example ├── requirements.txt ├── app.py ├── lambda-entrypoint.sh └── Dockerfile ├── infra └── tf │ ├── ecr.tf │ ├── .tflint.hcl │ ├── cloudwatch.tf │ ├── outputs.tf │ ├── lambda.tf │ ├── variables.tf │ ├── main.tf │ ├── iam.tf │ └── .terraform.lock.hcl ├── renovate.json ├── lambda ├── install-rie.sh └── lambda-entrypoint.sh ├── .dockerignore ├── LICENSE ├── Dockerfile ├── .gitignore ├── .github └── workflows │ └── ci.yml └── README.md /src/lambda-poc/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.tool-versions: -------------------------------------------------------------------------------- 1 | tflint 0.51.2 2 | -------------------------------------------------------------------------------- /simple-example/requirements.txt: -------------------------------------------------------------------------------- 1 | requests -------------------------------------------------------------------------------- /infra/tf/ecr.tf: -------------------------------------------------------------------------------- 1 | resource "aws_ecr_repository" "repo" { 2 | name = local.ecr_repository_name 3 | } 4 | -------------------------------------------------------------------------------- /renovate.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://docs.renovatebot.com/renovate-schema.json", 3 | "extends": [ 4 | "config:base" 5 | ] 6 | } 7 | -------------------------------------------------------------------------------- /infra/tf/.tflint.hcl: -------------------------------------------------------------------------------- 1 | plugin "aws" { 2 | enabled = true 3 | version = "0.32.0" 4 | source = "github.com/terraform-linters/tflint-ruleset-aws" 5 | } 6 | -------------------------------------------------------------------------------- /infra/tf/cloudwatch.tf: -------------------------------------------------------------------------------- 1 | resource "aws_cloudwatch_log_group" "this" { 2 | name = "/aws/lambda/${var.function_name}" 3 | retention_in_days = var.log_retention_days 4 | } 5 | -------------------------------------------------------------------------------- /infra/tf/outputs.tf: -------------------------------------------------------------------------------- 1 | output "lambda_arn" { 2 | value = aws_lambda_function.this.arn 3 | } 4 | 5 | output "function_url" { 6 | value = aws_lambda_function_url.this.function_url 7 | } 8 | 9 | output "lambda_image_uri" { 10 | value = aws_lambda_function.this.image_uri 11 | } 12 | -------------------------------------------------------------------------------- /simple-example/app.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | 5 | def handler(event, context): 6 | version = os.environ["APP_VERSION"] 7 | return { 8 | "statusCode": 200, 9 | "headers": {"Content-Type": "application/json"}, 10 | "body": json.dumps({"Version ": version}), 11 | } 12 | -------------------------------------------------------------------------------- /src/lambda-poc/lambda_function.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | 5 | def handler(event, context): 6 | version = os.environ["APP_VERSION"] 7 | return { 8 | "statusCode": 200, 9 | "headers": {"Content-Type": "application/json"}, 10 | "body": json.dumps({"Version ": version}), 11 | } 12 | -------------------------------------------------------------------------------- /lambda/install-rie.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | archSuffix="" 6 | if [[ "$1" == "arm64v8" ]] 7 | then 8 | archSuffix="-arm64" 9 | fi 10 | 11 | url="https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie$archSuffix" 12 | curl $url -Lo /usr/local/bin/aws-lambda-rie 13 | chmod +x /usr/local/bin/aws-lambda-rie 14 | -------------------------------------------------------------------------------- /lambda/lambda-entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | if [ $# -ne 1 ]; then 4 | echo "entrypoint requires the handler name to be the first argument" 1>&2 5 | exit 142 6 | fi 7 | 8 | if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then 9 | exec /usr/local/bin/aws-lambda-rie /var/lang/bin/python -m awslambdaric --log-level "debug" "$@" 10 | else 11 | exec /var/lang/bin/python -m awslambdaric "$@" 12 | fi 13 | -------------------------------------------------------------------------------- /simple-example/lambda-entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | if [ $# -ne 1 ]; then 4 | echo "entrypoint requires the handler name to be the first argument" 1>&2 5 | exit 142 6 | fi 7 | 8 | if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then 9 | exec /usr/local/bin/aws-lambda-rie /usr/local/bin/python -m awslambdaric --log-level "debug" "$@" 10 | else 11 | exec /usr/local/bin/python -m awslambdaric "$@" 12 | fi 13 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | .dockerignore 2 | **/.git 3 | **/.gitignore 4 | Dockerfile 5 | docker-compose.yml 6 | node_modules 7 | npm-debug.log* 8 | idea 9 | *.log 10 | coverage 11 | dist 12 | lib 13 | .nyc_output 14 | *.iml 15 | .DS_Store 16 | .build-tasksrc 17 | *.code-workspace 18 | /.vscode 19 | README.md 20 | renovate.json 21 | .terraform 22 | *.tfstate* 23 | .tool-versions 24 | venv 25 | .pytest_cache/ 26 | __pycache__/ 27 | *.py[cod] 28 | *$py.class -------------------------------------------------------------------------------- /infra/tf/lambda.tf: -------------------------------------------------------------------------------- 1 | resource "aws_lambda_function" "this" { 2 | function_name = var.function_name 3 | role = aws_iam_role.lambda.arn 4 | package_type = "Image" 5 | image_uri = "${aws_ecr_repository.repo.repository_url}:${var.image_tag}" 6 | 7 | depends_on = [aws_cloudwatch_log_group.this] 8 | } 9 | 10 | resource "aws_lambda_function_url" "this" { 11 | function_name = aws_lambda_function.this.function_name 12 | authorization_type = "NONE" 13 | } 14 | -------------------------------------------------------------------------------- /infra/tf/variables.tf: -------------------------------------------------------------------------------- 1 | variable "image_tag" { 2 | description = "the image tag for the docker image in ECR" 3 | type = string 4 | } 5 | 6 | variable "function_name" { 7 | description = "the name of the lambda function and its Cloudwatch log group" 8 | type = string 9 | default = "lambda-python-custom" 10 | } 11 | 12 | variable "log_retention_days" { 13 | description = "the number days to keep Cloudwatch logs" 14 | type = number 15 | default = 60 16 | } 17 | -------------------------------------------------------------------------------- /infra/tf/main.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_version = ">= 1.5" 3 | 4 | backend "s3" { 5 | bucket = "krp-project-tfstate" 6 | key = "lambda-python-custom.tfstate" 7 | region = "us-east-2" 8 | } 9 | 10 | required_providers { 11 | aws = { 12 | source = "hashicorp/aws" 13 | version = "5.66.0" 14 | } 15 | } 16 | } 17 | 18 | provider "aws" { 19 | region = "us-east-2" 20 | 21 | default_tags { 22 | tags = { 23 | environment = "dev" 24 | project = "https://github.com/keithly/lambda-python-custom" 25 | } 26 | } 27 | } 28 | 29 | locals { 30 | ecr_repository_name = var.function_name 31 | } 32 | -------------------------------------------------------------------------------- /infra/tf/iam.tf: -------------------------------------------------------------------------------- 1 | resource "aws_iam_role" "lambda" { 2 | name = "lambda-${var.function_name}" 3 | assume_role_policy = jsonencode({ 4 | Version = "2012-10-17" 5 | Statement = [ 6 | { 7 | "Effect" : "Allow", 8 | "Principal" : { 9 | "Service" : "lambda.amazonaws.com" 10 | }, 11 | "Action" : "sts:AssumeRole" 12 | } 13 | ] 14 | }) 15 | } 16 | 17 | resource "aws_iam_role_policy_attachment" "this" { 18 | role = aws_iam_role.lambda.name 19 | policy_arn = aws_iam_policy.lambda.arn 20 | } 21 | 22 | resource "aws_iam_policy" "lambda" { 23 | name = "lambda-${var.function_name}" 24 | 25 | policy = data.aws_iam_policy_document.lambda.json 26 | } 27 | 28 | data "aws_iam_policy_document" "lambda" { 29 | statement { 30 | actions = [ 31 | "logs:CreateLogStream", 32 | "logs:PutLogEvents" 33 | ] 34 | resources = ["${aws_cloudwatch_log_group.this.arn}:*"] 35 | effect = "Allow" 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Keith R. Petersen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /simple-example/Dockerfile: -------------------------------------------------------------------------------- 1 | # based on this example 2 | # https://github.com/aws/aws-lambda-python-runtime-interface-client/blob/970e9c1d2613e0ce9c388547c76ac30992ad0e96/README.md 3 | 4 | FROM public.ecr.aws/docker/library/python:3.12.5-slim-bookworm as build-image 5 | 6 | # Install aws-lambda-cpp build dependencies (for awslambdaric) 7 | RUN apt-get update && \ 8 | apt-get install -y --no-install-recommends \ 9 | g++ \ 10 | make \ 11 | cmake \ 12 | unzip \ 13 | libcurl4-openssl-dev && \ 14 | apt-get clean && \ 15 | rm -rf /var/lib/apt/lists/* 16 | 17 | ADD --chmod=555 https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/local/bin/aws-lambda-rie 18 | 19 | WORKDIR /src 20 | 21 | COPY app.py . 22 | COPY requirements.txt . 23 | RUN python3 -m pip install -U --no-cache-dir pip setuptools wheel && \ 24 | python3 -m pip install --no-cache-dir \ 25 | -r requirements.txt \ 26 | --target /src \ 27 | awslambdaric boto3 28 | 29 | FROM public.ecr.aws/docker/library/python:3.12.5-slim-bookworm 30 | 31 | WORKDIR /src 32 | 33 | COPY ./lambda-entrypoint.sh /lambda-entrypoint.sh 34 | COPY --from=build-image /usr/local/bin/aws-lambda-rie /usr/local/bin/aws-lambda-rie 35 | COPY --from=build-image /src . 36 | 37 | RUN useradd lambdauser 38 | USER lambdauser 39 | 40 | ENV APP_VERSION=1.0.0 41 | 42 | ENTRYPOINT [ "/lambda-entrypoint.sh"] 43 | CMD [ "app.handler" ] 44 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | ARG AL_PROVIDED_VERSION=al2023.2024.08.09.13 2 | ARG ARCH=x86_64 3 | FROM public.ecr.aws/lambda/provided:${AL_PROVIDED_VERSION}-${ARCH} as base 4 | RUN dnf -y update && \ 5 | dnf -y install shadow-utils && \ 6 | dnf clean all 7 | 8 | FROM base as builder 9 | RUN dnf -y update && \ 10 | dnf -y install gcc openssl-devel bzip2-devel libffi-devel xz-devel zlib-devel tar xz && \ 11 | dnf clean all 12 | 13 | ARG PYTHON_VERSION=3.12.5 14 | 15 | RUN cd "$(mktemp -d)" && \ 16 | curl -O https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz && \ 17 | tar xf Python-${PYTHON_VERSION}.tar.xz && \ 18 | cd Python-${PYTHON_VERSION} && \ 19 | ./configure --prefix=/var/lang --enable-optimizations --with-lto=full --with-computed-gotos --enable-loadable-sqlite-extensions && \ 20 | make -j "$(nproc)" && \ 21 | make install 22 | 23 | 24 | ARG ARCH=amd64 25 | FROM base 26 | COPY --from=builder /var/lang /var/lang 27 | RUN ln -s /var/lang/bin/python3 /var/lang/bin/python && \ 28 | ln -s /var/lang/bin/pip3 /var/lang/bin/pip && \ 29 | ln -s /var/lang/bin/pydoc3 /var/lang/bin/pydoc && \ 30 | ln -s /var/lang/bin/python3-config /var/lang/bin/python-config 31 | 32 | WORKDIR /var/task 33 | COPY lambda /var/task 34 | 35 | RUN ./install-rie.sh 36 | 37 | RUN python3 -m pip install -U --no-cache-dir pip setuptools wheel && \ 38 | python3 -m pip install --no-cache-dir --target /var/task awslambdaric boto3 39 | 40 | COPY src src 41 | 42 | RUN /usr/sbin/useradd lambdauser 43 | USER lambdauser 44 | 45 | ENV APP_VERSION=1.0.0 46 | 47 | ENTRYPOINT ["/var/task/lambda-entrypoint.sh", "src/lambda-poc/lambda_function.handler"] 48 | -------------------------------------------------------------------------------- /infra/tf/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "5.66.0" 6 | constraints = "5.66.0" 7 | hashes = [ 8 | "h1:/sZwNLukUujoH9Wo/LnbIcNR53OpQVHspiPmcOagDBk=", 9 | "h1:34+oxdNhh8cJCigeLUXBJZmUnKGkN4wxGO+4hLsrVtQ=", 10 | "h1:4GInuhb6IqucmxJ0wnkU8rn9kZ59usR5KpEhxbDiFHQ=", 11 | "h1:5FKAixQzIkKXqLp97tYU6TqAy43Pt2OSZP8scGnefag=", 12 | "h1:BixtkfzKQPGNw68gxxuRLDnZBluO900yOGv8wm6J4h4=", 13 | "h1:E3IqCLIq+m45oalIE+cJL8nhh6slVAEkTMQam5QC5Vg=", 14 | "h1:OK2O2sH0v0JP3YRNgTSRp3qzwgMiYSyFjRSv+5ddvJ0=", 15 | "h1:RHs4rOiKrKJqr8UhVW7yqfoMVwaofQ+9ChP41rAzc1A=", 16 | "h1:XcP+WoiB+pckH9Cs1AZIhmgF1MmzSLnoNRnOHZCk7sQ=", 17 | "h1:YUORddk17y81eM2IkHvux6UvcF4plznnzY4re6JTPqc=", 18 | "h1:bRu4VJCwrOzn+UWcuJxidyB22JSfphLCUoRBk/4z3bQ=", 19 | "h1:q04VHjxAyH71dKTfMvrUuap88czr8vpiS8MsN7mDn9A=", 20 | "h1:ssqKCgM2aaBGc57A+prQZ4faDHiYy4VWo9Y2rM7UQC4=", 21 | "h1:yGcVdhj9IKbS/b7BSHtgGjCiFnKK+81ImkK/x7UCgEI=", 22 | "zh:071c908eb18627f4becdaf0a9fe95d7a61f69be365080aba2ef5e24f6314392b", 23 | "zh:3dea2a474c6ad4be5b508de4e90064ec485e3fbcebb264cb6c4dec660e3ea8b5", 24 | "zh:56c0b81e3bbf4e9ccb2efb984f8758e2bc563ce179ff3aecc1145df268b046d1", 25 | "zh:5f34b75a9ef69cad8c79115ecc0697427d7f673143b81a28c3cf8d5decfd7f93", 26 | "zh:65632bc2c408775ee44cb32a72e7c48376001a9a7b3adbc2c9b4d088a7d58650", 27 | "zh:6d0550459941dfb39582fadd20bfad8816255a827bfaafb932d51d66030fcdd5", 28 | "zh:7f1811ef179e507fdcc9776eb8dc3d650339f8b84dd084642cf7314c5ca26745", 29 | "zh:8a793d816d7ef57e71758fe95bf830cfca70d121df70778b65cc11065ad004fd", 30 | "zh:8c7cda08adba01b5ae8cc4e5fbf16761451f0fab01327e5f44fc47b7248ba653", 31 | "zh:96d855f1771342771855c0fb2d47ff6a731e8f2fa5d242b18037c751fd63e6c3", 32 | "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", 33 | "zh:b2a62669b72c2471820410b58d764102b11c24e326831ddcfae85c7d20795acf", 34 | "zh:b4a6b251ac24c8f5522581f8d55238d249d0008d36f64475beefc3791f229e1d", 35 | "zh:ca519fa7ee1cac30439c7e2d311a0ecea6a5dae2d175fe8440f30133688b6272", 36 | "zh:fbcd54e7d65806b0038fc8a0fbdc717e1284298ff66e22aac39dcc5a22cc99e5", 37 | ] 38 | } 39 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ### Python template 2 | # Byte-compiled / optimized / DLL files 3 | __pycache__/ 4 | *.py[cod] 5 | *$py.class 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Distribution / packaging 11 | .Python 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | cover/ 54 | 55 | # Translations 56 | *.mo 57 | *.pot 58 | 59 | # Django stuff: 60 | *.log 61 | local_settings.py 62 | db.sqlite3 63 | db.sqlite3-journal 64 | 65 | # Flask stuff: 66 | instance/ 67 | .webassets-cache 68 | 69 | # Scrapy stuff: 70 | .scrapy 71 | 72 | # Sphinx documentation 73 | docs/_build/ 74 | 75 | # PyBuilder 76 | .pybuilder/ 77 | target/ 78 | 79 | # Jupyter Notebook 80 | .ipynb_checkpoints 81 | 82 | # IPython 83 | profile_default/ 84 | ipython_config.py 85 | 86 | # pyenv 87 | # For a library or package, you might want to ignore these files since the code is 88 | # intended to run in multiple environments; otherwise, check them in: 89 | # .python-version 90 | 91 | # pipenv 92 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 93 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 94 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 95 | # install all needed dependencies. 96 | #Pipfile.lock 97 | 98 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 99 | __pypackages__/ 100 | 101 | # Celery stuff 102 | celerybeat-schedule 103 | celerybeat.pid 104 | 105 | # SageMath parsed files 106 | *.sage.py 107 | 108 | # Environments 109 | .envrc 110 | .env 111 | .venv 112 | env/ 113 | venv/ 114 | ENV/ 115 | env.bak/ 116 | venv.bak/ 117 | 118 | # Spyder project settings 119 | .spyderproject 120 | .spyproject 121 | 122 | # Rope project settings 123 | .ropeproject 124 | 125 | # mkdocs documentation 126 | /site 127 | 128 | # mypy 129 | .mypy_cache/ 130 | .dmypy.json 131 | dmypy.json 132 | 133 | # Pyre type checker 134 | .pyre/ 135 | 136 | # pytype static type analyzer 137 | .pytype/ 138 | 139 | # Cython debug symbols 140 | cython_debug/ 141 | 142 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: Deploy to ECR and Lambda 2 | 3 | on: 4 | push: 5 | branches: [ "main" ] 6 | pull_request: 7 | 8 | permissions: 9 | id-token: write 10 | contents: read 11 | 12 | env: 13 | ECR_REPOSITORY: lambda-python-custom 14 | TF_DIR: infra/tf 15 | 16 | jobs: 17 | build-push-deploy: 18 | runs-on: ubuntu-latest 19 | 20 | steps: 21 | - name: Checkout 22 | uses: actions/checkout@v4 23 | 24 | - name: Configure AWS Credentials 25 | uses: aws-actions/configure-aws-credentials@v4 26 | with: 27 | aws-region: ${{ secrets.AWS_REGION }} 28 | role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/GitHubActionsAccess 29 | 30 | - name: Login to Amazon ECR 31 | id: login-ecr 32 | uses: aws-actions/amazon-ecr-login@v2 33 | 34 | - uses: docker/setup-buildx-action@v3 35 | 36 | - name: Docker Build 37 | uses: docker/build-push-action@v6 38 | with: 39 | context: . 40 | load: true 41 | tags: | 42 | ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ github.sha }} 43 | cache-from: type=gha 44 | cache-to: type=gha,mode=max 45 | provenance: false 46 | 47 | - name: Docker Build and Push 48 | uses: docker/build-push-action@v6 49 | if: github.ref == 'refs/heads/main' && github.event_name == 'push' 50 | with: 51 | context: . 52 | push: true 53 | tags: | 54 | ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ github.sha }} 55 | ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:latest 56 | cache-from: type=gha 57 | cache-to: type=gha,mode=max 58 | provenance: false 59 | 60 | - uses: actions/cache@v4 61 | name: Cache TFLint plugin dir 62 | with: 63 | path: ~/.tflint.d/plugins 64 | key: ${{ runner.os }}-tflint-${{ hashFiles('**/.tflint.hcl') }} 65 | 66 | - uses: terraform-linters/setup-tflint@v4 67 | name: Setup TFLint 68 | with: 69 | tflint_version: v0.51.2 70 | 71 | - name: Init TFLint 72 | run: cd ${{ env.TF_DIR }} && tflint --init 73 | 74 | - name: Show TFLint version 75 | run: cd ${{ env.TF_DIR }} && tflint --version 76 | 77 | - name: Run TFLint 78 | run: cd ${{ env.TF_DIR }} && tflint -f compact 79 | 80 | - uses: hashicorp/setup-terraform@v3 81 | with: 82 | terraform_version: 1.8.5 83 | 84 | - name: Config Terraform plugin cache 85 | run: | 86 | echo 'plugin_cache_dir="$HOME/.terraform.d/plugin-cache"' >~/.terraformrc 87 | mkdir --parents ~/.terraform.d/plugin-cache 88 | 89 | - name: Cache Terraform 90 | uses: actions/cache@v4 91 | with: 92 | path: | 93 | ~/.terraform.d/plugin-cache 94 | key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }} 95 | restore-keys: | 96 | ${{ runner.os }}-terraform- 97 | 98 | - name: Check Terraform Format 99 | id: fmt 100 | run: terraform -chdir='${{ env.TF_DIR }}' fmt -recursive -check 101 | 102 | - name: Terraform Init 103 | id: init 104 | run: terraform -chdir='${{ env.TF_DIR }}' init 105 | 106 | - name: Terraform Validate 107 | id: validate 108 | run: terraform -chdir='${{ env.TF_DIR }}' validate -no-color 109 | 110 | - name: Terraform Plan 111 | id: plan 112 | run: terraform -chdir='${{ env.TF_DIR }}' plan -var 'image_tag=${{ github.sha }}' -no-color -out=tfplan 113 | continue-on-error: true 114 | 115 | - name: Terraform Plan Status 116 | if: steps.plan.outcome == 'failure' 117 | run: exit 1 118 | 119 | - name: Terraform Apply 120 | id: apply 121 | if: github.ref == 'refs/heads/main' && github.event_name == 'push' 122 | run: terraform -chdir='${{ env.TF_DIR }}' apply -auto-approve tfplan 123 | continue-on-error: true 124 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # lambda-python-custom 2 | 3 | Use Any Python Version on AWS Lambda 4 | 5 | This project was created when AWS Lambda only supported Python versions 3.7 - 3.9, despite 3.10 and 3.11 having been 6 | released for quite a while. Now AWS is again keeping up with Python versions, but this project shows how to use any 7 | version by creating a custom runtime. The AWS documentation for how do this has improved but is still spread across 8 | several different sites and pages. 9 | 10 | ## Dockerfile 11 | 12 | The main Docker image is now based on the new 13 | [Amazon Linux 2023 Provided image for Lambda](https://gallery.ecr.aws/lambda/provided) (also see 14 | https://aws.amazon.com/blogs/compute/introducing-the-amazon-linux-2023-runtime-for-aws-lambda/). It's built via GitHub 15 | actions and deployed with Terraform. This means a modern version of OpenSSL is available without having to build it from 16 | source. However, the minimal image it's based on made verifying the Python source download more difficult 17 | (see https://github.com/keithly/lambda-python-custom/issues/78). 18 | 19 | The Dockerfile follows all the best practices I'm aware of. :) There are several ARGs for passing specific versions of 20 | the base image and Python, but I didn't attempt to pin every dependency. There's a tradeoff between reproducibility and 21 | convenience. 22 | 23 | - Starts with Amazon Linux 2023, creates a builder stage from it, copies build artifacts back into the base. 24 | - Builds Python from source The python build options optimize the build for speed of execution. The dependencies and 25 | build options could no doubt be tweaked, but this is the simplest solution I found that makes a functional Python 26 | build. 27 | - Links "python3" to "python" 28 | - Installs the latest version of 29 | the [AWS Lambda Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator/) 30 | - Installs the latest versions of pip, setuptools, wheel, 31 | then [awslambdaric](https://github.com/aws/aws-lambda-python-runtime-interface-client) and boto3 32 | - runs as a non-root user (though this may not matter for running on Lambda) 33 | 34 | There's also a [simpler example](simple-example) that starts with the Debian-based official Python image and 35 | therefore doesn't require building it from source. The only dependency truly required is 36 | the [awslambdaric](https://github.com/aws/aws-lambda-python-runtime-interface-client), and everything can be copied into 37 | the same directory in the image. 38 | 39 | ## AWS Lambda Runtime Interface Emulator 40 | 41 | The image contains 42 | the [AWS Lambda Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator/), which can 43 | emulate many features of AWS Lambda. Usage: 44 | 45 | ```bash 46 | docker build -t hello-world . 47 | docker run -it -p 9000:8080 hello-world:latest 48 | ``` 49 | 50 | From another shell: 51 | 52 | ```bash 53 | curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}' 54 | ``` 55 | 56 | ## Lambda Function Code 57 | 58 | [src/lambda-poc](src/lambda-poc) contains a basic function that returns HTTP 200 and some json. 59 | 60 | ## AWS Infrastructure 61 | 62 | [infra/tf](infra/tf) contains terraform code that creates an ECR repository, a lambda function that depends on it, and a 63 | simple [lambda function URL](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html). Terraform creates and 64 | manages the Cloudwatch log group that lambda would otherwise go cowboy with and create on its own, and the IAM policy is 65 | scoped accordingly. 66 | 67 | Note that when creating the infrastructure for the first time, there's a chicken-and-egg problem with the ECR Repository 68 | needing to exist first. The easiest way to handle this is to create it manually, then import to terraform: 69 | 70 | `terraform import aws_ecr_repository.repo ` 71 | 72 | ## GitHub Actions Workflow 73 | 74 | [.github/workflows/ci.yml](.github/workflows/ci.yml) 75 | 76 | - Builds and deploys the docker image to ECR 77 | - Images are tagged with the git SHA 78 | - Uses GitHub GHA caching 79 | - Testing the container before deploy is TBD 80 | - Lints, caches, and runs terraform plan and apply. Apply only runs on git push. 81 | --------------------------------------------------------------------------------