├── .github └── workflows │ └── test.yml ├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── docs ├── api-gateway.svg ├── lambda.svg ├── s3.svg └── serverless-pypi.png ├── example ├── Makefile └── terraform.tf ├── main.tf ├── outputs.tf ├── python ├── .python-version ├── Makefile ├── Pipfile ├── Pipfile.lock ├── index.py ├── index_test.py └── pyproject.toml └── variables.tf /.github/workflows/test.yml: -------------------------------------------------------------------------------- 1 | name: test 2 | on: 3 | pull_request: 4 | push: 5 | jobs: 6 | pytest: 7 | runs-on: ubuntu-latest 8 | steps: 9 | - uses: actions/checkout@v3 10 | - uses: actions/setup-python@v4 11 | with: 12 | python-version: "3.12" 13 | - uses: amancevice/setup-code-climate@v1 14 | with: 15 | cc_test_reporter_id: ${{ secrets.CC_TEST_REPORTER_ID }} 16 | - run: cc-test-reporter before-build 17 | - run: pip install pipenv 18 | - run: make test 19 | - run: cc-test-reporter after-build 20 | working-directory: python 21 | if: ${{ github.event_name != 'pull_request' }} 22 | terraform-validate: 23 | runs-on: ubuntu-latest 24 | steps: 25 | - uses: actions/checkout@v3 26 | - uses: hashicorp/setup-terraform@v2 27 | - run: make validate 28 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Local .terraform directories 2 | **/.terraform/ 3 | 4 | # .tfstate files 5 | *.tfstate 6 | *.tfstate.* 7 | 8 | # .tfvars files 9 | *.tfvars 10 | 11 | .docker/ 12 | .venv/ 13 | __pycache__/ 14 | *.coverage 15 | *.env 16 | *.iid 17 | *.whl 18 | *.zip 19 | .terraform.lock.hcl 20 | coverage.xml 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Alexander Mancevice 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: test validate 2 | 3 | clean: 4 | make -C example clean 5 | make -C python clean 6 | 7 | test: 8 | make -C python test 9 | 10 | validate: 11 | make -C example validate 12 | 13 | .PHONY: test validate 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Serverless PyPI 2 | 3 | [![terraform](https://img.shields.io/github/v/tag/amancevice/terraform-aws-serverless-pypi?color=62f&label=version&logo=terraform&style=flat-square)](https://registry.terraform.io/modules/amancevice/serverless-pypi/aws) 4 | [![test](https://img.shields.io/github/actions/workflow/status/amancevice/terraform-aws-serverless-pypi/test.yml?logo=github&style=flat-square)](https://github.com/amancevice/terraform-aws-serverless-pypi/actions/workflows/test.yml) 5 | [![coverage](https://img.shields.io/codeclimate/coverage/amancevice/terraform-aws-serverless-pypi?logo=code-climate&style=flat-square)](https://codeclimate.com/github/amancevice/terraform-aws-serverless-pypi/test_coverage) 6 | [![maintainability](https://img.shields.io/codeclimate/maintainability/amancevice/terraform-aws-serverless-pypi?logo=code-climate&style=flat-square)](https://codeclimate.com/github/amancevice/terraform-aws-serverless-pypi/maintainability) 7 | 8 | [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/smallweirdnumber) 9 | 10 | S3-backed serverless PyPI. 11 | 12 | Requests to your PyPI server will be proxied through a Lambda function that pulls content from an S3 bucket and responds with the same HTML content that you might find in a conventional PyPI server. 13 | 14 | Requests to the base path (eg, `/simple/`) will respond with the contents of an `index.html` file at the root of your S3 bucket. 15 | 16 | Requests to the package index (eg, `/simple/fizz/`) will dynamically generate an HTML file based on the contents of keys under that namespace (eg, `s3://your-bucket/fizz/`). URLs for package downloads are presigned S3 URLs with a default lifespan of 15 minutes. 17 | 18 | Package uploads/removals on S3 will trigger a Lambda function that reindexes the bucket and generates a new `index.html` at the root. This is done to save time when querying the base path when your bucket contains a multitude of packages. 19 | 20 | ![Serverless PyPI](./docs/serverless-pypi.png) 21 | 22 | ## Usage 23 | 24 | As of v7 users are expected to bring-your-own REST API (v1). This gives users greater flexibility in choosing how their API is set up. 25 | 26 | The most simplistic setup is as follows: 27 | 28 | ```terraform 29 | ####################### 30 | # SERVERLESS PYPI # 31 | ####################### 32 | 33 | module "serverless_pypi" { 34 | source = "amancevice/serverless-pypi/aws" 35 | version = "~> 7" 36 | 37 | api_execution_arn = aws_api_gateway_rest_api.pypi.execution_arn 38 | api_id = aws_api_gateway_rest_api.pypi.id 39 | api_root_resource_id = aws_api_gateway_rest_api.pypi.root_resource_id 40 | event_rule_name = "serverless-pypi-reindex" 41 | iam_role_name = "serverless-pypi" 42 | lambda_api_fallback_index_url = "https://pypi.org/simple/" 43 | lambda_api_function_name = "serverless-pypi-api" 44 | lambda_reindex_function_name = "serverless-pypi-reindex" 45 | s3_bucket_name = "serverless-pypi-us-west-2" 46 | 47 | # etc … 48 | } 49 | 50 | ################ 51 | # REST API # 52 | ################ 53 | 54 | resource "aws_api_gateway_rest_api" "pypi" { 55 | description = "Serverless PyPI example" 56 | name = "serverless-pypi" 57 | 58 | endpoint_configuration { types = ["REGIONAL"] } 59 | } 60 | 61 | resource "aws_api_gateway_deployment" "pypi" { 62 | rest_api_id = aws_api_gateway_rest_api.pypi.id 63 | 64 | triggers = { redeployment = module.serverless_pypi.api_deployment_trigger } 65 | 66 | lifecycle { create_before_destroy = true } 67 | } 68 | 69 | resource "aws_api_gateway_stage" "simple" { 70 | deployment_id = aws_api_gateway_deployment.pypi.id 71 | rest_api_id = aws_api_gateway_rest_api.pypi.id 72 | stage_name = "simple" 73 | } 74 | ``` 75 | 76 | ## S3 Bucket Organization 77 | 78 | This tool is highly opinionated about how your S3 bucket is organized. Your root key space should only contain the auto-generated `index.html` and "directories" of your PyPI packages. 79 | 80 | Packages should exist one level deep in the bucket where the prefix is the name of the project. 81 | 82 | Example: 83 | 84 | ```plain 85 | s3://your-bucket/ 86 | ├── index.html 87 | ├── my-cool-package/ 88 | │   ├── my-cool-package-0.1.2.tar.gz 89 | │   ├── my-cool-package-1.2.3.tar.gz 90 | │ └── my-cool-package-2.3.4.tar.gz 91 | └── my-other-package/ 92 |    ├── my-other-package-0.1.2.tar.gz 93 |    ├── my-other-package-1.2.3.tar.gz 94 | └── my-other-package-2.3.4.tar.gz 95 | ``` 96 | 97 | ## Fallback PyPI Index 98 | 99 | You can configure your PyPI index to fall back to a different PyPI in the event that a package is not found in your bucket. 100 | 101 | Without configuring a fallback index URL the following `pip install` command will surely fail (assuming you don't have `boto3` and all its dependencies in your S3 bucket): 102 | 103 | ```bash 104 | pip install boto3 --index-url https://my.private.pypi/simple/ 105 | ``` 106 | 107 | Instead, if you configure a fallback index URL in the terraform module, then requests for a pip that isn't found in the bucket will be re-routed to the fallback. 108 | 109 | ```terraform 110 | module "serverless_pypi" { 111 | source = "amancevice/serverless-pypi/aws" 112 | version = "~> 7" 113 | 114 | lambda_api_fallback_index_url = "https://pypi.org/simple/" 115 | 116 | # etc … 117 | } 118 | ``` 119 | 120 | ## Auth 121 | 122 | Please note that this tool provides **NO** authentication layer for your PyPI index out of the box. This is difficult to implement because `pip` is currently not very forgiving with any kind of auth pattern outside Basic Auth. 123 | 124 | Using a REST API configured for a private VPC is the easiest solution to this problem, but you could also write a custom authorizer for your API as well. 125 | -------------------------------------------------------------------------------- /docs/api-gateway.svg: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /docs/lambda.svg: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /docs/s3.svg: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /docs/serverless-pypi.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/amancevice/terraform-aws-serverless-pypi/512e24551140f348be1cf4b5825e9c5faf8ad876/docs/serverless-pypi.png -------------------------------------------------------------------------------- /example/Makefile: -------------------------------------------------------------------------------- 1 | BUCKET := us-west-2-serverless-pypi 2 | ENDPOINT := $(shell terraform output -raw endpoint) 3 | 4 | all: validate 5 | 6 | clean: 7 | rm -rf .terraform* 8 | 9 | curl: 10 | curl -s $(ENDPOINT)requests-iamauth/ | prettier --parser html | bat -l html 11 | 12 | logs: 13 | aws logs describe-log-groups --region us-west-2 \ 14 | | jq -r '.logGroups[].logGroupName' \ 15 | | grep serverless-pypi \ 16 | | fzf --no-info --reverse \ 17 | | xargs aws logs tail --follow 18 | 19 | ls: 20 | aws s3 ls s3://$(BUCKET)/ --recursive 21 | 22 | upload: requests_iamauth-0.7.0-py3-none-any.whl 23 | aws s3 cp $< s3://$(BUCKET)/requests-iamauth/$< 24 | 25 | validate: | .terraform 26 | terraform fmt -check 27 | AWS_REGION=us-east-1 terraform validate 28 | 29 | .PHONY: all curl logs ls upload validate 30 | 31 | requests_iamauth-%-py3-none-any.whl: 32 | pip download --no-deps requests-iamauth==$* 33 | 34 | .terraform: ../*.tf 35 | terraform init 36 | touch $@ 37 | -------------------------------------------------------------------------------- /example/terraform.tf: -------------------------------------------------------------------------------- 1 | ########### 2 | # AWS # 3 | ########### 4 | 5 | provider "aws" { 6 | region = local.region 7 | 8 | default_tags { tags = { Name = local.name } } 9 | } 10 | 11 | ############## 12 | # LOCALS # 13 | ############## 14 | 15 | locals { 16 | region = "us-east-1" 17 | name = "serverless-pypi" 18 | } 19 | 20 | ########### 21 | # DNS # 22 | ########### 23 | 24 | variable "domain_name" { type = string } 25 | 26 | data "aws_route53_zone" "zone" { 27 | name = var.domain_name 28 | } 29 | 30 | data "aws_acm_certificate" "ssl" { 31 | domain = var.domain_name 32 | statuses = ["ISSUED"] 33 | } 34 | 35 | resource "aws_api_gateway_domain_name" "pypi" { 36 | domain_name = "pypi.${var.domain_name}" 37 | regional_certificate_arn = data.aws_acm_certificate.ssl.arn 38 | 39 | endpoint_configuration { types = ["REGIONAL"] } 40 | } 41 | 42 | resource "aws_api_gateway_base_path_mapping" "pypi" { 43 | api_id = aws_api_gateway_rest_api.pypi.id 44 | base_path = aws_api_gateway_stage.simple.stage_name 45 | domain_name = aws_api_gateway_domain_name.pypi.domain_name 46 | stage_name = aws_api_gateway_stage.simple.stage_name 47 | } 48 | 49 | resource "aws_route53_record" "pypi" { 50 | for_each = toset(["A", "AAAA"]) 51 | 52 | name = aws_api_gateway_domain_name.pypi.domain_name 53 | set_identifier = local.region 54 | type = each.key 55 | zone_id = data.aws_route53_zone.zone.id 56 | 57 | alias { 58 | evaluate_target_health = false 59 | name = aws_api_gateway_domain_name.pypi.regional_domain_name 60 | zone_id = aws_api_gateway_domain_name.pypi.regional_zone_id 61 | } 62 | 63 | latency_routing_policy { 64 | region = local.region 65 | } 66 | } 67 | 68 | ################## 69 | # API GATEWAY # 70 | ################## 71 | 72 | resource "aws_api_gateway_rest_api" "pypi" { 73 | description = "Serverless PyPI example" 74 | disable_execute_api_endpoint = true 75 | name = "serverless-pypi" 76 | 77 | endpoint_configuration { types = ["REGIONAL"] } 78 | } 79 | 80 | resource "aws_api_gateway_deployment" "pypi" { 81 | rest_api_id = aws_api_gateway_rest_api.pypi.id 82 | 83 | triggers = { redeployment = module.serverless_pypi.api_deployment_trigger } 84 | 85 | lifecycle { create_before_destroy = true } 86 | } 87 | 88 | resource "aws_api_gateway_stage" "simple" { 89 | deployment_id = aws_api_gateway_deployment.pypi.id 90 | rest_api_id = aws_api_gateway_rest_api.pypi.id 91 | stage_name = "simple" 92 | } 93 | 94 | ####################### 95 | # SERVERLESS PYPI # 96 | ####################### 97 | 98 | module "serverless_pypi" { 99 | source = "./.." 100 | 101 | api_id = aws_api_gateway_rest_api.pypi.id 102 | api_execution_arn = aws_api_gateway_rest_api.pypi.execution_arn 103 | api_root_resource_id = aws_api_gateway_rest_api.pypi.root_resource_id 104 | event_rule_name = "${local.name}-reindex" 105 | iam_role_name = "${local.region}-${local.name}" 106 | lambda_api_fallback_index_url = "https://pypi.org/simple/" 107 | lambda_api_function_name = "${local.name}-api" 108 | lambda_reindex_function_name = "${local.name}-reindex" 109 | lambda_reindex_timeout = 14 110 | log_group_api_retention_in_days = 14 111 | log_group_reindex_retention_in_days = 14 112 | s3_bucket_name = "${local.region}-${local.name}" 113 | } 114 | 115 | ############### 116 | # OUTPUTS # 117 | ############### 118 | 119 | output "endpoint" { value = "https://${aws_api_gateway_base_path_mapping.pypi.domain_name}/${aws_api_gateway_base_path_mapping.pypi.base_path}/" } 120 | -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | ################# 2 | # TERRAFORM # 3 | ################# 4 | 5 | terraform { 6 | required_version = "~> 1.0" 7 | 8 | required_providers { 9 | archive = { 10 | source = "hashicorp/archive" 11 | version = "~> 2.0" 12 | } 13 | 14 | aws = { 15 | source = "hashicorp/aws" 16 | version = "~> 5.31" 17 | } 18 | } 19 | } 20 | 21 | ############## 22 | # LOCALS # 23 | ############## 24 | 25 | locals { 26 | event_rule = { 27 | description = var.event_rule_description 28 | name = var.event_rule_name 29 | } 30 | 31 | iam_role = { 32 | description = var.iam_role_description 33 | name = var.iam_role_name 34 | policy_name = var.iam_role_policy_name 35 | tags = var.iam_role_tags 36 | } 37 | 38 | lambda = { 39 | filename = data.archive_file.package.output_path 40 | runtime = var.lambda_runtime 41 | source_code_hash = data.archive_file.package.output_base64sha256 42 | } 43 | 44 | lambda_api = { 45 | description = var.lambda_api_description 46 | function_name = var.lambda_api_function_name 47 | memory_size = var.lambda_api_memory_size 48 | fallback_index_url = var.lambda_api_fallback_index_url 49 | tags = var.lambda_api_tags 50 | timeout = var.lambda_api_timeout 51 | } 52 | 53 | lambda_reindex = { 54 | description = var.lambda_reindex_description 55 | function_name = var.lambda_reindex_function_name 56 | memory_size = var.lambda_reindex_memory_size 57 | tags = var.lambda_reindex_tags 58 | timeout = var.lambda_reindex_timeout 59 | } 60 | 61 | log_group_api = { 62 | retention_in_days = var.log_group_api_retention_in_days 63 | tags = var.log_group_api_tags 64 | } 65 | 66 | log_group_reindex = { 67 | retention_in_days = var.log_group_reindex_retention_in_days 68 | tags = var.log_group_reindex_tags 69 | } 70 | 71 | rest_api = { 72 | authorization_type = var.api_authorization_type 73 | authorizer_id = var.api_authorizer_id 74 | execution_arn = var.api_execution_arn 75 | id = var.api_id 76 | root_resource_id = var.api_root_resource_id 77 | } 78 | 79 | routes = { 80 | "GET /" = { http_method : "GET", resource_id : local.rest_api.root_resource_id } 81 | "HEAD /" = { http_method : "HEAD", resource_id : local.rest_api.root_resource_id } 82 | "POST /" = { http_method : "POST", resource_id : local.rest_api.root_resource_id } 83 | "GET /{package+}" = { http_method : "GET", resource_id : aws_api_gateway_resource.proxy.id } 84 | "HEAD /{package+}" = { http_method : "HEAD", resource_id : aws_api_gateway_resource.proxy.id } 85 | } 86 | 87 | s3 = { 88 | bucket_name = var.s3_bucket_name 89 | bucket_tags = var.s3_bucket_tags 90 | presigned_url_ttl = var.s3_presigned_url_ttl 91 | } 92 | } 93 | 94 | #################### 95 | # S3 :: BUCKET # 96 | #################### 97 | 98 | resource "aws_s3_bucket" "pypi" { 99 | bucket = local.s3.bucket_name 100 | tags = local.s3.bucket_tags 101 | } 102 | 103 | resource "aws_s3_bucket_public_access_block" "pypi" { 104 | block_public_acls = true 105 | block_public_policy = true 106 | bucket = aws_s3_bucket.pypi.id 107 | ignore_public_acls = true 108 | restrict_public_buckets = true 109 | } 110 | 111 | resource "aws_s3_object" "index" { 112 | bucket = aws_s3_bucket.pypi.id 113 | key = "index.html" 114 | content = <<-EOT 115 | 116 | 117 | 118 | 119 | 120 | Simple index 121 | 122 | 123 | 124 |

Simple index

125 | 126 | 127 | 128 | EOT 129 | 130 | lifecycle { ignore_changes = [content] } 131 | } 132 | 133 | #################### 134 | # S3 :: EVENTS # 135 | #################### 136 | 137 | data "aws_caller_identity" "current" { 138 | } 139 | 140 | resource "aws_s3_bucket_notification" "reindex" { 141 | bucket = aws_s3_bucket.pypi.id 142 | eventbridge = true 143 | } 144 | 145 | ################### 146 | # EVENTBRIDGE # 147 | ################### 148 | 149 | resource "aws_cloudwatch_event_rule" "reindex" { 150 | description = local.event_rule.description 151 | name = local.event_rule.name 152 | 153 | event_pattern = jsonencode({ 154 | source = ["aws.s3"] 155 | detail-type = ["Object Created", "Object Deleted"] 156 | 157 | detail = { 158 | bucket = { name = [aws_s3_bucket.pypi.id] } 159 | object = { key = [{ anything-but = ["index.html"] }] } 160 | } 161 | }) 162 | } 163 | 164 | resource "aws_cloudwatch_event_target" "reindex" { 165 | arn = aws_lambda_function.reindex.arn 166 | input_path = "$.detail" 167 | rule = aws_cloudwatch_event_rule.reindex.name 168 | target_id = "reindex" 169 | } 170 | 171 | ########### 172 | # IAM # 173 | ########### 174 | 175 | resource "aws_iam_role" "role" { 176 | description = local.iam_role.description 177 | name = local.iam_role.name 178 | tags = local.iam_role.tags 179 | 180 | assume_role_policy = jsonencode({ 181 | Version = "2012-10-17" 182 | Statement = [{ 183 | Sid = "AssumeRole" 184 | Effect = "Allow" 185 | Action = "sts:AssumeRole" 186 | Principal = { Service = "lambda.amazonaws.com" } 187 | }] 188 | }) 189 | } 190 | 191 | resource "aws_iam_role_policy" "policy" { 192 | name = local.iam_role.policy_name 193 | role = aws_iam_role.role.id 194 | 195 | policy = jsonencode({ 196 | Version = "2012-10-17" 197 | Statement = [ 198 | { 199 | Sid = "ListBucket" 200 | Effect = "Allow" 201 | Action = "s3:ListBucket" 202 | Resource = aws_s3_bucket.pypi.arn 203 | }, 204 | { 205 | Sid = "GetObjects" 206 | Effect = "Allow" 207 | Action = "s3:GetObject" 208 | Resource = "${aws_s3_bucket.pypi.arn}/*" 209 | }, 210 | { 211 | Sid = "PutIndex" 212 | Effect = "Allow" 213 | Action = "s3:PutObject" 214 | Resource = "${aws_s3_bucket.pypi.arn}/index.html" 215 | }, 216 | { 217 | Sid = "WriteLambdaLogs" 218 | Effect = "Allow" 219 | Resource = "*" 220 | 221 | Action = [ 222 | "logs:CreateLogGroup", 223 | "logs:CreateLogStream", 224 | "logs:PutLogEvents", 225 | ] 226 | } 227 | ] 228 | }) 229 | } 230 | 231 | ######################### 232 | # LAMBDA :: PACKAGE # 233 | ######################### 234 | 235 | data "archive_file" "package" { 236 | source_file = "${path.module}/python/index.py" 237 | output_path = "${path.module}/python/package.zip" 238 | type = "zip" 239 | } 240 | 241 | ########################### 242 | # LAMBDA :: API PROXY # 243 | ########################### 244 | 245 | resource "aws_cloudwatch_log_group" "api" { 246 | name = "/aws/lambda/${aws_lambda_function.api.function_name}" 247 | retention_in_days = local.log_group_api.retention_in_days 248 | tags = local.log_group_api.tags 249 | } 250 | 251 | resource "aws_lambda_function" "api" { 252 | architectures = ["arm64"] 253 | description = local.lambda_api.description 254 | filename = local.lambda.filename 255 | function_name = local.lambda_api.function_name 256 | handler = "index.proxy_request" 257 | memory_size = local.lambda_api.memory_size 258 | role = aws_iam_role.role.arn 259 | runtime = local.lambda.runtime 260 | source_code_hash = local.lambda.source_code_hash 261 | tags = local.lambda_api.tags 262 | timeout = local.lambda_api.timeout 263 | 264 | environment { 265 | variables = { 266 | FALLBACK_INDEX_URL = local.lambda_api.fallback_index_url 267 | S3_BUCKET = aws_s3_bucket.pypi.bucket 268 | S3_PRESIGNED_URL_TTL = local.s3.presigned_url_ttl 269 | } 270 | } 271 | } 272 | 273 | resource "aws_lambda_permission" "api" { 274 | action = "lambda:InvokeFunction" 275 | function_name = aws_lambda_function.api.function_name 276 | principal = "apigateway.amazonaws.com" 277 | source_arn = "${local.rest_api.execution_arn}/*/*/*" 278 | } 279 | 280 | ########################### 281 | # LAMBDA :: REINDEXER # 282 | ########################### 283 | 284 | resource "aws_cloudwatch_log_group" "reindex" { 285 | name = "/aws/lambda/${aws_lambda_function.reindex.function_name}" 286 | retention_in_days = local.log_group_reindex.retention_in_days 287 | tags = local.log_group_reindex.tags 288 | } 289 | 290 | resource "aws_lambda_function" "reindex" { 291 | architectures = ["arm64"] 292 | description = local.lambda_reindex.description 293 | filename = local.lambda.filename 294 | function_name = local.lambda_reindex.function_name 295 | handler = "index.reindex_bucket" 296 | memory_size = local.lambda_reindex.memory_size 297 | role = aws_iam_role.role.arn 298 | runtime = local.lambda.runtime 299 | source_code_hash = local.lambda.source_code_hash 300 | tags = local.lambda_reindex.tags 301 | timeout = local.lambda_reindex.timeout 302 | 303 | environment { 304 | variables = { 305 | S3_BUCKET = aws_s3_bucket.pypi.bucket 306 | } 307 | } 308 | } 309 | 310 | resource "aws_lambda_permission" "reindex" { 311 | action = "lambda:InvokeFunction" 312 | function_name = aws_lambda_function.reindex.function_name 313 | principal = "events.amazonaws.com" 314 | source_arn = aws_cloudwatch_event_rule.reindex.arn 315 | } 316 | 317 | ########################### 318 | # API GATEWAY :: REST # 319 | ########################### 320 | 321 | resource "aws_api_gateway_resource" "proxy" { 322 | rest_api_id = local.rest_api.id 323 | parent_id = local.rest_api.root_resource_id 324 | path_part = "{package+}" 325 | } 326 | 327 | resource "aws_api_gateway_method" "methods" { 328 | for_each = local.routes 329 | authorization = local.rest_api.authorization_type 330 | authorizer_id = local.rest_api.authorizer_id 331 | http_method = each.value.http_method 332 | resource_id = each.value.resource_id 333 | rest_api_id = local.rest_api.id 334 | } 335 | 336 | resource "aws_api_gateway_integration" "integrations" { 337 | depends_on = [aws_api_gateway_method.methods] 338 | for_each = local.routes 339 | rest_api_id = local.rest_api.id 340 | resource_id = each.value.resource_id 341 | http_method = each.value.http_method 342 | integration_http_method = "POST" 343 | type = "AWS_PROXY" 344 | uri = aws_lambda_function.api.invoke_arn 345 | } 346 | -------------------------------------------------------------------------------- /outputs.tf: -------------------------------------------------------------------------------- 1 | output "api_deployment_trigger" { 2 | description = "API Gateway deployment trigger" 3 | 4 | value = sha1(jsonencode(concat( 5 | [aws_api_gateway_resource.proxy.id], 6 | [for x in aws_api_gateway_integration.integrations : x.id], 7 | [for x in aws_api_gateway_method.methods : x.id], 8 | ))) 9 | } 10 | 11 | output "api_integrations" { 12 | description = "API Gateway integrations" 13 | value = aws_api_gateway_integration.integrations 14 | } 15 | 16 | output "api_methods" { 17 | description = "API Gateway methods" 18 | value = aws_api_gateway_method.methods 19 | } 20 | 21 | output "api_resources" { 22 | description = "API Gateway resources" 23 | value = [aws_api_gateway_resource.proxy] 24 | } 25 | 26 | output "s3_bucket" { 27 | description = "PyPI S3 bucket" 28 | value = aws_s3_bucket.pypi 29 | } 30 | 31 | output "iam_role" { 32 | description = "PyPI API Lambda IAM role" 33 | value = aws_iam_role.role 34 | } 35 | 36 | output "lambda_api" { 37 | description = "PyPI API proxy Lambda function" 38 | value = aws_lambda_function.api 39 | } 40 | 41 | output "lambda_api_log_group" { 42 | description = "PyPI API proxy Lambda function CloudWatch log group" 43 | value = aws_cloudwatch_log_group.api 44 | } 45 | 46 | output "lambda_reindex" { 47 | description = "Reindexer Lambda function" 48 | value = aws_lambda_function.reindex 49 | } 50 | 51 | output "lambda_reindex_log_group" { 52 | description = "Reindexer Lambda function CloudWatch log group" 53 | value = aws_cloudwatch_log_group.reindex 54 | } 55 | -------------------------------------------------------------------------------- /python/.python-version: -------------------------------------------------------------------------------- 1 | 3.13.2 2 | -------------------------------------------------------------------------------- /python/Makefile: -------------------------------------------------------------------------------- 1 | all: test 2 | 3 | clean: 4 | pipenv --rm 5 | 6 | test: Pipfile.lock 7 | pipenv run black --check index.py index_test.py 8 | pipenv run pytest 9 | 10 | up: .env 11 | pipenv run python -m lambda_gateway index.proxy_request 12 | 13 | .PHONY: all clean test up 14 | 15 | Pipfile.lock: Pipfile | .venv 16 | pipenv lock 17 | 18 | .venv: .python-version Pipfile 19 | -pipenv --rm 20 | mkdir -p $@ 21 | pipenv install --dev 22 | touch $@ 23 | -------------------------------------------------------------------------------- /python/Pipfile: -------------------------------------------------------------------------------- 1 | [[source]] 2 | name = "pypi" 3 | url = "https://pypi.org/simple" 4 | verify_ssl = true 5 | 6 | [dev-packages] 7 | black = "*" 8 | ipdb = "*" 9 | ipython = "*" 10 | lambda-gateway = "*" 11 | pytest = "*" 12 | pytest-cov = "*" 13 | 14 | [packages] 15 | boto3 = "*" 16 | -------------------------------------------------------------------------------- /python/Pipfile.lock: -------------------------------------------------------------------------------- 1 | { 2 | "_meta": { 3 | "hash": { 4 | "sha256": "88f71af21f786227160b2e15f4a9622f654ebfa435ff35ba2fa9694e9f59a864" 5 | }, 6 | "pipfile-spec": 6, 7 | "requires": {}, 8 | "sources": [ 9 | { 10 | "name": "pypi", 11 | "url": "https://pypi.org/simple", 12 | "verify_ssl": true 13 | } 14 | ] 15 | }, 16 | "default": { 17 | "boto3": { 18 | "hashes": [ 19 | "sha256:873f8f5d2f6f85f1018cbb0535b03cceddc7b655b61f66a0a56995238804f41f", 20 | "sha256:d6f6096bdab35a0c0deff469563b87d184a28df7689790f7fe7be98502b7c590" 21 | ], 22 | "index": "pypi", 23 | "markers": "python_version >= '3.8'", 24 | "version": "==1.34.162" 25 | }, 26 | "botocore": { 27 | "hashes": [ 28 | "sha256:2d918b02db88d27a75b48275e6fb2506e9adaaddbec1ffa6a8a0898b34e769be", 29 | "sha256:adc23be4fb99ad31961236342b7cbf3c0bfc62532cd02852196032e8c0d682f3" 30 | ], 31 | "markers": "python_version >= '3.8'", 32 | "version": "==1.34.162" 33 | }, 34 | "jmespath": { 35 | "hashes": [ 36 | "sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980", 37 | "sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe" 38 | ], 39 | "markers": "python_version >= '3.7'", 40 | "version": "==1.0.1" 41 | }, 42 | "python-dateutil": { 43 | "hashes": [ 44 | "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", 45 | "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427" 46 | ], 47 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 48 | "version": "==2.9.0.post0" 49 | }, 50 | "s3transfer": { 51 | "hashes": [ 52 | "sha256:0711534e9356d3cc692fdde846b4a1e4b0cb6519971860796e6bc4c7aea00ef6", 53 | "sha256:eca1c20de70a39daee580aef4986996620f365c4e0fda6a86100231d62f1bf69" 54 | ], 55 | "markers": "python_version >= '3.8'", 56 | "version": "==0.10.2" 57 | }, 58 | "six": { 59 | "hashes": [ 60 | "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", 61 | "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" 62 | ], 63 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 64 | "version": "==1.16.0" 65 | }, 66 | "urllib3": { 67 | "hashes": [ 68 | "sha256:a448b2f64d686155468037e1ace9f2d2199776e17f0a46610480d311f73e3472", 69 | "sha256:dd505485549a7a552833da5e6063639d0d177c04f23bc3864e41e5dc5f612168" 70 | ], 71 | "markers": "python_version >= '3.10'", 72 | "version": "==2.2.2" 73 | } 74 | }, 75 | "develop": { 76 | "asttokens": { 77 | "hashes": [ 78 | "sha256:051ed49c3dcae8913ea7cd08e46a606dba30b79993209636c4875bc1d637bc24", 79 | "sha256:b03869718ba9a6eb027e134bfdf69f38a236d681c83c160d510768af11254ba0" 80 | ], 81 | "version": "==2.4.1" 82 | }, 83 | "black": { 84 | "hashes": [ 85 | "sha256:09cdeb74d494ec023ded657f7092ba518e8cf78fa8386155e4a03fdcc44679e6", 86 | "sha256:1f13f7f386f86f8121d76599114bb8c17b69d962137fc70efe56137727c7047e", 87 | "sha256:2500945420b6784c38b9ee885af039f5e7471ef284ab03fa35ecdde4688cd83f", 88 | "sha256:2b59b250fdba5f9a9cd9d0ece6e6d993d91ce877d121d161e4698af3eb9c1018", 89 | "sha256:3c4285573d4897a7610054af5a890bde7c65cb466040c5f0c8b732812d7f0e5e", 90 | "sha256:505289f17ceda596658ae81b61ebbe2d9b25aa78067035184ed0a9d855d18afd", 91 | "sha256:62e8730977f0b77998029da7971fa896ceefa2c4c4933fcd593fa599ecbf97a4", 92 | "sha256:649f6d84ccbae73ab767e206772cc2d7a393a001070a4c814a546afd0d423aed", 93 | "sha256:6e55d30d44bed36593c3163b9bc63bf58b3b30e4611e4d88a0c3c239930ed5b2", 94 | "sha256:707a1ca89221bc8a1a64fb5e15ef39cd755633daa672a9db7498d1c19de66a42", 95 | "sha256:72901b4913cbac8972ad911dc4098d5753704d1f3c56e44ae8dce99eecb0e3af", 96 | "sha256:73bbf84ed136e45d451a260c6b73ed674652f90a2b3211d6a35e78054563a9bb", 97 | "sha256:7c046c1d1eeb7aea9335da62472481d3bbf3fd986e093cffd35f4385c94ae368", 98 | "sha256:81c6742da39f33b08e791da38410f32e27d632260e599df7245cccee2064afeb", 99 | "sha256:837fd281f1908d0076844bc2b801ad2d369c78c45cf800cad7b61686051041af", 100 | "sha256:972085c618ee94f402da1af548a4f218c754ea7e5dc70acb168bfaca4c2542ed", 101 | "sha256:9e84e33b37be070ba135176c123ae52a51f82306def9f7d063ee302ecab2cf47", 102 | "sha256:b19c9ad992c7883ad84c9b22aaa73562a16b819c1d8db7a1a1a49fb7ec13c7d2", 103 | "sha256:d6417535d99c37cee4091a2f24eb2b6d5ec42b144d50f1f2e436d9fe1916fe1a", 104 | "sha256:eab4dd44ce80dea27dc69db40dab62d4ca96112f87996bca68cd75639aeb2e4c", 105 | "sha256:f490dbd59680d809ca31efdae20e634f3fae27fba3ce0ba3208333b713bc3920", 106 | "sha256:fb6e2c0b86bbd43dee042e48059c9ad7830abd5c94b0bc518c0eeec57c3eddc1" 107 | ], 108 | "index": "pypi", 109 | "markers": "python_version >= '3.8'", 110 | "version": "==24.8.0" 111 | }, 112 | "click": { 113 | "hashes": [ 114 | "sha256:ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28", 115 | "sha256:ca9853ad459e787e2192211578cc907e7594e294c7ccc834310722b41b9ca6de" 116 | ], 117 | "markers": "python_version >= '3.7'", 118 | "version": "==8.1.7" 119 | }, 120 | "coverage": { 121 | "extras": [ 122 | "toml" 123 | ], 124 | "hashes": [ 125 | "sha256:06a737c882bd26d0d6ee7269b20b12f14a8704807a01056c80bb881a4b2ce6ca", 126 | "sha256:07e2ca0ad381b91350c0ed49d52699b625aab2b44b65e1b4e02fa9df0e92ad2d", 127 | "sha256:0c0420b573964c760df9e9e86d1a9a622d0d27f417e1a949a8a66dd7bcee7bc6", 128 | "sha256:0dbde0f4aa9a16fa4d754356a8f2e36296ff4d83994b2c9d8398aa32f222f989", 129 | "sha256:1125ca0e5fd475cbbba3bb67ae20bd2c23a98fac4e32412883f9bcbaa81c314c", 130 | "sha256:13b0a73a0896988f053e4fbb7de6d93388e6dd292b0d87ee51d106f2c11b465b", 131 | "sha256:166811d20dfea725e2e4baa71fffd6c968a958577848d2131f39b60043400223", 132 | "sha256:170d444ab405852903b7d04ea9ae9b98f98ab6d7e63e1115e82620807519797f", 133 | "sha256:1f4aa8219db826ce6be7099d559f8ec311549bfc4046f7f9fe9b5cea5c581c56", 134 | "sha256:225667980479a17db1048cb2bf8bfb39b8e5be8f164b8f6628b64f78a72cf9d3", 135 | "sha256:260933720fdcd75340e7dbe9060655aff3af1f0c5d20f46b57f262ab6c86a5e8", 136 | "sha256:2bdb062ea438f22d99cba0d7829c2ef0af1d768d1e4a4f528087224c90b132cb", 137 | "sha256:2c09f4ce52cb99dd7505cd0fc8e0e37c77b87f46bc9c1eb03fe3bc9991085388", 138 | "sha256:3115a95daa9bdba70aea750db7b96b37259a81a709223c8448fa97727d546fe0", 139 | "sha256:3e0cadcf6733c09154b461f1ca72d5416635e5e4ec4e536192180d34ec160f8a", 140 | "sha256:3f1156e3e8f2872197af3840d8ad307a9dd18e615dc64d9ee41696f287c57ad8", 141 | "sha256:4421712dbfc5562150f7554f13dde997a2e932a6b5f352edcce948a815efee6f", 142 | "sha256:44df346d5215a8c0e360307d46ffaabe0f5d3502c8a1cefd700b34baf31d411a", 143 | "sha256:502753043567491d3ff6d08629270127e0c31d4184c4c8d98f92c26f65019962", 144 | "sha256:547f45fa1a93154bd82050a7f3cddbc1a7a4dd2a9bf5cb7d06f4ae29fe94eaf8", 145 | "sha256:5621a9175cf9d0b0c84c2ef2b12e9f5f5071357c4d2ea6ca1cf01814f45d2391", 146 | "sha256:609b06f178fe8e9f89ef676532760ec0b4deea15e9969bf754b37f7c40326dbc", 147 | "sha256:645786266c8f18a931b65bfcefdbf6952dd0dea98feee39bd188607a9d307ed2", 148 | "sha256:6878ef48d4227aace338d88c48738a4258213cd7b74fd9a3d4d7582bb1d8a155", 149 | "sha256:6a89ecca80709d4076b95f89f308544ec8f7b4727e8a547913a35f16717856cb", 150 | "sha256:6db04803b6c7291985a761004e9060b2bca08da6d04f26a7f2294b8623a0c1a0", 151 | "sha256:6e2cd258d7d927d09493c8df1ce9174ad01b381d4729a9d8d4e38670ca24774c", 152 | "sha256:6e81d7a3e58882450ec4186ca59a3f20a5d4440f25b1cff6f0902ad890e6748a", 153 | "sha256:702855feff378050ae4f741045e19a32d57d19f3e0676d589df0575008ea5004", 154 | "sha256:78b260de9790fd81e69401c2dc8b17da47c8038176a79092a89cb2b7d945d060", 155 | "sha256:7bb65125fcbef8d989fa1dd0e8a060999497629ca5b0efbca209588a73356232", 156 | "sha256:7dea0889685db8550f839fa202744652e87c60015029ce3f60e006f8c4462c93", 157 | "sha256:8284cf8c0dd272a247bc154eb6c95548722dce90d098c17a883ed36e67cdb129", 158 | "sha256:877abb17e6339d96bf08e7a622d05095e72b71f8afd8a9fefc82cf30ed944163", 159 | "sha256:8929543a7192c13d177b770008bc4e8119f2e1f881d563fc6b6305d2d0ebe9de", 160 | "sha256:8ae539519c4c040c5ffd0632784e21b2f03fc1340752af711f33e5be83a9d6c6", 161 | "sha256:8f59d57baca39b32db42b83b2a7ba6f47ad9c394ec2076b084c3f029b7afca23", 162 | "sha256:9054a0754de38d9dbd01a46621636689124d666bad1936d76c0341f7d71bf569", 163 | "sha256:953510dfb7b12ab69d20135a0662397f077c59b1e6379a768e97c59d852ee51d", 164 | "sha256:95cae0efeb032af8458fc27d191f85d1717b1d4e49f7cb226cf526ff28179778", 165 | "sha256:9bc572be474cafb617672c43fe989d6e48d3c83af02ce8de73fff1c6bb3c198d", 166 | "sha256:9c56863d44bd1c4fe2abb8a4d6f5371d197f1ac0ebdee542f07f35895fc07f36", 167 | "sha256:9e0b2df163b8ed01d515807af24f63de04bebcecbd6c3bfeff88385789fdf75a", 168 | "sha256:a09ece4a69cf399510c8ab25e0950d9cf2b42f7b3cb0374f95d2e2ff594478a6", 169 | "sha256:a1ac0ae2b8bd743b88ed0502544847c3053d7171a3cff9228af618a068ed9c34", 170 | "sha256:a318d68e92e80af8b00fa99609796fdbcdfef3629c77c6283566c6f02c6d6704", 171 | "sha256:a4acd025ecc06185ba2b801f2de85546e0b8ac787cf9d3b06e7e2a69f925b106", 172 | "sha256:a6d3adcf24b624a7b778533480e32434a39ad8fa30c315208f6d3e5542aeb6e9", 173 | "sha256:a78d169acd38300060b28d600344a803628c3fd585c912cacc9ea8790fe96862", 174 | "sha256:a95324a9de9650a729239daea117df21f4b9868ce32e63f8b650ebe6cef5595b", 175 | "sha256:abd5fd0db5f4dc9289408aaf34908072f805ff7792632250dcb36dc591d24255", 176 | "sha256:b06079abebbc0e89e6163b8e8f0e16270124c154dc6e4a47b413dd538859af16", 177 | "sha256:b43c03669dc4618ec25270b06ecd3ee4fa94c7f9b3c14bae6571ca00ef98b0d3", 178 | "sha256:b48f312cca9621272ae49008c7f613337c53fadca647d6384cc129d2996d1133", 179 | "sha256:b5d7b556859dd85f3a541db6a4e0167b86e7273e1cdc973e5b175166bb634fdb", 180 | "sha256:b9f222de8cded79c49bf184bdbc06630d4c58eec9459b939b4a690c82ed05657", 181 | "sha256:c3c02d12f837d9683e5ab2f3d9844dc57655b92c74e286c262e0fc54213c216d", 182 | "sha256:c44fee9975f04b33331cb8eb272827111efc8930cfd582e0320613263ca849ca", 183 | "sha256:cf4b19715bccd7ee27b6b120e7e9dd56037b9c0681dcc1adc9ba9db3d417fa36", 184 | "sha256:d0c212c49b6c10e6951362f7c6df3329f04c2b1c28499563d4035d964ab8e08c", 185 | "sha256:d3296782ca4eab572a1a4eca686d8bfb00226300dcefdf43faa25b5242ab8a3e", 186 | "sha256:d85f5e9a5f8b73e2350097c3756ef7e785f55bd71205defa0bfdaf96c31616ff", 187 | "sha256:da511e6ad4f7323ee5702e6633085fb76c2f893aaf8ce4c51a0ba4fc07580ea7", 188 | "sha256:e05882b70b87a18d937ca6768ff33cc3f72847cbc4de4491c8e73880766718e5", 189 | "sha256:e61c0abb4c85b095a784ef23fdd4aede7a2628478e7baba7c5e3deba61070a02", 190 | "sha256:e6a08c0be454c3b3beb105c0596ebdc2371fab6bb90c0c0297f4e58fd7e1012c", 191 | "sha256:e9a6e0eb86070e8ccaedfbd9d38fec54864f3125ab95419970575b42af7541df", 192 | "sha256:ed37bd3c3b063412f7620464a9ac1314d33100329f39799255fb8d3027da50d3", 193 | "sha256:f1adfc8ac319e1a348af294106bc6a8458a0f1633cc62a1446aebc30c5fa186a", 194 | "sha256:f5796e664fe802da4f57a168c85359a8fbf3eab5e55cd4e4569fbacecc903959", 195 | "sha256:fc5a77d0c516700ebad189b587de289a20a78324bc54baee03dd486f0855d234", 196 | "sha256:fd21f6ae3f08b41004dfb433fa895d858f3f5979e7762d052b12aef444e29afc" 197 | ], 198 | "markers": "python_version >= '3.8'", 199 | "version": "==7.6.1" 200 | }, 201 | "decorator": { 202 | "hashes": [ 203 | "sha256:637996211036b6385ef91435e4fae22989472f9d571faba8927ba8253acbc330", 204 | "sha256:b8c3f85900b9dc423225913c5aace94729fe1fa9763b38939a95226f02d37186" 205 | ], 206 | "markers": "python_version >= '3.5'", 207 | "version": "==5.1.1" 208 | }, 209 | "executing": { 210 | "hashes": [ 211 | "sha256:35afe2ce3affba8ee97f2d69927fa823b08b472b7b994e36a52a964b93d16147", 212 | "sha256:eac49ca94516ccc753f9fb5ce82603156e590b27525a8bc32cce8ae302eb61bc" 213 | ], 214 | "markers": "python_version >= '3.5'", 215 | "version": "==2.0.1" 216 | }, 217 | "iniconfig": { 218 | "hashes": [ 219 | "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3", 220 | "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374" 221 | ], 222 | "markers": "python_version >= '3.7'", 223 | "version": "==2.0.0" 224 | }, 225 | "ipdb": { 226 | "hashes": [ 227 | "sha256:45529994741c4ab6d2388bfa5d7b725c2cf7fe9deffabdb8a6113aa5ed449ed4", 228 | "sha256:e3ac6018ef05126d442af680aad863006ec19d02290561ac88b8b1c0b0cfc726" 229 | ], 230 | "index": "pypi", 231 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 232 | "version": "==0.13.13" 233 | }, 234 | "ipython": { 235 | "hashes": [ 236 | "sha256:1cec0fbba8404af13facebe83d04436a7434c7400e59f47acf467c64abd0956c", 237 | "sha256:e6b347c27bdf9c32ee9d31ae85defc525755a1869f14057e900675b9e8d6e6ff" 238 | ], 239 | "index": "pypi", 240 | "markers": "python_version >= '3.10'", 241 | "version": "==8.26.0" 242 | }, 243 | "jedi": { 244 | "hashes": [ 245 | "sha256:cf0496f3651bc65d7174ac1b7d043eff454892c708a87d1b683e57b569927ffd", 246 | "sha256:e983c654fe5c02867aef4cdfce5a2fbb4a50adc0af145f70504238f18ef5e7e0" 247 | ], 248 | "markers": "python_version >= '3.6'", 249 | "version": "==0.19.1" 250 | }, 251 | "lambda-gateway": { 252 | "hashes": [ 253 | "sha256:92ce7bae31c79e1c77ec967e78c37d8ab775ad932b0e612d212bec9abb4ca8c0", 254 | "sha256:b555e3ab4ef1fdc1abab084686a203e88b9dd75bc008881ef1278123377095e1" 255 | ], 256 | "index": "pypi", 257 | "markers": "python_version >= '3.8'", 258 | "version": "==1.1.0" 259 | }, 260 | "matplotlib-inline": { 261 | "hashes": [ 262 | "sha256:8423b23ec666be3d16e16b60bdd8ac4e86e840ebd1dd11a30b9f117f2fa0ab90", 263 | "sha256:df192d39a4ff8f21b1895d72e6a13f5fcc5099f00fa84384e0ea28c2cc0653ca" 264 | ], 265 | "markers": "python_version >= '3.8'", 266 | "version": "==0.1.7" 267 | }, 268 | "mypy-extensions": { 269 | "hashes": [ 270 | "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d", 271 | "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782" 272 | ], 273 | "markers": "python_version >= '3.5'", 274 | "version": "==1.0.0" 275 | }, 276 | "packaging": { 277 | "hashes": [ 278 | "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002", 279 | "sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124" 280 | ], 281 | "markers": "python_version >= '3.8'", 282 | "version": "==24.1" 283 | }, 284 | "parso": { 285 | "hashes": [ 286 | "sha256:a418670a20291dacd2dddc80c377c5c3791378ee1e8d12bffc35420643d43f18", 287 | "sha256:eb3a7b58240fb99099a345571deecc0f9540ea5f4dd2fe14c2a99d6b281ab92d" 288 | ], 289 | "markers": "python_version >= '3.6'", 290 | "version": "==0.8.4" 291 | }, 292 | "pathspec": { 293 | "hashes": [ 294 | "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", 295 | "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712" 296 | ], 297 | "markers": "python_version >= '3.8'", 298 | "version": "==0.12.1" 299 | }, 300 | "pexpect": { 301 | "hashes": [ 302 | "sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523", 303 | "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f" 304 | ], 305 | "markers": "sys_platform != 'win32' and sys_platform != 'emscripten'", 306 | "version": "==4.9.0" 307 | }, 308 | "platformdirs": { 309 | "hashes": [ 310 | "sha256:2d7a1657e36a80ea911db832a8a6ece5ee53d8de21edd5cc5879af6530b1bfee", 311 | "sha256:38b7b51f512eed9e84a22788b4bce1de17c0adb134d6becb09836e37d8654cd3" 312 | ], 313 | "markers": "python_version >= '3.8'", 314 | "version": "==4.2.2" 315 | }, 316 | "pluggy": { 317 | "hashes": [ 318 | "sha256:2cffa88e94fdc978c4c574f15f9e59b7f4201d439195c3715ca9e2486f1d0cf1", 319 | "sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669" 320 | ], 321 | "markers": "python_version >= '3.8'", 322 | "version": "==1.5.0" 323 | }, 324 | "prompt-toolkit": { 325 | "hashes": [ 326 | "sha256:0d7bfa67001d5e39d02c224b663abc33687405033a8c422d0d675a5a13361d10", 327 | "sha256:1e1b29cb58080b1e69f207c893a1a7bf16d127a5c30c9d17a25a5d77792e5360" 328 | ], 329 | "markers": "python_full_version >= '3.7.0'", 330 | "version": "==3.0.47" 331 | }, 332 | "ptyprocess": { 333 | "hashes": [ 334 | "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35", 335 | "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220" 336 | ], 337 | "version": "==0.7.0" 338 | }, 339 | "pure-eval": { 340 | "hashes": [ 341 | "sha256:1db8e35b67b3d218d818ae653e27f06c3aa420901fa7b081ca98cbedc874e0d0", 342 | "sha256:5f4e983f40564c576c7c8635ae88db5956bb2229d7e9237d03b3c0b0190eaf42" 343 | ], 344 | "version": "==0.2.3" 345 | }, 346 | "pygments": { 347 | "hashes": [ 348 | "sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199", 349 | "sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a" 350 | ], 351 | "markers": "python_version >= '3.8'", 352 | "version": "==2.18.0" 353 | }, 354 | "pytest": { 355 | "hashes": [ 356 | "sha256:4ba08f9ae7dcf84ded419494d229b48d0903ea6407b030eaec46df5e6a73bba5", 357 | "sha256:c132345d12ce551242c87269de812483f5bcc87cdbb4722e48487ba194f9fdce" 358 | ], 359 | "index": "pypi", 360 | "markers": "python_version >= '3.8'", 361 | "version": "==8.3.2" 362 | }, 363 | "pytest-cov": { 364 | "hashes": [ 365 | "sha256:4f0764a1219df53214206bf1feea4633c3b558a2925c8b59f144f682861ce652", 366 | "sha256:5837b58e9f6ebd335b0f8060eecce69b662415b16dc503883a02f45dfeb14857" 367 | ], 368 | "index": "pypi", 369 | "markers": "python_version >= '3.8'", 370 | "version": "==5.0.0" 371 | }, 372 | "six": { 373 | "hashes": [ 374 | "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", 375 | "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" 376 | ], 377 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 378 | "version": "==1.16.0" 379 | }, 380 | "stack-data": { 381 | "hashes": [ 382 | "sha256:836a778de4fec4dcd1dcd89ed8abff8a221f58308462e1c4aa2a3cf30148f0b9", 383 | "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695" 384 | ], 385 | "version": "==0.6.3" 386 | }, 387 | "traitlets": { 388 | "hashes": [ 389 | "sha256:9ed0579d3502c94b4b3732ac120375cda96f923114522847de4b3bb98b96b6b7", 390 | "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f" 391 | ], 392 | "markers": "python_version >= '3.8'", 393 | "version": "==5.14.3" 394 | }, 395 | "wcwidth": { 396 | "hashes": [ 397 | "sha256:3da69048e4540d84af32131829ff948f1e022c1c6bdb8d6102117aac784f6859", 398 | "sha256:72ea0c06399eb286d978fdedb6923a9eb47e1c486ce63e9b4e64fc18303972b5" 399 | ], 400 | "version": "==0.2.13" 401 | } 402 | } 403 | } 404 | -------------------------------------------------------------------------------- /python/index.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | import os 4 | import re 5 | from string import Template 6 | from xml.etree import ElementTree as xml 7 | 8 | import boto3 9 | 10 | S3 = boto3.client("s3") 11 | S3_BUCKET = os.environ["S3_BUCKET"] 12 | S3_PAGINATOR = S3.get_paginator("list_objects") 13 | S3_PRESIGNED_URL_TTL = int(os.getenv("S3_PRESIGNED_URL_TTL", "900")) 14 | 15 | FALLBACK_INDEX_URL = os.getenv("FALLBACK_INDEX_URL") 16 | LOG_LEVEL = os.getenv("LOG_LEVEL") or "INFO" 17 | LOG_FORMAT = os.getenv("LOG_FORMAT") or "%(levelname)s %(reqid)s %(message)s" 18 | 19 | 20 | class SuppressFilter(logging.Filter): 21 | """ 22 | Suppress Log Records from registered logger 23 | 24 | Taken from ``aws_lambda_powertools.logging.filters.SuppressFilter`` 25 | """ 26 | 27 | def __init__(self, logger): 28 | self.logger = logger 29 | 30 | def filter(self, record): 31 | logger = record.name 32 | return False if self.logger in logger else True 33 | 34 | 35 | class LambdaLoggerAdapter(logging.LoggerAdapter): 36 | """ 37 | Lambda logger adapter. 38 | """ 39 | 40 | def __init__(self, name, level=None, format_string=None): 41 | # Get logger, formatter 42 | logger = logging.getLogger(name) 43 | 44 | # Set log level 45 | logger.setLevel(level or LOG_LEVEL) 46 | 47 | # Set handler if necessary 48 | if not logger.handlers: # and not logger.parent.handlers: 49 | formatter = logging.Formatter(format_string or LOG_FORMAT) 50 | handler = logging.StreamHandler() 51 | handler.setFormatter(formatter) 52 | logger.addHandler(handler) 53 | 54 | # Suppress AWS logging for this logger 55 | for handler in logging.root.handlers: 56 | logFilter = SuppressFilter(name) 57 | handler.addFilter(logFilter) 58 | 59 | # Initialize adapter with null RequestId 60 | super().__init__(logger, dict(reqid="-")) 61 | 62 | def attach(self, handler): 63 | """ 64 | Decorate Lambda handler to attach logger to AWS request. 65 | 66 | :Example: 67 | 68 | >>> logger = lambo.getLogger(__name__) 69 | >>> 70 | >>> @logger.attach 71 | ... def handler(event, context): 72 | ... logger.info('Hello, world!') 73 | ... return {'ok': True} 74 | ... 75 | >>> handler({'fizz': 'buzz'}) 76 | >>> # => INFO RequestId: {awsRequestId} EVENT {"fizz": "buzz"} 77 | >>> # => INFO RequestId: {awsRequestId} Hello, world! 78 | >>> # => INFO RequestId: {awsRequestId} RETURN {"ok": True} 79 | """ 80 | 81 | def wrapper(event=None, context=None): 82 | try: 83 | self.addContext(context) 84 | self.info("EVENT %s", json.dumps(event, default=str)) 85 | result = handler(event, context) 86 | self.info("RETURN %s", json.dumps(result, default=str)) 87 | return result 88 | finally: 89 | self.dropContext() 90 | 91 | return wrapper 92 | 93 | def addContext(self, context=None): 94 | """ 95 | Add runtime context to logger. 96 | """ 97 | try: 98 | reqid = f"RequestId: {context.aws_request_id}" 99 | except AttributeError: 100 | reqid = "-" 101 | self.extra.update(reqid=reqid) 102 | return self 103 | 104 | def dropContext(self): 105 | """ 106 | Drop runtime context from logger. 107 | """ 108 | self.extra.update(reqid="-") 109 | return self 110 | 111 | 112 | logger = LambdaLoggerAdapter("PyPI") 113 | 114 | 115 | # LAMBDA HANDLERS 116 | 117 | 118 | @logger.attach 119 | def proxy_request(event, *_): 120 | """ 121 | Handle API Gateway proxy request. 122 | """ 123 | # Get HTTP request method, path, and body 124 | if event.get("version") == "2.0": 125 | method, package, body = parse_payload_v2(event) 126 | else: 127 | method, package, body = parse_payload_v1(event) 128 | 129 | # Get HTTP response 130 | try: 131 | res = ROUTES[method](package, body) 132 | except KeyError: 133 | res = reject(403, message="Forbidden") 134 | 135 | # Return proxy response 136 | logger.info("RESPONSE [%s]", res["statusCode"]) 137 | return res 138 | 139 | 140 | @logger.attach 141 | def reindex_bucket(*_): 142 | """ 143 | Reindex S3 bucket. 144 | """ 145 | # Get package names from common prefixes 146 | pages = S3_PAGINATOR.paginate(Bucket=S3_BUCKET, Delimiter="/") 147 | pkgs = ( 148 | x.get("Prefix").strip("/") 149 | for page in pages 150 | for x in page.get("CommonPrefixes", []) 151 | ) 152 | 153 | # Construct HTML 154 | anchors = (ANCHOR.safe_substitute(href=pkg, name=pkg) for pkg in pkgs) 155 | body = INDEX.safe_substitute(title="Simple index", anchors="".join(anchors)) 156 | 157 | # Upload to S3 as index.html 158 | res = S3.put_object(Bucket=S3_BUCKET, Key="index.html", Body=body.encode()) 159 | return res 160 | 161 | 162 | # LAMBDA HELPERS 163 | 164 | 165 | def get_index(): 166 | """ 167 | GET / 168 | 169 | :return dict: Response 170 | """ 171 | index = S3.get_object(Bucket=S3_BUCKET, Key="index.html") 172 | body = index["Body"].read().decode() 173 | return proxy_reponse(body) 174 | 175 | 176 | def get_package_index(name): 177 | """ 178 | GET / 179 | 180 | :param str name: Package name 181 | :return dict: Response 182 | """ 183 | # Get keys for given package 184 | pages = S3_PAGINATOR.paginate(Bucket=S3_BUCKET, Prefix=f"{name}/") 185 | keys = [key.get("Key") for page in pages for key in page.get("Contents") or []] 186 | 187 | # Go to fallback index if no keys 188 | if FALLBACK_INDEX_URL and not any(keys): 189 | fallback_url = os.path.join(FALLBACK_INDEX_URL, name, "") 190 | return redirect(fallback_url) 191 | 192 | # Respond with 404 if no keys and no fallback index 193 | elif not any(keys): 194 | return reject(404, message="Not Found") 195 | 196 | # Convert keys to presigned URLs 197 | hrefs = [presign(key) for key in keys] 198 | 199 | # Extract names of packages from keys 200 | names = [os.path.split(x)[-1] for x in keys] 201 | 202 | # Construct HTML 203 | anchors = [ 204 | ANCHOR.safe_substitute(href=href, name=name) for href, name in zip(hrefs, names) 205 | ] 206 | body = INDEX.safe_substitute(title=f"Links for {name}", anchors="".join(anchors)) 207 | 208 | # Convert to Lambda proxy response 209 | return proxy_reponse(body) 210 | 211 | 212 | def get_response(package, *_): 213 | """ 214 | GET /* 215 | 216 | :param str path: Request path 217 | :return dict: Response 218 | """ 219 | # GET / 220 | if package is None: 221 | return get_index() 222 | 223 | # GET /* 224 | return get_package_index(package) 225 | 226 | 227 | def head_response(package, *_): 228 | """ 229 | HEAD /* 230 | 231 | :param str path: Request path 232 | :return dict: Response 233 | """ 234 | res = get_response(package) 235 | res.update(body="") 236 | return res 237 | 238 | 239 | def parse_payload_v1(event): 240 | """ 241 | Get HTTP request method/path/body for v1 payloads. 242 | """ 243 | body = event.get("body") 244 | method = event.get("httpMethod") 245 | try: 246 | package, *_ = event["pathParameters"]["package"].split("/") 247 | except (KeyError, TypeError): 248 | package = None 249 | return (method, package, body) 250 | 251 | 252 | def parse_payload_v2(event): 253 | """ 254 | Get HTTP request method/path/body for v2 payloads. 255 | """ 256 | body = event.get("body") 257 | routeKey = event.get("routeKey") 258 | method, _ = routeKey.split(" ") 259 | try: 260 | package, *_ = event["pathParameters"]["package"].split("/") 261 | except KeyError: 262 | package = None 263 | return (method, package, body) 264 | 265 | 266 | def post_response(package, body): 267 | """ 268 | POST / 269 | 270 | :param str path: POST path 271 | :param str body: POST body 272 | :return dict: Response 273 | """ 274 | return reject(405, message="Not Allowed") 275 | 276 | 277 | def presign(key): 278 | """ 279 | Presign package URLs. 280 | 281 | :param str key: S3 key to presign 282 | :return str: Presigned URL 283 | """ 284 | url = S3.generate_presigned_url( 285 | "get_object", 286 | ExpiresIn=S3_PRESIGNED_URL_TTL, 287 | HttpMethod="GET", 288 | Params=dict(Bucket=S3_BUCKET, Key=key), 289 | ) 290 | return url 291 | 292 | 293 | def proxy_reponse(body, content_type=None): 294 | """ 295 | Convert HTML to API Gateway response. 296 | 297 | :param str body: HTML body 298 | :return dict: API Gateway Lambda proxy response 299 | """ 300 | content_type = content_type or "text/html" 301 | # Wrap HTML in proxy response object 302 | res = { 303 | "body": body, 304 | "statusCode": 200, 305 | "headers": { 306 | "content-length": len(body), 307 | "content-type": f"{content_type}; charset=utf-8", 308 | }, 309 | } 310 | return res 311 | 312 | 313 | def redirect(path): 314 | """ 315 | Redirect requests. 316 | 317 | :param str path: Rejection status code 318 | :return dict: Redirection response 319 | """ 320 | return dict(statusCode=301, headers={"Location": path}) 321 | 322 | 323 | def reject(status_code, **kwargs): 324 | """ 325 | Bad request. 326 | 327 | :param int status_code: Rejection status code 328 | :param dict kwargs: Rejection body JSON 329 | :return dict: Rejection response 330 | """ 331 | body = json.dumps(kwargs) if kwargs else "" 332 | headers = { 333 | "content-length": len(body), 334 | "content-type": "application/json; charset=utf-8", 335 | } 336 | return dict(body=body, headers=headers, statusCode=status_code) 337 | 338 | 339 | class MiniTemplate(Template): 340 | def __init__(self, template): 341 | super().__init__(re.sub(r"\n *", "", template)) 342 | 343 | 344 | ROUTES = dict(GET=get_response, HEAD=head_response, POST=post_response) 345 | ANCHOR = MiniTemplate('$name
') 346 | INDEX = MiniTemplate( 347 | """ 348 | 349 | 350 | 351 | 352 | $title 353 | 354 | 355 |

$title

356 | $anchors 357 | 358 | 359 | """ 360 | ) 361 | -------------------------------------------------------------------------------- /python/index_test.py: -------------------------------------------------------------------------------- 1 | import io 2 | import json 3 | import os 4 | import re 5 | from unittest import mock 6 | 7 | import pytest 8 | 9 | os.environ["S3_BUCKET"] = "serverless-pypi" 10 | 11 | with mock.patch("boto3.client"): 12 | import index 13 | from index import ANCHOR, INDEX 14 | 15 | SIMPLE_INDEX = INDEX.safe_substitute( 16 | title="Simple index", 17 | anchors=str.join( 18 | "", 19 | [ 20 | ANCHOR.safe_substitute(href="fizz", name="fizz"), 21 | ANCHOR.safe_substitute(href="buzz", name="buzz"), 22 | ], 23 | ), 24 | ) 25 | PACKAGE_INDEX = INDEX.safe_substitute( 26 | title="Links for fizz", 27 | anchors=str.join( 28 | "", 29 | [ 30 | ANCHOR.safe_substitute(href="presigned-url", name="fizz-0.1.2.tar.gz"), 31 | ANCHOR.safe_substitute(href="presigned-url", name="fizz-1.2.3.tar.gz"), 32 | ], 33 | ), 34 | ) 35 | S3_REINDEX_RESPONSE = [ 36 | {"CommonPrefixes": [{"Prefix": "fizz/"}, {"Prefix": "buzz/"}]}, 37 | ] 38 | S3_INDEX_RESPONSE = [ 39 | { 40 | "Contents": [ 41 | {"Key": "fizz/fizz-0.1.2.tar.gz"}, 42 | {"Key": "fizz/fizz-1.2.3.tar.gz"}, 43 | ], 44 | }, 45 | ] 46 | 47 | 48 | def test_proxy_reponse(): 49 | body = "FIZZ" 50 | ret = index.proxy_reponse("FIZZ") 51 | exp = { 52 | "body": body, 53 | "statusCode": 200, 54 | "headers": { 55 | "content-length": len(body), 56 | "content-type": "text/html; charset=utf-8", 57 | }, 58 | } 59 | assert ret == exp 60 | 61 | 62 | def test_get_index(): 63 | index.S3.get_object.return_value = { 64 | "Body": io.BytesIO(SIMPLE_INDEX.encode()), 65 | } 66 | ret = index.get_index() 67 | exp = { 68 | "body": SIMPLE_INDEX, 69 | "statusCode": 200, 70 | "headers": { 71 | "content-length": len(SIMPLE_INDEX), 72 | "content-type": "text/html; charset=utf-8", 73 | }, 74 | } 75 | assert ret == exp 76 | 77 | 78 | def test_get_package_index(): 79 | index.S3.generate_presigned_url.return_value = "presigned-url" 80 | index.S3_PAGINATOR.paginate.return_value = iter(S3_INDEX_RESPONSE) 81 | ret = index.get_package_index("fizz") 82 | exp = { 83 | "body": PACKAGE_INDEX, 84 | "statusCode": 200, 85 | "headers": { 86 | "content-length": len(PACKAGE_INDEX), 87 | "content-type": "text/html; charset=utf-8", 88 | }, 89 | } 90 | assert ret == exp 91 | 92 | 93 | def test_get_package_index_fallback(): 94 | index.FALLBACK_INDEX_URL = "https://pypi.org/simple/" 95 | index.S3_PAGINATOR.paginate.return_value = iter([]) 96 | ret = index.get_package_index("buzz") 97 | exp = { 98 | "statusCode": 301, 99 | "headers": { 100 | "Location": "https://pypi.org/simple/buzz/", 101 | }, 102 | } 103 | assert ret == exp 104 | 105 | 106 | def test_get_package_index_not_found(): 107 | index.FALLBACK_INDEX_URL = "" 108 | index.S3_PAGINATOR.paginate.return_value = iter([]) 109 | body = json.dumps({"message": "Not Found"}) 110 | ret = index.get_package_index("buzz") 111 | exp = { 112 | "body": body, 113 | "statusCode": 404, 114 | "headers": { 115 | "content-length": len(body), 116 | "content-type": "application/json; charset=utf-8", 117 | }, 118 | } 119 | assert ret == exp 120 | 121 | 122 | def test_redirect(): 123 | ret = index.redirect("simple") 124 | exp = {"headers": {"Location": "simple"}, "statusCode": 301} 125 | assert ret == exp 126 | 127 | 128 | def test_reject(): 129 | body = json.dumps({"message": "Unauthorized"}) 130 | ret = index.reject(401, message="Unauthorized") 131 | exp = { 132 | "body": body, 133 | "statusCode": 401, 134 | "headers": { 135 | "content-length": len(body), 136 | "content-type": "application/json; charset=utf-8", 137 | }, 138 | } 139 | assert ret == exp 140 | 141 | 142 | @pytest.mark.parametrize( 143 | ("event", "exp"), 144 | [ 145 | ( 146 | {"version": "2.0", "routeKey": "GET /"}, 147 | { 148 | "statusCode": 200, 149 | "body": "", 150 | "headers": { 151 | "content-length": 0, 152 | "content-type": "text/html; charset=utf-8", 153 | }, 154 | }, 155 | ), 156 | ], 157 | ) 158 | def test_handler_get_root(event, exp): 159 | ret = index.proxy_request(event) 160 | assert ret == exp 161 | 162 | 163 | @pytest.mark.parametrize( 164 | "event", 165 | [ 166 | {"version": "2.0", "routeKey": "GET /"}, 167 | {"version": "2.0", "routeKey": "HEAD /"}, 168 | {"httpMethod": "GET"}, 169 | {"httpMethod": "HEAD"}, 170 | ], 171 | ) 172 | def test_proxy_request_get(event): 173 | with mock.patch("index.get_index") as mock_idx: 174 | mock_idx.return_value = index.proxy_reponse(SIMPLE_INDEX) 175 | index.proxy_request(event) 176 | mock_idx.assert_called_once_with() 177 | 178 | 179 | @pytest.mark.parametrize( 180 | "event", 181 | [ 182 | { 183 | "version": "2.0", 184 | "routeKey": "GET /fizz", 185 | "pathParameters": {"package": "fizz"}, 186 | }, 187 | {"httpMethod": "GET", "pathParameters": {"package": "fizz"}}, 188 | ], 189 | ) 190 | def test_proxy_request_get_package(event): 191 | with mock.patch("index.get_package_index") as mock_pkg: 192 | mock_pkg.return_value = index.proxy_reponse(PACKAGE_INDEX) 193 | index.proxy_request(event) 194 | mock_pkg.assert_called_once_with("fizz") 195 | 196 | 197 | @pytest.mark.parametrize( 198 | ("event", "status_code", "msg"), 199 | [ 200 | ( 201 | { 202 | "version": "2.0", 203 | "routeKey": "OPTIONS /fizz", 204 | "pathParameters": {"package": "fizz"}, 205 | }, 206 | 403, 207 | "Forbidden", 208 | ), 209 | ( 210 | { 211 | "version": "2.0", 212 | "routeKey": "POST /fizz", 213 | "pathParameters": {"package": "fizz"}, 214 | }, 215 | 405, 216 | "Not Allowed", 217 | ), 218 | ], 219 | ) 220 | def test_proxy_request_reject(event, status_code, msg): 221 | ret = index.proxy_request(event) 222 | exp = index.reject(status_code, message=msg) 223 | assert ret == exp 224 | 225 | 226 | def test_reindex_bucket(): 227 | index.S3_PAGINATOR.paginate.return_value = iter(S3_REINDEX_RESPONSE) 228 | index.reindex_bucket({}) 229 | index.S3.put_object.assert_called_once_with( 230 | Bucket=index.S3_BUCKET, 231 | Key="index.html", 232 | Body=SIMPLE_INDEX.encode(), 233 | ) 234 | -------------------------------------------------------------------------------- /python/pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.pytest.ini_options] 2 | minversion = "6.0" 3 | addopts = "--verbose --cov index --cov index_test --cov-report term-missing --cov-report xml" 4 | -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | variable "api_authorization_type" { 2 | description = "API Gateway REST API routes authorization type" 3 | default = "NONE" 4 | } 5 | 6 | variable "api_authorizer_id" { 7 | description = "API Gateway REST API routes authorizer ID" 8 | default = null 9 | } 10 | 11 | variable "api_id" { 12 | description = "API Gateway REST API ID" 13 | } 14 | 15 | variable "api_execution_arn" { 16 | description = "API Gateway REST API execution ARN" 17 | } 18 | 19 | variable "api_root_resource_id" { 20 | description = "API Gateway REST API root resource ID" 21 | } 22 | 23 | variable "event_rule_name" { 24 | description = "EventBridge reinexer rule name" 25 | } 26 | 27 | variable "event_rule_description" { 28 | description = "EventBridge reinexer rule description" 29 | default = "Serverless PyPI reindex" 30 | } 31 | 32 | variable "iam_role_description" { 33 | description = "Lambda function IAM role description" 34 | default = "PyPI Lambda permissions" 35 | } 36 | 37 | variable "iam_role_name" { 38 | description = "Lambda function role name" 39 | } 40 | 41 | variable "iam_role_policy_name" { 42 | description = "IAM role inline policy name" 43 | default = "pypi-lambda-permissions" 44 | } 45 | 46 | variable "iam_role_tags" { 47 | description = "Resource tags" 48 | type = map(string) 49 | default = {} 50 | } 51 | 52 | variable "lambda_api_alias_name" { 53 | description = "PyPI API Lambda alias name" 54 | default = "prod" 55 | } 56 | 57 | variable "lambda_api_alias_function_version" { 58 | description = "PyPI API Lambda alias target function version" 59 | default = "$LATEST" 60 | } 61 | 62 | variable "lambda_api_description" { 63 | description = "REST API Lambda function description" 64 | default = "PyPI service REST API" 65 | } 66 | 67 | variable "lambda_api_fallback_index_url" { 68 | description = "Optional fallback PyPI index URL" 69 | default = null 70 | } 71 | 72 | variable "lambda_api_function_name" { 73 | description = "PyPI API Lambda function name" 74 | } 75 | 76 | variable "lambda_api_memory_size" { 77 | description = "PyPI API Lambda function memory size" 78 | default = 128 79 | } 80 | 81 | variable "lambda_api_tags" { 82 | description = "Resource tags" 83 | type = map(string) 84 | default = {} 85 | } 86 | 87 | variable "lambda_api_timeout" { 88 | description = "Lambda function timeout" 89 | default = 3 90 | } 91 | 92 | variable "lambda_reindex_alias_name" { 93 | description = "Reindexer Lambda alias name" 94 | default = "prod" 95 | } 96 | 97 | variable "lambda_reindex_alias_function_version" { 98 | description = "Reindexer Lambda alias target function version" 99 | default = "$LATEST" 100 | } 101 | 102 | variable "lambda_reindex_description" { 103 | description = "Reindexer Lambda function decription" 104 | default = "PyPI service reindexer" 105 | } 106 | 107 | variable "lambda_reindex_function_name" { 108 | description = "Reindexer Lambda function name" 109 | } 110 | 111 | variable "lambda_reindex_memory_size" { 112 | description = "Reindexer Lambda function memory size" 113 | default = 128 114 | } 115 | 116 | variable "lambda_reindex_tags" { 117 | description = "Resource tags" 118 | type = map(string) 119 | default = {} 120 | } 121 | 122 | variable "lambda_reindex_timeout" { 123 | description = "Lambda function timeout" 124 | default = 3 125 | } 126 | 127 | variable "lambda_runtime" { 128 | description = "Lambda runtime" 129 | default = "python3.13" 130 | } 131 | 132 | variable "log_group_api_retention_in_days" { 133 | description = "CloudWatch log group retention period" 134 | default = 0 135 | } 136 | 137 | variable "log_group_api_tags" { 138 | description = "Resource tags" 139 | type = map(string) 140 | default = {} 141 | } 142 | 143 | variable "log_group_reindex_retention_in_days" { 144 | description = "CloudWatch log group retention period" 145 | default = 0 146 | } 147 | 148 | variable "log_group_reindex_tags" { 149 | description = "Resource tags" 150 | type = map(string) 151 | default = {} 152 | } 153 | 154 | variable "s3_bucket_name" { 155 | description = "PyPI index S3 bucket name" 156 | } 157 | 158 | variable "s3_bucket_tags" { 159 | description = "Resource tags" 160 | type = map(string) 161 | default = {} 162 | } 163 | 164 | variable "s3_presigned_url_ttl" { 165 | description = "PyPI package presigned URL expiration in seconds" 166 | default = 900 167 | } 168 | --------------------------------------------------------------------------------