├── .gitignore ├── .gitmodules ├── README.md └── fragmenty-infra ├── .gitignore ├── ECS ├── ecs-infra │ ├── .terraform.lock.hcl │ ├── backend.tf │ ├── container_build_push.tf │ ├── ecr.tf │ ├── ecs.tf │ ├── external.tf │ ├── get_sha_module.sh │ ├── lambda.tf │ ├── loadbalancer.tf │ ├── outputs.tf │ ├── provider.tf │ ├── route53.tf │ └── variables.tf └── ecs-remote-state │ ├── .terraform.lock.hcl │ ├── outputs.tf │ ├── remote_state.tf │ └── variables.tf └── MongoDBAtlas ├── mongo-infra ├── .terraform.lock.hcl ├── backend.tf ├── main.tf ├── outputs.tf └── variables.tf └── mongo-remote-state ├── .terraform.lock.hcl ├── outputs.tf ├── remote_state.tf └── variables.tf /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | .DS_Store -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "fragmenty-api"] 2 | path = fragmenty-api 3 | url = https://github.com/Maders/fragmenty-api.git 4 | [submodule "fragmenty-spider"] 5 | path = fragmenty-spider 6 | url = https://github.com/Maders/fragmenty-spider.git 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Fragmenty 2 | 3 | ## What is the project? 4 | 5 | This is a small side project that crawls the [Fragment](https://fragment.com/numbers) Telegram platform to extract data about phone numbers, and provides a RESTful API, WebSocket API, and visualization of the data through a chart. 6 | 7 | The goal of this project is to extract data and basic insights about Telegram numbers auction, also learn more about the Play framework, Scala, Terraform and AWS. 8 | 9 | ## Stack: 10 | 11 | - [Scrapy framework](https://docs.scrapy.org/en/latest/) and Python language for Crawler part 12 | - [Play framework](https://www.playframework.com/documentation/2.8.x/ScalaHome) and Scala language for API server 13 | - [Plotly](https://plotly.com/graphing-libraries/) for data visualization 14 | - [MongoDB](https://www.mongodb.com/docs/) as data persistence 15 | - [Terraform](https://terraform.io/) infrastructure automation for provisioning 16 | - [AWS service](https://aws.amazon.com/) cloud infrastructure 17 | - [MongoDB Atlas](https://www.mongodb.com/atlas/database) cloud database service 18 | 19 | ## Infrastructure Architecture 20 | 21 | ![AWS Infrastructure architecture](https://user-images.githubusercontent.com/9904514/232190918-bc74bec6-857b-4f08-8e73-68c4c07df27e.png) 22 | 23 | This project uses Amazon Web Services (AWS) for infrastructure provisioning. The infrastructure is organized into different components, with each component residing in its own directory under the `fragmenty-infra` directory. 24 | 25 | ### Components: 26 | 27 | 1. Elastic Container Service (ECS) - Deploy and manage the containerized applications 28 | 2. MongoDB Atlas - Host the MongoDB instance for data persistence 29 | 30 | #### Elastic Container Service (ECS) 31 | 32 | The ECS infrastructure is set up using Terraform and includes the following resources: 33 | 34 | - Elastic Container Registry (ECR) for storing container images 35 | - ECS Cluster, ECS Service, and ECS Task Definition for running the containerized applications 36 | - AWS Lambda for running the Scrapy crawler periodically 37 | - Application Load Balancer (ALB) for distributing traffic to the ECS tasks 38 | - Route 53 for managing DNS records 39 | - AWS Certificate Manager (ACM) for SSL certificate provisioning 40 | 41 | #### MongoDB Atlas 42 | 43 | The MongoDB Atlas infrastructure is also set up using Terraform and consists of the following resources: 44 | 45 | - MongoDB Atlas Cluster 46 | - MongoDB Atlas Database Users 47 | 48 | ![graphviz terraform graph](https://user-images.githubusercontent.com/9904514/232057419-4d79fc97-0260-49de-a5d0-e5ec9e804178.svg) 49 | 50 |

51 | Visualized Terraform graph 52 |

53 | 54 | ## Deployment Workflow 55 | 56 | The deployment process is automated using Terraform. The `external.tf` file is used to extract the latest Git commit SHA for the `spider` and `api` modules. These SHAs are used as container image tags. Terraform uses `container_build_push.tf` to build and push the container images to the ECR. The `ecs.tf` file contains the resources required to run the containerized applications on ECS. 57 | 58 | The Lambda function, defined in `lambda.tf`, is responsible for running the Scrapy crawler periodically. The function is triggered by a CloudWatch Event Rule that specifies the desired frequency. 59 | 60 | The `loadbalancer.tf` file defines an Application Load Balancer (ALB) that routes traffic to the ECS tasks. Route 53 is used to create a custom domain name and an SSL certificate, as specified in the `route53.tf` file. 61 | 62 | ## Git Submodules 63 | 64 | This project consists of two Git submodules: 65 | 66 | 1. [fragmenty-api](https://github.com/Maders/fragmenty-api.git) - This submodule contains the source code for the API server, which is built using the Play framework and Scala. The `fragmenty-api` directory contains a Dockerfile for building the container image, configuration files, and the application's source code. 67 | 68 | 2. [fragmenty-spider](https://github.com/Maders/fragmenty-spider.git) - This submodule contains the source code for the Scrapy crawler that extracts data from Telegram's Fragment platform. The `fragmenty-spider` directory contains a Dockerfile for building the container image, a build script, a sample environment file, and the Scrapy spider's source code. 69 | 70 | These submodules are automatically checked out when the main repository is cloned with the `--recurse-submodules` option: 71 | 72 | ```sh 73 | git clone --recurse-submodules https://github.com/Maders/fragmenty.git 74 | ``` 75 | 76 | ## Useful Commands 77 | 78 | To initialize the working directory, run the following command in the respective directories: 79 | 80 | ```sh 81 | terraform init 82 | ``` 83 | 84 | To apply the infrastructure changes, run the following command in the respective directories: 85 | 86 | ```sh 87 | terraform apply 88 | ``` 89 | 90 | To destroy the infrastructure resources, run the following command in the respective directories: 91 | 92 | ```sh 93 | terraform destroy 94 | ``` 95 | -------------------------------------------------------------------------------- /fragmenty-infra/.gitignore: -------------------------------------------------------------------------------- 1 | terraform.tfvars 2 | 3 | # Local .terraform directories 4 | **/.terraform/* 5 | 6 | # Terraform override files 7 | override.tf 8 | override.tf.json 9 | *_override.tf 10 | *_override.tf.json 11 | 12 | # Terraform state files 13 | *.tfstate 14 | *.tfstate.backup 15 | 16 | # Terraform plan output files 17 | *.tfplan 18 | 19 | # Terraform crash log files 20 | crash.log 21 | 22 | # Custom sensitive files 23 | *.tfvars 24 | secrets.auto.tfvars 25 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "4.61.0" 6 | constraints = "~> 4.0" 7 | hashes = [ 8 | "h1:mJSchOA6VkYwEsi+tuspadRmyyE+FGZGYJFUt5kHV+M=", 9 | "h1:qyBawxoNN6EpiiX5h5ZG5P2dHsBeA5Z67xESl2c1HRk=", 10 | "zh:051e2588410b7448a5c4c30d668948dd6fdfa8037700bfc00fb228986ccbf3a5", 11 | "zh:082fbcf9706b48d0880ba552a11c29527e228dadd6d83668d0789abda24e5922", 12 | "zh:0e0e72f214fb24f4f9c601cab088a2d8e00ec3327c451bc753911951d773214a", 13 | "zh:3af6d38ca733ca66cce15c6a5735ded7c18348ad26040ebd9a59778b2cd9cf6c", 14 | "zh:404898bc2258bbb9527fa06c72cb927ca011fd9bc3f4b90931c0912652c3f9e9", 15 | "zh:4f617653b0f17a7708bc896f029c4ab0b677a1a1c987bd77166acad1d82db469", 16 | "zh:5dbe393355ac137aa3fd329e3d24871f27012d3ba93d714485b55820df240349", 17 | "zh:6067c2127eb5c879227aca671f101de6dcba909d0d8d15d5711480351962a248", 18 | "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", 19 | "zh:a939f94461f91aa3b7ec7096271e2714309bd917fe9a03e02f68afb556d65e0f", 20 | "zh:b21227b9082e5fafe8b7c415dc6a99c0d82da05492457377a5fe7d4acaed80e2", 21 | "zh:b8d9f09ed5fc8c654b768b7bee1237eaf1e2287c898249e740695055fb0fe072", 22 | "zh:d360e1e185b148ff6b1d0ed4f7d574e08f2391697ab43df62085b04a1a5b1284", 23 | "zh:da962da17ddda744911cb1e92b983fa3874d73a28f3ee72faa9ddb6680a63774", 24 | "zh:e2f1c4f5ebeb4fd7ef690178168a4c529025b54a91bb7a087dcea48e0b82737a", 25 | ] 26 | } 27 | 28 | provider "registry.terraform.io/hashicorp/external" { 29 | version = "2.3.1" 30 | hashes = [ 31 | "h1:bROCw6g5D/3fFnWeJ01L4IrdnJl1ILU8DGDgXCtYzaY=", 32 | "h1:gznGscVJ0USxy4CdihpjRKPsKvyGr/zqPvBoFLJTQDc=", 33 | "zh:001e2886dc81fc98cf17cf34c0d53cb2dae1e869464792576e11b0f34ee92f54", 34 | "zh:2eeac58dd75b1abdf91945ac4284c9ccb2bfb17fa9bdb5f5d408148ff553b3ee", 35 | "zh:2fc39079ba61411a737df2908942e6970cb67ed2f4fb19090cd44ce2082903dd", 36 | "zh:472a71c624952cff7aa98a7b967f6c7bb53153dbd2b8f356ceb286e6743bb4e2", 37 | "zh:4cff06d31272aac8bc35e9b7faec42cf4554cbcbae1092eaab6ab7f643c215d9", 38 | "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", 39 | "zh:7ed16ccd2049fa089616b98c0bd57219f407958f318f3c697843e2397ddf70df", 40 | "zh:842696362c92bf2645eb85c739410fd51376be6c488733efae44f4ce688da50e", 41 | "zh:8985129f2eccfd7f1841ce06f3bf2bbede6352ec9e9f926fbaa6b1a05313b326", 42 | "zh:a5f0602d8ec991a5411ef42f872aa90f6347e93886ce67905c53cfea37278e05", 43 | "zh:bf4ab82cbe5256dcef16949973bf6aa1a98c2c73a98d6a44ee7bc40809d002b8", 44 | "zh:e70770be62aa70198fa899526d671643ff99eecf265bf1a50e798fc3480bd417", 45 | ] 46 | } 47 | 48 | provider "registry.terraform.io/hashicorp/null" { 49 | version = "3.2.1" 50 | hashes = [ 51 | "h1:FbGfc+muBsC17Ohy5g806iuI1hQc4SIexpYCrQHQd8w=", 52 | "h1:ydA0/SNRVB1o95btfshvYsmxA+jZFRZcvKzZSB+4S1M=", 53 | "zh:58ed64389620cc7b82f01332e27723856422820cfd302e304b5f6c3436fb9840", 54 | "zh:62a5cc82c3b2ddef7ef3a6f2fedb7b9b3deff4ab7b414938b08e51d6e8be87cb", 55 | "zh:63cff4de03af983175a7e37e52d4bd89d990be256b16b5c7f919aff5ad485aa5", 56 | "zh:74cb22c6700e48486b7cabefa10b33b801dfcab56f1a6ac9b6624531f3d36ea3", 57 | "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", 58 | "zh:79e553aff77f1cfa9012a2218b8238dd672ea5e1b2924775ac9ac24d2a75c238", 59 | "zh:a1e06ddda0b5ac48f7e7c7d59e1ab5a4073bbcf876c73c0299e4610ed53859dc", 60 | "zh:c37a97090f1a82222925d45d84483b2aa702ef7ab66532af6cbcfb567818b970", 61 | "zh:e4453fbebf90c53ca3323a92e7ca0f9961427d2f0ce0d2b65523cc04d5d999c2", 62 | "zh:e80a746921946d8b6761e77305b752ad188da60688cfd2059322875d363be5f5", 63 | "zh:fbdb892d9822ed0e4cb60f2fedbdbb556e4da0d88d3b942ae963ed6ff091e48f", 64 | "zh:fca01a623d90d0cad0843102f9b8b9fe0d3ff8244593bd817f126582b52dd694", 65 | ] 66 | } 67 | 68 | provider "registry.terraform.io/hashicorp/random" { 69 | version = "3.4.3" 70 | hashes = [ 71 | "h1:saZR+mhthL0OZl4SyHXZraxyaBNVMxiZzks78nWcZ2o=", 72 | "h1:xZGZf18JjMS06pFa4NErzANI98qi59SEcBsOcS2P2yQ=", 73 | "zh:41c53ba47085d8261590990f8633c8906696fa0a3c4b384ff6a7ecbf84339752", 74 | "zh:59d98081c4475f2ad77d881c4412c5129c56214892f490adf11c7e7a5a47de9b", 75 | "zh:686ad1ee40b812b9e016317e7f34c0d63ef837e084dea4a1f578f64a6314ad53", 76 | "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", 77 | "zh:84103eae7251384c0d995f5a257c72b0096605048f757b749b7b62107a5dccb3", 78 | "zh:8ee974b110adb78c7cd18aae82b2729e5124d8f115d484215fd5199451053de5", 79 | "zh:9dd4561e3c847e45de603f17fa0c01ae14cae8c4b7b4e6423c9ef3904b308dda", 80 | "zh:bb07bb3c2c0296beba0beec629ebc6474c70732387477a65966483b5efabdbc6", 81 | "zh:e891339e96c9e5a888727b45b2e1bb3fcbdfe0fd7c5b4396e4695459b38c8cb1", 82 | "zh:ea4739860c24dfeaac6c100b2a2e357106a89d18751f7693f3c31ecf6a996f8d", 83 | "zh:f0c76ac303fd0ab59146c39bc121c5d7d86f878e9a69294e29444d4c653786f8", 84 | "zh:f143a9a5af42b38fed328a161279906759ff39ac428ebcfe55606e05e1518b93", 85 | ] 86 | } 87 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/backend.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_providers { 3 | aws = { 4 | source = "hashicorp/aws" 5 | version = "~> 4.0" 6 | } 7 | } 8 | 9 | backend "s3" { 10 | bucket = "terraform-state-bucket-fragmenty" 11 | key = "terraform-state/terraform.tfstate" 12 | dynamodb_table = "terraform-state-lock-fragmenty" 13 | region = "eu-central-1" 14 | profile = "terraform" 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/container_build_push.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | ecr_repository_url = aws_ecr_repository.scrapy_lambda_repository.repository_url 3 | api_hash = data.external.git_hash_api.result["sha_commit"] 4 | spider_hash = data.external.git_hash_spider.result["sha_commit"] 5 | } 6 | 7 | resource "null_resource" "push_spider_image" { 8 | depends_on = [null_resource.push_play_image] 9 | triggers = { 10 | always_run = local.spider_hash 11 | } 12 | 13 | # if you want to build images in terraform apply you should clone this repo with --recursive flag to have fragmenty-spider and fragmenty-api suubmodule contents 14 | # if you don't so comment the docker build lines (for example in that case that CI/CD made the images and pushed to the ECR repo) 15 | provisioner "local-exec" { 16 | command = <> /etc/ecs/ecs.config 81 | EOF 82 | } 83 | 84 | 85 | resource "random_string" "play-app_secret" { 86 | length = 25 87 | special = true 88 | } 89 | 90 | resource "aws_ecs_task_definition" "fragmenty" { 91 | depends_on = [null_resource.push_play_image] 92 | 93 | family = "fragmenty-task" 94 | container_definitions = jsonencode([ 95 | { 96 | "name" : "fragmenty-scala-play-service", 97 | "image" : "${aws_ecr_repository.scrapy_lambda_repository.repository_url}:${var.play_container_image_name}", 98 | "portMappings" : [ 99 | { 100 | "containerPort" : 9000, 101 | "hostPort" : 80, 102 | "protocol" : "tcp" 103 | } 104 | ], 105 | "memory" : 800, 106 | "essential" : true, 107 | "environment" : [ 108 | { 109 | "name" : "APPLICATION_SECRET", 110 | # the secret should not repreduce every time randomly but I use the random string for now 111 | "value" : "${random_string.play-app_secret.result}" 112 | }, 113 | { 114 | "name" : "MONGODB_URI", 115 | "value" : "${var.mongo_uri}" 116 | }, 117 | { 118 | "name" : "ALLOWED_HOSTS", 119 | "value" : "${var.api_allowed_hosts}" 120 | } 121 | 122 | ], 123 | "logConfiguration" : { 124 | "logDriver" : "awslogs", 125 | "options" : { 126 | "awslogs-group" : "/ecs/fragmenty-scala-play-app", 127 | "awslogs-region" : "${var.aws_region}", 128 | "awslogs-stream-prefix" : "ecs" 129 | } 130 | } 131 | } 132 | ] 133 | ) 134 | } 135 | 136 | resource "aws_ecs_service" "fragmenty" { 137 | depends_on = [aws_ecs_task_definition.fragmenty, aws_lb_target_group.fragmenty] 138 | name = "fragmenty-scala-play-service" 139 | cluster = aws_ecs_cluster.fragmenty.id 140 | task_definition = aws_ecs_task_definition.fragmenty.arn 141 | desired_count = 1 142 | launch_type = "EC2" 143 | 144 | load_balancer { 145 | target_group_arn = aws_lb_target_group.fragmenty.arn 146 | container_name = "fragmenty-scala-play-service" 147 | container_port = 9000 148 | } 149 | } 150 | 151 | resource "aws_cloudwatch_log_group" "fragmenty" { 152 | name = "/ecs/fragmenty-scala-play-app" 153 | retention_in_days = 14 154 | 155 | } 156 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/external.tf: -------------------------------------------------------------------------------- 1 | data "external" "git_hash_spider" { 2 | program = ["bash", "${path.module}/get_sha_module.sh"] 3 | 4 | query = { 5 | module = "spider" 6 | } 7 | } 8 | 9 | data "external" "git_hash_api" { 10 | program = ["bash", "${path.module}/get_sha_module.sh"] 11 | 12 | query = { 13 | module = "api" 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/get_sha_module.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if ! command -v jq &> /dev/null; then 4 | echo "Error: jq is not installed. Please install jq and try again." 5 | exit 1 6 | fi 7 | 8 | if ! command -v git &> /dev/null; then 9 | echo "Error: git is not installed. Please install git and try again." 10 | exit 1 11 | fi 12 | 13 | # Read input JSON from Terraform 14 | input=$(cat) 15 | # Extract the "module" value from the input JSON 16 | input_module=$(echo "$input" | jq -r '.module') 17 | if [ -z "$input_module" ]; then 18 | echo "Error: No module provided in the input JSON." 19 | exit 1 20 | fi 21 | 22 | # Find the matching submodule and store the result 23 | git submodule status | grep $input_module | while read -r line; do 24 | sha=$(echo $line | cut -d' ' -f1) 25 | module_path=$(echo $line | cut -d' ' -f2) 26 | module_name=$(basename $module_path) 27 | jq -n --arg module_name "sha_commit" --arg sha "$sha" '{($module_name): $sha}' 28 | break 29 | done 30 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/lambda.tf: -------------------------------------------------------------------------------- 1 | resource "aws_iam_role" "lambda_exec_role" { 2 | name = "ScrapyLambdaExecutionRole" 3 | 4 | assume_role_policy = jsonencode({ 5 | Version = "2012-10-17" 6 | Statement = [ 7 | { 8 | Action = "sts:AssumeRole" 9 | Effect = "Allow" 10 | Principal = { 11 | Service = "lambda.amazonaws.com" 12 | } 13 | } 14 | ] 15 | }) 16 | } 17 | 18 | resource "aws_iam_role_policy" "lambda_exec_policy" { 19 | name = "ScrapyLambdaExecutionPolicy" 20 | role = aws_iam_role.lambda_exec_role.id 21 | 22 | policy = jsonencode({ 23 | Version = "2012-10-17" 24 | Statement = [ 25 | { 26 | Action = [ 27 | "logs:CreateLogGroup", 28 | "logs:CreateLogStream", 29 | "logs:PutLogEvents" 30 | ] 31 | Effect = "Allow" 32 | Resource = "arn:aws:logs:*:*:*" 33 | } 34 | ] 35 | }) 36 | } 37 | 38 | # -----------------------------------------------------------------------| 39 | # The following comments blocks are sample config for create s3 bucket - 40 | # to upload your zip version of the lambda deployment package. also in - 41 | # aws_lambda_function block there are 4 line comments to specify lambda - 42 | # through s3 bucket instead of container. 43 | 44 | 45 | # resource "aws_s3_bucket" "example_bucket" { 46 | # bucket = "lambda-fn-fragmenty" 47 | # } 48 | 49 | # resource "aws_s3_bucket_acl" "example_bucket" { 50 | # bucket = aws_s3_bucket.example_bucket.id 51 | # acl = "private" 52 | # } 53 | 54 | # resource "aws_s3_object" "lambda_deployment_package" { 55 | # bucket = aws_s3_bucket.example_bucket.id 56 | # key = "your_zip_file_key" 57 | # source = "lambda_package.zip" 58 | # etag = filemd5("lambda_package.zip") 59 | # acl = "private" 60 | # content_type = "application/zip" 61 | # } 62 | # -----------------------------------------------------------------------| 63 | 64 | resource "aws_lambda_function" "scrapy_lambda" { 65 | function_name = "ScrapyLambdaFunction" 66 | # if you want to deploy lamnbda by zip file the following line are config sample 67 | # s3_bucket = aws_s3_object.lambda_deployment_package.bucket 68 | # s3_key = aws_s3_object.lambda_deployment_package.key 69 | # handler = "fragmenty.lambda_handler.lambda_handler" 70 | # runtime = "python3.9" 71 | role = aws_iam_role.lambda_exec_role.arn 72 | 73 | package_type = "Image" 74 | image_uri = "${aws_ecr_repository.scrapy_lambda_repository.repository_url}:${var.lambda_container_image_name}" 75 | 76 | depends_on = [null_resource.push_spider_image] 77 | 78 | timeout = 120 # Adjust the timeout based on your project's requirements 79 | memory_size = 256 # Adjust the memory size based on your project's requirements 80 | 81 | environment { 82 | variables = { 83 | MONGO_URI = var.mongo_uri 84 | } 85 | } 86 | } 87 | 88 | resource "aws_cloudwatch_event_rule" "scrapy_lambda_schedule" { 89 | name = "ScrapyLambdaSchedule" 90 | description = "Trigger Scrapy Lambda function on a schedule" 91 | schedule_expression = "rate(30 minutes)" # Adjust the schedule based on your desired frequency 92 | } 93 | 94 | resource "aws_cloudwatch_log_group" "scrapy_lambda_schedule" { 95 | name = "/aws/lambda/ScrapyLambdaFunction" 96 | retention_in_days = 7 # Change this value to the desired number of days 97 | } 98 | 99 | 100 | resource "aws_cloudwatch_event_target" "scrapy_lambda_schedule_target" { 101 | rule = aws_cloudwatch_event_rule.scrapy_lambda_schedule.name 102 | target_id = "ScrapyLambda" 103 | arn = aws_lambda_function.scrapy_lambda.arn 104 | } 105 | 106 | resource "aws_lambda_permission" "allow_cloudwatch_events" { 107 | statement_id = "AllowCloudWatchEvents" 108 | action = "lambda:InvokeFunction" 109 | function_name = aws_lambda_function.scrapy_lambda.function_name 110 | principal = "events.amazonaws.com" 111 | source_arn = aws_cloudwatch_event_rule.scrapy_lambda_schedule.arn 112 | } 113 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/loadbalancer.tf: -------------------------------------------------------------------------------- 1 | data "aws_elb_service_account" "main" {} 2 | 3 | resource "aws_s3_bucket" "lb_access_logs" { 4 | bucket = "fragmenty-lb-access-log" 5 | } 6 | 7 | resource "aws_s3_bucket_acl" "lb_access_log_acl" { 8 | bucket = aws_s3_bucket.lb_access_logs.id 9 | acl = "private" 10 | } 11 | 12 | resource "aws_s3_bucket_policy" "lb_access_logs_policy" { 13 | bucket = aws_s3_bucket.lb_access_logs.id 14 | 15 | policy = jsonencode({ 16 | Version = "2012-10-17" 17 | Statement = [ 18 | { 19 | Action = [ 20 | "s3:PutObject", 21 | "s3:PutObjectAcl", 22 | "s3:PutObjectVersionAcl" 23 | ] 24 | Effect = "Allow" 25 | Resource = "${aws_s3_bucket.lb_access_logs.arn}/*" 26 | Principal = { 27 | AWS = "arn:aws:iam::${data.aws_elb_service_account.main.id}:root" 28 | } 29 | }, 30 | { 31 | Action = "s3:GetBucketAcl" 32 | Effect = "Allow" 33 | Resource = aws_s3_bucket.lb_access_logs.arn 34 | Principal = { 35 | AWS = "arn:aws:iam::${data.aws_elb_service_account.main.id}:root" 36 | } 37 | } 38 | ] 39 | }) 40 | } 41 | 42 | 43 | resource "aws_security_group" "allow_web" { 44 | name = "allow_web" 45 | description = "Allow inbound traffic on port 80" 46 | vpc_id = var.vpc_id 47 | 48 | ingress { 49 | from_port = 80 50 | to_port = 80 51 | protocol = "tcp" 52 | cidr_blocks = ["0.0.0.0/0"] 53 | } 54 | } 55 | 56 | resource "aws_security_group" "allow_https" { 57 | name = "allow_https" 58 | description = "Allow inbound traffic on port 443" 59 | vpc_id = var.vpc_id 60 | 61 | ingress { 62 | from_port = 443 63 | to_port = 443 64 | protocol = "tcp" 65 | cidr_blocks = ["0.0.0.0/0"] 66 | } 67 | } 68 | 69 | 70 | resource "aws_lb" "fragmenty" { 71 | name = "fragmenty-lb" 72 | internal = false 73 | load_balancer_type = "application" 74 | security_groups = [ 75 | aws_security_group.allow_web.id, 76 | aws_security_group.allow_https.id, 77 | var.default_sg 78 | ] 79 | subnets = var.subnets 80 | 81 | depends_on = [aws_s3_bucket.lb_access_logs] 82 | access_logs { 83 | bucket = aws_s3_bucket.lb_access_logs.bucket 84 | prefix = "load-balancer-logs" 85 | enabled = true 86 | } 87 | } 88 | 89 | resource "aws_lb_target_group" "fragmenty" { 90 | depends_on = [aws_lb.fragmenty] 91 | name = "fragmenty-tg" 92 | port = 80 93 | protocol = "HTTP" 94 | vpc_id = var.vpc_id 95 | 96 | health_check { 97 | path = "/" 98 | healthy_threshold = 5 99 | unhealthy_threshold = 2 100 | } 101 | } 102 | 103 | resource "aws_lb_listener" "fragmenty" { 104 | load_balancer_arn = aws_lb.fragmenty.arn 105 | port = 80 106 | protocol = "HTTP" 107 | 108 | default_action { 109 | type = "forward" 110 | target_group_arn = aws_lb_target_group.fragmenty.arn 111 | } 112 | } 113 | 114 | resource "aws_lb_listener_rule" "redirect_http_to_https" { 115 | listener_arn = aws_lb_listener.fragmenty.arn 116 | priority = 1 117 | 118 | action { 119 | type = "redirect" 120 | redirect { 121 | port = "443" 122 | protocol = "HTTPS" 123 | status_code = "HTTP_301" 124 | } 125 | } 126 | 127 | condition { 128 | path_pattern { 129 | values = ["/*"] 130 | } 131 | } 132 | } 133 | 134 | resource "aws_lb_listener" "https" { 135 | load_balancer_arn = aws_lb.fragmenty.arn 136 | port = 443 137 | protocol = "HTTPS" 138 | ssl_policy = "ELBSecurityPolicy-2016-08" 139 | certificate_arn = aws_acm_certificate_validation.cert.certificate_arn 140 | 141 | default_action { 142 | type = "forward" 143 | target_group_arn = aws_lb_target_group.fragmenty.arn 144 | } 145 | } 146 | 147 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/outputs.tf: -------------------------------------------------------------------------------- 1 | output "play_app_secret" { 2 | value = random_string.play-app_secret.result 3 | } 4 | 5 | output "spider_sha" { 6 | value = data.external.git_hash_spider.result["sha_commit"] 7 | } 8 | 9 | output "api_sha" { 10 | value = data.external.git_hash_api.result["sha_commit"] 11 | } 12 | 13 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/provider.tf: -------------------------------------------------------------------------------- 1 | provider "aws" { 2 | profile = var.aws_profile 3 | region = var.aws_region 4 | } 5 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/route53.tf: -------------------------------------------------------------------------------- 1 | resource "aws_route53_zone" "fragmenty" { 2 | name = var.custom_domain 3 | } 4 | 5 | resource "aws_route53_record" "fragmenty" { 6 | zone_id = aws_route53_zone.fragmenty.id 7 | name = var.custom_domain 8 | type = "A" 9 | 10 | alias { 11 | name = aws_lb.fragmenty.dns_name 12 | zone_id = aws_lb.fragmenty.zone_id 13 | evaluate_target_health = true 14 | } 15 | } 16 | 17 | resource "aws_acm_certificate" "cert" { 18 | domain_name = var.custom_domain 19 | validation_method = "DNS" 20 | 21 | tags = { 22 | Terraform = "true" 23 | } 24 | 25 | lifecycle { 26 | create_before_destroy = true 27 | } 28 | } 29 | 30 | resource "aws_route53_record" "cert_validation" { 31 | depends_on = [aws_route53_zone.fragmenty] 32 | 33 | name = tolist(aws_acm_certificate.cert.domain_validation_options)[0].resource_record_name 34 | type = tolist(aws_acm_certificate.cert.domain_validation_options)[0].resource_record_type 35 | zone_id = aws_route53_zone.fragmenty.id 36 | ttl = 60 37 | records = [tolist(aws_acm_certificate.cert.domain_validation_options)[0].resource_record_value] 38 | 39 | } 40 | 41 | resource "aws_acm_certificate_validation" "cert" { 42 | certificate_arn = aws_acm_certificate.cert.arn 43 | validation_record_fqdns = [aws_route53_record.cert_validation.fqdn] 44 | } 45 | 46 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-infra/variables.tf: -------------------------------------------------------------------------------- 1 | variable "aws_profile" { 2 | description = "AWS profile to use for authentication" 3 | default = "terraform" 4 | } 5 | 6 | variable "subnets" { 7 | description = "A list of subnet IDs where the resources will be deployed" 8 | type = list(string) 9 | } 10 | 11 | variable "aws_region" { 12 | description = "AWS region to deploy the resources" 13 | default = "eu-central-1" 14 | } 15 | 16 | variable "vpc_id" { 17 | description = "The VPC ID where resources will be deployed" 18 | } 19 | 20 | variable "custom_domain" { 21 | description = "The custom domain name for the application" 22 | } 23 | 24 | variable "s3_bucket_name" { 25 | description = "The s3 bucket name used for remote state" 26 | sensitive = true 27 | } 28 | 29 | variable "dynamodb_table_name" { 30 | description = "The DynamoDB table name used for locking remote state" 31 | sensitive = true 32 | } 33 | 34 | variable "mongo_uri" { 35 | description = "The MongoDB atlas connection string" 36 | sensitive = true 37 | } 38 | 39 | variable "lambda_container_image_name" { 40 | description = "The name of the application docker image that you build locally" 41 | default = "spider" 42 | } 43 | 44 | variable "play_container_image_name" { 45 | description = "The name of the application docker image that you build locally" 46 | default = "api-0.0.3" 47 | } 48 | 49 | variable "api_allowed_hosts" { 50 | description = "The play framework allowed hosts" 51 | default = "." 52 | } 53 | 54 | variable "default_sg" { 55 | description = "The default security group for your vpc" 56 | } 57 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-remote-state/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "4.61.0" 6 | hashes = [ 7 | "h1:qyBawxoNN6EpiiX5h5ZG5P2dHsBeA5Z67xESl2c1HRk=", 8 | "zh:051e2588410b7448a5c4c30d668948dd6fdfa8037700bfc00fb228986ccbf3a5", 9 | "zh:082fbcf9706b48d0880ba552a11c29527e228dadd6d83668d0789abda24e5922", 10 | "zh:0e0e72f214fb24f4f9c601cab088a2d8e00ec3327c451bc753911951d773214a", 11 | "zh:3af6d38ca733ca66cce15c6a5735ded7c18348ad26040ebd9a59778b2cd9cf6c", 12 | "zh:404898bc2258bbb9527fa06c72cb927ca011fd9bc3f4b90931c0912652c3f9e9", 13 | "zh:4f617653b0f17a7708bc896f029c4ab0b677a1a1c987bd77166acad1d82db469", 14 | "zh:5dbe393355ac137aa3fd329e3d24871f27012d3ba93d714485b55820df240349", 15 | "zh:6067c2127eb5c879227aca671f101de6dcba909d0d8d15d5711480351962a248", 16 | "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", 17 | "zh:a939f94461f91aa3b7ec7096271e2714309bd917fe9a03e02f68afb556d65e0f", 18 | "zh:b21227b9082e5fafe8b7c415dc6a99c0d82da05492457377a5fe7d4acaed80e2", 19 | "zh:b8d9f09ed5fc8c654b768b7bee1237eaf1e2287c898249e740695055fb0fe072", 20 | "zh:d360e1e185b148ff6b1d0ed4f7d574e08f2391697ab43df62085b04a1a5b1284", 21 | "zh:da962da17ddda744911cb1e92b983fa3874d73a28f3ee72faa9ddb6680a63774", 22 | "zh:e2f1c4f5ebeb4fd7ef690178168a4c529025b54a91bb7a087dcea48e0b82737a", 23 | ] 24 | } 25 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-remote-state/outputs.tf: -------------------------------------------------------------------------------- 1 | output "s3_bucket_name" { 2 | value = aws_s3_bucket.terraform_state.bucket 3 | } 4 | 5 | output "dynamodb_table_name" { 6 | value = aws_dynamodb_table.terraform_state_lock.name 7 | } 8 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-remote-state/remote_state.tf: -------------------------------------------------------------------------------- 1 | provider "aws" { 2 | profile = var.aws_profile 3 | region = var.aws_region 4 | } 5 | 6 | resource "aws_s3_bucket" "terraform_state" { 7 | bucket = var.s3_bucket_name 8 | 9 | lifecycle { 10 | prevent_destroy = true 11 | } 12 | } 13 | 14 | resource "aws_s3_bucket_acl" "terraform_state" { 15 | bucket = aws_s3_bucket.terraform_state.id 16 | acl = "private" 17 | } 18 | 19 | 20 | resource "aws_s3_bucket_versioning" "terraform_state" { 21 | bucket = aws_s3_bucket.terraform_state.id 22 | versioning_configuration { 23 | status = "Enabled" 24 | } 25 | } 26 | 27 | resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" { 28 | bucket = aws_s3_bucket.terraform_state.id 29 | 30 | rule { 31 | apply_server_side_encryption_by_default { 32 | sse_algorithm = "AES256" 33 | } 34 | } 35 | } 36 | 37 | 38 | resource "aws_dynamodb_table" "terraform_state_lock" { 39 | name = var.dynamodb_table_name 40 | billing_mode = "PAY_PER_REQUEST" 41 | hash_key = "LockID" 42 | attribute { 43 | name = "LockID" 44 | type = "S" 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /fragmenty-infra/ECS/ecs-remote-state/variables.tf: -------------------------------------------------------------------------------- 1 | variable "aws_profile" { 2 | description = "AWS profile to use for authentication" 3 | default = "terraform" 4 | } 5 | 6 | variable "aws_region" { 7 | description = "AWS region to deploy the resources" 8 | default = "eu-central-1" 9 | } 10 | 11 | variable "s3_bucket_name" { 12 | description = "The s3 bucket name that you want to create" 13 | default = "terraform-state-bucket-fragmenty" 14 | } 15 | 16 | variable "dynamodb_table_name" { 17 | description = "The s3 bucket name that you want to create" 18 | default = "terraform-state-lock-fragmenty" 19 | } 20 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-infra/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/mongodb/mongodbatlas" { 5 | version = "1.8.2" 6 | constraints = "~> 1.8.0" 7 | hashes = [ 8 | "h1:njw68OlwLID3z60kkezyABNds5er0kJciNNsmxhw7/Y=", 9 | "zh:12beb93beb5e4f05cc0b77fc2492b349bceeb742e4f97ff7a1e50f77b69ac2a3", 10 | "zh:1511a8f093b082a4f71ab0b2f6d6877fda1e57918155802503b9dd5f9bfebdf9", 11 | "zh:20754df5e83b940f07558568f39a51f306f0ae92f2cf3848cdecea77ab3016d1", 12 | "zh:254f24f0dd5e2016348b5e968a52f19d3768d897ee1e181dc4c086955da2997e", 13 | "zh:27d288696f09c55fa0a9e7d0517a76bf736a36734862a31163805b8ecbdc5935", 14 | "zh:38d4fcc04f9fb4f7b0f333e04f00587036d76b6dc8b8a4acac327d8c1d61ad0d", 15 | "zh:9d3e3be0ee2094350e143c83a2bdffefa767dd51ea6b980db8e708f56e7bd814", 16 | "zh:abc2c6005ffb61f48471a4a93bbcfb38ed4f3d9f8b1b2f99ec53a7f5b9919108", 17 | "zh:b970e4bc9d4dfe170a697353e9bcc41cd112d500529e19781b50a49738d81b2e", 18 | "zh:bc2bbf73dfe3f8aac57d5b361ad0626544c670efa65e574522d667bb5083f39d", 19 | "zh:cd24f3d6b1a46561438d166f9f00a709006b4d618c79a30f7ca2e3459e3777bd", 20 | "zh:d50aa5cb9d4b2ecb6887529bebbd188a6bee6f3c75204f383a80508d2fae5bd4", 21 | "zh:eb931ce779b2b013583e6d0c31658528d2babfd835eb83cb67c7def7c9f47e46", 22 | "zh:fb204907819ad65bb9b295efd7ae36d013ff3a633d57dc96c58d025ecdd38136", 23 | ] 24 | } 25 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-infra/backend.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_providers { 3 | mongodbatlas = { 4 | source = "mongodb/mongodbatlas" 5 | version = "~> 1.8.0" 6 | } 7 | } 8 | 9 | backend "s3" { 10 | bucket = "terraform-state-bucket-mongodb" 11 | key = "terraform-state/terraform.tfstate" 12 | dynamodb_table = "terraform-state-lock-mongodb" 13 | region = "eu-central-1" 14 | profile = "terraform" 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-infra/main.tf: -------------------------------------------------------------------------------- 1 | provider "mongodbatlas" { 2 | public_key = var.mongodb_atlas_public_key 3 | private_key = var.mongodb_atlas_private_key 4 | } 5 | 6 | resource "mongodbatlas_project" "fragmenty" { 7 | name = "fragmenty" 8 | org_id = var.mongodb_atlas_organs_id 9 | } 10 | 11 | 12 | # https://www.mongodb.com/docs/atlas/reference/amazon-aws/#amazon-web-services--aws- 13 | # https://www.mongodb.com/docs/atlas/reference/free-shared-limitations/#service-m0--free-cluster---m2--and-m5-limitations 14 | resource "mongodbatlas_cluster" "my_cluster" { 15 | project_id = mongodbatlas_project.fragmenty.id 16 | name = "my-atlas-cluster" 17 | 18 | // Provider Settings 19 | # https://github.com/mongodb/terraform-provider-mongodbatlas/issues/64 20 | provider_name = "TENANT" // FOR AWS Free Tier Cluster 21 | backing_provider_name = "AWS" 22 | provider_region_name = "EU_CENTRAL_1" 23 | provider_instance_size_name = "M0" // Free Tier 24 | } 25 | 26 | resource "mongodbatlas_database_user" "my_database_user" { 27 | username = var.database_username 28 | password = var.database_password 29 | project_id = mongodbatlas_project.fragmenty.id 30 | auth_database_name = "admin" 31 | # delete_after_date = "2023-04-30T12:00:00Z" // Set the date when the user should be deleted. 32 | roles { 33 | role_name = "readWrite" 34 | database_name = var.database_name 35 | } 36 | } 37 | 38 | # https://www.mongodb.com/docs/atlas/security-vpc-peering 39 | resource "mongodbatlas_project_ip_access_list" "everywhere" { 40 | project_id = mongodbatlas_project.fragmenty.id 41 | cidr_block = "0.0.0.0/0" 42 | comment = "accessing from everywhere" 43 | } 44 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-infra/outputs.tf: -------------------------------------------------------------------------------- 1 | output "mongo_connection_string" { 2 | value = mongodbatlas_cluster.my_cluster.connection_strings[0].standard_srv 3 | # .my_cluster.connection_strings.standard_srv 4 | sensitive = true 5 | } 6 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-infra/variables.tf: -------------------------------------------------------------------------------- 1 | variable "mongodb_atlas_public_key" { 2 | description = "The MongoDB Atlas public API key" 3 | } 4 | 5 | variable "mongodb_atlas_private_key" { 6 | description = "The MongoDB Atlas private API key" 7 | sensitive = true 8 | } 9 | 10 | variable "mongodb_atlas_organs_id" { 11 | description = "The MongoDB Atlas Organization Id" 12 | } 13 | 14 | variable "database_username" { 15 | description = "The created database username" 16 | sensitive = true 17 | } 18 | 19 | variable "database_password" { 20 | description = "The created database password" 21 | sensitive = true 22 | } 23 | 24 | variable "database_name" { 25 | description = "The cteated database name" 26 | default = "fragmenty" 27 | } 28 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-remote-state/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/aws" { 5 | version = "4.61.0" 6 | hashes = [ 7 | "h1:qyBawxoNN6EpiiX5h5ZG5P2dHsBeA5Z67xESl2c1HRk=", 8 | "zh:051e2588410b7448a5c4c30d668948dd6fdfa8037700bfc00fb228986ccbf3a5", 9 | "zh:082fbcf9706b48d0880ba552a11c29527e228dadd6d83668d0789abda24e5922", 10 | "zh:0e0e72f214fb24f4f9c601cab088a2d8e00ec3327c451bc753911951d773214a", 11 | "zh:3af6d38ca733ca66cce15c6a5735ded7c18348ad26040ebd9a59778b2cd9cf6c", 12 | "zh:404898bc2258bbb9527fa06c72cb927ca011fd9bc3f4b90931c0912652c3f9e9", 13 | "zh:4f617653b0f17a7708bc896f029c4ab0b677a1a1c987bd77166acad1d82db469", 14 | "zh:5dbe393355ac137aa3fd329e3d24871f27012d3ba93d714485b55820df240349", 15 | "zh:6067c2127eb5c879227aca671f101de6dcba909d0d8d15d5711480351962a248", 16 | "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", 17 | "zh:a939f94461f91aa3b7ec7096271e2714309bd917fe9a03e02f68afb556d65e0f", 18 | "zh:b21227b9082e5fafe8b7c415dc6a99c0d82da05492457377a5fe7d4acaed80e2", 19 | "zh:b8d9f09ed5fc8c654b768b7bee1237eaf1e2287c898249e740695055fb0fe072", 20 | "zh:d360e1e185b148ff6b1d0ed4f7d574e08f2391697ab43df62085b04a1a5b1284", 21 | "zh:da962da17ddda744911cb1e92b983fa3874d73a28f3ee72faa9ddb6680a63774", 22 | "zh:e2f1c4f5ebeb4fd7ef690178168a4c529025b54a91bb7a087dcea48e0b82737a", 23 | ] 24 | } 25 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-remote-state/outputs.tf: -------------------------------------------------------------------------------- 1 | output "s3_bucket_name" { 2 | value = aws_s3_bucket.terraform_state.bucket 3 | } 4 | 5 | output "dynamodb_table_name" { 6 | value = aws_dynamodb_table.terraform_state_lock.name 7 | } 8 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-remote-state/remote_state.tf: -------------------------------------------------------------------------------- 1 | provider "aws" { 2 | profile = var.aws_profile 3 | region = var.aws_region 4 | } 5 | 6 | resource "aws_s3_bucket" "terraform_state" { 7 | bucket = var.s3_bucket_name 8 | 9 | lifecycle { 10 | prevent_destroy = true 11 | } 12 | } 13 | 14 | resource "aws_s3_bucket_acl" "terraform_state" { 15 | bucket = aws_s3_bucket.terraform_state.id 16 | acl = "private" 17 | } 18 | 19 | 20 | resource "aws_s3_bucket_versioning" "terraform_state" { 21 | bucket = aws_s3_bucket.terraform_state.id 22 | versioning_configuration { 23 | status = "Enabled" 24 | } 25 | } 26 | 27 | resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" { 28 | bucket = aws_s3_bucket.terraform_state.id 29 | 30 | rule { 31 | apply_server_side_encryption_by_default { 32 | sse_algorithm = "AES256" 33 | } 34 | } 35 | } 36 | 37 | 38 | resource "aws_dynamodb_table" "terraform_state_lock" { 39 | name = var.dynamodb_table_name 40 | billing_mode = "PAY_PER_REQUEST" 41 | hash_key = "LockID" 42 | attribute { 43 | name = "LockID" 44 | type = "S" 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /fragmenty-infra/MongoDBAtlas/mongo-remote-state/variables.tf: -------------------------------------------------------------------------------- 1 | variable "aws_profile" { 2 | description = "AWS profile to use for authentication" 3 | default = "terraform" 4 | } 5 | 6 | variable "aws_region" { 7 | description = "AWS region to deploy the resources" 8 | default = "eu-central-1" 9 | } 10 | 11 | variable "s3_bucket_name" { 12 | description = "The s3 bucket name that you want to create" 13 | default = "terraform-state-bucket-mongodb" 14 | } 15 | 16 | variable "dynamodb_table_name" { 17 | description = "The s3 bucket name that you want to create" 18 | default = "terraform-state-lock-mongodb" 19 | } 20 | --------------------------------------------------------------------------------