├── .gitignore ├── Dockerfile ├── README.md ├── cloudformation ├── README.md ├── fargate.private.yaml ├── fargate.yaml ├── retool-workflows.ec2.yaml ├── retool-workflows.fargate.yaml └── retool.yaml ├── compose.yaml ├── install.sh ├── kubernetes-with-temporal ├── retool-code-executor-container.yaml ├── retool-container.yaml ├── retool-jobs-runner.yaml ├── retool-postgres.yaml ├── retool-secrets.template.yaml ├── retool-workflows-backend-container.yaml ├── retool-workflows-worker-container.yaml └── temporal │ ├── LICENSE │ ├── retool-temporal-secrets.template.yaml │ ├── temporal-configmaps.yaml │ ├── temporal-debugging-containers.yaml │ ├── temporal-frontend-container.yaml │ ├── temporal-history-container.yaml │ ├── temporal-matching-container.yaml │ └── temporal-worker-container.yaml ├── kubernetes ├── retool-code-executor-container.yaml ├── retool-container.yaml ├── retool-jobs-runner.yaml ├── retool-postgres.yaml ├── retool-secrets.template.yaml ├── retool-workflows-backend-container.yaml └── retool-workflows-worker-container.yaml ├── temporal.yaml └── upgrade.sh /.gitignore: -------------------------------------------------------------------------------- 1 | # Config file generated by ./docker_setup 2 | docker.env 3 | retooldb.env 4 | 5 | # Config file that might be created when using the retool-secrets.template.yaml 6 | retool-secrets.yaml 7 | 8 | helm/charts 9 | .idea/ 10 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # Check Dockerhub for available tags: https://hub.docker.com/r/tryretool/backend/tags 2 | 3 | ARG VERSION=X.Y.Z-stable 4 | 5 | FROM tryretool/code-executor-service:${VERSION} AS code-executor 6 | 7 | FROM tryretool/backend:${VERSION} 8 | 9 | CMD ./docker_scripts/start_api.sh 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

2 | 3 | 4 | Retool Logo 5 | 6 |

7 |

The best way to build internal software

8 | 9 |
10 | 11 | Below are the instructions for deploying with Docker Compose, see [our docs](https://docs.retool.com/docs/deploy-guide-overview) for more specific details for [AWS](https://docs.retool.com/docs/deploy-with-aws-ec2), [GCP](https://docs.retool.com/docs/deploy-with-gcp), or [Azure](https://docs.retool.com/docs/deploy-with-azure-vm), as well as for deploying with [Helm](https://docs.retool.com/docs/deploy-with-helm), [Kubernetes](https://docs.retool.com/docs/deploy-with-kubernetes), or [ECS](https://docs.retool.com/docs/deploy-with-ecs-fargate). Check out our [Community Forums](https://community.retool.com/) if you have questions or issues, and see our [deprecated-onpremise repo](https://github.com/tryretool/deprecated-onpremise) if you need to reference legacy deployment instructions. 12 | 13 |
14 | 15 | Deploy with Docker Compose 16 | ------ 17 | 18 | [Install](#install) → [Configure](#configure) → [Run](#run) → [Upgrade](#upgrade) 19 | 20 |
21 | 22 | Install 23 | ------ 24 | > [!IMPORTANT] 25 | > We test and support running on Ubuntu. If on a different platform, you may need to manually install requirements like Docker. 26 | 27 | 1. Download this repo 28 | 29 | ``` 30 | git clone https://github.com/tryretool/retool-onpremise retool && cd retool 31 | ``` 32 | 33 | 2. Run our install script to attempt to set up Docker and initialize the `.env` files 34 | 35 | ``` 36 | ./install.sh 37 | ``` 38 | The script will create `docker.env` and `retooldb.env` if successful, else it should call out potential issues to address before rerunning. 39 | 40 | > [!WARNING] 41 | > We now assume Compose v2 is installed as a plugin accessed through `docker compose`, we no longer use the legacy v1 `docker-compose` syntax. You may need to use the latter based on your OS and installation, see [Docker's docs](https://docs.docker.com/compose/releases/migrate/) for more context on the migration. 42 | 43 |
44 | 45 | Configure 46 | ------ 47 | > [!TIP] 48 | > Optionally run `sudo usermod -aG docker $USER` and log out/back in to not require `sudo` for every Docker command moving forward. Not required, but we'll assume this in the guide 49 | 50 | 1. Check the generated `.env` files to make sure the license key and randomized keys were set as expected during the installation. 51 | 52 | 2. Save off the `ENCRYPTION_KEY` value, since this is needed to encrypt/decrypt values saved into the Postgres database the Retool instance runs on. 53 | 54 | 3. Replace `X.Y.Z-stable` in `Dockerfile` with the desired Retool version listed in our [Dockerhub repo](https://hub.docker.com/r/tryretool/backend/tags), we recommend the latest patch of the most recent [stable version](https://hub.docker.com/r/tryretool/backend/tags?name=stable). 55 | 56 | 4. To set up HTTPS, you'll need your domain pointing to your server's IP address. If that's in place, make sure `DOMAINS` is correct in `docker.env`, and then set `STAGE=production` in `compose.yaml` for the `https-portal` container to attempt to get and use a free `Let's Encrypt` cert for your domain on startup. 57 | 58 | > [!WARNING] 59 | > You must set `COOKIE_INSECURE=true` in `docker.env` to allow logging into Retool without HTTPS configured (not recommended) 60 | 61 | 5. By default, the deployment will include a Temporal container for Workflows. If you have an Enterprise license and would like to instead use Retool's managed Temporal cluster, comment out the `include` block in `compose.yaml` and the `WORKFLOW_TEMPORAL_...` environment variables in `docker.env`. Check out [our docs](https://docs.retool.com/self-hosted/concepts/temporal) for more information on Temporal deployment options. 62 | 63 |
64 | 65 | Run 66 | ------ 67 | 68 | 1. Bring up containers 69 | 70 | ``` 71 | docker compose up -d 72 | ``` 73 | 74 | 2. Check your container statuses after a few minutes 75 | 76 | ``` 77 | docker compose ps 78 | ``` 79 | 80 | 3. Check your container logs if any container isn't up and running 81 | 82 | ``` 83 | docker compose logs 84 | ``` 85 | 86 | 4. Go to your domain or IP in a browser and click `Sign up` to initialize and log into the new instance 87 | 88 |
89 | 90 | Upgrade 91 | ------ 92 | 93 | Set the new version in `Dockerfile`, and either run `./upgrade.sh` or follow the below steps: 94 | 95 | 1. Download and build the new images 96 | 97 | ``` 98 | docker compose build 99 | ``` 100 | 101 | 2. Bring up the new containers to replace the old ones 102 | 103 | ``` 104 | docker compose up -d 105 | ``` 106 | 107 | 3. Remove the old images from the system 108 | ``` 109 | docker image prune -a -f 110 | ``` 111 | 112 |
113 | -------------------------------------------------------------------------------- /cloudformation/README.md: -------------------------------------------------------------------------------- 1 | # Cloudformation file explained 2 | 3 | ``` 4 | 5 | Parameters: 6 | 7 | # 'staging' --> sent to cloudwatch logs 8 | Environment 9 | 10 | # select 2 public subnets for the load balancer 11 | SubnetId 12 | 13 | # the ECS cluster(this should have already been created) 14 | Cluster 15 | 16 | # the docker image (tryretool/backend:X.Y.Z, where X.Y.Z is a specific Retool version number. To get help choosing a version number, see [Retool Release Versions](https://docs.retool.com/docs/updating-retool-on-premise#retool-release-versions).) 17 | Image 18 | 19 | # default number of tasks to run 20 | DesiredCount 21 | 22 | # maximum number of tasks that are allowed to be RUNNING or PENDING during an update 23 | MaximumPercent 24 | 25 | # lower limit number of tasks that must remain RUNNING during update 26 | MinimumHealthyPercent 27 | 28 | # the VPC in which to run Retool 29 | VpcId 30 | 31 | # Used to force the deployment even when the image and parameters are otherwised unchanged 32 | Force 33 | 34 | Resources: 35 | 36 | # creates a security group for the load balancer 37 | ALBSecurityGroup 38 | 39 | # creates an inbound rule for ALBSecurityGroup listening on port 80 40 | GlobalHttpInbound 41 | 42 | # creates Cloudwatch logs for monitoring the Retool container 43 | CloudwatchLogsGroup 44 | 45 | # creates a security group for the Retool DB 46 | RDSSecurityGroup 47 | 48 | # creates an inbound rule for RDSSecurityGroup listening on port 5432 49 | RetoolECSPostgresInbound 50 | 51 | # ECS Tasks are basically blueprints for how to run a containerized app (e.g. which docker image to use, which ports to open, any necessary volumes, environment variables, etc.) 52 | # specify the Retool Task with the container tryretool/backend, the cloudwatch logs group specified above, the necessary env variables, the port mapping 80:3000, and the command to start the retool server 53 | RetoolTask 54 | 55 | # retool environment variable 56 | RetoolJWTSecret 57 | 58 | # retool environment variable 59 | RetoolEncryptionKeySecret 60 | 61 | # retool environment variable 62 | RetoolRDSSecret 63 | 64 | # The retool database 65 | RetoolRDSInstance 66 | 67 | # create an internet-facing load balancer to sit in front of the retool server 68 | # requires 2 public subnets in 2 different availability zones 69 | ECSALB 70 | 71 | # a listener for the load balancer, listening on port 80 72 | ALBListener 73 | 74 | # add a listener rule for the load balancer, forwarding requests to a specific target group 75 | ECSALBListenerRule 76 | 77 | # the target group that the load balancer forwards requests to; receives traffic on port 80 78 | ECSTG 79 | 80 | # an ECS service can run multiple ECS tasks and makes sure the correct number of tasks are always running 81 | # runs RetoolTask the number of times specified in DesiredCount 82 | RetoolECSservice 83 | 84 | # create an IAM role for the ECS Service 85 | RetoolServiceRole 86 | 87 | # create an IAM role for the ECS Task 88 | RetoolTaskRole 89 | 90 | # create the Application Load Balancer DNS URL by which to access retool in a browser 91 | Outputs 92 | ``` 93 | -------------------------------------------------------------------------------- /cloudformation/fargate.private.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | Environment: 4 | Type: String 5 | Description: Environment string sent back to plogger with the logs 6 | LoadBalancerSubnetId: 7 | Type: List 8 | Description: Select at least two public subnets in your selected VPC (this is for the load balancer) 9 | SubnetId: 10 | Type: List 11 | Description: Select at least two private subnets in your selected VPC (this is for the ECS Tasks) 12 | Cluster: 13 | Type: String 14 | Description: Cluster to put service in. 15 | Image: 16 | Type: String 17 | Description: Image to use in the service. 18 | DesiredCount: 19 | Type: Number 20 | Description: Default number of tasks to run 21 | MaximumPercent: 22 | Type: Number 23 | Description: Maximum percentage of tasks to run during a deployment 24 | Default: 150 25 | MinimumHealthyPercent: 26 | Type: Number 27 | Default: 50 28 | Description: Maximum percentage of tasks to run during a deployment 29 | VpcId: 30 | Type: AWS::EC2::VPC::Id 31 | Description: Select a VPC that allows instances access to the Internet. 32 | Force: 33 | Type: String 34 | Description: "Used to force the deployment even when the image and parameters are otherwised unchanged." 35 | Default: "false" 36 | 37 | Resources: 38 | ALBSecurityGroup: 39 | Type: AWS::EC2::SecurityGroup 40 | Properties: 41 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'load balancer security group']] 42 | VpcId: !Ref 'VpcId' 43 | 44 | GlobalHttpInbound: 45 | Type: AWS::EC2::SecurityGroupIngress 46 | Properties: 47 | GroupId: !Ref 'ALBSecurityGroup' 48 | IpProtocol: tcp 49 | FromPort: '3000' 50 | ToPort: '3000' 51 | CidrIp: '0.0.0.0/0' 52 | 53 | CloudwatchLogsGroup: 54 | Type: AWS::Logs::LogGroup 55 | Properties: 56 | LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']] 57 | RetentionInDays: 14 58 | 59 | RDSSecurityGroup: 60 | Type: AWS::EC2::SecurityGroup 61 | Properties: 62 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'database security group']] 63 | VpcId: !Ref 'VpcId' 64 | 65 | RetoolECSPostgresInbound: 66 | Type: AWS::EC2::SecurityGroupIngress 67 | Properties: 68 | GroupId: !GetAtt [RDSSecurityGroup, GroupId] 69 | IpProtocol: tcp 70 | FromPort: '5432' 71 | ToPort: '5432' 72 | CidrIp: '0.0.0.0/0' 73 | 74 | RetoolTask: 75 | Type: AWS::ECS::TaskDefinition 76 | Properties: 77 | NetworkMode: awsvpc 78 | Cpu: '1024' 79 | Memory: '4096' 80 | Family: 'retool' 81 | TaskRoleArn: !Ref 'RetoolTaskRole' 82 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 83 | RequiresCompatibilities: 84 | - FARGATE 85 | ContainerDefinitions: 86 | - Name: 'retool' 87 | Essential: 'true' 88 | Image: !Ref 'Image' 89 | LogConfiguration: 90 | LogDriver: awslogs 91 | Options: 92 | awslogs-group: !Ref 'CloudwatchLogsGroup' 93 | awslogs-region: !Ref 'AWS::Region' 94 | awslogs-stream-prefix: "SERVICE_RETOOL" 95 | Environment: 96 | - Name: DEPLOYMENT_TEMPLATE_TYPE 97 | Value: "aws-ecs-fargate" 98 | - Name: NODE_ENV 99 | Value: production 100 | - Name: SERVICE_TYPE 101 | Value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 102 | - Name: "FORCE_DEPLOYMENT" 103 | Value: !Ref "Force" 104 | - Name: POSTGRES_DB 105 | Value: hammerhead_production 106 | - Name: POSTGRES_HOST 107 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 108 | - Name: POSTGRES_SSL_ENABLED 109 | Value: "true" 110 | - Name: POSTGRES_PORT 111 | Value: "5432" 112 | - Name: POSTGRES_USER 113 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 114 | - Name: POSTGRES_PASSWORD 115 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 116 | - Name: JWT_SECRET 117 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 118 | - Name: ENCRYPTION_KEY 119 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 120 | - Name: LICENSE_KEY 121 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 122 | 123 | # Remove below when serving Retool over https 124 | - Name: COOKIE_INSECURE 125 | Value: "true" 126 | PortMappings: 127 | - ContainerPort: '3000' 128 | # HostPort: '80' 129 | Command: ["./docker_scripts/start_api.sh"] 130 | 131 | RetoolJobsRunnerTask: 132 | Type: AWS::ECS::TaskDefinition 133 | Properties: 134 | NetworkMode: awsvpc 135 | Cpu: '2048' 136 | Memory: '4096' 137 | Family: 'retool' 138 | TaskRoleArn: !Ref 'RetoolTaskRole' 139 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 140 | RequiresCompatibilities: 141 | - FARGATE 142 | ContainerDefinitions: 143 | - Name: 'retool-jobs-runner' 144 | Essential: 'true' 145 | Image: !Ref 'Image' 146 | LogConfiguration: 147 | LogDriver: awslogs 148 | Options: 149 | awslogs-group: !Ref 'CloudwatchLogsGroup' 150 | awslogs-region: !Ref 'AWS::Region' 151 | awslogs-stream-prefix: "SERVICE_RETOOL" 152 | Environment: 153 | - Name: DEPLOYMENT_TEMPLATE_TYPE 154 | Value: "aws-ecs-fargate" 155 | - Name: NODE_ENV 156 | Value: production 157 | - Name: SERVICE_TYPE 158 | Value: JOBS_RUNNER 159 | - Name: "FORCE_DEPLOYMENT" 160 | Value: !Ref "Force" 161 | - Name: POSTGRES_DB 162 | Value: hammerhead_production 163 | - Name: POSTGRES_HOST 164 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 165 | - Name: POSTGRES_SSL_ENABLED 166 | Value: "true" 167 | - Name: POSTGRES_PORT 168 | Value: "5432" 169 | - Name: POSTGRES_USER 170 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 171 | - Name: POSTGRES_PASSWORD 172 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 173 | - Name: JWT_SECRET 174 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 175 | - Name: ENCRYPTION_KEY 176 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 177 | - Name: LICENSE_KEY 178 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 179 | Command: ["./docker_scripts/start_api.sh"] 180 | 181 | RetoolJWTSecret: 182 | Type: AWS::SecretsManager::Secret 183 | Properties: 184 | Description: 'This is the secret for Retool JWTs' 185 | GenerateSecretString: 186 | SecretStringTemplate: '{}' 187 | GenerateStringKey: 'password' 188 | PasswordLength: 16 189 | ExcludeCharacters: '"@/\' 190 | 191 | RetoolEncryptionKeySecret: 192 | Type: AWS::SecretsManager::Secret 193 | Properties: 194 | Description: 'This is the secret for encrypting credentials' 195 | GenerateSecretString: 196 | SecretStringTemplate: '{}' 197 | GenerateStringKey: 'password' 198 | PasswordLength: 16 199 | ExcludeCharacters: '"@/\' 200 | 201 | RetoolRDSSecret: 202 | Type: AWS::SecretsManager::Secret 203 | Properties: 204 | Description: 'This is the secret for the Retool RDS instance' 205 | GenerateSecretString: 206 | SecretStringTemplate: '{"username": "retool"}' 207 | GenerateStringKey: 'password' 208 | PasswordLength: 16 209 | ExcludeCharacters: '"@/\' 210 | 211 | 212 | RetoolRDSInstance: 213 | Type: AWS::RDS::DBInstance 214 | Properties: 215 | AllocatedStorage: "80" 216 | DBInstanceClass: "db.m5.large" 217 | Engine: postgres 218 | EngineVersion: "13.11" 219 | DBName: "hammerhead_production" 220 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 221 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 222 | Port: "5432" 223 | VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 224 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 225 | 226 | RDSSubnetGroup: 227 | Type: AWS::RDS::DBSubnetGroup 228 | Properties: 229 | DBSubnetGroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'rds subnet security group']] 230 | SubnetIds: !Ref 'SubnetId' 231 | 232 | ECSALB: 233 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 234 | Properties: 235 | Name: !Join ['-', [!Ref 'AWS::StackName', 'lb']] 236 | Scheme: "internet-facing" 237 | LoadBalancerAttributes: 238 | - Key: idle_timeout.timeout_seconds 239 | Value: '60' 240 | Subnets: !Ref 'LoadBalancerSubnetId' 241 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 242 | 243 | ALBListener: 244 | Type: AWS::ElasticLoadBalancingV2::Listener 245 | DependsOn: RetoolServiceRole 246 | Properties: 247 | DefaultActions: 248 | - Type: forward 249 | TargetGroupArn: !Ref 'ECSTG' 250 | LoadBalancerArn: !Ref 'ECSALB' 251 | Port: '3000' 252 | Protocol: HTTP 253 | 254 | ECSALBListenerRule: 255 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 256 | DependsOn: ALBListener 257 | Properties: 258 | Actions: 259 | - Type: forward 260 | TargetGroupArn: !Ref 'ECSTG' 261 | Conditions: 262 | - Field: path-pattern 263 | Values: [/] 264 | ListenerArn: !Ref 'ALBListener' 265 | Priority: 1 266 | 267 | ECSTG: 268 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 269 | DependsOn: ECSALB 270 | Properties: 271 | TargetType: ip 272 | HealthCheckIntervalSeconds: 61 273 | HealthCheckPath: '/api/checkHealth' 274 | HealthCheckProtocol: HTTP 275 | HealthCheckTimeoutSeconds: 60 276 | HealthyThresholdCount: 4 277 | Name: !Join ['-', [!Ref 'AWS::StackName', 'tg']] 278 | Port: '3000' 279 | Protocol: HTTP 280 | UnhealthyThresholdCount: 10 281 | VpcId: !Ref 'VpcId' 282 | TargetGroupAttributes: 283 | - Key: deregistration_delay.timeout_seconds 284 | Value: '30' 285 | 286 | RetoolECSservice: 287 | Type: AWS::ECS::Service 288 | DependsOn: ALBListener 289 | Properties: 290 | NetworkConfiguration: 291 | AwsvpcConfiguration: 292 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 293 | Subnets: !Ref 'SubnetId' 294 | Cluster: !Ref 'Cluster' 295 | DesiredCount: !Ref 'DesiredCount' 296 | LaunchType: FARGATE 297 | DeploymentConfiguration: 298 | MaximumPercent: !Ref 'MaximumPercent' 299 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 300 | LoadBalancers: 301 | - ContainerName: 'retool' 302 | ContainerPort: '3000' 303 | TargetGroupArn: !Ref 'ECSTG' 304 | # Role: !Ref 'RetoolServiceRole' 305 | TaskDefinition: !Ref 'RetoolTask' 306 | 307 | RetoolJobsRunnerECSservice: 308 | Type: AWS::ECS::Service 309 | Properties: 310 | NetworkConfiguration: 311 | AwsvpcConfiguration: 312 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 313 | Subnets: !Ref 'SubnetId' 314 | Cluster: !Ref 'Cluster' 315 | LaunchType: FARGATE 316 | DesiredCount: 1 317 | TaskDefinition: !Ref 'RetoolJobsRunnerTask' 318 | 319 | RetoolServiceRole: 320 | Type: AWS::IAM::Role 321 | Properties: 322 | AssumeRolePolicyDocument: 323 | Statement: 324 | - Effect: Allow 325 | Principal: 326 | Service: [ecs.amazonaws.com] 327 | Action: ['sts:AssumeRole'] 328 | Path: / 329 | Policies: 330 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'service-policy']] 331 | PolicyDocument: 332 | Statement: 333 | - Effect: Allow 334 | Action: [ 335 | 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 336 | 'elasticloadbalancing:DeregisterTargets', 337 | 'elasticloadbalancing:Describe*', 338 | 'elasticloadbalancing:RegisterInstancesWithLoadBalancer', 339 | 'elasticloadbalancing:RegisterTargets', 340 | 'ec2:Describe*', 341 | 'ec2:AuthorizeSecurityGroupIngress'] 342 | Resource: '*' 343 | 344 | RetoolTaskRole: 345 | Type: AWS::IAM::Role 346 | Properties: 347 | AssumeRolePolicyDocument: 348 | Statement: 349 | - Effect: Allow 350 | Principal: 351 | Service: ['ecs-tasks.amazonaws.com'] 352 | Action: ['sts:AssumeRole'] 353 | Path: / 354 | Policies: [] 355 | 356 | RetoolExecutionRole: 357 | Type: AWS::IAM::Role 358 | Properties: 359 | AssumeRolePolicyDocument: 360 | Statement: 361 | - Effect: Allow 362 | Principal: 363 | Service: ['ecs-tasks.amazonaws.com'] 364 | Action: ['sts:AssumeRole'] 365 | Path: / 366 | Policies: 367 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'execution-policy']] 368 | PolicyDocument: 369 | Statement: 370 | - Effect: Allow 371 | Action: [ 372 | "ecr:GetAuthorizationToken", 373 | "ecr:BatchCheckLayerAvailability", 374 | "ecr:GetDownloadUrlForLayer", 375 | "ecr:BatchGetImage", 376 | "logs:CreateLogStream", 377 | "logs:PutLogEvents"] 378 | Resource: '*' 379 | 380 | Outputs: 381 | ECSALB: 382 | Description: Your ALB DNS URL 383 | Value: !GetAtt [ECSALB, DNSName] 384 | -------------------------------------------------------------------------------- /cloudformation/fargate.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | Environment: 4 | Type: String 5 | Description: Environment string sent back to plogger with the logs 6 | SubnetId: 7 | Type: List 8 | Description: Select at two subnets in your selected VPC. 9 | Cluster: 10 | Type: String 11 | Description: Cluster to put service in. 12 | Image: 13 | Type: String 14 | Description: Image to use in the service. 15 | DesiredCount: 16 | Type: Number 17 | Description: Default number of tasks to run 18 | MaximumPercent: 19 | Type: Number 20 | Description: Maximum percentage of tasks to run during a deployment 21 | Default: 150 22 | MinimumHealthyPercent: 23 | Type: Number 24 | Default: 50 25 | Description: Maximum percentage of tasks to run during a deployment 26 | VpcId: 27 | Type: AWS::EC2::VPC::Id 28 | Description: Select a VPC that allows instances access to the Internet. 29 | Force: 30 | Type: String 31 | Description: "Used to force the deployment even when the image and parameters are otherwised unchanged." 32 | Default: "false" 33 | 34 | Resources: 35 | ALBSecurityGroup: 36 | Type: AWS::EC2::SecurityGroup 37 | Properties: 38 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'load balancer security group']] 39 | VpcId: !Ref 'VpcId' 40 | 41 | GlobalHttpInbound: 42 | Type: AWS::EC2::SecurityGroupIngress 43 | Properties: 44 | GroupId: !Ref 'ALBSecurityGroup' 45 | IpProtocol: tcp 46 | FromPort: '3000' 47 | ToPort: '3000' 48 | CidrIp: '0.0.0.0/0' 49 | 50 | CloudwatchLogsGroup: 51 | Type: AWS::Logs::LogGroup 52 | Properties: 53 | LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']] 54 | RetentionInDays: 14 55 | 56 | RDSSecurityGroup: 57 | Type: AWS::EC2::SecurityGroup 58 | Properties: 59 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'database security group']] 60 | VpcId: !Ref 'VpcId' 61 | 62 | RetoolECSPostgresInbound: 63 | Type: AWS::EC2::SecurityGroupIngress 64 | Properties: 65 | GroupId: !GetAtt [RDSSecurityGroup, GroupId] 66 | IpProtocol: tcp 67 | FromPort: '5432' 68 | ToPort: '5432' 69 | CidrIp: '0.0.0.0/0' 70 | 71 | RetoolTask: 72 | Type: AWS::ECS::TaskDefinition 73 | Properties: 74 | NetworkMode: awsvpc 75 | Cpu: '1024' 76 | Memory: '4096' 77 | Family: 'retool' 78 | TaskRoleArn: !Ref 'RetoolTaskRole' 79 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 80 | RequiresCompatibilities: 81 | - FARGATE 82 | ContainerDefinitions: 83 | - Name: 'retool' 84 | Essential: 'true' 85 | Image: !Ref 'Image' 86 | LogConfiguration: 87 | LogDriver: awslogs 88 | Options: 89 | awslogs-group: !Ref 'CloudwatchLogsGroup' 90 | awslogs-region: !Ref 'AWS::Region' 91 | awslogs-stream-prefix: "SERVICE_RETOOL" 92 | Environment: 93 | - Name: DEPLOYMENT_TEMPLATE_TYPE 94 | Value: "aws-ecs-fargate" 95 | - Name: NODE_ENV 96 | Value: production 97 | - Name: SERVICE_TYPE 98 | Value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 99 | - Name: "FORCE_DEPLOYMENT" 100 | Value: !Ref "Force" 101 | - Name: POSTGRES_DB 102 | Value: hammerhead_production 103 | - Name: POSTGRES_HOST 104 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 105 | - Name: POSTGRES_SSL_ENABLED 106 | Value: "true" 107 | - Name: POSTGRES_PORT 108 | Value: "5432" 109 | - Name: POSTGRES_USER 110 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 111 | - Name: POSTGRES_PASSWORD 112 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 113 | - Name: JWT_SECRET 114 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 115 | - Name: ENCRYPTION_KEY 116 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 117 | - Name: LICENSE_KEY 118 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 119 | 120 | # Remove below when serving Retool over https 121 | - Name: COOKIE_INSECURE 122 | Value: "true" 123 | PortMappings: 124 | - ContainerPort: '3000' 125 | # HostPort: '80' 126 | Command: ["./docker_scripts/start_api.sh"] 127 | 128 | RetoolJobsRunnerTask: 129 | Type: AWS::ECS::TaskDefinition 130 | Properties: 131 | NetworkMode: awsvpc 132 | Cpu: '2048' 133 | Memory: '4096' 134 | Family: 'retool' 135 | TaskRoleArn: !Ref 'RetoolTaskRole' 136 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 137 | RequiresCompatibilities: 138 | - FARGATE 139 | ContainerDefinitions: 140 | - Name: 'retool-jobs-runner' 141 | Essential: 'true' 142 | Image: !Ref 'Image' 143 | LogConfiguration: 144 | LogDriver: awslogs 145 | Options: 146 | awslogs-group: !Ref 'CloudwatchLogsGroup' 147 | awslogs-region: !Ref 'AWS::Region' 148 | awslogs-stream-prefix: "SERVICE_RETOOL" 149 | Environment: 150 | - Name: DEPLOYMENT_TEMPLATE_TYPE 151 | Value: "aws-ecs-fargate" 152 | - Name: NODE_ENV 153 | Value: production 154 | - Name: SERVICE_TYPE 155 | Value: JOBS_RUNNER 156 | - Name: "FORCE_DEPLOYMENT" 157 | Value: !Ref "Force" 158 | - Name: POSTGRES_DB 159 | Value: hammerhead_production 160 | - Name: POSTGRES_HOST 161 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 162 | - Name: POSTGRES_SSL_ENABLED 163 | Value: "true" 164 | - Name: POSTGRES_PORT 165 | Value: "5432" 166 | - Name: POSTGRES_USER 167 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 168 | - Name: POSTGRES_PASSWORD 169 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 170 | - Name: JWT_SECRET 171 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 172 | - Name: ENCRYPTION_KEY 173 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 174 | - Name: LICENSE_KEY 175 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 176 | Command: ["./docker_scripts/start_api.sh"] 177 | 178 | RetoolJWTSecret: 179 | Type: AWS::SecretsManager::Secret 180 | Properties: 181 | Description: 'This is the secret for Retool JWTs' 182 | GenerateSecretString: 183 | SecretStringTemplate: '{}' 184 | GenerateStringKey: 'password' 185 | PasswordLength: 16 186 | ExcludeCharacters: '"@/\' 187 | 188 | RetoolEncryptionKeySecret: 189 | Type: AWS::SecretsManager::Secret 190 | Properties: 191 | Description: 'This is the secret for encrypting credentials' 192 | GenerateSecretString: 193 | SecretStringTemplate: '{}' 194 | GenerateStringKey: 'password' 195 | PasswordLength: 16 196 | ExcludeCharacters: '"@/\' 197 | 198 | RetoolRDSSecret: 199 | Type: AWS::SecretsManager::Secret 200 | Properties: 201 | Description: 'This is the secret for the Retool RDS instance' 202 | GenerateSecretString: 203 | SecretStringTemplate: '{"username": "retool"}' 204 | GenerateStringKey: 'password' 205 | PasswordLength: 16 206 | ExcludeCharacters: '"@/\' 207 | 208 | 209 | RetoolRDSInstance: 210 | Type: AWS::RDS::DBInstance 211 | Properties: 212 | AllocatedStorage: "80" 213 | DBInstanceClass: "db.m5.large" 214 | Engine: postgres 215 | EngineVersion: "13.11" 216 | DBName: "hammerhead_production" 217 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 218 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 219 | Port: "5432" 220 | VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 221 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 222 | 223 | RDSSubnetGroup: 224 | Type: AWS::RDS::DBSubnetGroup 225 | Properties: 226 | DBSubnetGroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'rds subnet security group']] 227 | SubnetIds: !Ref 'SubnetId' 228 | 229 | ECSALB: 230 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 231 | Properties: 232 | Name: !Join ['-', [!Ref 'AWS::StackName', 'lb']] 233 | Scheme: "internet-facing" 234 | LoadBalancerAttributes: 235 | - Key: idle_timeout.timeout_seconds 236 | Value: '60' 237 | Subnets: !Ref 'SubnetId' 238 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 239 | 240 | ALBListener: 241 | Type: AWS::ElasticLoadBalancingV2::Listener 242 | DependsOn: RetoolServiceRole 243 | Properties: 244 | DefaultActions: 245 | - Type: forward 246 | TargetGroupArn: !Ref 'ECSTG' 247 | LoadBalancerArn: !Ref 'ECSALB' 248 | Port: '3000' 249 | Protocol: HTTP 250 | 251 | ECSALBListenerRule: 252 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 253 | DependsOn: ALBListener 254 | Properties: 255 | Actions: 256 | - Type: forward 257 | TargetGroupArn: !Ref 'ECSTG' 258 | Conditions: 259 | - Field: path-pattern 260 | Values: [/] 261 | ListenerArn: !Ref 'ALBListener' 262 | Priority: 1 263 | 264 | ECSTG: 265 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 266 | DependsOn: ECSALB 267 | Properties: 268 | TargetType: ip 269 | HealthCheckIntervalSeconds: 61 270 | HealthCheckPath: '/api/checkHealth' 271 | HealthCheckProtocol: HTTP 272 | HealthCheckTimeoutSeconds: 60 273 | HealthyThresholdCount: 4 274 | Name: !Join ['-', [!Ref 'AWS::StackName', 'tg']] 275 | Port: '3000' 276 | Protocol: HTTP 277 | UnhealthyThresholdCount: 10 278 | VpcId: !Ref 'VpcId' 279 | TargetGroupAttributes: 280 | - Key: deregistration_delay.timeout_seconds 281 | Value: '30' 282 | 283 | RetoolECSservice: 284 | Type: AWS::ECS::Service 285 | DependsOn: ALBListener 286 | Properties: 287 | NetworkConfiguration: 288 | AwsvpcConfiguration: 289 | AssignPublicIp: ENABLED 290 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 291 | Subnets: !Ref 'SubnetId' 292 | Cluster: !Ref 'Cluster' 293 | DesiredCount: !Ref 'DesiredCount' 294 | LaunchType: FARGATE 295 | DeploymentConfiguration: 296 | MaximumPercent: !Ref 'MaximumPercent' 297 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 298 | LoadBalancers: 299 | - ContainerName: 'retool' 300 | ContainerPort: '3000' 301 | TargetGroupArn: !Ref 'ECSTG' 302 | # Role: !Ref 'RetoolServiceRole' 303 | TaskDefinition: !Ref 'RetoolTask' 304 | 305 | RetoolJobsRunnerECSservice: 306 | Type: AWS::ECS::Service 307 | Properties: 308 | NetworkConfiguration: 309 | AwsvpcConfiguration: 310 | AssignPublicIp: ENABLED 311 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 312 | Subnets: !Ref 'SubnetId' 313 | Cluster: !Ref 'Cluster' 314 | DesiredCount: 1 315 | LaunchType: FARGATE 316 | TaskDefinition: !Ref 'RetoolJobsRunnerTask' 317 | 318 | RetoolServiceRole: 319 | Type: AWS::IAM::Role 320 | Properties: 321 | AssumeRolePolicyDocument: 322 | Statement: 323 | - Effect: Allow 324 | Principal: 325 | Service: [ecs.amazonaws.com] 326 | Action: ['sts:AssumeRole'] 327 | Path: / 328 | Policies: 329 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'service-policy']] 330 | PolicyDocument: 331 | Statement: 332 | - Effect: Allow 333 | Action: [ 334 | 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 335 | 'elasticloadbalancing:DeregisterTargets', 336 | 'elasticloadbalancing:Describe*', 337 | 'elasticloadbalancing:RegisterInstancesWithLoadBalancer', 338 | 'elasticloadbalancing:RegisterTargets', 339 | 'ec2:Describe*', 340 | 'ec2:AuthorizeSecurityGroupIngress'] 341 | Resource: '*' 342 | 343 | RetoolTaskRole: 344 | Type: AWS::IAM::Role 345 | Properties: 346 | AssumeRolePolicyDocument: 347 | Statement: 348 | - Effect: Allow 349 | Principal: 350 | Service: ['ecs-tasks.amazonaws.com'] 351 | Action: ['sts:AssumeRole'] 352 | Path: / 353 | Policies: [] 354 | 355 | RetoolExecutionRole: 356 | Type: AWS::IAM::Role 357 | Properties: 358 | AssumeRolePolicyDocument: 359 | Statement: 360 | - Effect: Allow 361 | Principal: 362 | Service: ['ecs-tasks.amazonaws.com'] 363 | Action: ['sts:AssumeRole'] 364 | Path: / 365 | Policies: 366 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'execution-policy']] 367 | PolicyDocument: 368 | Statement: 369 | - Effect: Allow 370 | Action: [ 371 | "ecr:GetAuthorizationToken", 372 | "ecr:BatchCheckLayerAvailability", 373 | "ecr:GetDownloadUrlForLayer", 374 | "ecr:BatchGetImage", 375 | "logs:CreateLogStream", 376 | "logs:PutLogEvents"] 377 | Resource: '*' 378 | 379 | Outputs: 380 | ECSALB: 381 | Description: Your ALB DNS URL 382 | Value: !GetAtt [ECSALB, DNSName] 383 | -------------------------------------------------------------------------------- /cloudformation/retool-workflows.ec2.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | Environment: 4 | Type: String 5 | Description: Environment string sent back to plogger with the logs 6 | SubnetId: 7 | Type: List 8 | Description: Select at two private subnets in your selected VPC. 9 | ALBSubnetId: 10 | Type: List 11 | Description: Select the subnets for your Application Load Balancer. 12 | Cluster: 13 | Type: String 14 | Description: Cluster to put service in. 15 | RetoolVersion: 16 | Type: String 17 | Description: Retool version tag (e.g. 3.114.7-stable) 18 | Default: 3.114.7-stable 19 | DesiredCount: 20 | Type: Number 21 | Description: Default number of API container tasks to run 22 | Default: 1 23 | DesiredWorkflowsCount: 24 | Type: Number 25 | Description: Default number of tasks to run for Workflows Backend and Workflows Worker containers to run 26 | Default: 1 27 | DesiredCodeExecutorCount: 28 | Type: Number 29 | Description: Default number of tasks to run for Retool Code Executor containers 30 | Default: 1 31 | MaximumPercent: 32 | Type: Number 33 | Description: Maximum percentage of tasks to run during a deployment 34 | Default: 150 35 | MinimumHealthyPercent: 36 | Type: Number 37 | Default: 50 38 | Description: Maximum percentage of tasks to run during a deployment 39 | VpcId: 40 | Type: AWS::EC2::VPC::Id 41 | Description: Select a VPC that allows instances access to the Internet. 42 | Force: 43 | Type: String 44 | Description: "Used to force the deployment even when the image and parameters are otherwised unchanged." 45 | Default: "false" 46 | 47 | Resources: 48 | ALBSecurityGroup: 49 | Type: AWS::EC2::SecurityGroup 50 | Properties: 51 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'load balancer security group']] 52 | VpcId: !Ref 'VpcId' 53 | 54 | GlobalHttpInbound: 55 | Type: AWS::EC2::SecurityGroupIngress 56 | Properties: 57 | GroupId: !GetAtt [ALBSecurityGroup, GroupId] 58 | IpProtocol: tcp 59 | FromPort: '80' 60 | ToPort: '80' 61 | CidrIp: '0.0.0.0/0' 62 | 63 | RetoolSecurityGroup: 64 | Type: AWS::EC2::SecurityGroup 65 | Properties: 66 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'retool container security group']] 67 | VpcId: !Ref 'VpcId' 68 | 69 | ALBInbound: 70 | Type: AWS::EC2::SecurityGroupIngress 71 | Properties: 72 | GroupId: !GetAtt [RetoolSecurityGroup, GroupId] 73 | IpProtocol: tcp 74 | FromPort: '3000' 75 | ToPort: '3000' 76 | SourceSecurityGroupId: !GetAtt [ALBSecurityGroup, GroupId] 77 | 78 | RetoolSelfIngress: 79 | Type: AWS::EC2::SecurityGroupIngress 80 | Properties: 81 | GroupId: !GetAtt [RetoolSecurityGroup, GroupId] 82 | IpProtocol: -1 83 | SourceSecurityGroupId: !GetAtt [RetoolSecurityGroup, GroupId] 84 | 85 | TemporalSecurityGroup: 86 | Type: AWS::EC2::SecurityGroup 87 | Properties: 88 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'temporal security group']] 89 | SecurityGroupEgress: 90 | - CidrIp: 0.0.0.0/0 91 | Description: Allow all outbound traffic by default 92 | IpProtocol: "-1" 93 | VpcId: !Ref 'VpcId' 94 | 95 | TemporalFrontendIngress: 96 | Type: AWS::EC2::SecurityGroupIngress 97 | Properties: 98 | GroupId: !GetAtt [TemporalSecurityGroup, GroupId] 99 | IpProtocol: tcp 100 | FromPort: 7233 101 | ToPort: 7233 102 | SourceSecurityGroupId: !GetAtt [RetoolSecurityGroup, GroupId] 103 | 104 | TemporalSelfIngress: 105 | Type: AWS::EC2::SecurityGroupIngress 106 | Properties: 107 | GroupId: !GetAtt [TemporalSecurityGroup, GroupId] 108 | IpProtocol: -1 109 | SourceSecurityGroupId: !GetAtt [TemporalSecurityGroup, GroupId] 110 | 111 | 112 | CloudwatchLogsGroup: 113 | Type: AWS::Logs::LogGroup 114 | Properties: 115 | LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']] 116 | RetentionInDays: 14 117 | 118 | RDSSecurityGroup: 119 | Type: AWS::EC2::SecurityGroup 120 | Properties: 121 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'database security group']] 122 | VpcId: !Ref 'VpcId' 123 | 124 | RetoolECSPostgresInbound: 125 | Type: AWS::EC2::SecurityGroupIngress 126 | Properties: 127 | GroupId: !GetAtt [RDSSecurityGroup, GroupId] 128 | IpProtocol: tcp 129 | FromPort: '5432' 130 | ToPort: '5432' 131 | CidrIp: '0.0.0.0/0' 132 | 133 | RetoolTask: 134 | Type: AWS::ECS::TaskDefinition 135 | Properties: 136 | Family: 'retool' 137 | TaskRoleArn: !Ref 'RetoolTaskRole' 138 | NetworkMode: awsvpc 139 | ContainerDefinitions: 140 | - Name: 'retool' 141 | Cpu: '2048' 142 | Memory: '4096' 143 | Essential: 'true' 144 | Image: !Sub tryretool/backend:${RetoolVersion} 145 | LogConfiguration: 146 | LogDriver: awslogs 147 | Options: 148 | awslogs-group: !Ref 'CloudwatchLogsGroup' 149 | awslogs-region: !Ref 'AWS::Region' 150 | awslogs-stream-prefix: "SERVICE_RETOOL" 151 | Environment: 152 | - Name: DEPLOYMENT_TEMPLATE_TYPE 153 | Value: "aws-ecs-ec2" 154 | - Name: NODE_ENV 155 | Value: production 156 | - Name: SERVICE_TYPE 157 | Value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 158 | - Name: "FORCE_DEPLOYMENT" 159 | Value: !Ref "Force" 160 | - Name: POSTGRES_DB 161 | Value: hammerhead_production 162 | - Name: POSTGRES_HOST 163 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 164 | - Name: POSTGRES_SSL_ENABLED 165 | Value: "true" 166 | - Name: POSTGRES_PORT 167 | Value: "5432" 168 | - Name: POSTGRES_USER 169 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 170 | - Name: POSTGRES_PASSWORD 171 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 172 | - Name: JWT_SECRET 173 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 174 | - Name: ENCRYPTION_KEY 175 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 176 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 177 | Value: temporal.retoolsvc 178 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 179 | Value: "7233" 180 | - Name: WORKFLOW_BACKEND_HOST 181 | Value: http://workflows-backend.retoolsvc:3000 182 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 183 | Value: http://code-executor.retoolsvc:3004 184 | - Name: LICENSE_KEY 185 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 186 | # Remove below when serving Retool over https 187 | - Name: COOKIE_INSECURE 188 | Value: "true" 189 | PortMappings: 190 | - ContainerPort: '3000' 191 | # HostPort: '80' 192 | Command: ["./docker_scripts/start_api.sh"] 193 | 194 | RetoolJobsRunnerTask: 195 | Type: AWS::ECS::TaskDefinition 196 | Properties: 197 | Family: 'retool-jobs-runner' 198 | TaskRoleArn: !Ref 'RetoolTaskRole' 199 | NetworkMode: awsvpc 200 | ContainerDefinitions: 201 | - Name: 'retool-jobs-runner' 202 | Cpu: '1024' 203 | Memory: '2048' 204 | Essential: 'true' 205 | Image: !Sub tryretool/backend:${RetoolVersion} 206 | LogConfiguration: 207 | LogDriver: awslogs 208 | Options: 209 | awslogs-group: !Ref 'CloudwatchLogsGroup' 210 | awslogs-region: !Ref 'AWS::Region' 211 | awslogs-stream-prefix: "SERVICE_RETOOL" 212 | Environment: 213 | - Name: DEPLOYMENT_TEMPLATE_TYPE 214 | Value: "aws-ecs-ec2" 215 | - Name: NODE_ENV 216 | Value: production 217 | - Name: SERVICE_TYPE 218 | Value: JOBS_RUNNER 219 | - Name: "FORCE_DEPLOYMENT" 220 | Value: !Ref "Force" 221 | - Name: POSTGRES_DB 222 | Value: hammerhead_production 223 | - Name: POSTGRES_HOST 224 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 225 | - Name: POSTGRES_SSL_ENABLED 226 | Value: "true" 227 | - Name: POSTGRES_PORT 228 | Value: "5432" 229 | - Name: POSTGRES_USER 230 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 231 | - Name: POSTGRES_PASSWORD 232 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 233 | - Name: JWT_SECRET 234 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 235 | - Name: ENCRYPTION_KEY 236 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 237 | - Name: LICENSE_KEY 238 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 239 | Command: ["./docker_scripts/start_api.sh"] 240 | 241 | RetoolWorkflowsBackendTask: 242 | Type: AWS::ECS::TaskDefinition 243 | Properties: 244 | Family: 'retool-workflows-backend' 245 | TaskRoleArn: !Ref 'RetoolTaskRole' 246 | NetworkMode: awsvpc 247 | ContainerDefinitions: 248 | - Name: 'retool-workflows-backend' 249 | Cpu: '2048' 250 | Memory: '4096' 251 | Essential: 'true' 252 | Image: !Sub tryretool/backend:${RetoolVersion} 253 | LogConfiguration: 254 | LogDriver: awslogs 255 | Options: 256 | awslogs-group: !Ref 'CloudwatchLogsGroup' 257 | awslogs-region: !Ref 'AWS::Region' 258 | awslogs-stream-prefix: "SERVICE_RETOOL" 259 | Environment: 260 | - Name: DEPLOYMENT_TEMPLATE_TYPE 261 | Value: "aws-ecs-ec2" 262 | - Name: NODE_ENV 263 | Value: production 264 | - Name: SERVICE_TYPE 265 | Value: WORKFLOW_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 266 | - Name: "FORCE_DEPLOYMENT" 267 | Value: !Ref "Force" 268 | - Name: POSTGRES_DB 269 | Value: hammerhead_production 270 | - Name: POSTGRES_HOST 271 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 272 | - Name: POSTGRES_SSL_ENABLED 273 | Value: "true" 274 | - Name: POSTGRES_PORT 275 | Value: "5432" 276 | - Name: POSTGRES_USER 277 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 278 | - Name: POSTGRES_PASSWORD 279 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 280 | - Name: JWT_SECRET 281 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 282 | - Name: ENCRYPTION_KEY 283 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 284 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 285 | Value: temporal.retoolsvc 286 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 287 | Value: "7233" 288 | - Name: WORKFLOW_BACKEND_HOST 289 | Value: http://workflows-backend.retoolsvc:3000 290 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 291 | Value: http://code-executor.retoolsvc:3004 292 | - Name: LICENSE_KEY 293 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 294 | # Remove below when serving Retool over https 295 | - Name: COOKIE_INSECURE 296 | Value: "true" 297 | PortMappings: 298 | - ContainerPort: '3000' 299 | Command: ["./docker_scripts/start_api.sh"] 300 | 301 | RetoolWorkflowsWorkerTask: 302 | Type: AWS::ECS::TaskDefinition 303 | Properties: 304 | Family: 'retool-workflows-worker' 305 | TaskRoleArn: !Ref 'RetoolTaskRole' 306 | NetworkMode: awsvpc 307 | ContainerDefinitions: 308 | - Name: 'retool-workflows-worker' 309 | Cpu: '2048' 310 | Memory: '4096' 311 | Essential: 'true' 312 | Image: !Sub tryretool/backend:${RetoolVersion} 313 | LogConfiguration: 314 | LogDriver: awslogs 315 | Options: 316 | awslogs-group: !Ref 'CloudwatchLogsGroup' 317 | awslogs-region: !Ref 'AWS::Region' 318 | awslogs-stream-prefix: "SERVICE_RETOOL" 319 | Environment: 320 | - Name: DEPLOYMENT_TEMPLATE_TYPE 321 | Value: "aws-ecs-ec2" 322 | - Name: NODE_ENV 323 | Value: production 324 | - Name: SERVICE_TYPE 325 | Value: WORKFLOW_TEMPORAL_WORKER 326 | - Name: "FORCE_DEPLOYMENT" 327 | Value: !Ref "Force" 328 | - Name: POSTGRES_DB 329 | Value: hammerhead_production 330 | - Name: POSTGRES_HOST 331 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 332 | - Name: POSTGRES_SSL_ENABLED 333 | Value: "true" 334 | - Name: POSTGRES_PORT 335 | Value: "5432" 336 | - Name: POSTGRES_USER 337 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 338 | - Name: POSTGRES_PASSWORD 339 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 340 | - Name: JWT_SECRET 341 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 342 | - Name: ENCRYPTION_KEY 343 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 344 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 345 | Value: temporal.retoolsvc 346 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 347 | Value: "7233" 348 | - Name: WORKFLOW_BACKEND_HOST 349 | Value: http://workflows-backend.retoolsvc:3000 350 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 351 | Value: http://code-executor.retoolsvc:3004 352 | - Name: LICENSE_KEY 353 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 354 | Command: ["./docker_scripts/start_api.sh"] 355 | 356 | RetoolCodeExecutorTask: 357 | Type: AWS::ECS::TaskDefinition 358 | Properties: 359 | Family: 'retool-code-executor' 360 | TaskRoleArn: !Ref 'RetoolTaskRole' 361 | NetworkMode: awsvpc 362 | ContainerDefinitions: 363 | - Name: "retool-code-executor" 364 | Cpu: '2048' 365 | Memory: '4096' 366 | Essential: true 367 | Image: !Sub tryretool/code-executor-service:${RetoolVersion} 368 | LogConfiguration: 369 | LogDriver: awslogs 370 | Options: 371 | awslogs-group: !Ref CloudwatchLogsGroup 372 | awslogs-region: !Ref AWS::Region 373 | awslogs-stream-prefix: SERVICE_RETOOL 374 | # required to use nsjail sandboxing, which is required for custom libraries for JS and Python 375 | # Learn more here: https://docs.retool.com/self-hosted/concepts/architecture#code-executor 376 | Privileged: true 377 | # If not using nsjail sandboxing, remove the above line and instead use: 378 | # Privileged: false 379 | # User: "1001:1001" 380 | Environment: 381 | - Name: NODE_OPTIONS 382 | Value: "--max_old_space_size=1024" 383 | - Name: NODE_ENV 384 | Value: "production" 385 | PortMappings: 386 | - ContainerPort: 3004 387 | HostPort: 3004 388 | Protocol: "tcp" 389 | Command: 390 | - ./start.sh 391 | 392 | RetoolJWTSecret: 393 | Type: AWS::SecretsManager::Secret 394 | Properties: 395 | Description: 'This is the secret for Retool JWTs' 396 | GenerateSecretString: 397 | SecretStringTemplate: '{}' 398 | GenerateStringKey: 'password' 399 | PasswordLength: 16 400 | ExcludeCharacters: '"@/\' 401 | 402 | RetoolEncryptionKeySecret: 403 | Type: AWS::SecretsManager::Secret 404 | Properties: 405 | Description: 'This is the secret for encrypting credentials' 406 | GenerateSecretString: 407 | SecretStringTemplate: '{}' 408 | GenerateStringKey: 'password' 409 | PasswordLength: 16 410 | ExcludeCharacters: '"@/\' 411 | 412 | RetoolRDSSecret: 413 | Type: AWS::SecretsManager::Secret 414 | Properties: 415 | Description: 'This is the secret for the Retool RDS instance' 416 | GenerateSecretString: 417 | SecretStringTemplate: '{"username": "retool"}' 418 | GenerateStringKey: 'password' 419 | PasswordLength: 16 420 | ExcludeCharacters: '"@/\' 421 | 422 | 423 | RetoolRDSInstance: 424 | Type: AWS::RDS::DBInstance 425 | Properties: 426 | AllocatedStorage: "80" 427 | DBInstanceClass: "db.m5.large" 428 | Engine: postgres 429 | EngineVersion: "15.10" 430 | DBName: "hammerhead_production" 431 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 432 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 433 | Port: "5432" 434 | VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 435 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 436 | 437 | RDSSubnetGroup: 438 | Type: AWS::RDS::DBSubnetGroup 439 | Properties: 440 | DBSubnetGroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'rds subnet security group']] 441 | SubnetIds: !Ref 'SubnetId' 442 | 443 | ECSALB: 444 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 445 | Properties: 446 | Name: !Join ['-', [!Ref 'AWS::StackName', 'lb']] 447 | Scheme: "internet-facing" 448 | LoadBalancerAttributes: 449 | - Key: idle_timeout.timeout_seconds 450 | Value: '60' 451 | Subnets: !Ref 'ALBSubnetId' 452 | SecurityGroups: [!GetAtt [ALBSecurityGroup, GroupId]] 453 | 454 | ALBListener: 455 | Type: AWS::ElasticLoadBalancingV2::Listener 456 | DependsOn: RetoolServiceRole 457 | Properties: 458 | DefaultActions: 459 | - Type: forward 460 | TargetGroupArn: !Ref 'ECSTG' 461 | LoadBalancerArn: !Ref 'ECSALB' 462 | Port: '80' 463 | Protocol: HTTP 464 | 465 | ECSALBListenerRule: 466 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 467 | DependsOn: ALBListener 468 | Properties: 469 | Actions: 470 | - Type: forward 471 | TargetGroupArn: !Ref 'ECSTG' 472 | Conditions: 473 | - Field: path-pattern 474 | Values: [/] 475 | ListenerArn: !Ref 'ALBListener' 476 | Priority: 1 477 | 478 | ECSTG: 479 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 480 | DependsOn: ECSALB 481 | Properties: 482 | TargetType: ip 483 | HealthCheckIntervalSeconds: 61 484 | HealthCheckPath: '/api/checkHealth' 485 | HealthCheckProtocol: HTTP 486 | HealthCheckTimeoutSeconds: 60 487 | HealthyThresholdCount: 4 488 | Name: !Join ['-', [!Ref 'AWS::StackName', 'tg']] 489 | Port: '3000' 490 | Protocol: HTTP 491 | UnhealthyThresholdCount: 2 492 | VpcId: !Ref 'VpcId' 493 | TargetGroupAttributes: 494 | - Key: deregistration_delay.timeout_seconds 495 | Value: '30' 496 | 497 | RetoolECSservice: 498 | Type: AWS::ECS::Service 499 | DependsOn: ALBListener 500 | Properties: 501 | NetworkConfiguration: 502 | AwsvpcConfiguration: 503 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 504 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 505 | Subnets: !Ref 'SubnetId' 506 | Cluster: !Ref 'Cluster' 507 | DesiredCount: !Ref 'DesiredCount' 508 | DeploymentConfiguration: 509 | MaximumPercent: !Ref 'MaximumPercent' 510 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 511 | LoadBalancers: 512 | - ContainerName: 'retool' 513 | ContainerPort: '3000' 514 | TargetGroupArn: !Ref 'ECSTG' 515 | TaskDefinition: !Ref 'RetoolTask' 516 | 517 | RetoolJobsRunnerECSservice: 518 | Type: AWS::ECS::Service 519 | Properties: 520 | NetworkConfiguration: 521 | AwsvpcConfiguration: 522 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 523 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 524 | Subnets: !Ref 'SubnetId' 525 | Cluster: !Ref 'Cluster' 526 | DesiredCount: 1 527 | TaskDefinition: !Ref 'RetoolJobsRunnerTask' 528 | 529 | RetoolWorkflowsWorkerECSService: 530 | Type: AWS::ECS::Service 531 | Properties: 532 | NetworkConfiguration: 533 | AwsvpcConfiguration: 534 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 535 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 536 | Subnets: !Ref 'SubnetId' 537 | Cluster: !Ref 'Cluster' 538 | DesiredCount: !Ref 'DesiredWorkflowsCount' 539 | DeploymentConfiguration: 540 | MaximumPercent: !Ref 'MaximumPercent' 541 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 542 | TaskDefinition: !Ref 'RetoolWorkflowsWorkerTask' 543 | 544 | RetoolWorkflowsBackendECSService: 545 | DependsOn: RetoolWorkflowBackendServiceCloudmapService 546 | Type: AWS::ECS::Service 547 | Properties: 548 | NetworkConfiguration: 549 | AwsvpcConfiguration: 550 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 551 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 552 | Subnets: !Ref 'SubnetId' 553 | ServiceRegistries: 554 | - RegistryArn: !GetAtt [RetoolWorkflowBackendServiceCloudmapService, Arn] 555 | Cluster: !Ref 'Cluster' 556 | DesiredCount: !Ref 'DesiredWorkflowsCount' 557 | DeploymentConfiguration: 558 | MaximumPercent: !Ref 'MaximumPercent' 559 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 560 | TaskDefinition: !Ref 'RetoolWorkflowsBackendTask' 561 | 562 | RetoolCodeExecutorECSService: 563 | DependsOn: RetoolCodeExecutorServiceCloudmapService 564 | Type: AWS::ECS::Service 565 | Properties: 566 | NetworkConfiguration: 567 | AwsvpcConfiguration: 568 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 569 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 570 | Subnets: !Ref 'SubnetId' 571 | ServiceRegistries: 572 | - RegistryArn: !GetAtt [RetoolCodeExecutorServiceCloudmapService, Arn] 573 | Cluster: !Ref 'Cluster' 574 | DesiredCount: !Ref 'DesiredCodeExecutorCount' 575 | DeploymentConfiguration: 576 | MaximumPercent: !Ref 'MaximumPercent' 577 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 578 | TaskDefinition: !Ref 'RetoolCodeExecutorTask' 579 | 580 | RetoolServiceRole: 581 | Type: AWS::IAM::Role 582 | Properties: 583 | AssumeRolePolicyDocument: 584 | Statement: 585 | - Effect: Allow 586 | Principal: 587 | Service: [ecs.amazonaws.com] 588 | Action: ['sts:AssumeRole'] 589 | Path: / 590 | Policies: 591 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'service-policy']] 592 | PolicyDocument: 593 | Statement: 594 | - Effect: Allow 595 | Action: [ 596 | 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 597 | 'elasticloadbalancing:DeregisterTargets', 598 | 'elasticloadbalancing:Describe*', 599 | 'elasticloadbalancing:RegisterInstancesWithLoadBalancer', 600 | 'elasticloadbalancing:RegisterTargets', 601 | 'ec2:Describe*', 602 | 'ec2:AuthorizeSecurityGroupIngress'] 603 | Resource: '*' 604 | 605 | RetoolTaskRole: 606 | Type: AWS::IAM::Role 607 | Properties: 608 | AssumeRolePolicyDocument: 609 | Statement: 610 | - Effect: Allow 611 | Principal: 612 | Service: ['ecs-tasks.amazonaws.com'] 613 | Action: ['sts:AssumeRole'] 614 | Path: / 615 | Policies: [] 616 | 617 | # Workflows 618 | WorkflowsCloudMapNamespace: 619 | Type: AWS::ServiceDiscovery::PrivateDnsNamespace 620 | Properties: 621 | Name: retoolsvc 622 | Vpc: !Ref 'VpcId' 623 | 624 | RetoolWorkflowBackendServiceCloudmapService: 625 | Type: AWS::ServiceDiscovery::Service 626 | Properties: 627 | DnsConfig: 628 | DnsRecords: 629 | - TTL: 60 630 | Type: A 631 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 632 | RoutingPolicy: MULTIVALUE 633 | HealthCheckCustomConfig: 634 | FailureThreshold: 1 635 | Name: workflows-backend 636 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 637 | 638 | RetoolCodeExecutorServiceCloudmapService: 639 | Type: AWS::ServiceDiscovery::Service 640 | Properties: 641 | DnsConfig: 642 | DnsRecords: 643 | - TTL: 60 644 | Type: A 645 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 646 | RoutingPolicy: MULTIVALUE 647 | HealthCheckCustomConfig: 648 | FailureThreshold: 1 649 | Name: code-executor 650 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 651 | 652 | # Temporal 653 | RetoolTemporalRDSSecret: 654 | Type: AWS::SecretsManager::Secret 655 | Properties: 656 | Description: 'This is the secret for the Retool Temporal RDS instance' 657 | GenerateSecretString: 658 | SecretStringTemplate: '{"username": "retool"}' 659 | GenerateStringKey: 'password' 660 | PasswordLength: 16 661 | ExcludeCharacters: '"@/\' 662 | 663 | # RDS-based Temporal DB 664 | # RetoolTemporalRDSInstance: 665 | # Type: AWS::RDS::DBInstance 666 | # Properties: 667 | # AllocatedStorage: "80" 668 | # DBInstanceClass: "db.m4.large" 669 | # Engine: postgres 670 | # EngineVersion: "11.12" 671 | # DBName: "temporal" 672 | # MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 673 | # MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 674 | # Port: "5432" 675 | # VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 676 | # DBSubnetGroupName: !Ref 'RDSSubnetGroup' 677 | 678 | # Aurora-based Temporal DB 679 | RetoolTemporalRDSCluster: 680 | Type: AWS::RDS::DBCluster 681 | Properties: 682 | Engine: "aurora-postgresql" 683 | EngineVersion: "14.5" 684 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 685 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 686 | Port: "5432" 687 | VpcSecurityGroupIds: [!GetAtt [RDSSecurityGroup, GroupId]] 688 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 689 | ServerlessV2ScalingConfiguration: 690 | MinCapacity: 0.5 691 | MaxCapacity: 10 692 | RetoolTemporalRDSInstance: 693 | Type: AWS::RDS::DBInstance 694 | Properties: 695 | Engine: "aurora-postgresql" 696 | DBInstanceClass: "db.serverless" 697 | DBClusterIdentifier: !Ref 'RetoolTemporalRDSCluster' 698 | 699 | TemporalClusterFrontendTask: 700 | Type: AWS::ECS::TaskDefinition 701 | Properties: 702 | Family: 'retool-temporal-frontend' 703 | TaskRoleArn: !Ref 'RetoolTaskRole' 704 | NetworkMode: awsvpc 705 | ContainerDefinitions: 706 | - Name: temporal-cluster-frontend 707 | Cpu: '256' 708 | Memory: '512' 709 | Essential: 'true' 710 | Image: tryretool/one-offs:retool-temporal-1.1.6 711 | LogConfiguration: 712 | LogDriver: awslogs 713 | Options: 714 | awslogs-group: !Ref 'CloudwatchLogsGroup' 715 | awslogs-region: !Ref 'AWS::Region' 716 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 717 | Environment: 718 | - Name: SERVICES 719 | Value: frontend 720 | - Name: LOG_LEVEL 721 | Value: debug,info 722 | - Name: NUM_HISTORY_SHARDS 723 | Value: "128" 724 | - Name: DB 725 | Value: postgresql 726 | - Name: POSTGRES_HOST 727 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 728 | - Name: POSTGRES_PORT 729 | Value: 5432 730 | - Name: POSTGRES_USER 731 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 732 | - Name: POSTGRES_PASSWORD 733 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 734 | # - Name: SQL_TLS_ENABLED 735 | # Value: "true" 736 | # - Name: SQL_TLS 737 | # Value: "true" 738 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 739 | # Value: "true" 740 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 741 | # Value: "true" 742 | - Name: DBNAME 743 | Value: temporal 744 | - Name: DBNAME_VISIBILITY 745 | Value: temporal_visibility 746 | - Name: DYNAMIC_CONFIG_FILE_PATH 747 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 748 | - Name: ECS_DEPLOYED 749 | Value: "true" 750 | PortMappings: 751 | - ContainerPort: 7233 752 | HostPort: 7233 753 | Protocol: tcp 754 | - ContainerPort: 6933 755 | HostPort: 6933 756 | Protocol: tcp 757 | TemporalClusterHistoryTask: 758 | Type: AWS::ECS::TaskDefinition 759 | Properties: 760 | Family: 'retool-temporal-history' 761 | TaskRoleArn: !Ref 'RetoolTaskRole' 762 | NetworkMode: awsvpc 763 | ContainerDefinitions: 764 | - Name: temporal-cluster-history 765 | Cpu: '512' 766 | Memory: '1024' 767 | Essential: 'true' 768 | Image: tryretool/one-offs:retool-temporal-1.1.6 769 | LogConfiguration: 770 | LogDriver: awslogs 771 | Options: 772 | awslogs-group: !Ref 'CloudwatchLogsGroup' 773 | awslogs-region: !Ref 'AWS::Region' 774 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 775 | Environment: 776 | - Name: SERVICES 777 | Value: history 778 | - Name: LOG_LEVEL 779 | Value: debug,info 780 | - Name: NUM_HISTORY_SHARDS 781 | Value: "128" 782 | - Name: DB 783 | Value: postgresql 784 | - Name: POSTGRES_HOST 785 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 786 | - Name: POSTGRES_PORT 787 | Value: 5432 788 | - Name: POSTGRES_USER 789 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 790 | - Name: POSTGRES_PASSWORD 791 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 792 | # - Name: SQL_TLS_ENABLED 793 | # Value: "true" 794 | # - Name: SQL_TLS 795 | # Value: "true" 796 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 797 | # Value: "true" 798 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 799 | # Value: "true" 800 | - Name: DBNAME 801 | Value: temporal 802 | - Name: DBNAME_VISIBILITY 803 | Value: temporal_visibility 804 | - Name: DYNAMIC_CONFIG_FILE_PATH 805 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 806 | - Name: PUBLIC_FRONTEND_ADDRESS 807 | Value: temporal.retoolsvc:7233 808 | - Name: ECS_DEPLOYED 809 | Value: "true" 810 | PortMappings: 811 | - ContainerPort: 7234 812 | HostPort: 7234 813 | Protocol: tcp 814 | - ContainerPort: 6934 815 | HostPort: 6934 816 | Protocol: tcp 817 | TemporalClusterMatchingTask: 818 | Type: AWS::ECS::TaskDefinition 819 | Properties: 820 | Family: 'retool-temporal-matching' 821 | TaskRoleArn: !Ref 'RetoolTaskRole' 822 | NetworkMode: awsvpc 823 | ContainerDefinitions: 824 | - Name: temporal-cluster-matching 825 | Cpu: '512' 826 | Memory: '1024' 827 | Essential: 'true' 828 | Image: tryretool/one-offs:retool-temporal-1.1.6 829 | LogConfiguration: 830 | LogDriver: awslogs 831 | Options: 832 | awslogs-group: !Ref 'CloudwatchLogsGroup' 833 | awslogs-region: !Ref 'AWS::Region' 834 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 835 | Environment: 836 | - Name: SERVICES 837 | Value: matching 838 | - Name: LOG_LEVEL 839 | Value: debug,info 840 | - Name: NUM_HISTORY_SHARDS 841 | Value: "128" 842 | - Name: DB 843 | Value: postgresql 844 | - Name: POSTGRES_HOST 845 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 846 | - Name: POSTGRES_PORT 847 | Value: 5432 848 | - Name: POSTGRES_USER 849 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 850 | - Name: POSTGRES_PASSWORD 851 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 852 | # - Name: SQL_TLS_ENABLED 853 | # Value: "true" 854 | # - Name: SQL_TLS 855 | # Value: "true" 856 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 857 | # Value: "true" 858 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 859 | # Value: "true" 860 | - Name: DBNAME 861 | Value: temporal 862 | - Name: DBNAME_VISIBILITY 863 | Value: temporal_visibility 864 | - Name: DYNAMIC_CONFIG_FILE_PATH 865 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 866 | - Name: PUBLIC_FRONTEND_ADDRESS 867 | Value: temporal.retoolsvc:7233 868 | - Name: ECS_DEPLOYED 869 | Value: "true" 870 | PortMappings: 871 | - ContainerPort: 7235 872 | HostPort: 7235 873 | Protocol: tcp 874 | - ContainerPort: 6935 875 | HostPort: 6935 876 | Protocol: tcp 877 | TemporalClusterWorkerTask: 878 | Type: AWS::ECS::TaskDefinition 879 | Properties: 880 | Family: 'retool-temporal-worker' 881 | TaskRoleArn: !Ref 'RetoolTaskRole' 882 | NetworkMode: awsvpc 883 | ContainerDefinitions: 884 | - Name: temporal-cluster-worker 885 | Cpu: '256' 886 | Memory: '512' 887 | Essential: 'true' 888 | Image: tryretool/one-offs:retool-temporal-1.1.6 889 | LogConfiguration: 890 | LogDriver: awslogs 891 | Options: 892 | awslogs-group: !Ref 'CloudwatchLogsGroup' 893 | awslogs-region: !Ref 'AWS::Region' 894 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 895 | Environment: 896 | - Name: SERVICES 897 | Value: worker 898 | - Name: LOG_LEVEL 899 | Value: debug,info 900 | - Name: NUM_HISTORY_SHARDS 901 | Value: "128" 902 | - Name: DB 903 | Value: postgresql 904 | - Name: POSTGRES_HOST 905 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 906 | - Name: POSTGRES_PORT 907 | Value: 5432 908 | - Name: POSTGRES_USER 909 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 910 | - Name: POSTGRES_PASSWORD 911 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 912 | # - Name: SQL_TLS_ENABLED 913 | # Value: "true" 914 | # - Name: SQL_TLS 915 | # Value: "true" 916 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 917 | # Value: "true" 918 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 919 | # Value: "true" 920 | - Name: DBNAME 921 | Value: temporal 922 | - Name: DBNAME_VISIBILITY 923 | Value: temporal_visibility 924 | - Name: DYNAMIC_CONFIG_FILE_PATH 925 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 926 | - Name: PUBLIC_FRONTEND_ADDRESS 927 | Value: temporal.retoolsvc:7233 928 | - Name: ECS_DEPLOYED 929 | Value: "true" 930 | PortMappings: 931 | - ContainerPort: 7239 932 | HostPort: 7239 933 | Protocol: tcp 934 | - ContainerPort: 6939 935 | HostPort: 6939 936 | Protocol: tcp 937 | TemporalClusterWebTask: 938 | Type: AWS::ECS::TaskDefinition 939 | Properties: 940 | Family: 'retool-temporal-web' 941 | TaskRoleArn: !Ref 'RetoolTaskRole' 942 | NetworkMode: awsvpc 943 | ContainerDefinitions: 944 | - Name: temporal-cluster-web 945 | Cpu: '256' 946 | Memory: '512' 947 | Image: temporalio/ui:2.14.0 948 | LogConfiguration: 949 | LogDriver: awslogs 950 | Options: 951 | awslogs-group: !Ref 'CloudwatchLogsGroup' 952 | awslogs-region: !Ref 'AWS::Region' 953 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 954 | Environment: 955 | - Name: TEMPORAL_ADDRESS 956 | Value: temporal.retoolsvc:7233 957 | PortMappings: 958 | - ContainerPort: 8088 959 | HostPort: 8088 960 | Protocol: tcp 961 | TemporalFrontendService: 962 | Type: AWS::ECS::Service 963 | Properties: 964 | NetworkConfiguration: 965 | AwsvpcConfiguration: 966 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 967 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 968 | Subnets: !Ref 'SubnetId' 969 | ServiceRegistries: 970 | - RegistryArn: !GetAtt [TemporalFrontendCloudmapService, Arn] 971 | Cluster: !Ref 'Cluster' 972 | DesiredCount: !Ref 'DesiredWorkflowsCount' 973 | DeploymentConfiguration: 974 | MaximumPercent: !Ref 'MaximumPercent' 975 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 976 | TaskDefinition: !Ref 'TemporalClusterFrontendTask' 977 | TemporalHistoryService: 978 | Type: AWS::ECS::Service 979 | DependsOn: TemporalFrontendService 980 | Properties: 981 | NetworkConfiguration: 982 | AwsvpcConfiguration: 983 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 984 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 985 | Subnets: !Ref 'SubnetId' 986 | Cluster: !Ref 'Cluster' 987 | DesiredCount: !Ref 'DesiredWorkflowsCount' 988 | DeploymentConfiguration: 989 | MaximumPercent: !Ref 'MaximumPercent' 990 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 991 | TaskDefinition: !Ref 'TemporalClusterHistoryTask' 992 | TemporalMatchingService: 993 | Type: AWS::ECS::Service 994 | DependsOn: TemporalFrontendService 995 | Properties: 996 | NetworkConfiguration: 997 | AwsvpcConfiguration: 998 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 999 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1000 | Subnets: !Ref 'SubnetId' 1001 | Cluster: !Ref 'Cluster' 1002 | DesiredCount: !Ref 'DesiredWorkflowsCount' 1003 | DeploymentConfiguration: 1004 | MaximumPercent: !Ref 'MaximumPercent' 1005 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1006 | TaskDefinition: !Ref 'TemporalClusterMatchingTask' 1007 | TemporalWorkerService: 1008 | Type: AWS::ECS::Service 1009 | DependsOn: TemporalFrontendService 1010 | Properties: 1011 | NetworkConfiguration: 1012 | AwsvpcConfiguration: 1013 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 1014 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1015 | Subnets: !Ref 'SubnetId' 1016 | Cluster: !Ref 'Cluster' 1017 | DesiredCount: 1 1018 | DeploymentConfiguration: 1019 | MaximumPercent: !Ref 'MaximumPercent' 1020 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1021 | TaskDefinition: !Ref 'TemporalClusterWorkerTask' 1022 | TemporalClusterWebService: 1023 | Type: AWS::ECS::Service 1024 | DependsOn: TemporalFrontendService 1025 | Properties: 1026 | NetworkConfiguration: 1027 | AwsvpcConfiguration: 1028 | AssignPublicIp: DISABLED # must be run in private subnet with NAT gateway 1029 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1030 | Subnets: !Ref 'SubnetId' 1031 | Cluster: !Ref 'Cluster' 1032 | DesiredCount: 1 1033 | DeploymentConfiguration: 1034 | MaximumPercent: !Ref 'MaximumPercent' 1035 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1036 | TaskDefinition: !Ref 'TemporalClusterWebTask' 1037 | TemporalFrontendCloudmapService: 1038 | Type: AWS::ServiceDiscovery::Service 1039 | Properties: 1040 | DnsConfig: 1041 | DnsRecords: 1042 | - TTL: 60 1043 | Type: A 1044 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 1045 | RoutingPolicy: MULTIVALUE 1046 | HealthCheckCustomConfig: 1047 | FailureThreshold: 1 1048 | Name: temporal 1049 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 1050 | 1051 | Outputs: 1052 | ECSALB: 1053 | Description: Your ALB DNS URL 1054 | Value: !GetAtt [ECSALB, DNSName] 1055 | -------------------------------------------------------------------------------- /cloudformation/retool-workflows.fargate.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | Environment: 4 | Type: String 5 | Description: Environment string sent back to plogger with the logs 6 | SubnetId: 7 | Type: List 8 | Description: Select at two subnets in your selected VPC. 9 | ALBSubnetId: 10 | Type: List 11 | Description: Select the subnets for your Application Load Balancer. 12 | Cluster: 13 | Type: String 14 | Description: Cluster to put service in. 15 | RetoolVersion: 16 | Type: String 17 | Description: Retool version tag (e.g. 3.114.7-stable) 18 | Default: 3.114.7-stable 19 | DesiredCount: 20 | Type: Number 21 | Description: Default number of API container tasks to run 22 | Default: 1 23 | DesiredWorkflowsCount: 24 | Type: Number 25 | Description: Default number of tasks to run for Workflows Backend and Workflows Worker containers to run 26 | Default: 1 27 | DesiredCodeExecutorCount: 28 | Type: Number 29 | Description: Default number of tasks to run for Retool Code Executor containers 30 | Default: 1 31 | MaximumPercent: 32 | Type: Number 33 | Description: Maximum percentage of tasks to run during a deployment 34 | Default: 150 35 | MinimumHealthyPercent: 36 | Type: Number 37 | Default: 50 38 | Description: Maximum percentage of tasks to run during a deployment 39 | VpcId: 40 | Type: AWS::EC2::VPC::Id 41 | Description: Select a VPC that allows instances access to the Internet. 42 | Force: 43 | Type: String 44 | Description: "Used to force the deployment even when the image and parameters are otherwised unchanged." 45 | Default: "false" 46 | 47 | Resources: 48 | ALBSecurityGroup: 49 | Type: AWS::EC2::SecurityGroup 50 | Properties: 51 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'load balancer security group']] 52 | VpcId: !Ref 'VpcId' 53 | 54 | GlobalHttpInbound: 55 | Type: AWS::EC2::SecurityGroupIngress 56 | Properties: 57 | GroupId: !GetAtt [ALBSecurityGroup, GroupId] 58 | IpProtocol: tcp 59 | FromPort: '80' 60 | ToPort: '80' 61 | CidrIp: '0.0.0.0/0' 62 | 63 | RetoolSecurityGroup: 64 | Type: AWS::EC2::SecurityGroup 65 | Properties: 66 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'retool container security group']] 67 | VpcId: !Ref 'VpcId' 68 | 69 | ALBInbound: 70 | Type: AWS::EC2::SecurityGroupIngress 71 | Properties: 72 | GroupId: !GetAtt [RetoolSecurityGroup, GroupId] 73 | IpProtocol: tcp 74 | FromPort: '3000' 75 | ToPort: '3000' 76 | SourceSecurityGroupId: !GetAtt [ALBSecurityGroup, GroupId] 77 | 78 | RetoolSelfIngress: 79 | Type: AWS::EC2::SecurityGroupIngress 80 | Properties: 81 | GroupId: !GetAtt [RetoolSecurityGroup, GroupId] 82 | IpProtocol: -1 83 | SourceSecurityGroupId: !GetAtt [RetoolSecurityGroup, GroupId] 84 | 85 | TemporalSecurityGroup: 86 | Type: AWS::EC2::SecurityGroup 87 | Properties: 88 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'temporal security group']] 89 | SecurityGroupEgress: 90 | - CidrIp: 0.0.0.0/0 91 | Description: Allow all outbound traffic by default 92 | IpProtocol: "-1" 93 | VpcId: !Ref 'VpcId' 94 | 95 | TemporalFrontendIngress: 96 | Type: AWS::EC2::SecurityGroupIngress 97 | Properties: 98 | GroupId: !GetAtt [TemporalSecurityGroup, GroupId] 99 | IpProtocol: tcp 100 | FromPort: 7233 101 | ToPort: 7233 102 | SourceSecurityGroupId: !GetAtt [RetoolSecurityGroup, GroupId] 103 | 104 | TemporalSelfIngress: 105 | Type: AWS::EC2::SecurityGroupIngress 106 | Properties: 107 | GroupId: !GetAtt [TemporalSecurityGroup, GroupId] 108 | IpProtocol: -1 109 | SourceSecurityGroupId: !GetAtt [TemporalSecurityGroup, GroupId] 110 | 111 | 112 | CloudwatchLogsGroup: 113 | Type: AWS::Logs::LogGroup 114 | Properties: 115 | LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']] 116 | RetentionInDays: 14 117 | 118 | RDSSecurityGroup: 119 | Type: AWS::EC2::SecurityGroup 120 | Properties: 121 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'database security group']] 122 | VpcId: !Ref 'VpcId' 123 | 124 | RetoolECSPostgresInbound: 125 | Type: AWS::EC2::SecurityGroupIngress 126 | Properties: 127 | GroupId: !GetAtt [RDSSecurityGroup, GroupId] 128 | IpProtocol: tcp 129 | FromPort: '5432' 130 | ToPort: '5432' 131 | CidrIp: '0.0.0.0/0' 132 | 133 | RetoolTask: 134 | Type: AWS::ECS::TaskDefinition 135 | Properties: 136 | Family: 'retool' 137 | TaskRoleArn: !Ref 'RetoolTaskRole' 138 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 139 | RequiresCompatibilities: 140 | - FARGATE 141 | NetworkMode: awsvpc 142 | Cpu: '2048' 143 | Memory: '4096' 144 | ContainerDefinitions: 145 | - Name: 'retool' 146 | Essential: 'true' 147 | Image: !Sub tryretool/backend:${RetoolVersion} 148 | LogConfiguration: 149 | LogDriver: awslogs 150 | Options: 151 | awslogs-group: !Ref 'CloudwatchLogsGroup' 152 | awslogs-region: !Ref 'AWS::Region' 153 | awslogs-stream-prefix: "SERVICE_RETOOL" 154 | Environment: 155 | - Name: DEPLOYMENT_TEMPLATE_TYPE 156 | Value: "aws-ecs-fargate" 157 | - Name: NODE_ENV 158 | Value: production 159 | - Name: SERVICE_TYPE 160 | Value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 161 | - Name: "FORCE_DEPLOYMENT" 162 | Value: !Ref "Force" 163 | - Name: POSTGRES_DB 164 | Value: hammerhead_production 165 | - Name: POSTGRES_HOST 166 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 167 | - Name: POSTGRES_SSL_ENABLED 168 | Value: "true" 169 | - Name: POSTGRES_PORT 170 | Value: "5432" 171 | - Name: POSTGRES_USER 172 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 173 | - Name: POSTGRES_PASSWORD 174 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 175 | - Name: JWT_SECRET 176 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 177 | - Name: ENCRYPTION_KEY 178 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 179 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 180 | Value: temporal.retoolsvc 181 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 182 | Value: "7233" 183 | - Name: WORKFLOW_BACKEND_HOST 184 | Value: http://workflows-backend.retoolsvc:3000 185 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 186 | Value: http://code-executor.retoolsvc:3004 187 | - Name: LICENSE_KEY 188 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 189 | # Remove below when serving Retool over https 190 | - Name: COOKIE_INSECURE 191 | Value: "true" 192 | PortMappings: 193 | - ContainerPort: '3000' 194 | # HostPort: '80' 195 | Command: ["./docker_scripts/start_api.sh"] 196 | 197 | RetoolJobsRunnerTask: 198 | Type: AWS::ECS::TaskDefinition 199 | Properties: 200 | Family: 'retool-jobs-runner' 201 | TaskRoleArn: !Ref 'RetoolTaskRole' 202 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 203 | RequiresCompatibilities: 204 | - FARGATE 205 | NetworkMode: awsvpc 206 | Cpu: '1024' 207 | Memory: '2048' 208 | ContainerDefinitions: 209 | - Name: 'retool-jobs-runner' 210 | Essential: 'true' 211 | Image: !Sub tryretool/backend:${RetoolVersion} 212 | LogConfiguration: 213 | LogDriver: awslogs 214 | Options: 215 | awslogs-group: !Ref 'CloudwatchLogsGroup' 216 | awslogs-region: !Ref 'AWS::Region' 217 | awslogs-stream-prefix: "SERVICE_RETOOL" 218 | Environment: 219 | - Name: DEPLOYMENT_TEMPLATE_TYPE 220 | Value: "aws-ecs-fargate" 221 | - Name: NODE_ENV 222 | Value: production 223 | - Name: SERVICE_TYPE 224 | Value: JOBS_RUNNER 225 | - Name: "FORCE_DEPLOYMENT" 226 | Value: !Ref "Force" 227 | - Name: POSTGRES_DB 228 | Value: hammerhead_production 229 | - Name: POSTGRES_HOST 230 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 231 | - Name: POSTGRES_SSL_ENABLED 232 | Value: "true" 233 | - Name: POSTGRES_PORT 234 | Value: "5432" 235 | - Name: POSTGRES_USER 236 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 237 | - Name: POSTGRES_PASSWORD 238 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 239 | - Name: JWT_SECRET 240 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 241 | - Name: ENCRYPTION_KEY 242 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 243 | - Name: LICENSE_KEY 244 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 245 | Command: ["./docker_scripts/start_api.sh"] 246 | 247 | RetoolWorkflowsBackendTask: 248 | Type: AWS::ECS::TaskDefinition 249 | Properties: 250 | Family: 'retool-workflows-backend' 251 | TaskRoleArn: !Ref 'RetoolTaskRole' 252 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 253 | RequiresCompatibilities: 254 | - FARGATE 255 | NetworkMode: awsvpc 256 | Cpu: '2048' 257 | Memory: '4096' 258 | ContainerDefinitions: 259 | - Name: 'retool-workflows-backend' 260 | Essential: 'true' 261 | Image: !Sub tryretool/backend:${RetoolVersion} 262 | LogConfiguration: 263 | LogDriver: awslogs 264 | Options: 265 | awslogs-group: !Ref 'CloudwatchLogsGroup' 266 | awslogs-region: !Ref 'AWS::Region' 267 | awslogs-stream-prefix: "SERVICE_RETOOL" 268 | Environment: 269 | - Name: DEPLOYMENT_TEMPLATE_TYPE 270 | Value: "aws-ecs-fargate" 271 | - Name: NODE_ENV 272 | Value: production 273 | - Name: SERVICE_TYPE 274 | Value: WORKFLOW_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 275 | - Name: "FORCE_DEPLOYMENT" 276 | Value: !Ref "Force" 277 | - Name: POSTGRES_DB 278 | Value: hammerhead_production 279 | - Name: POSTGRES_HOST 280 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 281 | - Name: POSTGRES_SSL_ENABLED 282 | Value: "true" 283 | - Name: POSTGRES_PORT 284 | Value: "5432" 285 | - Name: POSTGRES_USER 286 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 287 | - Name: POSTGRES_PASSWORD 288 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 289 | - Name: JWT_SECRET 290 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 291 | - Name: ENCRYPTION_KEY 292 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 293 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 294 | Value: temporal.retoolsvc 295 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 296 | Value: "7233" 297 | - Name: WORKFLOW_BACKEND_HOST 298 | Value: http://workflows-backend.retoolsvc:3000 299 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 300 | Value: http://code-executor.retoolsvc:3004 301 | - Name: LICENSE_KEY 302 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 303 | # Remove below when serving Retool over https 304 | - Name: COOKIE_INSECURE 305 | Value: "true" 306 | Command: ["./docker_scripts/start_api.sh"] 307 | 308 | RetoolWorkflowsWorkerTask: 309 | Type: AWS::ECS::TaskDefinition 310 | Properties: 311 | Family: 'retool-workflows-worker' 312 | TaskRoleArn: !Ref 'RetoolTaskRole' 313 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 314 | RequiresCompatibilities: 315 | - FARGATE 316 | NetworkMode: awsvpc 317 | Cpu: '2048' 318 | Memory: '4096' 319 | ContainerDefinitions: 320 | - Name: 'retool-workflows-worker' 321 | Essential: 'true' 322 | Image: !Sub tryretool/backend:${RetoolVersion} 323 | LogConfiguration: 324 | LogDriver: awslogs 325 | Options: 326 | awslogs-group: !Ref 'CloudwatchLogsGroup' 327 | awslogs-region: !Ref 'AWS::Region' 328 | awslogs-stream-prefix: "SERVICE_RETOOL" 329 | Environment: 330 | - Name: DEPLOYMENT_TEMPLATE_TYPE 331 | Value: "aws-ecs-fargate" 332 | - Name: NODE_ENV 333 | Value: production 334 | - Name: SERVICE_TYPE 335 | Value: WORKFLOW_TEMPORAL_WORKER 336 | - Name: "FORCE_DEPLOYMENT" 337 | Value: !Ref "Force" 338 | - Name: POSTGRES_DB 339 | Value: hammerhead_production 340 | - Name: POSTGRES_HOST 341 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 342 | - Name: POSTGRES_SSL_ENABLED 343 | Value: "true" 344 | - Name: POSTGRES_PORT 345 | Value: "5432" 346 | - Name: POSTGRES_USER 347 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 348 | - Name: POSTGRES_PASSWORD 349 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 350 | - Name: JWT_SECRET 351 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 352 | - Name: ENCRYPTION_KEY 353 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 354 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 355 | Value: temporal.retoolsvc 356 | - Name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 357 | Value: "7233" 358 | - Name: WORKFLOW_BACKEND_HOST 359 | Value: http://workflows-backend.retoolsvc:3000 360 | - Name: CODE_EXECUTOR_INGRESS_DOMAIN 361 | Value: http://code-executor.retoolsvc:3004 362 | - Name: LICENSE_KEY 363 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 364 | Command: ["./docker_scripts/start_api.sh"] 365 | 366 | RetoolCodeExecutorTask: 367 | Type: AWS::ECS::TaskDefinition 368 | Properties: 369 | Family: 'retool-code-executor' 370 | TaskRoleArn: !Ref 'RetoolTaskRole' 371 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 372 | RequiresCompatibilities: 373 | - FARGATE 374 | NetworkMode: awsvpc 375 | Cpu: '2048' 376 | Memory: '4096' 377 | ContainerDefinitions: 378 | - Name: "retool-code-executor" 379 | Essential: true 380 | Image: !Sub tryretool/code-executor-service:${RetoolVersion} 381 | LogConfiguration: 382 | LogDriver: awslogs 383 | Options: 384 | awslogs-group: !Ref CloudwatchLogsGroup 385 | awslogs-region: !Ref AWS::Region 386 | awslogs-stream-prefix: SERVICE_RETOOL 387 | User: "1001:1001" 388 | Environment: 389 | - Name: CONTAINER_UNPRIVILEGED_MODE 390 | Value: true 391 | - Name: NODE_OPTIONS 392 | Value: "--max_old_space_size=1024" 393 | - Name: NODE_ENV 394 | Value: "production" 395 | PortMappings: 396 | - ContainerPort: 3004 397 | HostPort: 3004 398 | Protocol: "tcp" 399 | Command: 400 | - ./start.sh 401 | 402 | 403 | RetoolJWTSecret: 404 | Type: AWS::SecretsManager::Secret 405 | Properties: 406 | Description: 'This is the secret for Retool JWTs' 407 | GenerateSecretString: 408 | SecretStringTemplate: '{}' 409 | GenerateStringKey: 'password' 410 | PasswordLength: 16 411 | ExcludeCharacters: '"@/\' 412 | 413 | RetoolEncryptionKeySecret: 414 | Type: AWS::SecretsManager::Secret 415 | Properties: 416 | Description: 'This is the secret for encrypting credentials' 417 | GenerateSecretString: 418 | SecretStringTemplate: '{}' 419 | GenerateStringKey: 'password' 420 | PasswordLength: 16 421 | ExcludeCharacters: '"@/\' 422 | 423 | RetoolRDSSecret: 424 | Type: AWS::SecretsManager::Secret 425 | Properties: 426 | Description: 'This is the secret for the Retool RDS instance' 427 | GenerateSecretString: 428 | SecretStringTemplate: '{"username": "retool"}' 429 | GenerateStringKey: 'password' 430 | PasswordLength: 16 431 | ExcludeCharacters: '"@/\' 432 | 433 | 434 | RetoolRDSInstance: 435 | Type: AWS::RDS::DBInstance 436 | Properties: 437 | AllocatedStorage: "80" 438 | DBInstanceClass: "db.m5.large" 439 | Engine: postgres 440 | EngineVersion: "15.10" 441 | DBName: "hammerhead_production" 442 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 443 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 444 | Port: "5432" 445 | VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 446 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 447 | 448 | RDSSubnetGroup: 449 | Type: AWS::RDS::DBSubnetGroup 450 | Properties: 451 | DBSubnetGroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'rds subnet security group']] 452 | SubnetIds: !Ref 'SubnetId' 453 | 454 | ECSALB: 455 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 456 | Properties: 457 | Name: !Join ['-', [!Ref 'AWS::StackName', 'lb']] 458 | Scheme: "internet-facing" 459 | LoadBalancerAttributes: 460 | - Key: idle_timeout.timeout_seconds 461 | Value: '60' 462 | Subnets: !Ref 'ALBSubnetId' 463 | SecurityGroups: [!GetAtt [ALBSecurityGroup, GroupId]] 464 | 465 | ALBListener: 466 | Type: AWS::ElasticLoadBalancingV2::Listener 467 | DependsOn: RetoolServiceRole 468 | Properties: 469 | DefaultActions: 470 | - Type: forward 471 | TargetGroupArn: !Ref 'ECSTG' 472 | LoadBalancerArn: !Ref 'ECSALB' 473 | Port: '80' 474 | Protocol: HTTP 475 | 476 | ECSALBListenerRule: 477 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 478 | DependsOn: ALBListener 479 | Properties: 480 | Actions: 481 | - Type: forward 482 | TargetGroupArn: !Ref 'ECSTG' 483 | Conditions: 484 | - Field: path-pattern 485 | Values: [/] 486 | ListenerArn: !Ref 'ALBListener' 487 | Priority: 1 488 | 489 | ECSTG: 490 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 491 | DependsOn: ECSALB 492 | Properties: 493 | TargetType: ip 494 | HealthCheckIntervalSeconds: 61 495 | HealthCheckPath: '/api/checkHealth' 496 | HealthCheckProtocol: HTTP 497 | HealthCheckTimeoutSeconds: 60 498 | HealthyThresholdCount: 4 499 | Name: !Join ['-', [!Ref 'AWS::StackName', 'tg']] 500 | Port: '3000' 501 | Protocol: HTTP 502 | UnhealthyThresholdCount: 2 503 | VpcId: !Ref 'VpcId' 504 | TargetGroupAttributes: 505 | - Key: deregistration_delay.timeout_seconds 506 | Value: '30' 507 | 508 | RetoolECSservice: 509 | Type: AWS::ECS::Service 510 | DependsOn: ALBListener 511 | Properties: 512 | NetworkConfiguration: 513 | AwsvpcConfiguration: 514 | AssignPublicIp: ENABLED 515 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 516 | Subnets: !Ref 'SubnetId' 517 | Cluster: !Ref 'Cluster' 518 | DesiredCount: !Ref 'DesiredCount' 519 | LaunchType: FARGATE 520 | DeploymentConfiguration: 521 | MaximumPercent: !Ref 'MaximumPercent' 522 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 523 | LoadBalancers: 524 | - ContainerName: 'retool' 525 | ContainerPort: '3000' 526 | TargetGroupArn: !Ref 'ECSTG' 527 | TaskDefinition: !Ref 'RetoolTask' 528 | 529 | RetoolJobsRunnerECSservice: 530 | Type: AWS::ECS::Service 531 | Properties: 532 | NetworkConfiguration: 533 | AwsvpcConfiguration: 534 | AssignPublicIp: ENABLED 535 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 536 | Subnets: !Ref 'SubnetId' 537 | Cluster: !Ref 'Cluster' 538 | DesiredCount: 1 539 | LaunchType: FARGATE 540 | TaskDefinition: !Ref 'RetoolJobsRunnerTask' 541 | 542 | RetoolWorkflowsWorkerECSService: 543 | Type: AWS::ECS::Service 544 | Properties: 545 | NetworkConfiguration: 546 | AwsvpcConfiguration: 547 | AssignPublicIp: ENABLED 548 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 549 | Subnets: !Ref 'SubnetId' 550 | Cluster: !Ref 'Cluster' 551 | DesiredCount: !Ref 'DesiredWorkflowsCount' 552 | DeploymentConfiguration: 553 | MaximumPercent: !Ref 'MaximumPercent' 554 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 555 | LaunchType: FARGATE 556 | TaskDefinition: !Ref 'RetoolWorkflowsWorkerTask' 557 | 558 | RetoolWorkflowsBackendECSService: 559 | DependsOn: RetoolWorkflowBackendFargateServiceCloudmapService 560 | Type: AWS::ECS::Service 561 | Properties: 562 | NetworkConfiguration: 563 | AwsvpcConfiguration: 564 | AssignPublicIp: ENABLED 565 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 566 | Subnets: !Ref 'SubnetId' 567 | ServiceRegistries: 568 | - RegistryArn: !GetAtt [RetoolWorkflowBackendFargateServiceCloudmapService, Arn] 569 | Cluster: !Ref 'Cluster' 570 | DesiredCount: !Ref 'DesiredWorkflowsCount' 571 | DeploymentConfiguration: 572 | MaximumPercent: !Ref 'MaximumPercent' 573 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 574 | LaunchType: FARGATE 575 | TaskDefinition: !Ref 'RetoolWorkflowsBackendTask' 576 | 577 | RetoolCodeExecutorECSService: 578 | DependsOn: RetoolCodeExecutorFargateServiceCloudmapService 579 | Type: AWS::ECS::Service 580 | Properties: 581 | NetworkConfiguration: 582 | AwsvpcConfiguration: 583 | AssignPublicIp: ENABLED 584 | SecurityGroups: [!GetAtt [RetoolSecurityGroup, GroupId]] 585 | Subnets: !Ref 'SubnetId' 586 | ServiceRegistries: 587 | - RegistryArn: !GetAtt [RetoolCodeExecutorFargateServiceCloudmapService, Arn] 588 | Cluster: !Ref 'Cluster' 589 | DesiredCount: !Ref 'DesiredCodeExecutorCount' 590 | DeploymentConfiguration: 591 | MaximumPercent: !Ref 'MaximumPercent' 592 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 593 | LaunchType: FARGATE 594 | TaskDefinition: !Ref 'RetoolCodeExecutorTask' 595 | 596 | RetoolServiceRole: 597 | Type: AWS::IAM::Role 598 | Properties: 599 | AssumeRolePolicyDocument: 600 | Statement: 601 | - Effect: Allow 602 | Principal: 603 | Service: [ecs.amazonaws.com] 604 | Action: ['sts:AssumeRole'] 605 | Path: / 606 | Policies: 607 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'service-policy']] 608 | PolicyDocument: 609 | Statement: 610 | - Effect: Allow 611 | Action: [ 612 | 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 613 | 'elasticloadbalancing:DeregisterTargets', 614 | 'elasticloadbalancing:Describe*', 615 | 'elasticloadbalancing:RegisterInstancesWithLoadBalancer', 616 | 'elasticloadbalancing:RegisterTargets', 617 | 'ec2:Describe*', 618 | 'ec2:AuthorizeSecurityGroupIngress'] 619 | Resource: '*' 620 | 621 | RetoolTaskRole: 622 | Type: AWS::IAM::Role 623 | Properties: 624 | AssumeRolePolicyDocument: 625 | Statement: 626 | - Effect: Allow 627 | Principal: 628 | Service: ['ecs-tasks.amazonaws.com'] 629 | Action: ['sts:AssumeRole'] 630 | Path: / 631 | Policies: [] 632 | 633 | # FARGATE ONLY 634 | RetoolExecutionRole: 635 | Type: AWS::IAM::Role 636 | Properties: 637 | AssumeRolePolicyDocument: 638 | Statement: 639 | - Effect: Allow 640 | Principal: 641 | Service: ['ecs-tasks.amazonaws.com'] 642 | Action: ['sts:AssumeRole'] 643 | Path: / 644 | Policies: 645 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'execution-policy']] 646 | PolicyDocument: 647 | Statement: 648 | - Effect: Allow 649 | Action: [ 650 | "ecr:GetAuthorizationToken", 651 | "ecr:BatchCheckLayerAvailability", 652 | "ecr:GetDownloadUrlForLayer", 653 | "ecr:BatchGetImage", 654 | "logs:CreateLogStream", 655 | "logs:PutLogEvents"] 656 | Resource: '*' 657 | 658 | # Workflows 659 | WorkflowsCloudMapNamespace: 660 | Type: AWS::ServiceDiscovery::PrivateDnsNamespace 661 | Properties: 662 | Name: retoolsvc 663 | Vpc: !Ref 'VpcId' 664 | 665 | RetoolWorkflowBackendFargateServiceCloudmapService: 666 | Type: AWS::ServiceDiscovery::Service 667 | Properties: 668 | DnsConfig: 669 | DnsRecords: 670 | - TTL: 60 671 | Type: A 672 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 673 | RoutingPolicy: MULTIVALUE 674 | HealthCheckCustomConfig: 675 | FailureThreshold: 1 676 | Name: workflows-backend 677 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 678 | 679 | RetoolCodeExecutorFargateServiceCloudmapService: 680 | Type: AWS::ServiceDiscovery::Service 681 | Properties: 682 | DnsConfig: 683 | DnsRecords: 684 | - TTL: 60 685 | Type: A 686 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 687 | RoutingPolicy: MULTIVALUE 688 | HealthCheckCustomConfig: 689 | FailureThreshold: 1 690 | Name: code-executor 691 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 692 | 693 | # Temporal 694 | RetoolTemporalRDSSecret: 695 | Type: AWS::SecretsManager::Secret 696 | Properties: 697 | Description: 'This is the secret for the Retool Temporal RDS instance' 698 | GenerateSecretString: 699 | SecretStringTemplate: '{"username": "retool"}' 700 | GenerateStringKey: 'password' 701 | PasswordLength: 16 702 | ExcludeCharacters: '"@/\' 703 | 704 | # RDS-based Temporal DB 705 | # RetoolTemporalRDSInstance: 706 | # Type: AWS::RDS::DBInstance 707 | # Properties: 708 | # AllocatedStorage: "80" 709 | # DBInstanceClass: "db.m4.large" 710 | # Engine: postgres 711 | # EngineVersion: "11.12" 712 | # DBName: "temporal" 713 | # MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 714 | # MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 715 | # Port: "5432" 716 | # VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 717 | # DBSubnetGroupName: !Ref 'RDSSubnetGroup' 718 | 719 | # Aurora-based Temporal DB 720 | RetoolTemporalRDSCluster: 721 | Type: AWS::RDS::DBCluster 722 | Properties: 723 | Engine: "aurora-postgresql" 724 | EngineVersion: "14.5" 725 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 726 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 727 | Port: "5432" 728 | VpcSecurityGroupIds: [!GetAtt [RDSSecurityGroup, GroupId]] 729 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 730 | ServerlessV2ScalingConfiguration: 731 | MinCapacity: 0.5 732 | MaxCapacity: 10 733 | RetoolTemporalRDSInstance: 734 | Type: AWS::RDS::DBInstance 735 | Properties: 736 | Engine: "aurora-postgresql" 737 | DBInstanceClass: "db.serverless" 738 | DBClusterIdentifier: !Ref 'RetoolTemporalRDSCluster' 739 | 740 | TemporalClusterFrontendTask: 741 | Type: AWS::ECS::TaskDefinition 742 | Properties: 743 | Family: 'retool-temporal-frontend' 744 | TaskRoleArn: !Ref 'RetoolTaskRole' 745 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 746 | RequiresCompatibilities: 747 | - FARGATE 748 | NetworkMode: awsvpc 749 | Cpu: '256' 750 | Memory: '512' 751 | ContainerDefinitions: 752 | - Name: temporal-cluster-frontend 753 | Essential: 'true' 754 | Image: tryretool/one-offs:retool-temporal-1.1.6 755 | LogConfiguration: 756 | LogDriver: awslogs 757 | Options: 758 | awslogs-group: !Ref 'CloudwatchLogsGroup' 759 | awslogs-region: !Ref 'AWS::Region' 760 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 761 | Environment: 762 | - Name: SERVICES 763 | Value: frontend 764 | - Name: LOG_LEVEL 765 | Value: debug,info 766 | - Name: NUM_HISTORY_SHARDS 767 | Value: "128" 768 | - Name: DB 769 | Value: postgresql 770 | - Name: POSTGRES_HOST 771 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 772 | - Name: POSTGRES_PORT 773 | Value: 5432 774 | - Name: POSTGRES_USER 775 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 776 | - Name: POSTGRES_PASSWORD 777 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 778 | # - Name: SQL_TLS_ENABLED 779 | # Value: "true" 780 | # - Name: SQL_TLS 781 | # Value: "true" 782 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 783 | # Value: "true" 784 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 785 | # Value: "true" 786 | - Name: DBNAME 787 | Value: temporal 788 | - Name: DBNAME_VISIBILITY 789 | Value: temporal_visibility 790 | - Name: DYNAMIC_CONFIG_FILE_PATH 791 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 792 | - Name: ECS_DEPLOYED 793 | Value: "true" 794 | PortMappings: 795 | - ContainerPort: 7233 796 | HostPort: 7233 797 | Protocol: tcp 798 | - ContainerPort: 6933 799 | HostPort: 6933 800 | Protocol: tcp 801 | TemporalClusterHistoryTask: 802 | Type: AWS::ECS::TaskDefinition 803 | Properties: 804 | Family: 'retool-temporal-history' 805 | TaskRoleArn: !Ref 'RetoolTaskRole' 806 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 807 | RequiresCompatibilities: 808 | - FARGATE 809 | NetworkMode: awsvpc 810 | Cpu: '512' 811 | Memory: '1024' 812 | ContainerDefinitions: 813 | - Name: temporal-cluster-history 814 | Essential: 'true' 815 | Image: tryretool/one-offs:retool-temporal-1.1.6 816 | LogConfiguration: 817 | LogDriver: awslogs 818 | Options: 819 | awslogs-group: !Ref 'CloudwatchLogsGroup' 820 | awslogs-region: !Ref 'AWS::Region' 821 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 822 | Environment: 823 | - Name: SERVICES 824 | Value: history 825 | - Name: LOG_LEVEL 826 | Value: debug,info 827 | - Name: NUM_HISTORY_SHARDS 828 | Value: "128" 829 | - Name: DB 830 | Value: postgresql 831 | - Name: POSTGRES_HOST 832 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 833 | - Name: POSTGRES_PORT 834 | Value: 5432 835 | - Name: POSTGRES_USER 836 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 837 | - Name: POSTGRES_PASSWORD 838 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 839 | # - Name: SQL_TLS_ENABLED 840 | # Value: "true" 841 | # - Name: SQL_TLS 842 | # Value: "true" 843 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 844 | # Value: "true" 845 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 846 | # Value: "true" 847 | - Name: DBNAME 848 | Value: temporal 849 | - Name: DBNAME_VISIBILITY 850 | Value: temporal_visibility 851 | - Name: DYNAMIC_CONFIG_FILE_PATH 852 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 853 | - Name: PUBLIC_FRONTEND_ADDRESS 854 | Value: temporal.retoolsvc:7233 855 | - Name: ECS_DEPLOYED 856 | Value: "true" 857 | PortMappings: 858 | - ContainerPort: 7234 859 | HostPort: 7234 860 | Protocol: tcp 861 | - ContainerPort: 6934 862 | HostPort: 6934 863 | Protocol: tcp 864 | TemporalClusterMatchingTask: 865 | Type: AWS::ECS::TaskDefinition 866 | Properties: 867 | Family: 'retool-temporal-matching' 868 | TaskRoleArn: !Ref 'RetoolTaskRole' 869 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 870 | RequiresCompatibilities: 871 | - FARGATE 872 | NetworkMode: awsvpc 873 | Cpu: '512' 874 | Memory: '1024' 875 | ContainerDefinitions: 876 | - Name: temporal-cluster-matching 877 | Essential: 'true' 878 | Image: tryretool/one-offs:retool-temporal-1.1.6 879 | LogConfiguration: 880 | LogDriver: awslogs 881 | Options: 882 | awslogs-group: !Ref 'CloudwatchLogsGroup' 883 | awslogs-region: !Ref 'AWS::Region' 884 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 885 | Environment: 886 | - Name: SERVICES 887 | Value: matching 888 | - Name: LOG_LEVEL 889 | Value: debug,info 890 | - Name: NUM_HISTORY_SHARDS 891 | Value: "128" 892 | - Name: DB 893 | Value: postgresql 894 | - Name: POSTGRES_HOST 895 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 896 | - Name: POSTGRES_PORT 897 | Value: 5432 898 | - Name: POSTGRES_USER 899 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 900 | - Name: POSTGRES_PASSWORD 901 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 902 | # - Name: SQL_TLS_ENABLED 903 | # Value: "true" 904 | # - Name: SQL_TLS 905 | # Value: "true" 906 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 907 | # Value: "true" 908 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 909 | # Value: "true" 910 | - Name: DBNAME 911 | Value: temporal 912 | - Name: DBNAME_VISIBILITY 913 | Value: temporal_visibility 914 | - Name: DYNAMIC_CONFIG_FILE_PATH 915 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 916 | - Name: PUBLIC_FRONTEND_ADDRESS 917 | Value: temporal.retoolsvc:7233 918 | - Name: ECS_DEPLOYED 919 | Value: "true" 920 | PortMappings: 921 | - ContainerPort: 7235 922 | HostPort: 7235 923 | Protocol: tcp 924 | - ContainerPort: 6935 925 | HostPort: 6935 926 | Protocol: tcp 927 | TemporalClusterWorkerTask: 928 | Type: AWS::ECS::TaskDefinition 929 | Properties: 930 | Family: 'retool-temporal-worker' 931 | TaskRoleArn: !Ref 'RetoolTaskRole' 932 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 933 | RequiresCompatibilities: 934 | - FARGATE 935 | NetworkMode: awsvpc 936 | Cpu: '256' 937 | Memory: '512' 938 | ContainerDefinitions: 939 | - Name: temporal-cluster-worker 940 | Essential: 'true' 941 | Image: tryretool/one-offs:retool-temporal-1.1.6 942 | LogConfiguration: 943 | LogDriver: awslogs 944 | Options: 945 | awslogs-group: !Ref 'CloudwatchLogsGroup' 946 | awslogs-region: !Ref 'AWS::Region' 947 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 948 | Environment: 949 | - Name: SERVICES 950 | Value: worker 951 | - Name: LOG_LEVEL 952 | Value: debug,info 953 | - Name: NUM_HISTORY_SHARDS 954 | Value: "128" 955 | - Name: DB 956 | Value: postgresql 957 | - Name: POSTGRES_HOST 958 | Value: !GetAtt [RetoolTemporalRDSInstance, Endpoint.Address] 959 | - Name: POSTGRES_PORT 960 | Value: 5432 961 | - Name: POSTGRES_USER 962 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:username}}' ]] 963 | - Name: POSTGRES_PASSWORD 964 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolTemporalRDSSecret, ':SecretString:password}}' ]] 965 | # - Name: SQL_TLS_ENABLED 966 | # Value: "true" 967 | # - Name: SQL_TLS 968 | # Value: "true" 969 | # - Name: SQL_TLS_SKIP_HOST_VERIFICATION 970 | # Value: "true" 971 | # - Name: SQL_TLS_DISABLE_HOST_VERIFICATION 972 | # Value: "true" 973 | - Name: DBNAME 974 | Value: temporal 975 | - Name: DBNAME_VISIBILITY 976 | Value: temporal_visibility 977 | - Name: DYNAMIC_CONFIG_FILE_PATH 978 | Value: /etc/temporal/ecs/dynamic_config/dynamicconfig-sql.yaml 979 | - Name: PUBLIC_FRONTEND_ADDRESS 980 | Value: temporal.retoolsvc:7233 981 | - Name: ECS_DEPLOYED 982 | Value: "true" 983 | PortMappings: 984 | - ContainerPort: 7239 985 | HostPort: 7239 986 | Protocol: tcp 987 | - ContainerPort: 6939 988 | HostPort: 6939 989 | Protocol: tcp 990 | TemporalClusterWebTask: 991 | Type: AWS::ECS::TaskDefinition 992 | Properties: 993 | Family: 'retool-temporal-web' 994 | TaskRoleArn: !Ref 'RetoolTaskRole' 995 | ExecutionRoleArn: !Ref 'RetoolExecutionRole' 996 | RequiresCompatibilities: 997 | - FARGATE 998 | NetworkMode: awsvpc 999 | Cpu: '256' 1000 | Memory: '512' 1001 | ContainerDefinitions: 1002 | - Name: temporal-cluster-web 1003 | Image: temporalio/ui:2.14.0 1004 | LogConfiguration: 1005 | LogDriver: awslogs 1006 | Options: 1007 | awslogs-group: !Ref 'CloudwatchLogsGroup' 1008 | awslogs-region: !Ref 'AWS::Region' 1009 | awslogs-stream-prefix: "SERVICE_RETOOL_TEMPORAL" 1010 | Environment: 1011 | - Name: TEMPORAL_ADDRESS 1012 | Value: temporal.retoolsvc:7233 1013 | PortMappings: 1014 | - ContainerPort: 8088 1015 | HostPort: 8088 1016 | Protocol: tcp 1017 | TemporalFrontendService: 1018 | Type: AWS::ECS::Service 1019 | Properties: 1020 | NetworkConfiguration: 1021 | AwsvpcConfiguration: 1022 | AssignPublicIp: ENABLED 1023 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1024 | Subnets: !Ref 'SubnetId' 1025 | ServiceRegistries: 1026 | - RegistryArn: !GetAtt [TemporalFrontendCloudmapService, Arn] 1027 | Cluster: !Ref 'Cluster' 1028 | DesiredCount: !Ref 'DesiredWorkflowsCount' 1029 | LaunchType: FARGATE 1030 | DeploymentConfiguration: 1031 | MaximumPercent: !Ref 'MaximumPercent' 1032 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1033 | TaskDefinition: !Ref 'TemporalClusterFrontendTask' 1034 | TemporalHistoryService: 1035 | Type: AWS::ECS::Service 1036 | DependsOn: TemporalFrontendService 1037 | Properties: 1038 | NetworkConfiguration: 1039 | AwsvpcConfiguration: 1040 | AssignPublicIp: ENABLED 1041 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1042 | Subnets: !Ref 'SubnetId' 1043 | Cluster: !Ref 'Cluster' 1044 | DesiredCount: !Ref 'DesiredWorkflowsCount' 1045 | LaunchType: FARGATE 1046 | DeploymentConfiguration: 1047 | MaximumPercent: !Ref 'MaximumPercent' 1048 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1049 | TaskDefinition: !Ref 'TemporalClusterHistoryTask' 1050 | TemporalMatchingService: 1051 | Type: AWS::ECS::Service 1052 | DependsOn: TemporalFrontendService 1053 | Properties: 1054 | NetworkConfiguration: 1055 | AwsvpcConfiguration: 1056 | AssignPublicIp: ENABLED 1057 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1058 | Subnets: !Ref 'SubnetId' 1059 | Cluster: !Ref 'Cluster' 1060 | DesiredCount: !Ref 'DesiredWorkflowsCount' 1061 | LaunchType: FARGATE 1062 | DeploymentConfiguration: 1063 | MaximumPercent: !Ref 'MaximumPercent' 1064 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1065 | TaskDefinition: !Ref 'TemporalClusterMatchingTask' 1066 | TemporalWorkerService: 1067 | Type: AWS::ECS::Service 1068 | DependsOn: TemporalFrontendService 1069 | Properties: 1070 | NetworkConfiguration: 1071 | AwsvpcConfiguration: 1072 | AssignPublicIp: ENABLED 1073 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1074 | Subnets: !Ref 'SubnetId' 1075 | Cluster: !Ref 'Cluster' 1076 | DesiredCount: 1 1077 | LaunchType: FARGATE 1078 | DeploymentConfiguration: 1079 | MaximumPercent: !Ref 'MaximumPercent' 1080 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1081 | TaskDefinition: !Ref 'TemporalClusterWorkerTask' 1082 | TemporalClusterWebService: 1083 | Type: AWS::ECS::Service 1084 | DependsOn: TemporalFrontendService 1085 | Properties: 1086 | NetworkConfiguration: 1087 | AwsvpcConfiguration: 1088 | AssignPublicIp: ENABLED 1089 | SecurityGroups: [!GetAtt [TemporalSecurityGroup, GroupId]] 1090 | Subnets: !Ref 'SubnetId' 1091 | Cluster: !Ref 'Cluster' 1092 | DesiredCount: 1 1093 | LaunchType: FARGATE 1094 | DeploymentConfiguration: 1095 | MaximumPercent: !Ref 'MaximumPercent' 1096 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 1097 | TaskDefinition: !Ref 'TemporalClusterWebTask' 1098 | TemporalFrontendCloudmapService: 1099 | Type: AWS::ServiceDiscovery::Service 1100 | Properties: 1101 | DnsConfig: 1102 | DnsRecords: 1103 | - TTL: 60 1104 | Type: A 1105 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 1106 | RoutingPolicy: MULTIVALUE 1107 | HealthCheckCustomConfig: 1108 | FailureThreshold: 1 1109 | Name: temporal 1110 | NamespaceId: !GetAtt [WorkflowsCloudMapNamespace, Id] 1111 | 1112 | Outputs: 1113 | ECSALB: 1114 | Description: Your ALB DNS URL 1115 | Value: !GetAtt [ECSALB, DNSName] 1116 | -------------------------------------------------------------------------------- /cloudformation/retool.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Parameters: 3 | Environment: 4 | Type: String 5 | Description: Environment string sent back to plogger with the logs 6 | SubnetId: 7 | Type: List 8 | Description: Select at two subnets in your selected VPC. 9 | Cluster: 10 | Type: String 11 | Description: Cluster to put service in. 12 | Image: 13 | Type: String 14 | Description: Image to use in the service. 15 | DesiredCount: 16 | Type: Number 17 | Description: Default number of tasks to run 18 | MaximumPercent: 19 | Type: Number 20 | Description: Maximum percentage of tasks to run during a deployment 21 | Default: 150 22 | MinimumHealthyPercent: 23 | Type: Number 24 | Default: 50 25 | Description: Maximum percentage of tasks to run during a deployment 26 | VpcId: 27 | Type: AWS::EC2::VPC::Id 28 | Description: Select a VPC that allows instances access to the Internet. 29 | Force: 30 | Type: String 31 | Description: "Used to force the deployment even when the image and parameters are otherwised unchanged." 32 | Default: "false" 33 | 34 | Resources: 35 | ALBSecurityGroup: 36 | Type: AWS::EC2::SecurityGroup 37 | Properties: 38 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'load balancer security group']] 39 | VpcId: !Ref 'VpcId' 40 | 41 | GlobalHttpInbound: 42 | Type: AWS::EC2::SecurityGroupIngress 43 | Properties: 44 | GroupId: !Ref 'ALBSecurityGroup' 45 | IpProtocol: tcp 46 | FromPort: '80' 47 | ToPort: '80' 48 | CidrIp: '0.0.0.0/0' 49 | 50 | CloudwatchLogsGroup: 51 | Type: AWS::Logs::LogGroup 52 | Properties: 53 | LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']] 54 | RetentionInDays: 14 55 | 56 | RDSSecurityGroup: 57 | Type: AWS::EC2::SecurityGroup 58 | Properties: 59 | GroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'database security group']] 60 | VpcId: !Ref 'VpcId' 61 | 62 | RetoolECSPostgresInbound: 63 | Type: AWS::EC2::SecurityGroupIngress 64 | Properties: 65 | GroupId: !GetAtt [RDSSecurityGroup, GroupId] 66 | IpProtocol: tcp 67 | FromPort: '5432' 68 | ToPort: '5432' 69 | CidrIp: '0.0.0.0/0' 70 | 71 | RetoolTask: 72 | Type: AWS::ECS::TaskDefinition 73 | Properties: 74 | Family: 'retool' 75 | TaskRoleArn: !Ref 'RetoolTaskRole' 76 | ContainerDefinitions: 77 | - Name: 'retool' 78 | Cpu: '1024' 79 | Essential: 'true' 80 | Image: !Ref 'Image' 81 | Memory: '4096' 82 | LogConfiguration: 83 | LogDriver: awslogs 84 | Options: 85 | awslogs-group: !Ref 'CloudwatchLogsGroup' 86 | awslogs-region: !Ref 'AWS::Region' 87 | awslogs-stream-prefix: "SERVICE_RETOOL" 88 | Environment: 89 | - Name: DEPLOYMENT_TEMPLATE_TYPE 90 | Value: "aws-ecs-ec2" 91 | - Name: NODE_ENV 92 | Value: production 93 | - Name: SERVICE_TYPE 94 | Value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 95 | - Name: "FORCE_DEPLOYMENT" 96 | Value: !Ref "Force" 97 | - Name: POSTGRES_DB 98 | Value: hammerhead_production 99 | - Name: POSTGRES_HOST 100 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 101 | - Name: POSTGRES_SSL_ENABLED 102 | Value: "true" 103 | - Name: POSTGRES_PORT 104 | Value: "5432" 105 | - Name: POSTGRES_USER 106 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 107 | - Name: POSTGRES_PASSWORD 108 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 109 | - Name: JWT_SECRET 110 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 111 | - Name: ENCRYPTION_KEY 112 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 113 | - Name: LICENSE_KEY 114 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 115 | # Remove below when serving Retool over https 116 | - Name: COOKIE_INSECURE 117 | Value: "true" 118 | PortMappings: 119 | - ContainerPort: '3000' 120 | HostPort: '80' 121 | Command: ["./docker_scripts/start_api.sh"] 122 | 123 | RetoolJobsRunnerTask: 124 | Type: AWS::ECS::TaskDefinition 125 | Properties: 126 | Family: 'retool' 127 | TaskRoleArn: !Ref 'RetoolTaskRole' 128 | ContainerDefinitions: 129 | - Name: 'retool-jobs-runner' 130 | Cpu: '2048' 131 | Memory: '4096' 132 | Essential: 'true' 133 | Image: !Ref 'Image' 134 | LogConfiguration: 135 | LogDriver: awslogs 136 | Options: 137 | awslogs-group: !Ref 'CloudwatchLogsGroup' 138 | awslogs-region: !Ref 'AWS::Region' 139 | awslogs-stream-prefix: "SERVICE_RETOOL" 140 | Environment: 141 | - Name: DEPLOYMENT_TEMPLATE_TYPE 142 | Value: "aws-ecs-ec2" 143 | - Name: NODE_ENV 144 | Value: production 145 | - Name: SERVICE_TYPE 146 | Value: JOBS_RUNNER 147 | - Name: "FORCE_DEPLOYMENT" 148 | Value: !Ref "Force" 149 | - Name: POSTGRES_DB 150 | Value: hammerhead_production 151 | - Name: POSTGRES_HOST 152 | Value: !GetAtt [RetoolRDSInstance, Endpoint.Address] 153 | - Name: POSTGRES_SSL_ENABLED 154 | Value: "true" 155 | - Name: POSTGRES_PORT 156 | Value: "5432" 157 | - Name: POSTGRES_USER 158 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 159 | - Name: POSTGRES_PASSWORD 160 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 161 | - Name: JWT_SECRET 162 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolJWTSecret, ':SecretString:password}}' ]] 163 | - Name: ENCRYPTION_KEY 164 | Value: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolEncryptionKeySecret, ':SecretString:password}}' ]] 165 | - Name: LICENSE_KEY 166 | Value: "EXPIRED-LICENSE-KEY-TRIAL" 167 | Command: ["./docker_scripts/start_api.sh"] 168 | 169 | RetoolJWTSecret: 170 | Type: AWS::SecretsManager::Secret 171 | Properties: 172 | Description: 'This is the secret for Retool JWTs' 173 | GenerateSecretString: 174 | SecretStringTemplate: '{}' 175 | GenerateStringKey: 'password' 176 | PasswordLength: 16 177 | ExcludeCharacters: '"@/\' 178 | 179 | RetoolEncryptionKeySecret: 180 | Type: AWS::SecretsManager::Secret 181 | Properties: 182 | Description: 'This is the secret for encrypting credentials' 183 | GenerateSecretString: 184 | SecretStringTemplate: '{}' 185 | GenerateStringKey: 'password' 186 | PasswordLength: 16 187 | ExcludeCharacters: '"@/\' 188 | 189 | RetoolRDSSecret: 190 | Type: AWS::SecretsManager::Secret 191 | Properties: 192 | Description: 'This is the secret for the Retool RDS instance' 193 | GenerateSecretString: 194 | SecretStringTemplate: '{"username": "retool"}' 195 | GenerateStringKey: 'password' 196 | PasswordLength: 16 197 | ExcludeCharacters: '"@/\' 198 | 199 | 200 | RetoolRDSInstance: 201 | Type: AWS::RDS::DBInstance 202 | Properties: 203 | AllocatedStorage: "80" 204 | DBInstanceClass: "db.m5.large" 205 | Engine: postgres 206 | EngineVersion: "13.11" 207 | DBName: "hammerhead_production" 208 | MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:username}}' ]] 209 | MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref RetoolRDSSecret, ':SecretString:password}}' ]] 210 | Port: "5432" 211 | VPCSecurityGroups: [!GetAtt [RDSSecurityGroup, GroupId]] 212 | DBSubnetGroupName: !Ref 'RDSSubnetGroup' 213 | 214 | RDSSubnetGroup: 215 | Type: AWS::RDS::DBSubnetGroup 216 | Properties: 217 | DBSubnetGroupDescription: !Join [" ", [!Ref 'AWS::StackName', 'rds subnet security group']] 218 | SubnetIds: !Ref 'SubnetId' 219 | 220 | ECSALB: 221 | Type: AWS::ElasticLoadBalancingV2::LoadBalancer 222 | Properties: 223 | Name: !Join ['-', [!Ref 'AWS::StackName', 'lb']] 224 | Scheme: "internet-facing" 225 | LoadBalancerAttributes: 226 | - Key: idle_timeout.timeout_seconds 227 | Value: '60' 228 | Subnets: !Ref 'SubnetId' 229 | SecurityGroups: [!Ref 'ALBSecurityGroup'] 230 | 231 | ALBListener: 232 | Type: AWS::ElasticLoadBalancingV2::Listener 233 | DependsOn: RetoolServiceRole 234 | Properties: 235 | DefaultActions: 236 | - Type: forward 237 | TargetGroupArn: !Ref 'ECSTG' 238 | LoadBalancerArn: !Ref 'ECSALB' 239 | Port: '80' 240 | Protocol: HTTP 241 | 242 | ECSALBListenerRule: 243 | Type: AWS::ElasticLoadBalancingV2::ListenerRule 244 | DependsOn: ALBListener 245 | Properties: 246 | Actions: 247 | - Type: forward 248 | TargetGroupArn: !Ref 'ECSTG' 249 | Conditions: 250 | - Field: path-pattern 251 | Values: [/] 252 | ListenerArn: !Ref 'ALBListener' 253 | Priority: 1 254 | 255 | ECSTG: 256 | Type: AWS::ElasticLoadBalancingV2::TargetGroup 257 | DependsOn: ECSALB 258 | Properties: 259 | HealthCheckIntervalSeconds: 10 260 | HealthCheckPath: '/api/checkHealth' 261 | HealthCheckProtocol: HTTP 262 | HealthCheckTimeoutSeconds: 5 263 | HealthyThresholdCount: 4 264 | Name: !Join ['-', [!Ref 'AWS::StackName', 'tg']] 265 | Port: '80' 266 | Protocol: HTTP 267 | UnhealthyThresholdCount: 2 268 | VpcId: !Ref 'VpcId' 269 | TargetGroupAttributes: 270 | - Key: deregistration_delay.timeout_seconds 271 | Value: '30' 272 | 273 | RetoolECSservice: 274 | Type: AWS::ECS::Service 275 | DependsOn: ALBListener 276 | Properties: 277 | Cluster: !Ref 'Cluster' 278 | DesiredCount: !Ref 'DesiredCount' 279 | DeploymentConfiguration: 280 | MaximumPercent: !Ref 'MaximumPercent' 281 | MinimumHealthyPercent: !Ref 'MinimumHealthyPercent' 282 | LoadBalancers: 283 | - ContainerName: 'retool' 284 | ContainerPort: '3000' 285 | TargetGroupArn: !Ref 'ECSTG' 286 | Role: !Ref 'RetoolServiceRole' 287 | TaskDefinition: !Ref 'RetoolTask' 288 | 289 | RetoolJobsRunnerECSservice: 290 | Type: AWS::ECS::Service 291 | Properties: 292 | Cluster: !Ref 'Cluster' 293 | DesiredCount: 1 294 | TaskDefinition: !Ref 'RetoolJobsRunnerTask' 295 | 296 | RetoolServiceRole: 297 | Type: AWS::IAM::Role 298 | Properties: 299 | AssumeRolePolicyDocument: 300 | Statement: 301 | - Effect: Allow 302 | Principal: 303 | Service: [ecs.amazonaws.com] 304 | Action: ['sts:AssumeRole'] 305 | Path: / 306 | Policies: 307 | - PolicyName: !Join ['-', ['Retool', !Ref 'Environment', 'service-policy']] 308 | PolicyDocument: 309 | Statement: 310 | - Effect: Allow 311 | Action: [ 312 | 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 313 | 'elasticloadbalancing:DeregisterTargets', 314 | 'elasticloadbalancing:Describe*', 315 | 'elasticloadbalancing:RegisterInstancesWithLoadBalancer', 316 | 'elasticloadbalancing:RegisterTargets', 317 | 'ec2:Describe*', 318 | 'ec2:AuthorizeSecurityGroupIngress'] 319 | Resource: '*' 320 | 321 | RetoolTaskRole: 322 | Type: AWS::IAM::Role 323 | Properties: 324 | AssumeRolePolicyDocument: 325 | Statement: 326 | - Effect: Allow 327 | Principal: 328 | Service: ['ecs-tasks.amazonaws.com'] 329 | Action: ['sts:AssumeRole'] 330 | Path: / 331 | Policies: [] 332 | 333 | Outputs: 334 | ECSALB: 335 | Description: Your ALB DNS URL 336 | Value: !GetAtt [ECSALB, DNSName] 337 | -------------------------------------------------------------------------------- /compose.yaml: -------------------------------------------------------------------------------- 1 | # Comment out the below 'include' block to use Retool-managed Temporal (Enterprise license) 2 | include: 3 | - temporal.yaml 4 | 5 | services: 6 | api: 7 | build: 8 | context: . 9 | env_file: docker.env 10 | environment: 11 | - SERVICE_TYPE=MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 12 | ports: 13 | - 3000:3000 14 | networks: 15 | - frontend 16 | - backend 17 | - code-executor 18 | depends_on: 19 | - postgres 20 | restart: always 21 | 22 | jobs-runner: 23 | build: 24 | context: . 25 | env_file: docker.env 26 | environment: 27 | - SERVICE_TYPE=JOBS_RUNNER 28 | networks: 29 | - backend 30 | depends_on: 31 | - postgres 32 | restart: always 33 | 34 | workflows-worker: 35 | build: 36 | context: . 37 | env_file: docker.env 38 | environment: 39 | - SERVICE_TYPE=WORKFLOW_TEMPORAL_WORKER 40 | - NODE_OPTIONS=--max_old_space_size=1024 41 | networks: 42 | - backend 43 | - code-executor 44 | depends_on: 45 | - postgres 46 | restart: always 47 | 48 | workflows-backend: 49 | build: 50 | context: . 51 | env_file: docker.env 52 | environment: 53 | - SERVICE_TYPE=WORKFLOW_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 54 | networks: 55 | - backend 56 | - code-executor 57 | depends_on: 58 | - postgres 59 | restart: always 60 | 61 | code-executor: 62 | build: 63 | context: . 64 | target: code-executor 65 | environment: 66 | - NODE_OPTIONS=--max_old_space_size=1024 67 | networks: 68 | - code-executor 69 | # Privileged is required to sandbox user code execution and to use custom libraries 70 | # Set to false if your deployment method does not allow this 71 | privileged: true 72 | restart: always 73 | 74 | # Retool's internal DB, we recommend using an externally hosted database: https://docs.retool.com/docs/configuring-retools-storage-database 75 | postgres: 76 | image: postgres:16.8 77 | env_file: docker.env 78 | networks: 79 | - backend 80 | volumes: 81 | - data:/var/lib/postgresql/data 82 | restart: always 83 | 84 | retooldb-postgres: 85 | image: postgres:16.8 86 | env_file: retooldb.env 87 | networks: 88 | - backend 89 | volumes: 90 | - retooldb-data:/var/lib/postgresql/data 91 | restart: always 92 | 93 | # Optional Nginx container for handling TLS for your domain (requires setting DOMAINS and STAGE) 94 | https-portal: 95 | image: tryretool/https-portal:latest 96 | env_file: docker.env 97 | environment: 98 | # Change 'local' -> 'production' below once your domain is pointing to this server 99 | STAGE: local 100 | CLIENT_MAX_BODY_SIZE: 40M 101 | KEEPALIVE_TIMEOUT: 605 102 | PROXY_CONNECT_TIMEOUT: 600 103 | PROXY_SEND_TIMEOUT: 600 104 | PROXY_READ_TIMEOUT: 600 105 | ports: 106 | - 80:80 107 | - 443:443 108 | networks: 109 | - frontend 110 | restart: always 111 | 112 | networks: 113 | frontend: 114 | backend: 115 | code-executor: 116 | 117 | volumes: 118 | data: 119 | retooldb-data: 120 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "" 4 | echo "Checking installation requirements..." 5 | if grep Ubuntu /etc/issue &> /dev/null; then 6 | echo " ✅ Running on Ubuntu" 7 | else 8 | echo " ⚠️ This script is supported on Ubuntu, other distros may need Docker and Docker Compose installed manually" 9 | fi 10 | 11 | if command -v docker &> /dev/null; then 12 | echo " ✅ Docker is installed: $(docker --version)" 13 | 14 | if docker compose version &> /dev/null; then 15 | echo " ✅ Docker Compose plugin is installed: $(docker compose version)" 16 | else 17 | echo " ❌ Docker Compose plugin is not installed." 18 | echo " You can install it following the instructions at: https://docs.docker.com/compose/install" 19 | exit 1 20 | fi 21 | else 22 | echo " Docker is not yet installed" 23 | if command -v wget &> /dev/null; then 24 | echo " ✅ wget is installed" 25 | else 26 | echo " ❌ wget not installed, needed to download Docker's install script" 27 | exit 1 28 | fi 29 | echo " Attempting to run Docker's install script (https://get.docker.com)..." 30 | wget -qO- https://get.docker.com/ | sh 31 | echo " Rechecking Docker and Docker Compose..." 32 | if ! command -v docker &> /dev/null || ! docker compose version &> /dev/null; then 33 | echo " ❌ Docker or Docker Compose plugin still not installed" 34 | echo " See Docker docs (https://docs.docker.com/install) to install manually before rerunning this script" 35 | exit 1 36 | else 37 | echo " ✅ Docker is installed: $(docker --version)" 38 | echo " ✅ Docker Compose plugin is installed: $(docker compose version)" 39 | fi 40 | fi 41 | 42 | echo "" 43 | 44 | [[ -f docker.env ]] && echo "⚠️ docker.env file already exists, skipping initializing it!" && exit 1 45 | 46 | echo "Prompting for optional configuration..." 47 | 48 | read -p " Retool license key: " licenseKey 49 | licenseKey=${licenseKey:-EXPIRED-LICENSE-KEY-TRIAL} 50 | 51 | read -p " Domain (e.g. retool.company.com) pointing to this server: " hostname 52 | hostname=${hostname:-$(dig +short myip.opendns.com @resolver1.opendns.com)} 53 | echo "" 54 | 55 | # Create docker.env with values 56 | 57 | random() { cat /dev/urandom | base64 | head -c "$1" | tr -d +/ ; } 58 | 59 | cat << EOF > docker.env 60 | # Environment variables reference: docs.retool.com/docs/environment-variables 61 | DEPLOYMENT_TEMPLATE_TYPE=docker-compose 62 | 63 | # Retool's internal Postgres credentials 64 | POSTGRES_HOST=postgres 65 | POSTGRES_DB=hammerhead_production 66 | POSTGRES_PORT=5432 67 | POSTGRES_USER=retool_internal_user 68 | POSTGRES_PASSWORD=$(random 64) 69 | 70 | # Retool DB credentials 71 | RETOOLDB_POSTGRES_HOST=retooldb-postgres 72 | RETOOLDB_POSTGRES_DB=postgres 73 | RETOOLDB_POSTGRES_PORT=5432 74 | RETOOLDB_POSTGRES_USER=root 75 | RETOOLDB_POSTGRES_PASSWORD=$(random 64) 76 | 77 | # Workflows configuration 78 | WORKFLOW_BACKEND_HOST=http://workflows-backend:3000 79 | CODE_EXECUTOR_INGRESS_DOMAIN=http://code-executor:3004 80 | 81 | # Comment out below to use Retool-managed Temporal (Enterprise license) 82 | WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST=temporal 83 | WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT=7233 84 | 85 | # Key to encrypt/decrypt sensitive values stored in the Postgres database 86 | ENCRYPTION_KEY=$(random 64) 87 | 88 | # Key to sign requests for authentication with Retool's backend API server 89 | JWT_SECRET=$(random 256) 90 | 91 | # License you received from my.retool.com or your Retool contact 92 | LICENSE_KEY=$licenseKey 93 | 94 | # Make sure $hostname is your domain to set up HTTPS (e.g. retool.company.com) 95 | DOMAINS=$hostname -> http://api:3000 96 | 97 | # Used to create links like user invitations and password resets 98 | # Retool tries to guess this, but it can be incorrect if using a proxy in front of the instance 99 | BASE_DOMAIN=https://$hostname 100 | 101 | # If your domain/HTTPS isn't in place yet 102 | # COOKIE_INSECURE=true 103 | 104 | EOF 105 | 106 | echo "✅ Created docker.env" 107 | 108 | # Pull Retool DB config from docker.env if retooldb.env doesn't exist 109 | 110 | [[ -f retooldb.env ]] || grep RETOOLDB docker.env | cut -c 10- > retooldb.env && echo "✅ Created retooldb.env" 111 | 112 | # Next steps 113 | 114 | echo "" 115 | echo "Done! Check docker.env and retooldb.env files for expected values, and confirm" 116 | echo "the Retool version in Dockerfile. We suggest the most recent X.Y.Z-stable version," 117 | echo "see Dockerhub for available tags: https://hub.docker.com/r/tryretool/backend/tags" 118 | echo "" 119 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-code-executor-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | retoolService: code-executor 6 | name: code-executor 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | retoolService: code-executor 12 | revisionHistoryLimit: 3 13 | template: 14 | metadata: 15 | labels: 16 | retoolService: code-executor 17 | spec: 18 | containers: 19 | - args: 20 | - bash 21 | - start.sh 22 | env: 23 | - name: DEPLOYMENT_TEMPLATE_TYPE 24 | value: k8s-manifests 25 | - name: NODE_ENV 26 | value: production 27 | - name: NODE_OPTIONS 28 | value: --max_old_space_size=1024 29 | - name: PORT 30 | value: "3004" 31 | image: tryretool/code-executor-service:X.Y.Z 32 | name: code-executor 33 | ports: 34 | - containerPort: 3004 35 | name: retool 36 | protocol: TCP 37 | resources: 38 | limits: 39 | cpu: 2 40 | memory: 4Gi 41 | requests: 42 | cpu: 1 43 | memory: 1Gi 44 | # code executor uses nsjail to sandbox code execution. nsjail requires privileged container access. 45 | # If your deployment does not support privileged access, you can set this to false to not use nsjail. 46 | # Without nsjail, all code is run without sandboxing within your deployment. 47 | securityContext: 48 | privileged: true 49 | --- 50 | apiVersion: v1 51 | kind: Service 52 | metadata: 53 | labels: 54 | retoolService: code-executor 55 | name: code-executor 56 | namespace: retool 57 | spec: 58 | selector: 59 | retoolService: code-executor 60 | ports: 61 | - protocol: TCP 62 | port: 3004 63 | targetPort: 3004 64 | name: retool 65 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: api 6 | name: api 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: api 12 | template: 13 | metadata: 14 | labels: 15 | app: api 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: DEPLOYMENT_TEMPLATE_TYPE 25 | value: k8s-manifests 26 | - name: JWT_SECRET 27 | valueFrom: 28 | secretKeyRef: 29 | name: retoolsecrets 30 | key: jwt_secret 31 | - name: SERVICE_TYPE 32 | value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 69 | value: "retool-temporal-frontend" 70 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 71 | value: "7233" 72 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 73 | value: "workflows" 74 | - name: WORKFLOW_BACKEND_HOST 75 | value: http://workflows-api 76 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 77 | value: http://code-executor:3004 78 | image: tryretool/backend:X.Y.Z 79 | name: api 80 | ports: 81 | - containerPort: 3000 82 | resources: 83 | limits: 84 | memory: 8Gi 85 | cpu: 2 86 | requests: 87 | cpu: 1 88 | memory: 4Gi 89 | volumeMounts: 90 | - name: retool-pv 91 | mountPath: /retool_backend/pv-data 92 | restartPolicy: Always 93 | volumes: 94 | - name: retool-pv 95 | persistentVolumeClaim: 96 | claimName: retool-pvc 97 | --- 98 | apiVersion: v1 99 | kind: Service 100 | metadata: 101 | labels: 102 | app: api 103 | name: api 104 | spec: 105 | type: LoadBalancer 106 | ports: 107 | - name: "3000" 108 | port: 3000 109 | targetPort: 3000 110 | selector: 111 | app: api 112 | --- 113 | apiVersion: v1 114 | kind: PersistentVolumeClaim 115 | metadata: 116 | name: retool-pvc 117 | spec: 118 | accessModes: 119 | - ReadWriteOnce 120 | resources: 121 | requests: 122 | storage: 1Gi 123 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-jobs-runner.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: jobs-runner 6 | name: jobs-runner 7 | spec: 8 | replicas: 1 # jobs-runner pod should never be more than 1 9 | selector: 10 | matchLabels: 11 | app: jobs-runner 12 | template: 13 | metadata: 14 | labels: 15 | app: jobs-runner 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: DEPLOYMENT_TEMPLATE_TYPE 25 | value: k8s-manifests 26 | - name: JWT_SECRET 27 | valueFrom: 28 | secretKeyRef: 29 | name: retoolsecrets 30 | key: jwt_secret 31 | - name: SERVICE_TYPE 32 | value: JOBS_RUNNER 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | image: tryretool/backend:X.Y.Z 69 | name: jobs-runner 70 | ports: 71 | - containerPort: 3000 72 | resources: 73 | limits: 74 | cpu: 4 75 | memory: 8Gi 76 | requests: 77 | cpu: 2 78 | memory: 4Gi 79 | restartPolicy: Always 80 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-postgres.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: postgres-pvc 5 | labels: 6 | name: postgres-pvc 7 | spec: 8 | accessModes: 9 | - ReadWriteOnce 10 | resources: 11 | requests: 12 | storage: 60Gi 13 | 14 | --- 15 | apiVersion: apps/v1 16 | kind: Deployment 17 | metadata: 18 | name: postgres 19 | labels: 20 | app: postgres 21 | spec: 22 | replicas: 1 23 | selector: 24 | matchLabels: 25 | app: postgres 26 | strategy: 27 | type: Recreate 28 | template: 29 | metadata: 30 | labels: 31 | app: postgres 32 | spec: 33 | containers: 34 | - env: 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | name: postgres 49 | image: postgres:11.13 50 | resources: {} 51 | volumeMounts: 52 | - mountPath: /var/lib/postgresql/data 53 | subPath: postgres 54 | name: postgres-pv 55 | restartPolicy: Always 56 | volumes: 57 | - name: postgres-pv 58 | persistentVolumeClaim: 59 | claimName: postgres-pvc 60 | --- 61 | apiVersion: v1 62 | kind: Service 63 | metadata: 64 | labels: 65 | app: postgres 66 | name: postgres 67 | spec: 68 | clusterIP: None 69 | ports: 70 | - name: headless 71 | port: 55555 72 | targetPort: 0 73 | selector: 74 | app: postgres 75 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-secrets.template.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: retoolsecrets 5 | type: Opaque 6 | data: 7 | jwt_secret: {{ random base64 encoded string to sign jwt tokens }} 8 | encryption_key: {{ random base64 encoded string to encrypt database credentials }} 9 | postgres_password: {{ random base64 encoded string to set as the internal retool db password }} 10 | license_key: {{ base64 encoded string of the license key Retool will provide you }} 11 | google_client_id: {{ google client id encoded in base64 }} 12 | google_client_secret: {{ google client secret encoded in base64 }} 13 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-workflows-backend-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: workflows-api 6 | name: workflows-api 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: workflows-api 12 | template: 13 | metadata: 14 | labels: 15 | app: workflows-api 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: DEPLOYMENT_TEMPLATE_TYPE 25 | value: k8s-manifests 26 | - name: JWT_SECRET 27 | valueFrom: 28 | secretKeyRef: 29 | name: retoolsecrets 30 | key: jwt_secret 31 | - name: SERVICE_TYPE 32 | value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 69 | value: "retool-temporal-frontend" 70 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 71 | value: "7233" 72 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 73 | value: "workflows" 74 | - name: WORKFLOW_BACKEND_HOST 75 | value: http://workflows-api 76 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 77 | value: http://code-executor:3004 78 | image: tryretool/backend:X.Y.Z 79 | name: workflows-api 80 | ports: 81 | - containerPort: 3000 82 | resources: 83 | limits: 84 | memory: 8Gi 85 | cpu: 2 86 | requests: 87 | cpu: 1 88 | memory: 4Gi 89 | restartPolicy: Always 90 | --- 91 | apiVersion: v1 92 | kind: Service 93 | metadata: 94 | labels: 95 | app: workflows-api 96 | name: workflows-api 97 | spec: 98 | type: LoadBalancer 99 | ports: 100 | - protocol: TCP 101 | port: 80 102 | targetPort: 3000 103 | selector: 104 | app: workflows-api 105 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/retool-workflows-worker-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | retoolService: workflow-worker 6 | name: workflow-worker 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | retoolService: workflow-worker 12 | revisionHistoryLimit: 3 13 | template: 14 | metadata: 15 | labels: 16 | retoolService: workflow-worker 17 | spec: 18 | containers: 19 | - args: 20 | - bash 21 | - -c 22 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 23 | ./docker_scripts/start_api.sh 24 | env: 25 | - name: DEPLOYMENT_TEMPLATE_TYPE 26 | value: k8s-manifests 27 | - name: JWT_SECRET 28 | valueFrom: 29 | secretKeyRef: 30 | name: retoolsecrets 31 | key: jwt_secret 32 | - name: SERVICE_TYPE 33 | value: WORKFLOW_TEMPORAL_WORKER 34 | - name: NODE_ENV 35 | value: production 36 | - name: POSTGRES_DB 37 | value: hammerhead_production 38 | - name: POSTGRES_HOST 39 | value: postgres 40 | - name: POSTGRES_PORT 41 | value: "5432" 42 | - name: POSTGRES_USER 43 | value: retool_internal_user 44 | - name: POSTGRES_PASSWORD 45 | valueFrom: 46 | secretKeyRef: 47 | name: retoolsecrets 48 | key: postgres_password 49 | - name: ENCRYPTION_KEY 50 | valueFrom: 51 | secretKeyRef: 52 | name: retoolsecrets 53 | key: encryption_key 54 | - name: LICENSE_KEY 55 | valueFrom: 56 | secretKeyRef: 57 | name: retoolsecrets 58 | key: license_key 59 | - name: CLIENT_ID 60 | valueFrom: 61 | secretKeyRef: 62 | name: retoolsecrets 63 | key: google_client_id 64 | - name: CLIENT_SECRET 65 | valueFrom: 66 | secretKeyRef: 67 | name: retoolsecrets 68 | key: google_client_secret 69 | - name: NODE_OPTIONS 70 | value: --max_old_space_size=1024 71 | - name: DISABLE_DATABASE_MIGRATIONS 72 | value: "true" 73 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST 74 | value: "retool-temporal-frontend" 75 | - name: WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT 76 | value: "7233" 77 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 78 | value: "workflows" 79 | - name: WORKFLOW_BACKEND_HOST 80 | value: http://workflows-api 81 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 82 | value: http://code-executor:3004 83 | image: tryretool/backend:X.Y.Z 84 | name: workflow-worker 85 | ports: 86 | - containerPort: 3005 87 | name: retool 88 | protocol: TCP 89 | - containerPort: 9090 90 | name: metrics 91 | protocol: TCP 92 | resources: 93 | limits: 94 | cpu: 2 95 | memory: 2Gi 96 | requests: 97 | cpu: 1 98 | memory: 1Gi 99 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License 2 | 3 | Copyright (c) 2020 Temporal Technologies Inc. All rights reserved. 4 | 5 | Copyright (c) 2020 Uber Technologies, Inc. 6 | 7 | Permission is hereby granted, free of charge, to any person obtaining a copy 8 | of this software and associated documentation files (the "Software"), to deal 9 | in the Software without restriction, including without limitation the rights 10 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | copies of the Software, and to permit persons to whom the Software is 12 | furnished to do so, subject to the following conditions: 13 | 14 | The above copyright notice and this permission notice shall be included in 15 | all copies or substantial portions of the Software. 16 | 17 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 23 | THE SOFTWARE. -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/retool-temporal-secrets.template.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: retooltemporalsecrets 5 | type: Opaque 6 | data: 7 | postgres_password: {{ random base64 encoded string to set as the internal temporal db password }} -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-configmaps.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ConfigMap 3 | metadata: 4 | name: "retool-temporal-dynamic-config" 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | data: 9 | dynamic_config.yaml: |- 10 | limit.maxIDLength: 11 | - value: 255 12 | constraints: {} 13 | --- 14 | apiVersion: v1 15 | kind: ConfigMap 16 | metadata: 17 | name: "retool-temporal-server-config" 18 | data: 19 | config_template.yaml: |- 20 | log: 21 | stdout: true 22 | level: "debug,info" 23 | 24 | persistence: 25 | defaultStore: default 26 | visibilityStore: visibility 27 | numHistoryShards: 128 28 | datastores: 29 | default: 30 | sql: 31 | pluginName: "postgres" 32 | driverName: "postgres" 33 | databaseName: "temporal" 34 | connectAddr: "postgres:5432" 35 | connectProtocol: "tcp" 36 | user: "retool_internal_user" 37 | password: "{{ .Env.TEMPORAL_STORE_PASSWORD }}" 38 | maxConnLifetime: 1h 39 | maxConns: 20 40 | visibility: 41 | sql: 42 | pluginName: "postgres" 43 | driverName: "postgres" 44 | databaseName: "temporal_visibility" 45 | connectAddr: "postgres:5432" 46 | connectProtocol: "tcp" 47 | user: "retool_internal_user" 48 | password: "{{ .Env.TEMPORAL_VISIBILITY_STORE_PASSWORD }}" 49 | maxConnLifetime: 1h 50 | maxConns: 20 51 | 52 | global: 53 | membership: 54 | name: temporal 55 | maxJoinDuration: 30s 56 | broadcastAddress: {{ default .Env.POD_IP "0.0.0.0" }} 57 | 58 | pprof: 59 | port: 7936 60 | 61 | services: 62 | frontend: 63 | rpc: 64 | grpcPort: 7233 65 | membershipPort: 6933 66 | bindOnIP: "0.0.0.0" 67 | 68 | history: 69 | rpc: 70 | grpcPort: 7234 71 | membershipPort: 6934 72 | bindOnIP: "0.0.0.0" 73 | 74 | matching: 75 | rpc: 76 | grpcPort: 7235 77 | membershipPort: 6935 78 | bindOnIP: "0.0.0.0" 79 | 80 | worker: 81 | rpc: 82 | grpcPort: 7239 83 | membershipPort: 6939 84 | bindOnIP: "0.0.0.0" 85 | clusterMetadata: 86 | enableGlobalDomain: false 87 | failoverVersionIncrement: 10 88 | masterClusterName: "active" 89 | currentClusterName: "active" 90 | clusterInformation: 91 | active: 92 | enabled: true 93 | initialFailoverVersion: 1 94 | rpcName: "temporal-frontend" 95 | rpcAddress: "127.0.0.1:7933" 96 | 97 | dcRedirectionPolicy: 98 | policy: "noop" 99 | toDC: "" 100 | 101 | archival: 102 | status: "disabled" 103 | 104 | publicClient: 105 | hostPort: "retool-temporal-frontend:7233" 106 | 107 | dynamicConfigClient: 108 | filepath: "/etc/temporal/dynamic_config/dynamic_config.yaml" 109 | pollInterval: "10s" 110 | -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-debugging-containers.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ConfigMap 3 | metadata: 4 | name: retool-temporal-web-config 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | app.kubernetes.io/component: web 9 | data: 10 | config.yml: | 11 | auth: 12 | enabled: false 13 | routing: 14 | default_to_namespace: workflows 15 | --- 16 | apiVersion: apps/v1 17 | kind: Deployment 18 | metadata: 19 | name: retool-temporal-web 20 | labels: 21 | app.kubernetes.io/name: retool-temporal 22 | app.kubernetes.io/instance: retool 23 | app.kubernetes.io/component: web 24 | spec: 25 | replicas: 1 26 | selector: 27 | matchLabels: 28 | app.kubernetes.io/name: retool-temporal 29 | app.kubernetes.io/instance: retool 30 | app.kubernetes.io/component: web 31 | template: 32 | metadata: 33 | labels: 34 | app.kubernetes.io/name: retool-temporal 35 | app.kubernetes.io/instance: retool 36 | app.kubernetes.io/component: web 37 | spec: 38 | 39 | volumes: 40 | - name: retool-temporal-web-config 41 | configMap: 42 | name: retool-temporal-web-config 43 | containers: 44 | - name: retool-temporal-web 45 | image: "temporalio/ui:latest" 46 | imagePullPolicy: IfNotPresent 47 | env: 48 | - name: TEMPORAL_ADDRESS 49 | value: "retool-temporal-frontend:7233" 50 | ports: 51 | - name: http 52 | containerPort: 8080 53 | protocol: TCP 54 | resources: 55 | limits: 56 | cpu: 100m 57 | memory: 128Mi 58 | --- 59 | apiVersion: v1 60 | kind: Service 61 | metadata: 62 | name: retool-temporal-web 63 | labels: 64 | app.kubernetes.io/name: retool-temporal 65 | app.kubernetes.io/instance: retool 66 | app.kubernetes.io/component: web 67 | spec: 68 | type: ClusterIP 69 | ports: 70 | - port: 8080 71 | targetPort: 8080 72 | protocol: TCP 73 | name: http 74 | 75 | selector: 76 | app.kubernetes.io/name: retool-temporal 77 | app.kubernetes.io/instance: retool 78 | app.kubernetes.io/component: web 79 | --- 80 | apiVersion: apps/v1 81 | kind: Deployment 82 | metadata: 83 | name: retool-temporal-admintools 84 | labels: 85 | app.kubernetes.io/name: retool-temporal 86 | app.kubernetes.io/instance: retool 87 | app.kubernetes.io/component: admintools 88 | spec: 89 | replicas: 1 90 | selector: 91 | matchLabels: 92 | app.kubernetes.io/name: retool-temporal 93 | app.kubernetes.io/instance: retool 94 | app.kubernetes.io/component: admintools 95 | template: 96 | metadata: 97 | labels: 98 | app.kubernetes.io/name: retool-temporal 99 | app.kubernetes.io/instance: retool 100 | app.kubernetes.io/component: admintools 101 | spec: 102 | 103 | containers: 104 | - name: admin-tools 105 | image: "temporalio/admin-tools:1.18.5" 106 | imagePullPolicy: IfNotPresent 107 | ports: 108 | - name: http 109 | containerPort: 22 110 | protocol: TCP 111 | env: 112 | - name: TEMPORAL_CLI_ADDRESS 113 | value: retool-temporal-frontend:7233 114 | livenessProbe: 115 | exec: 116 | command: 117 | - ls 118 | - / 119 | initialDelaySeconds: 5 120 | periodSeconds: 5 121 | resources: 122 | limits: 123 | cpu: 100m 124 | memory: 128Mi 125 | --- 126 | apiVersion: v1 127 | kind: Service 128 | metadata: 129 | name: retool-temporal-admintools 130 | labels: 131 | app.kubernetes.io/name: retool-temporal 132 | app.kubernetes.io/instance: retool 133 | app.kubernetes.io/component: admintools 134 | spec: 135 | type: ClusterIP 136 | ports: 137 | - port: 22 138 | targetPort: 22 139 | protocol: TCP 140 | name: ssh 141 | 142 | selector: 143 | app.kubernetes.io/name: retool-temporal 144 | app.kubernetes.io/instance: retool 145 | app.kubernetes.io/component: admintools -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-frontend-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: retool-temporal-frontend 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | app.kubernetes.io/component: frontend 9 | spec: 10 | type: ClusterIP 11 | ports: 12 | - port: 7233 13 | targetPort: rpc 14 | protocol: TCP 15 | name: grpc-rpc 16 | selector: 17 | app.kubernetes.io/name: retool-temporal 18 | app.kubernetes.io/instance: retool 19 | app.kubernetes.io/component: frontend 20 | --- 21 | apiVersion: v1 22 | kind: Service 23 | metadata: 24 | name: retool-temporal-frontend-headless 25 | labels: 26 | app.kubernetes.io/name: retool-temporal 27 | app.kubernetes.io/instance: retool 28 | app.kubernetes.io/component: frontend 29 | app.kubernetes.io/headless: 'true' 30 | 31 | annotations: 32 | # Use this annotation in addition to the actual field below because the 33 | # annotation will stop being respected soon but the field is broken in 34 | # some versions of Kubernetes: 35 | # https://github.com/kubernetes/kubernetes/issues/58662 36 | service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" 37 | spec: 38 | type: ClusterIP 39 | clusterIP: None 40 | publishNotReadyAddresses: true 41 | ports: 42 | - port: 7233 43 | targetPort: rpc 44 | protocol: TCP 45 | name: grpc-rpc 46 | selector: 47 | app.kubernetes.io/name: retool-temporal 48 | app.kubernetes.io/instance: retool 49 | app.kubernetes.io/component: frontend 50 | --- 51 | apiVersion: apps/v1 52 | kind: Deployment 53 | metadata: 54 | name: retool-temporal-frontend 55 | labels: 56 | app.kubernetes.io/name: retool-temporal 57 | app.kubernetes.io/instance: retool 58 | app.kubernetes.io/component: frontend 59 | spec: 60 | replicas: 1 61 | selector: 62 | matchLabels: 63 | app.kubernetes.io/name: retool-temporal 64 | app.kubernetes.io/instance: retool 65 | app.kubernetes.io/component: frontend 66 | template: 67 | metadata: 68 | labels: 69 | app.kubernetes.io/name: retool-temporal 70 | app.kubernetes.io/instance: retool 71 | app.kubernetes.io/component: frontend 72 | spec: 73 | securityContext: 74 | fsGroup: 1000 #temporal group 75 | runAsUser: 1000 #temporal user 76 | containers: 77 | - name: retool-temporal-frontend 78 | image: "tryretool/one-offs:retool-temporal-1.1.6" 79 | imagePullPolicy: IfNotPresent 80 | env: 81 | - name: POD_IP 82 | valueFrom: 83 | fieldRef: 84 | fieldPath: status.podIP 85 | - name: ENABLE_ES 86 | value: "false" 87 | - name: SERVICES 88 | value: frontend 89 | - name: DEFAULT_NAMESPACE 90 | value: workflows 91 | - name: DB 92 | value: postgresql 93 | - name: DBNAME 94 | value: "temporal" 95 | - name: VISIBILITY_DBNAME 96 | value: "temporal_visibility" 97 | - name: POSTGRES_HOST 98 | value: postgres 99 | - name: POSTGRES_PORT 100 | value: "5432" 101 | - name: POSTGRES_USER 102 | value: "retool_internal_user" 103 | - name: POSTGRES_PASSWORD 104 | valueFrom: 105 | secretKeyRef: 106 | name: retooltemporalsecrets 107 | key: postgres_password 108 | - name: TEMPORAL_STORE_PASSWORD 109 | valueFrom: 110 | secretKeyRef: 111 | name: retooltemporalsecrets 112 | key: postgres_password 113 | - name: TEMPORAL_VISIBILITY_STORE_PASSWORD 114 | valueFrom: 115 | secretKeyRef: 116 | name: retooltemporalsecrets 117 | key: postgres_password 118 | ports: 119 | - name: rpc 120 | containerPort: 7233 121 | protocol: TCP 122 | livenessProbe: 123 | initialDelaySeconds: 150 124 | tcpSocket: 125 | port: rpc 126 | volumeMounts: 127 | - name: config 128 | mountPath: /etc/temporal/config/config_template.yaml 129 | subPath: config_template.yaml 130 | - name: dynamic-config 131 | mountPath: /etc/temporal/dynamic_config 132 | resources: 133 | limits: 134 | cpu: 500m 135 | memory: 1024Mi 136 | requests: 137 | cpu: 100m 138 | memory: 128Mi 139 | volumes: 140 | - name: config 141 | configMap: 142 | name: "retool-temporal-server-config" 143 | - name: dynamic-config 144 | configMap: 145 | name: "retool-temporal-dynamic-config" 146 | items: 147 | - key: dynamic_config.yaml 148 | path: dynamic_config.yaml -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-history-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: retool-temporal-history-headless 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | app.kubernetes.io/component: history 9 | app.kubernetes.io/headless: 'true' 10 | 11 | annotations: 12 | # Use this annotation in addition to the actual field below because the 13 | # annotation will stop being respected soon but the field is broken in 14 | # some versions of Kubernetes: 15 | # https://github.com/kubernetes/kubernetes/issues/58662 16 | service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" 17 | spec: 18 | type: ClusterIP 19 | clusterIP: None 20 | publishNotReadyAddresses: true 21 | ports: 22 | - port: 7233 23 | targetPort: rpc 24 | protocol: TCP 25 | name: grpc-rpc 26 | selector: 27 | app.kubernetes.io/name: retool-temporal 28 | app.kubernetes.io/instance: retool 29 | app.kubernetes.io/component: history 30 | --- 31 | apiVersion: apps/v1 32 | kind: Deployment 33 | metadata: 34 | name: retool-temporal-history 35 | labels: 36 | app.kubernetes.io/name: retool-temporal 37 | app.kubernetes.io/instance: retool 38 | app.kubernetes.io/component: history 39 | spec: 40 | replicas: 1 41 | selector: 42 | matchLabels: 43 | app.kubernetes.io/name: retool-temporal 44 | app.kubernetes.io/instance: retool 45 | app.kubernetes.io/component: history 46 | template: 47 | metadata: 48 | labels: 49 | app.kubernetes.io/name: retool-temporal 50 | app.kubernetes.io/instance: retool 51 | app.kubernetes.io/component: history 52 | spec: 53 | securityContext: 54 | fsGroup: 1000 #temporal group 55 | runAsUser: 1000 #temporal user 56 | containers: 57 | - name: retool-temporal-history 58 | image: "tryretool/one-offs:retool-temporal-1.1.6" 59 | imagePullPolicy: IfNotPresent 60 | env: 61 | - name: POD_IP 62 | valueFrom: 63 | fieldRef: 64 | fieldPath: status.podIP 65 | - name: ENABLE_ES 66 | value: "false" 67 | - name: SERVICES 68 | value: history 69 | - name: DEFAULT_NAMESPACE 70 | value: workflows 71 | - name: DB 72 | value: postgresql 73 | - name: DBNAME 74 | value: "temporal" 75 | - name: VISIBILITY_DBNAME 76 | value: "temporal_visibility" 77 | - name: POSTGRES_HOST 78 | value: postgres 79 | - name: POSTGRES_PORT 80 | value: "5432" 81 | - name: POSTGRES_USER 82 | value: "retool_internal_user" 83 | - name: POSTGRES_PASSWORD 84 | valueFrom: 85 | secretKeyRef: 86 | name: retooltemporalsecrets 87 | key: postgres_password 88 | - name: TEMPORAL_CLI_ADDRESS 89 | value: retool-temporal-frontend:7233 90 | - name: TEMPORAL_STORE_PASSWORD 91 | valueFrom: 92 | secretKeyRef: 93 | name: retooltemporalsecrets 94 | key: postgres_password 95 | - name: TEMPORAL_VISIBILITY_STORE_PASSWORD 96 | valueFrom: 97 | secretKeyRef: 98 | name: retooltemporalsecrets 99 | key: postgres_password 100 | ports: 101 | - name: rpc 102 | containerPort: 7234 103 | protocol: TCP 104 | livenessProbe: 105 | initialDelaySeconds: 150 106 | tcpSocket: 107 | port: rpc 108 | volumeMounts: 109 | - name: config 110 | mountPath: /etc/temporal/config/config_template.yaml 111 | subPath: config_template.yaml 112 | - name: dynamic-config 113 | mountPath: /etc/temporal/dynamic_config 114 | resources: 115 | limits: 116 | cpu: 500m 117 | memory: 1024Mi 118 | requests: 119 | cpu: 100m 120 | memory: 128Mi 121 | volumes: 122 | - name: config 123 | configMap: 124 | name: "retool-temporal-server-config" 125 | - name: dynamic-config 126 | configMap: 127 | name: "retool-temporal-dynamic-config" 128 | items: 129 | - key: dynamic_config.yaml 130 | path: dynamic_config.yaml -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-matching-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: retool-temporal-matching-headless 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | app.kubernetes.io/component: matching 9 | app.kubernetes.io/headless: 'true' 10 | 11 | annotations: 12 | # Use this annotation in addition to the actual field below because the 13 | # annotation will stop being respected soon but the field is broken in 14 | # some versions of Kubernetes: 15 | # https://github.com/kubernetes/kubernetes/issues/58662 16 | service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" 17 | spec: 18 | type: ClusterIP 19 | clusterIP: None 20 | publishNotReadyAddresses: true 21 | ports: 22 | - port: 7233 23 | targetPort: rpc 24 | protocol: TCP 25 | name: grpc-rpc 26 | selector: 27 | app.kubernetes.io/name: retool-temporal 28 | app.kubernetes.io/instance: retool 29 | app.kubernetes.io/component: matching 30 | --- 31 | apiVersion: apps/v1 32 | kind: Deployment 33 | metadata: 34 | name: retool-temporal-matching 35 | labels: 36 | app.kubernetes.io/name: retool-temporal 37 | app.kubernetes.io/instance: retool 38 | app.kubernetes.io/component: matching 39 | spec: 40 | replicas: 1 41 | selector: 42 | matchLabels: 43 | app.kubernetes.io/name: retool-temporal 44 | app.kubernetes.io/instance: retool 45 | app.kubernetes.io/component: matching 46 | template: 47 | metadata: 48 | labels: 49 | app.kubernetes.io/name: retool-temporal 50 | app.kubernetes.io/instance: retool 51 | app.kubernetes.io/component: matching 52 | spec: 53 | securityContext: 54 | fsGroup: 1000 #temporal group 55 | runAsUser: 1000 #temporal user 56 | containers: 57 | - name: retool-temporal-matching 58 | image: "tryretool/one-offs:retool-temporal-1.1.6" 59 | imagePullPolicy: IfNotPresent 60 | env: 61 | - name: POD_IP 62 | valueFrom: 63 | fieldRef: 64 | fieldPath: status.podIP 65 | - name: ENABLE_ES 66 | value: "false" 67 | - name: SERVICES 68 | value: matching 69 | - name: DEFAULT_NAMESPACE 70 | value: workflows 71 | - name: DB 72 | value: postgresql 73 | - name: DBNAME 74 | value: "temporal" 75 | - name: VISIBILITY_DBNAME 76 | value: "temporal_visibility" 77 | - name: POSTGRES_HOST 78 | value: postgres 79 | - name: POSTGRES_PORT 80 | value: "5432" 81 | - name: POSTGRES_USER 82 | value: "retool_internal_user" 83 | - name: POSTGRES_PASSWORD 84 | valueFrom: 85 | secretKeyRef: 86 | name: retooltemporalsecrets 87 | key: postgres_password 88 | - name: TEMPORAL_CLI_ADDRESS 89 | value: retool-temporal-frontend:7233 90 | - name: TEMPORAL_STORE_PASSWORD 91 | valueFrom: 92 | secretKeyRef: 93 | name: retooltemporalsecrets 94 | key: postgres_password 95 | - name: TEMPORAL_VISIBILITY_STORE_PASSWORD 96 | valueFrom: 97 | secretKeyRef: 98 | name: retooltemporalsecrets 99 | key: postgres_password 100 | ports: 101 | - name: rpc 102 | containerPort: 7235 103 | protocol: TCP 104 | livenessProbe: 105 | initialDelaySeconds: 150 106 | tcpSocket: 107 | port: rpc 108 | volumeMounts: 109 | - name: config 110 | mountPath: /etc/temporal/config/config_template.yaml 111 | subPath: config_template.yaml 112 | - name: dynamic-config 113 | mountPath: /etc/temporal/dynamic_config 114 | resources: 115 | limits: 116 | cpu: 500m 117 | memory: 1024Mi 118 | requests: 119 | cpu: 100m 120 | memory: 128Mi 121 | volumes: 122 | - name: config 123 | configMap: 124 | name: "retool-temporal-server-config" 125 | - name: dynamic-config 126 | configMap: 127 | name: "retool-temporal-dynamic-config" 128 | items: 129 | - key: dynamic_config.yaml 130 | path: dynamic_config.yaml -------------------------------------------------------------------------------- /kubernetes-with-temporal/temporal/temporal-worker-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: retool-temporal-worker-headless 5 | labels: 6 | app.kubernetes.io/name: retool-temporal 7 | app.kubernetes.io/instance: retool 8 | app.kubernetes.io/component: worker 9 | app.kubernetes.io/headless: 'true' 10 | 11 | annotations: 12 | # Use this annotation in addition to the actual field below because the 13 | # annotation will stop being respected soon but the field is broken in 14 | # some versions of Kubernetes: 15 | # https://github.com/kubernetes/kubernetes/issues/58662 16 | service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" 17 | spec: 18 | type: ClusterIP 19 | clusterIP: None 20 | publishNotReadyAddresses: true 21 | ports: 22 | - port: 7233 23 | targetPort: rpc 24 | protocol: TCP 25 | name: grpc-rpc 26 | selector: 27 | app.kubernetes.io/name: retool-temporal 28 | app.kubernetes.io/instance: retool 29 | app.kubernetes.io/component: worker 30 | --- 31 | apiVersion: apps/v1 32 | kind: Deployment 33 | metadata: 34 | name: retool-temporal-worker 35 | labels: 36 | app.kubernetes.io/name: retool-temporal 37 | app.kubernetes.io/instance: retool 38 | app.kubernetes.io/component: worker 39 | spec: 40 | replicas: 1 41 | selector: 42 | matchLabels: 43 | app.kubernetes.io/name: retool-temporal 44 | app.kubernetes.io/instance: retool 45 | app.kubernetes.io/component: worker 46 | template: 47 | metadata: 48 | labels: 49 | app.kubernetes.io/name: retool-temporal 50 | app.kubernetes.io/instance: retool 51 | app.kubernetes.io/component: worker 52 | spec: 53 | securityContext: 54 | fsGroup: 1000 #temporal group 55 | runAsUser: 1000 #temporal user 56 | containers: 57 | - name: retool-temporal-worker 58 | image: "tryretool/one-offs:retool-temporal-1.1.6" 59 | imagePullPolicy: IfNotPresent 60 | env: 61 | - name: POD_IP 62 | valueFrom: 63 | fieldRef: 64 | fieldPath: status.podIP 65 | - name: ENABLE_ES 66 | value: "false" 67 | - name: SERVICES 68 | value: worker 69 | - name: DEFAULT_NAMESPACE 70 | value: workflows 71 | - name: DB 72 | value: postgresql 73 | - name: DBNAME 74 | value: "temporal" 75 | - name: VISIBILITY_DBNAME 76 | value: "temporal_visibility" 77 | - name: POSTGRES_HOST 78 | value: postgres 79 | - name: POSTGRES_PORT 80 | value: "5432" 81 | - name: POSTGRES_USER 82 | value: "retool_internal_user" 83 | - name: POSTGRES_PASSWORD 84 | valueFrom: 85 | secretKeyRef: 86 | name: retooltemporalsecrets 87 | key: postgres_password 88 | - name: TEMPORAL_CLI_ADDRESS 89 | value: retool-temporal-frontend:7233 90 | - name: TEMPORAL_STORE_PASSWORD 91 | valueFrom: 92 | secretKeyRef: 93 | name: retooltemporalsecrets 94 | key: postgres_password 95 | - name: TEMPORAL_VISIBILITY_STORE_PASSWORD 96 | valueFrom: 97 | secretKeyRef: 98 | name: retooltemporalsecrets 99 | key: postgres_password 100 | ports: 101 | - name: rpc 102 | containerPort: 7239 103 | protocol: TCP 104 | volumeMounts: 105 | - name: config 106 | mountPath: /etc/temporal/config/config_template.yaml 107 | subPath: config_template.yaml 108 | - name: dynamic-config 109 | mountPath: /etc/temporal/dynamic_config 110 | resources: 111 | limits: 112 | cpu: 500m 113 | memory: 1024Mi 114 | requests: 115 | cpu: 100m 116 | memory: 128Mi 117 | volumes: 118 | - name: config 119 | configMap: 120 | name: "retool-temporal-server-config" 121 | - name: dynamic-config 122 | configMap: 123 | name: "retool-temporal-dynamic-config" 124 | items: 125 | - key: dynamic_config.yaml 126 | path: dynamic_config.yaml -------------------------------------------------------------------------------- /kubernetes/retool-code-executor-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | retoolService: code-executor 6 | name: code-executor 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | retoolService: code-executor 12 | revisionHistoryLimit: 3 13 | template: 14 | metadata: 15 | labels: 16 | retoolService: code-executor 17 | spec: 18 | containers: 19 | - args: 20 | - bash 21 | - start.sh 22 | env: 23 | - name: DEPLOYMENT_TEMPLATE_TYPE 24 | value: k8s-manifests 25 | - name: NODE_ENV 26 | value: production 27 | - name: NODE_OPTIONS 28 | value: --max_old_space_size=1024 29 | - name: PORT 30 | value: "3004" 31 | image: tryretool/code-executor-service:X.Y.Z 32 | name: code-executor 33 | ports: 34 | - containerPort: 3004 35 | name: retool 36 | protocol: TCP 37 | resources: 38 | limits: 39 | cpu: 2 40 | memory: 4Gi 41 | requests: 42 | cpu: 1 43 | memory: 1Gi 44 | # code executor uses nsjail to sandbox code execution. nsjail requires privileged container access. 45 | # If your deployment does not support privileged access, you can set this to false to not use nsjail. 46 | # Without nsjail, all code is run without sandboxing within your deployment. 47 | securityContext: 48 | privileged: true 49 | --- 50 | apiVersion: v1 51 | kind: Service 52 | metadata: 53 | name: code-executor 54 | spec: 55 | selector: 56 | retoolService: code-executor 57 | ports: 58 | - protocol: TCP 59 | port: 80 60 | targetPort: 3004 61 | name: http-server 62 | -------------------------------------------------------------------------------- /kubernetes/retool-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: api 6 | name: api 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: api 12 | template: 13 | metadata: 14 | labels: 15 | app: api 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: JWT_SECRET 25 | valueFrom: 26 | secretKeyRef: 27 | name: retoolsecrets 28 | key: jwt_secret 29 | - name: DEPLOYMENT_TEMPLATE_TYPE 30 | value: k8s-manifests 31 | - name: SERVICE_TYPE 32 | value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 69 | value: "workflows" 70 | - name: WORKFLOW_BACKEND_HOST 71 | value: http://workflows-api 72 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 73 | value: http://code-executor:3004 74 | image: tryretool/backend:X.Y.Z 75 | name: api 76 | ports: 77 | - containerPort: 3000 78 | resources: 79 | limits: 80 | memory: 8Gi 81 | cpu: 2 82 | requests: 83 | cpu: 1 84 | memory: 4Gi 85 | volumeMounts: 86 | - name: retool-pv 87 | mountPath: /retool_backend/pv-data 88 | restartPolicy: Always 89 | volumes: 90 | - name: retool-pv 91 | persistentVolumeClaim: 92 | claimName: retool-pvc 93 | --- 94 | apiVersion: v1 95 | kind: Service 96 | metadata: 97 | labels: 98 | app: api 99 | name: api 100 | spec: 101 | type: LoadBalancer 102 | ports: 103 | - name: "3000" 104 | port: 3000 105 | targetPort: 3000 106 | selector: 107 | app: api 108 | --- 109 | apiVersion: v1 110 | kind: PersistentVolumeClaim 111 | metadata: 112 | name: retool-pvc 113 | spec: 114 | accessModes: 115 | - ReadWriteOnce 116 | resources: 117 | requests: 118 | storage: 1Gi 119 | -------------------------------------------------------------------------------- /kubernetes/retool-jobs-runner.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: jobs-runner 6 | name: jobs-runner 7 | spec: 8 | replicas: 1 # jobs-runner pod should never be more than 1 9 | selector: 10 | matchLabels: 11 | app: jobs-runner 12 | template: 13 | metadata: 14 | labels: 15 | app: jobs-runner 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: JWT_SECRET 25 | valueFrom: 26 | secretKeyRef: 27 | name: retoolsecrets 28 | key: jwt_secret 29 | - name: DEPLOYMENT_TEMPLATE_TYPE 30 | value: k8s-manifests 31 | - name: SERVICE_TYPE 32 | value: JOBS_RUNNER 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | image: tryretool/backend:X.Y.Z 69 | name: jobs-runner 70 | ports: 71 | - containerPort: 3000 72 | resources: 73 | limits: 74 | cpu: 4 75 | memory: 8Gi 76 | requests: 77 | cpu: 2 78 | memory: 4Gi 79 | restartPolicy: Always 80 | -------------------------------------------------------------------------------- /kubernetes/retool-postgres.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | name: postgres-pvc 5 | labels: 6 | name: postgres-pvc 7 | spec: 8 | accessModes: 9 | - ReadWriteOnce 10 | resources: 11 | requests: 12 | storage: 60Gi 13 | 14 | --- 15 | apiVersion: apps/v1 16 | kind: Deployment 17 | metadata: 18 | name: postgres 19 | labels: 20 | app: postgres 21 | spec: 22 | replicas: 1 23 | selector: 24 | matchLabels: 25 | app: postgres 26 | strategy: 27 | type: Recreate 28 | template: 29 | metadata: 30 | labels: 31 | app: postgres 32 | spec: 33 | containers: 34 | - env: 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | name: postgres 49 | image: postgres:11.13 50 | resources: {} 51 | volumeMounts: 52 | - mountPath: /var/lib/postgresql/data 53 | subPath: postgres 54 | name: postgres-pv 55 | restartPolicy: Always 56 | volumes: 57 | - name: postgres-pv 58 | persistentVolumeClaim: 59 | claimName: postgres-pvc 60 | --- 61 | apiVersion: v1 62 | kind: Service 63 | metadata: 64 | labels: 65 | app: postgres 66 | name: postgres 67 | spec: 68 | clusterIP: None 69 | ports: 70 | - name: headless 71 | port: 55555 72 | targetPort: 0 73 | selector: 74 | app: postgres 75 | -------------------------------------------------------------------------------- /kubernetes/retool-secrets.template.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Secret 3 | metadata: 4 | name: retoolsecrets 5 | type: Opaque 6 | data: 7 | jwt_secret: {{ random base64 encoded string to sign jwt tokens }} 8 | encryption_key: {{ random base64 encoded string to encrypt database credentials }} 9 | postgres_password: {{ random base64 encoded string to set as the internal retool db password }} 10 | license_key: {{ base64 encoded string of the license key Retool will provide you }} 11 | google_client_id: {{ google client id encoded in base64 }} 12 | google_client_secret: {{ google client secret encoded in base64 }} 13 | -------------------------------------------------------------------------------- /kubernetes/retool-workflows-backend-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | app: workflows-api 6 | name: workflows-api 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: workflows-api 12 | template: 13 | metadata: 14 | labels: 15 | app: workflows-api 16 | spec: 17 | containers: 18 | - args: 19 | - bash 20 | - -c 21 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 22 | ./docker_scripts/start_api.sh 23 | env: 24 | - name: JWT_SECRET 25 | valueFrom: 26 | secretKeyRef: 27 | name: retoolsecrets 28 | key: jwt_secret 29 | - name: DEPLOYMENT_TEMPLATE_TYPE 30 | value: k8s-manifests 31 | - name: SERVICE_TYPE 32 | value: MAIN_BACKEND,DB_CONNECTOR,DB_SSH_CONNECTOR 33 | - name: NODE_ENV 34 | value: production 35 | - name: POSTGRES_DB 36 | value: hammerhead_production 37 | - name: POSTGRES_HOST 38 | value: postgres 39 | - name: POSTGRES_PORT 40 | value: "5432" 41 | - name: POSTGRES_USER 42 | value: retool_internal_user 43 | - name: POSTGRES_PASSWORD 44 | valueFrom: 45 | secretKeyRef: 46 | name: retoolsecrets 47 | key: postgres_password 48 | - name: ENCRYPTION_KEY 49 | valueFrom: 50 | secretKeyRef: 51 | name: retoolsecrets 52 | key: encryption_key 53 | - name: LICENSE_KEY 54 | valueFrom: 55 | secretKeyRef: 56 | name: retoolsecrets 57 | key: license_key 58 | - name: CLIENT_ID 59 | valueFrom: 60 | secretKeyRef: 61 | name: retoolsecrets 62 | key: google_client_id 63 | - name: CLIENT_SECRET 64 | valueFrom: 65 | secretKeyRef: 66 | name: retoolsecrets 67 | key: google_client_secret 68 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 69 | value: "workflows" 70 | - name: WORKFLOW_BACKEND_HOST 71 | value: http://workflows-api 72 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 73 | value: http://code-executor:3004 74 | image: tryretool/backend:X.Y.Z 75 | name: workflows-api 76 | ports: 77 | - containerPort: 3000 78 | resources: 79 | limits: 80 | memory: 8Gi 81 | cpu: 2 82 | requests: 83 | cpu: 1 84 | memory: 4Gi 85 | restartPolicy: Always 86 | --- 87 | apiVersion: v1 88 | kind: Service 89 | metadata: 90 | labels: 91 | app: workflows-api 92 | name: workflows-api 93 | spec: 94 | type: LoadBalancer 95 | ports: 96 | - protocol: TCP 97 | port: 80 98 | targetPort: 3000 99 | selector: 100 | app: workflows-api 101 | -------------------------------------------------------------------------------- /kubernetes/retool-workflows-worker-container.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | labels: 5 | retoolService: workflow-worker 6 | name: workflow-worker 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | retoolService: workflow-worker 12 | revisionHistoryLimit: 3 13 | template: 14 | metadata: 15 | labels: 16 | retoolService: workflow-worker 17 | spec: 18 | containers: 19 | - args: 20 | - bash 21 | - -c 22 | - ./docker_scripts/wait-for-it.sh -t 0 $POSTGRES_HOST:$POSTGRES_PORT; 23 | ./docker_scripts/start_api.sh 24 | env: 25 | - name: JWT_SECRET 26 | valueFrom: 27 | secretKeyRef: 28 | name: retoolsecrets 29 | key: jwt_secret 30 | - name: DEPLOYMENT_TEMPLATE_TYPE 31 | value: k8s-manifests 32 | - name: SERVICE_TYPE 33 | value: WORKFLOW_TEMPORAL_WORKER 34 | - name: NODE_ENV 35 | value: production 36 | - name: POSTGRES_DB 37 | value: hammerhead_production 38 | - name: POSTGRES_HOST 39 | value: postgres 40 | - name: POSTGRES_PORT 41 | value: "5432" 42 | - name: POSTGRES_USER 43 | value: retool_internal_user 44 | - name: POSTGRES_PASSWORD 45 | valueFrom: 46 | secretKeyRef: 47 | name: retoolsecrets 48 | key: postgres_password 49 | - name: ENCRYPTION_KEY 50 | valueFrom: 51 | secretKeyRef: 52 | name: retoolsecrets 53 | key: encryption_key 54 | - name: LICENSE_KEY 55 | valueFrom: 56 | secretKeyRef: 57 | name: retoolsecrets 58 | key: license_key 59 | - name: CLIENT_ID 60 | valueFrom: 61 | secretKeyRef: 62 | name: retoolsecrets 63 | key: google_client_id 64 | - name: CLIENT_SECRET 65 | valueFrom: 66 | secretKeyRef: 67 | name: retoolsecrets 68 | key: google_client_secret 69 | - name: NODE_OPTIONS 70 | value: --max_old_space_size=1024 71 | - name: DISABLE_DATABASE_MIGRATIONS 72 | value: "true" 73 | - name: WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE 74 | value: "workflows" 75 | - name: WORKFLOW_BACKEND_HOST 76 | value: http://workflows-api 77 | - name: CODE_EXECUTOR_INGRESS_DOMAIN 78 | value: http://code-executor:3004 79 | image: tryretool/backend:X.Y.Z 80 | name: workflow-worker 81 | ports: 82 | - containerPort: 3005 83 | name: retool 84 | protocol: TCP 85 | - containerPort: 9090 86 | name: metrics 87 | protocol: TCP 88 | resources: 89 | limits: 90 | cpu: 2 91 | memory: 2Gi 92 | requests: 93 | cpu: 1 94 | memory: 1Gi 95 | 96 | -------------------------------------------------------------------------------- /temporal.yaml: -------------------------------------------------------------------------------- 1 | services: 2 | temporal: 3 | image: tryretool/one-offs:retool-temporal-1.1.6 4 | env_file: docker.env 5 | environment: 6 | # To enable TLS between temporal and external postgres, set both below variables to true 7 | - SQL_TLS_ENABLED=false 8 | - SQL_TLS=false 9 | # Defined twice because temporal-server and temporal-sql-tool use different envvars 10 | - SQL_TLS_DISABLE_HOST_VERIFICATION=true 11 | - SQL_HOST_VERIFICATION=false 12 | ports: 13 | - 7233:7233 14 | networks: 15 | - backend 16 | - temporal 17 | 18 | # Optional Temporal tools 19 | 20 | # temporal-admin-tools: 21 | # image: temporalio/admin-tools:1.18.5 22 | # environment: 23 | # - TEMPORAL_CLI_ADDRESS=temporal:7233 24 | # networks: 25 | # - temporal 26 | # depends_on: 27 | # - temporal 28 | # stdin_open: true 29 | # tty: true 30 | 31 | # temporal-ui: 32 | # image: temporalio/ui:2.9.1 33 | # environment: 34 | # - TEMPORAL_ADDRESS=temporal:7233 35 | # - TEMPORAL_CORS_ORIGINS=http://localhost:3000 36 | # networks: 37 | # - temporal 38 | # depends_on: 39 | # - temporal 40 | # ports: 41 | # - 8080:8080 42 | 43 | networks: 44 | temporal: 45 | -------------------------------------------------------------------------------- /upgrade.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "Downloading and building new images..." 4 | docker compose build 5 | 6 | echo "Bringing up new containers to replace existing ones..." 7 | docker compose up -d 8 | 9 | echo "Removing unused images..." 10 | docker image prune -a -f 11 | --------------------------------------------------------------------------------