├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── bin └── ecs-gpu-demo.ts ├── build_image.sh ├── cdk.json ├── image ├── Dockerfile └── test.py ├── img └── diagram.png ├── lib ├── adot-config.yaml └── ecs-gpu-demo-stack.ts ├── package.json ├── python3 ├── tsconfig.json └── yarn.lock /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazon ECS Auto Scaling for GPU-based Machine Learning Workloads 2 | 3 | This repository is intended for engineers looking to horizontally scale GPU-based Machine Learning (ML) workloads on Amazon ECS. This example is for demonstrative purposes only and is not intended for production use. 4 | 5 | ## How it works 6 | 7 | ![](/img/diagram.png) 8 | 9 | * By default, GPU utilization metrics are **not** part of the [predefined metrics](https://docs.aws.amazon.com/autoscaling/application/APIReference/API_PredefinedMetricSpecification.html) available with [Application Autoscaling](https://docs.aws.amazon.com/autoscaling/application/userguide/what-is-application-auto-scaling.html). 10 | 11 | * As such, you implement auto scaling based on custom metrics. See [Autoscaling Amazon ECS services based on custom metrics with Application Auto Scaling](https://aws.amazon.com/blogs/containers/autoscaling-amazon-ecs-services-based-on-custom-metrics-with-application-auto-scaling/) 12 | 13 | * For NVIDIA-based GPUs, you use [DCGM-Exporter](https://github.com/NVIDIA/dcgm-exporter) in your container to expose GPU metrics. You can then use metrics such as `DCGM_FI_DEV_GPU_UTIL` and `DCGM_FI_DEV_GPU_TEMP` to determine your auto scaling behavior. Learn more about [NVIDIA DGCM](https://developer.nvidia.com/dcgm). 14 | 15 | ## Setup 16 | 17 | - Fill the proper values on the `.env` file. 18 | 19 | - Install [AWS CDK](https://aws.amazon.com/getting-started/guides/setup-cdk/module-two/). 20 | 21 | - Use AWS CDK to deploy the AWS infrastructure. 22 | 23 | ``` 24 | cdk deploy --require-approval never 25 | ``` 26 | 27 | - Build and push image to Amazon ECR. 28 | 29 | ``` 30 | ./build_image.sh 31 | 32 | ``` 33 | 34 | - Open 2 terminal session and exec into the ECS task. 35 | 36 | ``` 37 | TASK_ARN= 38 | aws ecs execute-command \ 39 | --region us-east-1 \ 40 | --cluster ecs-gpu-demo \ 41 | --task ${TASK_ARN} \ 42 | --container gpu \ 43 | --command "/bin/bash" \ 44 | --interactive 45 | ``` 46 | 47 | - On one terminal, watch the GPU utilization. 48 | 49 | ``` 50 | watch -n0.1 nvidia-smi 51 | ``` 52 | 53 | - On the other terminal, stress test the GPU. 54 | 55 | ``` 56 | python3 test.py 57 | ``` 58 | 59 | ## Security 60 | 61 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 62 | 63 | ## License 64 | 65 | This library is licensed under the MIT-0 License. See the LICENSE file. -------------------------------------------------------------------------------- /bin/ecs-gpu-demo.ts: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env node 2 | import * as dotenv from "dotenv"; 3 | dotenv.config({ path: `.env`, override: true }); 4 | dotenv.config({ path: `.env.local`, override: true }); 5 | 6 | import "source-map-support/register"; 7 | import * as cdk from "aws-cdk-lib"; 8 | import { EcsGpuDemoStack } from "../lib/ecs-gpu-demo-stack"; 9 | 10 | const app = new cdk.App(); 11 | new EcsGpuDemoStack(app, "EcsGpuDemoStack", { 12 | env: { 13 | account: process.env.AWS_ACCOUNT_ID, 14 | region: process.env.AWS_REGION || "ap-southeast-1", 15 | }, 16 | }); 17 | -------------------------------------------------------------------------------- /build_image.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | export $(grep -v '^#' .env | xargs) 4 | export $(grep -v '^#' .env.local | xargs) 5 | 6 | aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com 7 | 8 | TAG=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest 9 | 10 | echo ${TAG} 11 | 12 | docker buildx create --use 13 | 14 | docker buildx build --platform linux/amd64 --push -t ${TAG} -f ./image/Dockerfile ./image 15 | -------------------------------------------------------------------------------- /cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "npx ts-node --prefer-ts-exts bin/ecs-gpu-demo.ts", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "**/*.d.ts", 11 | "**/*.js", 12 | "tsconfig.json", 13 | "package*.json", 14 | "yarn.lock", 15 | "node_modules", 16 | "test" 17 | ] 18 | }, 19 | "context": { 20 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 21 | "@aws-cdk/core:checkSecretUsage": true, 22 | "@aws-cdk/core:target-partitions": [ 23 | "aws", 24 | "aws-cn" 25 | ], 26 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 27 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 28 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 29 | "@aws-cdk/aws-iam:minimizePolicies": true, 30 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 31 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 32 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 33 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 34 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 35 | "@aws-cdk/core:enablePartitionLiterals": true, 36 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 37 | "@aws-cdk/aws-iam:standardizedServicePrincipals": true, 38 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 39 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 40 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 41 | "@aws-cdk/aws-route53-patters:useCertificate": true, 42 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 43 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 44 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 45 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 46 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 47 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 48 | "@aws-cdk/aws-redshift:columnId": true, 49 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true 50 | } 51 | } 52 | -------------------------------------------------------------------------------- /image/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM nvcr.io/nvidia/k8s/dcgm-exporter:3.3.0-3.2.0-ubuntu22.04 2 | 3 | WORKDIR /usr/src/app 4 | 5 | RUN apt-get install -y ca-certificates && \ 6 | apt-get update && \ 7 | apt-get install -y python3 python3-pip 8 | 9 | RUN pip install torch 10 | 11 | COPY test.py . -------------------------------------------------------------------------------- /image/test.py: -------------------------------------------------------------------------------- 1 | import torch 2 | 3 | x = torch.linspace(0, 4, 16 * 1024 ** 2).cuda() 4 | 5 | while True: 6 | x = x * (1.0 - x) 7 | -------------------------------------------------------------------------------- /img/diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/ecs-gpu-scaling/55721efb5e1bae7b82c5d695e840e0dfb0c1734b/img/diagram.png -------------------------------------------------------------------------------- /lib/adot-config.yaml: -------------------------------------------------------------------------------- 1 | extensions: 2 | ecs_observer: 3 | cluster_name: "my-ecs-cluster" 4 | cluster_region: "us-west-2" 5 | result_file: "/etc/ecs_sd_targets.yaml" 6 | refresh_interval: 60s 7 | job_label_name: prometheus_job 8 | services: 9 | - name_pattern: "^.*gpu-service$" 10 | metrics_ports: 11 | - 9400 12 | job_name: gpu-prometheus-exporter 13 | receivers: 14 | prometheus: 15 | config: 16 | scrape_configs: 17 | - job_name: "ecssd" 18 | file_sd_configs: 19 | - files: 20 | - "/etc/ecs_sd_targets.yaml" 21 | relabel_configs: 22 | - source_labels: [__meta_ecs_cluster_name] 23 | action: replace 24 | target_label: ClusterName 25 | - source_labels: [__meta_ecs_service_name] 26 | action: replace 27 | target_label: ServiceName 28 | - source_labels: [__meta_ecs_task_definition_family] 29 | action: replace 30 | target_label: TaskDefinitionFamily 31 | - source_labels: [__meta_ecs_task_launch_type] 32 | action: replace 33 | target_label: LaunchType 34 | - source_labels: [__meta_ecs_container_name] 35 | action: replace 36 | target_label: container_name 37 | - action: labelmap 38 | regex: ^__meta_ecs_container_labels_(.+)$ 39 | replacement: "$$1" 40 | processors: 41 | resource: 42 | attributes: 43 | - key: receiver 44 | value: "prometheus" 45 | action: insert 46 | metricstransform: 47 | transforms: 48 | - include: ".*" 49 | match_type: regexp 50 | action: update 51 | operations: 52 | - label: prometheus_job 53 | new_label: job 54 | action: update_label 55 | 56 | exporters: 57 | awsemf: 58 | namespace: ECS/ContainerInsights/Prometheus 59 | log_group_name: "/aws/ecs/containerinsights/my-ecs-cluster/prometheus" 60 | dimension_rollup_option: NoDimensionRollup 61 | metric_declarations: 62 | - dimensions: [[ClusterName, TaskDefinitionFamily, ServiceName]] 63 | label_matchers: 64 | - label_names: 65 | - ServiceName 66 | regex: "^.*gpu-service$" 67 | metric_name_selectors: 68 | - "^gpu_.*$" 69 | service: 70 | extensions: [ecs_observer] 71 | pipelines: 72 | metrics: 73 | receivers: [prometheus] 74 | processors: [resource, metricstransform] 75 | exporters: [awsemf] 76 | -------------------------------------------------------------------------------- /lib/ecs-gpu-demo-stack.ts: -------------------------------------------------------------------------------- 1 | import * as dotenv from "dotenv"; 2 | dotenv.config(); 3 | dotenv.config({ path: `.env.local`, override: true }); 4 | 5 | import * as cdk from "aws-cdk-lib"; 6 | import { Construct } from "constructs"; 7 | import * as autoscaling from "aws-cdk-lib/aws-autoscaling"; 8 | import * as appscaling from "aws-cdk-lib/aws-applicationautoscaling"; 9 | import * as cloudwatch from "aws-cdk-lib/aws-cloudwatch"; 10 | import * as ec2 from "aws-cdk-lib/aws-ec2"; 11 | import * as ecr from "aws-cdk-lib/aws-ecr"; 12 | import * as ecs from "aws-cdk-lib/aws-ecs"; 13 | import * as iam from "aws-cdk-lib/aws-iam"; 14 | import * as logs from "aws-cdk-lib/aws-logs"; 15 | import * as ssm from "aws-cdk-lib/aws-ssm"; 16 | 17 | export class EcsGpuDemoStack extends cdk.Stack { 18 | constructor(scope: Construct, id: string, props?: cdk.StackProps) { 19 | super(scope, id, props); 20 | 21 | const name = "ecs-gpu-demo"; 22 | const instanceType = "g5.xlarge"; 23 | // const instanceType = "g5.12xlarge"; 24 | const keyName = process.env.KEY_NAME; 25 | const region = cdk.Stack.of(this).region; 26 | 27 | const repository = new ecr.Repository(this, "Repository", { 28 | repositoryName: "gpu-service", 29 | removalPolicy: cdk.RemovalPolicy.DESTROY, 30 | }); 31 | 32 | const vpc = new ec2.Vpc(this, "Vpc", { 33 | vpcName: name, 34 | ipAddresses: ec2.IpAddresses.cidr("10.0.0.0/16"), 35 | natGateways: 1, 36 | }); 37 | 38 | const cluster = new ecs.Cluster(this, "Cluster", { 39 | clusterName: name, 40 | vpc, 41 | containerInsights: true, 42 | }); 43 | new cdk.CfnOutput(this, "ClusterName", { 44 | value: cluster.clusterName, 45 | }); 46 | 47 | // ECS Capacity Provider 48 | const asg = new autoscaling.AutoScalingGroup(this, "Asg", { 49 | autoScalingGroupName: `${name}`, 50 | vpc, 51 | instanceType: new ec2.InstanceType(instanceType), 52 | machineImage: ecs.EcsOptimizedImage.amazonLinux2(ecs.AmiHardwareType.GPU), 53 | desiredCapacity: 1, 54 | minCapacity: 1, 55 | maxCapacity: 3, 56 | }); 57 | asg.role.addManagedPolicy( 58 | iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore") 59 | ); 60 | asg.role.addManagedPolicy( 61 | iam.ManagedPolicy.fromAwsManagedPolicyName("CloudWatchAgentServerPolicy") 62 | ); 63 | const cp = new ecs.AsgCapacityProvider(this, "Cp", { 64 | capacityProviderName: "default", 65 | autoScalingGroup: asg, 66 | enableManagedTerminationProtection: false, 67 | canContainersAccessInstanceRole: true, 68 | }); 69 | cp.autoScalingGroup.addUserData( 70 | "echo ECS_ENABLE_GPU_SUPPORT=true >> /etc/ecs/ecs.config" 71 | ); 72 | cluster.addAsgCapacityProvider(cp); 73 | 74 | const securityGroup = new ec2.SecurityGroup(this, "Sg", { 75 | vpc, 76 | allowAllOutbound: true, 77 | }); 78 | securityGroup.addIngressRule( 79 | ec2.Peer.ipv4(vpc.vpcCidrBlock), 80 | ec2.Port.allTraffic() 81 | ); 82 | cp.autoScalingGroup.addSecurityGroup(securityGroup); 83 | cluster.addAsgCapacityProvider(cp); 84 | 85 | cluster.addDefaultCapacityProviderStrategy([ 86 | { capacityProvider: cp.capacityProviderName, weight: 1 }, 87 | ]); 88 | 89 | // ECS GPU Task 90 | const gpuTaskExecutionRole = new iam.Role(this, "GpuTaskExecutionRole", { 91 | roleName: `${name}-gpu-task-execution-role`, 92 | assumedBy: new iam.ServicePrincipal("ecs-tasks.amazonaws.com"), 93 | }); 94 | gpuTaskExecutionRole.addManagedPolicy( 95 | iam.ManagedPolicy.fromAwsManagedPolicyName( 96 | "service-role/AmazonECSTaskExecutionRolePolicy" 97 | ) 98 | ); 99 | 100 | const gptTaskDef = new ecs.Ec2TaskDefinition(this, "GpuTaskDef", { 101 | executionRole: gpuTaskExecutionRole, 102 | networkMode: ecs.NetworkMode.AWS_VPC, 103 | }); 104 | gptTaskDef.addContainer("Gpu", { 105 | containerName: "gpu", 106 | image: ecs.ContainerImage.fromRegistry(repository.repositoryUri), 107 | portMappings: [ 108 | { 109 | name: "http-metrics", 110 | containerPort: 9400, 111 | protocol: ecs.Protocol.TCP, 112 | }, 113 | ], 114 | logging: ecs.LogDrivers.awsLogs({ streamPrefix: "gpu" }), 115 | cpu: 2048, 116 | memoryReservationMiB: 4096, 117 | gpuCount: 1, 118 | }); 119 | 120 | const gpuService = new ecs.Ec2Service(this, "GpuService", { 121 | cluster, 122 | serviceName: "gpu-service", 123 | taskDefinition: gptTaskDef, 124 | enableExecuteCommand: true, 125 | desiredCount: 1, 126 | capacityProviderStrategies: [ 127 | { capacityProvider: cp.capacityProviderName, weight: 1 }, 128 | ], 129 | securityGroups: [securityGroup], 130 | }); 131 | 132 | // ECS ADOT Task 133 | new logs.LogGroup(this, "LogGroup", { 134 | logGroupName: `/aws/ecs/containerinsights/${name}/prometheus`, 135 | removalPolicy: cdk.RemovalPolicy.DESTROY, 136 | }); 137 | 138 | const adotParameter = new ssm.StringParameter(this, "AdotParameter", { 139 | parameterName: `${name}-adot-config`, 140 | stringValue: ` 141 | extensions: 142 | ecs_observer: 143 | cluster_name: "${name}" 144 | cluster_region: "${region}" 145 | result_file: "/etc/ecs_sd_targets.yaml" 146 | refresh_interval: 60s 147 | job_label_name: prometheus_job 148 | services: 149 | - name_pattern: "^.*gpu-service$" 150 | metrics_ports: 151 | - 9400 152 | job_name: gpu-prometheus-exporter 153 | receivers: 154 | prometheus: 155 | config: 156 | scrape_configs: 157 | - job_name: "ecssd" 158 | file_sd_configs: 159 | - files: 160 | - "/etc/ecs_sd_targets.yaml" 161 | relabel_configs: 162 | - source_labels: [__meta_ecs_cluster_name] 163 | action: replace 164 | target_label: ClusterName 165 | - source_labels: [__meta_ecs_service_name] 166 | action: replace 167 | target_label: ServiceName 168 | - source_labels: [__meta_ecs_task_definition_family] 169 | action: replace 170 | target_label: TaskDefinitionFamily 171 | - source_labels: [__meta_ecs_task_launch_type] 172 | action: replace 173 | target_label: LaunchType 174 | - source_labels: [__meta_ecs_container_name] 175 | action: replace 176 | target_label: container_name 177 | - action: labelmap 178 | regex: ^__meta_ecs_container_labels_(.+)$ 179 | replacement: "$$1" 180 | processors: 181 | resource: 182 | attributes: 183 | - key: receiver 184 | value: "prometheus" 185 | action: insert 186 | metricstransform: 187 | transforms: 188 | - include: ".*" 189 | match_type: regexp 190 | action: update 191 | operations: 192 | - label: prometheus_job 193 | new_label: job 194 | action: update_label 195 | 196 | exporters: 197 | awsemf: 198 | namespace: ECS/ContainerInsights/Prometheus 199 | log_group_name: "/aws/ecs/containerinsights/${name}/prometheus" 200 | dimension_rollup_option: NoDimensionRollup 201 | metric_declarations: 202 | - dimensions: [[ClusterName, ServiceName]] 203 | label_matchers: 204 | - label_names: 205 | - ServiceName 206 | regex: "^.*gpu-service$" 207 | metric_name_selectors: 208 | - "DCGM_FI_DEV_GPU_UTIL" 209 | - "DCGM_FI_DEV_GPU_TEMP" 210 | service: 211 | extensions: [ecs_observer] 212 | pipelines: 213 | metrics: 214 | receivers: [prometheus] 215 | processors: [resource, metricstransform] 216 | exporters: [awsemf] 217 | `, 218 | }); 219 | 220 | const adotTaskDef = new ecs.Ec2TaskDefinition(this, "AdotTaskDef", { 221 | networkMode: ecs.NetworkMode.AWS_VPC, 222 | }); 223 | adotTaskDef.taskRole.addToPrincipalPolicy( 224 | new iam.PolicyStatement({ 225 | resources: ["*"], 226 | actions: [ 227 | "ec2:DescribeInstances", 228 | "ecs:ListTasks", 229 | "ecs:ListServices", 230 | "ecs:DescribeContainerInstances", 231 | "ecs:DescribeServices", 232 | "ecs:DescribeTasks", 233 | "ecs:DescribeTaskDefinition", 234 | ], 235 | }) 236 | ); 237 | adotTaskDef.addContainer("Adot", { 238 | containerName: "adot", 239 | // https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/5373 240 | image: ecs.ContainerImage.fromRegistry( 241 | "amazon/aws-otel-collector:v0.11.0" 242 | ), 243 | secrets: { 244 | AOT_CONFIG_CONTENT: ecs.Secret.fromSsmParameter(adotParameter), 245 | }, 246 | logging: ecs.LogDrivers.awsLogs({ streamPrefix: "adot" }), 247 | cpu: 256, 248 | memoryReservationMiB: 512, 249 | }); 250 | new ecs.Ec2Service(this, "AdotService", { 251 | cluster, 252 | serviceName: "adot", 253 | taskDefinition: adotTaskDef, 254 | enableExecuteCommand: true, 255 | desiredCount: 1, 256 | capacityProviderStrategies: [ 257 | { capacityProvider: cp.capacityProviderName, weight: 1 }, 258 | ], 259 | }); 260 | 261 | // App Scalling 262 | const target = new appscaling.ScalableTarget(this, "ScalableTarget", { 263 | serviceNamespace: appscaling.ServiceNamespace.ECS, 264 | scalableDimension: "ecs:service:DesiredCount", 265 | resourceId: `service/${cluster.clusterName}/${gpuService.serviceName}`, 266 | minCapacity: 1, 267 | maxCapacity: 3, 268 | }); 269 | 270 | new appscaling.TargetTrackingScalingPolicy(this, "ScalingPolicy", { 271 | scalingTarget: target, 272 | policyName: name, 273 | targetValue: 75, 274 | scaleOutCooldown: cdk.Duration.minutes(1), 275 | scaleInCooldown: cdk.Duration.minutes(1), 276 | customMetric: new cloudwatch.Metric({ 277 | namespace: "ECS/ContainerInsights/Prometheus", 278 | metricName: "DCGM_FI_DEV_GPU_UTIL", 279 | dimensionsMap: { 280 | ClusterName: name, 281 | ServiceName: "gpu-service", 282 | }, 283 | statistic: cloudwatch.Stats.AVERAGE, 284 | period: cdk.Duration.minutes(1), 285 | }), 286 | disableScaleIn: false, 287 | }); 288 | 289 | // // Dev VM 290 | // const vmRole = new iam.Role(this, "VmRole", { 291 | // assumedBy: new iam.ServicePrincipal("ec2.amazonaws.com"), 292 | // }); 293 | 294 | // vmRole.addManagedPolicy( 295 | // iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore") 296 | // ); 297 | 298 | // const instance = new ec2.Instance(this, "Instance", { 299 | // instanceName: name, 300 | // vpc, 301 | // vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, 302 | // instanceType: ec2.InstanceType.of( 303 | // ec2.InstanceClass.G4DN, 304 | // ec2.InstanceSize.XLARGE 305 | // ), 306 | // machineImage: ec2.MachineImage.genericLinux({ 307 | // "us-east-1": "ami-02ea7c238b7ba36af", 308 | // }), 309 | // role: vmRole, 310 | // keyName, 311 | // blockDevices: [ 312 | // { 313 | // deviceName: "/dev/sda1", 314 | // volume: ec2.BlockDeviceVolume.ebs(1000), 315 | // }, 316 | // ], 317 | // }); 318 | 319 | // new cdk.CfnOutput(this, "InstanceId", { value: instance.instanceId }); 320 | } 321 | } 322 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "ecs-gpu-demo", 3 | "version": "0.1.0", 4 | "bin": { 5 | "ecs-gpu-demo": "bin/ecs-gpu-demo.js" 6 | }, 7 | "scripts": { 8 | "build": "tsc", 9 | "watch": "tsc -w", 10 | "cdk": "cdk" 11 | }, 12 | "devDependencies": { 13 | "@types/node": "18.15.11", 14 | "aws-cdk": "2.74.0", 15 | "ts-node": "^10.9.1", 16 | "typescript": "5.0.4" 17 | }, 18 | "dependencies": { 19 | "aws-cdk-lib": "2.74.0", 20 | "constructs": "^10.0.0", 21 | "dotenv": "^16.0.3", 22 | "source-map-support": "^0.5.21" 23 | } 24 | } 25 | -------------------------------------------------------------------------------- /python3: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/ecs-gpu-scaling/55721efb5e1bae7b82c5d695e840e0dfb0c1734b/python3 -------------------------------------------------------------------------------- /tsconfig.json: -------------------------------------------------------------------------------- 1 | { 2 | "compilerOptions": { 3 | "target": "ES2020", 4 | "module": "commonjs", 5 | "lib": [ 6 | "es2020" 7 | ], 8 | "declaration": true, 9 | "strict": true, 10 | "noImplicitAny": true, 11 | "strictNullChecks": true, 12 | "noImplicitThis": true, 13 | "alwaysStrict": true, 14 | "noUnusedLocals": false, 15 | "noUnusedParameters": false, 16 | "noImplicitReturns": true, 17 | "noFallthroughCasesInSwitch": false, 18 | "inlineSourceMap": true, 19 | "inlineSources": true, 20 | "experimentalDecorators": true, 21 | "strictPropertyInitialization": false, 22 | "typeRoots": [ 23 | "./node_modules/@types" 24 | ] 25 | }, 26 | "exclude": [ 27 | "node_modules", 28 | "cdk.out" 29 | ] 30 | } 31 | -------------------------------------------------------------------------------- /yarn.lock: -------------------------------------------------------------------------------- 1 | # THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY. 2 | # yarn lockfile v1 3 | 4 | 5 | "@aws-cdk/asset-awscli-v1@^2.2.97": 6 | version "2.2.139" 7 | resolved "https://registry.yarnpkg.com/@aws-cdk/asset-awscli-v1/-/asset-awscli-v1-2.2.139.tgz#4236502e21e96153734c3faad6e09693c89ec865" 8 | integrity sha512-PGRtKbDfI8BmLEik4awqe/u02GKhpdyYvgI6XWGvSyn2qx+f4rVmyLqM2wiFI8wFWFt6YBAQaegKJS1cEZVe2A== 9 | 10 | "@aws-cdk/asset-kubectl-v20@^2.1.1": 11 | version "2.1.1" 12 | resolved "https://registry.yarnpkg.com/@aws-cdk/asset-kubectl-v20/-/asset-kubectl-v20-2.1.1.tgz#d01c1efb867fb7f2cfd8c8b230b8eae16447e156" 13 | integrity sha512-U1ntiX8XiMRRRH5J1IdC+1t5CE89015cwyt5U63Cpk0GnMlN5+h9WsWMlKlPXZR4rdq/m806JRlBMRpBUB2Dhw== 14 | 15 | "@aws-cdk/asset-node-proxy-agent-v5@^2.0.77": 16 | version "2.0.115" 17 | resolved "https://registry.yarnpkg.com/@aws-cdk/asset-node-proxy-agent-v5/-/asset-node-proxy-agent-v5-2.0.115.tgz#e95a73089b7ba637cf2e20f2d073076f01298fc0" 18 | integrity sha512-MJ5/tMBOo6m7Bazs5TqEijIZNo4DH9NOAuv6mXjFT9k1yNAdOAr0imzG9L3uk2JpSmKrGRDSPRkrJFOszNtSaw== 19 | 20 | "@balena/dockerignore@^1.0.2": 21 | version "1.0.2" 22 | resolved "https://registry.yarnpkg.com/@balena/dockerignore/-/dockerignore-1.0.2.tgz#9ffe4726915251e8eb69f44ef3547e0da2c03e0d" 23 | integrity sha512-wMue2Sy4GAVTk6Ic4tJVcnfdau+gx2EnG7S+uAEe+TWJFqE4YoWN4/H8MSLj4eYJKxGg26lZwboEniNiNwZQ6Q== 24 | 25 | "@cspotcode/source-map-support@^0.8.0": 26 | version "0.8.1" 27 | resolved "https://registry.yarnpkg.com/@cspotcode/source-map-support/-/source-map-support-0.8.1.tgz#00629c35a688e05a88b1cda684fb9d5e73f000a1" 28 | integrity sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw== 29 | dependencies: 30 | "@jridgewell/trace-mapping" "0.3.9" 31 | 32 | "@jridgewell/resolve-uri@^3.0.3": 33 | version "3.1.1" 34 | resolved "https://registry.yarnpkg.com/@jridgewell/resolve-uri/-/resolve-uri-3.1.1.tgz#c08679063f279615a3326583ba3a90d1d82cc721" 35 | integrity sha512-dSYZh7HhCDtCKm4QakX0xFpsRDqjjtZf/kjI/v3T3Nwt5r8/qz/M19F9ySyOqU94SXBmeG9ttTul+YnR4LOxFA== 36 | 37 | "@jridgewell/sourcemap-codec@^1.4.10": 38 | version "1.4.15" 39 | resolved "https://registry.yarnpkg.com/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.4.15.tgz#d7c6e6755c78567a951e04ab52ef0fd26de59f32" 40 | integrity sha512-eF2rxCRulEKXHTRiDrDy6erMYWqNw4LPdQ8UQA4huuxaQsVeRPFl2oM8oDGxMFhJUWZf9McpLtJasDDZb/Bpeg== 41 | 42 | "@jridgewell/trace-mapping@0.3.9": 43 | version "0.3.9" 44 | resolved "https://registry.yarnpkg.com/@jridgewell/trace-mapping/-/trace-mapping-0.3.9.tgz#6534fd5933a53ba7cbf3a17615e273a0d1273ff9" 45 | integrity sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ== 46 | dependencies: 47 | "@jridgewell/resolve-uri" "^3.0.3" 48 | "@jridgewell/sourcemap-codec" "^1.4.10" 49 | 50 | "@tsconfig/node10@^1.0.7": 51 | version "1.0.9" 52 | resolved "https://registry.yarnpkg.com/@tsconfig/node10/-/node10-1.0.9.tgz#df4907fc07a886922637b15e02d4cebc4c0021b2" 53 | integrity sha512-jNsYVVxU8v5g43Erja32laIDHXeoNvFEpX33OK4d6hljo3jDhCBDhx5dhCCTMWUojscpAagGiRkBKxpdl9fxqA== 54 | 55 | "@tsconfig/node12@^1.0.7": 56 | version "1.0.11" 57 | resolved "https://registry.yarnpkg.com/@tsconfig/node12/-/node12-1.0.11.tgz#ee3def1f27d9ed66dac6e46a295cffb0152e058d" 58 | integrity sha512-cqefuRsh12pWyGsIoBKJA9luFu3mRxCA+ORZvA4ktLSzIuCUtWVxGIuXigEwO5/ywWFMZ2QEGKWvkZG1zDMTag== 59 | 60 | "@tsconfig/node14@^1.0.0": 61 | version "1.0.3" 62 | resolved "https://registry.yarnpkg.com/@tsconfig/node14/-/node14-1.0.3.tgz#e4386316284f00b98435bf40f72f75a09dabf6c1" 63 | integrity sha512-ysT8mhdixWK6Hw3i1V2AeRqZ5WfXg1G43mqoYlM2nc6388Fq5jcXyr5mRsqViLx/GJYdoL0bfXD8nmF+Zn/Iow== 64 | 65 | "@tsconfig/node16@^1.0.2": 66 | version "1.0.3" 67 | resolved "https://registry.yarnpkg.com/@tsconfig/node16/-/node16-1.0.3.tgz#472eaab5f15c1ffdd7f8628bd4c4f753995ec79e" 68 | integrity sha512-yOlFc+7UtL/89t2ZhjPvvB/DeAr3r+Dq58IgzsFkOAvVC6NMJXmCGjbptdXdR9qsX7pKcTL+s87FtYREi2dEEQ== 69 | 70 | "@types/node@18.15.11": 71 | version "18.15.11" 72 | resolved "https://registry.yarnpkg.com/@types/node/-/node-18.15.11.tgz#b3b790f09cb1696cffcec605de025b088fa4225f" 73 | integrity sha512-E5Kwq2n4SbMzQOn6wnmBjuK9ouqlURrcZDVfbo9ftDDTFt3nk7ZKK4GMOzoYgnpQJKcxwQw+lGaBvvlMo0qN/Q== 74 | 75 | acorn-walk@^8.1.1: 76 | version "8.2.0" 77 | resolved "https://registry.yarnpkg.com/acorn-walk/-/acorn-walk-8.2.0.tgz#741210f2e2426454508853a2f44d0ab83b7f69c1" 78 | integrity sha512-k+iyHEuPgSw6SbuDpGQM+06HQUa04DZ3o+F6CSzXMvvI5KMvnaEqXe+YVe555R9nn6GPt404fos4wcgpw12SDA== 79 | 80 | acorn@^8.4.1: 81 | version "8.8.2" 82 | resolved "https://registry.yarnpkg.com/acorn/-/acorn-8.8.2.tgz#1b2f25db02af965399b9776b0c2c391276d37c4a" 83 | integrity sha512-xjIYgE8HBrkpd/sJqOGNspf8uHG+NOHGOw6a/Urj8taM2EXfdNAH2oFcPeIFfsv3+kz/mJrS5VuMqbNLjCa2vw== 84 | 85 | ajv@^8.0.1: 86 | version "8.12.0" 87 | resolved "https://registry.yarnpkg.com/ajv/-/ajv-8.12.0.tgz#d1a0527323e22f53562c567c00991577dfbe19d1" 88 | integrity sha512-sRu1kpcO9yLtYxBKvqfTeh9KzZEwO3STyX1HT+4CaDzC6HpTGYhIhPIzj9XuKU7KYDwnaeh5hcOwjy1QuJzBPA== 89 | dependencies: 90 | fast-deep-equal "^3.1.1" 91 | json-schema-traverse "^1.0.0" 92 | require-from-string "^2.0.2" 93 | uri-js "^4.2.2" 94 | 95 | ansi-regex@^5.0.1: 96 | version "5.0.1" 97 | resolved "https://registry.yarnpkg.com/ansi-regex/-/ansi-regex-5.0.1.tgz#082cb2c89c9fe8659a311a53bd6a4dc5301db304" 98 | integrity sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ== 99 | 100 | ansi-styles@^4.0.0: 101 | version "4.3.0" 102 | resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-4.3.0.tgz#edd803628ae71c04c85ae7a0906edad34b648937" 103 | integrity sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg== 104 | dependencies: 105 | color-convert "^2.0.1" 106 | 107 | arg@^4.1.0: 108 | version "4.1.3" 109 | resolved "https://registry.yarnpkg.com/arg/-/arg-4.1.3.tgz#269fc7ad5b8e42cb63c896d5666017261c144089" 110 | integrity sha512-58S9QDqG0Xx27YwPSt9fJxivjYl432YCwfDMfZ+71RAqUrZef7LrKQZ3LHLOwCS4FLNBplP533Zx895SeOCHvA== 111 | 112 | astral-regex@^2.0.0: 113 | version "2.0.0" 114 | resolved "https://registry.yarnpkg.com/astral-regex/-/astral-regex-2.0.0.tgz#483143c567aeed4785759c0865786dc77d7d2e31" 115 | integrity sha512-Z7tMw1ytTXt5jqMcOP+OQteU1VuNK9Y02uuJtKQ1Sv69jXQKKg5cibLwGJow8yzZP+eAc18EmLGPal0bp36rvQ== 116 | 117 | at-least-node@^1.0.0: 118 | version "1.0.0" 119 | resolved "https://registry.yarnpkg.com/at-least-node/-/at-least-node-1.0.0.tgz#602cd4b46e844ad4effc92a8011a3c46e0238dc2" 120 | integrity sha512-+q/t7Ekv1EDY2l6Gda6LLiX14rU9TV20Wa3ofeQmwPFZbOMo9DXrLbOjFaaclkXKWidIaopwAObQDqwWtGUjqg== 121 | 122 | aws-cdk-lib@2.74.0: 123 | version "2.74.0" 124 | resolved "https://registry.yarnpkg.com/aws-cdk-lib/-/aws-cdk-lib-2.74.0.tgz#dce596bb3c54d8895e7870032ac00b82c1db2259" 125 | integrity sha512-cPoYEVmJ06pi57q4uW2v/dwXvkr9JHDeSd4+ylBALccbYiRPlZw0OiNH+F2B+hrDVAiaYZWV3HE8hVDDON4+ZA== 126 | dependencies: 127 | "@aws-cdk/asset-awscli-v1" "^2.2.97" 128 | "@aws-cdk/asset-kubectl-v20" "^2.1.1" 129 | "@aws-cdk/asset-node-proxy-agent-v5" "^2.0.77" 130 | "@balena/dockerignore" "^1.0.2" 131 | case "1.6.3" 132 | fs-extra "^9.1.0" 133 | ignore "^5.2.4" 134 | jsonschema "^1.4.1" 135 | minimatch "^3.1.2" 136 | punycode "^2.3.0" 137 | semver "^7.3.8" 138 | table "^6.8.1" 139 | yaml "1.10.2" 140 | 141 | aws-cdk@2.74.0: 142 | version "2.74.0" 143 | resolved "https://registry.yarnpkg.com/aws-cdk/-/aws-cdk-2.74.0.tgz#da834764a369799f945606bac0e1a8301fdf6e5a" 144 | integrity sha512-pc6QO9uR7Ii0qQ74nujskkFqPCGoWTTMyt03CFaGW0CwxMfpduGC0+bvlLBbJISAe5ZGuRuYIIxxDMkNi3AIcw== 145 | optionalDependencies: 146 | fsevents "2.3.2" 147 | 148 | balanced-match@^1.0.0: 149 | version "1.0.2" 150 | resolved "https://registry.yarnpkg.com/balanced-match/-/balanced-match-1.0.2.tgz#e83e3a7e3f300b34cb9d87f615fa0cbf357690ee" 151 | integrity sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw== 152 | 153 | brace-expansion@^1.1.7: 154 | version "1.1.11" 155 | resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.11.tgz#3c7fcbf529d87226f3d2f52b966ff5271eb441dd" 156 | integrity sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA== 157 | dependencies: 158 | balanced-match "^1.0.0" 159 | concat-map "0.0.1" 160 | 161 | buffer-from@^1.0.0: 162 | version "1.1.2" 163 | resolved "https://registry.yarnpkg.com/buffer-from/-/buffer-from-1.1.2.tgz#2b146a6fd72e80b4f55d255f35ed59a3a9a41bd5" 164 | integrity sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ== 165 | 166 | case@1.6.3: 167 | version "1.6.3" 168 | resolved "https://registry.yarnpkg.com/case/-/case-1.6.3.tgz#0a4386e3e9825351ca2e6216c60467ff5f1ea1c9" 169 | integrity sha512-mzDSXIPaFwVDvZAHqZ9VlbyF4yyXRuX6IvB06WvPYkqJVO24kX1PPhv9bfpKNFZyxYFmmgo03HUiD8iklmJYRQ== 170 | 171 | color-convert@^2.0.1: 172 | version "2.0.1" 173 | resolved "https://registry.yarnpkg.com/color-convert/-/color-convert-2.0.1.tgz#72d3a68d598c9bdb3af2ad1e84f21d896abd4de3" 174 | integrity sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ== 175 | dependencies: 176 | color-name "~1.1.4" 177 | 178 | color-name@~1.1.4: 179 | version "1.1.4" 180 | resolved "https://registry.yarnpkg.com/color-name/-/color-name-1.1.4.tgz#c2a09a87acbde69543de6f63fa3995c826c536a2" 181 | integrity sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA== 182 | 183 | concat-map@0.0.1: 184 | version "0.0.1" 185 | resolved "https://registry.yarnpkg.com/concat-map/-/concat-map-0.0.1.tgz#d8a96bd77fd68df7793a73036a3ba0d5405d477b" 186 | integrity sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg== 187 | 188 | constructs@^10.0.0: 189 | version "10.1.310" 190 | resolved "https://registry.yarnpkg.com/constructs/-/constructs-10.1.310.tgz#e4b88d2f8719356e9b60398170a293ba773b8577" 191 | integrity sha512-hfw9zaXmdPItWbDyeL9MaCxjB77W0FSfPvN5N9uKXtXRLHJHk5EJx2NtGPNNf4po1YanrZTCpp6rxpkP3nQkGw== 192 | 193 | create-require@^1.1.0: 194 | version "1.1.1" 195 | resolved "https://registry.yarnpkg.com/create-require/-/create-require-1.1.1.tgz#c1d7e8f1e5f6cfc9ff65f9cd352d37348756c333" 196 | integrity sha512-dcKFX3jn0MpIaXjisoRvexIJVEKzaq7z2rZKxf+MSr9TkdmHmsU4m2lcLojrj/FHl8mk5VxMmYA+ftRkP/3oKQ== 197 | 198 | diff@^4.0.1: 199 | version "4.0.2" 200 | resolved "https://registry.yarnpkg.com/diff/-/diff-4.0.2.tgz#60f3aecb89d5fae520c11aa19efc2bb982aade7d" 201 | integrity sha512-58lmxKSA4BNyLz+HHMUzlOEpg09FV+ev6ZMe3vJihgdxzgcwZ8VoEEPmALCZG9LmqfVoNMMKpttIYTVG6uDY7A== 202 | 203 | dotenv@^16.0.3: 204 | version "16.0.3" 205 | resolved "https://registry.yarnpkg.com/dotenv/-/dotenv-16.0.3.tgz#115aec42bac5053db3c456db30cc243a5a836a07" 206 | integrity sha512-7GO6HghkA5fYG9TYnNxi14/7K9f5occMlp3zXAuSxn7CKCxt9xbNWG7yF8hTCSUchlfWSe3uLmlPfigevRItzQ== 207 | 208 | emoji-regex@^8.0.0: 209 | version "8.0.0" 210 | resolved "https://registry.yarnpkg.com/emoji-regex/-/emoji-regex-8.0.0.tgz#e818fd69ce5ccfcb404594f842963bf53164cc37" 211 | integrity sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A== 212 | 213 | fast-deep-equal@^3.1.1: 214 | version "3.1.3" 215 | resolved "https://registry.yarnpkg.com/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz#3a7d56b559d6cbc3eb512325244e619a65c6c525" 216 | integrity sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q== 217 | 218 | fs-extra@^9.1.0: 219 | version "9.1.0" 220 | resolved "https://registry.yarnpkg.com/fs-extra/-/fs-extra-9.1.0.tgz#5954460c764a8da2094ba3554bf839e6b9a7c86d" 221 | integrity sha512-hcg3ZmepS30/7BSFqRvoo3DOMQu7IjqxO5nCDt+zM9XWjb33Wg7ziNT+Qvqbuc3+gWpzO02JubVyk2G4Zvo1OQ== 222 | dependencies: 223 | at-least-node "^1.0.0" 224 | graceful-fs "^4.2.0" 225 | jsonfile "^6.0.1" 226 | universalify "^2.0.0" 227 | 228 | fsevents@2.3.2: 229 | version "2.3.2" 230 | resolved "https://registry.yarnpkg.com/fsevents/-/fsevents-2.3.2.tgz#8a526f78b8fdf4623b709e0b975c52c24c02fd1a" 231 | integrity sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA== 232 | 233 | graceful-fs@^4.1.6, graceful-fs@^4.2.0: 234 | version "4.2.11" 235 | resolved "https://registry.yarnpkg.com/graceful-fs/-/graceful-fs-4.2.11.tgz#4183e4e8bf08bb6e05bbb2f7d2e0c8f712ca40e3" 236 | integrity sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ== 237 | 238 | ignore@^5.2.4: 239 | version "5.2.4" 240 | resolved "https://registry.yarnpkg.com/ignore/-/ignore-5.2.4.tgz#a291c0c6178ff1b960befe47fcdec301674a6324" 241 | integrity sha512-MAb38BcSbH0eHNBxn7ql2NH/kX33OkB3lZ1BNdh7ENeRChHTYsTvWrMubiIAMNS2llXEEgZ1MUOBtXChP3kaFQ== 242 | 243 | is-fullwidth-code-point@^3.0.0: 244 | version "3.0.0" 245 | resolved "https://registry.yarnpkg.com/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz#f116f8064fe90b3f7844a38997c0b75051269f1d" 246 | integrity sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg== 247 | 248 | json-schema-traverse@^1.0.0: 249 | version "1.0.0" 250 | resolved "https://registry.yarnpkg.com/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz#ae7bcb3656ab77a73ba5c49bf654f38e6b6860e2" 251 | integrity sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug== 252 | 253 | jsonfile@^6.0.1: 254 | version "6.1.0" 255 | resolved "https://registry.yarnpkg.com/jsonfile/-/jsonfile-6.1.0.tgz#bc55b2634793c679ec6403094eb13698a6ec0aae" 256 | integrity sha512-5dgndWOriYSm5cnYaJNhalLNDKOqFwyDB/rr1E9ZsGciGvKPs8R2xYGCacuf3z6K1YKDz182fd+fY3cn3pMqXQ== 257 | dependencies: 258 | universalify "^2.0.0" 259 | optionalDependencies: 260 | graceful-fs "^4.1.6" 261 | 262 | jsonschema@^1.4.1: 263 | version "1.4.1" 264 | resolved "https://registry.yarnpkg.com/jsonschema/-/jsonschema-1.4.1.tgz#cc4c3f0077fb4542982973d8a083b6b34f482dab" 265 | integrity sha512-S6cATIPVv1z0IlxdN+zUk5EPjkGCdnhN4wVSBlvoUO1tOLJootbo9CquNJmbIh4yikWHiUedhRYrNPn1arpEmQ== 266 | 267 | lodash.truncate@^4.4.2: 268 | version "4.4.2" 269 | resolved "https://registry.yarnpkg.com/lodash.truncate/-/lodash.truncate-4.4.2.tgz#5a350da0b1113b837ecfffd5812cbe58d6eae193" 270 | integrity sha512-jttmRe7bRse52OsWIMDLaXxWqRAmtIUccAQ3garviCqJjafXOfNMO0yMfNpdD6zbGaTU0P5Nz7e7gAT6cKmJRw== 271 | 272 | lru-cache@^6.0.0: 273 | version "6.0.0" 274 | resolved "https://registry.yarnpkg.com/lru-cache/-/lru-cache-6.0.0.tgz#6d6fe6570ebd96aaf90fcad1dafa3b2566db3a94" 275 | integrity sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA== 276 | dependencies: 277 | yallist "^4.0.0" 278 | 279 | make-error@^1.1.1: 280 | version "1.3.6" 281 | resolved "https://registry.yarnpkg.com/make-error/-/make-error-1.3.6.tgz#2eb2e37ea9b67c4891f684a1394799af484cf7a2" 282 | integrity sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw== 283 | 284 | minimatch@^3.1.2: 285 | version "3.1.2" 286 | resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-3.1.2.tgz#19cd194bfd3e428f049a70817c038d89ab4be35b" 287 | integrity sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw== 288 | dependencies: 289 | brace-expansion "^1.1.7" 290 | 291 | punycode@^2.1.0, punycode@^2.3.0: 292 | version "2.3.0" 293 | resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.3.0.tgz#f67fa67c94da8f4d0cfff981aee4118064199b8f" 294 | integrity sha512-rRV+zQD8tVFys26lAGR9WUuS4iUAngJScM+ZRSKtvl5tKeZ2t5bvdNFdNHBW9FWR4guGHlgmsZ1G7BSm2wTbuA== 295 | 296 | require-from-string@^2.0.2: 297 | version "2.0.2" 298 | resolved "https://registry.yarnpkg.com/require-from-string/-/require-from-string-2.0.2.tgz#89a7fdd938261267318eafe14f9c32e598c36909" 299 | integrity sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw== 300 | 301 | semver@^7.3.8: 302 | version "7.4.0" 303 | resolved "https://registry.yarnpkg.com/semver/-/semver-7.4.0.tgz#8481c92feffc531ab1e012a8ffc15bdd3a0f4318" 304 | integrity sha512-RgOxM8Mw+7Zus0+zcLEUn8+JfoLpj/huFTItQy2hsM4khuC1HYRDp0cU482Ewn/Fcy6bCjufD8vAj7voC66KQw== 305 | dependencies: 306 | lru-cache "^6.0.0" 307 | 308 | slice-ansi@^4.0.0: 309 | version "4.0.0" 310 | resolved "https://registry.yarnpkg.com/slice-ansi/-/slice-ansi-4.0.0.tgz#500e8dd0fd55b05815086255b3195adf2a45fe6b" 311 | integrity sha512-qMCMfhY040cVHT43K9BFygqYbUPFZKHOg7K73mtTWJRb8pyP3fzf4Ixd5SzdEJQ6MRUg/WBnOLxghZtKKurENQ== 312 | dependencies: 313 | ansi-styles "^4.0.0" 314 | astral-regex "^2.0.0" 315 | is-fullwidth-code-point "^3.0.0" 316 | 317 | source-map-support@^0.5.21: 318 | version "0.5.21" 319 | resolved "https://registry.yarnpkg.com/source-map-support/-/source-map-support-0.5.21.tgz#04fe7c7f9e1ed2d662233c28cb2b35b9f63f6e4f" 320 | integrity sha512-uBHU3L3czsIyYXKX88fdrGovxdSCoTGDRZ6SYXtSRxLZUzHg5P/66Ht6uoUlHu9EZod+inXhKo3qQgwXUT/y1w== 321 | dependencies: 322 | buffer-from "^1.0.0" 323 | source-map "^0.6.0" 324 | 325 | source-map@^0.6.0: 326 | version "0.6.1" 327 | resolved "https://registry.yarnpkg.com/source-map/-/source-map-0.6.1.tgz#74722af32e9614e9c287a8d0bbde48b5e2f1a263" 328 | integrity sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g== 329 | 330 | string-width@^4.2.3: 331 | version "4.2.3" 332 | resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010" 333 | integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g== 334 | dependencies: 335 | emoji-regex "^8.0.0" 336 | is-fullwidth-code-point "^3.0.0" 337 | strip-ansi "^6.0.1" 338 | 339 | strip-ansi@^6.0.1: 340 | version "6.0.1" 341 | resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9" 342 | integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A== 343 | dependencies: 344 | ansi-regex "^5.0.1" 345 | 346 | table@^6.8.1: 347 | version "6.8.1" 348 | resolved "https://registry.yarnpkg.com/table/-/table-6.8.1.tgz#ea2b71359fe03b017a5fbc296204471158080bdf" 349 | integrity sha512-Y4X9zqrCftUhMeH2EptSSERdVKt/nEdijTOacGD/97EKjhQ/Qs8RTlEGABSJNNN8lac9kheH+af7yAkEWlgneA== 350 | dependencies: 351 | ajv "^8.0.1" 352 | lodash.truncate "^4.4.2" 353 | slice-ansi "^4.0.0" 354 | string-width "^4.2.3" 355 | strip-ansi "^6.0.1" 356 | 357 | ts-node@^10.9.1: 358 | version "10.9.1" 359 | resolved "https://registry.yarnpkg.com/ts-node/-/ts-node-10.9.1.tgz#e73de9102958af9e1f0b168a6ff320e25adcff4b" 360 | integrity sha512-NtVysVPkxxrwFGUUxGYhfux8k78pQB3JqYBXlLRZgdGUqTO5wU/UyHop5p70iEbGhB7q5KmiZiU0Y3KlJrScEw== 361 | dependencies: 362 | "@cspotcode/source-map-support" "^0.8.0" 363 | "@tsconfig/node10" "^1.0.7" 364 | "@tsconfig/node12" "^1.0.7" 365 | "@tsconfig/node14" "^1.0.0" 366 | "@tsconfig/node16" "^1.0.2" 367 | acorn "^8.4.1" 368 | acorn-walk "^8.1.1" 369 | arg "^4.1.0" 370 | create-require "^1.1.0" 371 | diff "^4.0.1" 372 | make-error "^1.1.1" 373 | v8-compile-cache-lib "^3.0.1" 374 | yn "3.1.1" 375 | 376 | typescript@5.0.4: 377 | version "5.0.4" 378 | resolved "https://registry.yarnpkg.com/typescript/-/typescript-5.0.4.tgz#b217fd20119bd61a94d4011274e0ab369058da3b" 379 | integrity sha512-cW9T5W9xY37cc+jfEnaUvX91foxtHkza3Nw3wkoF4sSlKn0MONdkdEndig/qPBWXNkmplh3NzayQzCiHM4/hqw== 380 | 381 | universalify@^2.0.0: 382 | version "2.0.0" 383 | resolved "https://registry.yarnpkg.com/universalify/-/universalify-2.0.0.tgz#75a4984efedc4b08975c5aeb73f530d02df25717" 384 | integrity sha512-hAZsKq7Yy11Zu1DE0OzWjw7nnLZmJZYTDZZyEFHZdUhV8FkH5MCfoU1XMaxXovpyW5nq5scPqq0ZDP9Zyl04oQ== 385 | 386 | uri-js@^4.2.2: 387 | version "4.4.1" 388 | resolved "https://registry.yarnpkg.com/uri-js/-/uri-js-4.4.1.tgz#9b1a52595225859e55f669d928f88c6c57f2a77e" 389 | integrity sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg== 390 | dependencies: 391 | punycode "^2.1.0" 392 | 393 | v8-compile-cache-lib@^3.0.1: 394 | version "3.0.1" 395 | resolved "https://registry.yarnpkg.com/v8-compile-cache-lib/-/v8-compile-cache-lib-3.0.1.tgz#6336e8d71965cb3d35a1bbb7868445a7c05264bf" 396 | integrity sha512-wa7YjyUGfNZngI/vtK0UHAN+lgDCxBPCylVXGp0zu59Fz5aiGtNXaq3DhIov063MorB+VfufLh3JlF2KdTK3xg== 397 | 398 | yallist@^4.0.0: 399 | version "4.0.0" 400 | resolved "https://registry.yarnpkg.com/yallist/-/yallist-4.0.0.tgz#9bb92790d9c0effec63be73519e11a35019a3a72" 401 | integrity sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A== 402 | 403 | yaml@1.10.2: 404 | version "1.10.2" 405 | resolved "https://registry.yarnpkg.com/yaml/-/yaml-1.10.2.tgz#2301c5ffbf12b467de8da2333a459e29e7920e4b" 406 | integrity sha512-r3vXyErRCYJ7wg28yvBY5VSoAF8ZvlcW9/BwUzEtUsjvX/DKs24dIkuwjtuprwJJHsbyUbLApepYTR1BN4uHrg== 407 | 408 | yn@3.1.1: 409 | version "3.1.1" 410 | resolved "https://registry.yarnpkg.com/yn/-/yn-3.1.1.tgz#1e87401a09d767c1d5eab26a6e4c185182d2eb50" 411 | integrity sha512-Ux4ygGWsu2c7isFWe8Yu1YluJmqVhxqK2cLXNQA5AcC3QfbGNpM7fu0Y8b/z16pXLnFxZYvWhd3fhBY9DLmC6Q== 412 | --------------------------------------------------------------------------------