├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── langfuse-v2 ├── .gitignore ├── README.md ├── app.py ├── assets │ ├── 01-langfuse-main-page.png │ ├── 02-sign-up.png │ ├── 03-new-project.png │ ├── 04-create-api-keys.png │ ├── 05-traces.png │ └── 06-trace-detail.png ├── cdk.context.json ├── cdk.json ├── cdk_stacks │ ├── __init__.py │ ├── alb.py │ ├── aurora_postgresql.py │ ├── ecr.py │ ├── ecs_alb_fargate_service.py │ ├── ecs_cluster.py │ ├── ecs_task.py │ └── vpc.py ├── examples │ └── tracing_for_langchain_bedrock.ipynb ├── langfuse-on-aws-ecs-fargate-arch.svg ├── requirements.txt └── source.bat └── langfuse-v3 ├── .envrc ├── .example.cdk.context.json ├── .gitignore ├── README.md ├── app.py ├── assets ├── 01-langfuse-main-page.png ├── 02-sign-up.png ├── 03-new-project.png ├── 04-create-api-keys.png ├── 05-traces.png └── 06-trace-detail.png ├── cdk.json ├── cdk_stacks ├── __init__.py ├── alb_langfuse_web.py ├── aurora_postgresql.py ├── ecr.py ├── ecs_cluster.py ├── ecs_fargate_service_clickhouse.py ├── ecs_fargate_service_langfuse_web.py ├── ecs_fargate_service_langfuse_worker.py ├── ecs_task_clickhouse.py ├── ecs_task_langfuse_web.py ├── ecs_task_langfuse_worker.py ├── efs_clickhouse.py ├── redis.py ├── s3.py ├── service_discovery.py └── vpc.py ├── examples └── tracing_for_langchain_bedrock.ipynb ├── langfuse-v3-on-aws-ecs-fargate-arch.svg ├── requirements.txt └── source.bat /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # Hosting Langfuse on Amazon ECS with Fargate using CDK Python 3 | 4 | This repository contains the AWS CDK Python code for deploying the [Langfuse](https://langfuse.com/) application using Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS). 5 | 6 | Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. 7 | 8 | | Project | Architecture | 9 | |---------|--------------| 10 | | [langfuse-v2](./langfuse-v2/) | ![lanfuse-v2-arch](./langfuse-v2/langfuse-on-aws-ecs-fargate-arch.svg) | 11 | | [langfuse-v3](./langfuse-v3/) | ![lanfuse-v3-arch](./langfuse-v3/langfuse-v3-on-aws-ecs-fargate-arch.svg) | 12 | 13 | ## References 14 | 15 | * [(Official) Self-host Langfuse Guide](https://langfuse.com/self-hosting) 16 | * [(GitHub) langfuse](https://github.com/langfuse/langfuse/) 17 | * [(GitHub) langfuse-examples](https://github.com/langfuse/langfuse-examples) 18 | * [AWS CDK Reference Documentation](https://docs.aws.amazon.com/cdk/api/v2/) 19 | 20 | ## Security 21 | 22 | See [CONTRIBUTING](./CONTRIBUTING.md#security-issue-notifications) for more information. 23 | 24 | ## License 25 | 26 | This library is licensed under the MIT-0 License. See the LICENSE file. 27 | -------------------------------------------------------------------------------- /langfuse-v2/.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | package-lock.json 3 | __pycache__ 4 | .pytest_cache 5 | .venv 6 | *.egg-info 7 | .DS_Store 8 | .gitignore 9 | .vscode 10 | 11 | # CDK asset staging directory 12 | .cdk.staging 13 | cdk.out 14 | 15 | .example.cdk.context.json 16 | .template.cdk.context.json 17 | .*.log 18 | *.bak -------------------------------------------------------------------------------- /langfuse-v2/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Hosting Langfuse on Amazon ECS with Fargate using CDK Python 3 | 4 | > :information_source: This guide covers Langfuse v2. For Langfuse v3, see the [v3 documentation](https://langfuse.com/self-hosting). 5 | 6 | ![lanfuse-on-aws-ecs-fargate-arch](./langfuse-on-aws-ecs-fargate-arch.svg) 7 | 8 | This repository contains the AWS CDK Python code for deploying the [Langfuse](https://langfuse.com/) application using Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS). 9 | 10 | Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. 11 | 12 | The `cdk.json` file tells the CDK Toolkit how to execute your app. 13 | 14 | This project is set up like a standard Python project. The initialization 15 | process also creates a virtualenv within this project, stored under the `.venv` 16 | directory. To create the virtualenv it assumes that there is a `python3` 17 | (or `python` for Windows) executable in your path with access to the `venv` 18 | package. If for any reason the automatic creation of the virtualenv fails, 19 | you can create the virtualenv manually. 20 | 21 | To manually create a virtualenv on MacOS and Linux: 22 | 23 | ``` 24 | $ git clone --depth=1 https://github.com/aws-samples/deploy-langfuse-on-ecs-with-fargate.git 25 | $ cd deploy-langfuse-on-ecs-with-fargate 26 | $ git sparse-checkout init --cone 27 | $ git sparse-checkout set langfuse-v2 28 | $ cd langfuse-v2 29 | 30 | $ python3 -m venv .venv 31 | ``` 32 | 33 | After the init process completes and the virtualenv is created, you can use the following 34 | step to activate your virtualenv. 35 | 36 | ``` 37 | $ source .venv/bin/activate 38 | ``` 39 | 40 | If you are a Windows platform, you would activate the virtualenv like this: 41 | 42 | ``` 43 | % .venv\Scripts\activate.bat 44 | ``` 45 | 46 | Once the virtualenv is activated, you can install the required dependencies. 47 | 48 | ``` 49 | (.venv) $ pip install -r requirements.txt 50 | ``` 51 | 52 | > To add additional dependencies, for example other CDK libraries, just add 53 | them to your `setup.py` file and rerun the `pip install -r requirements.txt` 54 | command. 55 | 56 | ### Set up `cdk.context.json` 57 | 58 | Then, we need to set approperly the cdk context configuration file, `cdk.context.json`. 59 | 60 | For example, 61 | 62 | ``` 63 | { 64 | "db_cluster_name": "langfuse-db", 65 | "ecs_cluster_name": "langfuse-cluster", 66 | "ecs_service_name": "langfuse-alb-service", 67 | "image_version": "2", 68 | "langfuse_env": { 69 | "NODE_ENV": "production", 70 | "NEXTAUTH_SECRET": "mysecret", 71 | "SALT": "mysalt", 72 | "TELEMETRY_ENABLED": "true", 73 | "NEXT_PUBLIC_SIGN_UP_DISABLED": "false", 74 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true" 75 | } 76 | } 77 | ``` 78 | 79 | :information_source: This guide covers Langfuse v2. `image_version` should be set to `2`. 80 | 81 | :information_source: `NEXTAUTH_SECRET` and `SALT` can be created using `openssl rand -base64 32` on `MacOS` or `Ubuntu`. 82 | (For more information, see [**Langfuse Configuring Environment Variables**](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables)) 83 | 84 | ### Deploy 85 | 86 | At this point you can now synthesize the CloudFormation template for this code. 87 | 88 | ``` 89 | (.venv) $ export CDK_DEFAULT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text) 90 | (.venv) $ export CDK_DEFAULT_REGION=$(aws configure get region) 91 | (.venv) $ cdk synth --all 92 | ``` 93 | 94 | Use `cdk deploy` command to create the stack shown above. 95 | 96 | ``` 97 | (.venv) $ cdk deploy --require-approval never --all 98 | ``` 99 | 100 | We can list all the CDK stacks by using the `cdk list` command prior to deployment. 101 | 102 | ``` 103 | (.venv) $ cdk list 104 | LangFuseECRStack 105 | LangFuseVpcStack 106 | LangFuseALBStack 107 | LangFuseAuroraPostgreSQLStack 108 | LangFuseECSClusterStack 109 | LangFuseECSTaskStack 110 | LangFuseECSAlbFargateServiceStack 111 | ``` 112 | 113 | ## Clean Up 114 | 115 | Delete the CloudFormation stack by running the below command. 116 | 117 | ``` 118 | (.venv) $ cdk destroy --force --all 119 | ``` 120 | 121 | ## Useful commands 122 | 123 | * `cdk ls` list all stacks in the app 124 | * `cdk synth` emits the synthesized CloudFormation template 125 | * `cdk deploy` deploy this stack to your default AWS account/region 126 | * `cdk diff` compare deployed stack with current state 127 | * `cdk docs` open CDK documentation 128 | 129 | Enjoy! 130 | 131 | ## Tracing for your LLM Application with Langfuse 132 | 133 | After deploying all CDK stacks, you can find the **Langfuse URL** using the following command: 134 | 135 | ```bash 136 | aws cloudformation describe-stacks --stack-name LangFuseECSAlbFargateServiceStack --region ${CDK_DEFAULT_REGION} | \ 137 | jq -r '.Stacks[0].Outputs | map(select(.OutputKey == "LoadBalancerDNS")) | .[0].OutputValue' 138 | ``` 139 | 140 | Next, open the **Langfuse URL** in your browser to create a new project for tracking your LLM application with Langfuse. 141 | 142 | ### Create a New Project in Langfuse 143 | 144 | 1. Create a Langfuse Account 145 | 146 | ![Login](./assets/01-langfuse-main-page.png) 147 | 148 | ![Singup](./assets/02-sign-up.png) 149 | 2. Create a New Project 150 | ![New-Project](./assets/03-new-project.png) 151 | 3. Create New API Credentials in the Project Settings 152 | ![API-Keys](./assets/04-create-api-keys.png) 153 | 154 | ### Log Your First LLM Call to Langfuse 155 | 156 | Open the `tracing_for_langchain_bedrock` notebook in the `examples` folder and run it. (See [here](./examples/tracing_for_langchain_bedrock.ipynb) for more information) 157 | 158 | You will the see the list of traces as follows: 159 | ![Traces](./assets/05-traces.png) 160 | 161 | You will also see the details of the selected trace as follows: 162 | 163 | ![Trace-detail](./assets/06-trace-detail.png) 164 | 165 | ## References 166 | 167 | * [(Workshop) GenAIOps - Observability for GenAI applications with Amazon Bedrock and Langfuse](https://catalog.workshops.aws/genaiops-langfuse) 168 | * [(AWS Korea Tech Blog) Hosting Langfuse on Amazon ECS with Fargate using AWS CDK Python (2024-08-09)](https://aws.amazon.com/ko/blogs/tech/hosting-langfuse-with-aws-cdk-python-using-amazon-ecs-and-aws-fargate/) 169 | * [Langfuse Official Documents](https://langfuse.com/docs) - Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. 170 | * [Langfuse Self-Hosting Guide (Docker)](https://langfuse.com/docs/deployment/self-host) 171 | * [Langfuse Configuring Environment Variables](https://langfuse.com/docs/deployment/self-host#configuring-environment-variables) 172 | * [Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing) 173 | * [Langfuse Python SDK (Low-level)](https://langfuse.com/docs/sdk/python/low-level-sdk) 174 | * [Langfuse Cookbook](https://github.com/langfuse/langfuse-docs/tree/main/cookbook) 175 | * [(GitHub) Langfuse](https://github.com/langfuse/langfuse) - 🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 176 | * [Langfuse ECR/ECS Deployment CDK TypeScript](https://github.com/AI4Organization/langfuse-ecr-ecs-deployment-cdk) - Deploy Langfuse to ECR and ECS with Fargate and AWS CDK TypeScript. 177 | * [cdk-ecr-deployment](https://github.com/cdklabs/cdk-ecr-deployment) - A CDK construct to deploy docker image to Amazon ECR. 178 | 179 | ## Security 180 | 181 | See [CONTRIBUTING](../CONTRIBUTING.md#security-issue-notifications) for more information. 182 | 183 | ## License 184 | 185 | This library is licensed under the MIT-0 License. See the LICENSE file. 186 | -------------------------------------------------------------------------------- /langfuse-v2/app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import os 6 | 7 | import aws_cdk as cdk 8 | 9 | from cdk_stacks import ( 10 | ApplicationLoadBalancerStack, 11 | AuroraPostgresqlStack, 12 | ECRStack, 13 | ECSAlbFargateServiceStack, 14 | ECSClusterStack, 15 | ECSTaskStack, 16 | VpcStack 17 | ) 18 | 19 | AWS_ENV = cdk.Environment( 20 | account=os.environ["CDK_DEFAULT_ACCOUNT"], 21 | region=os.environ["CDK_DEFAULT_REGION"] 22 | ) 23 | 24 | app = cdk.App() 25 | 26 | ecr_stack = ECRStack(app, "LangFuseECRStack", 27 | env=AWS_ENV 28 | ) 29 | 30 | vpc_stack = VpcStack(app, "LangFuseVpcStack", 31 | env=AWS_ENV 32 | ) 33 | vpc_stack.add_dependency(ecr_stack) 34 | 35 | alb_stack = ApplicationLoadBalancerStack(app, "LangFuseALBStack", 36 | vpc_stack.vpc, 37 | env=AWS_ENV 38 | ) 39 | alb_stack.add_dependency(vpc_stack) 40 | 41 | rds_stack = AuroraPostgresqlStack(app, "LangFuseAuroraPostgreSQLStack", 42 | vpc_stack.vpc, 43 | env=AWS_ENV 44 | ) 45 | rds_stack.add_dependency(alb_stack) 46 | 47 | ecs_cluster_stack = ECSClusterStack(app, "LangFuseECSClusterStack", 48 | vpc_stack.vpc, 49 | env=AWS_ENV 50 | ) 51 | ecs_cluster_stack.add_dependency(rds_stack) 52 | 53 | ecs_task_stack = ECSTaskStack(app, "LangFuseECSTaskStack", 54 | ecr_stack.repository, 55 | rds_stack.database_secret, 56 | alb_stack.load_balancer_url, 57 | env=AWS_ENV 58 | ) 59 | ecs_task_stack.add_dependency(ecs_cluster_stack) 60 | 61 | ecs_fargate_stack = ECSAlbFargateServiceStack(app, "LangFuseECSAlbFargateServiceStack", 62 | vpc_stack.vpc, 63 | ecs_cluster_stack.ecs_cluster, 64 | ecs_task_stack.ecs_task_definition, 65 | alb_stack.load_balancer, 66 | rds_stack.sg_rds_client, 67 | env=AWS_ENV 68 | ) 69 | ecs_fargate_stack.add_dependency(ecs_task_stack) 70 | 71 | app.synth() 72 | -------------------------------------------------------------------------------- /langfuse-v2/assets/01-langfuse-main-page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/01-langfuse-main-page.png -------------------------------------------------------------------------------- /langfuse-v2/assets/02-sign-up.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/02-sign-up.png -------------------------------------------------------------------------------- /langfuse-v2/assets/03-new-project.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/03-new-project.png -------------------------------------------------------------------------------- /langfuse-v2/assets/04-create-api-keys.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/04-create-api-keys.png -------------------------------------------------------------------------------- /langfuse-v2/assets/05-traces.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/05-traces.png -------------------------------------------------------------------------------- /langfuse-v2/assets/06-trace-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v2/assets/06-trace-detail.png -------------------------------------------------------------------------------- /langfuse-v2/cdk.context.json: -------------------------------------------------------------------------------- 1 | { 2 | "db_cluster_name": "langfuse-db", 3 | "ecs_cluster_name": "langfuse-cluster", 4 | "ecs_service_name": "langfuse-alb-service", 5 | "image_version": "2", 6 | "langfuse_env": { 7 | "NODE_ENV": "production", 8 | "NEXTAUTH_SECRET": "secret (generate by running 'openssl rand -base64 32')", 9 | "SALT": "salt (generate by running 'openssl rand -base64 32')", 10 | "TELEMETRY_ENABLED": "true", 11 | "NEXT_PUBLIC_SIGN_UP_DISABLED": "false", 12 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true" 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /langfuse-v2/cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "requirements*.txt", 11 | "source.bat", 12 | "**/__init__.py", 13 | "**/__pycache__", 14 | "tests" 15 | ] 16 | }, 17 | "context": { 18 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 19 | "@aws-cdk/core:checkSecretUsage": true, 20 | "@aws-cdk/core:target-partitions": [ 21 | "aws", 22 | "aws-cn" 23 | ], 24 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 25 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 26 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 27 | "@aws-cdk/aws-iam:minimizePolicies": true, 28 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 29 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 30 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 31 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 32 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 33 | "@aws-cdk/core:enablePartitionLiterals": true, 34 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 35 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 36 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 37 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 38 | "@aws-cdk/aws-route53-patters:useCertificate": true, 39 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 40 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 41 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 42 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 43 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 44 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 45 | "@aws-cdk/aws-redshift:columnId": true, 46 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, 47 | "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, 48 | "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, 49 | "@aws-cdk/aws-kms:aliasNameRef": true, 50 | "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, 51 | "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, 52 | "@aws-cdk/aws-efs:denyAnonymousAccess": true, 53 | "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, 54 | "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, 55 | "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, 56 | "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, 57 | "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, 58 | "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true, 59 | "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true, 60 | "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true, 61 | "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, 62 | "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true, 63 | "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, 64 | "@aws-cdk/aws-eks:nodegroupNameAttribute": true, 65 | "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true, 66 | "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true, 67 | "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false, 68 | "@aws-cdk/aws-stepfunctions-tasks:ecsReduceRunTaskPermissions": true 69 | } 70 | } 71 | -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/__init__.py: -------------------------------------------------------------------------------- 1 | from .alb import ApplicationLoadBalancerStack 2 | from .aurora_postgresql import AuroraPostgresqlStack 3 | from .ecr import ECRStack 4 | from .ecs_alb_fargate_service import ECSAlbFargateServiceStack 5 | from .ecs_cluster import ECSClusterStack 6 | from .ecs_task import ECSTaskStack 7 | from .vpc import VpcStack -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/alb.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_elasticloadbalancingv2 as elbv2 11 | ) 12 | 13 | from constructs import Construct 14 | 15 | 16 | class ApplicationLoadBalancerStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 19 | 20 | super().__init__(scope, construct_id, **kwargs) 21 | 22 | service_name = self.node.try_get_context('ecs_service_name') or "langfuse-alb-service" 23 | 24 | sg_alb = aws_ec2.SecurityGroup(self, 'LoadBalancerSG', 25 | vpc=vpc, 26 | allow_all_outbound=True, 27 | description="Allow inbound from VPC for ECS Fargate Service", 28 | security_group_name=f'{service_name}-alb-sg' 29 | ) 30 | sg_alb.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 31 | connection=aws_ec2.Port.tcp(80)) 32 | sg_alb.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 33 | connection=aws_ec2.Port.tcp(443)) 34 | cdk.Tags.of(sg_alb).add('Name', 'ecs-service-alb-sg') 35 | 36 | self.load_balancer = elbv2.ApplicationLoadBalancer(self, "LoadBalancer", 37 | vpc=vpc, 38 | internet_facing=True, 39 | security_group=sg_alb 40 | ) 41 | self.load_balancer.apply_removal_policy(cdk.RemovalPolicy.DESTROY) 42 | 43 | self.load_balancer_url = f'http://{self.load_balancer.load_balancer_dns_name}' 44 | 45 | 46 | cdk.CfnOutput(self, "LoadBalancerDNS", 47 | value=self.load_balancer_url, 48 | export_name=f'{self.stack_name}-LoadBalancerDNS') 49 | -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/aurora_postgresql.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import json 6 | 7 | import aws_cdk as cdk 8 | 9 | from aws_cdk import ( 10 | Stack, 11 | aws_ec2, 12 | aws_logs, 13 | aws_rds, 14 | aws_secretsmanager 15 | ) 16 | 17 | from constructs import Construct 18 | 19 | 20 | class AuroraPostgresqlStack(Stack): 21 | 22 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 23 | super().__init__(scope, construct_id, **kwargs) 24 | 25 | db_cluster_name = self.node.try_get_context('db_cluster_name') or 'langfuse-db' 26 | 27 | sg_postgresql_client = aws_ec2.SecurityGroup(self, 'PostgreSQLClientSG', 28 | vpc=vpc, 29 | allow_all_outbound=True, 30 | description='security group for postgresql client', 31 | security_group_name=f'{db_cluster_name}-postgresql-client-sg' 32 | ) 33 | cdk.Tags.of(sg_postgresql_client).add('Name', 'postgresql-client-sg') 34 | 35 | sg_postgresql_server = aws_ec2.SecurityGroup(self, 'PostgreSQLServerSG', 36 | vpc=vpc, 37 | allow_all_outbound=True, 38 | description='security group for postgresql', 39 | security_group_name=f'{db_cluster_name}-postgresql-server-sg' 40 | ) 41 | sg_postgresql_server.add_ingress_rule(peer=sg_postgresql_client, connection=aws_ec2.Port.tcp(5432), 42 | description='postgresql-client-sg') 43 | sg_postgresql_server.add_ingress_rule(peer=sg_postgresql_server, connection=aws_ec2.Port.all_tcp(), 44 | description='postgresql-server-sg') 45 | cdk.Tags.of(sg_postgresql_server).add('Name', 'postgresql-server-sg') 46 | 47 | rds_subnet_group = aws_rds.SubnetGroup(self, 'PostgreSQLSubnetGroup', 48 | description='subnet group for postgresql', 49 | subnet_group_name=f'aurora-postgresql-{self.stack_name}', 50 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS), 51 | vpc=vpc 52 | ) 53 | 54 | #XXX: In order to exclude punctuations when generating a password 55 | # use aws_secretsmanager.Secret instead of aws_rds.DatabaseSecret. 56 | # Othwerise, an error occurred such as: 57 | # "All characters of the desired type have been excluded" 58 | db_secret = aws_secretsmanager.Secret(self, 'DatabaseSecret', 59 | generate_secret_string=aws_secretsmanager.SecretStringGenerator( 60 | secret_string_template=json.dumps({"username": "postgres"}), 61 | generate_string_key="password", 62 | exclude_punctuation=True, 63 | password_length=8 64 | ) 65 | ) 66 | rds_credentials = aws_rds.Credentials.from_secret(db_secret) 67 | 68 | rds_engine = aws_rds.DatabaseClusterEngine.aurora_postgres(version=aws_rds.AuroraPostgresEngineVersion.VER_15_4) 69 | 70 | #XXX: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Reference.ParameterGroups.html#AuroraPostgreSQL.Reference.Parameters.Cluster 71 | rds_cluster_param_group = aws_rds.ParameterGroup(self, 'AuroraPostgreSQLClusterParamGroup', 72 | engine=rds_engine, 73 | description='Custom cluster parameter group for aurora-postgresql15', 74 | parameters={ 75 | 'log_min_duration_statement': '15000', # 15 sec 76 | 'default_transaction_isolation': 'read committed', 77 | 'client_encoding': 'UTF8' 78 | } 79 | ) 80 | 81 | #XXX: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Reference.ParameterGroups.html#AuroraPostgreSQL.Reference.Parameters.Instance 82 | rds_db_param_group = aws_rds.ParameterGroup(self, 'AuroraPostgreSQLDBParamGroup', 83 | engine=rds_engine, 84 | description='Custom parameter group for aurora-postgresql15', 85 | parameters={ 86 | 'log_min_duration_statement': '15000', # 15 sec 87 | 'default_transaction_isolation': 'read committed' 88 | } 89 | ) 90 | 91 | db_cluster = aws_rds.DatabaseCluster(self, 'Database', 92 | engine=rds_engine, 93 | credentials=rds_credentials, # A username of 'admin' (or 'postgres' for PostgreSQL) and SecretsManager-generated password 94 | writer=aws_rds.ClusterInstance.provisioned("writer", 95 | instance_type=aws_ec2.InstanceType.of(aws_ec2.InstanceClass.R6G, aws_ec2.InstanceSize.LARGE), 96 | parameter_group=rds_db_param_group, 97 | auto_minor_version_upgrade=False, 98 | ), 99 | readers=[ 100 | aws_rds.ClusterInstance.provisioned("reader", 101 | instance_type=aws_ec2.InstanceType.of(aws_ec2.InstanceClass.R6G, aws_ec2.InstanceSize.LARGE), 102 | parameter_group=rds_db_param_group, 103 | auto_minor_version_upgrade=False 104 | ) 105 | ], 106 | parameter_group=rds_cluster_param_group, 107 | cloudwatch_logs_retention=aws_logs.RetentionDays.THREE_DAYS, 108 | cluster_identifier=db_cluster_name, 109 | subnet_group=rds_subnet_group, 110 | backup=aws_rds.BackupProps( 111 | retention=cdk.Duration.days(3), 112 | preferred_window="03:00-04:00" 113 | ), 114 | security_groups=[sg_postgresql_server], 115 | vpc=vpc, 116 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS) 117 | ) 118 | 119 | self.sg_rds_client = sg_postgresql_client 120 | self.database_secret = db_cluster.secret 121 | self.database = db_cluster 122 | 123 | 124 | cdk.CfnOutput(self, 'DBClusterEndpoint', 125 | value=db_cluster.cluster_endpoint.socket_address, 126 | export_name=f'{self.stack_name}-DBClusterEndpoint') 127 | cdk.CfnOutput(self, 'DBClusterReadEndpoint', 128 | value=db_cluster.cluster_read_endpoint.socket_address, 129 | export_name=f'{self.stack_name}-DBClusterReadEndpoint') 130 | cdk.CfnOutput(self, 'RDSClientSecurityGroupId', 131 | value=sg_postgresql_client.security_group_id, 132 | export_name=f'{self.stack_name}-RDSClientSecurityGroupId') 133 | cdk.CfnOutput(self, 'DBSecretName', 134 | value=db_cluster.secret.secret_name, 135 | export_name=f'{self.stack_name}-DBSecretName') -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/ecr.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecr, 10 | ) 11 | from constructs import Construct 12 | import cdk_ecr_deployment as ecr_deploy 13 | 14 | class ECRStack(Stack): 15 | 16 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 17 | super().__init__(scope, construct_id, **kwargs) 18 | 19 | repository_name = self.node.try_get_context('ecr_repository_name') or "langfuse" 20 | self.repository = aws_ecr.Repository(self, "LangFuseECRRepository", 21 | empty_on_delete=True, 22 | encryption=aws_ecr.RepositoryEncryption.AES_256, 23 | removal_policy=cdk.RemovalPolicy.DESTROY, 24 | repository_name=repository_name 25 | ) 26 | 27 | # delete images older than 7 days 28 | self.repository.add_lifecycle_rule(max_image_age=cdk.Duration.days(7), rule_priority=1, tag_status=aws_ecr.TagStatus.UNTAGGED) 29 | # keep last 3 images 30 | self.repository.add_lifecycle_rule(max_image_count=3, rule_priority=2, tag_status=aws_ecr.TagStatus.ANY) 31 | 32 | src_docker_image_version = 'ghcr.io/langfuse/langfuse' 33 | image_version = self.node.try_get_context('image_version') or "2" 34 | deploy_image_versions = ["2", image_version] if image_version == "latest" else [image_version, "latest"] 35 | 36 | for i, deploy_image_version in enumerate(deploy_image_versions): 37 | ecr_deploy.ECRDeployment(self, f"LangFuseECRDeployment-{i:0>3}", 38 | src=ecr_deploy.DockerImageName(f'{src_docker_image_version}:{image_version}'), 39 | dest=ecr_deploy.DockerImageName(self.repository.repository_uri_for_tag_or_digest(deploy_image_version)) 40 | ) 41 | 42 | 43 | cdk.CfnOutput(self, 'ECRRepositoryArn', 44 | value=self.repository.repository_arn, 45 | export_name=f'{self.stack_name}-ECRRepositoryArn' 46 | ) 47 | cdk.CfnOutput(self, 'ECRRepositoryName', 48 | value=self.repository.repository_name, 49 | export_name=f'{self.stack_name}-ECRRepositoryName' 50 | ) 51 | -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/ecs_alb_fargate_service.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_ecs_patterns, 11 | ) 12 | 13 | from constructs import Construct 14 | 15 | 16 | class ECSAlbFargateServiceStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, 19 | vpc, ecs_cluster, ecs_task_definition, 20 | load_balancer, sg_rds_client, **kwargs) -> None: 21 | 22 | super().__init__(scope, construct_id, **kwargs) 23 | 24 | service_name = self.node.try_get_context('ecs_service_name') or "langfuse-alb-service" 25 | 26 | sg_fargate_service = aws_ec2.SecurityGroup(self, 'ECSFargateServiceSG', 27 | vpc=vpc, 28 | allow_all_outbound=True, 29 | description="Allow inbound from VPC for ECS Fargate Service", 30 | security_group_name=f'{service_name}-ecs-service-sg' 31 | ) 32 | sg_fargate_service.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 33 | connection=aws_ec2.Port.tcp(3000), 34 | description='langfuse-server') 35 | cdk.Tags.of(sg_fargate_service).add('Name', 'ecs-service-alb-sg') 36 | 37 | fargate_service = aws_ecs_patterns.ApplicationLoadBalancedFargateService(self, "ALBFargateService", 38 | service_name=service_name, 39 | cluster=ecs_cluster, 40 | task_definition=ecs_task_definition, 41 | load_balancer=load_balancer, 42 | security_groups=[sg_fargate_service, sg_rds_client] 43 | ) 44 | 45 | # Setup autoscaling policy 46 | scalable_target = fargate_service.service.auto_scale_task_count(max_capacity=2) 47 | scalable_target.scale_on_cpu_utilization( 48 | id="Autoscaling", 49 | target_utilization_percent=70, 50 | scale_in_cooldown=cdk.Duration.seconds(60), 51 | scale_out_cooldown=cdk.Duration.seconds(60), 52 | ) 53 | 54 | cdk.CfnOutput(self, "LoadBalancerDNS", 55 | value=f'http://{fargate_service.load_balancer.load_balancer_dns_name}', 56 | export_name=f'{self.stack_name}-LoadBalancerDNS') -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/ecs_cluster.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecs as ecs, 10 | ) 11 | 12 | from constructs import Construct 13 | 14 | 15 | class ECSClusterStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 18 | 19 | super().__init__(scope, construct_id, **kwargs) 20 | 21 | cluster_name = self.node.try_get_context('ecs_cluster_name') or "langfuse-cluster" 22 | cluster = ecs.Cluster(self, "ECSCluster", 23 | cluster_name=cluster_name, 24 | vpc=vpc 25 | ) 26 | 27 | self.ecs_cluster = cluster 28 | 29 | cdk.CfnOutput(self, 'ClusterName', 30 | value=self.ecs_cluster.cluster_name, 31 | export_name=f'{self.stack_name}-ClusterName') 32 | cdk.CfnOutput(self, 'ClusterArn', 33 | value=self.ecs_cluster.cluster_arn, 34 | export_name=f'{self.stack_name}-ClusterArn') 35 | -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/ecs_task.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecs, 10 | aws_iam 11 | ) 12 | 13 | from constructs import Construct 14 | from typing import List 15 | 16 | 17 | def check_env_variables(envars: dict, vars: List[str]): 18 | for k in vars: 19 | assert envars.get(k) 20 | 21 | 22 | class ECSTaskStack(Stack): 23 | 24 | def __init__(self, scope: Construct, construct_id: str, ecr_repository, database_secret, load_balancer_url, **kwargs) -> None: 25 | 26 | super().__init__(scope, construct_id, **kwargs) 27 | 28 | task_role_policy_doc = aws_iam.PolicyDocument() 29 | task_role_policy_doc.add_statements(aws_iam.PolicyStatement(**{ 30 | "effect": aws_iam.Effect.ALLOW, 31 | "resources": ["*"], 32 | "actions": [ 33 | "ecr:GetAuthorizationToken", 34 | "ecr:BatchCheckLayerAvailability", 35 | "ecr:GetDownloadUrlForLayer", 36 | "ecr:BatchGetImage", 37 | "logs:CreateLogStream", 38 | "logs:PutLogEvents" 39 | ] 40 | })) 41 | 42 | task_role = aws_iam.Role(self, "ECSTaskRole", 43 | role_name=f'ECSTaskRole-{self.stack_name}', 44 | assumed_by=aws_iam.ServicePrincipal(service="ecs-tasks.amazonaws.com"), 45 | inline_policies={ 46 | 'ecs_task_role_policy': task_role_policy_doc 47 | }, 48 | managed_policies=[ 49 | aws_iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AmazonECSTaskExecutionRolePolicy") 50 | ] 51 | ) 52 | 53 | task_definition = aws_ecs.FargateTaskDefinition(self, "LangFuseServer", 54 | task_role=task_role, 55 | cpu=1 * 1024, 56 | memory_limit_mib=2 * 1024 57 | ) 58 | 59 | db_conn_info = { 60 | "DATABASE_HOST": database_secret.secret_value_from_json("host").unsafe_unwrap(), 61 | "DATABASE_PORT": database_secret.secret_value_from_json("port").unsafe_unwrap(), 62 | "DATABASE_USERNAME": database_secret.secret_value_from_json("username").unsafe_unwrap(), 63 | "DATABASE_PASSWORD": database_secret.secret_value_from_json("password").unsafe_unwrap(), 64 | "DATABASE_NAME": "postgres" 65 | } 66 | DATABASE_URL = "postgresql://{DATABASE_USERNAME}:{DATABASE_PASSWORD}@{DATABASE_HOST}:{DATABASE_PORT}/{DATABASE_NAME}".format(**db_conn_info) 67 | 68 | docker_run_args = self.node.try_get_context('langfuse_env') 69 | task_env = { 70 | **docker_run_args, 71 | "NEXTAUTH_URL": load_balancer_url, 72 | "DATABASE_URL": DATABASE_URL 73 | } 74 | 75 | check_env_variables(task_env, 76 | [ 77 | 'NODE_ENV', 'NEXTAUTH_SECRET', 'SALT', 78 | 'TELEMETRY_ENABLED', 'NEXTAUTH_URL', 'NEXT_PUBLIC_SIGN_UP_DISABLED', 79 | 'LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES', 'DATABASE_URL' 80 | ] 81 | ) 82 | 83 | container = task_definition.add_container("LangFuseServer", 84 | image=aws_ecs.ContainerImage.from_ecr_repository(ecr_repository, tag="latest"), 85 | environment=task_env, 86 | logging=aws_ecs.LogDriver.aws_logs(stream_prefix="langfuse-server"), 87 | ) 88 | 89 | port_mapping = aws_ecs.PortMapping( 90 | container_port=3000, 91 | host_port=3000, 92 | protocol=aws_ecs.Protocol.TCP 93 | ) 94 | container.add_port_mappings(port_mapping) 95 | 96 | self.ecs_task_definition = task_definition 97 | 98 | 99 | cdk.CfnOutput(self, 'TaskDefinitionArn', 100 | value=self.ecs_task_definition.task_definition_arn, 101 | export_name=f'{self.stack_name}-TaskDefinitionArn') 102 | -------------------------------------------------------------------------------- /langfuse-v2/cdk_stacks/vpc.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import os 6 | import aws_cdk as cdk 7 | 8 | from aws_cdk import ( 9 | Stack, 10 | aws_ec2, 11 | ) 12 | from constructs import Construct 13 | 14 | 15 | class VpcStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 18 | super().__init__(scope, construct_id, **kwargs) 19 | 20 | #XXX: For creating this CDK Stack in the existing VPC, 21 | # remove comments from the below codes and 22 | # comments out vpc = aws_ec2.Vpc(..) codes, 23 | # then pass -c vpc_name=your-existing-vpc to cdk command 24 | # for example, 25 | # cdk -c vpc_name=your-existing-vpc syth 26 | # 27 | if str(os.environ.get('USE_DEFAULT_VPC', 'false')).lower() == 'true': 28 | vpc_name = self.node.try_get_context('vpc_name') or "default" 29 | self.vpc = aws_ec2.Vpc.from_lookup(self, 'ExistingVPC', 30 | is_default=True, 31 | vpc_name=vpc_name 32 | ) 33 | else: 34 | #XXX: To use more than 2 AZs, be sure to specify the account and region on your stack. 35 | #XXX: https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_ec2/Vpc.html 36 | self.vpc = aws_ec2.Vpc(self, 'VPC', 37 | ip_addresses=aws_ec2.IpAddresses.cidr("10.0.0.0/16"), 38 | max_azs=3, 39 | 40 | # 'subnetConfiguration' specifies the "subnet groups" to create. 41 | # Every subnet group will have a subnet for each AZ, so this 42 | # configuration will create `2 groups × 3 AZs = 6` subnets. 43 | subnet_configuration=[ 44 | { 45 | "cidrMask": 20, 46 | "name": "Public", 47 | "subnetType": aws_ec2.SubnetType.PUBLIC, 48 | }, 49 | { 50 | "cidrMask": 20, 51 | "name": "Private", 52 | "subnetType": aws_ec2.SubnetType.PRIVATE_WITH_EGRESS 53 | } 54 | ], 55 | gateway_endpoints={ 56 | "S3": aws_ec2.GatewayVpcEndpointOptions( 57 | service=aws_ec2.GatewayVpcEndpointAwsService.S3 58 | ) 59 | } 60 | ) 61 | 62 | cdk.CfnOutput(self, 'VPCID', value=self.vpc.vpc_id, 63 | export_name=f'{self.stack_name}-VPCID') 64 | -------------------------------------------------------------------------------- /langfuse-v2/examples/tracing_for_langchain_bedrock.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "fa62892f-5737-428d-a192-c53b07f0923c", 6 | "metadata": {}, 7 | "source": [ 8 | "# Tracing for LangChain (Python)\n", 9 | "\n", 10 | "This notebook works well with the `Data Science 2.0 kernel` on a SageMaker Studio `ml.t3.medium` instance." 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "id": "44f1d80e-3d36-4e4f-aba8-7923459c62ec", 16 | "metadata": {}, 17 | "source": [ 18 | "### Install required packages" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "id": "18390522-ae27-4a63-86c8-b3e1401fa1d3", 25 | "metadata": { 26 | "tags": [] 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "%%capture --no-stderr\n", 31 | "\n", 32 | "!pip install -U langfuse==2.39.3\n", 33 | "!pip install -U langchain==0.2.11\n", 34 | "!pip install -U langchain-community==0.2.10\n", 35 | "!pip install -U langchain-aws==0.1.12" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "id": "432648db-9d10-4f5e-bf3d-7553fc4a7309", 42 | "metadata": { 43 | "tags": [] 44 | }, 45 | "outputs": [ 46 | { 47 | "name": "stdout", 48 | "output_type": "stream", 49 | "text": [ 50 | "langchain==0.2.11\n", 51 | "langchain-aws==0.1.12\n", 52 | "langchain-community==0.2.10\n", 53 | "langchain-core==0.2.24\n", 54 | "langchain-text-splitters==0.2.2\n", 55 | "langfuse==2.39.3\n" 56 | ] 57 | } 58 | ], 59 | "source": [ 60 | "!pip freeze | grep -E \"langchain|langfuse\"" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "id": "bb8c9e1e-56db-4a4b-b88a-b084adf1543f", 66 | "metadata": {}, 67 | "source": [ 68 | "### Environment Setup and LangFuse Initialization" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": null, 74 | "id": "3dc5e5c0-4765-45df-b8ce-509092f0ef57", 75 | "metadata": { 76 | "tags": [] 77 | }, 78 | "outputs": [], 79 | "source": [ 80 | "import os\n", 81 | "from langfuse.callback import CallbackHandler\n", 82 | "\n", 83 | "# Get the API key from the LangFuse project settings and set it as an environment variable\n", 84 | "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = 'pk-lf-***'\n", 85 | "os.environ[\"LANGFUSE_SECRET_KEY\"] = 'sk-lf-***'\n", 86 | "os.environ[\"LANGFUSE_HOST\"] = \"http://langfu-***-****.{region}.elb.amazonaws.com\"\n", 87 | "\n", 88 | "# Initialize LangFuse Callback Handler\n", 89 | "langfuse_handler = CallbackHandler()" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "id": "929e4944-e8ea-45fb-8378-0d8afde50d38", 96 | "metadata": { 97 | "tags": [] 98 | }, 99 | "outputs": [ 100 | { 101 | "data": { 102 | "text/plain": [ 103 | "True" 104 | ] 105 | }, 106 | "execution_count": null, 107 | "metadata": {}, 108 | "output_type": "execute_result" 109 | } 110 | ], 111 | "source": [ 112 | "# connection test\n", 113 | "langfuse_handler.auth_check()" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": null, 119 | "id": "aed2146a-2410-4f8e-a8f6-c14e53e3f248", 120 | "metadata": { 121 | "tags": [] 122 | }, 123 | "outputs": [], 124 | "source": [ 125 | "import boto3\n", 126 | "from langchain_aws import ChatBedrock as BedrockChat\n", 127 | "\n", 128 | "region = boto3.Session().region_name\n", 129 | "\n", 130 | "model_kwargs={\n", 131 | " \"max_tokens\": 512,\n", 132 | " \"temperature\": 0,\n", 133 | " \"top_p\": 0.9\n", 134 | "}\n", 135 | "\n", 136 | "model_id = \"anthropic.claude-3-sonnet-20240229-v1:0\"\n", 137 | "\n", 138 | "llm = BedrockChat(model_id=model_id, region_name=region, model_kwargs=model_kwargs)\n", 139 | "llm" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "id": "19d1f0cc-0802-4672-a480-51192cff77f1", 146 | "metadata": { 147 | "tags": [] 148 | }, 149 | "outputs": [], 150 | "source": [ 151 | "from langchain.prompts import ChatPromptTemplate\n", 152 | "from langchain.schema import StrOutputParser\n", 153 | "from operator import itemgetter\n", 154 | "\n", 155 | "prompt1 = ChatPromptTemplate.from_template(\"What is the city {person} is from?\")\n", 156 | "prompt2 = ChatPromptTemplate.from_template(\n", 157 | " \"What country is the city {city} in? Respond in {language}\"\n", 158 | ")\n", 159 | "\n", 160 | "chain1 = prompt1 | llm | StrOutputParser()\n", 161 | "chain2 = (\n", 162 | " {\"city\": chain1, \"language\": itemgetter(\"language\")}\n", 163 | " | prompt2\n", 164 | " | llm\n", 165 | " | StrOutputParser()\n", 166 | ")" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "id": "ae985454-ceba-4195-970b-277d6e8afddd", 173 | "metadata": { 174 | "tags": [] 175 | }, 176 | "outputs": [], 177 | "source": [ 178 | "# Invoke chain with LangFuse callback handler\n", 179 | "result = chain2.invoke(\n", 180 | " {\"person\": \"Obama\", \"language\": \"Spanish\"},\n", 181 | " config={\"callbacks\": [langfuse_handler]}\n", 182 | ")" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "id": "93977840-b304-4b87-bb74-33df2e1ab336", 189 | "metadata": { 190 | "tags": [] 191 | }, 192 | "outputs": [ 193 | { 194 | "name": "stdout", 195 | "output_type": "stream", 196 | "text": [ 197 | "La ciudad de Honolulu, donde nació Barack Obama el 4 de agosto de 1961, está ubicada en Hawái, un estado de los Estados Unidos.\n", 198 | "\n", 199 | "Aunque pasó gran parte de su infancia en Indonesia y vivió en varios otros lugares como Los Ángeles, Nueva York, Boston y Chicago antes de convertirse en presidente, Chicago se considera su base política principal. Allí trabajó como organizador comunitario, enseñó derecho constitucional en la Universidad de Chicago e inició su carrera política sirviendo en el Senado estatal de Illinois y luego en el Senado de los Estados Unidos representando a Illinois antes de ser elegido como el 44º presidente de los Estados Unidos en 2008.\n", 200 | "\n", 201 | "Si bien nació en Honolulu, Hawái, las conexiones de Barack Obama con Chicago como adulto fueron muy sólidas e integrales para su ascenso en la política antes de convertirse en presidente.\n" 202 | ] 203 | } 204 | ], 205 | "source": [ 206 | "print(result)" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "id": "cb44a99f-3abe-4310-9815-15d7ebdfde1e", 212 | "metadata": {}, 213 | "source": [ 214 | "## References\n", 215 | "\n", 216 | "- [Langfuse Documents](https://langfuse.com/docs)\n", 217 | " - [Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing)" 218 | ] 219 | } 220 | ], 221 | "metadata": { 222 | "availableInstances": [ 223 | { 224 | "_defaultOrder": 0, 225 | "_isFastLaunch": true, 226 | "category": "General purpose", 227 | "gpuNum": 0, 228 | "hideHardwareSpecs": false, 229 | "memoryGiB": 4, 230 | "name": "ml.t3.medium", 231 | "vcpuNum": 2 232 | }, 233 | { 234 | "_defaultOrder": 1, 235 | "_isFastLaunch": false, 236 | "category": "General purpose", 237 | "gpuNum": 0, 238 | "hideHardwareSpecs": false, 239 | "memoryGiB": 8, 240 | "name": "ml.t3.large", 241 | "vcpuNum": 2 242 | }, 243 | { 244 | "_defaultOrder": 2, 245 | "_isFastLaunch": false, 246 | "category": "General purpose", 247 | "gpuNum": 0, 248 | "hideHardwareSpecs": false, 249 | "memoryGiB": 16, 250 | "name": "ml.t3.xlarge", 251 | "vcpuNum": 4 252 | }, 253 | { 254 | "_defaultOrder": 3, 255 | "_isFastLaunch": false, 256 | "category": "General purpose", 257 | "gpuNum": 0, 258 | "hideHardwareSpecs": false, 259 | "memoryGiB": 32, 260 | "name": "ml.t3.2xlarge", 261 | "vcpuNum": 8 262 | }, 263 | { 264 | "_defaultOrder": 4, 265 | "_isFastLaunch": true, 266 | "category": "General purpose", 267 | "gpuNum": 0, 268 | "hideHardwareSpecs": false, 269 | "memoryGiB": 8, 270 | "name": "ml.m5.large", 271 | "vcpuNum": 2 272 | }, 273 | { 274 | "_defaultOrder": 5, 275 | "_isFastLaunch": false, 276 | "category": "General purpose", 277 | "gpuNum": 0, 278 | "hideHardwareSpecs": false, 279 | "memoryGiB": 16, 280 | "name": "ml.m5.xlarge", 281 | "vcpuNum": 4 282 | }, 283 | { 284 | "_defaultOrder": 6, 285 | "_isFastLaunch": false, 286 | "category": "General purpose", 287 | "gpuNum": 0, 288 | "hideHardwareSpecs": false, 289 | "memoryGiB": 32, 290 | "name": "ml.m5.2xlarge", 291 | "vcpuNum": 8 292 | }, 293 | { 294 | "_defaultOrder": 7, 295 | "_isFastLaunch": false, 296 | "category": "General purpose", 297 | "gpuNum": 0, 298 | "hideHardwareSpecs": false, 299 | "memoryGiB": 64, 300 | "name": "ml.m5.4xlarge", 301 | "vcpuNum": 16 302 | }, 303 | { 304 | "_defaultOrder": 8, 305 | "_isFastLaunch": false, 306 | "category": "General purpose", 307 | "gpuNum": 0, 308 | "hideHardwareSpecs": false, 309 | "memoryGiB": 128, 310 | "name": "ml.m5.8xlarge", 311 | "vcpuNum": 32 312 | }, 313 | { 314 | "_defaultOrder": 9, 315 | "_isFastLaunch": false, 316 | "category": "General purpose", 317 | "gpuNum": 0, 318 | "hideHardwareSpecs": false, 319 | "memoryGiB": 192, 320 | "name": "ml.m5.12xlarge", 321 | "vcpuNum": 48 322 | }, 323 | { 324 | "_defaultOrder": 10, 325 | "_isFastLaunch": false, 326 | "category": "General purpose", 327 | "gpuNum": 0, 328 | "hideHardwareSpecs": false, 329 | "memoryGiB": 256, 330 | "name": "ml.m5.16xlarge", 331 | "vcpuNum": 64 332 | }, 333 | { 334 | "_defaultOrder": 11, 335 | "_isFastLaunch": false, 336 | "category": "General purpose", 337 | "gpuNum": 0, 338 | "hideHardwareSpecs": false, 339 | "memoryGiB": 384, 340 | "name": "ml.m5.24xlarge", 341 | "vcpuNum": 96 342 | }, 343 | { 344 | "_defaultOrder": 12, 345 | "_isFastLaunch": false, 346 | "category": "General purpose", 347 | "gpuNum": 0, 348 | "hideHardwareSpecs": false, 349 | "memoryGiB": 8, 350 | "name": "ml.m5d.large", 351 | "vcpuNum": 2 352 | }, 353 | { 354 | "_defaultOrder": 13, 355 | "_isFastLaunch": false, 356 | "category": "General purpose", 357 | "gpuNum": 0, 358 | "hideHardwareSpecs": false, 359 | "memoryGiB": 16, 360 | "name": "ml.m5d.xlarge", 361 | "vcpuNum": 4 362 | }, 363 | { 364 | "_defaultOrder": 14, 365 | "_isFastLaunch": false, 366 | "category": "General purpose", 367 | "gpuNum": 0, 368 | "hideHardwareSpecs": false, 369 | "memoryGiB": 32, 370 | "name": "ml.m5d.2xlarge", 371 | "vcpuNum": 8 372 | }, 373 | { 374 | "_defaultOrder": 15, 375 | "_isFastLaunch": false, 376 | "category": "General purpose", 377 | "gpuNum": 0, 378 | "hideHardwareSpecs": false, 379 | "memoryGiB": 64, 380 | "name": "ml.m5d.4xlarge", 381 | "vcpuNum": 16 382 | }, 383 | { 384 | "_defaultOrder": 16, 385 | "_isFastLaunch": false, 386 | "category": "General purpose", 387 | "gpuNum": 0, 388 | "hideHardwareSpecs": false, 389 | "memoryGiB": 128, 390 | "name": "ml.m5d.8xlarge", 391 | "vcpuNum": 32 392 | }, 393 | { 394 | "_defaultOrder": 17, 395 | "_isFastLaunch": false, 396 | "category": "General purpose", 397 | "gpuNum": 0, 398 | "hideHardwareSpecs": false, 399 | "memoryGiB": 192, 400 | "name": "ml.m5d.12xlarge", 401 | "vcpuNum": 48 402 | }, 403 | { 404 | "_defaultOrder": 18, 405 | "_isFastLaunch": false, 406 | "category": "General purpose", 407 | "gpuNum": 0, 408 | "hideHardwareSpecs": false, 409 | "memoryGiB": 256, 410 | "name": "ml.m5d.16xlarge", 411 | "vcpuNum": 64 412 | }, 413 | { 414 | "_defaultOrder": 19, 415 | "_isFastLaunch": false, 416 | "category": "General purpose", 417 | "gpuNum": 0, 418 | "hideHardwareSpecs": false, 419 | "memoryGiB": 384, 420 | "name": "ml.m5d.24xlarge", 421 | "vcpuNum": 96 422 | }, 423 | { 424 | "_defaultOrder": 20, 425 | "_isFastLaunch": false, 426 | "category": "General purpose", 427 | "gpuNum": 0, 428 | "hideHardwareSpecs": true, 429 | "memoryGiB": 0, 430 | "name": "ml.geospatial.interactive", 431 | "supportedImageNames": [ 432 | "sagemaker-geospatial-v1-0" 433 | ], 434 | "vcpuNum": 0 435 | }, 436 | { 437 | "_defaultOrder": 21, 438 | "_isFastLaunch": true, 439 | "category": "Compute optimized", 440 | "gpuNum": 0, 441 | "hideHardwareSpecs": false, 442 | "memoryGiB": 4, 443 | "name": "ml.c5.large", 444 | "vcpuNum": 2 445 | }, 446 | { 447 | "_defaultOrder": 22, 448 | "_isFastLaunch": false, 449 | "category": "Compute optimized", 450 | "gpuNum": 0, 451 | "hideHardwareSpecs": false, 452 | "memoryGiB": 8, 453 | "name": "ml.c5.xlarge", 454 | "vcpuNum": 4 455 | }, 456 | { 457 | "_defaultOrder": 23, 458 | "_isFastLaunch": false, 459 | "category": "Compute optimized", 460 | "gpuNum": 0, 461 | "hideHardwareSpecs": false, 462 | "memoryGiB": 16, 463 | "name": "ml.c5.2xlarge", 464 | "vcpuNum": 8 465 | }, 466 | { 467 | "_defaultOrder": 24, 468 | "_isFastLaunch": false, 469 | "category": "Compute optimized", 470 | "gpuNum": 0, 471 | "hideHardwareSpecs": false, 472 | "memoryGiB": 32, 473 | "name": "ml.c5.4xlarge", 474 | "vcpuNum": 16 475 | }, 476 | { 477 | "_defaultOrder": 25, 478 | "_isFastLaunch": false, 479 | "category": "Compute optimized", 480 | "gpuNum": 0, 481 | "hideHardwareSpecs": false, 482 | "memoryGiB": 72, 483 | "name": "ml.c5.9xlarge", 484 | "vcpuNum": 36 485 | }, 486 | { 487 | "_defaultOrder": 26, 488 | "_isFastLaunch": false, 489 | "category": "Compute optimized", 490 | "gpuNum": 0, 491 | "hideHardwareSpecs": false, 492 | "memoryGiB": 96, 493 | "name": "ml.c5.12xlarge", 494 | "vcpuNum": 48 495 | }, 496 | { 497 | "_defaultOrder": 27, 498 | "_isFastLaunch": false, 499 | "category": "Compute optimized", 500 | "gpuNum": 0, 501 | "hideHardwareSpecs": false, 502 | "memoryGiB": 144, 503 | "name": "ml.c5.18xlarge", 504 | "vcpuNum": 72 505 | }, 506 | { 507 | "_defaultOrder": 28, 508 | "_isFastLaunch": false, 509 | "category": "Compute optimized", 510 | "gpuNum": 0, 511 | "hideHardwareSpecs": false, 512 | "memoryGiB": 192, 513 | "name": "ml.c5.24xlarge", 514 | "vcpuNum": 96 515 | }, 516 | { 517 | "_defaultOrder": 29, 518 | "_isFastLaunch": true, 519 | "category": "Accelerated computing", 520 | "gpuNum": 1, 521 | "hideHardwareSpecs": false, 522 | "memoryGiB": 16, 523 | "name": "ml.g4dn.xlarge", 524 | "vcpuNum": 4 525 | }, 526 | { 527 | "_defaultOrder": 30, 528 | "_isFastLaunch": false, 529 | "category": "Accelerated computing", 530 | "gpuNum": 1, 531 | "hideHardwareSpecs": false, 532 | "memoryGiB": 32, 533 | "name": "ml.g4dn.2xlarge", 534 | "vcpuNum": 8 535 | }, 536 | { 537 | "_defaultOrder": 31, 538 | "_isFastLaunch": false, 539 | "category": "Accelerated computing", 540 | "gpuNum": 1, 541 | "hideHardwareSpecs": false, 542 | "memoryGiB": 64, 543 | "name": "ml.g4dn.4xlarge", 544 | "vcpuNum": 16 545 | }, 546 | { 547 | "_defaultOrder": 32, 548 | "_isFastLaunch": false, 549 | "category": "Accelerated computing", 550 | "gpuNum": 1, 551 | "hideHardwareSpecs": false, 552 | "memoryGiB": 128, 553 | "name": "ml.g4dn.8xlarge", 554 | "vcpuNum": 32 555 | }, 556 | { 557 | "_defaultOrder": 33, 558 | "_isFastLaunch": false, 559 | "category": "Accelerated computing", 560 | "gpuNum": 4, 561 | "hideHardwareSpecs": false, 562 | "memoryGiB": 192, 563 | "name": "ml.g4dn.12xlarge", 564 | "vcpuNum": 48 565 | }, 566 | { 567 | "_defaultOrder": 34, 568 | "_isFastLaunch": false, 569 | "category": "Accelerated computing", 570 | "gpuNum": 1, 571 | "hideHardwareSpecs": false, 572 | "memoryGiB": 256, 573 | "name": "ml.g4dn.16xlarge", 574 | "vcpuNum": 64 575 | }, 576 | { 577 | "_defaultOrder": 35, 578 | "_isFastLaunch": false, 579 | "category": "Accelerated computing", 580 | "gpuNum": 1, 581 | "hideHardwareSpecs": false, 582 | "memoryGiB": 61, 583 | "name": "ml.p3.2xlarge", 584 | "vcpuNum": 8 585 | }, 586 | { 587 | "_defaultOrder": 36, 588 | "_isFastLaunch": false, 589 | "category": "Accelerated computing", 590 | "gpuNum": 4, 591 | "hideHardwareSpecs": false, 592 | "memoryGiB": 244, 593 | "name": "ml.p3.8xlarge", 594 | "vcpuNum": 32 595 | }, 596 | { 597 | "_defaultOrder": 37, 598 | "_isFastLaunch": false, 599 | "category": "Accelerated computing", 600 | "gpuNum": 8, 601 | "hideHardwareSpecs": false, 602 | "memoryGiB": 488, 603 | "name": "ml.p3.16xlarge", 604 | "vcpuNum": 64 605 | }, 606 | { 607 | "_defaultOrder": 38, 608 | "_isFastLaunch": false, 609 | "category": "Accelerated computing", 610 | "gpuNum": 8, 611 | "hideHardwareSpecs": false, 612 | "memoryGiB": 768, 613 | "name": "ml.p3dn.24xlarge", 614 | "vcpuNum": 96 615 | }, 616 | { 617 | "_defaultOrder": 39, 618 | "_isFastLaunch": false, 619 | "category": "Memory Optimized", 620 | "gpuNum": 0, 621 | "hideHardwareSpecs": false, 622 | "memoryGiB": 16, 623 | "name": "ml.r5.large", 624 | "vcpuNum": 2 625 | }, 626 | { 627 | "_defaultOrder": 40, 628 | "_isFastLaunch": false, 629 | "category": "Memory Optimized", 630 | "gpuNum": 0, 631 | "hideHardwareSpecs": false, 632 | "memoryGiB": 32, 633 | "name": "ml.r5.xlarge", 634 | "vcpuNum": 4 635 | }, 636 | { 637 | "_defaultOrder": 41, 638 | "_isFastLaunch": false, 639 | "category": "Memory Optimized", 640 | "gpuNum": 0, 641 | "hideHardwareSpecs": false, 642 | "memoryGiB": 64, 643 | "name": "ml.r5.2xlarge", 644 | "vcpuNum": 8 645 | }, 646 | { 647 | "_defaultOrder": 42, 648 | "_isFastLaunch": false, 649 | "category": "Memory Optimized", 650 | "gpuNum": 0, 651 | "hideHardwareSpecs": false, 652 | "memoryGiB": 128, 653 | "name": "ml.r5.4xlarge", 654 | "vcpuNum": 16 655 | }, 656 | { 657 | "_defaultOrder": 43, 658 | "_isFastLaunch": false, 659 | "category": "Memory Optimized", 660 | "gpuNum": 0, 661 | "hideHardwareSpecs": false, 662 | "memoryGiB": 256, 663 | "name": "ml.r5.8xlarge", 664 | "vcpuNum": 32 665 | }, 666 | { 667 | "_defaultOrder": 44, 668 | "_isFastLaunch": false, 669 | "category": "Memory Optimized", 670 | "gpuNum": 0, 671 | "hideHardwareSpecs": false, 672 | "memoryGiB": 384, 673 | "name": "ml.r5.12xlarge", 674 | "vcpuNum": 48 675 | }, 676 | { 677 | "_defaultOrder": 45, 678 | "_isFastLaunch": false, 679 | "category": "Memory Optimized", 680 | "gpuNum": 0, 681 | "hideHardwareSpecs": false, 682 | "memoryGiB": 512, 683 | "name": "ml.r5.16xlarge", 684 | "vcpuNum": 64 685 | }, 686 | { 687 | "_defaultOrder": 46, 688 | "_isFastLaunch": false, 689 | "category": "Memory Optimized", 690 | "gpuNum": 0, 691 | "hideHardwareSpecs": false, 692 | "memoryGiB": 768, 693 | "name": "ml.r5.24xlarge", 694 | "vcpuNum": 96 695 | }, 696 | { 697 | "_defaultOrder": 47, 698 | "_isFastLaunch": false, 699 | "category": "Accelerated computing", 700 | "gpuNum": 1, 701 | "hideHardwareSpecs": false, 702 | "memoryGiB": 16, 703 | "name": "ml.g5.xlarge", 704 | "vcpuNum": 4 705 | }, 706 | { 707 | "_defaultOrder": 48, 708 | "_isFastLaunch": false, 709 | "category": "Accelerated computing", 710 | "gpuNum": 1, 711 | "hideHardwareSpecs": false, 712 | "memoryGiB": 32, 713 | "name": "ml.g5.2xlarge", 714 | "vcpuNum": 8 715 | }, 716 | { 717 | "_defaultOrder": 49, 718 | "_isFastLaunch": false, 719 | "category": "Accelerated computing", 720 | "gpuNum": 1, 721 | "hideHardwareSpecs": false, 722 | "memoryGiB": 64, 723 | "name": "ml.g5.4xlarge", 724 | "vcpuNum": 16 725 | }, 726 | { 727 | "_defaultOrder": 50, 728 | "_isFastLaunch": false, 729 | "category": "Accelerated computing", 730 | "gpuNum": 1, 731 | "hideHardwareSpecs": false, 732 | "memoryGiB": 128, 733 | "name": "ml.g5.8xlarge", 734 | "vcpuNum": 32 735 | }, 736 | { 737 | "_defaultOrder": 51, 738 | "_isFastLaunch": false, 739 | "category": "Accelerated computing", 740 | "gpuNum": 1, 741 | "hideHardwareSpecs": false, 742 | "memoryGiB": 256, 743 | "name": "ml.g5.16xlarge", 744 | "vcpuNum": 64 745 | }, 746 | { 747 | "_defaultOrder": 52, 748 | "_isFastLaunch": false, 749 | "category": "Accelerated computing", 750 | "gpuNum": 4, 751 | "hideHardwareSpecs": false, 752 | "memoryGiB": 192, 753 | "name": "ml.g5.12xlarge", 754 | "vcpuNum": 48 755 | }, 756 | { 757 | "_defaultOrder": 53, 758 | "_isFastLaunch": false, 759 | "category": "Accelerated computing", 760 | "gpuNum": 4, 761 | "hideHardwareSpecs": false, 762 | "memoryGiB": 384, 763 | "name": "ml.g5.24xlarge", 764 | "vcpuNum": 96 765 | }, 766 | { 767 | "_defaultOrder": 54, 768 | "_isFastLaunch": false, 769 | "category": "Accelerated computing", 770 | "gpuNum": 8, 771 | "hideHardwareSpecs": false, 772 | "memoryGiB": 768, 773 | "name": "ml.g5.48xlarge", 774 | "vcpuNum": 192 775 | }, 776 | { 777 | "_defaultOrder": 55, 778 | "_isFastLaunch": false, 779 | "category": "Accelerated computing", 780 | "gpuNum": 8, 781 | "hideHardwareSpecs": false, 782 | "memoryGiB": 1152, 783 | "name": "ml.p4d.24xlarge", 784 | "vcpuNum": 96 785 | }, 786 | { 787 | "_defaultOrder": 56, 788 | "_isFastLaunch": false, 789 | "category": "Accelerated computing", 790 | "gpuNum": 8, 791 | "hideHardwareSpecs": false, 792 | "memoryGiB": 1152, 793 | "name": "ml.p4de.24xlarge", 794 | "vcpuNum": 96 795 | }, 796 | { 797 | "_defaultOrder": 57, 798 | "_isFastLaunch": false, 799 | "category": "Accelerated computing", 800 | "gpuNum": 0, 801 | "hideHardwareSpecs": false, 802 | "memoryGiB": 32, 803 | "name": "ml.trn1.2xlarge", 804 | "vcpuNum": 8 805 | }, 806 | { 807 | "_defaultOrder": 58, 808 | "_isFastLaunch": false, 809 | "category": "Accelerated computing", 810 | "gpuNum": 0, 811 | "hideHardwareSpecs": false, 812 | "memoryGiB": 512, 813 | "name": "ml.trn1.32xlarge", 814 | "vcpuNum": 128 815 | }, 816 | { 817 | "_defaultOrder": 59, 818 | "_isFastLaunch": false, 819 | "category": "Accelerated computing", 820 | "gpuNum": 0, 821 | "hideHardwareSpecs": false, 822 | "memoryGiB": 512, 823 | "name": "ml.trn1n.32xlarge", 824 | "vcpuNum": 128 825 | } 826 | ], 827 | "instance_type": "ml.t3.medium", 828 | "kernelspec": { 829 | "display_name": "Python 3 (Data Science 3.0)", 830 | "language": "python", 831 | "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:123456012:image/sagemaker-data-science-310-v1" 832 | }, 833 | "language_info": { 834 | "codemirror_mode": { 835 | "name": "ipython", 836 | "version": 3 837 | }, 838 | "file_extension": ".py", 839 | "mimetype": "text/x-python", 840 | "name": "python", 841 | "nbconvert_exporter": "python", 842 | "pygments_lexer": "ipython3", 843 | "version": "3.10.6" 844 | } 845 | }, 846 | "nbformat": 4, 847 | "nbformat_minor": 5 848 | } 849 | -------------------------------------------------------------------------------- /langfuse-v2/requirements.txt: -------------------------------------------------------------------------------- 1 | aws-cdk-lib==2.172.0 2 | constructs>=10.0.0,<11.0.0 3 | cdk-ecr-deployment==3.0.82 4 | -------------------------------------------------------------------------------- /langfuse-v2/source.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem The sole purpose of this script is to make the command 4 | rem 5 | rem source .venv/bin/activate 6 | rem 7 | rem (which activates a Python virtualenv on Linux or Mac OS X) work on Windows. 8 | rem On Windows, this command just runs this batch file (the argument is ignored). 9 | rem 10 | rem Now we don't need to document a Windows command for activating a virtualenv. 11 | 12 | echo Executing .venv\Scripts\activate.bat for you 13 | .venv\Scripts\activate.bat 14 | -------------------------------------------------------------------------------- /langfuse-v3/.envrc: -------------------------------------------------------------------------------- 1 | export USE_DEFAULT_VPC=true 2 | -------------------------------------------------------------------------------- /langfuse-v3/.example.cdk.context.json: -------------------------------------------------------------------------------- 1 | { 2 | "private_dns_namespace_name": "langfuse.local", 3 | "db_cluster_name": "langfuse-db", 4 | "ecr": [ 5 | { 6 | "repository_name": "langfuse-web", 7 | "docker_image_name": "langfuse/langfuse", 8 | "tag": "3" 9 | }, 10 | { 11 | "repository_name": "langfuse-worker", 12 | "docker_image_name": "langfuse/langfuse-worker", 13 | "tag": "3" 14 | }, 15 | { 16 | "repository_name": "clickhouse", 17 | "docker_image_name": "clickhouse", 18 | "tag": "24.12.3.47" 19 | } 20 | ], 21 | "ecs_cluster_name": "langfuse", 22 | "langfuse_worker_desired_count": 1, 23 | "langfuse_worker_env": { 24 | "NODE_ENV": "production", 25 | "SALT": "salt (generate by running 'openssl rand -base64 32')", 26 | "ENCRYPTION_KEY": "encryption key (generate by running 'openssl rand -hex 32')", 27 | "TELEMETRY_ENABLED": "true", 28 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true" 29 | }, 30 | "langfuse_web_env": { 31 | "NODE_ENV": "production", 32 | "NEXTAUTH_SECRET": "secret (generate by running 'openssl rand -base64 32')", 33 | "SALT": "salt (generate by running 'openssl rand -base64 32')", 34 | "ENCRYPTION_KEY": "encryption key (generate by running 'openssl rand -hex 32')", 35 | "HOSTNAME": "0.0.0.0", 36 | "LANGFUSE_S3_MEDIA_DOWNLOAD_URL_EXPIRY_SECONDS": "604800", 37 | "TELEMETRY_ENABLED": "true", 38 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true", 39 | "LANGFUSE_SDK_CI_SYNC_PROCESSING_ENABLED": "false", 40 | "LANGFUSE_READ_FROM_POSTGRES_ONLY": "false", 41 | "LANGFUSE_READ_FROM_CLICKHOUSE_ONLY": "true", 42 | "LANGFUSE_RETURN_FROM_CLICKHOUSE": "true" 43 | } 44 | } -------------------------------------------------------------------------------- /langfuse-v3/.gitignore: -------------------------------------------------------------------------------- 1 | .gitignore 2 | *.swp 3 | package-lock.json 4 | __pycache__ 5 | .pytest_cache 6 | .venv 7 | *.egg-info 8 | .DS_Store 9 | *.bak 10 | 11 | # CDK asset staging directory 12 | .cdk.staging 13 | cdk.out 14 | cdk.context.json 15 | -------------------------------------------------------------------------------- /langfuse-v3/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Hosting Langfuse V3 on Amazon ECS with Fargate using CDK Python 3 | 4 | This repository contains the AWS CDK Python code for deploying the [Langfuse](https://langfuse.com/) application using Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS). 5 | 6 | Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. 7 | 8 | ![lanfuse-v3-on-aws-ecs-fargate-arch](./langfuse-v3-on-aws-ecs-fargate-arch.svg) 9 | > :information_source: For more information on Langfuse's architecture, please check [the official documentation](https://langfuse.com/self-hosting#architecture) 10 | 11 | The `cdk.json` file tells the CDK Toolkit how to execute your app. 12 | 13 | This project is set up like a standard Python project. The initialization 14 | process also creates a virtualenv within this project, stored under the `.venv` 15 | directory. To create the virtualenv it assumes that there is a `python3` 16 | (or `python` for Windows) executable in your path with access to the `venv` 17 | package. If for any reason the automatic creation of the virtualenv fails, 18 | you can create the virtualenv manually. 19 | 20 | To manually create a virtualenv on MacOS and Linux: 21 | 22 | ``` 23 | $ git clone --depth=1 https://github.com/aws-samples/deploy-langfuse-on-ecs-with-fargate.git 24 | $ cd deploy-langfuse-on-ecs-with-fargate 25 | $ git sparse-checkout init --cone 26 | $ git sparse-checkout set langfuse-v3 27 | $ cd langfuse-v3 28 | 29 | $ python3 -m venv .venv 30 | ``` 31 | 32 | After the init process completes and the virtualenv is created, you can use the following 33 | step to activate your virtualenv. 34 | 35 | ``` 36 | $ source .venv/bin/activate 37 | ``` 38 | 39 | If you are a Windows platform, you would activate the virtualenv like this: 40 | 41 | ``` 42 | % .venv\Scripts\activate.bat 43 | ``` 44 | 45 | Once the virtualenv is activated, you can install the required dependencies. 46 | 47 | ``` 48 | (.venv) $ pip install -r requirements.txt 49 | ``` 50 | > To add additional dependencies, for example other CDK libraries, just add 51 | them to your `setup.py` file and rerun the `pip install -r requirements.txt` 52 | command. 53 | 54 | ## Prerequisites 55 | 56 | **Set up `cdk.context.json`** 57 | 58 | Then, we need to set approperly the cdk context configuration file, `cdk.context.json`. 59 | 60 | For example, 61 | 62 | ``` 63 | { 64 | "private_dns_namespace_name": "langfuse.local", 65 | "db_cluster_name": "langfuse-db", 66 | "ecr": [ 67 | { 68 | "repository_name": "langfuse-web", 69 | "docker_image_name": "langfuse/langfuse", 70 | "tag": "3" 71 | }, 72 | { 73 | "repository_name": "langfuse-worker", 74 | "docker_image_name": "langfuse/langfuse-worker", 75 | "tag": "3" 76 | }, 77 | { 78 | "repository_name": "clickhouse", 79 | "docker_image_name": "clickhouse", 80 | "tag": "24.12.3.47" 81 | } 82 | ], 83 | "ecs_cluster_name": "langfuse", 84 | "langfuse_worker_desired_count": 1, 85 | "langfuse_worker_env": { 86 | "NODE_ENV": "production", 87 | "SALT": "salt (generate by running 'openssl rand -base64 32')", 88 | "ENCRYPTION_KEY": "encryption key (generate by running 'openssl rand -hex 32')", 89 | "TELEMETRY_ENABLED": "true", 90 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true" 91 | }, 92 | "langfuse_web_env": { 93 | "NODE_ENV": "production", 94 | "NEXTAUTH_SECRET": "secret (generate by running 'openssl rand -base64 32')", 95 | "SALT": "salt (generate by running 'openssl rand -base64 32')", 96 | "ENCRYPTION_KEY": "encryption key (generate by running 'openssl rand -hex 32')", 97 | "HOSTNAME": "0.0.0.0", 98 | "LANGFUSE_S3_MEDIA_DOWNLOAD_URL_EXPIRY_SECONDS": "604800", 99 | "TELEMETRY_ENABLED": "true", 100 | "LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES": "true", 101 | "LANGFUSE_SDK_CI_SYNC_PROCESSING_ENABLED": "false", 102 | "LANGFUSE_READ_FROM_POSTGRES_ONLY": "false", 103 | "LANGFUSE_READ_FROM_CLICKHOUSE_ONLY": "true", 104 | "LANGFUSE_RETURN_FROM_CLICKHOUSE": "true" 105 | } 106 | } 107 | ``` 108 | 109 | :information_source: This guide covers Langfuse v3. The docker image version (`tag`) of `langfuse-web` and `langfuse-worker` should be set to `3`. 110 | 111 | :information_source: For more details on environment variables for the Langfuse Web (`langfuse_web_env`) and Langfuse Worker (`langfuse_worker_env`) containers, please refer to the [official configuration guide](https://langfuse.com/self-hosting/configuration#environment-variables). 112 | 113 | **Bootstrap AWS environment for AWS CDK app** 114 | 115 | Also, before any AWS CDK app can be deployed, you have to bootstrap your AWS environment to create certain AWS resources that the AWS CDK CLI (Command Line Interface) uses to deploy your AWS CDK app. 116 | 117 | Run the `cdk bootstrap` command to bootstrap the AWS environment. 118 | 119 | ``` 120 | (.venv) $ cdk bootstrap 121 | ``` 122 | 123 | ### Deploy 124 | 125 | At this point you can now synthesize the CloudFormation template for this code. 126 | 127 | ``` 128 | (.venv) $ export CDK_DEFAULT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text) 129 | (.venv) $ export CDK_DEFAULT_REGION=$(aws configure get region) 130 | (.venv) $ cdk synth --all 131 | ``` 132 | 133 | Use `cdk deploy` command to create the stack shown above. 134 | 135 | ``` 136 | (.venv) $ cdk deploy --require-approval never --all 137 | ``` 138 | 139 | We can list all the CDK stacks by using the `cdk list` command prior to deployment. 140 | 141 | ``` 142 | (.venv) $ cdk list 143 | LangfuseECRStack 144 | LangfuseVpcStack 145 | LangfuseWebALBStack 146 | LangfuseCacheStack 147 | LangfuseAuroraPostgreSQLStack 148 | LangfuseS3BucketStack 149 | LangfuseServiceDiscoveryStack 150 | LangfuseECSClusterStack 151 | LangfuseClickhouseEFSStack 152 | LangfuseClickhouseECSTaskStack 153 | LangfuseClickhouseECSServiceStack 154 | LangfuseWorkerECSTaskStack 155 | LangfuseWorkerECSServiceStack 156 | LangfuseWebECSTaskStack 157 | LangfuseWebECSServiceStack 158 | ``` 159 | 160 | ## Clean Up 161 | 162 | Delete the CloudFormation stack by running the below command. 163 | 164 | ``` 165 | (.venv) $ cdk destroy --force --all 166 | ``` 167 | 168 | ## Useful commands 169 | 170 | * `cdk ls` list all stacks in the app 171 | * `cdk synth` emits the synthesized CloudFormation template 172 | * `cdk deploy` deploy this stack to your default AWS account/region 173 | * `cdk diff` compare deployed stack with current state 174 | * `cdk docs` open CDK documentation 175 | 176 | Enjoy! 177 | 178 | ## Tracing for your LLM Application with Langfuse 179 | 180 | After deploying all CDK stacks, you can find the **Langfuse URL** using the following command: 181 | 182 | ```bash 183 | aws cloudformation describe-stacks --stack-name LangfuseWebECSServiceStack --region ${CDK_DEFAULT_REGION} | \ 184 | jq -r '.Stacks[0].Outputs | map(select(.OutputKey == "LoadBalancerDNS")) | .[0].OutputValue' 185 | ``` 186 | 187 | Next, open the **Langfuse URL** in your browser to create a new project for tracking your LLM application with Langfuse. 188 | 189 | ### Create a New Project in Langfuse 190 | 191 | 1. Create a Langfuse Account 192 | 193 | ![Login](./assets/01-langfuse-main-page.png) 194 | 195 | ![Singup](./assets/02-sign-up.png) 196 | 2. Create a New Project 197 | ![New-Project](./assets/03-new-project.png) 198 | 3. Create New API Credentials in the Project Settings 199 | ![API-Keys](./assets/04-create-api-keys.png) 200 | 201 | ### Log Your First LLM Call to Langfuse 202 | 203 | Open the `tracing_for_langchain_bedrock` notebook in the `examples` folder and run it. (See [here](./examples/tracing_for_langchain_bedrock.ipynb) for more information) 204 | 205 | You will the see the list of traces as follows: 206 | ![Traces](./assets/05-traces.png) 207 | 208 | You will also see the details of the selected trace as follows: 209 | 210 | ![Trace-detail](./assets/06-trace-detail.png) 211 | 212 | ## References 213 | 214 | #### General 215 | 216 | * [(Official) Self-host Langfuse](https://langfuse.com/self-hosting) 217 | * [(Official) Langfuse configuration guide](https://langfuse.com/self-hosting/configuration) 218 | * [(GitHub) langfuse](https://github.com/langfuse/langfuse/) 219 | * [(GitHub) Langfuse v3 Terraform Module Sample](https://github.com/tubone24/langfuse-v3-terraform/) 220 | * [AWS CDK Reference Documentation](https://docs.aws.amazon.com/cdk/api/v2/) 221 | * [cdk-ecr-deployment](https://github.com/cdklabs/cdk-ecr-deployment) - A CDK construct to deploy docker image to Amazon ECR. 222 | * [Terraform - AWS Provider Docs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) 223 | 224 | #### Langfuse User Guide 225 | 226 | * [(Workshop) GenAIOps - Observability for GenAI applications with Amazon Bedrock and Langfuse](https://catalog.workshops.aws/genaiops-langfuse) 227 | * [Get Started with Langfuse Tracing](https://langfuse.com/docs/get-started) 228 | * [Observability & Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing) 229 | 230 | #### Amazon ElastiCache 231 | 232 | * [Amazon ElastiCache Supported node types](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/CacheNodes.SupportedTypes.html#CacheNodes.CurrentGen) 233 | * [Amazon ElastiCache Supported engines and versions](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/supported-engine-versions.html) 234 | * [Comparing Valkey, Memcached, and Redis OSS self-designed caches](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/SelectEngine.html) 235 | 236 | #### Amazon Aurora PostgreSQL 237 | 238 | * [Working with Amazon Aurora PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.AuroraPostgreSQL.html) 239 | 240 | #### Clickhouse 241 | 242 | * [Clickhouse for self-hosting Langfuse v3](https://langfuse.com/self-hosting/infrastructure/clickhouse) 243 | * [(DockerHub) Clickhouse Docker Official Image](https://hub.docker.com/_/clickhouse) 244 | * [Clickhouse CLI](https://clickhouse.com/docs/en/integrations/sql-clients/clickhouse-client-local) 245 | 246 | ## Security 247 | 248 | See [CONTRIBUTING](../CONTRIBUTING.md#security-issue-notifications) for more information. 249 | 250 | ## License 251 | 252 | This library is licensed under the MIT-0 License. See the LICENSE file. 253 | -------------------------------------------------------------------------------- /langfuse-v3/app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import os 6 | 7 | import aws_cdk as cdk 8 | 9 | from cdk_stacks import ( 10 | ALBLangfuseWebStack, 11 | AuroraPostgresqlStack, 12 | ECRStack, 13 | ECSClusterStack, 14 | ECSTaskClickhouseStack, 15 | ECSFargateServiceClickhouseStack, 16 | EFSStack, 17 | ECSTaskLangfuseWebStack, 18 | ECSTaskLangfuseWorkerStack, 19 | ECSFargateServiceLangfuseWebStack, 20 | ECSFargateServiceLangfuseWorkerStack, 21 | RedisClusterStack, 22 | S3BucketStack, 23 | ServiceDiscoveryStack, 24 | VpcStack 25 | ) 26 | 27 | AWS_ENV = cdk.Environment( 28 | account=os.environ["CDK_DEFAULT_ACCOUNT"], 29 | region=os.environ["CDK_DEFAULT_REGION"] 30 | ) 31 | 32 | app = cdk.App() 33 | 34 | ecr_stack = ECRStack(app, "LangfuseECRStack", 35 | env=AWS_ENV 36 | ) 37 | 38 | vpc_stack = VpcStack(app, "LangfuseVpcStack", 39 | env=AWS_ENV 40 | ) 41 | vpc_stack.add_dependency(ecr_stack) 42 | 43 | langfuse_web_alb_stack = ALBLangfuseWebStack(app, "LangfuseWebALBStack", 44 | vpc_stack.vpc, 45 | env=AWS_ENV 46 | ) 47 | langfuse_web_alb_stack.add_dependency(vpc_stack) 48 | 49 | redis_stack = RedisClusterStack(app, "LangfuseCacheStack", 50 | vpc_stack.vpc, 51 | env=AWS_ENV 52 | ) 53 | redis_stack.add_dependency(langfuse_web_alb_stack) 54 | 55 | rds_stack = AuroraPostgresqlStack(app, "LangfuseAuroraPostgreSQLStack", 56 | vpc_stack.vpc, 57 | env=AWS_ENV 58 | ) 59 | rds_stack.add_dependency(redis_stack) 60 | 61 | s3_buckets = S3BucketStack(app, "LangfuseS3BucketStack", 62 | env=AWS_ENV 63 | ) 64 | s3_buckets.add_dependency(rds_stack) 65 | 66 | service_discovery_stack = ServiceDiscoveryStack(app, "LangfuseServiceDiscoveryStack", 67 | vpc_stack.vpc, 68 | env=AWS_ENV) 69 | service_discovery_stack.add_dependency(s3_buckets) 70 | 71 | ecs_cluster_stack = ECSClusterStack(app, "LangfuseECSClusterStack", 72 | vpc_stack.vpc, 73 | env=AWS_ENV 74 | ) 75 | ecs_cluster_stack.add_dependency(service_discovery_stack) 76 | 77 | clickhouse_efs_stack = EFSStack(app, "LangfuseClickhouseEFSStack", 78 | vpc_stack.vpc, 79 | env=AWS_ENV 80 | ) 81 | clickhouse_efs_stack.add_dependency(ecs_cluster_stack) 82 | 83 | clickhouse_stack = ECSTaskClickhouseStack(app, "LangfuseClickhouseECSTaskStack", 84 | ecr_stack.repositories, 85 | clickhouse_efs_stack.efs_file_system, 86 | env=AWS_ENV 87 | ) 88 | clickhouse_stack.add_dependency(clickhouse_efs_stack) 89 | 90 | clickhouse_service_stack = ECSFargateServiceClickhouseStack(app, "LangfuseClickhouseECSServiceStack", 91 | vpc_stack.vpc, 92 | ecs_cluster_stack.ecs_cluster, 93 | clickhouse_stack.ecs_task_definition, 94 | clickhouse_efs_stack.sg_efs_inbound, 95 | service_discovery_stack.service, 96 | env=AWS_ENV 97 | ) 98 | clickhouse_service_stack.add_dependency(clickhouse_stack) 99 | 100 | langfuse_worker_stack = ECSTaskLangfuseWorkerStack(app, "LangfuseWorkerECSTaskStack", 101 | ecr_repositories=ecr_stack.repositories, 102 | database_secret=rds_stack.database_secret, 103 | clickhouse_secret=clickhouse_stack.clickhouse_secret, 104 | clickhouse_migration_url=clickhouse_service_stack.clickhouse_migration_url, 105 | clickhouse_url=clickhouse_service_stack.clickhouse_url, 106 | redis_cluster=redis_stack.redis_cluster, 107 | s3_blob_bucket=s3_buckets.blob_bucket, 108 | s3_event_bucket=s3_buckets.event_bucket, 109 | env=AWS_ENV 110 | ) 111 | langfuse_worker_stack.add_dependency(clickhouse_service_stack) 112 | 113 | langfuse_worker_service_stack = ECSFargateServiceLangfuseWorkerStack(app, "LangfuseWorkerECSServiceStack", 114 | vpc=vpc_stack.vpc, 115 | ecs_cluster=ecs_cluster_stack.ecs_cluster, 116 | ecs_task_definition=langfuse_worker_stack.ecs_task_definition, 117 | sg_redis_client=redis_stack.sg_elasticache_client, 118 | sg_rds_client=rds_stack.sg_rds_client, 119 | sg_clickhouse_client=clickhouse_service_stack.sg_clickhouse_client, 120 | env=AWS_ENV 121 | ) 122 | langfuse_worker_service_stack.add_dependency(langfuse_worker_stack) 123 | 124 | langfuse_web_stack = ECSTaskLangfuseWebStack(app, "LangfuseWebECSTaskStack", 125 | ecr_repositories=ecr_stack.repositories, 126 | database_secret=rds_stack.database_secret, 127 | clickhouse_secret=clickhouse_stack.clickhouse_secret, 128 | clickhouse_migration_url=clickhouse_service_stack.clickhouse_migration_url, 129 | clickhouse_url=clickhouse_service_stack.clickhouse_url, 130 | redis_cluster=redis_stack.redis_cluster, 131 | s3_blob_bucket=s3_buckets.blob_bucket, 132 | s3_event_bucket=s3_buckets.event_bucket, 133 | load_balancer_url=langfuse_web_alb_stack.load_balancer_url, 134 | env=AWS_ENV 135 | ) 136 | langfuse_web_stack.add_dependency(langfuse_worker_service_stack) 137 | 138 | langfuse_web_service_stack = ECSFargateServiceLangfuseWebStack(app, "LangfuseWebECSServiceStack", 139 | vpc=vpc_stack.vpc, 140 | ecs_cluster=ecs_cluster_stack.ecs_cluster, 141 | ecs_task_definition=langfuse_web_stack.ecs_task_definition, 142 | sg_redis_client=redis_stack.sg_elasticache_client, 143 | sg_rds_client=rds_stack.sg_rds_client, 144 | sg_clickhouse_client=clickhouse_service_stack.sg_clickhouse_client, 145 | alb_listener=langfuse_web_alb_stack.alb_listener, 146 | env=AWS_ENV 147 | ) 148 | langfuse_web_service_stack.add_dependency(langfuse_web_stack) 149 | 150 | app.synth() 151 | -------------------------------------------------------------------------------- /langfuse-v3/assets/01-langfuse-main-page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/01-langfuse-main-page.png -------------------------------------------------------------------------------- /langfuse-v3/assets/02-sign-up.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/02-sign-up.png -------------------------------------------------------------------------------- /langfuse-v3/assets/03-new-project.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/03-new-project.png -------------------------------------------------------------------------------- /langfuse-v3/assets/04-create-api-keys.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/04-create-api-keys.png -------------------------------------------------------------------------------- /langfuse-v3/assets/05-traces.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/05-traces.png -------------------------------------------------------------------------------- /langfuse-v3/assets/06-trace-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/deploy-langfuse-on-ecs-with-fargate/904d421be145e76ab4074fd3e90246561790e210/langfuse-v3/assets/06-trace-detail.png -------------------------------------------------------------------------------- /langfuse-v3/cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "requirements*.txt", 11 | "source.bat", 12 | "**/__init__.py", 13 | "**/__pycache__", 14 | "tests" 15 | ] 16 | }, 17 | "context": { 18 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 19 | "@aws-cdk/core:checkSecretUsage": true, 20 | "@aws-cdk/core:target-partitions": [ 21 | "aws", 22 | "aws-cn" 23 | ], 24 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 25 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 26 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 27 | "@aws-cdk/aws-iam:minimizePolicies": true, 28 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 29 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 30 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 31 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 32 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 33 | "@aws-cdk/core:enablePartitionLiterals": true, 34 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 35 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 36 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 37 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 38 | "@aws-cdk/aws-route53-patters:useCertificate": true, 39 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 40 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 41 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 42 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 43 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 44 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 45 | "@aws-cdk/aws-redshift:columnId": true, 46 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, 47 | "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, 48 | "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, 49 | "@aws-cdk/aws-kms:aliasNameRef": true, 50 | "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, 51 | "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, 52 | "@aws-cdk/aws-efs:denyAnonymousAccess": true, 53 | "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, 54 | "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, 55 | "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, 56 | "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, 57 | "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, 58 | "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true, 59 | "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true, 60 | "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true, 61 | "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, 62 | "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true, 63 | "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, 64 | "@aws-cdk/aws-eks:nodegroupNameAttribute": true, 65 | "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true, 66 | "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true, 67 | "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false, 68 | "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false, 69 | "@aws-cdk/aws-ecs:enableImdsBlockingDeprecatedFeature": false, 70 | "@aws-cdk/aws-ecs:disableEcsImdsBlocking": true, 71 | "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true, 72 | "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true, 73 | "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true, 74 | "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true, 75 | "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true, 76 | "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true, 77 | "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true, 78 | "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true, 79 | "@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true, 80 | "@aws-cdk/aws-route53-targets:userPoolDomainNameMethodWithoutCustomResource": true, 81 | "@aws-cdk/aws-elasticloadbalancingV2:albDualstackWithoutPublicIpv4SecurityGroupRulesDefault": true, 82 | "@aws-cdk/aws-iam:oidcRejectUnauthorizedConnections": true 83 | } 84 | } 85 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/__init__.py: -------------------------------------------------------------------------------- 1 | from .alb_langfuse_web import ALBLangfuseWebStack 2 | from .aurora_postgresql import AuroraPostgresqlStack 3 | from .ecr import ECRStack 4 | from .ecs_cluster import ECSClusterStack 5 | from .ecs_fargate_service_clickhouse import ECSFargateServiceClickhouseStack 6 | from .ecs_fargate_service_langfuse_web import ECSFargateServiceLangfuseWebStack 7 | from .ecs_fargate_service_langfuse_worker import ECSFargateServiceLangfuseWorkerStack 8 | from .ecs_task_clickhouse import ECSTaskClickhouseStack 9 | from .ecs_task_langfuse_web import ECSTaskLangfuseWebStack 10 | from .ecs_task_langfuse_worker import ECSTaskLangfuseWorkerStack 11 | from .efs_clickhouse import EFSStack 12 | from .redis import RedisClusterStack 13 | from .s3 import S3BucketStack 14 | from .service_discovery import ServiceDiscoveryStack 15 | from .vpc import VpcStack 16 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/alb_langfuse_web.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_elasticloadbalancingv2 as elbv2 11 | ) 12 | 13 | from constructs import Construct 14 | 15 | 16 | class ALBLangfuseWebStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 19 | 20 | super().__init__(scope, construct_id, **kwargs) 21 | 22 | sg_alb = aws_ec2.SecurityGroup(self, 'LoadBalancerSG', 23 | vpc=vpc, 24 | allow_all_outbound=True, 25 | description="Allow inbound for Langfuse Web", 26 | security_group_name=f'{self.stack_name.lower()}-langfuse-web-alb-sg' 27 | ) 28 | sg_alb.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 29 | connection=aws_ec2.Port.tcp(80)) 30 | sg_alb.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 31 | connection=aws_ec2.Port.tcp(443)) 32 | cdk.Tags.of(sg_alb).add('Name', 'langfuse-web-alb-sg') 33 | 34 | self.load_balancer = elbv2.ApplicationLoadBalancer(self, "LoadBalancer", 35 | vpc=vpc, 36 | internet_facing=True, 37 | security_group=sg_alb 38 | ) 39 | self.load_balancer.apply_removal_policy(cdk.RemovalPolicy.DESTROY) 40 | 41 | self.alb_listener = self.load_balancer.add_listener("Listener", 42 | protocol=elbv2.ApplicationProtocol.HTTP) 43 | 44 | self.load_balancer_url = f'http://{self.load_balancer.load_balancer_dns_name}' 45 | 46 | 47 | cdk.CfnOutput(self, "LoadBalancerDNS", 48 | value=self.load_balancer_url, 49 | export_name=f'{self.stack_name}-LoadBalancerDNS') 50 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/aurora_postgresql.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import json 6 | 7 | import aws_cdk as cdk 8 | 9 | from aws_cdk import ( 10 | Stack, 11 | aws_ec2, 12 | aws_logs, 13 | aws_rds, 14 | aws_secretsmanager 15 | ) 16 | 17 | from constructs import Construct 18 | 19 | 20 | class AuroraPostgresqlStack(Stack): 21 | 22 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 23 | super().__init__(scope, construct_id, **kwargs) 24 | 25 | db_cluster_name = self.node.try_get_context('db_cluster_name') or 'langfuse-db' 26 | 27 | sg_postgresql_client = aws_ec2.SecurityGroup(self, 'PostgreSQLClientSG', 28 | vpc=vpc, 29 | allow_all_outbound=True, 30 | description='security group for postgresql client', 31 | security_group_name=f'{db_cluster_name}-postgresql-client-sg' 32 | ) 33 | cdk.Tags.of(sg_postgresql_client).add('Name', 'postgresql-client-sg') 34 | 35 | sg_postgresql_server = aws_ec2.SecurityGroup(self, 'PostgreSQLServerSG', 36 | vpc=vpc, 37 | allow_all_outbound=True, 38 | description='security group for postgresql', 39 | security_group_name=f'{db_cluster_name}-postgresql-server-sg' 40 | ) 41 | sg_postgresql_server.add_ingress_rule(peer=sg_postgresql_client, connection=aws_ec2.Port.tcp(5432), 42 | description='postgresql-client-sg') 43 | sg_postgresql_server.add_ingress_rule(peer=sg_postgresql_server, connection=aws_ec2.Port.all_tcp(), 44 | description='postgresql-server-sg') 45 | cdk.Tags.of(sg_postgresql_server).add('Name', 'postgresql-server-sg') 46 | 47 | rds_subnet_group = aws_rds.SubnetGroup(self, 'PostgreSQLSubnetGroup', 48 | description='subnet group for postgresql', 49 | subnet_group_name=f'aurora-postgresql-{self.stack_name}', 50 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS), 51 | vpc=vpc 52 | ) 53 | 54 | #XXX: In order to exclude punctuations when generating a password 55 | # use aws_secretsmanager.Secret instead of aws_rds.DatabaseSecret. 56 | # Othwerise, an error occurred such as: 57 | # "All characters of the desired type have been excluded" 58 | db_secret = aws_secretsmanager.Secret(self, 'DatabaseSecret', 59 | generate_secret_string=aws_secretsmanager.SecretStringGenerator( 60 | secret_string_template=json.dumps({"username": "postgres"}), 61 | generate_string_key="password", 62 | exclude_punctuation=True, 63 | password_length=8 64 | ) 65 | ) 66 | rds_credentials = aws_rds.Credentials.from_secret(db_secret) 67 | 68 | rds_engine = aws_rds.DatabaseClusterEngine.aurora_postgres(version=aws_rds.AuroraPostgresEngineVersion.VER_15_4) 69 | 70 | #XXX: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Reference.ParameterGroups.html#AuroraPostgreSQL.Reference.Parameters.Cluster 71 | rds_cluster_param_group = aws_rds.ParameterGroup(self, 'AuroraPostgreSQLClusterParamGroup', 72 | engine=rds_engine, 73 | description='Custom cluster parameter group for aurora-postgresql15', 74 | parameters={ 75 | 'log_min_duration_statement': '15000', # 15 sec 76 | 'default_transaction_isolation': 'read committed', 77 | 'client_encoding': 'UTF8' 78 | } 79 | ) 80 | 81 | #XXX: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Reference.ParameterGroups.html#AuroraPostgreSQL.Reference.Parameters.Instance 82 | rds_db_param_group = aws_rds.ParameterGroup(self, 'AuroraPostgreSQLDBParamGroup', 83 | engine=rds_engine, 84 | description='Custom parameter group for aurora-postgresql15', 85 | parameters={ 86 | 'log_min_duration_statement': '15000', # 15 sec 87 | 'default_transaction_isolation': 'read committed' 88 | } 89 | ) 90 | 91 | db_cluster = aws_rds.DatabaseCluster(self, 'Database', 92 | engine=rds_engine, 93 | credentials=rds_credentials, # A username of 'admin' (or 'postgres' for PostgreSQL) and SecretsManager-generated password 94 | writer=aws_rds.ClusterInstance.provisioned("writer", 95 | instance_type=aws_ec2.InstanceType.of(aws_ec2.InstanceClass.R6G, aws_ec2.InstanceSize.LARGE), 96 | parameter_group=rds_db_param_group, 97 | auto_minor_version_upgrade=False, 98 | ), 99 | readers=[ 100 | aws_rds.ClusterInstance.provisioned("reader", 101 | instance_type=aws_ec2.InstanceType.of(aws_ec2.InstanceClass.R6G, aws_ec2.InstanceSize.LARGE), 102 | parameter_group=rds_db_param_group, 103 | auto_minor_version_upgrade=False 104 | ) 105 | ], 106 | parameter_group=rds_cluster_param_group, 107 | cloudwatch_logs_retention=aws_logs.RetentionDays.THREE_DAYS, 108 | cluster_identifier=db_cluster_name, 109 | subnet_group=rds_subnet_group, 110 | backup=aws_rds.BackupProps( 111 | retention=cdk.Duration.days(3), 112 | preferred_window="03:00-04:00" 113 | ), 114 | security_groups=[sg_postgresql_server], 115 | vpc=vpc, 116 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS) 117 | ) 118 | 119 | self.sg_rds_client = sg_postgresql_client 120 | self.database_secret = db_cluster.secret 121 | self.database = db_cluster 122 | 123 | 124 | cdk.CfnOutput(self, 'DBClusterEndpoint', 125 | value=db_cluster.cluster_endpoint.socket_address, 126 | export_name=f'{self.stack_name}-DBClusterEndpoint') 127 | cdk.CfnOutput(self, 'DBClusterReadEndpoint', 128 | value=db_cluster.cluster_read_endpoint.socket_address, 129 | export_name=f'{self.stack_name}-DBClusterReadEndpoint') 130 | cdk.CfnOutput(self, 'RDSClientSecurityGroupId', 131 | value=sg_postgresql_client.security_group_id, 132 | export_name=f'{self.stack_name}-RDSClientSecurityGroupId') 133 | cdk.CfnOutput(self, 'DBSecretName', 134 | value=db_cluster.secret.secret_name, 135 | export_name=f'{self.stack_name}-DBSecretName') -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecr.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecr, 10 | ) 11 | from constructs import Construct 12 | import cdk_ecr_deployment as ecr_deploy 13 | 14 | 15 | class ECRStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 18 | super().__init__(scope, construct_id, **kwargs) 19 | 20 | ecr_repositories = self.node.try_get_context('ecr') 21 | 22 | self.repositories = {} 23 | 24 | for elem in ecr_repositories: 25 | repository_name = elem['repository_name'] 26 | 27 | repository = aws_ecr.Repository(self, f"{repository_name}ECRRepository", 28 | empty_on_delete=True, 29 | encryption=aws_ecr.RepositoryEncryption.AES_256, 30 | removal_policy=cdk.RemovalPolicy.DESTROY, 31 | repository_name=repository_name 32 | ) 33 | self.repositories[repository_name] = repository 34 | 35 | # delete images older than 7 days 36 | repository.add_lifecycle_rule( 37 | max_image_age=cdk.Duration.days(7), 38 | rule_priority=1, 39 | tag_status=aws_ecr.TagStatus.UNTAGGED 40 | ) 41 | 42 | # keep last 3 images 43 | repository.add_lifecycle_rule( 44 | max_image_count=3, 45 | rule_priority=2, 46 | tag_status=aws_ecr.TagStatus.ANY 47 | ) 48 | 49 | src_docker_image_version = elem['docker_image_name'] 50 | image_version = elem.get('tag', 'latest') 51 | deploy_image_versions = image_version if image_version == "latest" else [image_version, "latest"] 52 | 53 | for idx, deploy_image_version in enumerate(deploy_image_versions): 54 | ecr_deploy.ECRDeployment(self, f"{repository_name}ECRDeployment-{idx:0>3}", 55 | src=ecr_deploy.DockerImageName(f'{src_docker_image_version}:{image_version}'), 56 | dest=ecr_deploy.DockerImageName(repository.repository_uri_for_tag_or_digest(deploy_image_version)) 57 | ) 58 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_cluster.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecs 10 | ) 11 | 12 | from constructs import Construct 13 | 14 | 15 | class ECSClusterStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 18 | 19 | super().__init__(scope, construct_id, **kwargs) 20 | 21 | cluster_name = self.node.try_get_context('ecs_cluster_name') or "langfuse" 22 | self.ecs_cluster = aws_ecs.Cluster(self, "ECSCluster", 23 | cluster_name=cluster_name, 24 | vpc=vpc 25 | ) 26 | cdk.Tags.of(self.ecs_cluster).add('Name', 'langfuse') 27 | 28 | 29 | cdk.CfnOutput(self, 'ClusterName', 30 | value=self.ecs_cluster.cluster_name, 31 | export_name=f'{self.stack_name}-ClusterName') 32 | cdk.CfnOutput(self, 'ClusterArn', 33 | value=self.ecs_cluster.cluster_arn, 34 | export_name=f'{self.stack_name}-ClusterArn') 35 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_fargate_service_clickhouse.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_ecs 11 | ) 12 | 13 | from constructs import Construct 14 | 15 | 16 | class ECSFargateServiceClickhouseStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, 19 | vpc, 20 | ecs_cluster, 21 | ecs_task_definition, 22 | sg_efs_inbound, 23 | cloud_map_service, 24 | **kwargs) -> None: 25 | 26 | super().__init__(scope, construct_id, **kwargs) 27 | 28 | self.sg_clickhouse_client = aws_ec2.SecurityGroup(self, 'ClickhouseClientSG', 29 | vpc=vpc, 30 | allow_all_outbound=True, 31 | description='security group for clickhouse client', 32 | security_group_name=f'{self.stack_name.lower()}-clickhouse-client-sg' 33 | ) 34 | cdk.Tags.of(self.sg_clickhouse_client).add('Name', 'clickhouse-client-sg') 35 | 36 | sg_clickhouse_server = aws_ec2.SecurityGroup(self, 'ECSFargateServiceSG', 37 | vpc=vpc, 38 | allow_all_outbound=True, 39 | description="Allow inbound from VPC for ECS Fargate Service", 40 | security_group_name=f'{self.stack_name.lower()}-clickhouse-server-sg' 41 | ) 42 | sg_clickhouse_server.add_ingress_rule(peer=self.sg_clickhouse_client, 43 | connection=aws_ec2.Port.tcp(8123), 44 | description='clickhouse http interface') 45 | sg_clickhouse_server.add_ingress_rule(peer=self.sg_clickhouse_client, 46 | connection=aws_ec2.Port.tcp(9000), 47 | description='clickhouse http interface') 48 | cdk.Tags.of(sg_clickhouse_server).add('Name', 'clickhouse-server-sg') 49 | 50 | self.fargate_service = aws_ecs.FargateService(self, "FargateService", 51 | service_name="clickhouse", 52 | cluster=ecs_cluster, 53 | task_definition=ecs_task_definition, 54 | desired_count=1, 55 | min_healthy_percent=50, 56 | security_groups=[sg_clickhouse_server, sg_efs_inbound], 57 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS), 58 | ) 59 | self.fargate_service.associate_cloud_map_service( 60 | service=cloud_map_service, 61 | container_port=self.fargate_service.task_definition.default_container.container_port) 62 | cdk.Tags.of(self.fargate_service).add('Name', 'langfuse_clickhouse') 63 | 64 | self.clickhouse_migration_url = f"clickhouse://{cloud_map_service.service_name}.{cloud_map_service.namespace.namespace_name}:9000" 65 | self.clickhouse_url = f"http://{cloud_map_service.service_name}.{cloud_map_service.namespace.namespace_name}:8123" 66 | # Outputs: 67 | # ClickhouseMigrationUrl = clickhouse://clickhouse.langfuse.local:9000 68 | # ClickhouseUrl = http://clickhouse.langfuse.local:8123 69 | 70 | 71 | cdk.CfnOutput(self, "ClickhouseUrl", 72 | value=self.clickhouse_url, 73 | export_name=f'{self.stack_name}-ClickhouseUrl') 74 | cdk.CfnOutput(self, "ClickhouseMigrationUrl", 75 | value=self.clickhouse_migration_url, 76 | export_name=f'{self.stack_name}-ClickhouseMigrationUrl') 77 | cdk.CfnOutput(self, "FargateServiceName", 78 | value=self.fargate_service.service_name, 79 | export_name=f'{self.stack_name}-FargateServiceName') 80 | cdk.CfnOutput(self, "FargateServiceArn", 81 | value=self.fargate_service.service_arn, 82 | export_name=f'{self.stack_name}-FargateServiceArn') 83 | cdk.CfnOutput(self, "TaskDefinitionArn", 84 | value=self.fargate_service.task_definition.task_definition_arn, 85 | export_name=f'{self.stack_name}-TaskDefinitionArn') 86 | cdk.CfnOutput(self, "ClusterName", 87 | value=self.fargate_service.cluster.cluster_name, 88 | export_name=f'{self.stack_name}-ClusterName') 89 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_fargate_service_langfuse_web.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_ecs, 11 | aws_elasticloadbalancingv2 as elbv2 12 | ) 13 | 14 | from constructs import Construct 15 | 16 | 17 | class ECSFargateServiceLangfuseWebStack(Stack): 18 | 19 | def __init__(self, scope: Construct, construct_id: str, 20 | vpc, 21 | ecs_cluster, 22 | ecs_task_definition, 23 | sg_redis_client, 24 | sg_rds_client, 25 | sg_clickhouse_client, 26 | alb_listener, 27 | **kwargs) -> None: 28 | 29 | super().__init__(scope, construct_id, **kwargs) 30 | 31 | sg_fargate_service = aws_ec2.SecurityGroup(self, 'ECSFargateServiceSG', 32 | vpc=vpc, 33 | allow_all_outbound=True, 34 | description="Allow inbound from VPC for ECS Fargate Service", 35 | security_group_name=f'{self.stack_name.lower()}-langfuse-web-sg' 36 | ) 37 | sg_fargate_service.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 38 | connection=aws_ec2.Port.tcp(3000), 39 | description='langfuse-web') 40 | cdk.Tags.of(sg_fargate_service).add('Name', 'langfuse-web-sg') 41 | 42 | self.fargate_service = aws_ecs.FargateService(self, "FargateService", 43 | service_name="langfuse_web", 44 | cluster=ecs_cluster, 45 | task_definition=ecs_task_definition, 46 | desired_count=1, 47 | min_healthy_percent=50, 48 | security_groups=[sg_fargate_service, sg_redis_client, sg_rds_client, sg_clickhouse_client], 49 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS), 50 | ) 51 | cdk.Tags.of(self.fargate_service).add('Name', 'langfuse_web') 52 | 53 | alb_target_group = alb_listener.add_targets("ECS", 54 | port=3000, 55 | protocol=elbv2.ApplicationProtocol.HTTP 56 | ) 57 | alb_target_group.add_target(self.fargate_service) 58 | 59 | # Setup autoscaling policy 60 | scalable_target = self.fargate_service.auto_scale_task_count(max_capacity=2) 61 | scalable_target.scale_on_cpu_utilization( 62 | id="Autoscaling", 63 | target_utilization_percent=70, 64 | scale_in_cooldown=cdk.Duration.seconds(60), 65 | scale_out_cooldown=cdk.Duration.seconds(60), 66 | ) 67 | 68 | 69 | cdk.CfnOutput(self, "FargateServiceName", 70 | value=self.fargate_service.service_name, 71 | export_name=f'{self.stack_name}-FargateServiceName') 72 | cdk.CfnOutput(self, "FargateServiceArn", 73 | value=self.fargate_service.service_arn, 74 | export_name=f'{self.stack_name}-FargateServiceArn') 75 | cdk.CfnOutput(self, "TaskDefinitionArn", 76 | value=self.fargate_service.task_definition.task_definition_arn, 77 | export_name=f'{self.stack_name}-TaskDefinitionArn') 78 | cdk.CfnOutput(self, "ClusterName", 79 | value=self.fargate_service.cluster.cluster_name, 80 | export_name=f'{self.stack_name}-ClusterName') 81 | cdk.CfnOutput(self, "LoadBalancerDNS", 82 | value=f'http://{alb_listener.load_balancer.load_balancer_dns_name}', 83 | export_name=f'{self.stack_name}-LoadBalancerDNS') 84 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_fargate_service_langfuse_worker.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_ecs 11 | ) 12 | 13 | from constructs import Construct 14 | 15 | 16 | class ECSFargateServiceLangfuseWorkerStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, 19 | vpc, 20 | ecs_cluster, 21 | ecs_task_definition, 22 | sg_redis_client, 23 | sg_rds_client, 24 | sg_clickhouse_client, 25 | **kwargs) -> None: 26 | 27 | super().__init__(scope, construct_id, **kwargs) 28 | 29 | sg_fargate_service = aws_ec2.SecurityGroup(self, 'ECSFargateServiceSG', 30 | vpc=vpc, 31 | allow_all_outbound=True, 32 | description="Allow inbound from VPC for ECS Fargate Service", 33 | security_group_name=f'{self.stack_name.lower()}-langfuse-worker-sg' 34 | ) 35 | sg_fargate_service.add_ingress_rule(peer=aws_ec2.Peer.ipv4("0.0.0.0/0"), 36 | connection=aws_ec2.Port.all_tcp(), 37 | description='langfuse-worker') 38 | cdk.Tags.of(sg_fargate_service).add('Name', 'langfuse-worker-sg') 39 | 40 | worker_desired_count = self.node.try_get_context('langfuse_worker_desired_count') or 1 41 | self.fargate_service = aws_ecs.FargateService(self, "FargateService", 42 | service_name="langfuse_worker", 43 | cluster=ecs_cluster, 44 | task_definition=ecs_task_definition, 45 | desired_count=int(worker_desired_count), 46 | min_healthy_percent=50, 47 | security_groups=[sg_fargate_service, sg_redis_client, sg_rds_client, sg_clickhouse_client], 48 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS), 49 | ) 50 | cdk.Tags.of(self.fargate_service).add('Name', 'langfuse_worker') 51 | 52 | 53 | cdk.CfnOutput(self, "FargateServiceName", 54 | value=self.fargate_service.service_name, 55 | export_name=f'{self.stack_name}-FargateServiceName') 56 | cdk.CfnOutput(self, "FargateServiceArn", 57 | value=self.fargate_service.service_arn, 58 | export_name=f'{self.stack_name}-FargateServiceArn') 59 | cdk.CfnOutput(self, "TaskDefinitionArn", 60 | value=self.fargate_service.task_definition.task_definition_arn, 61 | export_name=f'{self.stack_name}-TaskDefinitionArn') 62 | cdk.CfnOutput(self, "ClusterName", 63 | value=self.fargate_service.cluster.cluster_name, 64 | export_name=f'{self.stack_name}-ClusterName') 65 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_task_clickhouse.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import json 6 | 7 | import aws_cdk as cdk 8 | 9 | from aws_cdk import ( 10 | Stack, 11 | aws_ecs, 12 | aws_iam, 13 | aws_logs, 14 | aws_secretsmanager 15 | ) 16 | 17 | from constructs import Construct 18 | 19 | 20 | class ECSTaskClickhouseStack(Stack): 21 | 22 | def __init__(self, scope: Construct, construct_id: str, 23 | ecr_repositories, 24 | efs_file_system, 25 | **kwargs) -> None: 26 | 27 | super().__init__(scope, construct_id, **kwargs) 28 | 29 | task_execution_role_policy_doc = aws_iam.PolicyDocument() 30 | task_execution_role_policy_doc.add_statements(aws_iam.PolicyStatement(**{ 31 | "effect": aws_iam.Effect.ALLOW, 32 | "resources": ["*"], 33 | "actions": [ 34 | "ecr:GetAuthorizationToken", 35 | "ecr:BatchCheckLayerAvailability", 36 | "ecr:GetDownloadUrlForLayer", 37 | "ecr:BatchGetImage", 38 | "logs:CreateLogStream", 39 | "logs:PutLogEvents" 40 | ] 41 | })) 42 | 43 | task_execution_role = aws_iam.Role(self, "ECSTaskExecutionRole", 44 | role_name=f'ECSTaskExecutionRole-{self.stack_name}', 45 | assumed_by=aws_iam.ServicePrincipal(service="ecs-tasks.amazonaws.com"), 46 | inline_policies={ 47 | 'ecs_task_execution_role_policy': task_execution_role_policy_doc 48 | }, 49 | managed_policies=[ 50 | aws_iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AmazonECSTaskExecutionRolePolicy") 51 | ] 52 | ) 53 | 54 | task_role_policy_doc = aws_iam.PolicyDocument() 55 | task_role_policy_doc.add_statements(aws_iam.PolicyStatement(**{ 56 | "effect": aws_iam.Effect.ALLOW, 57 | "resources": ["*"], 58 | "actions": [ 59 | "secretsmanager:*", 60 | "s3:*", 61 | "elasticfilesystem:ClientMount", 62 | "elasticfilesystem:ClientWrite", 63 | ] 64 | })) 65 | 66 | task_role = aws_iam.Role(self, "ECSTaskRole", 67 | role_name=f'ECSTaskRole-{self.stack_name}', 68 | assumed_by=aws_iam.ServicePrincipal(service="ecs-tasks.amazonaws.com"), 69 | inline_policies={ 70 | 'ecs_task_role_policy': task_role_policy_doc 71 | }, 72 | managed_policies=[ 73 | aws_iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AmazonECSTaskExecutionRolePolicy"), 74 | ] 75 | ) 76 | 77 | efs_volume = aws_ecs.Volume( 78 | name="clickhouse_data", 79 | efs_volume_configuration=aws_ecs.EfsVolumeConfiguration( 80 | file_system_id=efs_file_system.file_system_id, 81 | root_directory="/" 82 | ) 83 | ) 84 | 85 | self.ecs_task_definition = aws_ecs.FargateTaskDefinition(self, "ClickhouseTaskDef", 86 | family="clickhouse", 87 | cpu=1*1024, 88 | memory_limit_mib=8*1024, 89 | execution_role=task_execution_role, 90 | task_role=task_role, 91 | volumes=[efs_volume] 92 | ) 93 | 94 | self.clickhouse_secret = aws_secretsmanager.Secret(self, "ClickhouseSecret", 95 | generate_secret_string=aws_secretsmanager.SecretStringGenerator( 96 | secret_string_template=json.dumps({ 97 | "database": "default", 98 | "user": "clickhouse" 99 | }), 100 | generate_string_key="password", 101 | exclude_punctuation=True, 102 | password_length=8 103 | ) 104 | ) 105 | 106 | task_env = { 107 | "CLICKHOUSE_DB": self.clickhouse_secret.secret_value_from_json("database").unsafe_unwrap(), 108 | "CLICKHOUSE_USER": self.clickhouse_secret.secret_value_from_json("user").unsafe_unwrap(), 109 | "CLICKHOUSE_PASSWORD": self.clickhouse_secret.secret_value_from_json("password").unsafe_unwrap() 110 | } 111 | 112 | task_health_check = aws_ecs.HealthCheck( 113 | command=["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1"], 114 | interval=cdk.Duration.minutes(5), # cdk.Duration.seconds(5) 115 | retries=10, 116 | start_period=cdk.Duration.minutes(1), # cdk.Duration.seconds(1) 117 | timeout=cdk.Duration.seconds(5) 118 | ) 119 | 120 | repository = ecr_repositories['clickhouse'] 121 | container = self.ecs_task_definition.add_container("Clickhouse", 122 | container_name="clickhouse", 123 | image=aws_ecs.ContainerImage.from_ecr_repository(repository, tag="latest"), 124 | cpu=1*1024, 125 | memory_limit_mib=8*1024, 126 | environment=task_env, 127 | health_check=task_health_check, 128 | logging=aws_ecs.LogDriver.aws_logs( 129 | stream_prefix="clickhouse", 130 | log_retention=aws_logs.RetentionDays.ONE_WEEK 131 | ), 132 | port_mappings=[ 133 | # ClickHouse HTTP interface 134 | aws_ecs.PortMapping( 135 | container_port=8123, 136 | host_port=8123, 137 | protocol=aws_ecs.Protocol.TCP 138 | ), 139 | # ClickHouse native interface 140 | aws_ecs.PortMapping( 141 | container_port=9000, 142 | host_port=9000, 143 | protocol=aws_ecs.Protocol.TCP 144 | ) 145 | ], 146 | ulimits=[ 147 | aws_ecs.Ulimit( 148 | name=aws_ecs.UlimitName.NOFILE, 149 | soft_limit=65535, 150 | hard_limit=65535 151 | ) 152 | ] 153 | ) 154 | 155 | mount_point = aws_ecs.MountPoint( 156 | container_path="/var/lib/clickhouse", 157 | read_only=False, 158 | source_volume="clickhouse_data" 159 | ) 160 | container.add_mount_points(mount_point) 161 | 162 | efs_file_system.grant_root_access(self.ecs_task_definition.task_role.grant_principal) 163 | 164 | 165 | cdk.CfnOutput(self, 'TaskDefinitionArn', 166 | value=self.ecs_task_definition.task_definition_arn, 167 | export_name=f'{self.stack_name}-TaskDefinitionArn') 168 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_task_langfuse_web.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecs, 10 | aws_iam, 11 | aws_logs 12 | ) 13 | 14 | from constructs import Construct 15 | 16 | 17 | class ECSTaskLangfuseWebStack(Stack): 18 | 19 | def __init__(self, scope: Construct, construct_id: str, 20 | ecr_repositories, 21 | database_secret, 22 | clickhouse_secret, 23 | clickhouse_migration_url, 24 | clickhouse_url, 25 | redis_cluster, 26 | s3_blob_bucket, 27 | s3_event_bucket, 28 | load_balancer_url, 29 | **kwargs) -> None: 30 | 31 | super().__init__(scope, construct_id, **kwargs) 32 | 33 | task_role_policy_doc = aws_iam.PolicyDocument() 34 | task_role_policy_doc.add_statements(aws_iam.PolicyStatement(**{ 35 | "effect": aws_iam.Effect.ALLOW, 36 | "resources": ["*"], 37 | "actions": [ 38 | "secretsmanager:*", 39 | "s3:*", 40 | "ecr:GetAuthorizationToken", 41 | "ecr:BatchCheckLayerAvailability", 42 | "ecr:GetDownloadUrlForLayer", 43 | "ecr:BatchGetImage", 44 | "logs:CreateLogStream", 45 | "logs:PutLogEvents" 46 | ] 47 | })) 48 | 49 | task_role = aws_iam.Role(self, "ECSTaskRole", 50 | role_name=f'ECSTaskRole-{self.stack_name}', 51 | assumed_by=aws_iam.ServicePrincipal(service="ecs-tasks.amazonaws.com"), 52 | inline_policies={ 53 | 'ecs_task_role_policy': task_role_policy_doc 54 | }, 55 | managed_policies=[ 56 | aws_iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AmazonECSTaskExecutionRolePolicy") 57 | ] 58 | ) 59 | 60 | self.ecs_task_definition = aws_ecs.FargateTaskDefinition(self, "LangfuseWebTaskDef", 61 | family="langfuse-web", 62 | task_role=task_role, 63 | cpu=2*1024, 64 | memory_limit_mib=4*1024 65 | ) 66 | 67 | db_conn_info = { 68 | "DATABASE_HOST": database_secret.secret_value_from_json("host").unsafe_unwrap(), 69 | "DATABASE_PORT": database_secret.secret_value_from_json("port").unsafe_unwrap(), 70 | "DATABASE_USERNAME": database_secret.secret_value_from_json("username").unsafe_unwrap(), 71 | "DATABASE_PASSWORD": database_secret.secret_value_from_json("password").unsafe_unwrap(), 72 | "DATABASE_NAME": "postgres" 73 | } 74 | DATABASE_URL = "postgresql://{DATABASE_USERNAME}:{DATABASE_PASSWORD}@{DATABASE_HOST}:{DATABASE_PORT}/{DATABASE_NAME}".format(**db_conn_info) 75 | 76 | redis_conn_info = { 77 | "REDIS_HOST": redis_cluster.attr_primary_end_point_address, 78 | "REDIS_PORT": redis_cluster.attr_primary_end_point_port 79 | } 80 | REDIS_CONNECTION_STRING = "redis://{REDIS_HOST}:{REDIS_PORT}".format(**redis_conn_info) 81 | 82 | # Environment Variables for the Langfuse Worker container 83 | # https://langfuse.com/self-hosting/configuration#environment-variables 84 | docker_run_args = self.node.try_get_context('langfuse_web_env') 85 | task_env = { 86 | **docker_run_args, 87 | 88 | "DATABASE_URL": DATABASE_URL, 89 | 90 | "CLICKHOUSE_MIGRATION_URL": clickhouse_migration_url, 91 | "CLICKHOUSE_URL": clickhouse_url, 92 | "CLICKHOUSE_DB": clickhouse_secret.secret_value_from_json("database").unsafe_unwrap(), 93 | "CLICKHOUSE_USER": clickhouse_secret.secret_value_from_json("user").unsafe_unwrap(), 94 | "CLICKHOUSE_PASSWORD": clickhouse_secret.secret_value_from_json("password").unsafe_unwrap(), 95 | "CLICKHOUSE_CLUSTER_ENABLED": "false", 96 | 97 | "REDIS_CONNECTION_STRING": REDIS_CONNECTION_STRING, 98 | 99 | "LANGFUSE_S3_EVENT_UPLOAD_BUCKET": s3_event_bucket.bucket_name, 100 | "LANGFUSE_S3_EVENT_UPLOAD_PREFIX": "events/", 101 | "LANGFUSE_S3_EVENT_UPLOAD_REGION": self.region, 102 | 103 | "LANGFUSE_S3_MEDIA_UPLOAD_BUCKET": s3_blob_bucket.bucket_name, 104 | # "LANGFUSE_S3_MEDIA_UPLOAD_PREFIX": "media/", # default: "" (the bucket root) 105 | "LANGFUSE_S3_MEDIA_UPLOAD_ENABLED": "true", 106 | 107 | "NEXTAUTH_URL": load_balancer_url, 108 | } 109 | 110 | repository = ecr_repositories['langfuse-web'] 111 | container = self.ecs_task_definition.add_container("LangfuseWebContainerDef", 112 | container_name="langfuse_web", 113 | image=aws_ecs.ContainerImage.from_ecr_repository(repository, tag="latest"), 114 | environment=task_env, 115 | logging=aws_ecs.LogDriver.aws_logs( 116 | stream_prefix="langfuse-web", 117 | log_retention=aws_logs.RetentionDays.ONE_WEEK 118 | ) 119 | ) 120 | 121 | port_mapping = aws_ecs.PortMapping( 122 | container_port=3000, 123 | host_port=3000, 124 | protocol=aws_ecs.Protocol.TCP 125 | ) 126 | container.add_port_mappings(port_mapping) 127 | 128 | 129 | cdk.CfnOutput(self, 'TaskDefinitionArn', 130 | value=self.ecs_task_definition.task_definition_arn, 131 | export_name=f'{self.stack_name}-TaskDefinitionArn') 132 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/ecs_task_langfuse_worker.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ecs, 10 | aws_iam, 11 | aws_logs 12 | ) 13 | 14 | from constructs import Construct 15 | 16 | 17 | class ECSTaskLangfuseWorkerStack(Stack): 18 | 19 | def __init__(self, scope: Construct, construct_id: str, 20 | ecr_repositories, 21 | database_secret, 22 | clickhouse_secret, 23 | clickhouse_migration_url, 24 | clickhouse_url, 25 | redis_cluster, 26 | s3_blob_bucket, 27 | s3_event_bucket, 28 | **kwargs) -> None: 29 | 30 | super().__init__(scope, construct_id, **kwargs) 31 | 32 | task_role_policy_doc = aws_iam.PolicyDocument() 33 | task_role_policy_doc.add_statements(aws_iam.PolicyStatement(**{ 34 | "effect": aws_iam.Effect.ALLOW, 35 | "resources": ["*"], 36 | "actions": [ 37 | "secretsmanager:*", 38 | "s3:*", 39 | "ecr:GetAuthorizationToken", 40 | "ecr:BatchCheckLayerAvailability", 41 | "ecr:GetDownloadUrlForLayer", 42 | "ecr:BatchGetImage", 43 | "logs:CreateLogStream", 44 | "logs:PutLogEvents" 45 | ] 46 | })) 47 | 48 | task_role = aws_iam.Role(self, "ECSTaskRole", 49 | role_name=f'ECSTaskRole-{self.stack_name}', 50 | assumed_by=aws_iam.ServicePrincipal(service="ecs-tasks.amazonaws.com"), 51 | inline_policies={ 52 | 'ecs_task_role_policy': task_role_policy_doc 53 | }, 54 | managed_policies=[ 55 | aws_iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AmazonECSTaskExecutionRolePolicy") 56 | ] 57 | ) 58 | 59 | self.ecs_task_definition = aws_ecs.FargateTaskDefinition(self, "LangfuseWorkerTaskDef", 60 | family="langfuse-worker", 61 | task_role=task_role, 62 | cpu=2*1024, 63 | memory_limit_mib=4*1024 64 | ) 65 | 66 | db_conn_info = { 67 | "DATABASE_HOST": database_secret.secret_value_from_json("host").unsafe_unwrap(), 68 | "DATABASE_PORT": database_secret.secret_value_from_json("port").unsafe_unwrap(), 69 | "DATABASE_USERNAME": database_secret.secret_value_from_json("username").unsafe_unwrap(), 70 | "DATABASE_PASSWORD": database_secret.secret_value_from_json("password").unsafe_unwrap(), 71 | "DATABASE_NAME": "postgres" 72 | } 73 | DATABASE_URL = "postgresql://{DATABASE_USERNAME}:{DATABASE_PASSWORD}@{DATABASE_HOST}:{DATABASE_PORT}/{DATABASE_NAME}".format(**db_conn_info) 74 | 75 | redis_conn_info = { 76 | "REDIS_HOST": redis_cluster.attr_primary_end_point_address, 77 | "REDIS_PORT": redis_cluster.attr_primary_end_point_port 78 | } 79 | REDIS_CONNECTION_STRING = "redis://{REDIS_HOST}:{REDIS_PORT}".format(**redis_conn_info) 80 | 81 | # Environment Variables for the Langfuse Worker container 82 | # https://langfuse.com/self-hosting/configuration#environment-variables 83 | docker_run_args = self.node.try_get_context('langfuse_worker_env') 84 | task_env = { 85 | **docker_run_args, 86 | 87 | "DATABASE_URL": DATABASE_URL, 88 | 89 | "CLICKHOUSE_MIGRATION_URL": clickhouse_migration_url, 90 | "CLICKHOUSE_URL": clickhouse_url, 91 | "CLICKHOUSE_DB": clickhouse_secret.secret_value_from_json("database").unsafe_unwrap(), 92 | "CLICKHOUSE_USER": clickhouse_secret.secret_value_from_json("user").unsafe_unwrap(), 93 | "CLICKHOUSE_PASSWORD": clickhouse_secret.secret_value_from_json("password").unsafe_unwrap(), 94 | "CLICKHOUSE_CLUSTER_ENABLED": "false", 95 | 96 | "REDIS_CONNECTION_STRING": REDIS_CONNECTION_STRING, 97 | 98 | "LANGFUSE_S3_EVENT_UPLOAD_BUCKET": s3_event_bucket.bucket_name, 99 | "LANGFUSE_S3_EVENT_UPLOAD_PREFIX": "events/", 100 | "LANGFUSE_S3_EVENT_UPLOAD_REGION": self.region, 101 | 102 | "LANGFUSE_S3_MEDIA_UPLOAD_BUCKET": s3_blob_bucket.bucket_name, 103 | # "LANGFUSE_S3_MEDIA_UPLOAD_PREFIX": "media/", # default: "" (the bucket root) 104 | "LANGFUSE_S3_MEDIA_UPLOAD_ENABLED": "true", 105 | } 106 | 107 | repository = ecr_repositories['langfuse-worker'] 108 | container = self.ecs_task_definition.add_container("LangfuseWorkerContainerDef", 109 | container_name="worker", 110 | image=aws_ecs.ContainerImage.from_ecr_repository(repository, tag="latest"), 111 | # essential=True, 112 | environment=task_env, 113 | logging=aws_ecs.LogDriver.aws_logs( 114 | stream_prefix="langfuse-worker", 115 | log_retention=aws_logs.RetentionDays.ONE_WEEK 116 | ) 117 | ) 118 | 119 | port_mapping = aws_ecs.PortMapping( 120 | container_port=3030, 121 | host_port=3030, 122 | protocol=aws_ecs.Protocol.TCP 123 | ) 124 | container.add_port_mappings(port_mapping) 125 | 126 | 127 | cdk.CfnOutput(self, 'TaskDefinitionArn', 128 | value=self.ecs_task_definition.task_definition_arn, 129 | export_name=f'{self.stack_name}-TaskDefinitionArn') 130 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/efs_clickhouse.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_efs, 11 | aws_iam, 12 | ) 13 | from constructs import Construct 14 | 15 | 16 | class EFSStack(Stack): 17 | 18 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 19 | super().__init__(scope, construct_id, **kwargs) 20 | 21 | self.sg_efs_inbound = aws_ec2.SecurityGroup(self, "EFSInboundSecurityGroup", 22 | vpc=vpc, 23 | allow_all_outbound=True, 24 | description='Security Group that allows inbound NFS traffic for ECS tasks', 25 | security_group_name=f'{self.stack_name.lower()}-efs-inbound-sg' 26 | ) 27 | self.sg_efs_inbound.apply_removal_policy(cdk.RemovalPolicy.DESTROY) 28 | cdk.Tags.of(self.sg_efs_inbound).add('Name', 'efs-inbound-sg') 29 | 30 | self.sg_efs_outbound = aws_ec2.SecurityGroup(self, "EFSOutboundGroup", 31 | vpc=vpc, 32 | allow_all_outbound=False, 33 | description='Security Group that allows outbound NFS traffic for ECS tasks', 34 | security_group_name=f'{self.stack_name.lower()}-efs-outbound-sg' 35 | ) 36 | self.sg_efs_outbound.add_ingress_rule(peer=self.sg_efs_inbound, 37 | connection=aws_ec2.Port.tcp(2049)) 38 | self.sg_efs_outbound.add_egress_rule(peer=self.sg_efs_inbound, 39 | connection=aws_ec2.Port.tcp(2049)) 40 | self.sg_efs_outbound.apply_removal_policy(cdk.RemovalPolicy.DESTROY) 41 | cdk.Tags.of(self.sg_efs_outbound).add('Name', 'efs-outbound-sg') 42 | 43 | self.efs_file_system = aws_efs.FileSystem(self, "EfsFileSystem", 44 | vpc=vpc, 45 | performance_mode=aws_efs.PerformanceMode.GENERAL_PURPOSE, # default 46 | removal_policy=cdk.RemovalPolicy.DESTROY, # default: RemovalPolicy.RETAIN 47 | security_group=self.sg_efs_outbound, 48 | vpc_subnets=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS) 49 | ) 50 | 51 | self.efs_file_system.add_to_resource_policy(aws_iam.PolicyStatement(**{ 52 | "effect": aws_iam.Effect.ALLOW, 53 | "actions": [ 54 | "elasticfilesystem:ClientMount", 55 | ], 56 | "principals": [ 57 | aws_iam.AnyPrincipal() 58 | ], 59 | "conditions": { 60 | "Bool": { 61 | "elasticfilesystem:AccessedViaMountTarget": "true" 62 | } 63 | } 64 | })) 65 | 66 | cdk.Tags.of(self.efs_file_system).add('Name', 'clickhouse-data') 67 | 68 | 69 | cdk.CfnOutput(self, 'EFSInboundSecurityGroupId', 70 | value=self.sg_efs_inbound.security_group_id, 71 | export_name=f'{self.stack_name}-EFSInboundSecurityGroupId') 72 | cdk.CfnOutput(self, 'EFSOutboundSecurityGroupId', 73 | value=self.sg_efs_outbound.security_group_id, 74 | export_name=f'{self.stack_name}-EFSOutboundSecurityGroupId') 75 | cdk.CfnOutput(self, 'EFSFileSystemId', 76 | value=self.efs_file_system.file_system_id, 77 | export_name=f'{self.stack_name}-EFSFileSystemId') 78 | cdk.CfnOutput(self, 'EFSFileSystemArn', 79 | value=self.efs_file_system.file_system_arn, 80 | export_name=f'{self.stack_name}-EFSFileSystemArn') 81 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/redis.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_ec2, 10 | aws_elasticache 11 | ) 12 | from constructs import Construct 13 | 14 | 15 | class RedisClusterStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 18 | super().__init__(scope, construct_id, **kwargs) 19 | 20 | self.sg_elasticache_client = aws_ec2.SecurityGroup(self, 'RedisClientSG', 21 | vpc=vpc, 22 | allow_all_outbound=True, 23 | description='security group for redis client', 24 | security_group_name=f'{self.stack_name}-redis-client-sg' 25 | ) 26 | cdk.Tags.of(self.sg_elasticache_client).add('Name', 'redis-client-sg') 27 | 28 | sg_elasticache_server = aws_ec2.SecurityGroup(self, 'RedisServerSG', 29 | vpc=vpc, 30 | allow_all_outbound=True, 31 | description='security group for redis server', 32 | security_group_name=f'{self.stack_name}-redis-server-sg' 33 | ) 34 | cdk.Tags.of(sg_elasticache_server).add('Name', 'redis-server-sg') 35 | 36 | sg_elasticache_server.add_ingress_rule(peer=self.sg_elasticache_client, 37 | connection=aws_ec2.Port.tcp(6379), 38 | description='redis-client-sg') 39 | sg_elasticache_server.add_ingress_rule(peer=sg_elasticache_server, 40 | connection=aws_ec2.Port.all_tcp(), 41 | description='redis-server-sg') 42 | 43 | redis_cluster_subnet_group = aws_elasticache.CfnSubnetGroup(self, 'RedisSubnetGroup', 44 | description='subnet group for redis', 45 | subnet_ids=vpc.select_subnets(subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS).subnet_ids, 46 | cache_subnet_group_name=f'{self.stack_name}-redis-cluster' 47 | ) 48 | 49 | self.redis_cluster = aws_elasticache.CfnReplicationGroup(self, 'RedisCluster', 50 | replication_group_id='langfuse-cache', 51 | replication_group_description='Langfuse Cache/Queue Replication Group', 52 | 53 | #XXX: Amazon ElastiCache Supported node types 54 | # https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/CacheNodes.SupportedTypes.html#CacheNodes.CurrentGen 55 | cache_node_type='cache.t3.small', 56 | engine='valkey', 57 | 58 | #XXX: Comparing Valkey, Memcached, and Redis OSS self-designed caches 59 | # https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/SelectEngine.html 60 | #XXX: Amazon ElastiCache Supported engines and versions 61 | # https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/supported-engine-versions.html 62 | engine_version='7.2', 63 | 64 | num_cache_clusters=1, 65 | automatic_failover_enabled=False, 66 | cache_parameter_group_name="default.valkey7", 67 | cache_subnet_group_name=redis_cluster_subnet_group.cache_subnet_group_name, 68 | security_group_ids=[sg_elasticache_server.security_group_id], 69 | snapshot_retention_limit=3, 70 | snapshot_window='19:00-21:00', 71 | preferred_maintenance_window='mon:21:00-mon:22:30', 72 | auto_minor_version_upgrade=False, 73 | transit_encryption_enabled=True, 74 | transit_encryption_mode='preferred', 75 | tags=[ 76 | cdk.CfnTag(key='Name', value='langfuse-cache') 77 | ] 78 | ) 79 | self.redis_cluster.add_dependency(redis_cluster_subnet_group) 80 | self.redis_cluster.apply_removal_policy(cdk.RemovalPolicy.DESTROY) 81 | 82 | 83 | cdk.CfnOutput(self, 'PrimaryEndpoint', 84 | value=self.redis_cluster.attr_primary_end_point_address, 85 | export_name=f'{self.stack_name}-PrimaryEndpoint') 86 | cdk.CfnOutput(self, 'PrimaryPort', 87 | value=self.redis_cluster.attr_primary_end_point_port, 88 | export_name=f'{self.stack_name}-PrimaryPort') 89 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/s3.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import random 6 | import string 7 | 8 | import aws_cdk as cdk 9 | 10 | from aws_cdk import ( 11 | Stack, 12 | aws_s3 as s3, 13 | ) 14 | from constructs import Construct 15 | 16 | random.seed(47) 17 | 18 | 19 | class S3BucketStack(Stack): 20 | 21 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 22 | super().__init__(scope, construct_id, **kwargs) 23 | 24 | S3_BUCKET_SUFFIX = ''.join(random.sample((string.ascii_lowercase + string.digits), k=7)) 25 | 26 | self.blob_bucket = s3.Bucket(self, "BlobBucket", 27 | bucket_name=f'langfuse-blob-{cdk.Aws.REGION}-{cdk.Aws.ACCOUNT_ID}-{S3_BUCKET_SUFFIX}', 28 | removal_policy=cdk.RemovalPolicy.DESTROY, #XXX: Default: core.RemovalPolicy.RETAIN - The bucket will be orphaned 29 | auto_delete_objects=True) 30 | 31 | self.event_bucket = s3.Bucket(self, "EventBucket", 32 | bucket_name=f'langfuse-event-{cdk.Aws.REGION}-{cdk.Aws.ACCOUNT_ID}-{S3_BUCKET_SUFFIX}', 33 | removal_policy=cdk.RemovalPolicy.DESTROY, #XXX: Default: core.RemovalPolicy.RETAIN - The bucket will be orphaned 34 | auto_delete_objects=True) 35 | 36 | 37 | cdk.CfnOutput(self, 'BlobBucketName', 38 | value=self.blob_bucket.bucket_name, 39 | export_name=f'{self.stack_name}-BlobBucketName') 40 | cdk.CfnOutput(self, 'EventBucketName', 41 | value=self.event_bucket.bucket_name, 42 | export_name=f'{self.stack_name}-EventBucketName') 43 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/service_discovery.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import aws_cdk as cdk 6 | 7 | from aws_cdk import ( 8 | Stack, 9 | aws_servicediscovery 10 | ) 11 | 12 | from constructs import Construct 13 | 14 | 15 | class ServiceDiscoveryStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, vpc, **kwargs) -> None: 18 | 19 | super().__init__(scope, construct_id, **kwargs) 20 | 21 | private_dns_namespace_name = self.node.try_get_context('private_dns_namespace_name') or "langfuse.local" 22 | self.namespace = aws_servicediscovery.PrivateDnsNamespace(self, "PrivateDnsNamespace", 23 | name=private_dns_namespace_name, 24 | description="Langfuse Service Discovery namespace", 25 | vpc=vpc 26 | ) 27 | cdk.Tags.of(self.namespace).add('Name', 'langfuse') 28 | 29 | self.service = self.namespace.create_service("Service", 30 | name="clickhouse", 31 | dns_record_type=aws_servicediscovery.DnsRecordType.A, 32 | dns_ttl=cdk.Duration.seconds(10), 33 | # load_balancer=False, # default: False 34 | custom_health_check=aws_servicediscovery.HealthCheckCustomConfig( 35 | failure_threshold=1 36 | ) 37 | ) 38 | cdk.Tags.of(self.service).add('Name', 'langfuse_clickhouse') 39 | 40 | 41 | cdk.CfnOutput(self, 'NamespaceName', 42 | value=self.namespace.namespace_name, 43 | export_name=f'{self.stack_name}-NamespaceName') 44 | cdk.CfnOutput(self, 'ServiceName', 45 | value=self.service.service_name, 46 | export_name=f'{self.stack_name}-ServiceName') 47 | 48 | -------------------------------------------------------------------------------- /langfuse-v3/cdk_stacks/vpc.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- encoding: utf-8 -*- 3 | # vim: tabstop=2 shiftwidth=2 softtabstop=2 expandtab 4 | 5 | import os 6 | import aws_cdk as cdk 7 | 8 | from aws_cdk import ( 9 | Stack, 10 | aws_ec2, 11 | ) 12 | from constructs import Construct 13 | 14 | 15 | class VpcStack(Stack): 16 | 17 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 18 | super().__init__(scope, construct_id, **kwargs) 19 | 20 | #XXX: For creating this CDK Stack in the existing VPC, 21 | # remove comments from the below codes and 22 | # comments out vpc = aws_ec2.Vpc(..) codes, 23 | # then pass -c vpc_name=your-existing-vpc to cdk command 24 | # for example, 25 | # cdk -c vpc_name=your-existing-vpc syth 26 | # 27 | if str(os.environ.get('USE_DEFAULT_VPC', 'false')).lower() == 'true': 28 | vpc_name = self.node.try_get_context('vpc_name') or "default" 29 | self.vpc = aws_ec2.Vpc.from_lookup(self, 'ExistingVPC', 30 | is_default=True, 31 | vpc_name=vpc_name 32 | ) 33 | else: 34 | #XXX: To use more than 2 AZs, be sure to specify the account and region on your stack. 35 | #XXX: https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_ec2/Vpc.html 36 | self.vpc = aws_ec2.Vpc(self, 'VPC', 37 | ip_addresses=aws_ec2.IpAddresses.cidr("10.0.0.0/16"), 38 | max_azs=3, 39 | 40 | # 'subnetConfiguration' specifies the "subnet groups" to create. 41 | # Every subnet group will have a subnet for each AZ, so this 42 | # configuration will create `2 groups × 3 AZs = 6` subnets. 43 | subnet_configuration=[ 44 | { 45 | "cidrMask": 20, 46 | "name": "Public", 47 | "subnetType": aws_ec2.SubnetType.PUBLIC, 48 | }, 49 | { 50 | "cidrMask": 20, 51 | "name": "Private", 52 | "subnetType": aws_ec2.SubnetType.PRIVATE_WITH_EGRESS 53 | } 54 | ], 55 | gateway_endpoints={ 56 | "S3": aws_ec2.GatewayVpcEndpointOptions( 57 | service=aws_ec2.GatewayVpcEndpointAwsService.S3 58 | ) 59 | } 60 | ) 61 | 62 | cdk.CfnOutput(self, 'VPCID', value=self.vpc.vpc_id, 63 | export_name=f'{self.stack_name}-VPCID') 64 | -------------------------------------------------------------------------------- /langfuse-v3/examples/tracing_for_langchain_bedrock.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "fa62892f-5737-428d-a192-c53b07f0923c", 6 | "metadata": {}, 7 | "source": [ 8 | "# Tracing for LangChain (Python)\n", 9 | "\n", 10 | "❗This notebook works well on `ml.t3.medium` instance with `Data Science 2.0` kernel from **SageMaker Studio Classic** or `Python3` kernel from **JupyterLab**." 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "id": "44f1d80e-3d36-4e4f-aba8-7923459c62ec", 16 | "metadata": {}, 17 | "source": [ 18 | "### Install required packages" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "id": "18390522-ae27-4a63-86c8-b3e1401fa1d3", 25 | "metadata": { 26 | "tags": [] 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "%%capture --no-stderr\n", 31 | "\n", 32 | "!pip install -U langfuse==2.57.13\n", 33 | "!pip install -U langchain==0.3.15\n", 34 | "!pip install -U langchain-community==0.3.15\n", 35 | "!pip install -U langchain-aws==0.2.11" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "id": "432648db-9d10-4f5e-bf3d-7553fc4a7309", 42 | "metadata": { 43 | "tags": [] 44 | }, 45 | "outputs": [ 46 | { 47 | "name": "stdout", 48 | "output_type": "stream", 49 | "text": [ 50 | "langchain==0.3.15\n", 51 | "langchain-aws==0.2.11\n", 52 | "langchain-community==0.3.15\n", 53 | "langchain-core==0.3.31\n", 54 | "langchain-text-splitters==0.3.5\n", 55 | "langfuse==2.57.13\n" 56 | ] 57 | } 58 | ], 59 | "source": [ 60 | "!pip freeze | grep -E \"langchain|langfuse\"" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "id": "bb8c9e1e-56db-4a4b-b88a-b084adf1543f", 66 | "metadata": {}, 67 | "source": [ 68 | "### Environment Setup and Langfuse Initialization" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": null, 74 | "id": "3dc5e5c0-4765-45df-b8ce-509092f0ef57", 75 | "metadata": { 76 | "tags": [] 77 | }, 78 | "outputs": [], 79 | "source": [ 80 | "import os\n", 81 | "from langfuse.callback import CallbackHandler\n", 82 | "\n", 83 | "# Initialize LangFuse Callback Handler\n", 84 | "langfuse_handler = CallbackHandler(\n", 85 | " public_key=\"pk-lf-***\",\n", 86 | " secret_key=\"sk-lf-***\",\n", 87 | " host=\"http://langfu-***-****.{region}.elb.amazonaws.com\"\n", 88 | ")" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "id": "929e4944-e8ea-45fb-8378-0d8afde50d38", 95 | "metadata": { 96 | "tags": [] 97 | }, 98 | "outputs": [ 99 | { 100 | "data": { 101 | "text/plain": [ 102 | "True" 103 | ] 104 | }, 105 | "execution_count": null, 106 | "metadata": {}, 107 | "output_type": "execute_result" 108 | } 109 | ], 110 | "source": [ 111 | "# connection test\n", 112 | "langfuse_handler.auth_check()" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "id": "aed2146a-2410-4f8e-a8f6-c14e53e3f248", 119 | "metadata": { 120 | "tags": [] 121 | }, 122 | "outputs": [], 123 | "source": [ 124 | "import boto3\n", 125 | "from langchain_aws import ChatBedrock as BedrockChat\n", 126 | "\n", 127 | "region = boto3.Session().region_name\n", 128 | "\n", 129 | "model_kwargs = {\n", 130 | " \"max_tokens\": 512,\n", 131 | " \"temperature\": 0,\n", 132 | " \"top_p\": 0.9\n", 133 | "}\n", 134 | "\n", 135 | "model_id = \"anthropic.claude-3-sonnet-20240229-v1:0\"\n", 136 | "\n", 137 | "llm = BedrockChat(model_id=model_id, region_name=region, model_kwargs=model_kwargs)\n", 138 | "llm" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "id": "19d1f0cc-0802-4672-a480-51192cff77f1", 145 | "metadata": { 146 | "tags": [] 147 | }, 148 | "outputs": [], 149 | "source": [ 150 | "from langchain.prompts import ChatPromptTemplate\n", 151 | "from langchain.schema import StrOutputParser\n", 152 | "from operator import itemgetter\n", 153 | "\n", 154 | "prompt1 = ChatPromptTemplate.from_template(\"What is the city {person} is from?\")\n", 155 | "prompt2 = ChatPromptTemplate.from_template(\n", 156 | " \"What country is the city {city} in? Respond in {language}\"\n", 157 | ")\n", 158 | "\n", 159 | "chain1 = prompt1 | llm | StrOutputParser()\n", 160 | "chain2 = (\n", 161 | " {\"city\": chain1, \"language\": itemgetter(\"language\")}\n", 162 | " | prompt2\n", 163 | " | llm\n", 164 | " | StrOutputParser()\n", 165 | ")" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": null, 171 | "id": "ae985454-ceba-4195-970b-277d6e8afddd", 172 | "metadata": { 173 | "tags": [] 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "# Invoke chain with LangFuse callback handler\n", 178 | "result = chain2.invoke(\n", 179 | " {\"person\": \"Obama\", \"language\": \"Spanish\"},\n", 180 | " config={\"callbacks\": [langfuse_handler]}\n", 181 | ")" 182 | ] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "execution_count": null, 187 | "id": "93977840-b304-4b87-bb74-33df2e1ab336", 188 | "metadata": { 189 | "tags": [] 190 | }, 191 | "outputs": [ 192 | { 193 | "name": "stdout", 194 | "output_type": "stream", 195 | "text": [ 196 | "La ciudad de Honolulu, donde nació Barack Obama el 4 de agosto de 1961, está ubicada en Hawái, un estado de los Estados Unidos. Aunque pasó gran parte de su infancia en Indonesia y vivió en varios otros lugares como Los Ángeles, Nueva York, Boston y Chicago antes de convertirse en presidente, Chicago se considera su base política principal. Fue allí donde trabajó como organizador comunitario, enseñó derecho constitucional en la Universidad de Chicago e inició su carrera política sirviendo en el Senado estatal de Illinois y luego en el Senado de los Estados Unidos representando a Illinois antes de ser elegido como el 44° presidente de los Estados Unidos en 2008. Así que si bien nació en Honolulu, Hawái, la ciudad adoptiva de Barack Obama y la más asociada con su ascenso a la prominencia nacional es Chicago, Illinois.\n" 197 | ] 198 | } 199 | ], 200 | "source": [ 201 | "print(result)" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "id": "cb44a99f-3abe-4310-9815-15d7ebdfde1e", 207 | "metadata": {}, 208 | "source": [ 209 | "## References\n", 210 | "\n", 211 | "- [Langfuse Documents](https://langfuse.com/docs)\n", 212 | " - [Get Started with Langfuse Tracing](https://langfuse.com/docs/get-started) \n", 213 | " - [Observability & Tracing for Langchain (Python & JS/TS)](https://langfuse.com/docs/integrations/langchain/tracing)" 214 | ] 215 | } 216 | ], 217 | "metadata": { 218 | "availableInstances": [ 219 | { 220 | "_defaultOrder": 0, 221 | "_isFastLaunch": true, 222 | "category": "General purpose", 223 | "gpuNum": 0, 224 | "hideHardwareSpecs": false, 225 | "memoryGiB": 4, 226 | "name": "ml.t3.medium", 227 | "vcpuNum": 2 228 | }, 229 | { 230 | "_defaultOrder": 1, 231 | "_isFastLaunch": false, 232 | "category": "General purpose", 233 | "gpuNum": 0, 234 | "hideHardwareSpecs": false, 235 | "memoryGiB": 8, 236 | "name": "ml.t3.large", 237 | "vcpuNum": 2 238 | }, 239 | { 240 | "_defaultOrder": 2, 241 | "_isFastLaunch": false, 242 | "category": "General purpose", 243 | "gpuNum": 0, 244 | "hideHardwareSpecs": false, 245 | "memoryGiB": 16, 246 | "name": "ml.t3.xlarge", 247 | "vcpuNum": 4 248 | }, 249 | { 250 | "_defaultOrder": 3, 251 | "_isFastLaunch": false, 252 | "category": "General purpose", 253 | "gpuNum": 0, 254 | "hideHardwareSpecs": false, 255 | "memoryGiB": 32, 256 | "name": "ml.t3.2xlarge", 257 | "vcpuNum": 8 258 | }, 259 | { 260 | "_defaultOrder": 4, 261 | "_isFastLaunch": true, 262 | "category": "General purpose", 263 | "gpuNum": 0, 264 | "hideHardwareSpecs": false, 265 | "memoryGiB": 8, 266 | "name": "ml.m5.large", 267 | "vcpuNum": 2 268 | }, 269 | { 270 | "_defaultOrder": 5, 271 | "_isFastLaunch": false, 272 | "category": "General purpose", 273 | "gpuNum": 0, 274 | "hideHardwareSpecs": false, 275 | "memoryGiB": 16, 276 | "name": "ml.m5.xlarge", 277 | "vcpuNum": 4 278 | }, 279 | { 280 | "_defaultOrder": 6, 281 | "_isFastLaunch": false, 282 | "category": "General purpose", 283 | "gpuNum": 0, 284 | "hideHardwareSpecs": false, 285 | "memoryGiB": 32, 286 | "name": "ml.m5.2xlarge", 287 | "vcpuNum": 8 288 | }, 289 | { 290 | "_defaultOrder": 7, 291 | "_isFastLaunch": false, 292 | "category": "General purpose", 293 | "gpuNum": 0, 294 | "hideHardwareSpecs": false, 295 | "memoryGiB": 64, 296 | "name": "ml.m5.4xlarge", 297 | "vcpuNum": 16 298 | }, 299 | { 300 | "_defaultOrder": 8, 301 | "_isFastLaunch": false, 302 | "category": "General purpose", 303 | "gpuNum": 0, 304 | "hideHardwareSpecs": false, 305 | "memoryGiB": 128, 306 | "name": "ml.m5.8xlarge", 307 | "vcpuNum": 32 308 | }, 309 | { 310 | "_defaultOrder": 9, 311 | "_isFastLaunch": false, 312 | "category": "General purpose", 313 | "gpuNum": 0, 314 | "hideHardwareSpecs": false, 315 | "memoryGiB": 192, 316 | "name": "ml.m5.12xlarge", 317 | "vcpuNum": 48 318 | }, 319 | { 320 | "_defaultOrder": 10, 321 | "_isFastLaunch": false, 322 | "category": "General purpose", 323 | "gpuNum": 0, 324 | "hideHardwareSpecs": false, 325 | "memoryGiB": 256, 326 | "name": "ml.m5.16xlarge", 327 | "vcpuNum": 64 328 | }, 329 | { 330 | "_defaultOrder": 11, 331 | "_isFastLaunch": false, 332 | "category": "General purpose", 333 | "gpuNum": 0, 334 | "hideHardwareSpecs": false, 335 | "memoryGiB": 384, 336 | "name": "ml.m5.24xlarge", 337 | "vcpuNum": 96 338 | }, 339 | { 340 | "_defaultOrder": 12, 341 | "_isFastLaunch": false, 342 | "category": "General purpose", 343 | "gpuNum": 0, 344 | "hideHardwareSpecs": false, 345 | "memoryGiB": 8, 346 | "name": "ml.m5d.large", 347 | "vcpuNum": 2 348 | }, 349 | { 350 | "_defaultOrder": 13, 351 | "_isFastLaunch": false, 352 | "category": "General purpose", 353 | "gpuNum": 0, 354 | "hideHardwareSpecs": false, 355 | "memoryGiB": 16, 356 | "name": "ml.m5d.xlarge", 357 | "vcpuNum": 4 358 | }, 359 | { 360 | "_defaultOrder": 14, 361 | "_isFastLaunch": false, 362 | "category": "General purpose", 363 | "gpuNum": 0, 364 | "hideHardwareSpecs": false, 365 | "memoryGiB": 32, 366 | "name": "ml.m5d.2xlarge", 367 | "vcpuNum": 8 368 | }, 369 | { 370 | "_defaultOrder": 15, 371 | "_isFastLaunch": false, 372 | "category": "General purpose", 373 | "gpuNum": 0, 374 | "hideHardwareSpecs": false, 375 | "memoryGiB": 64, 376 | "name": "ml.m5d.4xlarge", 377 | "vcpuNum": 16 378 | }, 379 | { 380 | "_defaultOrder": 16, 381 | "_isFastLaunch": false, 382 | "category": "General purpose", 383 | "gpuNum": 0, 384 | "hideHardwareSpecs": false, 385 | "memoryGiB": 128, 386 | "name": "ml.m5d.8xlarge", 387 | "vcpuNum": 32 388 | }, 389 | { 390 | "_defaultOrder": 17, 391 | "_isFastLaunch": false, 392 | "category": "General purpose", 393 | "gpuNum": 0, 394 | "hideHardwareSpecs": false, 395 | "memoryGiB": 192, 396 | "name": "ml.m5d.12xlarge", 397 | "vcpuNum": 48 398 | }, 399 | { 400 | "_defaultOrder": 18, 401 | "_isFastLaunch": false, 402 | "category": "General purpose", 403 | "gpuNum": 0, 404 | "hideHardwareSpecs": false, 405 | "memoryGiB": 256, 406 | "name": "ml.m5d.16xlarge", 407 | "vcpuNum": 64 408 | }, 409 | { 410 | "_defaultOrder": 19, 411 | "_isFastLaunch": false, 412 | "category": "General purpose", 413 | "gpuNum": 0, 414 | "hideHardwareSpecs": false, 415 | "memoryGiB": 384, 416 | "name": "ml.m5d.24xlarge", 417 | "vcpuNum": 96 418 | }, 419 | { 420 | "_defaultOrder": 20, 421 | "_isFastLaunch": false, 422 | "category": "General purpose", 423 | "gpuNum": 0, 424 | "hideHardwareSpecs": true, 425 | "memoryGiB": 0, 426 | "name": "ml.geospatial.interactive", 427 | "supportedImageNames": [ 428 | "sagemaker-geospatial-v1-0" 429 | ], 430 | "vcpuNum": 0 431 | }, 432 | { 433 | "_defaultOrder": 21, 434 | "_isFastLaunch": true, 435 | "category": "Compute optimized", 436 | "gpuNum": 0, 437 | "hideHardwareSpecs": false, 438 | "memoryGiB": 4, 439 | "name": "ml.c5.large", 440 | "vcpuNum": 2 441 | }, 442 | { 443 | "_defaultOrder": 22, 444 | "_isFastLaunch": false, 445 | "category": "Compute optimized", 446 | "gpuNum": 0, 447 | "hideHardwareSpecs": false, 448 | "memoryGiB": 8, 449 | "name": "ml.c5.xlarge", 450 | "vcpuNum": 4 451 | }, 452 | { 453 | "_defaultOrder": 23, 454 | "_isFastLaunch": false, 455 | "category": "Compute optimized", 456 | "gpuNum": 0, 457 | "hideHardwareSpecs": false, 458 | "memoryGiB": 16, 459 | "name": "ml.c5.2xlarge", 460 | "vcpuNum": 8 461 | }, 462 | { 463 | "_defaultOrder": 24, 464 | "_isFastLaunch": false, 465 | "category": "Compute optimized", 466 | "gpuNum": 0, 467 | "hideHardwareSpecs": false, 468 | "memoryGiB": 32, 469 | "name": "ml.c5.4xlarge", 470 | "vcpuNum": 16 471 | }, 472 | { 473 | "_defaultOrder": 25, 474 | "_isFastLaunch": false, 475 | "category": "Compute optimized", 476 | "gpuNum": 0, 477 | "hideHardwareSpecs": false, 478 | "memoryGiB": 72, 479 | "name": "ml.c5.9xlarge", 480 | "vcpuNum": 36 481 | }, 482 | { 483 | "_defaultOrder": 26, 484 | "_isFastLaunch": false, 485 | "category": "Compute optimized", 486 | "gpuNum": 0, 487 | "hideHardwareSpecs": false, 488 | "memoryGiB": 96, 489 | "name": "ml.c5.12xlarge", 490 | "vcpuNum": 48 491 | }, 492 | { 493 | "_defaultOrder": 27, 494 | "_isFastLaunch": false, 495 | "category": "Compute optimized", 496 | "gpuNum": 0, 497 | "hideHardwareSpecs": false, 498 | "memoryGiB": 144, 499 | "name": "ml.c5.18xlarge", 500 | "vcpuNum": 72 501 | }, 502 | { 503 | "_defaultOrder": 28, 504 | "_isFastLaunch": false, 505 | "category": "Compute optimized", 506 | "gpuNum": 0, 507 | "hideHardwareSpecs": false, 508 | "memoryGiB": 192, 509 | "name": "ml.c5.24xlarge", 510 | "vcpuNum": 96 511 | }, 512 | { 513 | "_defaultOrder": 29, 514 | "_isFastLaunch": true, 515 | "category": "Accelerated computing", 516 | "gpuNum": 1, 517 | "hideHardwareSpecs": false, 518 | "memoryGiB": 16, 519 | "name": "ml.g4dn.xlarge", 520 | "vcpuNum": 4 521 | }, 522 | { 523 | "_defaultOrder": 30, 524 | "_isFastLaunch": false, 525 | "category": "Accelerated computing", 526 | "gpuNum": 1, 527 | "hideHardwareSpecs": false, 528 | "memoryGiB": 32, 529 | "name": "ml.g4dn.2xlarge", 530 | "vcpuNum": 8 531 | }, 532 | { 533 | "_defaultOrder": 31, 534 | "_isFastLaunch": false, 535 | "category": "Accelerated computing", 536 | "gpuNum": 1, 537 | "hideHardwareSpecs": false, 538 | "memoryGiB": 64, 539 | "name": "ml.g4dn.4xlarge", 540 | "vcpuNum": 16 541 | }, 542 | { 543 | "_defaultOrder": 32, 544 | "_isFastLaunch": false, 545 | "category": "Accelerated computing", 546 | "gpuNum": 1, 547 | "hideHardwareSpecs": false, 548 | "memoryGiB": 128, 549 | "name": "ml.g4dn.8xlarge", 550 | "vcpuNum": 32 551 | }, 552 | { 553 | "_defaultOrder": 33, 554 | "_isFastLaunch": false, 555 | "category": "Accelerated computing", 556 | "gpuNum": 4, 557 | "hideHardwareSpecs": false, 558 | "memoryGiB": 192, 559 | "name": "ml.g4dn.12xlarge", 560 | "vcpuNum": 48 561 | }, 562 | { 563 | "_defaultOrder": 34, 564 | "_isFastLaunch": false, 565 | "category": "Accelerated computing", 566 | "gpuNum": 1, 567 | "hideHardwareSpecs": false, 568 | "memoryGiB": 256, 569 | "name": "ml.g4dn.16xlarge", 570 | "vcpuNum": 64 571 | }, 572 | { 573 | "_defaultOrder": 35, 574 | "_isFastLaunch": false, 575 | "category": "Accelerated computing", 576 | "gpuNum": 1, 577 | "hideHardwareSpecs": false, 578 | "memoryGiB": 61, 579 | "name": "ml.p3.2xlarge", 580 | "vcpuNum": 8 581 | }, 582 | { 583 | "_defaultOrder": 36, 584 | "_isFastLaunch": false, 585 | "category": "Accelerated computing", 586 | "gpuNum": 4, 587 | "hideHardwareSpecs": false, 588 | "memoryGiB": 244, 589 | "name": "ml.p3.8xlarge", 590 | "vcpuNum": 32 591 | }, 592 | { 593 | "_defaultOrder": 37, 594 | "_isFastLaunch": false, 595 | "category": "Accelerated computing", 596 | "gpuNum": 8, 597 | "hideHardwareSpecs": false, 598 | "memoryGiB": 488, 599 | "name": "ml.p3.16xlarge", 600 | "vcpuNum": 64 601 | }, 602 | { 603 | "_defaultOrder": 38, 604 | "_isFastLaunch": false, 605 | "category": "Accelerated computing", 606 | "gpuNum": 8, 607 | "hideHardwareSpecs": false, 608 | "memoryGiB": 768, 609 | "name": "ml.p3dn.24xlarge", 610 | "vcpuNum": 96 611 | }, 612 | { 613 | "_defaultOrder": 39, 614 | "_isFastLaunch": false, 615 | "category": "Memory Optimized", 616 | "gpuNum": 0, 617 | "hideHardwareSpecs": false, 618 | "memoryGiB": 16, 619 | "name": "ml.r5.large", 620 | "vcpuNum": 2 621 | }, 622 | { 623 | "_defaultOrder": 40, 624 | "_isFastLaunch": false, 625 | "category": "Memory Optimized", 626 | "gpuNum": 0, 627 | "hideHardwareSpecs": false, 628 | "memoryGiB": 32, 629 | "name": "ml.r5.xlarge", 630 | "vcpuNum": 4 631 | }, 632 | { 633 | "_defaultOrder": 41, 634 | "_isFastLaunch": false, 635 | "category": "Memory Optimized", 636 | "gpuNum": 0, 637 | "hideHardwareSpecs": false, 638 | "memoryGiB": 64, 639 | "name": "ml.r5.2xlarge", 640 | "vcpuNum": 8 641 | }, 642 | { 643 | "_defaultOrder": 42, 644 | "_isFastLaunch": false, 645 | "category": "Memory Optimized", 646 | "gpuNum": 0, 647 | "hideHardwareSpecs": false, 648 | "memoryGiB": 128, 649 | "name": "ml.r5.4xlarge", 650 | "vcpuNum": 16 651 | }, 652 | { 653 | "_defaultOrder": 43, 654 | "_isFastLaunch": false, 655 | "category": "Memory Optimized", 656 | "gpuNum": 0, 657 | "hideHardwareSpecs": false, 658 | "memoryGiB": 256, 659 | "name": "ml.r5.8xlarge", 660 | "vcpuNum": 32 661 | }, 662 | { 663 | "_defaultOrder": 44, 664 | "_isFastLaunch": false, 665 | "category": "Memory Optimized", 666 | "gpuNum": 0, 667 | "hideHardwareSpecs": false, 668 | "memoryGiB": 384, 669 | "name": "ml.r5.12xlarge", 670 | "vcpuNum": 48 671 | }, 672 | { 673 | "_defaultOrder": 45, 674 | "_isFastLaunch": false, 675 | "category": "Memory Optimized", 676 | "gpuNum": 0, 677 | "hideHardwareSpecs": false, 678 | "memoryGiB": 512, 679 | "name": "ml.r5.16xlarge", 680 | "vcpuNum": 64 681 | }, 682 | { 683 | "_defaultOrder": 46, 684 | "_isFastLaunch": false, 685 | "category": "Memory Optimized", 686 | "gpuNum": 0, 687 | "hideHardwareSpecs": false, 688 | "memoryGiB": 768, 689 | "name": "ml.r5.24xlarge", 690 | "vcpuNum": 96 691 | }, 692 | { 693 | "_defaultOrder": 47, 694 | "_isFastLaunch": false, 695 | "category": "Accelerated computing", 696 | "gpuNum": 1, 697 | "hideHardwareSpecs": false, 698 | "memoryGiB": 16, 699 | "name": "ml.g5.xlarge", 700 | "vcpuNum": 4 701 | }, 702 | { 703 | "_defaultOrder": 48, 704 | "_isFastLaunch": false, 705 | "category": "Accelerated computing", 706 | "gpuNum": 1, 707 | "hideHardwareSpecs": false, 708 | "memoryGiB": 32, 709 | "name": "ml.g5.2xlarge", 710 | "vcpuNum": 8 711 | }, 712 | { 713 | "_defaultOrder": 49, 714 | "_isFastLaunch": false, 715 | "category": "Accelerated computing", 716 | "gpuNum": 1, 717 | "hideHardwareSpecs": false, 718 | "memoryGiB": 64, 719 | "name": "ml.g5.4xlarge", 720 | "vcpuNum": 16 721 | }, 722 | { 723 | "_defaultOrder": 50, 724 | "_isFastLaunch": false, 725 | "category": "Accelerated computing", 726 | "gpuNum": 1, 727 | "hideHardwareSpecs": false, 728 | "memoryGiB": 128, 729 | "name": "ml.g5.8xlarge", 730 | "vcpuNum": 32 731 | }, 732 | { 733 | "_defaultOrder": 51, 734 | "_isFastLaunch": false, 735 | "category": "Accelerated computing", 736 | "gpuNum": 1, 737 | "hideHardwareSpecs": false, 738 | "memoryGiB": 256, 739 | "name": "ml.g5.16xlarge", 740 | "vcpuNum": 64 741 | }, 742 | { 743 | "_defaultOrder": 52, 744 | "_isFastLaunch": false, 745 | "category": "Accelerated computing", 746 | "gpuNum": 4, 747 | "hideHardwareSpecs": false, 748 | "memoryGiB": 192, 749 | "name": "ml.g5.12xlarge", 750 | "vcpuNum": 48 751 | }, 752 | { 753 | "_defaultOrder": 53, 754 | "_isFastLaunch": false, 755 | "category": "Accelerated computing", 756 | "gpuNum": 4, 757 | "hideHardwareSpecs": false, 758 | "memoryGiB": 384, 759 | "name": "ml.g5.24xlarge", 760 | "vcpuNum": 96 761 | }, 762 | { 763 | "_defaultOrder": 54, 764 | "_isFastLaunch": false, 765 | "category": "Accelerated computing", 766 | "gpuNum": 8, 767 | "hideHardwareSpecs": false, 768 | "memoryGiB": 768, 769 | "name": "ml.g5.48xlarge", 770 | "vcpuNum": 192 771 | }, 772 | { 773 | "_defaultOrder": 55, 774 | "_isFastLaunch": false, 775 | "category": "Accelerated computing", 776 | "gpuNum": 8, 777 | "hideHardwareSpecs": false, 778 | "memoryGiB": 1152, 779 | "name": "ml.p4d.24xlarge", 780 | "vcpuNum": 96 781 | }, 782 | { 783 | "_defaultOrder": 56, 784 | "_isFastLaunch": false, 785 | "category": "Accelerated computing", 786 | "gpuNum": 8, 787 | "hideHardwareSpecs": false, 788 | "memoryGiB": 1152, 789 | "name": "ml.p4de.24xlarge", 790 | "vcpuNum": 96 791 | }, 792 | { 793 | "_defaultOrder": 57, 794 | "_isFastLaunch": false, 795 | "category": "Accelerated computing", 796 | "gpuNum": 0, 797 | "hideHardwareSpecs": false, 798 | "memoryGiB": 32, 799 | "name": "ml.trn1.2xlarge", 800 | "vcpuNum": 8 801 | }, 802 | { 803 | "_defaultOrder": 58, 804 | "_isFastLaunch": false, 805 | "category": "Accelerated computing", 806 | "gpuNum": 0, 807 | "hideHardwareSpecs": false, 808 | "memoryGiB": 512, 809 | "name": "ml.trn1.32xlarge", 810 | "vcpuNum": 128 811 | }, 812 | { 813 | "_defaultOrder": 59, 814 | "_isFastLaunch": false, 815 | "category": "Accelerated computing", 816 | "gpuNum": 0, 817 | "hideHardwareSpecs": false, 818 | "memoryGiB": 512, 819 | "name": "ml.trn1n.32xlarge", 820 | "vcpuNum": 128 821 | } 822 | ], 823 | "instance_type": "ml.t3.medium", 824 | "kernelspec": { 825 | "display_name": "Python 3 (ipykernel)", 826 | "language": "python", 827 | "name": "python3" 828 | }, 829 | "language_info": { 830 | "codemirror_mode": { 831 | "name": "ipython", 832 | "version": 3 833 | }, 834 | "file_extension": ".py", 835 | "mimetype": "text/x-python", 836 | "name": "python", 837 | "nbconvert_exporter": "python", 838 | "pygments_lexer": "ipython3", 839 | "version": "3.11.11" 840 | } 841 | }, 842 | "nbformat": 4, 843 | "nbformat_minor": 5 844 | } 845 | -------------------------------------------------------------------------------- /langfuse-v3/requirements.txt: -------------------------------------------------------------------------------- 1 | aws-cdk-lib==2.177.0 2 | constructs>=10.0.0,<11.0.0 3 | cdk-ecr-deployment==3.0.82 4 | -------------------------------------------------------------------------------- /langfuse-v3/source.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem The sole purpose of this script is to make the command 4 | rem 5 | rem source .venv/bin/activate 6 | rem 7 | rem (which activates a Python virtualenv on Linux or Mac OS X) work on Windows. 8 | rem On Windows, this command just runs this batch file (the argument is ignored). 9 | rem 10 | rem Now we don't need to document a Windows command for activating a virtualenv. 11 | 12 | echo Executing .venv\Scripts\activate.bat for you 13 | .venv\Scripts\activate.bat 14 | --------------------------------------------------------------------------------